Skip to content

chore(tooling): remove the Dagger engine and its runner scale set - #2073

Merged
Smana merged 2 commits into
mainfrom
chore/remove-dagger-engine
Sep 21, 2026
Merged

Smana merged 2 commits into
mainfrom
chore/remove-dagger-engine

Conversation

@Smana

@Smana Smana commented Sep 21, 2026

Copy link
Copy Markdown
Owner

Removes the last of Dagger: the in-cluster engine and its GitHub Actions runner scale set. Dagger's CI step was already removed in #1810, and ADR-0039 (on #2061) records why. Closes #1595.

Merging this deletes both from the cluster through Flux prune. Both components were already commented out of tooling/aws-0/kustomization.yaml, and every workflow uses runs-on: ubuntu-latest, so nothing depends on them.

What goes

  • tooling/base/dagger-engine/ (6 files) and tooling/base/gha-runners/dagger-scale-set-helmrelease.yaml, plus their references:
    • the gha-runners kustomization and its network policy;
    • tooling/aws-0/kustomization.yaml;
    • .polaris.yaml exemptions;
    • a dead CHART_RENDER_OVERRIDES entry in scripts/flux-schema/render-bundle.py, keyed on the deleted HelmRelease.
  • Docs and data: ci-workflows.md, validation.md, further-reading.md, repo-tree.yaml, stack.yaml, and the orphaned dagger.svg logo with its license row.
  • docs/architecture/ci-pipeline.drawio: the "Dagger → daggerverse/pre-commit-tf" box, re-exported with the pinned drawio 30.3.6. Every other export came back byte-identical.

Kept, as history: ADR-0005, and the three ci.yaml comments explaining why Dagger left.

One unrelated fix rides along. The rendered-controller count quoted in validation.md, root AGENTS.md and scripts/AGENTS.md had drifted: it said 156, and the measured count is 160 (110 Deployment, 25 Job, 10 StatefulSet, 8 DaemonSet, 7 CronJob). All three now agree.

Installation removal, not a decision, so no ADR (ADR-0039 already covers Dagger). The old SPEC-009 branch chore/remove-dagger (2026-07-15, never opened) is superseded by this PR.

Evidence

Run on this branch, rebased on 42f567e8:

  • ./scripts/validate-manifests.sh exits 0: Summary: 2105 resources found in 276 files - Valid: 2105, Invalid: 0, and "All gates passed". Before this change it was 2115; the drop is exactly the 10 Dagger resources.
  • validate-links.sh, verify-doc-paths.sh and validate-doc-claims.sh (30/30) exit 0.
  • git grep -i dagger, outside the dated archive, returns 4 hits, all history.
  • .secrets.baseline loses only the deleted file's entry.

@github-actions

Copy link
Copy Markdown
Contributor

🔍 Rendered manifest diff — this PR vs main (desired state)

1 changed · 0 added · 10 removed

Rendered with kustomize build + helm template (source of truth = git), so Helm-expanded workloads are included. Shows what Flux will apply — not a diff against live cluster state (drift is alerted on separately), and not CRD-defaulted / webhook-mutated output. Secret values are redacted; per-render noise (webhook caBundles, checksum/* annotations, render timestamps) is normalized out.

🔴 removed — actions.github.com/v1alpha1/AutoscalingRunnerSet/tooling/dagger-gha-runner-scale-set
--- a/actions.github.com/v1alpha1/AutoscalingRunnerSet/tooling/dagger-gha-runner-scale-set
+++ b/actions.github.com/v1alpha1/AutoscalingRunnerSet/tooling/dagger-gha-runner-scale-set
@@ -1,82 +0,0 @@
-apiVersion: actions.github.com/v1alpha1
-kind: AutoscalingRunnerSet
-metadata:
-  annotations:
-    actions.github.com/cleanup-manager-role-binding: dagger-gha-runner-scale-set-gha-rs-manager
-    actions.github.com/cleanup-manager-role-name: dagger-gha-runner-scale-set-gha-rs-manager
-    actions.github.com/cleanup-no-permission-service-account-name: dagger-gha-runner-scale-set-gha-rs-no-permission
-    actions.github.com/values-hash: 65a78d8261515b7899e1f1106dd07edea5edde730309b1e13b809dba6ecccc9
-  labels:
-    actions.github.com/scale-set-name: dagger-gha-runner-scale-set
-    actions.github.com/scale-set-namespace: tooling
-    app.kubernetes.io/component: autoscaling-runner-set
-    app.kubernetes.io/instance: dagger-gha-runner-scale-set
-    app.kubernetes.io/managed-by: Helm
-    app.kubernetes.io/name: dagger-gha-runner-scale-set
-    app.kubernetes.io/part-of: gha-rs
-    app.kubernetes.io/version: 0.9.3
-    helm.sh/chart: gha-rs-0.9.3
-  name: dagger-gha-runner-scale-set
-  namespace: tooling
-spec:
-  githubConfigSecret: gha-runner-scale-set
-  githubConfigUrl: https://github.com/Smana/cloud-native-ref
-  maxRunners: 5
-  runnerGroup: default
-  template:
-    spec:
-      containers:
-      - command:
-        - /home/runner/run.sh
-        env:
-        - name: DOCKER_HOST
-          value: unix:///var/run/docker.sock
-        - name: RUNNER_WAIT_FOR_DOCKER_IN_SECONDS
-          value: '120'
-        image: ghcr.io/actions/actions-runner:2.337.0
-        name: runner
-        volumeMounts:
-        - mountPath: /home/runner/_work
-          name: work
-        - mountPath: /var/run
-          name: dind-sock
-      - args:
-        - dockerd
-        - --host=unix:///var/run/docker.sock
-        - --group=$(DOCKER_GROUP_GID)
-        env:
-        - name: DOCKER_GROUP_GID
-          value: '123'
-        image: docker:dind
-        name: dind
-        securityContext:
-          privileged: true
-        volumeMounts:
-        - mountPath: /home/runner/_work
-          name: work
-        - mountPath: /var/run
-          name: dind-sock
-        - mountPath: /home/runner/externals
-          name: dind-externals
-      initContainers:
-      - args:
-        - -r
-        - -v
-        - /home/runner/externals/.
-        - /home/runner/tmpDir/
-        command:
-        - cp
-        image: ghcr.io/actions/actions-runner:2.337.0
-        name: init-dind-externals
-        volumeMounts:
-        - mountPath: /home/runner/tmpDir
-          name: dind-externals
-      restartPolicy: Never
-      serviceAccountName: dagger-gha-runner-scale-set-gha-rs-no-permission
-      volumes:
-      - emptyDir: {}
-        name: dind-sock
-      - emptyDir: {}
-        name: dind-externals
-      - emptyDir: {}
-        name: work
🔴 removed — apps/v1/Deployment/tooling/dagger-engine
--- a/apps/v1/Deployment/tooling/dagger-engine
+++ b/apps/v1/Deployment/tooling/dagger-engine
@@ -1,102 +0,0 @@
-apiVersion: apps/v1
-kind: Deployment
-metadata:
-  annotations:
-    karpenter.sh/do-not-disrupt: 'true'
-  labels:
-    app.kubernetes.io/name: dagger-engine
-  name: dagger-engine
-  namespace: tooling
-spec:
-  replicas: 1
-  selector:
-    matchLabels:
-      app.kubernetes.io/name: dagger-engine
-  template:
-    metadata:
-      labels:
-        app.kubernetes.io/name: dagger-engine
-    spec:
-      affinity:
-        nodeAffinity:
-          requiredDuringSchedulingIgnoredDuringExecution:
-            nodeSelectorTerms:
-            - matchExpressions:
-              - key: karpenter.sh/nodepool
-                operator: In
-                values:
-                - io
-      containers:
-      - args:
-        - --addr
-        - tcp://0.0.0.0:8080
-        - --oci-max-parallelism
-        - num-cpu
-        image: registry.dagger.io/engine:v0.21.9
-        imagePullPolicy: IfNotPresent
-        name: dagger-engine
-        ports:
-        - containerPort: 8080
-        readinessProbe:
-          failureThreshold: 3
-          initialDelaySeconds: 5
-          periodSeconds: 10
-          tcpSocket:
-            port: 8080
-        resources:
-          limits:
-            ephemeral-storage: 100Gi
-            memory: 8Gi
-          requests:
-            cpu: '4'
-            ephemeral-storage: 100Gi
-            memory: 8Gi
-        securityContext:
-          capabilities:
-            add:
-            - ALL
-          privileged: true
-        volumeMounts:
-        - mountPath: /etc/dagger/engine.toml
-          name: dagger-engine-config
-          subPath: engine.toml
-        - mountPath: /var/lib/dagger
-          name: varlibdagger
-        - mountPath: /var/run/buildkit
-          name: varrundagger
-      securityContext:
-        fsGroup: 1001
-        fsGroupChangePolicy: OnRootMismatch
-        runAsGroup: 1001
-        runAsUser: 0
-      serviceAccountName: default
-      terminationGracePeriodSeconds: 300
-      tolerations:
-      - effect: NoSchedule
-        key: ogenki/io
-        operator: Exists
-      volumes:
-      - configMap:
-          items:
-          - key: engine.toml
-            path: engine.toml
-          name: dagger-engine
-        name: dagger-engine-config
-      - ephemeral:
-          volumeClaimTemplate:
-            spec:
-              accessModes:
-              - ReadWriteOnce
-              resources:
-                requests:
-                  storage: 50Gi
-        name: varlibdagger
-      - ephemeral:
-          volumeClaimTemplate:
-            spec:
-              accessModes:
-              - ReadWriteOnce
-              resources:
-                requests:
-                  storage: 50Gi
-        name: varrundagger
🔴 removed — cilium.io/v2/CiliumNetworkPolicy/tooling/dagger-engine
--- a/cilium.io/v2/CiliumNetworkPolicy/tooling/dagger-engine
+++ b/cilium.io/v2/CiliumNetworkPolicy/tooling/dagger-engine
@@ -1,40 +0,0 @@
-apiVersion: cilium.io/v2
-kind: CiliumNetworkPolicy
-metadata:
-  name: dagger-engine
-  namespace: tooling
-spec:
-  description: Allow internal traffic to the Dagger Engine service.
-  egress:
-  - toEndpoints:
-    - matchLabels:
-        k8s:io.kubernetes.pod.namespace: kube-system
-        k8s:k8s-app: kube-dns
-    toPorts:
-    - ports:
-      - port: '53'
-        protocol: UDP
-      - port: '53'
-        protocol: TCP
-  - toEntities:
-    - world
-    toPorts:
-    - ports:
-      - port: '80'
-        protocol: TCP
-      - port: '443'
-        protocol: TCP
-  endpointSelector:
-    matchLabels:
-      k8s:app.kubernetes.io/name: dagger-engine
-  ingress:
-  - fromEndpoints:
-    - matchLabels:
-        k8s:actions.github.com/scale-set-name: default-gha-runner-scale-set
-  - fromEndpoints:
-    - matchLabels:
-        k8s:actions.github.com/scale-set-name: dagger-gha-runner-scale-set
-    toPorts:
-    - ports:
-      - port: '8080'
-        protocol: TCP
🟡 changed — cilium.io/v2/CiliumNetworkPolicy/tooling/gha-runner-scale-set
--- a/cilium.io/v2/CiliumNetworkPolicy/tooling/gha-runner-scale-set
+++ b/cilium.io/v2/CiliumNetworkPolicy/tooling/gha-runner-scale-set
@@ -16,13 +16,6 @@
         protocol: UDP
       - port: '53'
         protocol: TCP
-  - toEndpoints:
-    - matchLabels:
-        k8s:app.kubernetes.io/name: dagger-engine
-    toPorts:
-    - ports:
-      - port: '8080'
-        protocol: TCP
   - toEntities:
     - world
     toPorts:
@@ -37,4 +30,3 @@
       operator: In
       values:
       - default-gha-runner-scale-set
-      - dagger-gha-runner-scale-set
🔴 removed — helm.toolkit.fluxcd.io/v2/HelmRelease/tooling/dagger-gha-runner-scale-set
--- a/helm.toolkit.fluxcd.io/v2/HelmRelease/tooling/dagger-gha-runner-scale-set
+++ b/helm.toolkit.fluxcd.io/v2/HelmRelease/tooling/dagger-gha-runner-scale-set
@@ -1,34 +0,0 @@
-apiVersion: helm.toolkit.fluxcd.io/v2
-kind: HelmRelease
-metadata:
-  name: dagger-gha-runner-scale-set
-  namespace: tooling
-spec:
-  chart:
-    spec:
-      chart: gha-runner-scale-set
-      sourceRef:
-        kind: HelmRepository
-        name: gha-runner-scale-set
-      version: 0.9.3
-  driftDetection:
-    mode: enabled
-  install:
-    remediation:
-      retries: 3
-  interval: 10m0s
-  releaseName: dagger-gha-runner-scale-set
-  values:
-    containerMode:
-      type: dind
-    githubConfigSecret: gha-runner-scale-set
-    githubConfigUrl: https://github.com/Smana/cloud-native-ref
-    maxRunners: 5
-    runnerGroup: default
-    template:
-      spec:
-        containers:
-        - command:
-          - /home/runner/run.sh
-          image: ghcr.io/actions/actions-runner:2.337.0
-          name: runner
🔴 removed — policy/v1/PodDisruptionBudget/tooling/dagger-engine
--- a/policy/v1/PodDisruptionBudget/tooling/dagger-engine
+++ b/policy/v1/PodDisruptionBudget/tooling/dagger-engine
@@ -1,10 +0,0 @@
-apiVersion: policy/v1
-kind: PodDisruptionBudget
-metadata:
-  name: dagger-engine
-  namespace: tooling
-spec:
-  minAvailable: 1
-  selector:
-    matchLabels:
-      app.kubernetes.io/name: dagger-engine
🔴 removed — rbac.authorization.k8s.io/v1/Role/tooling/dagger-gha-runner-scale-set-gha-rs-manager
--- a/rbac.authorization.k8s.io/v1/Role/tooling/dagger-gha-runner-scale-set-gha-rs-manager
+++ b/rbac.authorization.k8s.io/v1/Role/tooling/dagger-gha-runner-scale-set-gha-rs-manager
@@ -1,74 +0,0 @@
-apiVersion: rbac.authorization.k8s.io/v1
-kind: Role
-metadata:
-  finalizers:
-  - actions.github.com/cleanup-protection
-  labels:
-    actions.github.com/scale-set-name: dagger-gha-runner-scale-set
-    actions.github.com/scale-set-namespace: tooling
-    app.kubernetes.io/component: manager-role
-    app.kubernetes.io/instance: dagger-gha-runner-scale-set
-    app.kubernetes.io/managed-by: Helm
-    app.kubernetes.io/name: dagger-gha-runner-scale-set
-    app.kubernetes.io/part-of: gha-rs
-    app.kubernetes.io/version: 0.9.3
-    helm.sh/chart: gha-rs-0.9.3
-  name: dagger-gha-runner-scale-set-gha-rs-manager
-  namespace: tooling
-rules:
-- apiGroups:
-  - ''
-  resources:
-  - pods
-  verbs:
-  - create
-  - delete
-  - get
-- apiGroups:
-  - ''
-  resources:
-  - pods/status
-  verbs:
-  - get
-- apiGroups:
-  - ''
-  resources:
-  - secrets
-  verbs:
-  - create
-  - delete
-  - get
-  - list
-  - patch
-  - update
-- apiGroups:
-  - ''
-  resources:
-  - serviceaccounts
-  verbs:
-  - create
-  - delete
-  - get
-  - list
-  - patch
-  - update
-- apiGroups:
-  - rbac.authorization.k8s.io
-  resources:
-  - rolebindings
-  verbs:
-  - create
-  - delete
-  - get
-  - patch
-  - update
-- apiGroups:
-  - rbac.authorization.k8s.io
-  resources:
-  - roles
-  verbs:
-  - create
-  - delete
-  - get
-  - patch
-  - update
🔴 removed — rbac.authorization.k8s.io/v1/RoleBinding/tooling/dagger-gha-runner-scale-set-gha-rs-manager
--- a/rbac.authorization.k8s.io/v1/RoleBinding/tooling/dagger-gha-runner-scale-set-gha-rs-manager
+++ b/rbac.authorization.k8s.io/v1/RoleBinding/tooling/dagger-gha-runner-scale-set-gha-rs-manager
@@ -1,25 +0,0 @@
-apiVersion: rbac.authorization.k8s.io/v1
-kind: RoleBinding
-metadata:
-  finalizers:
-  - actions.github.com/cleanup-protection
-  labels:
-    actions.github.com/scale-set-name: dagger-gha-runner-scale-set
-    actions.github.com/scale-set-namespace: tooling
-    app.kubernetes.io/component: manager-role-binding
-    app.kubernetes.io/instance: dagger-gha-runner-scale-set
-    app.kubernetes.io/managed-by: Helm
-    app.kubernetes.io/name: dagger-gha-runner-scale-set
-    app.kubernetes.io/part-of: gha-rs
-    app.kubernetes.io/version: 0.9.3
-    helm.sh/chart: gha-rs-0.9.3
-  name: dagger-gha-runner-scale-set-gha-rs-manager
-  namespace: tooling
-roleRef:
-  apiGroup: rbac.authorization.k8s.io
-  kind: Role
-  name: dagger-gha-runner-scale-set-gha-rs-manager
-subjects:
-- kind: ServiceAccount
-  name: gha-runner-scale-set-controller-gha-rs-controller
-  namespace: tooling
🔴 removed — v1/ConfigMap/tooling/dagger-engine
--- a/v1/ConfigMap/tooling/dagger-engine
+++ b/v1/ConfigMap/tooling/dagger-engine
@@ -1,12 +0,0 @@
-apiVersion: v1
-data:
-  engine.toml: 'debug = true
-
-    '
-kind: ConfigMap
-metadata:
-  labels:
-    app.kubernetes.io/name: dagger-engine
-    app.kubernetes.io/version: v0.11.9
-  name: dagger-engine
-  namespace: tooling
🔴 removed — v1/Service/tooling/dagger-engine
--- a/v1/Service/tooling/dagger-engine
+++ b/v1/Service/tooling/dagger-engine
@@ -1,14 +0,0 @@
-apiVersion: v1
-kind: Service
-metadata:
-  labels:
-    app.kubernetes.io/name: dagger-engine
-  name: dagger-engine
-  namespace: tooling
-spec:
-  ports:
-  - port: 8080
-    protocol: TCP
-    targetPort: 8080
-  selector:
-    app.kubernetes.io/name: dagger-engine
🔴 removed — v1/ServiceAccount/tooling/dagger-gha-runner-scale-set-gha-rs-no-permission
--- a/v1/ServiceAccount/tooling/dagger-gha-runner-scale-set-gha-rs-no-permission
+++ b/v1/ServiceAccount/tooling/dagger-gha-runner-scale-set-gha-rs-no-permission
@@ -1,16 +0,0 @@
-apiVersion: v1
-kind: ServiceAccount
-metadata:
-  finalizers:
-  - actions.github.com/cleanup-protection
-  labels:
-    actions.github.com/scale-set-name: dagger-gha-runner-scale-set
-    actions.github.com/scale-set-namespace: tooling
-    app.kubernetes.io/instance: dagger-gha-runner-scale-set
-    app.kubernetes.io/managed-by: Helm
-    app.kubernetes.io/name: dagger-gha-runner-scale-set
-    app.kubernetes.io/part-of: gha-rs
-    app.kubernetes.io/version: 0.9.3
-    helm.sh/chart: gha-rs-0.9.3
-  name: dagger-gha-runner-scale-set-gha-rs-no-permission
-  namespace: tooling

Closes #1595. Dagger's CI usage was already removed in #1810 (ADR-0039);
this removes what was left: the in-cluster dagger-engine deployment,
the dedicated dagger-gha-runner-scale-set HelmRelease, and every
reference to them in Polaris exemptions, network policies, the
render-bundle overrides, the secrets baseline, the CI-pipeline diagram
and its export, and the docs site.

Both components were already disabled (commented out of
tooling/aws-0/kustomization.yaml), so merging this deletes nothing from
a live cluster beyond what Flux prune already excludes from the
rendered bundle validated in CI — the 10 resources they still
contributed to that validation-only render (ConfigMap, Service,
Deployment, PodDisruptionBudget, CiliumNetworkPolicy, HelmRelease,
ServiceAccount, Role, RoleBinding, AutoscalingRunnerSet) drop out of
it.
Removing dagger-engine and its runner scale set (3c08b250) dropped the
rendered bundle's controller count from 156 to the now-measured 160.
AGENTS.md and its scripts/AGENTS.md counterpart still said 156;
validation.md already said 160. Point both at the same number.
@Smana
Smana force-pushed the chore/remove-dagger-engine branch from 055245f to 942e823 Compare September 21, 2026 21:14
@Smana
Smana merged commit 7ead04c into main Sep 21, 2026
12 checks passed
@Smana
Smana deleted the chore/remove-dagger-engine branch September 21, 2026 21:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment