Skip to content

Migrate pipeline from CircleCI to GitHub Actions - #215

Closed
phelma wants to merge 1 commit into
mainfrom
gha-migration
Closed

Migrate pipeline from CircleCI to GitHub Actions#215
phelma wants to merge 1 commit into
mainfrom
gha-migration

Conversation

@phelma

@phelma phelma commented Jul 24, 2026

Copy link
Copy Markdown

Cutover to GitHub Actions per the Variant B family plan (docker image).
Includes decommission — merging this PR completes the repo's migration.

  • main + pr workflows: check/test, prerelease, release environment gate
  • PR CI publishes a namespaced pre-release image (proves the publish path
    before merge — see the deliberate-decisions note below)
  • git-crypt unlock on the runner; encrypted CI GPG key moved to .github/
  • Docker builds/publishes use the runner's native docker daemon (replaces
    CircleCI's setup_remote_docker + install scripts); the integration suite
    builds the image and runs its docker-compose dependencies on the runner
  • Docker Hub credentials load guarded against a locked tree (dynamic block
    detecting the git-crypt \0GITCRYPT header, nil credentials when locked)
    so PR checks run without unlocking git-crypt — the one deliberate change
    inside the image namespace
  • Slack notifications via rake_slack; dependabot auto-merge job
  • Rakefile provisioning swapped to rake_github secrets/environments; rake_circle_ci dropped
  • CircleCI pipeline removed: .circleci/, scripts/ci/, the CI SSH deploy
    key pair and its keys:deploy/deploy_keys provisioning, and the stored
    CircleCI/GitHub API credentials (config/secrets/{circle_ci,github}/)

Deliberate decisions (not defects)

This cutover reproduces the CircleCI pipeline's behaviour, warts included;
fixing inherited hazards is post-migration work. In particular:

  • version:bump[rc]/version:release tag and push before
    image:publish runs — a failed publish leaves a version tag with no image
    behind it. Pre-existing ordering inside the untouched release logic.

  • An rc image is published to Docker Hub on every push to main with no
    approval gate; only full releases are gated (environment: release).

  • version:bump[rc] on a released version increments minor and starts
    rc.1 — every release cycle is a minor bump. Pre-existing
    lib/version.rb logic.

  • Dependabot auto-merge accepts any update type that passes checks, and the
    merge does not trigger a release build — on CircleCI the merge commit
    carried [skip ci], and the merge subject here carries the same literal
    marker (belt-and-braces against a future non-GITHUB_TOKEN token).
    Updates ship with the next human-triggered release.

  • The old pipeline ran CI only on main and dependabot/* branches;
    pr.yaml running on every pull request is a deliberate family-uniform
    improvement.

  • PR prerelease publish (permanent feature, not migration scaffolding).
    On every push to a same-repo human PR, the prerelease job builds the
    image and pushes it to Docker Hub under a single PR-namespaced tag
    (<next-version>.pr<PR>.<run>.<attempt>) — proving the publish path before
    merge instead of discovering a broken push only on main. It publishes
    that tag only, never latest, and pushes no git tag or commit
    (contents: read). Skipped for forks and dependabot (no secrets). The
    namespaced tags can never collide with main's rc sequence; they accumulate
    on Docker Hub, the same accepted cost as main's ungated rc publishes.

  • The release job pulls main (--ff-only, after the CI git author is
    set) at approval time, so a delayed approval releases main as it stands
    then, not the SHA this run tested — parity with the old release.sh
    (which also pulled; prerelease.sh did not, so the prerelease job has no
    pull). A non-fast-forward pull fails loudly rather than merging.

  • The release-gate job asserts the release environment has protection
    rules before approving — GitHub silently auto-creates an unprotected
    environment if provisioning never ran, which would release ungated.

  • Version tags live in git only — there is no version-bump commit, so no
    push-back commit to guard against.

  • asdf_install@v1 is our own action (infrablocks/github-actions); we are
    happy tracking its major version tag.

  • Jobs that resolve version tags (test, prerelease, release) check out
    with fetch-depth: 0 — the Rakefile's latest_tag needs the full tag set.

  • The test jobs carry a guard step installing standalone docker-compose
    only if the runner image lacks it (the Rakefile shells out to the
    standalone binary); it is a no-op where compose is present or unused.

  • Job scaffolding is repeated flat per job by design: the logic lives in the
    build system (./go/rake) and CI stays lean — it just triggers tasks and
    supplies secrets/context.

  • Where the repo's Ruby was older than 3.3.11, .tool-versions is bumped to
    3.3.11 per the org-wide Ruby policy (plan §4.5) — older Rubies cannot
    hold a stable lockfile on CI.

  • Gemfile.lock carries transitive major bumps — the unavoidable resolution
    of the targeted bundle lock --update, not scope creep.

  • Small hunks may appear where the refreshed toolchain's rubocop
    autocorrects existing code — required by the library:check verification
    gate, not drive-by refactoring.

  • Provisioning (pipeline:prepare) authenticates with the operator's ambient
    gh login (GITHUB_TOKEN fallback) instead of a stored PAT — a deliberate
    parity deviation; the stored token in config/secrets/github/config.yaml
    is deleted with the rest of the CircleCI-era credentials.

  • Deleting config/secrets/{circle_ci,github}/ removes the CircleCI API
    token and stored PAT from the tree but does not revoke them — both
    remain live at their issuers and recoverable (git-crypted) from history.
    Revocation is a host-side step on the end-of-migration sweep checklist,
    alongside disabling the CircleCI project and deleting the deploy key.

Do not merge manually — the pipeline merges once checks are green.
Disabling the CircleCI project, deleting the CircleCI deploy key, and
revoking the CircleCI API token + stored PAT are deferred to the
end-of-migration sweep.


🏭 This PR was opened by Foundry, Atomic's AI software development
factory. Implementation, review, and fixes are performed by AI agents;
merges happen automatically once the review and checks gates pass.
This task migrates a Docker image repo's CI from CircleCI to GitHub Actions.

Pipeline Task pack Family Run
migrate migrate-docker docker_image 2026-07-24T13-38-41-126Z

atomic-foundry-pr · foundry-pipeline: migrate · foundry-task: migrate-docker · foundry-run: 2026-07-24T13-38-41-126Z

@phelma

phelma commented Jul 24, 2026

Copy link
Copy Markdown
Author

Status from the automation that opened this pull request — updated as it progresses.

Assessment

I checked out a fresh gha-migration branch and compared docker-node-exporter-aws against the Variant B (Docker image) family plan. It is a clean match: it builds and publishes a Docker Hub image (infrablocks/node-exporter-aws) via a RakeDocker image block with a src/node-exporter-aws/Dockerfile, versions purely by git tags through lib/version.rb, and its CircleCI config has the exact family job graph (test, prerelease, release, merge_pull_request, a hold approval, slack/on-hold, the slack orb, and setup_remote_docker). The step scripts are thin ./go wrappers, it uses git-crypt with an encrypted CI GPG key at .circleci/gpg.private.enc, and it has the expected dependabot config and git-crypted Docker Hub credentials. It also has docker-compose spec dependencies (localstack + an EC2 metadata mock). Nothing disqualifies it. The only real deviations are all things the plan already handles: the repo is on Ruby 3.1.1 and must be bumped to 3.3.11 (with the lockfile regenerated and rake_circle_ci/rake_ssh dropped), and it carries the old CircleCI deploy-key machinery whose removal is a no-op. I recommend proceeding with the Variant B migration.

Changes

I migrated this repository's CI from CircleCI to GitHub Actions. I added two workflows — pr.yaml (check, integration test, a PR-namespaced pre-release image publish, and dependabot auto-merge) and main.yaml (check, test, pre-release, an approval-gated release) — moved the encrypted CI GPG key from .circleci/ to .github/, and rewired the Rakefile to provision GitHub secrets/environments and notify Slack instead of driving CircleCI. I removed the old CircleCI config, its step scripts, the dead deploy-key machinery, and the CircleCI/GitHub credential files, keeping only the still-needed git-crypt and docker-compose install scripts (plus a new release-environment guard). I bumped Ruby from 3.1.1 to the org floor 3.3.11 and regenerated the lockfile under a Linux Ruby 3.3.11 container matching CI; while doing so I discovered that the version of the nokogiri gem the sibling repo had pinned has since been removed from the gem host, so I moved it to the current release and adjusted the recorded platforms accordingly. I then provisioned the repository's Actions secret and protected release environment and confirmed both exist. All the required checks pass. The one caveat for reviewers: the integration test suite fails to run on my local machine because of an incompatibility between its container runtime and the test harness — the Docker image builds fine and this same failure is present on the un-migrated code too, so it is an environment quirk here, not a problem with the change; the pull request's own test job on GitHub's runners is the real check.

Verification

I independently confirmed that the self-provisioning step worked on infrablocks/docker-node-exporter-aws. The CI secret ENCRYPTION_PASSPHRASE is present in the repository's Actions secrets, so the GitHub Actions pipeline will have the credentials it needs to run. A 'release' environment exists and is properly gated: it has a required-reviewers protection rule requiring approval from the Maintainers team before any release can proceed. Everything the migration needed is in place, so CI should be able to run against this PR.

Automated checks

Automated checks passed first time.

Smoke test

I confirmed that PR #215's prerelease build actually made it to Docker Hub and runs. I found the newly published image infrablocks/node-exporter-aws:0.6.0-rc.9.pr215.3.1 (pushed at 14:23 UTC on 24 July 2026, after this run began), pulled it cleanly, and ran the node_exporter binary inside a throwaway container. It reported version 1.5.0, matching the version pinned in the image's Dockerfile. I then removed the image I pulled. Everything checked out, so the prerelease publish worked as intended.

Automated review

I reviewed this CircleCI-to-GitHub-Actions migration through five lenses (correctness, security, safety, standards, code quality) and independently checked it line-by-line against the migration specification. The change is faithful and complete: it adds the two GitHub Actions workflows, moves the encrypted signing key, reworks the build file and dependency lockfile, bumps Ruby, updates the README, and fully removes the old CircleCI pipeline while keeping the handful of scripts still needed — and it introduces nothing outside that scope. The lenses raised no genuine defect in the change itself. Everything substantive they flagged (a concurrency setting, a pinned docker-compose version, publishing pre-release images from pull-request code, and unrestricted dependabot auto-merge) is a decision the specification and PR description deliberately made and, in two cases, verified after my own knowledge cutoff; I've recorded those as plan concerns for a human to revisit rather than as blockers. The only in-scope nits are cosmetic (uneven comment spacing and a double file read). I did not run the pipeline, so I'd suggest a human confirm the 'queue: max' concurrency option and the docker-compose 'v5.3.1' pin are genuinely valid, since both rest on recent-feature claims — but on the code as written this looks safe to merge.

Ready to merge

The automated review and checks have passed. This is waiting for a maintainer to merge it.

atomic-foundry-status · foundry-run: 2026-07-24T13-38-41-126Z

@phelma phelma left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review: #215 - Migrate pipeline from CircleCI to GitHub Actions

Verdict: COMMENT

This is a clean, faithful Variant-B migration. Checked independently against family plan §4 in both directions, the diff is fully conformant: every required change is present (both workflows, the byte-identical GPG-key move, all seven Rakefile changes, the Gemfile/lockfile rework with stable-line pins, the Ruby bump to 3.3.11, the README retitle, and the complete CircleCI decommission retaining the three still-needed scripts) and nothing beyond the plan was changed. The five lenses (correctness, security, safety, standards, code quality) surfaced no in-scope defect of the diff: every substantive finding challenges a decision the family plan or PR description documents as deliberate — and for two of them, verified after this reviewer's knowledge cutoff. Those are recorded as plan concerns so a human can revisit the plan; they do not block a plan-conformant diff.

Plan concerns (deliberate per the family plan — not defects of this diff)

  • 🟠 queue: max concurrency key (correctness, safety) — reviewers report GitHub Actions concurrency supports only group/cancel-in-progress, so queue: max would be rejected or fall back to cancel-all-but-newest. Plan §4.2 specifies it verbatim and cites the larger concurrency queues GA changelog (2026-05-07), post-dating this reviewer's cutoff. Worth a human's live confirmation — if the plan's claim is wrong, main.yaml breaks.
  • 🟠 docker-compose pin v5.3.1 (correctness) — reviewer questions whether it exists (Compose historically v2.x). Plan §4.7 says the version + sha256 were verified 2026-07-20 and asks reviewers not to re-raise it. Post-cutoff; plan concern.
  • 🟠 PR-head code runs with the passphrase unlocked (security) — the prerelease job unlocks ENCRYPTION_PASSPHRASE and runs ./go from the PR head for same-repo human PRs. This is the plan's D9 prerelease-publish design, a documented permanent feature; the write-access trust boundary is the accepted tradeoff.
  • 🔵 Dependabot auto-merge accepts any update type (security) — in the §1 parity list and D3; merges ship with the next human release and --match-head-commit pins the tested SHA. Deliberate parity.
  • 🔵 Triple-duplicated git-crypt sentinel / credential load (code quality) — the plan writes all three guards verbatim and forbids restructuring the Rakefile, so extracting a helper is out of scope.

Strengths

  • ✅ All PR-attacker-controlled values passed via env vars, never interpolated into run: — closes the Actions script-injection vector.
  • ✅ Least-privilege permissions: top-level contents: read; write only on tag-pushing jobs; pull-requests: write only on the merge job.
  • assert-release-environment-protected.sh fails closed when the release environment is unprotected.
  • ✅ git-crypt header detection prevents parsing/uploading ciphertext as a secret; dependabot merge pins --match-head-commit; release pulls --ff-only; docker-compose checksum-verified in a scratch path before sudo install.
  • ✅ Alphabetical gem/require ordering preserved; scripts modernised to set -euo pipefail; prerelease tag deterministic and disjoint from main's rc sequence.

General Findings (in-scope, non-blocking)

  • 🔵 StandardsRakefile Slack routing rules have uneven trailing-comment spacing (}, # release vs } # builds). Cosmetic; reproduces the plan's exact code and passes rubocop.
  • 🔵 Code quality — the dockerhub credentials file is read twice (binread for the header check, then YAML.load_file); a single branched read would be tidier. Bounded by the plan's no-restructure rule.

Review generated by /accelerator:review-pr

concurrency:
group: main-tags
cancel-in-progress: false
queue: max

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Plan concern (correctness, safety)queue: max is flagged as an unsupported concurrency key (only group and cancel-in-progress are documented), which would invalidate the workflow or fall back to cancel-all-but-newest — the opposite of the intended "keep every queued run". Family plan §4.2 specifies this line verbatim and cites the 2026-05-07 larger concurrency queues GA changelog, which post-dates this reviewer's knowledge cutoff, so it is recorded as a plan concern rather than a defect. Worth confirming live that the feature is GA and the key is honoured — if not, main.yaml breaks.

concurrency:
group: main-tags
cancel-in-progress: false
queue: max

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Plan concern (correctness, safety) — same queue: max question on the release job. If the plan's GA claim holds, a maintainer-approved release queued behind an in-progress run is preserved; if not, it could be silently superseded. Plan-documented (§4.2), non-blocking.

# fleet-wide change, not per-repo drift.
if ! command -v docker-compose >/dev/null 2>&1; then
curl -fsSL \
"https://github.com/docker/compose/releases/download/v5.3.1/docker-compose-linux-x86_64" \

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Plan concern (correctness) — reviewer questions whether docker-compose v5.3.1 exists (Compose has historically been a v2.x line). Plan §4.7 states the version and its sha256 were verified on 2026-07-20 (published 2026-07-07, latest stable) and explicitly asks reviewers not to re-raise it. Post-cutoff; recorded as a plan concern. If a runner lacks a standalone binary and the pin is wrong, the test job fails at tool install — the PR's own test check is the authoritative signal.

Comment thread .github/workflows/pr.yaml
- name: Install secrets tools
run: ./scripts/ci/common/install-git-crypt.sh
- name: Unlock git-crypt
run: ./go git_crypt:unlock_with_encrypted_gpg_key

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟠 Plan concern (security) — the prerelease job unlocks ENCRYPTION_PASSPHRASE and runs ./go from the PR head for same-repo, non-dependabot PRs, so a write-access collaborator could modify the tree to exfiltrate the passphrase from unreviewed code. This is the plan's D9 prerelease-publish design, documented as a permanent deliberate feature; the trust boundary (any write-access user) is the accepted tradeoff.

Comment thread .github/workflows/pr.yaml
needs: [check, test]
runs-on: ubuntu-latest
timeout-minutes: 10
if: github.event.pull_request.user.login == 'dependabot[bot]'

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Plan concern (security)merge-pull-request auto-merges any dependabot PR passing check/test with no update-type restriction. In the §1 parity list and D3: merges do not trigger a release and ship with the next human release; --match-head-commit pins the tested SHA. Deliberate parity; post-migration hardening may add a filter.

@phelma
phelma marked this pull request as ready for review July 24, 2026 14:32
@phelma

phelma commented Jul 24, 2026

Copy link
Copy Markdown
Author

Ready to merge. The automated review and checks have passed; a maintainer can merge this when ready.

atomic-foundry-event · foundry-run: 2026-07-24T13-38-41-126Z

@phelma

phelma commented Jul 28, 2026

Copy link
Copy Markdown
Author

Closing: re-running the migration pipeline from scratch against the amended family plan. Superseded by a fresh PR.

@phelma phelma closed this Jul 28, 2026
@phelma
phelma deleted the gha-migration branch July 28, 2026 10:19
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant