Skip to content

Commit 9a9381c

Browse files
aledbfclaude
andcommitted
refactor(ci): replace shell scripts with a Go devtool command
The CI helper scripts (parity-affected, coverage gate/report/merge, daily-changed, ci-prepare-runner, docker-cache-export, pkgcov.awk) carried real decision logic — the parity command selection, coverage floors and profile parsing — with zero tests and shell's fragility (a dropped trailing newline silently changed the parity scope). Port them to Go: pure, unit-tested logic in internal/devtools and a thin cmd/devtool multiplexer that wires stdin/git/`go tool`/docker around it. The Taskfile and workflow now call `go run ./cmd/devtool <sub>`. Behavior is preserved: every ported command was diffed against the original script (parity-affected mapping, affected-commands over a real git diff, coverage report/gate stdout+exit, daily-changed) and matches byte-for-byte. Those captures became the golden tables in the devtools tests. Also drops the experimental arm64 parity job: it never gated, ran only on manual dispatch, and is no longer wanted. Tests: internal/devtools covers ParityAffected (golden corpus incl. the trailing-newline behavior), ParseProfile/RenderReport/CoverageGate (golden vs the old awk+sh), and the DailyChanged boundary; the exec-the-shell daily test is replaced by the native one. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
1 parent 7ded813 commit 9a9381c

20 files changed

Lines changed: 938 additions & 473 deletions

‎.github/workflows/go-cli.yml‎

Lines changed: 17 additions & 46 deletions
Original file line numberDiff line numberDiff line change
@@ -34,7 +34,7 @@ jobs:
3434
- run: task coverage
3535
- name: Coverage summary
3636
if: always()
37-
run: bash scripts/coverage-report.sh coverage.out "unit (hermetic)" >> "$GITHUB_STEP_SUMMARY"
37+
run: go run ./cmd/devtool coverage-report coverage.out "unit (hermetic)" >> "$GITHUB_STEP_SUMMARY"
3838
- run: task test:integration
3939
- run: task build
4040
- run: ./devcontainer --version
@@ -76,7 +76,7 @@ jobs:
7676
- run: task coverage:e2e
7777
- name: Coverage summary
7878
if: always()
79-
run: bash scripts/coverage-report.sh artifacts/coverage/data/e2e "e2e" >> "$GITHUB_STEP_SUMMARY"
79+
run: go run ./cmd/devtool coverage-report artifacts/coverage/data/e2e "e2e" >> "$GITHUB_STEP_SUMMARY"
8080
- uses: actions/upload-artifact@v7
8181
if: always()
8282
with:
@@ -115,7 +115,7 @@ jobs:
115115
- run: task coverage:parity-contract
116116
- name: Coverage summary
117117
if: always()
118-
run: bash scripts/coverage-report.sh artifacts/coverage/data/contract "parity:contract" >> "$GITHUB_STEP_SUMMARY"
118+
run: go run ./cmd/devtool coverage-report artifacts/coverage/data/contract "parity:contract" >> "$GITHUB_STEP_SUMMARY"
119119
- if: always()
120120
run: |
121121
mkdir -p artifacts
@@ -135,7 +135,7 @@ jobs:
135135

136136
# Runtime lane of the parity matrix (creates real containers via Docker) on
137137
# push/PR, SCOPED to the commands affected by the change: the full matrix takes
138-
# ~45 min, so scripts/parity-affected.sh maps the diff to a
138+
# ~45 min, so the devtool parity-affected mapping reduces the diff to a
139139
# PARITY_COMMAND allowlist (or "all"/"none"). The daily parity-runtime-full job
140140
# is the backstop that always runs the whole thing.
141141
parity-runtime-shard:
@@ -163,6 +163,11 @@ jobs:
163163
submodules: recursive
164164
# Full history so the diff base (PR base / previous push) is present.
165165
fetch-depth: 0
166+
# devtool (the affected-commands mapping) runs before the composite setup,
167+
# so it needs the Go toolchain here.
168+
- uses: actions/setup-go@v6
169+
with:
170+
go-version-file: go.mod
166171
- name: Determine affected parity commands
167172
id: affected
168173
run: |
@@ -171,7 +176,7 @@ jobs:
171176
else
172177
base="${{ github.event.before }}"; head="${{ github.sha }}"
173178
fi
174-
bash scripts/parity-affected-commands.sh "$base" "$head" >> "$GITHUB_OUTPUT"
179+
go run ./cmd/devtool parity-affected-commands "$base" "$head" >> "$GITHUB_OUTPUT"
175180
# Toolchain + runner prep + reference oracle, shared with parity-runtime-full.
176181
- if: steps.affected.outputs.run == 'true'
177182
uses: ./.github/actions/setup-parity-runtime
@@ -212,7 +217,7 @@ jobs:
212217
213218
# Gate the expensive daily lanes on "did the repo change today?". A manual dispatch
214219
# always runs; a scheduled run only proceeds if HEAD has a commit within the last
215-
# day (scripts/daily-changed.sh). On a private/billed repo this skips the full
220+
# day (devtool daily-changed). On a private/billed repo this skips the full
216221
# runtime matrix + publish on quiet days. Runs on schedule/dispatch so it is never
217222
# skipped for those events — the heavy jobs `needs:` it, and a skipped need would
218223
# skip them regardless of its output.
@@ -223,13 +228,16 @@ jobs:
223228
run: ${{ steps.check.outputs.run }}
224229
steps:
225230
- uses: actions/checkout@v7
231+
- uses: actions/setup-go@v6
232+
with:
233+
go-version-file: go.mod
226234
- id: check
227235
run: |
228236
if [ "${{ github.event_name }}" = "workflow_dispatch" ]; then
229237
echo "manual dispatch → always run" >&2
230238
echo "run=true" >> "$GITHUB_OUTPUT"
231239
else
232-
bash scripts/daily-changed.sh >> "$GITHUB_OUTPUT"
240+
go run ./cmd/devtool daily-changed >> "$GITHUB_OUTPUT"
233241
fi
234242
235243
# Full runtime parity matrix. Runs once a day (schedule, only when the repo changed
@@ -267,7 +275,7 @@ jobs:
267275
- run: task coverage:parity-runtime
268276
- name: Coverage summary
269277
if: always()
270-
run: bash scripts/coverage-report.sh artifacts/coverage/data/runtime "parity:runtime shard ${{ matrix.shard }}" >> "$GITHUB_STEP_SUMMARY"
278+
run: go run ./cmd/devtool coverage-report artifacts/coverage/data/runtime "parity:runtime shard ${{ matrix.shard }}" >> "$GITHUB_STEP_SUMMARY"
271279
- if: always()
272280
run: |
273281
mkdir -p artifacts
@@ -306,7 +314,7 @@ jobs:
306314
- run: task coverage:parity-publish
307315
- name: Coverage summary
308316
if: always()
309-
run: bash scripts/coverage-report.sh artifacts/coverage/data/publish "parity:publish" >> "$GITHUB_STEP_SUMMARY"
317+
run: go run ./cmd/devtool coverage-report artifacts/coverage/data/publish "parity:publish" >> "$GITHUB_STEP_SUMMARY"
310318
- uses: actions/upload-artifact@v7
311319
if: always()
312320
with:
@@ -365,43 +373,6 @@ jobs:
365373
esac
366374
done
367375
368-
# Experimental, NON-gating: arm64 runtime via QEMU emulation. arm64 runtime is
369-
# unsupported for now, so its cases are skipped-arm64 in the gate above. This job
370-
# is SLOW (QEMU) and only runs on a manual trigger (workflow_dispatch), not on
371-
# every push/PR — enable it when validating arm64.
372-
parity-runtime-arm64-experimental:
373-
if: github.event_name == 'workflow_dispatch'
374-
runs-on: ubuntu-latest
375-
needs: lint-and-test
376-
continue-on-error: true
377-
env:
378-
PARITY_ARM64: "true"
379-
PARITY_RUNTIME_TIMEOUT: "15m"
380-
steps:
381-
- uses: actions/checkout@v7
382-
with:
383-
submodules: recursive
384-
- name: Set up QEMU (arm64 emulation)
385-
uses: docker/setup-qemu-action@v4
386-
with:
387-
platforms: arm64
388-
- uses: actions/setup-go@v6
389-
with:
390-
go-version-file: go.mod
391-
- uses: actions/setup-node@v6
392-
with:
393-
node-version: "20"
394-
- uses: go-task/setup-task@v2
395-
with:
396-
version: 3.x
397-
repo-token: ${{ secrets.GITHUB_TOKEN }}
398-
- run: task ci:prepare-runner
399-
- run: task reference
400-
- run: task parity:runtime
401-
- if: always()
402-
run: task clean
403-
404-
405376
# Cross-lane coverage: merge the per-PR lanes' covdata (unit + e2e + contract)
406377
# into the one number no single lane shows, and post it to the run summary.
407378
coverage-report:

‎Taskfile.yml‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -90,9 +90,9 @@ tasks:
9090
# -test.gocoverdir also emits binary covdata so `coverage:merge` can union it
9191
# with the e2e/parity lanes.
9292
- go test ./cmd/... ./internal/... -skip '^TestClient$|^TestE2E|^TestParityMatrix$|^TestPublishParity$' -coverprofile=coverage.out -count=1 -args -test.gocoverdir={{.ROOT_DIR}}/{{.COVER_DATA}}/unit
93-
- bash scripts/coverage-report.sh coverage.out "unit (hermetic)"
93+
- go run ./cmd/devtool coverage-report coverage.out "unit (hermetic)"
9494
# Ratchet: total + per-package floors (parsers must not mask lost CLI/OCI coverage).
95-
- bash scripts/coverage-gate.sh coverage.out TOTAL={{.COVERAGE_MIN}} cli={{.COVERAGE_CLI_MIN}} docker={{.COVERAGE_DOCKER_MIN}} lifecycle={{.COVERAGE_LIFECYCLE_MIN}} oci={{.COVERAGE_OCI_MIN}}
95+
- go run ./cmd/devtool coverage-gate coverage.out TOTAL={{.COVERAGE_MIN}} cli={{.COVERAGE_CLI_MIN}} docker={{.COVERAGE_DOCKER_MIN}} lifecycle={{.COVERAGE_LIFECYCLE_MIN}} oci={{.COVERAGE_OCI_MIN}}
9696

9797
coverage:e2e:
9898
desc: Coverage of the real in-process E2E paths (requires Docker)
@@ -104,7 +104,7 @@ tasks:
104104
# -coverpkg instruments the whole module (these tests invoke NewRootCommand
105105
# directly); -test.gocoverdir emits binary covdata for the merge.
106106
- go test ./internal/cli -run '^TestE2E' -count=1 -timeout 5m -coverpkg=./... -args -test.gocoverdir={{.ROOT_DIR}}/{{.COVER_DATA}}/e2e
107-
- bash scripts/coverage-report.sh {{.COVER_DATA}}/e2e "e2e"
107+
- go run ./cmd/devtool coverage-report {{.COVER_DATA}}/e2e "e2e"
108108

109109
# coverage:_parity-lane runs the instrumented CLI (build:coverage) through a
110110
# parity lane, collecting covdata into COVER_DATA/<LANE> and printing the table.
@@ -115,7 +115,7 @@ tasks:
115115
cmds:
116116
- rm -rf {{.COVER_DATA}}/{{.LANE}} && mkdir -p {{.COVER_DATA}}/{{.LANE}}
117117
- GOCOVERDIR={{.ROOT_DIR}}/{{.COVER_DATA}}/{{.LANE}} CLI_GO={{.ROOT_DIR}}/{{.COVER_DIR}}/bin/{{.BINARY}} {{.RUN}}
118-
- bash scripts/coverage-report.sh {{.COVER_DATA}}/{{.LANE}} "parity:{{.LANE}}"
118+
- go run ./cmd/devtool coverage-report {{.COVER_DATA}}/{{.LANE}} "parity:{{.LANE}}"
119119

120120
coverage:parity-contract:
121121
desc: Coverage of the instrumented binary through the hermetic contract parity lane
@@ -135,8 +135,8 @@ tasks:
135135
msg: Docker is required for runtime parity coverage
136136
- sh: docker info 2>/dev/null | grep -q io.containerd.snapshotter
137137
msg: The parity runtime lane requires the containerd image store (docker build --platform on local images); enable it with `task ci:prepare-runner`.
138-
- sh: bash scripts/docker-cache-export.sh
139-
msg: The Docker builder must support cache export (see scripts/ci-prepare-runner.sh)
138+
- sh: go run ./cmd/devtool docker-cache-export
139+
msg: The Docker builder must support cache export (see `devtool ci-prepare-runner`)
140140
cmds:
141141
- task: coverage:_parity-lane
142142
vars:
@@ -160,7 +160,7 @@ tasks:
160160
coverage:merge:
161161
desc: Union every per-lane covdata dir into the true cross-lane coverage report
162162
cmds:
163-
- bash scripts/coverage-merge.sh {{.COVER_DATA}} {{.COVER_DATA}}/merged "merged (all lanes)"
163+
- go run ./cmd/devtool coverage-merge {{.COVER_DATA}} {{.COVER_DATA}}/merged "merged (all lanes)"
164164

165165
test:e2e:
166166
desc: Run Go E2E tests with Docker
@@ -236,11 +236,11 @@ tasks:
236236
# Fail fast (~1s) if the builder can't export build cache — otherwise the
237237
# runtime cases (build.cache-to-*, build.output-oci-*) fail ~4 min deep with
238238
# a cryptic buildkit error. The default `docker` driver lacks cache export.
239-
- sh: bash scripts/docker-cache-export.sh
239+
- sh: go run ./cmd/devtool docker-cache-export
240240
msg: |
241241
The Docker builder cannot export build cache (needed by the runtime parity
242242
matrix: build.cache-to-*, build.output-oci-*). Enable the containerd image
243-
store (scripts/ci-prepare-runner.sh) or use a docker-container buildx builder.
243+
store (`devtool ci-prepare-runner`) or use a docker-container buildx builder.
244244
cmds:
245245
# -parallel default 2 for gate determinism: higher values overload the docker
246246
# daemon on contended runners and transiently fail compose/heavy cases (they
@@ -261,7 +261,7 @@ tasks:
261261
ci:prepare-runner:
262262
desc: "CI-ONLY: free disk + enable the containerd image store (uses sudo; not for local dev)"
263263
cmds:
264-
- bash scripts/ci-prepare-runner.sh
264+
- go run ./cmd/devtool ci-prepare-runner
265265

266266
clean:
267267
desc: Remove generated artifacts and test containers

0 commit comments

Comments
 (0)