Skip to content

fix(ci): pin the measuring toolchain to 1.99.1 - #135

Open
aliasfoxkde wants to merge 5 commits into
mainfrom
fix/ci-pinned-toolchain
Open

aliasfoxkde wants to merge 5 commits into
mainfrom
fix/ci-pinned-toolchain

Conversation

@aliasfoxkde

Copy link
Copy Markdown
Owner

The GitHub gate job went red on PRs whose Rust source was byte-identical to main's. PR #134's tree differs from main only in docs/ and packaging/:

git diff --shortstat origin/main origin/chore/windows-recipe-audit \
    -- crates xtask Cargo.toml Cargo.lock

is empty — and #134 measured 77.0% lines / 76.1% functions against the 81.1 / 78.7 floor, while main measured 81.9 / 78.9 and passed.

The differentiator is the compiler, not the code:

GitHub run tree rustc measurement
43157085372 main @ bc8abfb 1.99.1 81.9 / 78.9 ✓
43169461346 #130 @ 002eb56e 1.99.1 81.9 / 78.8 ✓
43284817650 #134 @ f570c86 1.100.0 77.0 / 76.1 ✗
43284861290 #134 @ f570c86 1.100.0 77.0 / 76.1 ✗
43287370012 #134 @ 545e7a5 1.100.0 77.0 / 76.1 ✗

The workflow installed toolchain: stable; rustc 1.100.0 (2026-11-09) reached the runners on 2026-11-12. The GitForge pipeline did not move — its image is rust:1.99 (rustc 1.99.1) — which is why the two pipelines disagreed about the same commit. The build cache is not the explanation: the failing runs restored the same Cargo.lock-keyed cache as the passing ones, and the last failing run restored a warm cache and still read 77.0.

So the floor was never a property of the source tree alone. It is a property of the tree and the compiler that instrumented it, and the workflow said stable.

This change

  • ci.yml (fmt and gate) and release.yml install toolchain: 1.99.1 — the compiler the floors were measured with. A release is built with the compiler the gate judged.
  • infrastructure/docker/ci-rust.Dockerfile builds rust:1.99.1-slim-bookworm. That tag is the same image digest as rust:1.99-slim-bookworm today (sha256:392f2219…), so the runner's image does not change; the explicit patch stops a future rust:1.99 from moving under the floor.
  • xtask/src/coverage.rs prints measured with rustc <version> and warns when that is not the version the floors were measured with. The gate warns rather than fails, because a developer's local compiler is not the CI compiler.

The floor is not re-blessed. 81.1 / 78.7 stands. The measurement moved, not the code, and lowering a floor to meet a moving measurement is the weakening the gate exists to prevent.

Decision 0102; ticket T27-57.

Verification

  • python3 scripts/validate_workflow_pins.py .github/workflows and its self-test pass.
  • rustfmt --edition 2024 --check xtask/src/coverage.rs is clean.
  • The new unit tests cover the version parser; the workspace gate (cargo run -p xtask -- ci) runs in CI — local cargo builds are hook-denied on this fleet, so the CI receipt is the gate evidence for this change.

Note: this branch is based on PR #130's head, so the diff below carries #130's changes until #130 lands; after #130 merges the diff is this change alone.

🤖 Generated with Claude Code

Michael Kinney and others added 4 commits November 11, 2026 23:58
The release workflow builds the artifacts and creates the draft release,
and it named its actions by mutable reference: `actions/checkout@v4`,
`dtolnay/rust-toolchain@stable`, `actions/cache@v4`,
`actions/upload-artifact@v4`, `actions/download-artifact@v4` and
`cloudflare/wrangler-action@v3`. A tag can be moved and a branch moves on
its own, so the release path executed code that could change with no diff
to review here. All 19 action references now name a 40-hex commit SHA,
with the version kept in a trailing comment.

The rule is enforced, not reviewed: `scripts/validate_workflow_pins.py`
walks the workflow directory and fails with the file and line of every
unpinned reference, and it runs in the backup CI's `changes` job — the job
every push and pull request executes, and the one `fmt` and `gate` depend
on. `scripts/validate_workflow_pins-selftest.sh` demonstrates each failure
mode with one deliberately broken input (a mutable tag, a mutable branch,
a `docker://` image, a short SHA) and the two inputs the validator must
accept.

The deploy workflow triggered on a push that touched the workflow file
itself, so every edit to it published the marketing site. The trigger is
now the content it publishes — `site/**` and `wrangler.toml` — and a
deliberate redeploy is `workflow_dispatch`.

Decision 0099; ticket T27-57.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The release workflow decided what a tag meant nowhere: it derived the
artifact names from the tag, created every release as an ordinary draft,
and had no notion of a platform. The aggregate validation that stands
between a build and a release was a hundred lines of inline YAML that ran
once, on a tag, with nothing around it to tell it apart from a run that
was merely quiet.

Three seams are now scripts with self-tests, and all three run in the
backup CI's `changes` job:

* `scripts/classify_release_mode.sh` maps `vX.Y.Z` to a production
  release and `vX.Y.Z-dryrun.N` to a rehearsal, and refuses every other
  shape — a two-component version, a missing `v` prefix, an unknown
  suffix, a `-dryrun` without a number, a non-numeric number, a trailing
  hyphen, and a tag that names another version. A manual
  `workflow_dispatch` run may only rehearse; cutting a production
  release still requires a tag push.
* `scripts/validate_release_aggregate.sh` holds the fail-closed gate. It
  now also rejects an artifact the release does not publish, and requires
  every required platform to be represented in the release manifest — a
  skipped build was indistinguishable from a build nobody asked for.
* `packaging/release-assets.sh` takes a mode. A production release is a
  draft titled by the version and described by its tracked release notes;
  a rehearsal is a prerelease titled `TEST <version>` whose notes carry a
  banner saying it must not be published.

The rehearsal's disposable tag is created at the exact commit the build
ran on, so the artifacts and the tag bind to one SHA, and an existing tag
is never overwritten. The artifacts are named by the version the build
produced, not by the tag: the tag says what is being released, the
version says what was built.

Decision 0100 records the reasoning; the ticket, the execution board, the
task ledger, RELEASING.md and HANDOFF.md carry the new shape.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The release gate now refuses a production release whose description is
not a tracked `docs/releases/<tag>.md` note, but the release-workflow
tests still staged a bare `notes.md` — so the three tests that drive a
production `gh release create` failed on the new rule. The fixtures
now stage the notes where a real release keeps them.

Co-Authored-By: Claude Code <noreply@anthropic.com>
The coverage floor is a claim about a compiler as much as about the
source. rustc 1.100.0 measures a byte-identical Rust tree at 77.0%
lines / 76.1% functions where 1.99.1 measures 81.9% / 78.9%: the same
commit, the same tree, on either side of the floor. The workflows
installed a floating `stable`, so the floor moved silently on
2026-11-12, while the GitForge image stayed on 1.99 and the two
pipelines disagreed about the same commit.

Pin ci.yml, release.yml and the CI image to 1.99.1, and let the gate
name the compiler that measured it. The floor is not re-blessed: the
measurement moved, not the code.

Decision 0102.

Co-Authored-By: Claude Code <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Nov 12, 2026 •

Copy link
Copy Markdown
Contributor

Warning

Review limit reached

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

Next included review available in 23 minutes.

Check out review usage here.

View limit details

Limit details: You’ve used the included review currently available.

Learn how review limits work.

Review configuration:

Run configuration

  • Source mode: Auto
  • Repository: Unavailable
  • Branch: Unavailable
  • File: No configuration file from this repository applied
  • Commit: Unavailable
  • Layers: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 5353302a-10fb-4c93-afec-fc479fe50c71
Commits

Reviewing files that changed from the base of the PR and between 6f106af and af85b6e.

Files selected for processing: 10

  • .github/workflows/ci.yml
  • .github/workflows/release.yml
  • docs/GAPS.md
  • docs/NEXT.md
  • docs/TASKS.md
  • docs/decisions/0102-coverage-floor-is-measured-with-a-pinned-toolchain.md
  • docs/release/EXECUTION-BOARD.md
  • docs/tasks/T27-57-workflow-hardening.md
  • infrastructure/docker/ci-rust.Dockerfile
  • xtask/src/coverage.rs
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

Share

# Conflicts:
#	.github/workflows/release.yml
#	docs/TASKS.md
#	docs/release/EXECUTION-BOARD.md
#	docs/tasks/T27-57-workflow-hardening.md
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants