fix(tooling): join runner observations to guarded sites on (file, job, step) - #13245
Merged
Conversation
…, step)
`measure-stall-guard-headroom` keyed its join on the step NAME alone, an
identifier that is unique only inside one job used as if it were global --
the same mistake `attachSiblings` documents one level up on the job axis.
Two guarded steps sharing a `name:` merged into one bucket and each site was
then judged against the worst reading of the union. The failure is silent:
the output is identical in shape to a correct run. It is also not
consistently conservative -- with asymmetric observation sets a site that
never ran is printed MEASURED and COVERED, quoting another job's reading.
Attribution now resolves every observation against the identity the workflow
file declares (workflow `name:`, the job's `name:` template with `${{ }}`
standing for one expansion). One survivor is attributed; none is EXCLUDED and
reported; more than one REFUSES rather than picking the worst, the first or
the average.
`--root <dir>` is what makes the refusal demonstrable: no duplicate step name
exists in this tree, and a refusal nobody can trigger is decoration.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01CPrUz21stTFhJRUirdc4yw
os-elon
marked this pull request as ready for review
August 29, 2026 14:53
os-elon
enabled auto-merge
August 29, 2026 14:53
github-merge-queue
Bot
removed this pull request from the merge queue due to no response for status checks
Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #13121
scripts/measure-stall-guard-headroom.mjsjoined GitHub's runner observations onto guarded sites by step name alone, while a site is identified by the triple(file, job, step)— the triple the tool already prints in every one of its own lines.The premise, measured before implementing
Re-derived on
origin/mainat5a9b7a0b10: 7 guard-wrapped steps, all names distinct, and(file, job, step)unique across the population — so the defect was LATENT and no reading on this tree was wrong. No duplicate name has arrived since the card was written, and no fourth join component is needed. The self-test now asserts both facts, so the day either stops holding is the day the suite says so.The card's sharpest claim is TRUE
The card says the error direction "is not even consistently conservative". It is not, and the failure is worse than a wrong number. Driving the pre-fix code over two co-named guarded steps in different jobs, with only the faster one actually executed:
The
tightsite never ran in that payload. The honest line isNOT OBSERVED. Instead the tool printed a confidentCOVEREDfor it, quoting a reading from a different job in its own evidence line, and exited 0. Reversing the observation sets flips the error the other way — a genuinely covered site reportedUNCOVERED by 50m00s. Both outputs are identical in shape to a correct run.What changed
Attribution resolves every observation against the identity the workflow file declares for it — the workflow's
name:, and the job'sname:template with each expression standing for one expansion, so a real matrix leg such asTest Core (3/6)matches. A component the payload does not carry is UNKNOWN: it can neither confirm nor exclude a candidate.NOT OBSERVED, the direction that cannot invent headroom;The narrowing is unconditional, not reserved for co-named steps. "The triple when the name happens to be ambiguous" is the same defect with the collision size fixed at one, and it is reachable with no duplicate
name:anywhere — an unguarded step copying a guarded step's name.On the collection end (
const wanted = new Set(swept.sites.map((s) => s.step))), asked about on the card: it stays keyed on the name, deliberately. As a filter the name yields a superset of what any per-site join can use, so it cannot under-collect. What did under-collect was the observation record — it discardedworkflow_name, leaving the join blind on two of the three axes even when the payload carried them. That is what changed.--root DIRis added because the refusal branch cannot be reached from this tree, and a refusal nobody can trigger is decoration.check-stall-guard-budget.mjsis not touched: itsT - C >= Wcriterion is deliberately independent ofpands, and this card is the join in the measurement tool. Its own self-test still passes 61 assertions.Verification — all at
39f19b2fe0--self-test: 32 assertions passed (was 21). Reverse proof, predictions named before each run, each mutation proven on disk by decomposed before/aftergrep -cFcounts on the removed and injected text plusgit hash-object, each restore proven by blob equality against the HEAD blob and an emptygit diff HEAD, under an absolute-pathtrap. No rebuild leg applies —nodeloads this.mjsfrom source, with nodiston the resolution path.tightsite printedCOVERED, 8m00s to sparequotingworst on \Loose Job`` — the defect reproduced verbatim. The real-tree case stayed green: that is the latencyDerived gate family —
node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack(12 families, derived from the real change set, not a hand-built diff):check:agent-test-spelling,check:bash32-floor,check:cli-command-ids,check:cross-package-test-inputs,check:entry-guard,check:parse-guard,check:pnpm-filter-targets,check:watch-hint-literal,check-ci-filter-parity.mjs,check-cross-package-test-inputs.mjs,check-shard-attestation.mjs— all green, each read from its own printed verdict line, each exit code captured before any pipe.check-test-completeness.mjsargument-less is aPREREQUISITE NOT METrefusal (exit 3), NOT a red: it grades a savedturbo run testlog CI tees for it, so it is NOT MEASURED locally by design.Repo-wide
pnpm lint(eslint . --no-inline-config, the whole farm, no narrowing):VERDICT command-exit 0, run throughscripts/pm/os-verify-lock.sh.check-nul-bytes: OK over 7342 text files, plus a direct control-byte scan of the changed file.No changeset: the diff is one internal
scripts/measurement tool that is not published and not wired into any workflow, so this PR releases nothing —skip-changeset.Filed separately, not fixed here
This tool's
--self-testis not wired into anycheck:*family, so its 32 assertions — the refusal branches included — never run in CI, unlike its sibling's (check:stall-guard-budget). Wiring it needs a new lint-farm entry, which is a new verification surface, so it is recorded rather than smuggled in. See the report.Generated by Claude Code
Generated by Claude Code