Skip to content

Survey: which scripts/** self-tests cannot prove they ran — and the two that now can - #13797

Merged
os-project-manager merged 5 commits into
mainfrom
claude/issue-13489-selftest-assertion-floor-survey
Aug 31, 2026
Merged

Survey: which scripts/** self-tests cannot prove they ran — and the two that now can#13797
os-project-manager merged 5 commits into
mainfrom
claude/issue-13489-selftest-assertion-floor-survey

Conversation

@claude

@claude claude Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Fixes #13489

Survey of every scripts/** self-test for the two holes PR #13487 measured on check-doc-authoring.mjs, plus the repair applied to a named tier and an instrument the follow-up cards can re-run.

All figures measured on 597020aa5 (the branch base) unless stated; the census numbers below are re-derived on this branch's head 0d58ba20e, where two of them have already moved because of this PR.

The criterion actually used, and how it is wider than the grep

The ruling's grep (failures.length === 0, return selfTest()) took the candidates. It is not what decided anything: it catches two spellings, and gates reach the same effect several other ways. The criterion is per hole, and the two holes are orthogonal — counted separately, never summed.

Hole 1 — no assertion floor. Static, and stated so. A floor is a comparison whose left side counts what actually ran and whose right side is a declared expectation, on a path that produces a failure. The important half of that sentence is "produces a failure": an earlier revision of this criterion keyed on the name SELF_TEST_BATTERIES and called a fixture floored after the roster had been deleted — the control caught it, nothing else would have. The criterion reads names, so a floor spelled with names it does not know reads as unfloored: its error runs in one direction only, and the population it reports is an upper bound. The tree was independently hand-swept for zero-case refusals (checked === 0, if (!ran), ...) outside those names; every hit was a production-scan refusal, not a self-test floor.

Hole 2 — no verdict handshake. Measured, not read. return selfTest() is only the most visible spelling. process.exit(selfTest()) is the same hole wearing a handshake's clothes: an early bare return yields undefined, and process.exit(undefined) is exit 0. selfTest(); main(); is the same hole again, and there the self-test's silence is then covered by a production success line. So the decision is behavioural: inject return; as the first statement of the function the --self-test dispatch calls, run it, read the exit code before any pipe. Exit 0 is the defect.

Two mechanical controls sit under every hole-2 reading: the mutated bytes are re-read from disk before the run (an editor step that matched nothing exits 0 exactly like one that landed), and a mutation whose output and exit code are byte-identical to the baseline is recorded NOT MEASURED rather than "held" — it never reached the executed path. Zero of the 158 tripped that second control, so every reading below is about a mutation that ran.

A2.5 — the positive control, and what it caught

The whole subject here is a green that asserted nothing, so an instrument that silently misses a class of files and reports zero commits this card's own defect. Both instruments are driven against a known-holed and a known-sound fixture on every invocation, and a control failure refuses — it does not degrade to a smaller number.

The positive control is the pre-fix shape of check-doc-authoring.mjs, rebuilt from the live file by stripping the verdict handshake and the roster:

P1 classifier on the PRE-FIX fixture : NONE ✓ caught
P1 classifier on the LIVE file      : ROSTER
   on-disk marker in mutated fixture   : 1
P2 probe: PRE-FIX fixture, unmutated : exit 0 (4772 bytes of output)
P2 probe: PRE-FIX fixture, early ret : exit 0 (0 bytes of output) ✓ DEFEATED — the probe sees the known hole

The control has already earned its keep once, on this run: the first classifyFloor was name-only and called the stripped fixture floored. The refusal is the reason that revision is not what shipped.

The live check-doc-authoring.mjs is the negative control and reads HELD, naming its own sentinel refusal.

A2.2 — the two numbers, separately

Population: 158 files under scripts/** whose code (comments masked) dispatches on --self-test. Not "mentions": 165 carry the literal in code; 7 of those hand it to a child process instead of dispatching on it.

Hole 1 — no assertion floor: 155 of 158 at the branch base. Floored: 1 (check-doc-authoring.mjs, the gate #13487 fixed). Two further files matched the count-comparison arm and were hand-read as false positivescheck-refd-timer-probe.mjs compares count < 100 in its production probe, and measure-stall-guard-headroom.mjs tests total < 0 for a sign. Neither is a case floor. On this branch's head the figure is 153 of 158.

Hole 2 — silently defeated by an early return: 146 of 154 measured at the branch base. 3 not applicable (the self-test is an inline top-level block with no callee to leave early), 1 not measurable, 8 held. 137 of the defeated printed nothing at all and still exited 0 — the exact shape the ruling names. On this branch's head: 144 DEFEATED, 9 HELD, 5 NOT MEASURED.

⛔ These are not two views of one set and are never added. check-doc-authoring.mjs is the only file clean on both. Of the 8 that held at base, exactly one held by design.

The 8 that held, and why 6 of them are luck rather than a guarantee

gate why it did not exit 0 is that protection?
check-doc-authoring.mjs its sentinel handshake refused yes, by design
check-page-declaration-shape.mjs process.exit(selfTest() ? 0 : 1), undefined is falsy partial — stops a bare return, not return true, and prints nothing while doing it
check-osv-exemptions.mjs const { passed, lines } = selfTest() destructured undefined no — a TypeError
typecheck-configs.mjs for (const failure of failures) iterated undefined no — a TypeError
check-platform-checklist.mjs spread of undefined no — a TypeError
checklist-select.mjs selfTest(); main();main() then rejected --self-test as a selector, exit 2 no — a usage error
run-with-stall-guard.mjs fell through to the option loop: "unknown option --self-test" no — an argument error
check-exported-any-returns.mts fell through to the production path, which requires --package no — a missing-argument throw

A self-test whose early return happens to crash its caller is not protected; it is lucky, and the luck is one refactor deep. Six of these eight are that.

Boundary — #13014 is adjacent, not the same

Kept apart, and the tree proves the two do not cover for each other. check-adr-links.mjs and check-doc-anchors.mjs both carry a full empty-scan refusal — "A green run over nothing is not a green run: either the directory moved or the extractor stopped matching." — and both are defeated by hole 2, measured. A gate can be clean on one class and defeated by the other, so #13014's coverage argues nothing away here.

A2.3 — the hits, tiered by transplantability

The #13487 shape does not transplant uniformly. Sorted by what the transplant actually costs:

Tier A — the handshake is mechanical everywhere (144 files). A sentinel constant, a return SENTINEL after the verdict, and a compare at the dispatch. Identical in every gate whatever the self-test's internal shape, because it touches only the boundary. This is the cheap half of the repair and it closes the hole the ruling calls the nastier one.

Tier B — the roster floor transplants directly (the majority of Tier A). Self-tests already sectioned into named groups — most carry // ── ... ── section banners or block-scoped groups — take the roster verbatim: declare the names, open each section with battery('...'), attribute each assertion to the open battery. Both gates fixed in this PR were Tier B, and neither needed a single case rewritten.

Tier C — reshaping needed before the roster means anything.

  • 3 inline top-level self-test blocks with no entry function at all: check-regen-pending.mjs, git-merge-regen.mjs, setup-git-hooks.mjs. There is nothing to return from, so the hole-2 mutation does not apply in that form; they need the block extracted into a function before either repair fits.
  • 2 multi-entry dispatches that call several self-test functions and combine their statuses: check-platform-checklist.mjs (four) and check-durability-degradation-log-level.mjs (two). One handshake per callee, or one wrapper — a decision, not a transplant.
  • table-driven self-tests with no battery concept, where the natural roster is the table's own named rows rather than sections.

Tier D — not measurable as it stands (1 file). scripts/pm/dispatch-gates.mjs: injecting at the anchor lands inside a template literal and produces a SyntaxError, so no run of it measures anything. Recorded NOT MEASURED, never as a pass.

A2.4 — what this PR fixes, and the criterion for that choice

Two gates, both fully: scripts/check-self-test-wired.mjs and scripts/check-self-test-workflow-commands.mjs.

The criterion is not "the easy ones" — Tier A is uniformly easy, and 144 of them exist. It is: the gates whose own subject is the gate farm's ability to prove itself. check-self-test-wired is the gate that enforces that every shipped --self-test is actually run by CI; check-self-test-workflow-commands is the gate that keeps self-test output from forging runner annotations. A hole in either is circular: the farm's proof that its self-tests run rested on a gate whose own self-test could be skipped in silence, printing nothing and exiting 0. Everything else in Tier A is one card of mechanical work behind a decision that is now made; these two had to stop being the instrument and the defect at once.

Both took the #13487 shape verbatim — frozen roster of battery names, per-battery case floor, roster size pinned, every assertion attributed to the most recently opened battery, refusal when the opened set differs from the declared set, verdict sentinel checked at the dispatch. Neither needed a case rewritten.

One floor is deliberately not pinned at what it measures, and says so in place: check-self-test-wired's live ledger battery registers 15 cases today, one per row of a ⛔ shrink-only ledger whose whole purpose is to lose rows. A floor at 15 would redden every legitimate shrink and train the next author to edit the floor — the one habit these floors exist to prevent. What is pinned there is the part that does not move with the list: the structural case ran and at least one row was actually audited (1 + 3).

Reverse verification — 7 ablations, each naming what stopped

Every leg: mutate, prove the mutation is on disk anchored on the text meant to move, run, restore with git checkout HEAD -- ABSOLUTE_PATH, prove the restore by observed state (git diff HEAD empty and the path's blob hash equal to its HEAD blob, non-empty — an empty hash is read as failure, not as "nothing to compare"). Absolute paths are resolved before any cd, and an EXIT INT TERM trap restores both files. No build or dist/ is in the path: both gates run from source, so there is no rebuild leg to get wrong. The mutations ran against committed implementations, so the restore leg had a real reference to restore to.

ablation exit what it said
A1 delete the alias resolution battery's cases 1 battery "alias resolution" DID NOT RUN — 0 cases registered, 4 pinned
B1 delete that battery's registry entry 1 declares 5 batteries, below the pinned 6 and "alias resolution" registered 4 case(s) but is not declared
C1 early return at the top of selfTest() 1 selfTest() returned without reaching its verdict
D1 remove one assertion from a battery 1 battery "comment mask" registered 6 case(s), below its pinned floor of 7
A2 delete the innocent output battery's cases 1 battery "innocent output" DID NOT RUN — 0 cases registered, 4 pinned
B2 delete that battery's registry entry 1 declares 5 batteries, below the pinned 6 and the undeclared-battery set difference
C2 early return at the top of selfTest() 1 selfTest() returned without reaching its verdict

D1 is the leg that separates a floor from a zero check: one assertion removed, the battery still non-empty, and the gate still reds naming the battery and both numbers. A1/A2 are the set difference the ruling asked for — they say which battery stopped, not that something did. B1/B2 close the registry's own escape: deleting an entry silences a floor exactly as effectively as zeroing it.

Restores were proven per leg, e.g. blob 8eff18efe83c26bcc1958b4d2d997e872519863b == HEAD blob 8eff18efe83c26bcc1958b4d2d997e872519863b, git diff HEAD empty. Final git status --porcelain was empty.

Hit but deferred — the four hot files

⭐ This section is part of the deliverable. All four were surveyed; none appears in this diff.

file hole(s) measured why not fixed here occupying card
scripts/pm/dispatch-gates.mjs hole 1 yes (no floor); hole 2 NOT MEASURED — the injection anchor lands inside a template literal and produces a SyntaxError, so no run of it measures anything occupied; and it needs a hand-read entry before hole 2 can be measured at all #13448 → PR #13783
scripts/check-durability-degradation-log-level.mjs hole 1 yes; hole 2 yes — its dispatch is process.exit(logLevelStatus || readSeamStatus ? 1 : 0), so an early return in the first callee yields undefined, the || falls through to the second status, and the run exits 0 occupied; and it is Tier C (multi-entry — one handshake per callee, or a wrapper: a decision, not a transplant) #13474, in flight
scripts/check-platform-checklist.mjs hole 1 yes; hole 2 NOT MEASURED — four self-test callees combined; a one-function probe measures a sub-battery and reads the downstream spread-of-undefined crash as a handshake occupied; Tier C for the same reason #13482, in flight
scripts/check-dispatcher-error-vocabulary.mjs hole 1 yes; hole 2 yes, defeated, printing nothing occupied. Note it is the sibling #13489 names as having landed the named-set form — it did, for its SHAPES/samples pair, which is a different property from its self-test's own floor. Being right about one is not being floored on the other #13478, in flight

Occupancy re-checked at 2026-08-31T12:45:05Z, by reading each card's live state — ⛔ not from the dispatch list's wording. At that moment: PR #13783 had merged, at 2026-08-31T12:37:30Z; #13474, #13482 and #13478 were all still open. scripts/pm/dispatch-gates.mjs is nevertheless still excluded from this diff, for a reason that outlived its occupancy: it is the one file the probe cannot mutate, so there is no measurement to repair against yet. It belongs to the Tier D follow-up, not to this PR.

The instrument, committed

scripts/measure-self-test-floor.mjs — the census, so the follow-up tiers can be measured rather than estimated.

node scripts/measure-self-test-floor.mjs           # static census (fast)
node scripts/measure-self-test-floor.mjs --probe   # + the dynamic probe (about 5 minutes)
node scripts/measure-self-test-floor.mjs --json    # machine-readable

It deliberately ships no --self-test flag: its controls run inline on every invocation, so they cannot become unrun. That is the inline route check-self-test-wired.mjs's own ledger already records, and it is the only placement consistent with what this card measured. It also excludes itself from its own census — the --self-test literals in it live in control-fixture strings, which are data, not a dispatch. The nine files whose entry cannot be resolved mechanically are resolved by reading, in a shrink-only ENTRY_BY_HAND table where a null is NOT MEASURED with a stated reason, never a guess and never a quiet pass.

Correction — this PR reported green on a red commit, and here is what that was

⚠️ An earlier revision of this body claimed a green union on 0d58ba20e. CI was red on that exact commit (Type Check · source gates, check-run 99495667232, failing step pnpm check:ratchet-remedy-authority; TypeScript Type Check red only as its aggregator, one root cause showing as two squares). The local union and the CI red were both true readings of the same tree, and the reason is recorded below and filed as #13813. The claim is withdrawn; the verification section is re-run on e7bf269bd.

What the gate said, verbatim

✗ check-ratchet-remedy-authority: 2 problem(s)

  • UNMARKED: scripts/check-self-test-wired.mjs hands the author a remedy that EXPANDS a shrink-only registry, and neither marks that path ⛔ MAINTAINER-ONLY nor turns it down outright (#8435).
    remedy: …add a SELF_TEST_RUN_OTHERWISE row in scripts/check-self-test-wired.mjs naming the evidence for that.…
    registry: SELF_TEST_RUN_OTHERWISE (testimony: shrink)
    Fix: say in the same message that this path belongs to a maintainer, or turn it down
    outright the way check-type-source-resolution.mjs does. Turning it down is the stronger
    shape and this gate treats it as fully compliant — it is not a lesser option.

  • UNCLASSIFIED: scripts/check-self-test-wired.mjs is reached by the sweep as "unmarked", and the control corpus in scripts/check-ratchet-remedy-authority.mjs does not cover it.
    Fix: record which shape it is. That is the correct fix and needs nobody's leave — the
    control is a declaration registry, not a ratchet.

Whose text it is — measured, because the obvious answer is wrong

The quoted remedy sentence is not new. It sits in origin/main verbatim, at the same line, and predates this card. What is new is proximity.

anchorFor decides whether an offer's target is a ratchet by reading the ±400 characters around every mention of the declared identifier. On main, SELF_TEST_RUN_OTHERWISE's own docblock opens with ⛔ SHRINK-ONLY — but that banner sits far enough above the const's name to fall outside the window, so stage 2 never reached the target and the offer was never examined. A comment this PR added elsewhere in the file names SELF_TEST_RUN_OTHERWISE and its shrink-only nature one line apart, which pulled the target inside the window for the first time.

Measured both ways rather than argued: a detached worktree at origin/main runs node scripts/check-ratchet-remedy-authority.mjs exit 0"159 hand out no ratchet-expanding remedy. Control corpus: 25 hand-classified scripts". This branch reddened. So the regression is genuinely this PR's, arriving through a route neither "my new text" nor "not my text" describes correctly.

⛔ Reverting that comment to buy green was rejected outright: the sentence is true and load-bearing for the floor it explains, and deleting it would restore a blind spot that exists only because a docblock is long. The offer had been unexamined for as long as it existed.

The repair, and why refusal rather than marking

The gate offers two compliant shapes. Refusal was chosen, and not on taste — on what the ledger's own docblock already says, unchanged since before this card:

Nothing joins this list to silence a finding: a gate that really does ship an unrun self-test gets wired instead.

The author-facing message had been contradicting that sentence for its whole life, inviting exactly the row the docblock refuses. It now agrees with it: wiring the self-test is the whole remedy and it is the landing author's. The legitimate row still exists and is still named — a drives wrapper or an inline run is real — but as a maintainer's judgement of evidence (⛔ MAINTAINER-ONLY) rather than as an escape from wiring.

The same token was added to this PR's own new expanding path, matching #13487: lowering a battery floor is now spelled ⛔ MAINTAINER-ONLY rather than merely discouraged.

The UNCLASSIFIED half is recorded in the control corpus in scripts/check-ratchet-remedy-authority.mjs, as the gate instructs — the sixth refused instance. ⚠️ The first attempt at that entry put the two refusal sentences inside the why string and reddened the gate's own self-test case (1): quotes shaped like offers are fine in a comment and are not fine in author-facing text, which is precisely the distinction that gate enforces. They live in the comment above the entry now.

Ablation E — proving the green is bought by the refusal

Restoring the pre-#13489 growth offer (mutation proven on disk: the refusal string 1 → 0, the old offer 0 → 1, −260 bytes) reddens check-ratchet-remedy-authority again, exit 1. ⚠️ Reported as observed, not as predicted: it reds as MISCLASSIFIED, not UNMARKED"the control says 'refused', the sweep says 'marked'" — because the file now also carries the authority token on its floor path, so removing the refusal lands on a shape that is itself compliant. The control-corpus entry is what catches it. That is a stronger result than the predicted one and it is the entry earning its keep. Restore proven by observed state; tree clean.

The finding this produced, filed as #13813

⭐ On the record rather than as a footnote, because it is this PR's own defect class one level up: a derivation that passes while not covering what its reader takes it to cover.

node scripts/pm/dispatch-gates.mjs --residue over the original three paths places pnpm check:ratchet-remedy-authority in Undetermined (source names no path at all — NOT known irrelevant), 1 of 39 — not matched, not silent. The gate sweeps scripts/*.{mjs,mts}, 176 files, but builds that population at runtime from readdirSync(SCRIPTS_DIR); the glob exists in the file only inside output messages and one comment, never as a declared literal. The eight families that did match the same three paths all matched at gate source 'scripts/**', because their populations are spelled where the derivation can read them.

So editing any scripts/** file makes this gate affected and no path derivation says so. The derivation's residue text is honest about this — it even says the bucket is largely unexamined — but a dispatch brief carries the matched list, and the caveat is in a section nobody pastes. Remedy shape and the 37 unexamined siblings are in #13813; ⛔ deliberately not fixed here, since it changes a third gate's declared population.

Verification

Re-run on e7bf269bd after the correction above. Repo-wide, not narrowed. Every exit code captured before any pipe, from a variable assigned directly after the command.

  • pnpm lint (eslint . --no-inline-config, whole repo) — GREEN, LINT_EXIT=0, zero findings. ⚠️ Run on its own: in an earlier batched attempt this command was killed by the container's 10-minute foreground cap and printed ELIFECYCLE Command failed with no findings at all — the signature of a kill, not of a lint failure. Re-run alone, it is green. A batched run that hits the cap is NOT MEASURED, not red.
  • Gate family re-derived after the final commit: node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack — 4 paths vs merge base eb717a12a, DG_EXIT=0. Now 15 families, and check:ratchet-remedy-authority is in them. ⚠️ Read why: matched via scripts/check-ratchet-remedy-authority.mjs ⇢ gate script. It is in the union only because this PR now edits that gate's own file — not because the gate sweeps the other three. dispatch-gates places check:ratchet-remedy-authority in "undetermined" — a scripts/** sweep that no derived family names, so a full local family run misses it #13813 is exactly that gap.
family result
check:agent-test-spelling, check:bash32-floor, check:cli-command-ids, check:cross-package-test-inputs, check:entry-guard, check:parse-guard, check:pnpm-filter-targets, check:watch-hint-literal GREEN, exit 0 each
check:ratchet-remedy-authority GREEN, exit 0 — both halves: OK self-test: the lexer holds, messages are bounded, both offer word orders and path-named registries are reached, declaration registries are not, the authority token cannot anchor itself, refusal is told apart from discouragement, and the sweep still reaches every known instance. then OK check-ratchet-remedy-authority: 176 scripts swept (scripts/*.{mjs,mts}); 11 mark the expanding remedy ⛔ MAINTAINER-ONLY, 6 turn it down outright, 159 hand out no ratchet-expanding remedy. Control corpus: 26 hand-classified scripts, set-equality audited both ways. (main: 5 refusals, 25 corpus entries)
check-ci-filter-parity, check-cross-package-test-inputs, check-self-test-wired, check-self-test-workflow-commands, check-shard-attestation GREEN, exit 0 each
check-test-completeness NOT MEASURED — exit 3, PREREQUISITE NOT MET: it grades a saved turbo run test log and no log exists locally. Its own text names this branch unreachable in CI. Not a red
  • Convention-triggered, both run because this diff edits gate scripts: node scripts/pm/bare-root-worklist.mjs --self-testGREEN, exit 0, 51 live row(s), 43 unreachable as spelled, 43 recorded verdict(s) — none stale, none missing, none contradicted. pnpm check:pm-dispatch-gatesGREEN, exit 0, ✓ dispatch-gates self-test: 1048 cases pass.
  • The two repaired gates in both modes: --self-test exit 0 (6 declared batteries, 43 cases registered / 6 declared batteries, 23 cases registered) and production exit 0.
  • node scripts/check-nul-bytes.mjsGREEN, exit 0, 7587 files. The touched files were additionally swept for raw control bytes directly: no match.
  • All seven battery ablations re-run on e7bf269bd: all red, each naming what stopped, every restore proven by observed state (git diff HEAD empty and blob hash equal to the HEAD blob), tree clean. Plus ablation E above.
  • No test suite in the tree names any of the four touched scripts (git grep over *.test.* / *.spec.ts).

No changeset: scripts/** publishes nothing from any package, and Check Changeset has no path exemption — its only exemption is the skip-changeset label, which is applied here.

Session for this work: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC


Generated by Claude Code

claude added 4 commits August 31, 2026 12:32
…est-wired

Its --self-test decided success by `failures.length === 0` alone and its
dispatch discarded the call's completion, so an early return printed nothing
and exited 0 -- in the gate whose subject is that a shipped self-test runs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC
…est-workflow-commands

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC
…nnot prove they ran

Two orthogonal holes, counted separately and never summed: no assertion floor
(static, published criterion) and no verdict handshake (measured by injecting
an early return and reading the exit code). Both instruments are driven against
known-holed and known-sound fixtures on every invocation, and a control failure
refuses instead of printing a smaller number.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC
@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

Follow-ups filed, by tier

The tiering criterion is in the PR body; these are the cards it produces. All three are unassigned and carry their measured numbers, so the next seat re-measures rather than re-derives.

#13798 and #13799 are deliberately two cards, not one: the holes are orthogonal, and a gate with a perfect floor is still defeated by the other.

Session for this work: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC


Generated by Claude Code

… and record the shape

check-ratchet-remedy-authority went red on this branch and is green on main:
the remedy sentence is older than this PR, but a comment added here named
SELF_TEST_RUN_OTHERWISE and its shrink-only nature one line apart, which pulled
the target inside anchorFor's 400-character window for the first time.

The message now says what the ledger's own docblock already said -- wiring the
self-test is the whole remedy -- and the legitimate row is named as a
maintainer's judgement of evidence. Shape recorded in the control corpus.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Pk26oZ12t5N1hwGW1m1MgC
@claude

claude Bot commented Aug 31, 2026

Copy link
Copy Markdown
Contributor Author

⚠️ CORRECTION (13:4xZ) — one claim below is WRONG, and it is the diagnosis, not the verdict

The comment below says: "Both problems are the PR's own new text — the remedy message it adds to check-self-test-wired.mjs", and "⛔ Not ported, not pre-existing, not a flake". The remedy message is pre-existing.

The dev found this by measurement and this seat then verified it independently on origin/main:

$ git show origin/main:scripts/check-self-test-wired.mjs | grep -n 'SELF_TEST_RUN_OTHERWISE'
268:        '    or its ordinary run executes its own cases — add a SELF_TEST_RUN_OTHERWISE\n' +

⇒ the author-facing offer CI flagged as UNMARKED has been on main all along, at that line, verbatim. This PR did not write it.

What the PR actually changed is PROXIMITY. anchorFor reads ±400 characters around each mention of a declared target. On main the ledger's SHRINK-ONLY banner sits far enough from the const's name that stage 2 never reached the target, so the offer went unexamined for as long as it existed. A comment this PR added names SELF_TEST_RUN_OTHERWISE and its shrink-only nature one line apart, pulling the target inside the window for the first time.

The verdict stands, the reasoning does not. The failure IS this PR's — a detached worktree at origin/main runs the gate at exit 0 and this branch reddens — so no re-run was warranted and none was spent. But it arrived by a route that neither "the PR's own new text" nor "not this PR's" describes, and the bolded claim below would send a reader looking for a defect in a line the PR never touched.

Original comment preserved unedited below.


CI status on 0d58ba20e — red, root-caused, dev sent back

Recording this on the PR so the state is legible without the dispatch channel.

Two red squares, one root cause.

check run reading
Type Check · source gates 99495667232 ⛔ the real failure — step pnpm check:ratchet-remedy-authority, exit 1
TypeScript Type Check 99496963301 not a second problem — the aggregator. Its log shows OS_NEEDS = typecheck-source-gates: failure, other three lanes success, and it exits on that one row

The failure, verbatim from the job log:

✗ check-ratchet-remedy-authority: 2 problem(s)

  • UNMARKED: scripts/check-self-test-wired.mjs hands the author a remedy that EXPANDS a
    shrink-only registry, and neither marks that path ⛔ MAINTAINER-ONLY nor turns it down
    outright (#8435).
    registry: SELF_TEST_RUN_OTHERWISE (testimony: shrink)

  • UNCLASSIFIED: scripts/check-self-test-wired.mjs is reached by the sweep as "unmarked",
    and the control corpus in scripts/check-ratchet-remedy-authority.mjs does not cover it.

This failure is this PR's. Both problems are the PR's own new text — the remedy message it adds to check-self-test-wired.mjs, and the control-corpus row that text now requires. ⛔ Not ported, not pre-existing, not a flake: no re-run is warranted and none has been spent.

⚠️ Why the dev's run and CI disagreed, and why it is worth recording

The completion report states the derived family union was re-run on 0d58ba20e with 14 families green. The last push to this branch is 0d58ba20e at 12:45:26Z; the failing run started at 12:59:01Z. Both readings are of the same tree — so this is not a stale-checkout disagreement.

⭐ The likely cause is that the union never contained this gate. check-ratchet-remedy-authority is a sweep gate: it reads the remedy prose of every scripts/** gate, so editing an author-facing message in any one file makes it affected — and no family mapping keyed on the edited file's own gate or test predicts that. If so, "14 families green" and "CI red" are both true statements about disjoint sets.

That hole is the same shape as this PR's own subject: a check that passes while not covering what it is taken to cover. It is being measured rather than assumed, and the result will be reported either way.

Boundaries given for the fix

  • ⛔ Do not widen the PR beyond these two files' own new text.
  • ⛔ Do not buy green with a suppression, an exclusion-list entry, or an exemption in check-ratchet-remedy-authority — this seat will not accept a green bought that way on a card whose entire subject is gates that pass without asserting anything.
  • The gate names two acceptable shapes for the UNMARKED problem (mark the path ⛔ MAINTAINER-ONLY, or turn it down outright); the dev picks one deliberately and states which and why.

Next event on this PR should be a push, not a comment.


Generated by Claude Code

@github-actions

Copy link
Copy Markdown
Contributor

⛔ merge queue 构建失败 — 先分诊,再决定要不要重排

队列构建 33403512271 红了。队列跑的是全量套件(PR 侧 CI 只跑 affected 子集),
所以失败的测试可能在本 PR 没碰过的包里 —— 那不是重排能修的。每次盲目重排都会让排在后面的所有 PR 重建一轮。

失败的 job(日志抽取,best effort):

  • Test Core (1/6) — 失败步骤: Run this shard's tests

    @objectstack/client:test:  FAIL  src/envelope-caller-census.test.ts > #13079 §3 — every call site is classified > the mechanical enumeration and the hand ledger agree, site for site
      ↳ 失败原因: @objectstack/client:test: AssertionError: expected { …(13) } to deeply equal { …(12) }
    @objectstack/client:test:  FAIL  src/envelope-caller-census.test.ts > #13079 §3 — every call site is classified > ⭐ THE NUMBER: zero call sites in this repo would break SILENTLY
      ↳ 失败原因: @objectstack/client:test: AssertionError: expected [ { …(5) }, { …(5) } ] to deeply equal []
    @objectstack/client:test:  FAIL  src/envelope-caller-census.test.ts > #13079 §3 — every call site is classified > records the split: 13 loud pin sites, 6 result-insensitive, 1 not-SDK
      ↳ 失败原因: @objectstack/client:test: AssertionError: expected 21 to be 19 // Object.is equality
    

↳ 失败原因 是判读的关键:超时Test timed out in … / Hook timed out in …)多半是负载/时序,不是本 PR 的回归;
断言AssertionError: …)才指向真实的行为改变。两者的 FAIL 行长得一模一样,只有这一行能区分。

跨 PR 相同签名(24h,按失败测试文件聚合):

历史信号:

  • 本 PR 过去 24h 无队列失败记录(首次)。
  • 过去 24h 队列共有 7 个失败构建(不含本次)。

分诊清单:

  1. 失败测试在本 PR 改动的包里 → 真回归,修 PR。
  2. 失败测试与本 PR 无关 → 看上面的「跨 PR 相同签名」;已有汇总 issue ⇒ flaky/环境问题实锤,去那张 issue 上谈,修好前重排只会再烧一轮全队列。
  3. 两者都不是 → 可能与同组 PR 语义冲突;等前面的 PR 落地或失败出队后再重排一次即可,不要连续重排。

Generated by Claude Code · merge-queue-triage workflow (#4859)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/l skip-changeset PR has no user-facing published change; bypasses the changeset gate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[finding] Survey: which other scripts/* --self-test modes decide success with no assertion floor, or exit 0 via return selfTest() before their verdict

2 participants