What is broken
scripts/measure-self-test-floor.mjs's probeEarlyReturn awards a HELD / ACCIDENT verdict even when the baseline run of the probed file already exits non-zero — i.e. when the mutation had nothing to defeat because the file cannot run in this tree at all.
Measured
On main 5b2ad1b41 (after #15372 landed the mutatedSpoke discriminator), in a checkout whose node_modules lacks a dependency of the audits census script:
scripts/audits/14744-before-update-per-row-value-census.mjs
verdict HELD entry runSelfTest baselineExit 1 mutatedExit 1 mutatedBytes 937
mutatedHead 'node:internal/modules/package_json_reader:314' mutatedSpoke true
The mutated run "spoke" — with a module-resolution stack trace, not a handshake — and the baseline was already red (baselineExit 1). The same file reads ACCIDENT (baselineExit 0, mutatedBytes 0) in an installed worktree (#15372's report, comment 5541003547). So one probe row has two verdicts depending on the environment, and the wrong one is the flattering one.
Why it matters
NOT MEASURED exists for exactly this: 34 rows read it in the same run. A row whose baseline cannot run belongs there, not under HELD. The #13799 census reads HELD counts as evidence; a fresh worktree without pnpm install inflates them.
Remedy (recommendation, not a ruling)
baselineExit !== 0 ⇒ verdict NOT MEASURED with the reason baseline run failed (exit N), before any mutated-run reading; a positive control fixture whose baseline fails. One function (probeEarlyReturn), same file as #15339 (PRODUCES_FAILURE ternary, classifyFloor) and #15371 (helper-spelling coverage) — serial on that file.
Found by the PM seat domain:devx @ objectstack (#6023) while probing #15372 on main, 2026-09-04.
Generated by Claude Code
What is broken
scripts/measure-self-test-floor.mjs'sprobeEarlyReturnawards aHELD/ACCIDENTverdict even when the baseline run of the probed file already exits non-zero — i.e. when the mutation had nothing to defeat because the file cannot run in this tree at all.Measured
On
main5b2ad1b41(after #15372 landed themutatedSpokediscriminator), in a checkout whosenode_moduleslacks a dependency of the audits census script:The mutated run "spoke" — with a module-resolution stack trace, not a handshake — and the baseline was already red (
baselineExit 1). The same file readsACCIDENT(baselineExit 0,mutatedBytes 0) in an installed worktree (#15372's report, comment 5541003547). So one probe row has two verdicts depending on the environment, and the wrong one is the flattering one.Why it matters
NOT MEASUREDexists for exactly this: 34 rows read it in the same run. A row whose baseline cannot run belongs there, not underHELD. The #13799 census readsHELDcounts as evidence; a fresh worktree withoutpnpm installinflates them.Remedy (recommendation, not a ruling)
baselineExit !== 0 ⇒ verdict NOT MEASUREDwith the reasonbaseline run failed (exit N), before any mutated-run reading; a positive control fixture whose baseline fails. One function (probeEarlyReturn), same file as #15339 (PRODUCES_FAILUREternary,classifyFloor) and #15371 (helper-spelling coverage) — serial on that file.Found by the PM seat
domain:devx @ objectstack(#6023) while probing #15372 onmain, 2026-09-04.Generated by Claude Code