Skip to content

Commit 6d98342

Browse files
os-elon-muskclaude
andauthored
fix(scripts): evaluate the census self-test's battery floor before its verdict, and set the handshake last (#19053)
Fixes #19029 Clause-②: no One file: `scripts/check-system-context-census.mjs`. A root `scripts/` gate, nothing published. ## Premise, re-read on the source first The card was filed bare and triage (comment 5730522407) asked the taker to read the order itself and refute the card if it was in fact compliant. Read on `origin/main` `b22db516d`: `selfTest()` printed its verdict line, set `selfTestReachedVerdict = true`, and only THEN evaluated its 46-line battery floor, whose own comment claimed it was "Evaluated ... BEFORE the verdict". Premise **confirmed**, and `AGENTS.md`'s `Writing a --self-test` band is explicit: "Set the flag as the self-test's last statement, after its success line prints". ## What moved - The floor block moves **above** the verdict write. Its 40 lines of code are **byte-identical** after the move (sha256 of the block `f6312a3aa320cbf0` before and after); the roster, the floor's rules and `SELF_TEST_BATTERY_FLOOR` are untouched, and so is `run()` and the entry block. - `selfTestReachedVerdict = true` becomes the **last statement** before the return, after the verdict line prints — the ordering `scripts/check-agent-model-declared.mjs` already carries (copied, ⛔ not imported). - The floor's comment now describes the order it relies on, and names why the order is load-bearing. - One new self-test case, `⭐ VERDICT ORDER`, filed under the battery already open — the roster is **not** widened. ## Why the new case is a STATIC read, and what it pins The floor prints only when it FAILS, so a passing run's bytes are identical whichever side of the verdict it sits on: no mutation of a green tree exposes the order, which is why the fleet probe on PR #19026's branch correctly reports that it observed nothing. So the case reads the order off this file's own text (the idiom the neighbouring `POPULATION DECLARATION` pins already use) and holds five facts, each anchor required to resolve **exactly once**: ``` {"floorBeforeVerdict":true,"floorAddsToTheCounterTheVerdictReads":true,"flagAfterVerdictLine":true, "nothingExitsBetweenVerdictAndFlag":true,"flagIsTheLastStatement":true, "anchorsResolvedExactlyOnce":{"floor":true,"verdict":true,"flag":true,"returnStatement":true}} ``` Chained, those are the behavioural claim: the floor adds to the same `failures` the verdict reads, the verdict prints after it, and the flag is the last statement with no `return` or `process.exit(` between — so a breached floor can only print the FAILED line and can only return non-zero. Comments are stripped before the two adjacency reads, in both directions: prose between the two statements must not red the case, and prose must not be able to hide a statement from it either (a line-anchored `^return` misses `if (x) return 0;` — measured, see leg 2). ## Measurements All four ablation legs ran through `scripts/ablation-replace.mjs` on the **committed** file: anchor hit count declared, the write verified against the disk and the blob hash, and the restore proven by `blob == HEAD` plus an empty `git diff HEAD`. Every leg printed `ok restored: blob == HEAD (dcaf90e) and git diff HEAD is empty`. **1. A passing run is unchanged except for the new case** (`--self-test` before vs after, full transcripts diffed): ``` 93a94 > ok ⭐ VERDICT ORDER: the battery floor is evaluated ABOVE the verdict line and ... ``` That is the whole diff: 95 lines / 18,636 bytes before, 96 lines / 19,201 bytes after, 80 cases to 81, success line still last, exit 0 both. Nothing else moved, because the floor prints nothing on a passing run. **2. A breached floor prints the FAILED line and no success line.** Mutation: one pin raised above its count (`'absence is loud': 1,` to `: 99,`). ``` ablation-replace: ok mutation landed: anchor 1 -> 0, blob dcaf90e -> 85fc130dabf9 FAIL self-test battery "absence is loud" registered 1 case(s), below its pinned floor of 99 — cases that used to run no longer do. FAIL A battery at or below its floor means cases STOPPED RUNNING — the battery is the bug, not the number. ... check-system-context-census --self-test: 2 case(s) FAILED ablation-replace: command exited 1 ``` No `all cases passed` anywhere in that run (grep count 0). Before this change the same breach printed `all cases passed` and then those FAIL lines. **3. An early exit between the verdict and the flag is caught twice.** Mutation: `return 0;` injected after the verdict write, before the flag. ``` FAIL ⭐ VERDICT ORDER: ... -- {"floorBeforeVerdict":true,...,"nothingExitsBetweenVerdictAndFlag":false,...} check-system-context-census --self-test: 1 case(s) FAILED ✗ check-system-context-census self-test: selfTest() returned without reaching its verdict, so no success line was printed. Exiting 0 here would report a self-test that never finished as a self-test that passed. ablation-replace: command exited 1 ``` The static case names the ordering; the handshake refuses the 0. Exit 1. **4. An early exit inside the floor region, new order.** Mutation: `return 0;` injected at the top of the floor region (before `const declaredBatteries`). ``` ok ⭐ VERDICT ORDER: ... (no verdict line at all: grep "all cases passed" = 0, grep "case(s) FAILED" = 0) ✗ check-system-context-census self-test: selfTest() returned without reaching its verdict, so no success line was printed. ... ablation-replace: command exited 1 ``` **5. ⭐ The identical injection on `origin/main`'s shape — the hole, empirically.** Same anchor, same replacement, run against `HEAD~1`'s bytes checked out on disk (blob `2b21f55e7045`, verified before mutating; restored to HEAD's `dcaf90ed4d1a` after, `git diff HEAD` empty): ``` ok: the origin/main shape is on disk (floor BELOW the verdict and BELOW the handshake) ablation-replace: ok mutation landed: anchor 1 -> 0, blob 2b21f55 -> ac63c9bade5a check-system-context-census --self-test: all cases passed ablation-replace: command exited 0 ``` A run that printed the success line, exited 0, and never evaluated its floor. ⚠️ By the escalation rule the triage seat wrote into comment 5730522407 ("注入一次真实早退后该运行仍绿 ⇒ 升 p1"), that reading is the named trigger — it is a measurement, not a re-grade: the priority label stays triage's to move, and this PR touches no label but `skip-changeset`. ### Gates `node scripts/pm/dispatch-gates.mjs --commands --repo objectstack-ai/objectstack` derived **30** commands from this tree (change set: 1 path, merge base `b22db516d`). All 30 ran, exit codes captured redirect-then-`$?`, and the record reconciles with `--ran`. Both halves of the subject gate are in that list and green: `check-system-context-census: OK — 110 elevation read sites in 20 packages across 45 files ...`. Named NOT MEASURED by the derivation itself, ⛔ not by me: 2 families taking a value from the workflow, 1 path-scheduled CI job's 5 steps, 11 wide-population families, 14 pending-changeset families and the 53 artifact-roster families — CI's. ### Lint, narrowed with its three readings `pnpm lint` is `eslint . --no-inline-config`, a whole-repo scan CI owns. The narrowing: (1) the population is eslint's own — `eslint.config.mjs` lints `**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}` and this file is in it (it produced a result object, not an ignore skip); (2) `--format json` reports **1** file linted, 0 errors, 0 warnings, 0 suppressed, exit 0; (3) the config **never enables type-aware linting for ANY file** (`eslint.config.mjs`: "no `parserOptions.project`, no typed `@typescript-eslint` rules", measured there with a positive control), so a one-file diff cannot move an untouched file's verdict. All readings taken at `1f684af84`, the final commit. ## Reader test Run `node scripts/ablation-replace.mjs --file scripts/check-system-context-census.mjs --anchor " const declaredBatteries = Object.keys(SELF_TEST_BATTERIES);" --replacement " return 0; const declaredBatteries = Object.keys(SELF_TEST_BATTERIES);" -- node scripts/check-system-context-census.mjs --self-test`: it must print **no** verdict line and exit 1 (the dispatch says the self-test never reached its verdict). On `origin/main` that same command prints `all cases passed` and exits 0. ## Acceptance notes - Out of scope, noted and ⛔ not filed: `floorFailure`'s body carries a 6-space indent on its `process.stdout.write` where 4 is the file's shape. It moved byte-identically with the block rather than being tidied, so the diff stays reviewable as a move; a style nit, and the successor is whoever next edits this floor. - The self-test case is filed under the battery already open rather than a new roster entry, because the dispatch froze the roster. The cost is one case of slack in that battery's floor; the case sits after every one of that battery's own cases, so an early exit inside that block stops the new case too. - `docs/audits/2026-09-self-test-shape-census.md`'s row for this file (ROSTER / HELD) stays true: the handshake still holds, and legs 3 and 4 above are its live readings. --- _Generated by [Claude Code](https://claude.ai/code/session_01BTeBejoPUvRHN8WdAJC6oF)_ Co-authored-by: Claude <noreply@anthropic.com>
1 parent 5380daa commit 6d98342

1 file changed

Lines changed: 75 additions & 10 deletions

File tree

‎scripts/check-system-context-census.mjs‎

Lines changed: 75 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2838,21 +2838,73 @@ function selfTest() {
28382838
!/[A-Za-z_$][\w$]*\s*\./.test(declSites[0][1]),
28392839
JSON.stringify(declSites.map((d) => d[1].replace(/\s+/g, ' ')))
28402840
);
2841+
2842+
// ⭐ The VERDICT ORDER is a STATIC property, so it is read here off this
2843+
// file's own text. The floor prints only when it FAILS, so a passing run's
2844+
// bytes are identical whichever side of the verdict the floor sits on: no
2845+
// mutation of a green tree can expose the order and the fleet's injection
2846+
// probe correctly reports that it observed nothing (#19029). What this pins
2847+
// is the chain that makes a breach loud -- the floor adds to the same
2848+
// `failures` the verdict reads, the verdict prints after it, and the
2849+
// handshake flag is the statement immediately below the verdict -- so the
2850+
// ordering cannot regress silently back to a success line printed over an
2851+
// unevaluated floor. ⛔ Filed under the battery opened above: the roster is
2852+
// frozen, and adding a name to it is not this case's to do.
2853+
const orderIndex = (re) => {
2854+
const hits = [...ownSource.matchAll(re)];
2855+
return hits.length === 1 ? hits[0].index : -1;
2856+
};
2857+
const floorAt = orderIndex(/^ {2}const floorFailure = \(message\) => \{$/gm);
2858+
const verdictAt = orderIndex(/^ {2}process\.stdout\.write\(\n {4}failures === 0$/gm);
2859+
const flagAt = orderIndex(/^ {2}selfTestReachedVerdict = true;$/gm);
2860+
const returnAt = orderIndex(/^ {2}return failures === 0 \? 0 : 1;$/gm);
2861+
const between = (from, to) => (from >= 0 && to > from ? ownSource.slice(from, to) : '');
2862+
// Comments are stripped before the two ADJACENCY reads below, in both
2863+
// directions: prose between the verdict and the flag must not red them, and
2864+
// prose must not be able to hide a statement from them either. ⛔ An
2865+
// exit spelled `if (x) return 0;` is not caught by a line-anchored
2866+
// `^return`, so the test is for the TOKEN in the surviving code.
2867+
const codeOnly = (text) => text.replace(/\/\*[\s\S]*?\*\//g, '').replace(/^\s*\/\/.*$/gm, '');
2868+
const order = {
2869+
floorBeforeVerdict: floorAt >= 0 && verdictAt > floorAt,
2870+
floorAddsToTheCounterTheVerdictReads: /failures \+= 1;/.test(between(floorAt, verdictAt)),
2871+
flagAfterVerdictLine: verdictAt >= 0 && flagAt > verdictAt,
2872+
nothingExitsBetweenVerdictAndFlag:
2873+
flagAt > verdictAt && !/\breturn\b|process\.exit\(/.test(codeOnly(between(verdictAt, flagAt))),
2874+
flagIsTheLastStatement:
2875+
flagAt >= 0 && returnAt > flagAt
2876+
&& codeOnly(between(flagAt, returnAt)).trim() === 'selfTestReachedVerdict = true;',
2877+
};
2878+
t(
2879+
'⭐ VERDICT ORDER: the battery floor is evaluated ABOVE the verdict line and the handshake flag is '
2880+
+ 'the last statement after it -- so a breached floor prints the FAILED line and returns non-zero, '
2881+
+ 'and no exit between the two can report a run that never evaluated its floor as one that passed',
2882+
Object.values(order).every(Boolean),
2883+
JSON.stringify({
2884+
...order,
2885+
anchorsResolvedExactlyOnce: {
2886+
floor: floorAt >= 0, verdict: verdictAt >= 0, flag: flagAt >= 0, returnStatement: returnAt >= 0,
2887+
},
2888+
})
2889+
);
28412890
}
28422891

28432892
Object.assign(process.env, savedGitEnv);
2844-
process.stdout.write(
2845-
failures === 0
2846-
? '\ncheck-system-context-census --self-test: all cases passed\n'
2847-
: `\ncheck-system-context-census --self-test: ${failures} case(s) FAILED\n`
2848-
);
2849-
selfTestReachedVerdict = true;
2893+
28502894
// ── The floor: every declared battery RAN, and ran its cases (#13489) ───
28512895
//
2852-
// Evaluated after every battery has had its chance and BEFORE the verdict, so
2853-
// the success line below can only be printed by a run in which the set of
2854-
// batteries that registered assertions EQUALS the set declared. A set
2855-
// difference names WHICH battery stopped; a count says only that something did.
2896+
// Evaluated after every battery has had its chance and BEFORE the verdict, and
2897+
// the STATEMENT ORDER here is the whole of what makes that sentence true
2898+
// (#19029: this block used to sit BELOW a verdict that had already printed the
2899+
// success line and set the handshake, so a breach printed "all cases passed"
2900+
// and then its own FAIL lines -- a transcript that contradicted itself, and an
2901+
// early exit anywhere in here left a run that printed success, exited 0 and
2902+
// never evaluated its floor). `floorFailure` adds to the same `failures` the
2903+
// verdict below reads, so the success line can only be printed by a run in
2904+
// which the set of batteries that registered assertions EQUALS the set
2905+
// declared, and a breached floor can only print the FAILED line and return
2906+
// non-zero. A set difference names WHICH battery stopped; a count says only
2907+
// that something did.
28562908
const floorFailure = (message) => {
28572909
failures += 1;
28582910
process.stdout.write(` FAIL ${message}\n`);
@@ -2894,6 +2946,19 @@ function selfTest() {
28942946
);
28952947
}
28962948

2949+
process.stdout.write(
2950+
failures === 0
2951+
? '\ncheck-system-context-census --self-test: all cases passed\n'
2952+
: `\ncheck-system-context-census --self-test: ${failures} case(s) FAILED\n`
2953+
);
2954+
// The handshake (`AGENTS.md`, the `Writing a --self-test` band): the flag is
2955+
// set as the LAST statement of this function, after the verdict line above has
2956+
// printed. Every exit ABOVE this line -- including one inside the floor --
2957+
// therefore leaves it false, and the dispatch SAYS the self-test never reached
2958+
// its verdict rather than reading a 0 as a pass. ⛔ Nothing may be inserted
2959+
// between the write above and this line; the ⭐ VERDICT ORDER case pins both
2960+
// halves of the ordering off this file's own text.
2961+
selfTestReachedVerdict = true;
28972962
return failures === 0 ? 0 : 1;
28982963
}
28992964

0 commit comments

Comments
 (0)