Commit c70581b
fix(spec): give four packages/spec self-tests a roster, a floor and a verdict handshake (#19076)
Refs #18919
Clause-②: no
Tranche 1 of #18919 — **4 of the 12** gates under
`packages/spec/scripts/` that dispatch on `--self-test` while carrying
no battery roster, no floor and no verdict handshake. ⛔ This PR
deliberately does **not** close the card; the card's own fence is 「⛔
不主张一次全做完」 and the eight-file residue is named below so the next tranche
needs no re-derivation.
## The hole reproduces inside this package — measured before any
retrofit
The card claims **absence of protection, ⛔ not existence of a defect**,
and says #18512's readings are not these files' readings. So the first
act was an ablation on one of the four, at the branch point `d4cb05cbf`,
through `scripts/ablation-replace.mjs` (anchor must hit, write verified
against the disk, restore proven):
| | reading |
|:--|:--|
| file | `packages/spec/scripts/check-exported-any.ts` |
| mutation | `['BareAny', 'InferredFromAnySchema']` → `['BareAny']` —
one of the two RED-leg detection pins stops being reached |
| anchor on disk | hits `1` → `0`; replacement `0` → `1` |
| blob before | `7150b80a00ead8fecb8b13f08f27d2627332fb28` |
| blob after | `3210b6b88928bf1baebc50cf520133a65133b2fe` |
| what the gate then said | `✅ self-test: detects \`any\` types and
\`any\`-output schemas, and nothing else.` — **byte-identical to the
unmutated run** |
| exit code | **0** |
| restore | `git checkout HEAD -- PATH`: blob back to `7150b80a00ea…`,
`git diff HEAD` empty |
A second reading, on the file whose dispatch hands the self-test's
return value straight to `process.exit()` —
`check-error-code-provenance.ts`, same tool, same tree, in a throwaway
worktree at `d4cb05cbf`:
| | reading |
|:--|:--|
| mutation | `return 0;` as the first statement of `selfTest()` |
| blob | `9e74797b6d2db68bb59bd9c9fd93ff81e9ef32e7` →
`2245217ca8d41e3b46fb1634c7b99637dd9b351d` |
| what the gate then said | **nothing — zero bytes** |
| exit code | **0** |
| restore | blob back to `9e74797b6d2d…`, `git diff HEAD` empty |
So both holes the card names are real here, not inherited: a battery can
shrink silently, and a `return` above the verdict prints nothing and
passes.
## What the four carry now, with the lit control
Predicate, re-established rather than quoted from the dispatch — `grep
-c -E 'SELF_TEST_BATTERIES|SELF_TEST_BATTERY_FLOOR'` (roster) and `grep
-c 'returned without reaching its verdict'` (handshake), matching lines:
| file | roster before → after | handshake before → after | self-test
verdict now |
|:--|:--|:--|:--|
| `check-exported-any.ts` | 0 → 10 | 0 → 1 | 14 case(s) across 5
batteries |
| `check-dual-source-exports.ts` | 0 → 10 | 0 → 1 | 9 case(s) across 4
batteries |
| `check-error-code-provenance.ts` | 0 → 10 | 0 → 1 | 13 case(s) across
5 batteries |
| `check-browser-reachable-entries.ts` | 0 → 10 | 0 → 1 | 29 case(s)
across 9 batteries |
| **lit control** `scripts/check-test-typecheck.mts` (unchanged) |
**10** | **1** | its own `--self-test` prints 11 semantic cases |
The same instrument answers 10 / 1 on the TS precedent and answered 0 /
0 on all four targets, so the four zeros were readings and not a dead
grep. The registered case counts are not derived from the roster — each
is the number the run itself registered, and the roster is a literal the
run is compared against.
Every floor was **measured, not guessed**: the roster was first declared
with a sentinel floor of 999 per battery and the run asked which
batteries registered how many cases; the floors are those answers. That
sentinel run is also a red-leg demonstration — the floor named all nine
`check-browser-reachable-entries.ts` batteries and exited 1.
## Reverse verification: the same mutation now reds, and names the
battery
From the committed retrofit, the ablation that was silent-green above,
re-run through the same tool:
```
✗ self-test floor: self-test battery "the RED leg: an exported TYPE that IS `any` is flagged" registered 1 case(s), below its pinned floor of 2 — 1 case(s) that used to run no longer do.
✗ self-test floor: A battery below its floor means cases STOPPED RUNNING — the battery is the bug, not the number. …
```
blob `99c2fbe72538…` → `f92324cceba0…`, command exit **1**, restore
proven (`blob == HEAD`, `git diff HEAD` empty).
And the handshake, one leg per file — `return;` (`return 0;` for the one
returning a number) injected as the first statement of `selfTest()`,
each from the committed state, each restored with the blob equal to
HEAD's and an empty `git diff HEAD`:
| file | mutated blob | exit | first line printed |
|:--|:--|:--|:--|
| `check-exported-any.ts` | `d51735a53c12…` | 1 | `✗ check-exported-any
self-test: selfTest() returned without reaching its verdict,` |
| `check-dual-source-exports.ts` | `21bf4cbbaedf…` | 1 | `✗
check-dual-source-exports self-test: …without reaching its verdict,` |
| `check-error-code-provenance.ts` | `d5bd8fe9c0e6…` | 1 | `✗
check-error-code-provenance self-test: …without reaching its verdict,` |
| `check-browser-reachable-entries.ts` | `33dc39d561dc…` | 1 | `✗
check-browser-reachable-entries self-test: …without reaching its
verdict,` |
In three of the four that `return` did not even stop the process before
this PR: `selfTest()` was typed `never` and the dispatch was a bare `if
(SELF_TEST) selfTest();`, so control fell through to the real audit,
which on a built tree prints its own green line and exits 0.
## The retrofit changes nothing about what these gates say about the
tree
Each file's audit path is **byte-identical** to the branch point
(`sha256` over the region, taken from the two blobs):
| file | region | lines | sha (base) | sha (head) |
|:--|:--|--:|:--|:--|
| `check-exported-any.ts` | `// ── Audit ─` → EOF | 69 |
`74d2dbb87e8e5ec6` | `74d2dbb87e8e5ec6` |
| `check-dual-source-exports.ts` | `// ── Audit ─` → EOF | 92 |
`f95e248f6ce2a74d` | `f95e248f6ce2a74d` |
| `check-error-code-provenance.ts` | file start → self-test banner | 368
| `8ad0283719c0f855` | `8ad0283719c0f855` |
| `check-browser-reachable-entries.ts` | file start → self-test banner |
714 | `82c3e7ee5ab505f9` | `82c3e7ee5ab505f9` |
The only change outside the self-test region is each file's dispatch
block. All four real (non-self-test) runs are green after a
`packages/spec` build, and say what they said before:
```
✅ no exported type resolves to `any`: 2378 types + 1457 schemas across 17 entry points.
✅ no new dual-source exports: 5001 names across 17 entry points — 204 re-exported (single declaration), 0 accepted dual-source (baseline).
OK — every registered-code stamp site is listed under its own owner key or carries a recorded waiver (10 waiver(s), all live)
✅ check:browser-reachable-entries — 2 declared browser-reachable entries link no zod; 44 bundle(s) scanned, 40 zod link(s) seen elsewhere (instrument calibrated); …
```
## Checker health is not a PR verdict — and now their own health is
falsifiable
`scripts/pm/dispatch-gates.mjs --commands` already separates this class
by name, printing a `--self-test`-only family as 「⚠ checker-health only
(--self-test) — NOT a PR verdict」. The repo therefore already knew a
self-test is not a verdict about the tree. What these four lacked is the
other half: a self-test whose own *health* can be falsified. A roster
with a floor makes a shrunk battery say so, and the handshake makes a
silent return say so.
Two design points worth naming, both copied from the landed precedents
rather than invented:
- **registration asserts REACH, not failure.** The registering call is
the first statement of each file's assertion sink, before the outcome is
consulted. Routed through the failure sink instead, a fully green run
would register zero cases and every battery would read DID NOT RUN — the
floor inverted rather than installed.
- **the flag is the self-test's last statement**, after the verdict line
prints and after the floor has been evaluated. A flag set before the
floor would report "reached" for a run that never evaluated it.
⛔ Nothing is imported: each of the four still runs standalone as `tsx
scripts/NAME.ts --self-test`, which is the whole point of the mechanism.
## Residue — the eight files this tranche does NOT touch
Measured at this branch's head, all eight still answer roster `0` /
handshake `0` on the predicates above:
```
packages/spec/scripts/build-declaration-map.ts
packages/spec/scripts/build-export-origins.ts
packages/spec/scripts/build-migration-registry.ts
packages/spec/scripts/check-llms-txt.ts
packages/spec/scripts/check-objectui-pin-citations.ts
packages/spec/scripts/check-skill-examples.ts
packages/spec/scripts/check-template-manifests.ts
packages/spec/scripts/check-yaml-examples.ts
```
12 today, not the card's 13: PR #18916 landed
`check-duration-unit-keys.ts`, which now carries all three pieces
(roster 10 / handshake 1) and is out of the population.
## Changeset
No changeset — **this diff releases nothing**, and the exemption label
is the gate's own prescription for that case. `@objectstack/spec` ships
`dist`, `json-schema`, `liveness`, `prompts`, `llms.txt`, `README.md`,
`src/**/*.zod.ts`, `CHANGELOG.md`, `api-surface`,
`api-surface-declarations` and `spec-changes.json`; the four changed
files live under `scripts/`, which is **not** in that list, so they
reach no tarball and a named changeset would publish a version whose
content is unchanged. This PR briefly carried a `patch` changeset
because the dispatch told the author to write one without checking
whether the package ships `scripts/` — that was the **seat's** error;
the author measured the contradiction and flagged it in `open_questions`
rather than resolving it silently, and the changeset was dropped in
`0e8b6567a49e` (one file, −31 lines, nothing else moved).
`skip-changeset` was then applied by the seat, a label write the author
is forbidden to make, after checking the one case where that label is
refused: ruling ② B on #18375 refuses it where a PR's `.changeset` rows
are CHANGED, and this PR's diff against base carries **zero**
`.changeset` rows of any status.
## Verification
Head of this branch when these were taken: `fe05b5ae1`.
| what | command | reading |
|:--|:--|:--|
| gate families | `node scripts/pm/dispatch-gates.mjs --commands --repo
objectstack-ai/objectstack` | **63 derived from the real change set** (5
paths vs merge base `0ec81857a`), every one run, exit code captured
before any pipe |
| census | `… --ran RECORD` | `✓ 63 derived famil(ies) accounted for —
61 run, 2 NOT-MEASURED (2 DERIVED from a recorded exit 3)` |
| **61** | — | exit **0** |
| **2** | `pnpm check:dual-build-cjs-loads`, `pnpm
check:lean-entry-closure` | exit **3 = PREREQUISITE NOT MET** — both
read BUILT output of every package (83 packages have no `dist` in this
container; only `packages/spec`'s closure was built). Their own
`--self-test` legs passed inside the same run. ⛔ Recorded as not
measured, not as a pass and not as a finding. |
| build | `pnpm --filter '@objectstack/spec...' build --concurrency=2`,
through `scripts/pm/os-verify-lock.sh` | `VERDICT command-exit 0` · held
the lock 145s · waited 0s |
| typecheck | `pnpm --filter @objectstack/spec typecheck` | exit 0 —
`tsc --noEmit`, `check:scripts-typecheck` (this is the leg that compiles
`scripts/**`) and `check:test-typecheck` |
| package tests | `pnpm --filter @objectstack/spec test`, under the same
lock | exit 0 — 491 files, **14299 tests** pass |
| the two suites that drive these four scripts | `vitest run
--maxWorkers=2 scripts/dist-freshness-adoption.test.ts
scripts/check-error-code-provenance.test.ts` | exit 0 — 24 tests. They
pin `--self-test` end to end: the spawned `--self-test` must exit 0
printing `self-test OK`, and both stale-dist cases require `--self-test`
to still run and exit 0 on a dist it never reads. |
| all four gates end to end | `pnpm --filter @objectstack/spec run
check:{exported-any,dual-source-exports,error-code-provenance,browser-reachable-entries}`
| exit 0 each — self-test then real audit |
| lint | `pnpm lint` (`eslint . --no-inline-config`, the whole repo) |
exit 0 — the full run, so **no narrowing to declare** |
| control bytes | `grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]'` over
the four files, plus `pnpm check:nul-bytes` | no match; gate exit 0 |
`origin/main` was merged into this branch before these readings (three
commits, `scripts/check-closing-target-claim.mjs`,
`scripts/measure-self-test-floor.mjs`, `scripts/pm/os-verify-lock.sh` —
all disjoint from this diff, none `merge=os-regen`, no regeneration debt
recorded).
## Acceptance notes
`noted, not filed`: `scripts/measure-self-test-floor.mjs` — the repo's
own self-test floor/handshake instrument — has a population of repo-root
`scripts/**` only (`population()` walks `join(ROOT, 'scripts')`), so the
twelve gates under `packages/spec/scripts/` that #18919 is about cannot
be probed by it at all, in either direction. Nothing in `AGENTS.md` or
that file's own contract promises that directory, so this is a scope gap
rather than a violated contract; widening it is its own card and its own
measurement. Next toucher: whoever dispatches tranche 2 of #18919, or
whoever regenerates `docs/audits/2026-09-self-test-shape-census.md`.
---
_Generated by [Claude Code](https://claude.ai/code)_
---------
Co-authored-by: Claude <noreply@anthropic.com>1 parent ab41a8f commit c70581b
4 files changed
Lines changed: 637 additions & 29 deletions
File tree
- packages/spec/scripts
Lines changed: 157 additions & 6 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
715 | 715 | | |
716 | 716 | | |
717 | 717 | | |
| 718 | + | |
| 719 | + | |
| 720 | + | |
| 721 | + | |
| 722 | + | |
| 723 | + | |
| 724 | + | |
| 725 | + | |
| 726 | + | |
| 727 | + | |
| 728 | + | |
| 729 | + | |
| 730 | + | |
| 731 | + | |
| 732 | + | |
| 733 | + | |
| 734 | + | |
| 735 | + | |
| 736 | + | |
| 737 | + | |
| 738 | + | |
| 739 | + | |
| 740 | + | |
| 741 | + | |
| 742 | + | |
| 743 | + | |
| 744 | + | |
| 745 | + | |
| 746 | + | |
| 747 | + | |
| 748 | + | |
| 749 | + | |
| 750 | + | |
| 751 | + | |
| 752 | + | |
| 753 | + | |
| 754 | + | |
| 755 | + | |
| 756 | + | |
| 757 | + | |
| 758 | + | |
| 759 | + | |
| 760 | + | |
| 761 | + | |
| 762 | + | |
| 763 | + | |
| 764 | + | |
| 765 | + | |
| 766 | + | |
| 767 | + | |
| 768 | + | |
| 769 | + | |
| 770 | + | |
| 771 | + | |
| 772 | + | |
| 773 | + | |
| 774 | + | |
| 775 | + | |
| 776 | + | |
718 | 777 | | |
719 | 778 | | |
720 | 779 | | |
721 | 780 | | |
722 | 781 | | |
723 | 782 | | |
724 | 783 | | |
725 | | - | |
| 784 | + | |
726 | 785 | | |
| 786 | + | |
| 787 | + | |
| 788 | + | |
| 789 | + | |
| 790 | + | |
| 791 | + | |
| 792 | + | |
| 793 | + | |
| 794 | + | |
| 795 | + | |
| 796 | + | |
| 797 | + | |
| 798 | + | |
| 799 | + | |
| 800 | + | |
| 801 | + | |
727 | 802 | | |
| 803 | + | |
| 804 | + | |
728 | 805 | | |
729 | 806 | | |
730 | 807 | | |
731 | 808 | | |
732 | 809 | | |
| 810 | + | |
733 | 811 | | |
734 | 812 | | |
735 | 813 | | |
| |||
746 | 824 | | |
747 | 825 | | |
748 | 826 | | |
| 827 | + | |
749 | 828 | | |
750 | 829 | | |
751 | 830 | | |
| |||
764 | 843 | | |
765 | 844 | | |
766 | 845 | | |
| 846 | + | |
767 | 847 | | |
768 | 848 | | |
769 | 849 | | |
| |||
813 | 893 | | |
814 | 894 | | |
815 | 895 | | |
| 896 | + | |
816 | 897 | | |
817 | 898 | | |
818 | 899 | | |
| |||
998 | 1079 | | |
999 | 1080 | | |
1000 | 1081 | | |
| 1082 | + | |
1001 | 1083 | | |
1002 | 1084 | | |
1003 | 1085 | | |
| |||
1033 | 1115 | | |
1034 | 1116 | | |
1035 | 1117 | | |
| 1118 | + | |
1036 | 1119 | | |
1037 | 1120 | | |
1038 | 1121 | | |
| |||
1044 | 1127 | | |
1045 | 1128 | | |
1046 | 1129 | | |
| 1130 | + | |
1047 | 1131 | | |
1048 | 1132 | | |
1049 | 1133 | | |
| |||
1077 | 1161 | | |
1078 | 1162 | | |
1079 | 1163 | | |
| 1164 | + | |
1080 | 1165 | | |
1081 | 1166 | | |
1082 | 1167 | | |
| |||
1092 | 1177 | | |
1093 | 1178 | | |
1094 | 1179 | | |
| 1180 | + | |
1095 | 1181 | | |
1096 | 1182 | | |
1097 | 1183 | | |
| |||
1131 | 1217 | | |
1132 | 1218 | | |
1133 | 1219 | | |
1134 | | - | |
1135 | | - | |
| 1220 | + | |
| 1221 | + | |
| 1222 | + | |
| 1223 | + | |
| 1224 | + | |
| 1225 | + | |
| 1226 | + | |
| 1227 | + | |
| 1228 | + | |
| 1229 | + | |
| 1230 | + | |
| 1231 | + | |
| 1232 | + | |
| 1233 | + | |
| 1234 | + | |
| 1235 | + | |
| 1236 | + | |
| 1237 | + | |
| 1238 | + | |
| 1239 | + | |
| 1240 | + | |
| 1241 | + | |
| 1242 | + | |
| 1243 | + | |
| 1244 | + | |
| 1245 | + | |
| 1246 | + | |
| 1247 | + | |
| 1248 | + | |
| 1249 | + | |
| 1250 | + | |
| 1251 | + | |
| 1252 | + | |
| 1253 | + | |
| 1254 | + | |
| 1255 | + | |
| 1256 | + | |
| 1257 | + | |
| 1258 | + | |
| 1259 | + | |
| 1260 | + | |
| 1261 | + | |
| 1262 | + | |
| 1263 | + | |
1136 | 1264 | | |
1137 | 1265 | | |
1138 | | - | |
1139 | | - | |
| 1266 | + | |
| 1267 | + | |
| 1268 | + | |
| 1269 | + | |
| 1270 | + | |
| 1271 | + | |
| 1272 | + | |
| 1273 | + | |
| 1274 | + | |
| 1275 | + | |
1140 | 1276 | | |
1141 | 1277 | | |
1142 | | - | |
| 1278 | + | |
| 1279 | + | |
| 1280 | + | |
| 1281 | + | |
| 1282 | + | |
| 1283 | + | |
| 1284 | + | |
| 1285 | + | |
| 1286 | + | |
| 1287 | + | |
| 1288 | + | |
| 1289 | + | |
| 1290 | + | |
| 1291 | + | |
| 1292 | + | |
| 1293 | + | |
1143 | 1294 | | |
0 commit comments