Skip to content

Commit 682c98d

Browse files
claude[bot]claude
andauthored
docs(agents): write down the shape a --self-test must have to be capable of failing (#18227)
Fixes #15410 Clause-②: no Reason: this is PROSE in `AGENTS.md` plus one ratchet ceiling raised by a maintainer-ruled amount. No gate rule, threshold, ledger, exemption list or population is widened — nothing that used to be refused is now admitted. ⛔ **GOVERNED SURFACE.** `AGENTS.md` is the rules layer. Review is requested from both authorized approvers (`GOVERNED_APPROVERS` = os-zhuang, hotlong). The dispatching seat will not arm auto-merge, will not enqueue and will not merge this. It lands by human approval or not at all. ## The ruling this revision executes, quoted Maintainer ruling on **decision batch #135 item 4**, verbatim and untranslated: > 135 同意 The form it was presented in, verbatim and untranslated: > B,目标 ≤ 24 行,并按实测行数抬上限 Recorded on #15410 as comment 5682595374. That ruling also (a) answers `os-zhuang`'s CHANGES_REQUESTED question 「这个关键文件没有长度限制吗?」 — yes, and this is the raise; (b) refuses option D, the cross-file move; and (c) rules that ⛔ no other ceiling moves. ## What changed since the CHANGES_REQUESTED round The previous round landed the section at **41 lines** and left `check:pm-skill-ratchet` **RED** on purpose, because raising a ceiling is a human-floor item. The ruling adopts letter **B**. This revision therefore: 1. **Compresses the section to its judgeable clauses** — the assertion-floor rule, the verdict-handshake rule, and one pointer line. 41 → **24 lines including the heading**. 2. **Moves the two worked non-handshake shapes** into `docs/audits/2026-09-self-test-shape-census.md`. ⛔ They were not deleted — see the reading below. 3. **Raises `['AGENTS.md', 1075]` to `1099`** in `scripts/pm/check-skill-line-ratchet.mjs` — exactly the measured count, not one line more. ⛔ `['AGENTS.md', 768]` in the widest-row table is untouched, and ⛔ no other entry in that map moved. ⛔ No existing rule line was deleted to pay for anything, and ⛔ no re-wrap was used as currency. ## The measurement, and the two numbers that must agree | reading | value | |:--|--:| | compressed section, heading through its closing separator | **24 lines** | | `AGENTS.md` on `origin/main` | 1075 lines | | `AGENTS.md` on this branch | **1099 lines** | | ceiling before / after | 1075 / **1099** | | the raise | **+24** | The raise equals the measured section count exactly. Independently checkable: `git diff origin/main...HEAD --numstat` reads `24 0 AGENTS.md`. **The ratchet, before and after, quoted from its own verdict lines.** Before (the state `os-zhuang` reviewed): ``` ✗ check-skill-line-ratchet: AGENTS.md is 1116 lines; the ratchet ceiling is 1075. ... Raising a ceiling requires a maintainer ruling quoted in the PR. ``` After compression, before the ceiling edit — the same gate's **second** axis fired, which is why the pointer is two physical lines and not one: ``` ✗ check-skill-line-ratchet: AGENTS.md has 1 line(s) over the 120-byte budget: L1048 (170B). ``` Now: ``` ✓ check-skill-line-ratchet: AGENTS.md is 1099 lines (ceiling 1099; headroom 0). ✓ check-skill-line-ratchet: AGENTS.md: widest table row is 768 bytes (pin 768; headroom 0). ✓ check-skill-line-ratchet self-test: 157 cases pass. ``` Headroom is 0 again by construction: the next author needing a line is back to compressing. ## The two ACCIDENT examples are readable in the audit document They landed under `### The ACCIDENT` in `docs/audits/2026-09-self-test-shape-census.md`, in a new subsection at **line 130**, as a side-by-side code block naming what each dispatch does under an early `return`: - `process.exit(selfTest());` — the DEFEATED shape, at line 138. - `process.exit(selfTest() === 0 ? 0 : 1);` — the ACCIDENT shape, at line 140. Readings, with a fire control so a zero would be visible: | grep | count | |:--|--:| | `### The ACCIDENT` (positive control) | 1 | | `process.exit(selfTest());` in the new block | 1 (plus 2 pre-existing rows in the 4-DEFEATED block) | | `selfTest() === 0` | 1 | | `selfTest() !== 0` (fire control — must be absent) | **0** | ## What the compressed section still says - **Floor** — pin battery NAMES with a per-battery minimum. Fail when a declared battery under-registers, when a registered case names no declared battery, or when the roster itself falls below its pinned battery count. ⛔ A printed count is evidence, not proof; one pinned TOTAL rots the moment a sibling battery grows. - **Handshake** — the verdict sets a module-level flag and the dispatch refuses when it is unset, and SAYS the self-test never reached its verdict. ⛔ An exit code alone is not a handshake. - **Copy a landed one, ⛔ never import one** — `scripts/check-agent-model-declared.mjs` carries all three parts; every self-test must keep running standalone. - **One pointer line** at `docs/audits/2026-09-self-test-shape-census.md` and the `scripts/measure-self-test-floor.mjs` docblock. What left the file: the incident narrative, the 2026-09-05 census totals, and the two worked shapes. ⚠️ **Truthful note on the instrument the pointer names.** On today's `origin/main` `scripts/measure-self-test-floor.mjs` **refuses and prints no census** — `THE HANDSHAKE CENSUS IS NOT COMPLETE -- no census printed`, naming #14968 and two files whose landed handshake its recogniser reads as `none`. It refuses rather than publishing a flattering number, which is the right behaviour, but it means the pointer's **how-to** is readable today in the docblock and the audit document, while the instrument's own census is not runnable until #14968 lands. The AGENTS.md line claims only where to read, not that a census prints. ## Gates Derived on the merged tree with `node scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands` — the tool's own changeset derivation, ⛔ not a hand-written path list. The branch was first merged with `origin/main` so the derivation is not taken from a stale tree (the tool refused to be quiet about that). **45 derived families, 45 run, 0 red.** Reconciled with `--ran`, exit codes recorded: ``` Run reconciliation — 45 derived, 45 run, 0 NOT-MEASURED, 0 UNRUN. ✓ dispatch-gates --ran: 45 derived famil(ies) accounted for — 45 run, 0 NOT-MEASURED (a DERIVED zero — all 45 recorded an exit code and none of them is 3). ``` Every exit code was captured with `cmd > log 2>&1; EXIT=$?`, ⛔ never across a pipe. One family first exited **3 — PREREQUISITE NOT MET**, which is not a finding: `check:doc-formula-expressions` needs `@objectstack/formula` and `@objectstack/lint` built. Built under the shared verify lock, re-run, exit 0. **Lint** — narrowed and declared, with the three readings that make the narrowing a measurement rather than a skip: 1. Population is read from `eslint.config.mjs` itself: its base block is `files: ['**/*.{ts,tsx,mts,cts,js,jsx,mjs,cjs}']` and no markdown or MDX plugin is configured, so `AGENTS.md` and the audit document are outside ESLint's population entirely. One changed file is in it. 2. `eslint --no-inline-config --format json scripts/pm/check-skill-line-ratchet.mjs` → **1 file linted, 0 errors, 0 warnings**, exit 0. 3. Immutability for untouched files: this repo runs one `eslint.config.mjs`, which "never enables type-aware linting (no `parserOptions.project`, no typed `@typescript-eslint` rules) for ANY file" (its own words, at the `QUERY_OPTIONS_TEST_GLOBS` header). A diff in one `.mjs` therefore cannot move the verdict on any file it does not contain. The repo-wide `pnpm lint` is CI's run. Also run, because `dispatch-gates` marks their rosters as sitting under a directory one of these paths is in and warns that their silence is evidence in neither direction: `check-published-list-mirrors`, `check-skills-token-ratchet`, `check:console-injection`, `check:dts-closure`, `check:engine-double-contract`, `check:i18n-stale-fill`, `check:pm-label-desc-cap` — all exit 0. `check:published-readme-exports` exits **3 (PREREQUISITE NOT MET — 44 packages not built)**: NOT MEASURED locally, declared to CI. It reads built type entries and this diff touches no package source. `skip-changeset` holds unchanged: `AGENTS.md`, `docs/audits/**` and `scripts/pm/**` are shipped by no package's `files[]`. ## 维护者速读(草稿) **改了什么** — 按您批的 B,把上一轮那 41 行压到 **24 行(含标题)**,只留可判定的部分:自检必须钉住 **battery 名字 + 每个 battery 的最小用例数**(不是一个总数),以及 **verdict 置一个模块级 flag、dispatch 读不到就拒绝**。剩下一行指针,指向审计文档和那支工具的文件头。上一轮那两个「看着像检查、其实什么都没查」的写法**搬进了审计文档**(不是删掉,文中给了行号和发火对照)。然后把 `AGENTS.md` 的天花板从 1075 抬到 **1099** —— 正好是实测的 24 行,一行不多。⛔ 同一个文件里另一张表的 `['AGENTS.md', 768]` 没碰,别的天花板也一个没动;⛔ 没删任何既有规则行来腾地方。 **为什么改** — 仓库强制要求 `--self-test`,也用门禁保证「它存在、CI 会跑它」,但**怎么写从来没写下来**。代价不是重复造轮子,是严谨度参差:`failures.length === 0` 作唯一成功条件时,「每个用例都过了」和「一个用例都没跑」打印出来一模一样。对「规则本身就是缺陷类」的门禁,自检是唯一的仪器 —— 自检空转,等于同一个失效往上挪了一层。这一节把那个形状变成可以据以判人的条款。 **风险与代价(含回滚)** — 风险低:散文 + 一行天花板数字,不动任何门禁逻辑、不动任何包、不发布任何东西。代价就是这 24 行的行数预算,已按您的裁定付掉,付完余量回到 0(下一个要加行的人照样得先压缩)。**回滚**:revert 本 PR 的那一个改动 commit,`AGENTS.md` 立刻回到 1075 行、天花板回到 1075,门禁全绿,不留残留。⚠️ 一个如实提示:指针指向的那支工具 `scripts/measure-self-test-floor.mjs` 今天在 `origin/main` 上是**拒绝出数**的(它自陈 handshake 识别器不完整,点名 #14968)。它拒绝而不是撒谎,是对的;但这意味着**指针目前只保证「去哪儿读」,不保证「跑得出普查表」**。上面正文里写明了这一点,没有把它说成能跑。 **席位意见** — (留空,待维护者填写) **你要做的** — 一个动作:确认这 24 行和 1075 → 1099 这个数,然后按受管路径走两个人工批准。⛔ 本 PR 不会由任何 agent 合并、入队或武装 auto-merge。如果您认为 24 行还嫌多,**请直接给一个目标行数** —— 执行席会再压一轮并重报实测行数,⛔ 不会靠塞水词或砍掉一条子句去凑。 --- _Generated by [Claude Code](https://claude.ai/code/session_017ef78bLdybu3AffehKkhfk)_ --- _Generated by [Claude Code](https://claude.ai/code)_ --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2a4a796 commit 682c98d

3 files changed

Lines changed: 63 additions & 1 deletion

File tree

‎AGENTS.md‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1026,6 +1026,30 @@ registry? Add it to `OPEN_CAPABILITY_REGISTRIES` in the same PR that fixes it.
10261026

10271027
---
10281028

1029+
## Writing a `--self-test` — it must be capable of failing when it runs nothing
1030+
1031+
**Floor — pin battery NAMES, never one total.** Declare a frozen roster of battery name → minimum
1032+
case count, register every case against a battery, and fail when a declared battery registers fewer
1033+
cases than its pin, when a registered case names no declared battery, or when the roster itself
1034+
falls below its pinned battery count. ⛔ A printed case count is EVIDENCE, NOT PROOF — a battery
1035+
falling 40 → 3 still prints a non-zero count — and one pinned TOTAL rots the moment a sibling grows.
1036+
1037+
**Handshake — the verdict sets a module-level flag, and the dispatch refuses when it is unset.**
1038+
Set the flag as the self-test's last statement, after its success line prints; the dispatch must
1039+
SAY the self-test never reached its verdict. ⛔ An exit code is not a handshake. Without this a
1040+
`return` above the verdict prints nothing and exits 0, and a perfect floor never runs either —
1041+
the two holes are ORTHOGONAL, so close both.
1042+
1043+
**Copy a landed one — ⛔ never import one.** `scripts/check-agent-model-declared.mjs` carries all
1044+
three parts (`SELF_TEST_BATTERIES`, `SELF_TEST_BATTERY_FLOOR`, `selfTestReachedVerdict`); every
1045+
self-test must keep running standalone as `node scripts/<x>.mjs --self-test`, so a shared assertion
1046+
module is one point of failure for every instrument at once.
1047+
1048+
Both non-handshake shapes, and how to classify and probe your own:
1049+
`docs/audits/2026-09-self-test-shape-census.md` and the `scripts/measure-self-test-floor.mjs` docblock.
1050+
1051+
---
1052+
10291053
## Post-Task Checklist
10301054

10311055
1. `pnpm test` — verify nothing broke. Touched a type-check-covered package? `pnpm typecheck` too.

‎docs/audits/2026-09-self-test-shape-census.md‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -127,6 +127,25 @@ instrument's header that is **not** a hold: a comparison against a missing retur
127127
to be false, and nothing detected anything. It belongs on the repair list beside the
128128
4 DEFEATED, not among the 165 that held.
129129

130+
#### The two non-handshake shapes — the worked examples, held here
131+
132+
`AGENTS.md`'s `--self-test` section points here for these two rather than carrying them itself.
133+
They are the two shapes an author reaches for when the dispatch is supposed to notice an early
134+
`return` above the verdict. Neither notices anything; the difference is only in the exit code:
135+
136+
```js
137+
// the DEFEATED shape — 4 rows above carry it
138+
process.exit(selfTest()); // early return → process.exit(undefined) → exit 0
139+
// the ACCIDENT shape — this row carries it
140+
process.exit(selfTest() === 0 ? 0 : 1); // early return → undefined === 0 → false → exit 1,
141+
// ZERO BYTES printed; the arithmetic did it, not a check
142+
```
143+
144+
⛔ Neither is a handshake. **A handshake is the dispatch reading a module-level flag the VERDICT
145+
set, and SAYING so when it is unset** — which is why a non-zero exit alone cannot be scored as a
146+
hold, and why the ACCIDENT row is classified apart from the 165. The 159 HELD rows that answer the
147+
mutation do it in one sentence: `selfTest() returned without reaching its verdict`.
148+
130149
### What the 9 NOT MEASURED are, and what they are not
131150

132151
They are limits of the probe, published per row rather than folded into a verdict — 9

‎scripts/pm/check-skill-line-ratchet.mjs‎

Lines changed: 20 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1361,7 +1361,26 @@ export const CEILINGS = new Map([
13611361
// and holds the gate's LEVEL axis unchanged). AGENTS.md is not a CROSS_FILE_MOVES
13621362
// destination, so no `ruledRaises` record applies. Landed count, headroom 0, same
13631363
// convention.
1364-
['AGENTS.md', 1075],
1364+
//
1365+
// 1075 → 1099 (card #15410): the `--self-test` shape section — nothing in this repo
1366+
// said what a self-test must look like, so every author re-derived it and the two
1367+
// holes (no assertion floor; no verdict handshake) kept being re-entered. Landed as
1368+
// the two judgeable clauses plus one pointer line: the floor pins battery NAMES with
1369+
// a per-battery minimum (never one total), and the verdict sets a module-level flag
1370+
// the dispatch refuses on when unset. The narrative, the census numbers and the two
1371+
// worked non-handshake shapes are NOT here — they were moved into
1372+
// `docs/audits/2026-09-self-test-shape-census.md`, which the pointer line names
1373+
// alongside the `scripts/measure-self-test-floor.mjs` docblock. +24 lines, the
1374+
// measured count of the compressed section including its heading, against a first
1375+
// draft that measured 41; the section has 0 lossless rewrap headroom at the
1376+
// surrounding ~100-byte prose width, and the ruling's single pointer line is two
1377+
// physical lines only because one would be 170B against this file's own 120-byte
1378+
// per-line budget. Maintainer ruling, verbatim and untranslated:
1379+
// 「135 同意」 (decision batch #135 item 4, 2026-09-15, presented as 「B,目标 ≤ 24 行,
1380+
// 并按实测行数抬上限」; recorded by the director on #15410 comment 5682595374, which
1381+
// also refuses option D). AGENTS.md is not a CROSS_FILE_MOVES destination, so no
1382+
// `ruledRaises` record applies. Landed count, headroom 0, same convention.
1383+
['AGENTS.md', 1099],
13651384
// #9965: root CLAUDE.md is the other repo-root instruction file — same read
13661385
// path (every seat session), same governance (Prime Directive #14). It is
13671386
// structurally growth-prone in the way the ratchet is built for: it exists to

0 commit comments

Comments
 (0)