ci(test-tiers): per-PR gate for a package adopting a tier file without reading OS_TEST_TIERS

[claude]

Fixes #16494

scripts/nightly-tiers.mjs (#16455, PR #16481) moved the e2e and live test tiers off the per-PR and merge-queue runs onto a nightly on main, gated by OS_TEST_TIERS. Its own header states what that does not buy: a package that adopts its first *.e2e.test.* or *.live.test.* file without reading the switch runs that file in the merge queue (nothing excludes it) and its whole suite under the nightly (nothing narrows it) — both silent. nightly-tiers.mjs --check makes that loud, but only on the nightly, one night after the PR landed, and its refusal then blocks the whole tier run for one package's omission.

This adds the per-PR half: scripts/check-tier-file-adoption.mjs, wired into lint.yml's lint job (the required Lint & Repo Gates context) as pnpm check:tier-file-adoption.

Which route, and why

The card left the choice open and called it a cost question. Static walk, not --check's vitest probe.

--check measures the truth — two vitest list --filesOnly runs per tier-owning package, judged by judgeCollection — and that is the right instrument for a nightly and the wrong one for every pull request: it boots vitest, twice per package, to answer a question about adoption. This gate spawns nothing. It answers the cheaper question statically: does the package's vitest configuration reach the switch at all? It imports --check's own tierPackages and NIGHTLY_TIERS from nightly-tiers.mjs, so the two can never disagree about which files are tier files or which packages own them, and it says so in its header that green here means "the switch is wired in", never "the partition is right" — the semantic verdict stays --check's, on the nightly.

Measured on this branch: the gate's full run (self-test + real-tree sweep) is sub-second.

Why the detector cannot be a grep for the variable name

This is the load-bearing measurement, taken on d03c3c96d6 against the entire population of tier-owning packages, which is one:

file	OS_TEST_TIERS raw	with comments masked
packages/cli/vitest.config.ts	5	0
packages/cli/vitest-tiers.ts	4	0
scripts/nightly-tiers.mjs	5	1

All nine occurrences in the two config files are prose. The real read is an import edge: the config imports ./vitest-tiers.js, which imports readTierMode and selectTierFiles from ../../scripts/nightly-tiers.mjs.

So both obvious detectors are wrong, in opposite directions:

• grep the config with comments included → green on prose; it would pass a package whose config merely mentions the switch in a comment explaining why it does not read it.
• grep the config with comments masked → red on packages/cli, the one package that does this correctly. A 100% fabrication rate over today's whole population.

The detector therefore walks the config's local relative-import closure and accepts either signal in a code position: a value import of the switch-reading surface (readTierMode / selectTierFiles / OS_TEST_TIERS) from the reader module, or the variable named directly. Comments are masked with the shared scripts/js-comment-mask.mjs (the module check:comment-mask-adoption exists to keep everyone on). Both wrong detectors are pinned as controls in the self-test, plus a type-only import (reads nothing at runtime → red) and an import of isNightlyTierFile alone (a filename predicate, not a switch read → red).

The three acceptance legs, measured

leg	expected	measured
tier file in a package that reads the switch	green	✓ two rows — direct import, and the transitive packages/cli shape through a local module
tier file in a package that does not	red, naming package and file	✓ two rows — blind config, and no config at all; the finding carries both
no tier files on disk at all	green, printing the zero	✓ asserts tierFileCount === 0, owners === 0, packagesWalked > 0, and that the printed line carries a 0 for each tier by name

⭐ Leg 3 is a reverse control, and a green that never looked would satisfy a weaker version of it. Two things make the zero real:

1. The zero is printed on every run, red or green: N nightly-tier test file(s) on disk (X e2e, Y live). "Clean" is not a sentence this gate can print without a count behind it.
2. A walk that finds zero workspace packages is exit 2, not a green — MEASUREMENT FAILED, distinct from a finding's 1. That is the one anti-vacuity floor kept; the selection floor its sibling check-registry-log-declared carries is deliberately inverted here and the header says why: zero engine-booting packages means that gate went blind, while zero tier files is a legitimate state of this tree — it is what the tree looked like before packages/cli adopted the tier.
3. A companion row proves leg 3 is not vacuous from the other side: a file named live-dialect-matrix.test.ts is not a tier file (live is a tier a package adopts by naming a file into it, not a substring this gate hunts), so the zero in leg 3 is a judged zero, not an empty directory.

The one-time red proof

Run against the real tree with one scratch tier file added to a non-reading package, then removed. Trap-guarded; the mutation was proven on disk before the run (grep -c on the injected marker), and the tree was proven clean afterwards (git status --porcelain empty). No permanent fixture.

$ node scripts/check-tier-file-adoption.mjs            # before
EXIT=0
OK: 80 workspace package(s) walked, 65 nightly-tier test file(s) on disk (65 e2e, 0 live),
    owned by 1 package(s); every one reads OS_TEST_TIERS.
    @objectstack/cli — 65 file(s); imports readTierMode, selectTierFiles from
      scripts/nightly-tiers.mjs (via packages/cli/vitest-tiers.ts)

# scratch file added: packages/types/src/scratch-red-proof.e2e.test.ts

$ node scripts/check-tier-file-adoption.mjs            # RED
EXIT=1
check-tier-file-adoption: 80 workspace package(s) walked, 66 nightly-tier test file(s) on disk
  (66 e2e, 0 live); 1 owning package(s) do not read OS_TEST_TIERS:

  @objectstack/types (packages/types) owns 1 nightly-tier test file(s) and its vitest
  configuration does not reach OS_TEST_TIERS:
      packages/types/src/scratch-red-proof.e2e.test.ts
    A package that owns a nightly-tier file must read OS_TEST_TIERS, or the file runs in
    the merge queue (nothing excludes it) and the package's WHOLE suite runs under the
    nightly (nothing narrows it) -- both silent.
    Wire the switch in vitest.config.ts by routing the file list through the one reader,
    the way packages/cli does: ...
    Then confirm the partition with: node scripts/nightly-tiers.mjs --check

# scratch file removed

$ node scripts/check-tier-file-adoption.mjs            # after — byte-identical to "before"
EXIT=0

The count moved 65 → 66 → 65 and the package name and file path both appear in the finding.

Census re-derived on this branch — it has drifted from the card

The card's numbers were taken at PR #16481 / 770dd18205. Re-measured at d03c3c96d6:

	card (770dd18205)	this branch (d03c3c96d6)
standalone vitest.config.ts files	72	73
root / workspace vitest config	none	none (0 vitest.workspace*, no root vitest.config.*)
packages owning tier files	1 (packages/cli)	1 (packages/cli)
tier-named files	60 *.e2e.test.ts	65 *.e2e.test.*
*.live.test.* files	0	0
check:*tier* script in package.json	none	none (so the card was not already satisfied)

Controls taken in the same pass, on the same tree: the *.live.test.* zero sits beside 3541 *.test.* files matched by the same enumeration, so the zero is a reading and not a broken glob; OS_TEST_TIERS occurs in 8 tracked files, and a nonsense control term returns 0.

⚠️ Stated plainly, because it is the honest limit of this PR: with zero *.live.test.* files anywhere in the tree, the gate's live arm — and leg 3's "no tier files" case for that tier — are validated against an empty real corpus. Their only coverage is the self-test's temporary fixture workspaces, which exercise the judging path and cannot tell you the arm has ever met a real file. The gate prints that fact itself on every self-test run (the OS_TEST_TIERS live-tier corpus is 0 file(s) — that arm is covered by FIXTURES ONLY) rather than leaving it to a PR body nobody re-reads. The e2e arm is not in that position: it is measured against 65 real files and one real reading package on every run.

Scope held

⛔ No test file changed. ⛔ No package made to read the switch. The Ruling is explicit that the gate reports and the adopting PR fixes, and the diff is three files: the new script, its package.json entry, and its lint.yml step.

Changeset — graded, not invented

No .changeset/*.md; skip-changeset instead, on a measurement rather than a guess. The diff touches repo-root scripts/, .github/workflows/lint.yml, and the root package.json, which is private: true (@objectstack/spec-monorepo) and ships nothing. Across all 70 published (non-private) workspace packages there are exactly 10 distinct files[] entries — CHANGELOG.md, README.md, api-surface, dist, json-schema, liveness, llms.txt, prompts, spec-changes.json, src/**/*.zod.ts — every one package-local, and none reaches repo-root scripts/ or .github/. Positive control on the same instrument: that enumeration does return dist for @objectstack/spec, so it is capable of reporting a genuinely published path. Nothing publishable moves, so an empty-frontmatter changeset is refused on principle (check:empty-changeset's ledger: it buys nothing the label does not and uniquely risks #4898) and no bump is invented.

Clause-② — the card's no still holds

Surface is scripts/** + lint.yml + a root package.json script entry. No packages/**/src/** path, no published surface. The implementation needed no change under packages/**, so the card's declaration is not contradicted.

维护者速读(草稿)

改了什么 — 新增一个 per-PR 门禁 pnpm check:tier-file-adoption,挂进 lint.yml。它静态判断:凡是拥有 *.e2e.test.* / *.live.test.* 文件的包,其 vitest 配置有没有真的读到 OS_TEST_TIERS 这个开关。没读到就红,点名包和文件。

为什么改 — #16455 把 e2e/live 两层测试从 PR 与合并队列挪到了 nightly。但一个新采用该层的包如果不读开关,那个文件照样在合并队列里跑,而 nightly 又会把它整包跑一遍——两边都静默。原有的 nightly-tiers.mjs --check 能发现,但只在 nightly 上、晚一夜,而且它一拒就把整轮 tier 运行为了一个包的疏漏全部挡住。这条门禁把发现点提前到引入它的那个 PR。

风险与代价(含回滚) — 代价接近零:不启动 vitest,只读约 80 个 manifest 和几个配置文件,亚秒级。风险面是误报:检测器如果写错方向,会把今天唯一做对的 packages/cli 判红。这一点已被实测钉死(见上表:该包里 OS_TEST_TIERS 共 9 次全在注释里,真正的读取是 import 边),两个错误方向都做成了 self-test 的控制用例。回滚 = 删掉这三处改动,没有任何包的源码或测试被触碰。⚠️ 已知局限:全树 *.live.test.* 文件为 0,该臂只有 fixture 覆盖,门禁自己每次都会把这句话打印出来。

席位意见 — (待填)

你要做的 — 看一眼「Which route, and why」那节的取舍是否符合预期(静态拼写检查 vs. 启动 vitest 的语义检查),以及 skip-changeset 的定级是否认可。其余无需操作。

Verification

• node scripts/check-tier-file-adoption.mjs --self-test — exit 0; 12 fixture-workspace cases + 8 pure-judge cases + population declaration + real-tree walk floor.
• node scripts/check-tier-file-adoption.mjs — exit 0 on this branch.
• The one-time red proof above — exit 1, restored, tree clean.
• Gate families derived with node scripts/pm/dispatch-gates.mjs --commands (from the merge-base changeset, not a hand-written list) and run; results in the report.

🤖 Generated with Claude Code

https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU

---

Generated by Claude Code