test(cli): split the suite into named unit and integration vitest tiers - #14536
Conversation
…jects Two named tiers in packages/cli/vitest.config.ts: `unit` (the local default) and `integration` (spawns the real CLI or boots a real kernel/driver; CI-mandatory, local on demand). `vitest run` with no --project still runs every file, so CI is unchanged by construction. The tier predicate is behavioural, not the *.e2e.test.ts name: an explicit INTEGRATION_FILES list, kept equal to what each file's comment-masked source does by test/vitest-tiers-partition.test.ts, which also pins that the two projects partition every test file on disk (a file matching no project would fall out of `pnpm test` too). Both projects use `extends: true` so the resolve.alias table and server.deps.external carry into each; the console-intercept disarm is repeated per project because the root one is inert under projects. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016yfqQh2dBgPAymYd7xipza
📓 Docs Drift Check1 anchor(s) derived from 1 changed package(s); no hand-written page names any of them, so this run has nothing to list — not a clean bill of health. This check sees only pages that NAME a derived anchor: one that documents this change in prose, or enumerates it in an authoring dialect, names none and stays invisible to it on every run. What this run could not see
Coarse fallback — 22 page(s) merely mention a changed package (the pre-#9192 predicate, kept for the deliberately-wide backstop): Which tree this was computed onThis run read A worktree cut from an older # while this PR is open — GitHub drops the merge commit once it closes
git fetch origin e2061c22e63e713ffe07d15788262907300a1a5b && git checkout e2061c22e63e713ffe07d15788262907300a1a5b
# afterwards, rebuild it from the two parents, which stay fetchable
git fetch origin a5b95ee6edd1e21e04cc43200f41fb730538b197 e33556e63bb36569f037d8d07c59869714726927 && git checkout -B drift-repro a5b95ee6edd1e21e04cc43200f41fb730538b197 && git merge --no-ff e33556e63bb36569f037d8d07c59869714726927
node scripts/docs-audit/affected-docs.mjs --json a5b95ee6edd1e21e04cc43200f41fb730538b197 |
os-trump
added
skip-changeset
Claude
⛔ merge queue 构建失败 — 先分诊，再决定要不要重排队列构建 33623396688 红了。队列跑的是全量套件（PR 侧 CI 只跑 affected 子集）， 失败的 job（日志抽取，best effort）：
跨 PR 相同签名（24h，按失败测试文件聚合）：
历史信号：
分诊清单：
Generated by Claude Code · merge-queue-triage workflow (#4859) |
The partition pin caught both in the merge queue, on main after this branch's base: test/build-multi-package-artifact.e2e.test.ts (new; spawns the CLI through bin/run-dev.js under tsx) and src/utils/schema-migration-plugins.declaration-boot-write-guard.test.ts (existing; now constructs a real ObjectQL engine). Both join INTEGRATION_FILES; the header's population line moves to the merged tree (230 files = 158 unit + 72 integration). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_016yfqQh2dBgPAymYd7xipza
2 participants
Part of #13504
Implements the maintainer's ruling on #13504 (director comment, 2026-09-01): the
packages/clisuite is split into two named vitest tiers —unit(the local default, minutes-scale) andintegration(spawns the real CLI or boots a real kernel/driver; CI-mandatory, run locally on demand). Nothing is skipped, weakened, deleted or converted to a double:vitest runwith no--projectruns every project, sopnpm test(the@objectstack/cli#testturbo task CI runs) still collects every file. The card stays open — the other half of the ruling, the Definition-of-done wording forpackages/clicards, lives inAGENTS.md, a governed surface, and is only proposed here (last section).Commands
No package script was added:
packages/cli/package.jsonandturbo.jsonare held by open PRs (#14065 / #13956 and #14346), so the tiers live entirely invitest.config.tsand are invoked through vitest's own--project. A named script is a follow-up for after those land; the split is usable without one.What changed
packages/cli/vitest.config.ts—test.projectswith two inline projects, bothextends: true. Vitest 4.1.10: an inline project inherits nothing by default — verified by resolving both projects programmatically (createVitest→project.vite.config.resolve.alias/project.config.server.deps.external): each carries the 4resolve.aliasentries and theserver.deps.externalpattern; withoutextendsthey would be declared at root and enforced nowhere. Each project repeatsdisableConsoleIntercept: true(root-level is inert under projects;check:console-intercept-disarmrequires it per project).unit= the default include minusINTEGRATION_FILES(spreadingconfigDefaults.excludefirst, sonode_modulesstays excluded);integration= exactlyINTEGRATION_FILES, an exported explicit list. A header section records the predicate, the population and the census reconciliation.packages/cli/test/vitest-tiers-partition.test.ts(unit tier; needs nodist/) — four pins: every test file on disk is onevitest list --filesOnlycollects with no--project;unitandintegrationpartition that population (none in both, none in neither);INTEGRATION_FILESequals the behavioural predicate re-derived from every file's comment-masked source (and equals what vitest resolves, so a dead entry fails too); and the pin classifies itself asunit.projectsconfigured, a file that matches no project is not run bypnpm testeither — the pin is what keeps that from happening silently.The predicate — what a file DOES, not what it is called
The ACCEPT on #13504 fixed that the
*.e2e.test.tsname disagrees with behaviour, so the tier is keyed on behaviour, measured over comment-masked source (scripts/js-comment-mask.mjs):runServe((the helper intest/helpers/serve-process.tswhose body spawns the source entry), or value-importsnode:child_processand (names an entry basenamerun-dev.js/run.js, or importsCLI/TSXfrom that helper, or names the.bin/tsxbinary).bootSchemaStackfromschema-migrate, orbetter-sqlite3, or any@objectstack/driver-*package, or constructsnew ObjectQL(.unit.Value imports only — an
import typefrom@objectstack/driver-sql, aCONTRACT_ONLY_SPELLINGSlist that says'better-sqlite3', andexpect(deps).toContain('better-sqlite3')boot nothing; each was a false positive of the text-match census this replaces.Reconciliation against the #13872 census (f532630, 220 files)
runServe(), 5 through the helper's exportedCLIpath constant; −3 that name an entry basename in an assertion without importingchild_process(invocation,plugin-commands,serve-built-cli-prerequisite); +1tsxchild of this package's own source (serve-host-fallback-base)CONTRACT_ONLY_SPELLINGSlists, a banner fixture, a connection-display formatter, a scaffold-dependency assertion and twoimport typedriver imports are not bootsmigrate-apply-refuses-before-ddlOn
00ff228fe0(this branch's base, 228 files; 229 with this pin): 70 integration (48 spawn, 23 kernel, 1 both) / 159 unit.Patch round (merge queue eviction, run 33623396688): on
3b5f8168b5(merge oforigin/main2a26536196, 230 files) the list is 72 integration (49 spawn, 24 kernel, 1 both) / 158 unit. The pin named two files and both were added:test/build-multi-package-artifact.e2e.test.ts(new in #14513 — value-importsnode:child_process, names therun-deventry basename and thetsxbinary: spawns the CLI) andsrc/utils/schema-migration-plugins.declaration-boot-write-guard.test.ts(modified in #14505 — now constructsnew ObjectQL(: kernel). The second is the shape a name-based tier can never see: an existing, non-.e2efile that started booting the real engine.Measured: the fast tier's wall
One lock hold, detached (
nohup, lock acquired inside the script), onfaf99c2620with the dependency closure built inside the same hold (turbo run build --filter=@objectstack/cli --concurrency=2: 56 tasks, 4 cached, exit 0, 365s). The fast tier,pnpm --filter @objectstack/cli exec vitest run --project unit --maxWorkers=2:4m28s wall for the local default, against 17m23s (#13872, exit 0) / 23m58s (the card) for the whole suite on this box — and the hold's 10m48s includes the build, the pin and the two ablations, not only the tier. Shared-box seconds (the lock's own caveat): three sibling seats queued behind this hold and unlocked gate work ran beside it. Read the SHAPE, which travels: the unit tier's
teststerm is 148s against the whole suite's 1741s (#13872), i.e. the tier keeps ~8.5% of test-body time while holding 159 of 229 files and 2077 of ~2600 cases; itsimportterm (346s) is now the dominant term, which is the per-file floor #13872 attributed and the part no split can move. The integration tier was not run locally (declared to CI, wherepnpm testruns both).The full suite was not re-run locally (CI owns it; #13872 measured it at 17–24 min under this lock). The partition proof is
vitest list --filesOnly(exit 0 each): 228 files with no--projecton the base tree, unchanged from before the split; 158 + 70 with--project unit/--project integration(159 + 70 with the pin); on the patch-round head3b5f8168b5230 = 158 + 72; no duplicates; union equal to the whole.The pin, red-first
Baseline on the committed tree, before the build (the pin needs no
dist/):vitest run --project unit test/vitest-tiers-partition.test.ts→Test Files 1 passed (1) · Tests 4 passed (4) · Duration 3.74s, exit 0.Two ablations, each: mutate → prove the mutation landed on disk by an anchored count → run the pin → restore with
git checkout HEAD -- ABSOLUTE_PATH(the config's absolute path) under anEXIT INT TERMtrap → prove the restore bygit diff HEADempty ANDgit hash-objectequal to the HEAD blobb46ba7a0e3db9d69f3c966d16970398ad6233a28:'test/capability-preflight.test.ts'appended to the unit project'sexclude): anchored count 0 → 1; pin exit 1 —AssertionError: vitest run collects a different population than the filesystem holds: expected [ …(228) ] to deeply equal [ …(229) ]namingtest/capability-preflight.test.ts; 1 failed / 3 passed. This is the CI-relevant direction: the file vanished from the no---projectrun too. Restore: diff-lines=0, blob equal.INTEGRATION_FILES(src/adr-0048-app-split.test.tsline deleted): anchored count 1 → 0; pin exit 1 —AssertionError: files that spawn the CLI or boot a kernel/driver but are NOT in INTEGRATION_FILES (add them): … "src/adr-0048-app-split.test.ts [objectQLCtor]"; 1 failed / 3 passed. Restore: diff-lines=0, blob equal.No
dist/is involved in either leg (the pin reads sources and spawnsvitest list), so no dist preflight applies.Patch round: on the merged tree
3b5f8168b5the pin went red exactly as in the queue (files that spawn the CLI or boot a kernel/driver but are NOT in INTEGRATION_FILES (add them)naming the two files above); after the two entries were added it is green under the lock:Test Files 1 passed (1) · Tests 4 passed (4) · Duration 3.69s,os-verify-lock: VERDICT command-exit 0 · held the lock 6s · waited 296s.The two file lists (on
e33556e63b)integration— 72 filesunit— 158 filesVerification
Gate union derived AFTER the final commit on
e33556e63bwithnode scripts/pm/dispatch-gates.mjs --repo objectstack-ai/objectstack --commands(no paths passed; the script took the change set from the merge base). Every exit code captured BEFORE any pipe (each command redirected to its own log,EXIT=$?read, log tailed after). Reconciliation by exactcomm -23over sorted lists: union named 27, ran 27, unreconciled 0.25 exit 0 · 2 exit 3 (NOT MEASURED, in the gates' own words) · 0 other.
NOT MEASURED (exit 3), quoted:
node scripts/check-test-completeness.mjs— Fix: pass a savedturbo run testlog — or, running the family locally, record this gate as NOT MEASURED.pnpm check:dual-build-cjs-loads— PREREQUISITE NOT MET — this gate reads built output, and some package has no dist/.Green, with each gate's own verdict line:
node scripts/check-ci-filter-parity.mjs(0s) — OK: all 130 declared cross-package glob(s) (92 unique) are covered bycoreorcrosspkg, everycrosspkgentry still covers one, and thetestjob'sif:still namenode scripts/check-comment-mask-adoption.mjs(2s) — OK check:comment-mask-adoption — 14 private comment-stripper(s) under packages/** + examples/**, all 14 recorded and every recorded row still reached (13 unconverted, 1node scripts/check-cross-package-test-inputs.mjs(0s) — OK: 25 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.node scripts/check-keyed-text-bounds.mjs(1s) — ✓ check:keyed-text-bounds: 112 .object.ts files under packages/* + apps/** + examples/** (walk is repo-wide; 0 outside), 117 object declarations, 251 declared index entnode scripts/check-plugin-teardown-shape.mjs(5s) — ✓ check:plugin-teardown-shape: 64 Plugin implementation(s) across 5267 source(s) under packages/**; every teardown-shaped method (stop / shutdown / close / dispose) sitsnode scripts/check-shard-attestation.mjs(0s) — ✓ check-shard-attestation: 2 aggregate gate(s) count 3 declared leg(s) across 3 attesting job(s).node scripts/check-system-context-census.mjs(4s) — check-system-context-census: OK — 109 elevation read sites in 20 packages across 45 files, all anchored; 145 anchors resolve, 27 declared non-read.node scripts/check-undeclared-dep-imports.mjs(3s) — ✓ check:undeclared-dep-imports: 79 workspace packages under packages/** + apps/** + examples/**, 2145 non-test src files, 1861 @objectstack/* specifiers (0 assembled, notnode scripts/docs-audit/check-affected-docs.mjs(4s) — ✓ affected-docs self-test: 516 cases pass.node scripts/docs-audit/check-drift-comment.mjs(1s) — ✓ check-drift-comment: 56 cases pass across 5 fixture diff(s).pnpm check:cli-test-child-env(11s) — ✓ check:cli-test-child-env: 46 spawner source(s) among 113 under packages/cli/test/**; no new bulk process.env copy reaches a spawned child, all 53 spawn call(s) declarepnpm check:cross-package-test-inputs(3s) — OK: 25 package(s) read outside themselves, all declared, and turbo.json hashes every declared glob.pnpm check:doc-authoring(10s) — ✓ doc authoring guard: sibling-package prose ids hold the baseline — 831 pinned site(s) across 231 file(s), 82793 string(s) read in 1124 parsed source(s), no growth, no bpnpm check:engine-double-contract(53s) — check-engine-double-contract: OK — 753 pinned, 134 in the DEBT ledger, 3 exempt.pnpm check:logger-receiver-detach(21s) — OK every log channel keeps its receiver: 2394 non-test TS file(s) walked, 0 detach(es) on the 5 declared receiver-sensitive sink spelling(s).pnpm check:objectql-double-limit(27s) — OK ObjectQL doublelimitconformance holds: 306 double(s) graded, 107 apply the caller's bound or refuse it loudly.pnpm check:page-declaration-shape(19s) — check-page-declaration-shape: OK — 34 page entries across 2406 sources under packages/, examples/, apps/** all reach the kernel through a discoverable declaration (`:pnpm check:published-files(2s) — ✓ check:published-files — 69 publishable package(s) of 79 workspace member(s) declare afileswhitelist that covers every entry point plus CHANGELOG.md and admits no tepnpm check:query-options-erasure(297s) — ✓ query-options-erasure ratchet holds: 67 unswept non-test site(s) in 17 file(s), none new, and every file measured parsed. Every other non-test file under packages/ is cpnpm check:slot-lookup(133s) — ✓ slot-lookup ratchet holds: 106 unswept site(s) in 25 file(s), none new, and every file in the population parsed. Every other file under packages/ is covered by `pnpm lipnpm check:test-source-alias(11s) — check-test-source-alias OK — 72 packages with tests scanned; 61 registered as still resolving a workspace dep throughdist/; 47 published subpath(s) resolved through evpnpm check:type-check-coverage(2s) — check-type-check-coverage: OK — 69/79 workspace packages type-checked (plus the root), 10 in the DEBT ledger (305 frozen raw errors, https://github.com/objectstack-ai/objpnpm check:type-source-resolution(2s) — check-type-source-resolution OK — 114 tsc program(s) across 78 packages scanned (everytsconfig*.jsoneach package'stypecheckscript names,check:type-source-resolutionreads each package'stsconfig.jsonand only that one — the siblingtsconfig.test.jsonthis repo prescribes as its own repair is a whole tsc program outside the gate's declared population #11490); 56 registered aspnpm check:where-matcher(11s) — ✓ where-matcher conformance holds: 329 matcher(s) discovered, 329 answer the combinator battery correctly or refuse it loudly (209 refuse).pnpm check:type-check-debt(450s (under the lock)) — check-type-check-coverage --re-measure: OK — 22 ledger entr(ies) re-measured in 447.0s, 755 raw tsc error(s) total, none above its recorded number.Beyond the union:
pnpm lint(whole repo,eslint . --no-inline-config) exit 0 in 213s on the patch-round tree, identical to the committed tree, not narrowed;pnpm --filter @objectstack/cli typecheckexit 0 (29s) one33556e63bwith the dependency closure built inside the same lock hold (turbo run build --filter=@objectstack/cli --concurrency=2: 56 tasks, 2 cached, exit 0, 637s); the fresh worktree had reported TS2307 for every workspace dependency before that build, the unbuilt-closure false red, NOT MEASURED, not red. As in round 1 that program compiles the 118src/**/*.test.tsfiles and nottest/(--listFiles: 0 hits for the pin), so the pin remains covered by the round-1 ad-hoc reading (tsc --noEmit --ignoreConfig --module NodeNext --moduleResolution NodeNext --target ES2022 --strict --types node test/vitest-tiers-partition.test.tsexit 0; the file is unchanged since). Re-run one33556e63boutside the union: eslint on both changed files exit 0;pnpm check:nul-bytesexit 0 (check-nul-bytes: OK (scanned 7979 text file(s) -- 7979 tracked, 0 untracked-not-ignored; skipped 7 binary; no raw ASCII control bytes));check:console-intercept-disarmexit 0 (OK: 72 vitest-running package(s), every one disarms console interception at the package root). Positive control after the merge:vitest list --filesOnlygives 230 files;--project unit158,--project integration72; 0 in both, union = all. Round-1 readings ond98ff795a9(the accepted head) stand in the PR history; every gate above was re-run one33556e63b.Deviations from the dispatch route
runServe,CLI/TSX), not the census's basename/text match: the census's file classes could not be reproduced without its file list, and re-deriving them exposed 11 helper-routed spawners and 7 non-booting text matches — the reconciliation table above is the explanation of every difference.projects, "a file in no project still runs in CI" (route step 4) is false — such a file is skipped bypnpm testtoo; the pin's first case exists for exactly that reason.Proposed Definition-of-done wording for
packages/clicardsFor the skills seat to land in
AGENTS.md(governed; not edited here). Proposed sentence, verbatim:Generated by Claude Code
🤖 Generated with Claude Code
https://claude.ai/code/session_016yfqQh2dBgPAymYd7xipza