ci(test): e2e and live filename tiers move to a nightly run on main (#16481)

* ci(test): e2e and live filename tiers move to a nightly run on main

One switch, OS_TEST_TIERS (queue | nightly, unset = queue), read once in
scripts/nightly-tiers.mjs and applied in packages/cli's tier walk — the
one config that owns tier files (60 *.e2e.test.ts; the tree owns no
*.live.test.*). turbo.json hashes the variable into every test task so a
nightly can never replay a queue-mode cache entry. ci.yml's shard step
spells queue; test-nightly-tiers.yml runs the two tiers on main, two
shards through the existing partitioner, and files or refreshes one
deduplicated card on red.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RfFHiRCSs3JXLK4cwcfox

* ci(nightly-tiers): pull_request paths trigger and a path-less population declaration

dispatch-gates refuses a family reachable only from a scheduled workflow, so
the nightly carries the patrol posture: a pull_request trigger paths-filtered
to its own file and the switch reader. The reader declares no-path-population
(its self-test drives a temp fixture) and resolves vitest's CLI entry by
walking up from require.resolve('vitest') instead of spelling a subpath the
derivation read as a population. Concurrency keyed per ref so a PR run never
queues behind main's.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RfFHiRCSs3JXLK4cwcfox

* test(cli): the secret-orphans driver contract declares the key its production-posture boot needs

The test sets NODE_ENV=production and boots a real kernel; LocalCryptoProvider
then needs a stable key, which this test never supplied. It was green only when
an earlier dev-mode boot in the same job had persisted one to the runner's
home directory (the ordering coupling serve-process.ts documents) — and with the
e2e tier no longer in the queue run, that sibling is gone. Set OS_SECRET_KEY to
a fresh random value in beforeAll, restored in afterAll beside NODE_ENV; nothing
is written to disk and nothing is depended on from a sibling. Ruled by the seat
for exactly this one file.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_019RfFHiRCSs3JXLK4cwcfox

---------

Co-authored-by: Claude <noreply@anthropic.com>

Author: claude[bot]

.github/workflows/ci.yml

@@ -497,6 +497,14 @@ jobs:
       - name: Run this shard's tests
         env:
           NODE_OPTIONS: --report-on-signal --report-signal=SIGUSR2 --report-directory=${{ runner.temp }}/stall-reports
+          # The `e2e` and `live` filename tiers run NIGHTLY on main
+          # (test-nightly-tiers.yml), not here: `queue` is the per-PR and
+          # merge-queue setting, read once in scripts/nightly-tiers.mjs. Spelled
+          # explicitly even though unset reads the same, so the setting this
+          # required check verifies is written where the check runs.
+          # turbo.json hashes it in the `test` task's `env`, which is what lets
+          # it reach vitest under strict env mode at all.
+          OS_TEST_TIERS: queue
         run: |
           if [ ! -s "$RUNNER_TEMP/shard-packages.txt" ]; then
             echo "No packages on this shard — nothing to test."

.github/workflows/test-nightly-tiers.yml

@@ -44,6 +44,7 @@
  */
 
 import { describe, it, expect, beforeAll, afterAll, vi } from 'vitest';
+import { randomBytes } from 'node:crypto';
 import { mkdtempSync, rmSync } from 'node:fs';
 import { tmpdir } from 'node:os';
 import { dirname, join, resolve } from 'node:path';
@@ -117,10 +118,15 @@ describe('os secret orphans — the concrete driver behind both reads (#14843)',
     }
     savedEnv.OS_ARTIFACT_PATH = process.env.OS_ARTIFACT_PATH;
     savedEnv.NODE_ENV = process.env.NODE_ENV;
+    savedEnv.OS_SECRET_KEY = process.env.OS_SECRET_KEY;
     // Deliberately absent: no compiled artifact, so the boot is the bare data
     // stack plus the two plugins the command passes.
     process.env.OS_ARTIFACT_PATH = join(dir, 'dist', 'objectstack.json');
     process.env.NODE_ENV = 'production';
+    // The key this production-posture boot needs, declared here rather than
+    // inherited from a sibling's persisted `$HOME/.objectstack/dev-crypto-key`
+    // (#16491): a fresh value per run, never written to disk.
+    process.env.OS_SECRET_KEY = randomBytes(32).toString('hex');
     // The command does not pass `projectRoot`, so its boot takes `process.cwd()`
     // for its state directory. Stand in the tempdir so the run under test keeps
     // its state there instead of in whatever directory vitest started in.
@@ -154,7 +160,7 @@ describe('os secret orphans — the concrete driver behind both reads (#14843)',
       if (savedEnv[key] === undefined) delete process.env[key];
       else process.env[key] = savedEnv[key];
     }
-    for (const key of ['OS_ARTIFACT_PATH', 'NODE_ENV'] as const) {
+    for (const key of ['OS_ARTIFACT_PATH', 'NODE_ENV', 'OS_SECRET_KEY'] as const) {
       if (savedEnv[key] === undefined) delete process.env[key];
       else process.env[key] = savedEnv[key];
     }

packages/cli/src/commands/secret/orphans.driver-contract.test.ts

@@ -85,11 +85,42 @@
  * tokens that must track the predicate, which is the exact class of copy this
  * change exists to delete. Revisit only with a measurement that says it costs
  * something real.
+ *
+ * ## The NIGHTLY tiers are a second, orthogonal cut — by NAME (#16455)
+ *
+ * The two tiers above answer "what does this file DO" and decide which
+ * project collects it. `OS_TEST_TIERS` answers a different question — "which
+ * RUN is this" — and is decided by the file's NAME alone: `*.e2e.test.*` and
+ * `*.live.test.*` are the nightly tiers, everything else is the queue's. The
+ * switch is read ONCE, in `scripts/nightly-tiers.mjs` (the values, the default
+ * and the refusal of anything else live there), and applied here in
+ * `testFilesOnDisk()` — the one walk both `integrationTestFiles()` and
+ * `unitTestFiles()` derive from — so the setting narrows the POPULATION and
+ * the behavioural predicate then partitions whatever is left:
+ *
+ *   OS_TEST_TIERS unset / queue   population = every test file that is NOT nightly-tier
+ *   OS_TEST_TIERS=nightly         population = exactly the nightly-tier files
+ *
+ * Both projects stay a partition of that population BY CONSTRUCTION
+ * (`unit` = population − predicate, `integration` = population ∩ predicate),
+ * and `test/vitest-tiers-partition.test.ts` still measures it under whichever
+ * setting it runs in: its `vitest list` child inherits the switch through
+ * `childEnv()` and its filesystem walk is this function, so the two sides of
+ * every equality it asserts are read under the same setting.
+ *
+ * ⛔ The two cuts deliberately disagree on 5 files and that is not a defect:
+ * a file that carries the `.e2e` name and spawns plain node is nightly-tier
+ * (name) AND `unit` (behaviour); a file that spawns the CLI without the name
+ * is queue (name) AND `integration` (behaviour). The ruling that landed the
+ * nightly selects by the existing filename tiers only and renames nothing, so
+ * the name decides the run and the behaviour decides the project, and no file
+ * is renamed to make the two agree.
  */
 
 import { readdirSync, readFileSync } from 'node:fs';
 import { join, relative, sep } from 'node:path';
 import { maskComments } from '../../scripts/js-comment-mask.mjs';
+import { readTierMode, selectTierFiles } from '../../scripts/nightly-tiers.mjs';
 
 // ---------------------------------------------------------------------------
 // The predicate
@@ -164,7 +195,13 @@ export function tierOfFile(pkgRoot: string, relPath: string): TierSignals {
 const TEST_FILE_RE = /\.(?:test|spec)\.[cm]?[jt]sx?$/;
 const SKIP_DIRS = new Set(['node_modules', 'dist', '.git', '.turbo', 'coverage']);
 
-/** `pkgRoot`-relative, POSIX-separated paths of every test file on disk, sorted. */
+/**
+ * `pkgRoot`-relative, POSIX-separated paths of every test file on disk that
+ * the current `OS_TEST_TIERS` setting selects, sorted — the nightly-tier files
+ * under `nightly`, everything else under `queue` (the default). See "The
+ * NIGHTLY tiers" in the header: this is the ONE place the switch narrows this
+ * package's population, and both tier derivations below read from it.
+ */
 export function testFilesOnDisk(pkgRoot: string): string[] {
   const out: string[] = [];
   const walk = (dir: string): void => {
@@ -176,16 +213,33 @@ export function testFilesOnDisk(pkgRoot: string): string[] {
     }
   };
   walk(pkgRoot);
-  return out.sort();
+  return selectTierFiles(out.sort(), readTierMode());
 }
 
 /**
- * The integration tier: every test file on disk the predicate calls integration.
+ * The integration tier: every selected test file the predicate calls integration.
  *
  * This is what `vitest.config.ts` feeds to the `integration` project's
- * `include` and the `unit` project's `exclude`, so the two projects stay a
+ * `include`; `unitTestFiles()` is its complement, so the two projects stay a
  * partition of the population by CONSTRUCTION rather than by maintenance.
  */
 export function integrationTestFiles(pkgRoot: string): string[] {
   return testFilesOnDisk(pkgRoot).filter((file) => isIntegration(tierOfFile(pkgRoot, file)));
 }
+
+/**
+ * The unit tier: the selected population MINUS the integration tier — the exact
+ * complement of `integrationTestFiles()` over the same walk, handed to the
+ * `unit` project's `include`. Spelled as an include list rather than as
+ * `exclude: INTEGRATION_FILES` so that BOTH projects read the switched
+ * population above; an exclude-shaped unit tier would fall back to vitest's
+ * default `include` and collect nightly-tier files the queue must not run.
+ *
+ * `integration` defaults to a fresh derivation; the config passes the list it
+ * already derived so the second tier costs one walk (~10ms) and no second
+ * classification pass (~250ms measured on the box this landed on).
+ */
+export function unitTestFiles(pkgRoot: string, integration: readonly string[] = integrationTestFiles(pkgRoot)): string[] {
+  const integrationSet = new Set(integration);
+  return testFilesOnDisk(pkgRoot).filter((file) => !integrationSet.has(file));
+}

packages/cli/vitest-tiers.ts

@@ -563,27 +563,66 @@
 // the PR that landed this section; re-measure it when the population moves,
 // and print the commit here.
 //
+// ## THE NIGHTLY TIERS (#16455) — a second cut, by NAME, read from `OS_TEST_TIERS`
+//
+// Maintainer direction (2026-09-07, verbatim): 「我想的是测试会不会太多，是否都是
+// 必要的，是不是应该砍，每次修改都要完整的测试吗」. The `e2e` and `live` tiers
+// leave the per-PR and merge-queue runs and run nightly on `main`; selection is
+// by the EXISTING filename tiers only (`*.e2e.test.*`, `*.live.test.*`) and no
+// file is renamed, deleted or edited to move it. Measured on 6eba38f5a3: this
+// package owns every one of the tree's 60 `*.e2e.test.ts` files and the tree
+// owns no `*.live.test.*` at all, so this is the one config that reads the
+// switch today — read through `scripts/nightly-tiers.mjs`, the single reader of
+// the variable, so a package that adopts a tier tomorrow imports rather than
+// re-spells it.
+//
+//   OS_TEST_TIERS unset / queue   this package's population = the 212 non-tier files (180 unit + 32 integration)
+//   OS_TEST_TIERS=nightly         this package's population = exactly the 60 tier files (1 unit + 59 integration)
+//
+// (272 test files on disk at 6eba38f5a3, read from `vitest list --filesOnly`
+// under each setting and each `--project`; 212 + 60 = 272, no file in both.)
+//
+// The cut is applied in ONE place — `testFilesOnDisk()` in `vitest-tiers.ts`,
+// the walk both derivations below read from — so the behavioural partition
+// into `unit` / `integration` operates on whatever population the setting
+// selected, and the two projects remain a partition of it by construction.
+// That is also why the unit tier is now an INCLUDE list rather than
+// `exclude: INTEGRATION_FILES`: an exclude-shaped unit project falls back to
+// vitest's default `include`, which would collect the tier files the queue
+// must not run. `test/vitest-tiers-partition.test.ts` measures all of this
+// under whichever setting it runs in (its `vitest list` child inherits the
+// switch); under `nightly` it is not itself collected — it carries no tier
+// name — which is the ruled behaviour, not a gap.
+//
+// ⛔ The switch reaches vitest only because `turbo.json` names `OS_TEST_TIERS`
+// in the `test` task's `env`: turbo 2.10 runs in STRICT env mode and strips
+// every undeclared variable before the task's shell sees it. It sits in `env`
+// (hashed) rather than `passThroughEnv` on purpose — a `test` task's cached
+// outcome depends on the setting, so the setting is in the hash and a nightly
+// can never replay a queue-mode cache entry as `>>> FULL TURBO`.
+//
 // ⚠️ INLINE PROJECTS INHERIT NOTHING BY DEFAULT — `extends: true` is what
 // carries this file's `resolve.alias` table and `test.server.deps.external`
 // into each project (vitest 4.1.10: an inline project without it gets a fresh
 // Vite config, so the source aliases the gate above guards would be declared
 // here and enforced nowhere). `disableConsoleIntercept: true` is repeated
 // inside every project because `check:console-intercept-disarm` measured the
-// root-level setting inert under projects. `exclude` for the unit tier spreads
-// `configDefaults.exclude` first: an `exclude` that names only the integration
-// files would drop the `node_modules` exclusion and start collecting
-// dependencies' own test files.
-import { configDefaults, defineConfig } from 'vitest/config';
+// root-level setting inert under projects. Both projects carry an explicit
+// `include` of exact paths, so neither reaches vitest's default `include`
+// (which would collect the whole tree) and neither needs its own
+// `node_modules` exclusion: an exact-path list matches nothing it does not name.
+import { defineConfig } from 'vitest/config';
 import path from 'path';
-import { integrationTestFiles } from './vitest-tiers.js';
+import { integrationTestFiles, unitTestFiles } from './vitest-tiers.js';
 
-// The integration tier, DERIVED from what the files DO — never written down.
-// `vitest-tiers.ts` holds the predicate, the walk and the argument for both;
-// `test/vitest-tiers-partition.test.ts` pins what a derivation cannot pin
-// about itself. Package-root-relative, POSIX-separated, sorted; each entry is
-// an exact path, which is what lets the same array serve as the integration
-// project's `include` and the unit project's `exclude`.
+// The two tiers, DERIVED from what the files DO — never written down — over
+// the population `OS_TEST_TIERS` selects. `vitest-tiers.ts` holds the
+// predicate, the walk and the argument for both; `test/vitest-tiers-partition.test.ts`
+// pins what a derivation cannot pin about itself. Package-root-relative,
+// POSIX-separated, sorted; each entry is an exact path, which is what lets
+// each array serve as its project's `include`.
 export const INTEGRATION_FILES = integrationTestFiles(__dirname);
+export const UNIT_FILES = unitTestFiles(__dirname, INTEGRATION_FILES);
 
 export default defineConfig({
   resolve: {
@@ -662,17 +701,18 @@ export default defineConfig({
         external: [/packages[\/]types[\/]dist/],
       },
     },
-    // The two tiers (#13504) — see the header section of the same name. Both
-    // `extends: true` so each project inherits the `resolve.alias` table and
-    // the `server.deps.external` entry above; each repeats the console-intercept
+    // The two tiers (#13504) — see the header section of the same name, and
+    // "THE NIGHTLY TIERS" for the population both read. Both `extends: true`
+    // so each project inherits the `resolve.alias` table and the
+    // `server.deps.external` entry above; each repeats the console-intercept
     // disarm because the root-level one is inert under projects.
     projects: [
       {
         extends: true,
         test: {
           name: 'unit',
           disableConsoleIntercept: true,
-          exclude: [...configDefaults.exclude, ...INTEGRATION_FILES],
+          include: UNIT_FILES,
         },
       },
       {

packages/cli/vitest.config.ts

@@ -0,0 +1,41 @@
+// Types for the `nightly-tiers.mjs` exports a TypeScript consumer reads -- the
+// same problem, and the same fix, as `js-comment-mask.d.mts` next door (#5475).
+//
+// The module itself stays `.mjs`: it carries a `--self-test` / `--packages` /
+// `--check` / `--failing-files` CLI run with bare `node` from the nightly
+// workflow, and every root script here is authored that way. What needs the
+// declaration is the other direction -- `packages/cli/vitest-tiers.ts` imports
+// the switch from inside a tsc program (`tsconfig.test.json`), where an untyped
+// `.mjs` import is TS7016 and `readTierMode` silently becomes `any`.
+//
+// PARTIAL BY DESIGN, the `check-regen-pending.d.mts` shape: the module exports
+// more names than this declares, and importing an undeclared one is `TS2305`
+// -- loud, red and immediate -- never a silent `any`. Keep this file in step
+// with the module by hand; `check:declaration-mirrors` holds the names, kinds
+// and required arities.
+
+/** The two tiers the switch moves off the per-PR and merge-queue runs. */
+export const NIGHTLY_TIERS: readonly string[];
+
+/** The two legal spellings of `OS_TEST_TIERS`. */
+export const TIER_MODES: readonly string[];
+
+/** The environment variable the switch is read from. */
+export const TIER_ENV: string;
+
+/** A test file in one of the nightly tiers, judged on its basename. */
+export const NIGHTLY_TIER_FILE_RE: RegExp;
+
+export function isNightlyTierFile(relPath: string): boolean;
+
+/**
+ * The switch's value: `queue` when unset or empty, else exactly `queue` or
+ * `nightly`; any other spelling throws.
+ */
+export function readTierMode(env?: Record<string, string | undefined>): 'queue' | 'nightly';
+
+/**
+ * The files `mode` selects out of `files`: under `queue` everything NOT in a
+ * nightly tier, under `nightly` exactly what is. Order preserved.
+ */
+export function selectTierFiles(files: string[], mode: 'queue' | 'nightly'): string[];