Skip to content

Commit d18bc32

Browse files
yinlianghuiclaude
andauthored
fix(devx): bound vitest's inner worker pool from the invocation layer (#12182)
* fix(devx): bound vitest's inner worker pool from the invocation layer turbo's outer `--concurrency=50%` (#11954) bounds how many package `test` tasks run at once, but not vitest's own pool inside each of them. 40 of the 41 `vitest.config.ts` files say nothing about pool sizing — the single mention, in `packages/cli`, is a comment recording a REJECTED lever — so every package takes vitest's default of `max(cores - 1, 1)`, which scales with the host rather than with the shard it was given. Peak workers is the product of the two, and both terms grow with core count. Measured on a 4-CPU/15GB container, the product law holds exactly: 2x3=6, 4x3=12, 4x2=8, 4x1=4 concurrent workers observed. The bound goes at the invocation layer, per #10149's recorded reasoning that worker allocation is a property of the shard rather than of any one package's config. No `vitest.config.ts` is touched. Two traps this shape exists to avoid, both measured rather than assumed: - turbo filters task environments, so `VITEST_MAX_WORKERS` alone does NOTHING. Through turbo it spawned 3 workers (the unbounded default) while the same variable on a direct `vitest run` spawned 1. The `globalPassThroughEnv` entry is what makes the lever real. - vitest's `maxWorkers` is a PIN, not a ceiling — `resolveMaxWorkers()` returns the configured value outright. A flat `4` produced 8 workers at outer=2 where the default produces 6, i.e. a flat number RAISES the count on small boxes. So the cap is computed against the host's own cores and only ever lowers. A no-op on any host with <= 5 cores, today's CI runners included. In the regime where it binds (outer=2, inner 8 -> 4 on the 7-package fleet) it cut worker RSS 5700MB -> 2475MB for 93s -> 95s of wall, inside this box's noise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6 * fix(devx): guard the worker-cap script's top level per check:entry-guard The file exports `workerCap`/`WORKER_CEILING` for tests and callers, so its top-level dispatch must not run inside an importer — `check:entry-guard` measured 8 of 39 unguarded exporters ending the importer mid-import, five of them exit 0. Value resolution moves into an exported `resolveValue()` and the write sits behind `isEntrypoint`. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UjM2ia8Av1v5NqfqQEQmC6 --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 2c4c59e commit d18bc32

5 files changed

Lines changed: 139 additions & 1 deletion

File tree

.github/workflows/ci.yml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -556,6 +556,13 @@ jobs:
556556
echo "No packages on this shard — nothing to test."
557557
exit 0
558558
fi
559+
# Bound vitest's INNER worker pool (#11958). A no-op on a runner with
560+
# <= 5 cores (it only ever LOWERS vitest's own `cores - 1` default);
561+
# it exists so a larger runner cannot multiply turbo's outer
562+
# --concurrency by a host-sized inner pool. Empty on failure, which is
563+
# vitest's own "use the default" signal. Needs turbo.json's
564+
# globalPassThroughEnv entry or turbo strips it — see the script header.
565+
export VITEST_MAX_WORKERS="$(node scripts/vitest-worker-cap.mjs)"
559566
FILTERS=$(sed 's/^/--filter=/' "$RUNNER_TEMP/shard-packages.txt" | tr '\n' ' ')
560567
mkdir -p "$RUNNER_TEMP/stall-reports"
561568
node scripts/run-with-stall-guard.mjs --log "$RUNNER_TEMP/test-core.log" --stall-minutes 10 \
@@ -1129,6 +1136,13 @@ jobs:
11291136
env:
11301137
NODE_OPTIONS: --report-on-signal --report-signal=SIGUSR2 --report-directory=${{ runner.temp }}/stall-reports
11311138
run: |
1139+
# Bound vitest's INNER worker pool (#11958). A no-op on a runner with
1140+
# <= 5 cores (it only ever LOWERS vitest's own `cores - 1` default);
1141+
# it exists so a larger runner cannot multiply turbo's outer
1142+
# --concurrency by a host-sized inner pool. Empty on failure, which is
1143+
# vitest's own "use the default" signal. Needs turbo.json's
1144+
# globalPassThroughEnv entry or turbo strips it — see the script header.
1145+
export VITEST_MAX_WORKERS="$(node scripts/vitest-worker-cap.mjs)"
11321146
mkdir -p "$RUNNER_TEMP/stall-reports"
11331147
node scripts/run-with-stall-guard.mjs --log "$RUNNER_TEMP/dogfood.log" --stall-minutes 10 \
11341148
--report-dir "$RUNNER_TEMP/stall-reports" -- \

.github/workflows/rerun-safety-nightly.yml

Lines changed: 14 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,13 @@ jobs:
8989
# frozen while healthy. The guard refuses a turbo run without it.
9090
- name: Test suite — pass 1
9191
run: |
92+
# Bound vitest's INNER worker pool (#11958). A no-op on a runner with
93+
# <= 5 cores (it only ever LOWERS vitest's own `cores - 1` default);
94+
# it exists so a larger runner cannot multiply turbo's outer
95+
# --concurrency by a host-sized inner pool. Empty on failure, which is
96+
# vitest's own "use the default" signal. Needs turbo.json's
97+
# globalPassThroughEnv entry or turbo strips it — see the script header.
98+
export VITEST_MAX_WORKERS="$(node scripts/vitest-worker-cap.mjs)"
9299
node scripts/run-with-stall-guard.mjs --log "$RUNNER_TEMP/rerun-pass1.log" --stall-minutes 15 -- \
93100
pnpm turbo run test --concurrency=4 --force --log-order=stream
94101
@@ -112,6 +119,13 @@ jobs:
112119
# there. Keep the two verdicts apart.
113120
- name: Test suite — pass 2 (same working tree)
114121
run: |
122+
# Bound vitest's INNER worker pool (#11958). A no-op on a runner with
123+
# <= 5 cores (it only ever LOWERS vitest's own `cores - 1` default);
124+
# it exists so a larger runner cannot multiply turbo's outer
125+
# --concurrency by a host-sized inner pool. Empty on failure, which is
126+
# vitest's own "use the default" signal. Needs turbo.json's
127+
# globalPassThroughEnv entry or turbo strips it — see the script header.
128+
export VITEST_MAX_WORKERS="$(node scripts/vitest-worker-cap.mjs)"
115129
status=0
116130
node scripts/run-with-stall-guard.mjs --log "$RUNNER_TEMP/rerun-pass2.log" --stall-minutes 15 -- \
117131
pnpm turbo run test --concurrency=4 --force --log-order=stream || status=$?

package.json

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@
1010
"dev:crm": "node scripts/check-dev-prereqs.mjs && pnpm check:console-sha && pnpm --filter @objectstack/example-crm dev",
1111
"dev:todo": "node scripts/check-dev-prereqs.mjs && pnpm check:console-sha && pnpm --filter @objectstack/example-todo dev",
1212
"spec:rebuild": "turbo run build --filter=...@objectstack/spec",
13-
"test": "turbo run test --concurrency=50%",
13+
"test": "VITEST_MAX_WORKERS=$(node scripts/vitest-worker-cap.mjs) turbo run test --concurrency=50%",
1414
"test:e2e": "turbo run test:e2e",
1515
"typecheck": "turbo run typecheck",
1616
"clean": "turbo run clean && rm -rf dist",

scripts/vitest-worker-cap.mjs

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
// Copyright (c) 2026 ObjectStack. Licensed under the Apache-2.0 license.
2+
3+
/**
4+
* Prints the value for `VITEST_MAX_WORKERS` — the bound on vitest's INNER
5+
* worker pool — for the root `test` script to export into `turbo run test`.
6+
*
7+
* ## Why this exists at all (#11958)
8+
*
9+
* Two fan-outs multiply and neither bounds the other:
10+
*
11+
* OUTER `turbo run test --concurrency=50%` — how many package `test` tasks
12+
* run at once. Bounded as a share of the host's cores (#11954/#11938).
13+
* INNER vitest's own pool inside EACH of those tasks. Unbounded: 40 of this
14+
* repo's 41 `vitest.config.ts` files say nothing about pool sizing
15+
* (the one mention, in `packages/cli`, is a COMMENT recording a
16+
* rejected lever), so every package takes vitest's default, which is
17+
* `max(availableParallelism() - 1, 1)` — i.e. it scales with the
18+
* HOST's core count, not with the shard it was given.
19+
*
20+
* Peak concurrent test-worker processes is therefore `outer × inner`, and both
21+
* terms grow with core count, so the product grows QUADRATICALLY. Measured on
22+
* a 4-CPU/15GB container, the product law holds exactly — 2×3=6, 4×3=12,
23+
* 4×2=8, 4×1=4 workers observed for those combinations.
24+
*
25+
* ## Why a CAP, computed here, and not a flat pinned number
26+
*
27+
* ⚠️ vitest's `maxWorkers` is a PIN, not a ceiling: `resolveMaxWorkers()`
28+
* returns the configured value outright rather than `min()`-ing it with the
29+
* default. Measured: `VITEST_MAX_WORKERS=4` on this 4-core box produced 8
30+
* concurrent workers at outer=2, where the DEFAULT produces 6. A flat number
31+
* small enough to protect a 64-core box would tax every small box, and a flat
32+
* number chosen for comfort would RAISE the count on small boxes. So the cap is
33+
* applied here, against this host's own core count, and only ever lowers.
34+
*
35+
* The ceiling is 4 rather than 1-2 because oversubscription is what makes this
36+
* suite fast — its cost is dominated by module IMPORT, not CPU. Holding the
37+
* ceiling at 4 keeps today's oversubscription ratio roughly constant as core
38+
* count grows (total ≈ 2 × cores) instead of letting it grow with the box.
39+
*
40+
* ## What it buys, measured in the regime where it binds
41+
*
42+
* On the 7-package fleet at outer=2, emulating a larger box by setting the
43+
* inner pool explicitly (peak RSS is of the vitest processes only):
44+
*
45+
* inner=8 (a 9-core box's default) 16 workers 5700 MB workers 93s
46+
* inner=4 (this cap) 8 workers 2475 MB workers 95s
47+
*
48+
* -57% worker RSS for ~0 wall-clock (93s vs 95s is inside this box's run-to-run
49+
* noise; two same-config repeats differed by 17s). On a host with <= 5 cores
50+
* this file returns the default unchanged, so it is a NO-OP for every box the
51+
* project runs on today, CI runners included — which is the point: it bounds
52+
* growth without taxing anyone now.
53+
*
54+
* ## The silent no-op this is paired with
55+
*
56+
* ⚠️ Exporting this variable does NOTHING on its own. Turbo filters task
57+
* environments, so the variable must also be declared in `turbo.json`
58+
* (`globalPassThroughEnv`). Measured before that line existed:
59+
* `VITEST_MAX_WORKERS=1` through turbo spawned 3 workers — the unbounded
60+
* default — while the same variable on a direct `vitest run` spawned 1. If you
61+
* change either half, verify by OBSERVING the worker count (`ps` for
62+
* `--experimental-import-meta-resolve` children), never by the value being
63+
* accepted without error.
64+
*
65+
* ⚠️ The value must be a plain integer. vitest reads this variable with
66+
* `Number.parseInt`, so a percentage — the spelling turbo's `--concurrency`
67+
* accepts — is silently truncated: `VITEST_MAX_WORKERS=50%` means FIFTY
68+
* workers, not half the box.
69+
*/
70+
71+
import os from 'node:os';
72+
import { isEntrypoint } from './invoked-as.mjs';
73+
74+
/** vitest's own default: `max(availableParallelism() - 1, 1)` (non-watch). */
75+
export function vitestDefaultWorkers(cores) {
76+
return Math.max(cores - 1, 1);
77+
}
78+
79+
/** The ceiling. Only ever lowers vitest's default — never raises it. */
80+
export const WORKER_CEILING = 4;
81+
82+
export function workerCap(cores) {
83+
return Math.min(vitestDefaultWorkers(cores), WORKER_CEILING);
84+
}
85+
86+
/**
87+
* Resolves the value to print. An explicit value from the environment wins: a
88+
* developer profiling one package, or a CI job that knows its own runner, is a
89+
* better judge of its shard than this file's host-relative guess. Only a
90+
* positive integer is honoured — anything else falls through to the computed
91+
* cap rather than reaching vitest as NaN.
92+
*/
93+
export function resolveValue(env = process.env) {
94+
const override = Number.parseInt(env.VITEST_MAX_WORKERS ?? '', 10);
95+
if (Number.isInteger(override) && override > 0) return override;
96+
const cores =
97+
typeof os.availableParallelism === 'function' ? os.availableParallelism() : os.cpus().length;
98+
return workerCap(cores);
99+
}
100+
101+
// Guarded per `check:entry-guard`: this file exports bindings, so its top level
102+
// must not run inside an importer.
103+
if (isEntrypoint(import.meta.url)) {
104+
// A non-integer here would reach vitest as NaN and break the pool, so the
105+
// output is validated rather than trusted. An empty value is vitest's own
106+
// "use the default" signal, which is the safe way to fail.
107+
const value = resolveValue();
108+
process.stdout.write(Number.isInteger(value) && value > 0 ? String(value) : '');
109+
}

turbo.json

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,7 @@
22
"$schema": "https://turbo.build/schema.json",
33
"globalDependencies": ["tsconfig.json", "tsup.config.ts"],
44
"globalEnv": ["OS_SKIP_DTS"],
5+
"globalPassThroughEnv": ["VITEST_MAX_WORKERS"],
56
"tasks": {
67
"build": {
78
"dependsOn": ["^build"],

0 commit comments

Comments
 (0)