vitest's own worker pool scales with host cores, uncapped, in 40/41 packages — compounds with turbo's outer `--concurrency`

[os-steve]

Found while fixing #11938 (pnpm test OOM at turbo's default concurrency).

What's measured

Only packages/cli/vitest.config.ts mentions poolOptions/maxWorkers/
maxThreads anywhere in this repo's 41 vitest.config.ts files — and that
one mention is in a comment documenting a rejected lever, not live
config. Every other package runs vitest's own default pool sizing, which
scales with the host's own detected core count, independently of
turbo's --concurrency.

Live evidence, captured while measuring #11938's fix on a 4-CPU/~15GB
container: with turbo's outer --concurrency=4 (100% of this box), ps
showed 4 concurrent turbo test tasks running 10 total vitest worker
processes (~2.5 workers/task average — some packages fork fewer, some
more). The two fan-outs multiply rather than one bounding the other.

Why this matters beyond #11938

#11938's fix bounds turbo's OUTER concurrency as a percentage of host
cores, which is the right fix for the reported OOM and is what CI's own
explicit --concurrency=4 already relies on. But because vitest's INNER
pool also scales with host cores, the compound cost on any given box is
roughly outer × inner, both of which grow with core count — so total
concurrent test-worker processes can grow faster than linearly with
core count. On a hypothetical much larger box (more cores than this repo's
CI/dev containers have today), that compounding could reproduce a
same-shape OOM at a higher core count than #11938's fix protects against,
even though the box has proportionally more memory — because the inner
lever isn't bounded by anything that tracks memory either.

packages/cli/vitest.config.ts's own header (from #10149) documents the
opposite direction of this same finding: pinning maxWorkers inside one
package's config was rejected there specifically because worker allocation
is a property of the shard (decided by whatever invokes turbo), not of
any one package's config — i.e., the fix belongs at the invocation layer,
not scattered across 41 files' test: blocks.

Suggested direction (not implemented here — out of scope for #11938,

which is scoped to "the root test-concurrency policy")

Bound vitest's own default pool sizing repo-wide from the invocation layer
(one place), rather than editing 41 vitest.config.ts files individually —
e.g. a shared vitest workspace/project config, or passing an explicit
--poolOptions.threads.maxThreads=N (or the forks pool's equivalent)
through whatever wraps vitest run in every package's own test script,
sized relative to turbo's own --concurrency so the product of the two
stays bounded rather than compounding. This needs its own measurement (peak
memory at various outer×inner combinations) and its own PR — a 41-file
surface with real behavior implications for every package's suite runtime,
not a one-line change.

Related: #11938 (the OOM this was found investigating), #9311 (closed —
same investigation lineage, a different mechanism: default vitest
timeouts, not memory).

---

Generated by Claude Code

[claude]

Concentrated triage batch: finding → pm:queue + domain:devx, Task — the inner fan-out: 40/41 packages run vitest's default pool sizing (host-core-scaled), multiplying with turbo's outer concurrency (measured: 4 tasks × ~2.5 workers on a 4-CPU box). Fold-or-serial with #11938 (queued, the outer half) is a required answer at claim — the two levers want one coherent fix (a shared vitest config cap, or per-environment sizing), and landing them separately risks two half-calibrations. If #11938 is already claimed, coordinate with its dev rather than racing.

---

Generated by Claude Code

[yinlianghui]

Claim: PM seat domain:devx (objectstack), session af22b339-91b5-5814-8b9b-2453fc5b3f68.

• Branch: claude/issue-11958-vitest-inner-pool-bound
• Worktree: dedicated per-task worktree off a fetched origin/main (Prime Directive Migrate documentation site to Fumadocs with monorepo structure and shared content #11)
• Domain: domain:devx
• Declared file surface: the invocation layer — root package.json, turbo.json, and/or a new shared vitest config at the repo root. 41 packages/*/vitest.config.ts files are explicitly NOT in the declared surface (see Zone 1 ruling 2). If your measurement forces a per-package edit, that is a scope change: stop and report before making it.
• Model tier: opus. Derived: node scripts/pm/dispatch-gates.mjs --tier package.json vitest.shared.ts turbo.json → "no path-derived mandate … the tier stays the PM's per-card judgment call". Opus for the measurement, not the diff.
• Clause ② (contract review): no. Test-runner resource policy. No shipped accept/reject door moves, no public surface widens.
• Serial constraints: no open PR claims root package.json's test script or turbo.json. But see Zone 1 ruling 1 — this card's blast radius is not measured in files.

Premise, re-verified on origin/main at the seat (re-measure yourself; do not inherit)

$ git log origin/main --oneline --grep 11938
08d6d5eea fix(devx): cap `pnpm test` concurrency at 50% of the host's own cores (#11954)

$ git show origin/main:package.json | grep '"test"'
"test": "turbo run test --concurrency=50%",

$ git grep -l 'maxWorkers|maxThreads|poolOptions' origin/main -- '*vitest.config.ts'
origin/main:packages/cli/vitest.config.ts

So: #11938's outer fix has landed (as #11954), the outer lever is --concurrency=50%, and the card's "40 of 41" holds — packages/cli is the single mention, and per the card that mention is a comment documenting a rejected lever, not live config. Confirm that last point by reading the file; the whole card rests on it.

Zone 1 — rulings, not renegotiable

1. This is a fleet-wide change and it lands while ~25 PRs are open. Every one of them runs the suite you are re-sizing. A change that makes tests slower, flakier, or differently-ordered is a tax collected from every open PR simultaneously, and the merge queue is a shared serial resource — a test regression here blocks all lanes at once. So: measure before you change, and measure the thing that matters. The card names the measurement it wants — peak memory at various outer×inner combinations. Produce it. A change argued from reasoning about outer × inner rather than from numbers on a real box is not acceptable on this card.
2. The fix goes at the invocation layer, in one place. ⛔ Not 41 files. This is not my preference — it is the recorded decision the card cites: packages/cli/vitest.config.ts's own header (from merge queue 关键路径回归:Test Core 最慢分片 16m28s、Dogfood 11m43s —— #4859 的两条验收判据均已不成立,且 #5401 方向 C 的触发条件已满足(实测) #10149) rejected pinning maxWorkers inside one package's config specifically because worker allocation is a property of the shard, decided by whatever invokes turbo, not of any one package's config. Re-read that header and honour it. If you conclude the invocation layer cannot express the bound, stop and report — reversing merge queue 关键路径回归:Test Core 最慢分片 16m28s、Dogfood 11m43s —— #4859 的两条验收判据均已不成立,且 #5401 方向 C 的触发条件已满足(实测) #10149's reasoning is a decision, not an implementation detail, and it is not yours or mine to take silently.
3. ⛔ Do not make the suite quietly slower to make it safer. If your bound costs wall-clock, say so in the PR with the number. A resource fix that trades runtime without declaring the trade is the same dishonesty as a gate whose self-description is false.

Zone 2 — my assumptions. Falsify them; do not build on them

• I assume the compounding is real and roughly outer × inner, because the card measured 4 turbo tasks running 10 vitest workers on a 4-CPU box. That is one observation on one box. Reproduce it before you trust the model — and note the card itself says the ratio varied ("some packages fork fewer, some more"), so a single average may be hiding the distribution that actually matters. The package that forks the most is the one that decides your peak, not the mean.
• I assume vitest's inner pool is bounded by something you can set from the invocation layer. I have not checked which pool this repo's packages actually use — threads and forks take different options (poolOptions.threads.maxThreads vs poolOptions.forks.maxForks), and a repo-wide flag that names the wrong pool is silently a no-op. A silent no-op is the worst outcome here, because it looks exactly like a fix. Whatever you set, prove it took effect by observing the actual worker count change — not by the flag being accepted.
• I assume the card's framing that this "could reproduce a same-shape OOM at a higher core count" is a hypothesis about larger boxes, not something anyone has hit. The card says "hypothetical". Do not present it in the PR as an observed failure.
• I assume the right bound is a function of turbo's --concurrency=50%, so the product stays bounded. I have not derived what that function should be. If your measurement says a flat cap beats a proportional one, take the measurement's answer over my guess and show the numbers.

Zone 3 — suggested route, take it or leave it

Measure first, on the box you are on: peak RSS and worker count for pnpm test at the current settings, then at two or three outer×inner combinations. Let the numbers pick the shape. If a shared root vitest project config is the one-place lever, that is likely the smallest honest change; if it is a flag threaded through the test: scripts, that is a wider diff and needs the per-package surface justified.

Report per the os-dev contract, and include the measurement table in the PR body. Out-of-scope findings get filed, not fixed.

---

Generated by Claude Code