Skip to content

Commit 13ec15f

Browse files
committed
feat(devx): shard @objectstack/cli below package granularity as file-level slice items
The partitioner's own pin 3 names this remedy: no split at any shard count can bin a package that exceeds 1.3x the mean, and @objectstack/cli measured 1231.52s against a 800.7s post-refresh mean. A shard item is now a package OR a k/n slice of one; the slice count is derived from that measurement rather than picked, and the balancing pins bin the sliced items so the pending dataset refresh lands instead of reding them. The generator gains the other half: turbo records a run's passthrough argv per task as `cliArguments`, so a package's slices are reassembled (summed) into one whole-package weight before the median rule sees it. Without that, the next refresh would record the heaviest suite at 1/n of its real cost. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Vbw3RPgdtqesx4azk9SbW8
1 parent d12940e commit 13ec15f

3 files changed

Lines changed: 745 additions & 26 deletions

File tree

scripts/check-test-completeness.mjs

Lines changed: 106 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -144,6 +144,12 @@ import path from 'node:path';
144144
import { fileURLToPath } from 'node:url';
145145
import process from 'node:process';
146146
import { isEntrypoint } from './invoked-as.mjs';
147+
// The shard-item grammar, from the script that WRITES the scheduled list. A
148+
// second reader here would be a second grammar, and the two would drift apart
149+
// silently -- an unparsed `@objectstack/cli 1/2` reaches describe() as a
150+
// package name the turbo ls document has never heard of, which this guard
151+
// (correctly, for its own contract) refuses the whole shard over.
152+
import { parseShardItem } from './partition-test-shards.mjs';
147153

148154
const REPO_ROOT = path.resolve(path.dirname(fileURLToPath(import.meta.url)), '..');
149155

@@ -298,14 +304,49 @@ export function parseFailedTestPackages(text) {
298304
return failed;
299305
}
300306

307+
/**
308+
* The scheduled list, folded to PACKAGE names.
309+
*
310+
* `shard-packages.txt` holds shard ITEMS, and since #16173 an item can be a
311+
* file-level slice of a package (`@objectstack/cli 1/2`). Q2 is asked per
312+
* package -- did this package report at all on this shard -- and the reported
313+
* set is keyed by the package name turbo prints, which carries no slice, so an
314+
* unfolded item would be a name the `turbo ls` document does not list and the
315+
* shard would be refused whole. Duplicates collapse for the same reason: a
316+
* package is complete when every slice of it scheduled HERE has reported, and
317+
* partition-test-shards.mjs refuses any split that puts two slices of one
318+
* package on one shard, so "every slice here" is "the one slice here".
319+
*/
320+
export function scheduledPackages(lines) {
321+
const out = [];
322+
const seen = new Set();
323+
for (const line of lines) {
324+
const { name } = parseShardItem(line);
325+
if (seen.has(name)) continue;
326+
seen.add(name);
327+
out.push(name);
328+
}
329+
return out;
330+
}
331+
301332
// true = every task turbo scheduled succeeded, so nothing was cancelled.
302333
// false = it stopped early. null = turbo never printed its roster (it died, or
303334
// the log was truncated) -- unknown, and treated as "not completed" by Rule B.
335+
//
336+
// EVERY roster line is folded, not just the last one. A shard's log holds one
337+
// roster per `turbo run` invocation, and since #16173 a shard that carries a
338+
// file-level slice runs TWO (the sliced package needs its own passthrough, which
339+
// turbo applies run-wide). Reading only the last would let a completed second
340+
// invocation vouch for a first one that stopped early -- and Rule B turns that
341+
// into a red on every package the abort left unreached, which is precisely the
342+
// false-red machine this file's header warns about.
304343
export function parseRunCompleted(text) {
305344
let completed = null;
306345
for (const line of text.split('\n')) {
307346
const m = line.match(TASKS);
308-
if (m) completed = Number(m[1]) === Number(m[2]);
347+
if (!m) continue;
348+
const thisRun = Number(m[1]) === Number(m[2]);
349+
completed = completed === null ? thisRun : completed && thisRun;
309350
}
310351
return completed;
311352
}
@@ -596,7 +637,7 @@ function reportVerdict(verdict) {
596637
// not red. A battery BELOW its floor means cases stopped running; the remedy is
597638
// to find what stopped registering.
598639
const SELF_TEST_BATTERIES = Object.freeze({
599-
'check-test-completeness self-test': 67,
640+
'check-test-completeness self-test': 79,
600641
});
601642

602643
// DELETING an entry silences that battery's floor exactly as effectively as
@@ -956,6 +997,63 @@ function selfTest({ quiet = false } = {}) {
956997
'exit codes: the refusal code collides with a verdict code',
957998
);
958999

1000+
// -- File-level slice items in the scheduled list (#16173) ----------------
1001+
//
1002+
// `shard-packages.txt` stopped being a list of package names the day one
1003+
// package started being sharded below package granularity. Both halves below
1004+
// fail in the false-RED direction if they regress, which is the direction
1005+
// this file's header spends its length warning about.
1006+
eq(scheduledPackages(['@objectstack/spec', '@objectstack/core']), ['@objectstack/spec', '@objectstack/core'],
1007+
'scheduled: plain package names were disturbed');
1008+
eq(scheduledPackages(['@objectstack/cli 1/2']), ['@objectstack/cli'],
1009+
'scheduled: a slice did not fold to its package name');
1010+
eq(scheduledPackages(['@objectstack/cli 1/2', '@objectstack/cli 2/2']), ['@objectstack/cli'],
1011+
'scheduled: two slices of one package did not collapse');
1012+
eq(scheduledPackages(['@objectstack/spec', '@objectstack/cli 2/2', '@objectstack/core']),
1013+
['@objectstack/spec', '@objectstack/cli', '@objectstack/core'],
1014+
'scheduled: folding reordered the list');
1015+
// END-TO-END: an unfolded item reaches describe() as a name the turbo ls
1016+
// document cannot resolve, and classifyShard refuses the whole shard over it.
1017+
// This asserts the fold is what stops that, not that describe() got lenient.
1018+
eq(
1019+
classifyShard({
1020+
scheduled: scheduledPackages(['@objectstack/cli 1/2']),
1021+
reported: new Set(['@objectstack/cli']),
1022+
failed: new Set(),
1023+
runCompleted: true,
1024+
describe: describe({ '@objectstack/cli': { hasTestScript: true, testFileCount: 268 } }),
1025+
}).silent,
1026+
[],
1027+
'scheduled: a slice that DID report was still graded silent',
1028+
);
1029+
eq(
1030+
threw(() =>
1031+
classifyShard({
1032+
scheduled: ['@objectstack/cli 1/2'],
1033+
reported: new Set(['@objectstack/cli']),
1034+
failed: new Set(),
1035+
runCompleted: true,
1036+
describe: describe({ '@objectstack/cli': { hasTestScript: true, testFileCount: 268 } }),
1037+
}),
1038+
),
1039+
true,
1040+
'scheduled: an UNFOLDED item was accepted, so the fold is not what makes this work',
1041+
);
1042+
1043+
// -- One roster per turbo invocation, and a shard can now run two (#16173) --
1044+
const roster = (ok, total) => `Tasks: ${ok} successful, ${total} total`;
1045+
eq(parseRunCompleted(roster(4, 4)), true, 'runCompleted: a complete single roster');
1046+
eq(parseRunCompleted(roster(3, 5)), false, 'runCompleted: an incomplete single roster');
1047+
eq(parseRunCompleted('no roster here'), null, 'runCompleted: a log with no roster is unknown, not complete');
1048+
eq(parseRunCompleted(`${roster(4, 4)}\n${roster(2, 2)}`), true,
1049+
'runCompleted: two complete rosters');
1050+
// The load-bearing one: the packages the FIRST invocation never reached would
1051+
// otherwise be charged as silent under Rule B.
1052+
eq(parseRunCompleted(`${roster(3, 9)}\n${roster(1, 1)}`), false,
1053+
'runCompleted: a completed second invocation vouched for a first that stopped early');
1054+
eq(parseRunCompleted(`${roster(9, 9)}\n${roster(0, 1)}`), false,
1055+
'runCompleted: a second invocation that stopped early was overlooked');
1056+
9591057
// ⚠️ The floor is scoped to the LOUD run on purpose. `selfTest({ quiet: true })`
9601058
// also runs on EVERY production invocation of this gate (see `main()`), and
9611059
// there nothing claims a self-test verdict — the floor exists to stop a green
@@ -1059,10 +1157,12 @@ function main() {
10591157
if (scheduledPath) {
10601158
let scheduled;
10611159
try {
1062-
scheduled = readFileSync(scheduledPath, 'utf8')
1063-
.split('\n')
1064-
.map((l) => l.trim())
1065-
.filter(Boolean);
1160+
scheduled = scheduledPackages(
1161+
readFileSync(scheduledPath, 'utf8')
1162+
.split('\n')
1163+
.map((l) => l.trim())
1164+
.filter(Boolean)
1165+
);
10661166
} catch (err) {
10671167
console.error(`check-test-completeness: cannot read ${scheduledPath} -- ${err.message}`);
10681168
process.exit(1);

0 commit comments

Comments
 (0)