Skip to content

Commit d958b34

Browse files
claude[bot]claude
andauthored
ci(test): a file-level slice builds its dependency closure in a passthrough-free run, so the sharded leg stops rebuilding it (#16868)
* wip(ci): slice leg builds its closure in a passthrough-free run * wip(ci): slice closure builds in its own guarded step --------- Co-authored-by: Claude <noreply@anthropic.com>
1 parent 25b0789 commit d958b34

1 file changed

Lines changed: 73 additions & 1 deletion

File tree

.github/workflows/ci.yml

Lines changed: 73 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -464,6 +464,70 @@ jobs:
464464
echo 'Items on this shard (a package name, or a package plus a k/n file-level slice):'
465465
cat "$RUNNER_TEMP/shard-packages.txt"
466466
467+
# ⛔ A FILE-LEVEL SLICE BUILDS ITS DEPENDENCY CLOSURE HERE, IN A RUN THAT
468+
# CARRIES NO PASSTHROUGH, so that the sharded run in the next step can be
469+
# `--only` (#16395).
470+
#
471+
# Turbo folds a run-level passthrough into the hash of EVERY task in the
472+
# run, not only the task that receives it -- and `-- "--shard=k/n"` is the
473+
# whole reason a slice gets its own invocation at all (the next step's
474+
# comment says why it cannot ride the shared run). Measured on turbo
475+
# 2.10.10, `--filter=@objectstack/cli`, `turbo run test ... --dry=json`
476+
# (60 tasks: 59 `build` + 1 `test`):
477+
#
478+
# plain vs plain 60 identical, 0 changed <- control, fires
479+
# plain vs -- --shard=1/2 0 identical, 60 changed
480+
# --shard=1/2 vs 2/2 0 identical, 60 changed
481+
#
482+
# So the sliced leg could hit NEITHER the main-seeded Turbo cache restored
483+
# above NOR the builds the shared leg ran seconds earlier in the SAME job:
484+
# it re-executed the closure every run, and shards 5/6 and 6/6 -- the two
485+
# that carry a slice -- paid that closure twice per job. Live reading,
486+
# `Test Core (5/6)` of run 34193080219 (a `packages/spec` PR, so the
487+
# affected set reaches cli): the slice leg reported
488+
# `Cached: 2 cached, 58 total` / `Time: 9m18.941s` while
489+
# `@objectstack/cli:test` itself measured `Duration 187.49s` -- six of those
490+
# nine minutes were the duplicate rebuild, on a job that was then standing
491+
# against a 30-minute wall (#16395's measurement).
492+
#
493+
# `turbo run build --filter=$PKG` is that closure and nothing more:
494+
# measured 59 build tasks, all 59 hash-IDENTICAL to the ones in the
495+
# passthrough-free test plan (0 differing, 0 extra, and 0 missing against
496+
# the test's own `^build` closure), so they REPLAY rather than re-execute.
497+
# `--filter=...^$PKG` was measured too and schedules 12 packages this
498+
# closure does not need. Locally, back-to-back invocations of this exact
499+
# command measured `57 cached, 57 total` / `Time: 153ms >>> FULL TURBO` on
500+
# the second, against `5 cached, 57 total` / `3m45.918s` on the first.
501+
#
502+
# ⚠ THIS IS ITS OWN STEP, not a second guarded run inside the step below,
503+
# because a guarded SITE is the triple (file, job, step) --
504+
# `measure-stall-guard-headroom` REFUSES to report a verdict when two
505+
# guarded runs share one, and refusing is right: the two would be judged
506+
# against the worst reading of their union. `pnpm check:stall-guard-budget`
507+
# and `pnpm check:stall-guard-headroom` both read this step, so it keeps
508+
# its own `--stall-minutes` and its own headroom row.
509+
#
510+
# A shard with no slice runs zero iterations here; every shard still
511+
# reaches the step, so its name is a stable site for those two gates.
512+
- name: Build the sliced package's dependency closure
513+
env:
514+
NODE_OPTIONS: --report-on-signal --report-signal=SIGUSR2 --report-directory=${{ runner.temp }}/stall-reports
515+
run: |
516+
if [ ! -s "$RUNNER_TEMP/shard-packages.txt" ]; then
517+
echo "No packages on this shard — nothing to build."
518+
exit 0
519+
fi
520+
mkdir -p "$RUNNER_TEMP/stall-reports"
521+
while read -r PKG SLICE; do
522+
[ -n "$PKG" ] || continue
523+
[ -n "$SLICE" ] || continue
524+
echo "Slice $PKG=$SLICE — building its dependency closure with no passthrough."
525+
LOG="$RUNNER_TEMP/test-core-slice-build-$(printf '%s' "$PKG" | tr -c 'A-Za-z0-9' '-').log"
526+
node scripts/run-with-stall-guard.mjs --log "$LOG" --stall-minutes 10 \
527+
--report-dir "$RUNNER_TEMP/stall-reports" -- \
528+
pnpm turbo run build "--filter=$PKG" --concurrency=4 --log-order=stream
529+
done < "$RUNNER_TEMP/shard-packages.txt"
530+
467531
# --concurrency=4: turbo's default (10) oversubscribes the 4-vCPU
468532
# hosted runner; matching the core count bounds peak memory and the
469533
# job is CPU-bound anyway.
@@ -571,7 +635,15 @@ jobs:
571635
PKG="${LEG%%=*}"
572636
SLICE="${LEG#*=}"
573637
LOG="$RUNNER_TEMP/test-core-slice-$(printf '%s' "$PKG" | tr -c 'A-Za-z0-9' '-').log"
574-
set -- pnpm turbo run test "--filter=$PKG" --concurrency=4 --summarize --log-order=stream -- "--shard=$SLICE"
638+
# `--only` (#16395): the step above already built this slice's
639+
# dependency closure in a passthrough-free run, so this run must
640+
# schedule the ONE task the passthrough is for. Without it turbo
641+
# re-hashes the whole `^build` closure under `--shard=k/n` and
642+
# rebuilds it -- that comment carries the measurement. ⚠ The build
643+
# step is load-bearing for this flag: a sliced package whose build
644+
# never ran fails LOUDLY here (its imports resolve to a missing
645+
# dist), never as a silent green.
646+
set -- pnpm turbo run test "--filter=$PKG" --only --concurrency=4 --summarize --log-order=stream -- "--shard=$SLICE"
575647
fi
576648
LOGS="$LOGS $LOG"
577649
node scripts/run-with-stall-guard.mjs --log "$LOG" --stall-minutes 10 \

0 commit comments

Comments
 (0)