measure-test-shard-timings: fold a package's test:repo leg into its sample - #17448
Merged
Merged
Conversation
…ples samplesFromSummary() kept only task.task === 'test', so the six packages split into `test` + `test:repo` by #16466 had their repo-scan seconds silently dropped -- under-weighing spec by roughly an eighth on the next refresh, in the direction --check-drift cannot see since it reads samples through the same function. Fold both legs per package by summing their execution windows; either leg being a cache HIT or a failure skips the whole package, same as today's single-leg rule. Self-test gains a two-task-summary battery (sum, either-leg- cached, either-leg-failed, and an un-split control proving the fold does not disturb ordinary packages), and its floor is raised 50 -> 56 accordingly. Fixes #16550 Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
baozhoutao
marked this pull request as ready for review
September 10, 2026 14:57
baozhoutao
enabled auto-merge
September 10, 2026 14:57
baozhoutao
deleted the
claude/issue-16550-shard-timings-fold-test-repo
branch
September 10, 2026 15:53
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #16550
What changed
samplesFromSummary()inscripts/measure-test-shard-timings.mjskept onlytask.task === 'test', so the six packages#16466split intotest+test:repo(core, objectql, rest, runtime, spec, types) had theirtest:repoexecution window silently dropped from every future refresh.samplesFromSummary()now folds a package'stestandtest:repolegs bysumming their execution windows into one whole-package reading. The
rejection rule composes across the fold rather than being re-derived per
leg: every leg present for a package must be
cache.status === 'MISS'and
exitCode === 0, or the whole package is skipped — one cached orfailed leg is exactly as disqualifying as it was for a lone
testtaskbefore. The existing throw paths (no execution window, negative seconds, no
package name) apply per leg, so they still fire for either task name.
The partitioner (
scripts/partition-test-shards.mjs) is untouched — it onlyweighs packages, and both
--check-driftand the shard's ownturbo run test test:repoalready run/read both tasks for a package throughthis same function.
--check-driftself-consistency (boundary check)Fed a synthetic two-leg summary for
@objectstack/spec(
test349.75s +test:repo53.9s) against the current, unrefreshedscripts/test-shard-timings.json(weight403.65, measured before thesplit when both halves still lived inside one
testtask):Measured and predicted agree (1.00x) because
--check-driftreads legsthrough the same now-fixed
samplesFromSummary()— before this change, onlythe 349.75s
testleg would have been read against the 403.65s prediction,a 0.87x undershoot invisible to nobody watching for it today but exactly the
"measured and predicted shrink together" blind spot the card names once the
dataset is refreshed and predicted also starts under-counting.
Dataset
scripts/test-shard-timings.jsonis not touched — this PR fixes thegenerator only; "until the refresh the existing rows are right" per the
card. Refreshing is
#16464's scheduled workflow's job.Tests
node scripts/measure-test-shard-timings.mjs --self-test— OK. Added atwo-task-summary battery: the sum case, an un-split control riding in the
same summary (proves the fold does not disturb ordinary packages —
the discriminating leg, since a before/after-identical reading on that
control alone would not catch a fold that leaked), either-leg-cached (both
orderings), and either-leg-failed. Floor raised
50 -> 56for the 6 newcases (155 of 158 scripts/** self-tests have no assertion floor: a battery that never ran is indistinguishable from one that passed #13799).
node scripts/partition-test-shards.mjs --self-test— OK (71 measuredpackages, unaffected — it imports
samplesFromSummarybut its own fixturesare single-task).
node scripts/pm/dispatch-gates.mjs --commandsnamed 34 families for this diff (editing a gated tooling script with its
own
--self-testpulls in the self-test-wiring/declaration-mirror family).All 34 ran green under
scripts/pm/os-verify-lock.sh; reconciled with--ran— 34 derived, 34 run, 0 NOT-MEASURED.pnpm exec eslint scripts/measure-test-shard-timings.mjs --no-inline-config --format json— 1 file checked, 0 errors, 0 warnings.grep -naP '[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]' scripts/measure-test-shard-timings.mjs— no control bytes.
Changeset
skip-changeset— measured, not assumed:scripts/measure-test-shard-timings.mjslives at the repo root, outside every package directory, and every
publishing package's
"files"array is limited todist/README.md/CHANGELOG.md(checked acrosspackages/*/package.json). No packagebundles or re-exports this script (
grep -rl measure-test-shard-timingsoutside
scripts/returns nothing). Nothing published moves. Applying theskip-changesetlabel on this PR now.Clause-②: no
🤖 Generated with Claude Code
https://claude.ai/code/session_012GKcPZbMoGq7WPzKLfRBTU
Generated by Claude Code