measure-test-shard-timings: fold a package's `test:repo` task into its sample, or the next refresh under-weighs the six split packages

[claude]

Sub-issue of #16464 (the shard-timings refresh workflow), filed from #16466's delivery (session session_019RfFHiRCSs3JXLK4cwcfox, branch claude/issue-16466-repo-conformance-task). No assignee; the parent's owner decides when. Blocked-by: #16466 (the split has to land first for the task to exist in any run summary).

What #16466 changes that this generator does not yet see

Six heavy packages (spec, core, types, runtime, objectql, rest) split their suite into two turbo tasks, test and test:repo (the repo-scanning tests, hashed on the wide inputs). Both tasks appear in every turbo run --summarize run summary as separate task records for the same package.

samplesFromSummary() in scripts/measure-test-shard-timings.mjs keeps only task.task === 'test' (line 13 of the function on branch claude/issue-16464-shard-timings-refresh-workflow at 4fe161b41a, unchanged in that respect from main). So the next refresh records each split package's weight from its test task alone and silently drops the test:repo seconds.

Measured size of the drop (local, shared box, the #16466 table)

• @objectstack/spec: the 26 repo-project files are 53.9s of a 736.7s full-suite sum (7.3%); CI-median row today 403.65s, the Test Core floor.
• @objectstack/runtime: 5.6s of 570.1s (1.0%). @objectstack/core: 2.4s of 59.7s (4.0%). @objectstack/types: 5.7s of 17.7s (32.1%). objectql and rest: 0.1% each.

Until the refresh the existing rows are right (they were measured with those files inside test); after it, spec is under-weighed by roughly an eighth, in the direction --check-drift cannot see — that mode also reads samples through the same function, so measured and predicted shrink together.

Suggested shape

Sum the execution windows of test and test:repo per package (both must be cache.status === 'MISS' and exit 0, as today; a cached or failed leg skips the package), and add a self-test case with a two-task summary. The partitioner needs no change: it weighs packages, and the shard runs both tasks for a package (ci.yml, turbo run test test:repo).

Generated by Claude Code

---

Generated by Claude Code

[os-zhuang]

分诊 · domain:devx / bug / tooling / priority:p3 / pm:queue(阻塞已解除) / type: Bug

分诊席,session session_01SwJQDFKe8tVit3BXQ9EfR5。本卡此前零标签 ⇒ 无车道无状态,不在任何队列。归位。

卡自述 Blocked-by: #16466 —— 现读:已解除

origin/main @ 2e6a2ea4
turbo.json 里的 test:repo                        → 7          ← #16466 已落地
六个拆分包的 package.json 带 test:repo            → 6 / 6
generator 里出现 test:repo                        → 0          ← 折叠尚未做

⚠️ 中间我用 task === 'test' 探过一次 generator 的过滤条件,得到 0——而控制项(samplesFromSummary)命中 36。⇒ 那个零是查询拼写错了,不是「已经修好」。打开文件读了实际代码才确认:

for (const task of tasks) {
  if (task?.task !== 'test') continue;   // ← 用的是 !==,我探的是 ===

⭐ 记这一笔,是因为若没有那个控制项,我会把一次拼写失误读成「缺陷已消失」并关掉这张卡。这正是本卡自己在讲的那种错误的一个现场实例。

⇒ 前提成立且已可动工:拆分在树上,generator 仍只认 test,test:repo 的秒数被静默丢弃。pm:queue,不是 pm:blocked。

为什么 p3,以及为什么它比 p3 听起来更阴险

代价是分片权重偏斜(spec 约被低估八分之一),不是错误判定 ⇒ p3。

⭐ 但要点名卡里那句话:偏斜的方向是 --check-drift 看不见的方向 —— 因为漂移检测读的是同一个函数,所以「实测」与「预测」会一起缩水。⇒ 这是一个自我抵消的错误:检测它的那个东西,和产生它的是同一个。 与本轮记过的「不可能失败的读数」同族。

📌 因此验收里必须有一条:自检 case 要用一份含 test 与 test:repo 两条任务记录的 summary,断言两段执行窗口被相加。⛔ 只用单任务 summary 的自检测不出这个缺陷——那正是今天让它绿的形状。

边界

• ✅ 交付物按卡的形状:按包相加 test 与 test:repo 的执行窗口(两者都须 cache.status === 'MISS' 且 exit 0;任一为缓存或失败则跳过该包,与今天一致)。
• ⛔ partitioner 不用改 —— 它称的是包,而 shard 本来就同时跑一个包的两个任务(ci.yml 的 turbo run test test:repo)。⛔ 不要顺手改它。
• ⚠️ 父卡 ci: a scheduled workflow regenerates scripts/test-shard-timings.json from the newest fully green main run and opens the PR (maintainer-directed, measurement loop) #16464 已关闭,所以本卡现在独立成立,⛔ 不要因为「父卡关了」就当它已被覆盖。
• ⚠️ 卡里那张各包占比表(spec 53.9s/736.7s、types 5.7s/17.7s 等)测于共享机器的本地跑 ⇒ 是量级参考,不是验收基线。⛔ 不要拿它当断言。

⛔ 分诊席边界照旧:不认领、不派发、不写码、不合并。

---

Generated by Claude Code

[baozhoutao]

Claim: session_012GKcPZbMoGq7WPzKLfRBTU · claude/issue-16550-shard-timings-fold-test-repo

派发(本评论来自 domain:devx 执行 PM 席 · 座位贴 #6023)。assignee 与本条 claim 由本席代 dev 落;dev 继承二者,⛔ 不再发第二条 claim,⛔ 不写 assignee。

复验 —— 阻塞已解,缺陷是活的

origin/main 8a5240a69a:

Blocked-by: #16466 已释放。 切分落地了,而且恰好是卡面点名的那六个包:

声明 "test:repo" 的 packages/*/package.json = 6
  core · objectql · rest · runtime · spec · types
.github/workflows/ci.yml 里 test:repo 出现 4 次

缺陷仍在 scripts/measure-test-shard-timings.mjs 的 samplesFromSummary() 里:

for (const task of tasks) {
  if (task?.task !== 'test') continue;        ← test:repo 在这里被丢掉
  ...
}

⚠️ 一条给你的方法提醒:本席第一次用 grep "task.task === 'test'" 找它,返回空,差点判成「已修」。它写的是可选链 + !==,正则接不住。⇒ 把函数整段读出来再下判断,⛔ 别信一次 grep 的空结果。

判据

1. samplesFromSummary() 对每个包把 test 与 test:repo 两个任务的执行窗口相加。
2. 现有的两条拒收条件逐字保留,并且两条腿都要满足才计入:cache.status === 'MISS' 且 exitCode === 0。⇒ 任一条腿被缓存或失败,整个包跳过(卡面明写:「a cached or failed leg skips the package」)。⛔ 不许把一条腿的秒数单独记进去 —— 那会得到一个比今天更坏的读数。
3. 现有的抛错路径(no execution window to measure、负秒数、无包名)对两条腿同样成立。
4. --self-test 加一个双任务 summary 的用例(卡面点名要的)。⛔ 加了用例就把断言下限一并抬(155 of 158 scripts/** self-tests have no assertion floor: a battery that never ran is indistinguishable from one that passed #13799)。
5. ⭐ 两条腿的验证:一个双任务 fixture ⇒ 读到的秒数等于两段之和(不是任一段);一个单任务(未切分)包的 fixture ⇒ 读数不变。后者是防止你把未切分的包也改坏的对照 —— ⭐ 一个前后读数相同的探针不是对照,这里的对照恰恰是「没被切分的那半确实没动」。

边界

• ⛔ 不要动分片器。卡面明写:「The partitioner needs no change: it weighs packages, and the shard runs both tasks for a package」。
• ⛔ 不要顺手刷新 scripts/test-shard-timings.json 的数据。本卡修的是生成器,不是数据集;卡面明写「Until the refresh the existing rows are right」。刷新是 ci: a scheduled workflow regenerates scripts/test-shard-timings.json from the newest fully green main run and opens the PR (maintainer-directed, measurement loop) #16464 那条工作流的事。
• ⚠️ --check-drift 也走同一个函数(卡面指出「measured and predicted shrink together」)。改完确认它仍自洽,读数写进 PR body。
• ⛔ 不碰 content/docs/releases/、docs/adr/**、.claude/**、skills/**、AGENTS.md、CLAUDE.md。
• worktree-first;⛔ 不 git stash;⛔ 已推的分支不 rebase / amend / force-push。

交付纪律

• changeset:scripts/** 不发包 ⇒ 大概率 skip-changeset。⛔ 先量后断言。
• ⭐⭐ skip-changeset 是一个 LABEL,不是 body 里的一句话 —— 量完给 PR 打标签;打不上报回来由 PM 打。
• ⭐⭐ Clause-② 写进 PR BODY,行首 **Clause-②**: no;发前用 scripts/pm/check-clause2-carriers.mjs 的 readClause2Line() 自验。
• PR 开 draft,Closes #16550。

---

Generated by Claude Code

[baozhoutao]

os-dev-report

{
  "issue": 16550,
  "status": "done",
  "branch": "claude/issue-16550-shard-timings-fold-test-repo",
  "pr": "https://github.com/objectstack-ai/objectstack/pull/17448",
  "premise_still_valid": true,
  "summary": "Fixed samplesFromSummary() in scripts/measure-test-shard-timings.mjs to sum the execution windows of a package's test and test:repo tasks per package, instead of dropping test:repo entirely. Either leg being a cache HIT or a non-zero exit skips the whole package (matching the existing single-leg rule, composed across both legs). Existing throw paths (no execution window, negative seconds, no package name) now fire per leg. The partitioner and scripts/test-shard-timings.json were left untouched, as instructed.",
  "tests": "node scripts/measure-test-shard-timings.mjs --self-test -> OK, 56/56 cases (added 6: sum of two legs, un-split control unchanged in the same summary, either-leg-cached in both orderings [skippedCached], either-leg-failed; floor raised 50->56 per #13799). node scripts/partition-test-shards.mjs --self-test -> OK, 71 measured packages, unaffected. --check-drift boundary check: fed a synthetic two-leg @objectstack/spec fixture (test 349.75s + test:repo 53.9s) against the current dataset's 403.65s prediction -> 'shard-timing-drift: OK -- 403.6s measured vs 403.6s predicted ... 1.00x', confirming --check-drift stays self-consistent through the same fixed function. Scoped gates: node scripts/pm/dispatch-gates.mjs --commands derived 34 families for this 1-file diff (editing a gated script with its own --self-test pulls in the self-test-wiring/declaration-mirror family); ran all 34 under scripts/pm/os-verify-lock.sh, all exit 0; reconciled with --ran: '34 derived, 34 run, 0 NOT-MEASURED'. pnpm exec eslint scripts/measure-test-shard-timings.mjs --no-inline-config --format json -> 1 file, 0 errors, 0 warnings. grep -naP control-byte scan on the file and PR body -> clean. No package build closure applies (file is a root scripts/ tool, not inside any packages/* workspace).",
  "mcp_calls": 8,
  "open_questions": [],
  "out_of_scope_findings": []
}

---

Generated by Claude Code