test(scripts): smol-worker A/B gate harness (verdict: FAIL, flags not landed) - #2304
test(scripts): smol-worker A/B gate harness (verdict: FAIL, flags not landed)#2304lidge-jun wants to merge 1 commit into
Conversation
…s not landed Fresh-child-process isolation per run (in-process sequential runs contaminate baselines via allocator page retention — the first version measured a phantom win). Peak from Subprocess.resourceUsage().maxRSS over the audited workload shape (100MB row materialization + aggregate JSON). Result on Bun 1.4.0 darwin/arm64: elapsed within bound but median peak RSS NOT reduced (447.76MB vs 447.81MB) — the burst-allocation batch shape is dominated by live data, not heap growth policy. Per the audited pre-landing gate, the production Worker call sites keep full-size heaps; harness + devlog record are the deliverable. Unit: devlog/_plan/260822_260822-bun14-followup-memory/030
|
✅ Deterministic PR hygiene checks passed. |
github-actions
Bot
added
the
chore
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. 🗂️ Base branches to auto review (2)
Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Path: .coderabbit.yaml Review profile: ASSERTIVE Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Ingwannu
left a comment
Ingwannu
left a comment
There was a problem hiding this comment.
Reviewed exact head 9c7f42f8db8307615c3f65abd1f1c38be0a04287 against its stacked base and the 030 pre-landing gate.
Keeping all production workers on the full heap after a failed experiment is the conservative outcome. The evidence artifact still needs correction before it is mergeable:
-
The document records
2026-08-22, but the current date is August 21, 2026. Use the actual measurement date and avoid a future-dated result. -
payloadMbandrunsaccept zero, negative, non-numeric, and arbitrarily large values. This is a memory-stress script underscripts/, where inputs must be explicit and bounded. For example,runs=0reachesmedian([])and writes a structurally incomplete gate, while a huge payload can exhaust the host. Validate finite positive integer ranges before creating the child or output directory, and make median/report generation total for every accepted value. -
The controlling document says “for each worker (history, restore, policy): build a large fixture and run the job.” This harness runs one synthetic array-plus-JSON workload and does not invoke or model the distinct production history, restore, and policy jobs. That is useful preliminary screening, but it is not a per-worker gate and cannot support the current claim that all three audited workload shapes were measured. Either scope the report/doc honestly as a synthetic screening result, or add separate fixtures/cells that exercise each production worker boundary.
git diff --check also fails on scripts/smol-worker-ab.ts because of the extra blank line at EOF. Please fix that, add focused argument/report regressions, and rerun exact-head validation. The parent documentation PR must be cleaned and landed before this sibling is retargeted to dev.
리뷰 · 우선순위 42 / 80지금 하네스. 구멍. (1)
해결방안: FAIL 유지. 이 댓글은 grok-bot이 작성했습니다 |
…ockers closed Rebuilds the harness halves of #2303 and #2304 directly on dev, without the #2302 runtime commit those PRs were stacked on. Merging them as stacked would have dragged in the extraMemorySize: 0 fabrication that #2302 still carries, and would also have reverted unrelated coordinator work that landed on src/cli/doctor.ts after the stack was cut. scripts/bun-gc-relief-eval.ts Records rssBeforeLoad and derives postLoadGrowth and recoveryFraction. The controlling 260731 gate is "at least 50% of post-load RSS GROWTH is gone", and the previous shape could not express that: rssAfterLoad - rssPlus60s cannot separate recovery from ordinary drift, and the recorded verdict divided recovered bytes by total post-load RSS, which answers a different question than the gate asks. recoveryFraction is null when growth was not measurable, so a cell that proves nothing does not read as 0% recovery. A child-side gc-error now rejects the waiting cell instead of expiring into a ten-second "gc receipt timeout" that hides the real cause. scripts/macos-rss-retention-harness-child.ts The SIGUSR2 collector is installed only under OCX_GC_EVAL=1. It was gated by a comment saying the 7h retention protocol never sends that signal, which is a claim about one sender rather than a property of the process; a stray signal would have collected inside the measurement that protocol exists to take. scripts/smol-worker-ab.ts payloadMb and runs are validated as bounded integers. Previously runs=0 produced a report claiming completionSuccess over an empty result set with the median fields silently absent, and a negative payload ran a meaningless workload instead of refusing. Medians are computed only once both arms are complete, so a verdict can never be derived from a partial set. The header claimed to measure the audited shapes of history, restore and policy workers; it imports none of them. It now says what it is: a synthetic screening of the array-plus-JSON burst shape those workers share. The FAIL verdicts both harnesses recorded stand. No production Bun.gc(true) call and no smol: true flag is landed here. The GC harness needs a live upstream fixture to produce new numbers, so the recorded RSS cells are NOT regenerated by this commit and the 020 table still carries the old denominator. Re-running the cells and rewriting that table around recoveryFraction is deliberately left as the next measurement pass rather than claimed here.
|
|
2 participants
Summary
Verification
Checklist