| FR-041 | Public A/B drift benchmark — coding agents with vs without MetaObjects, pre-registered, friction-first | 📋 **design settled 2026-09-12, unbuilt** — the "proving the value" work; it is what licenses the claims FR-042 §4 withholds. Revised after an adversarial two-reviewer design review (spec §14): the scored task set is now held out from the friction pass (the draft tuned Arm B on the tasks it would later be scored on, with no symmetric loop for the control), Arm A is derived from Arm B's generated output so the seeds differ only in the model and the gate, `n` is calibrated from a pilot instead of asserted (one identical config measured 25/51/28 turns at temperature 0 — 41% CV), H2 is time-to-**correct** rather than time-to-done, the primary is analysed intention-to-treat so an arm cannot win by not finishing, and escaped defects are reported split by whether `meta verify` already covers the invariant class (it covers four of five, so H1 is partly definitional for those). First deliverable is the Phase 0b friction log. Design: `docs/superpowers/specs/2026-09-11-fr-041-drift-ab-benchmark-design.md` | 1.x | — |
0 commit comments