fix(eval): M3 selectivity gate is exactly <= 1/3 (#209 amendment) - #215
Conversation
The M3 protocol's "median candidate fraction <= 0.33" was M1's otel-fresh value rounded to two decimals. The evaluator reproduces that value as exactly 1/3. Read literally, the bound fails its own reference and draws an arbitrary line (0.331 passes, 0.334 fails). The protocol was amended on #209 before any M3 capture exists: <= 1/3, in exact arithmetic. The gate now computes the median over Fraction(|localization|, |services|), so it is never decided by float rounding. For example, the median of 1/4 and 5/12 is exactly 1/3. The float median is still reported, and the exact ratio is added alongside it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
/review |
There was a problem hiding this comment.
REQUEST CHANGES
The comparison in decide is exact, and the new unit tests actually pin the float-rounding case. The decision record those tests never look at does not contain the value that was compared. CI is green. This is not a failing-test problem.
1. The criterion record is not the comparison the gate ran
BLOCKING.
decide computes Fraction median, compares that to Fraction(1, 3), then writes a different object into generalization_criteria: float(median) and the string "1/3". render_markdown — the as-is post for #209 — runs every criterion value through _num, i.e. two decimal places, and never reads median_candidate_fraction_exact.
So the three medians this PR exists to separate all print as the same line:
1/3passes, posts0.33 (pre-registered <= 1/3)331/1000passes, posts0.33 (pre-registered <= 1/3)334/1000(=167/500) fails, posts0.33 (pre-registered <= 1/3)
The table cell is the same 0.33. The exact ratio is on the arm dict inside the JSON and then dropped on the floor by the publisher. Before this patch the criteria triple was the pair of operands the branch used. It is not anymore: 0.334 <= "1/3" does not even type-check, and the Fraction that did get compared is not stored.
ಠ_ಠ You refused to gate on the float, then stored the float as the official criterion, so _num could keep rounding it back to the literal the amendment just retired.
Record the exact ratio in the criterion value and print that ratio in the verdict line and the selectivity row. The float can stay beside it. Add a render test for 1/3, 331/1000, and 334/1000 that asserts the posted lines are not identical.
VERDICT
The gate boolean matches the #209 amendment. The audit record does not. This is an abstraction failure: two representations of one threshold, and the one the protocol posts is the rounded one. Do not merge until the posted criterion is the ratio decide compared.
Sent by Cursor Automation: Code Reviewer
…pared (#215 review) decide() compared the exact Fraction median to Fraction(1, 3), but it recorded float(median). render_markdown then printed that float to two decimals. So 1/3 (pass), 331/1000 (pass) and 167/500 (fail) all posted the same "0.33", which is the literal the amendment retired. The criterion triple now records the operands the gate compared: the exact ratio "n/d" against "1/3". The verdict line and the selectivity row print that ratio, with its decimal beside it (e.g. "167/500 (≈0.3340)"). A render test asserts that the three posted lines differ. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
/review |


This encodes the M3 protocol amendment recorded on #209 before any capture exists (amendment). The selectivity bar is now median candidate fraction ≤ 1/3. It was "≤ 0.33".
Why
Change
CANDIDATE_FRACTION_MAX = Fraction(1, 3).Fraction(|localization|, |services|), so a result of exactly 1/3 can never fail on float rounding. For example, the median of 1/4 and 5/12 is exactly 1/3.median_candidate_fraction_exact, e.g."11/30". The criterion prints<= 1/3.Nothing else in the protocol or the evaluator changes. After capture, no criterion changes.
Tests
The test that pinned "exact 1/3 fails 0.33" is replaced by four tests:
make lint✅ ·make test-unit✅ (1587). A smoke run on trace-loc serializes and renders correctly.Eval delta
None on the product. This touches
src/eval/only;src/core/is unchanged. On the spent otel-fresh corpus the frozen arm's fraction is 0.139, so the gate verdict is unchanged there. The only behaviour change is a median landing in (0.33, 1/3].Refs #209.
🤖 Generated with Claude Code