You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
docs(receipt): Stage 2 council dispositions, full-population census transitions
The single seat returned REJECT with four MAJORs; every finding was checked
at source. Two were real planner defects (fixed in c8c105c); the other
four were evidence-class findings, answered here with artefacts rather
than prose: the golden gate is marked NOT MET AS WRITTEN and amended to
projected equality checked between named commits; the census attribution
now rests on all 5,434 questions through both arms (identical 1,844 /
crash 2,349 / whole-question-phrase 1,227 / different 14; zero
regressions), with the mechanism read from the frozen escape function
itself (1,217 of 1,227) and the instrument difference from run_census
stated; the gold parity and rescued-crasher claims are counted per item;
the studyloop suite attribution rests on a matched control at 80ee57b in
the same sandbox (299 of 305 failing ids in common, identical exception
profiles). The four census texts affected by the F2 fix keep their
recorded outcomes, so the Stage 2 receipts stand exactly.
| CLI ≡ MCP, ordered session lists | 91/91 |**91/91** — counted directly over the per-item `ranked` arrays (`stage2-census-transitions.json` → `gold_counters`), not inferred from equal aggregates|
28
28
| K-stratum paired delta vs frozen | not negative | K **0.182 → 0.303**; P 0.034 → 0.034; R 0.103 → 0.138 |
29
-
| planner golden | unchanged |**cases byte-identical**; row shape extended additively — see deviation|
29
+
| planner golden | unchanged |**NOT MET AS WRITTEN — gate amended** (see below): equal after projecting `message_id` away, checked between `80ee57bb` and `d060d3f2` by `scripts/eval/stage2_council_evidence.py`|
| latency | — | mcp p50 20.8 ms / p95 92 ms (was 18.5 / 94.5) |
32
32
33
-
Honest detail: of the 42 former crashers only **5** become hits at K=5. The crash class is dead;
33
+
Honest detail: of the 42 former crashers only **5** become hits at K=5 (`frozen_crash_and_live_hit_at_5: 5`, counted per item; `moved_outside_crash_class: []`). The crash class is dead;
34
34
those questions now have the same ranking problem as everything else. That is Stage 4's problem,
35
35
and the K/P/R shape says so: K and R moved, P did not.
36
36
37
-
## Deviation from the freeze wording — recorded, not hidden
37
+
## Gate amendment — the golden gate was changed, not met
38
38
39
-
The freeze said "golden file unchanged". The golden's `row_keys` gained `"message_id"` and each
40
-
case result gained its id; every case's query, ordering and preview is byte-identical (verified
41
-
by diff: five added lines, nothing removed). The gate's purpose — the explicit-syntax and simple
42
-
cases must not move while the planner changes — held. Real message ids were a stated Stage 2
43
-
requirement (the agent's interface had no citation handle and the census could not gate the tool
44
-
without them), so the additive change is the requirement, not drift.
39
+
The freeze said "golden file unchanged". That literal gate is **not met**: `row_keys` gained
40
+
`"message_id"` and each case result gained its id (15 changed lines in the diff stat: the
41
+
trailing-comma lines count as changed). The seat was right that labelling this "unchanged" was
42
+
wrong. The gate is hereby **amended** to: *the golden is equal to its Stage 1 form after
43
+
projecting `message_id` away* — checked between named commits `80ee57bb` and `d060d3f2` by
- Ranking, not vocabulary and not crashes, is the dominant miss on the corpus the product serves:
103
121
1,832 ranking misses vs 257 vocabulary misses.
104
122
105
-
## Council addendum
123
+
## Council addendum — single seat `openai.gpt-6-astra`, verdict **REJECT** (6 findings)
106
124
107
-
_(appended when the single-seat review returns)_
125
+
Brief 25.5 KB; 58 s; 844 words. Every finding verified at source before acting.
126
+
127
+
| id | sev. | finding | disposition |
128
+
|---|---|---|---|
129
+
| F1 | MAJOR | fallback after a rejected explicit query re-entered explicit detection; `fts:fts:alpha?` runs `alpha?` unguarded |**CONFIRMED at runtime** (reproducer crashed with `fts5: syntax error near "?"`). Fixed: the fallback calls `plan_natural_language()` only, strips every prefix; regression tests through MCP and CLI (`fts:fts:alpha?`, `fts:"alpha" OR ? AND`) |
130
+
| F2 | MAJOR | an uppercase operator inside a quoted span classified as explicit, so `"error OR warning" recovery` never widened |**CONFIRMED.** Operators now count only outside double-quoted spans (`_has_operator_outside_quotes`); tests pin the seat's discriminating case (upper == lower inside quotes; operator outside quotes still explicit) |
131
+
| F3 | MAJOR | census attribution overclaimed from a 400-sample; mechanism unverified |**ACCEPTED** → full-population per-question transition receipt; mechanism read from the frozen escape function (1,217/1,227); zero regressions across 5,434; bound and instrument difference stated |
132
+
| F4 | MAJOR | golden gate labelled met when it was changed; "five added lines" vs 15-line stat |**ACCEPTED** → gate marked NOT MET AS WRITTEN and amended to projected equality, checked between named commits by script; the 15 lines are trailing-comma changes on lines that also gained a comma |
133
+
| F5 | MINOR | 91/91 and "5 rescued" inferred, not counted |**ACCEPTED** → counted directly (`gold_counters`): 91/91 ordered lists, 5 of 42, `moved_outside_crash_class: []`|
134
+
| F6 | MINOR | studyloop attribution lacked a matched control |**ACCEPTED** → pre-Stage-2 worktree run in the same sandbox; 299/305 common ids, identical exception profiles |
135
+
136
+
After the corrections the code fix and tests are in `c8c105c4`; the gold and census receipts
137
+
were **not** regenerated for F1/F2 because neither reproducer pattern occurs in the gold set or
138
+
in the census population in a way that changes an outcome (checked over all 5,434 texts: zero start
139
+
with `fts:`; **4** carry an uppercase operator inside quotes — all four re-run through the fixed tool
140
+
keep their recorded own-session outcome, 3 hits and 1 miss, now via the `or` plan). The receipts
141
+
therefore stand exactly.
142
+
Escalation rule: a REJECT with MAJORs escalates to three seats — **pending Andy's call** whether
143
+
to run the three-seat council now or fold Stage 2 into the Stage 3 three-seat review already
0 commit comments