Skip to content

fix(spec): re-sync the #102-moved prose figures and bind them in verify:claims - #116

Merged
justin13888 merged 2 commits into
masterfrom
fix/113-sync-decision-figures-verify-claims
Sep 29, 2026
Merged

justin13888 merged 2 commits into
masterfrom
fix/113-sync-decision-figures-verify-claims

Conversation

@justin13888

Copy link
Copy Markdown
Collaborator

Summary

The #102 re-score moved spec/EXPERIMENTS.md §7.13, §7.10 and §11.9, but four prose figures that restate those cells by hand kept their old values, and verify:claims bound none of them.

  • spec/V0.8-DECISIONS.md — D1's prior evidence: LOO coder saving −7.4% → −7.5% (§7.13, per-index context backing off to order-0, LOO / vs fixed).
  • spec/README.md — CfL roadmap row: free −0.09% → −0.10%, paid +2.18% → +2.11% (§7.10). Scalefactor-bands roadmap row and the §15 summary paragraph: −0.13% → −0.14% (§11.9).
  • tools/comparison/src/verify-claims.ts — six new register entries binding those quotes (plus the CfL row's −0.90% code-3 figure), and spec/V0.8-DECISIONS.md added to the header's list of quoting files. The register now checks 34 figures across 6 files; registering spec/V0.8-DECISIONS.md also puts it under the unregistered-figure scan, which finds nothing there.

Validation

  • node tools/comparison/src/verify-claims.ts — exit 0: "Checked 34 quoted figure(s) across 6 files", no unregistered figure.
  • Falsifier: with the four old values temporarily restored in the two spec files (not committed), the same command exits 1 and reports all five stale quotes (7.4 vs 7.50, 0.09 vs 0.10, +2.18 vs 2.11, 0.13 vs 0.14 twice).
  • pnpm --prefix tools/comparison run format:check — pass.
  • pnpm --prefix tools/comparison run lint — pass.
  • pnpm --prefix tools/comparison exec tsc --noEmit — exit 0.

Coverage gaps

Risks and rollout

None. Documentation figures and a CI documentation gate only; no format, API or encoder change.

Decisions taken

  • Where §11.9's best arm is bound. §11.9 states −0.14% only in prose and has no table, and the register binds cells. Taken: bind both quotes to §9.5's re-source table, row Scalefactor bands, best arm, column Wikimedia, which holds the same measurement (−0.14%). Rejected: extending the register to bind prose (a new mechanism, and it would gate a sentence rather than a cell); adding a table to §11.9 (outside the manifest, and EXPERIMENTS.md edits belong with the sweep bindings). Reverse: change section/row/column on the two scalefactor-band entries.
  • How the paid CfL figure is bound. +2.11% is against the shipped row, not a table cell. Taken: bind the CfL paid, L24@5 C9@4 / ΔE00 cell and recompute against shipped / ΔE00 in a transform, the same shape as §11.14's WebP margin; the capture keeps the + and omits abs, like the 108 B entry, because the sentence's content is the direction. Rejected: abs, which would pass a flipped sign. Reverse: edit that entry's transform/pattern.
  • Binding the −0.90% code-3 figure too. Not named in fix(spec): re-sync prose figures the #108 re-score moved, and bind them in verify:claims #113, but it is the other end of the same sentence and the same column. Taken: register it. Reverse: delete the CfL with free gains at code 3 entry.
  • CI path filter and the §11.13 cell left out. Both are outside the manifest (.github/workflows/ci-comparison.yml, spec/EXPERIMENTS.md). Taken: filed as fix(comparison): run verify:claims on spec/V0.8-DECISIONS.md edits, and re-sync EXPERIMENTS.md §11.13's scalefactor cell #115 rather than widening. Reverse: fold fix(comparison): run verify:claims on spec/V0.8-DECISIONS.md edits, and re-sync EXPERIMENTS.md §11.13's scalefactor cell #115 into this branch.

Issue

Closes #113

…s to EXPERIMENTS.md

The #102 re-score moved EXPERIMENTS.md §7.13 (LOO coder saving −7.5%), §7.10 (CfL free −0.10%, paid +2.11%) and §11.9 (best scalefactor-band arm −0.14%), but the prose restating them in V0.8-DECISIONS.md and spec/README.md kept the old values.
…ify:claims

Registers the V0.8-DECISIONS.md LOO coder saving, spec/README.md's CfL free and paid figures, and both quotes of the best scalefactor-band arm, so a re-score that moves those cells fails the gate instead of leaving the prose stale.
@justin13888
justin13888 merged commit e595f26 into master Sep 29, 2026
4 checks passed
@justin13888
justin13888 deleted the fix/113-sync-decision-figures-verify-claims branch September 29, 2026 01:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(spec): re-sync prose figures the #108 re-score moved, and bind them in verify:claims

1 participant