Skip to content

Refresh README, ROADMAP and methodology against what actually shipped - #19

Merged
emrefbulut merged 4 commits into
fix/gate-semantics-and-provenancefrom
docs/refresh
Sep 14, 2026
Merged

emrefbulut merged 4 commits into
fix/gate-semantics-and-provenancefrom
docs/refresh

Conversation

@emrefbulut

Copy link
Copy Markdown
Owner

Documentation only — no source file is touched. Stacked on #18; the base retargets to main once that merges.

1. README (af5b513)

Four problems, one of them actively misleading.

measure-leakage was documented as running "at split seed 42 and train seed 0". That is the Phase 5 regression described as if it were the design. The command measures 15 seed pairs; the single pair was the bug this project has spent the last stretch removing. A README that documents the defect as the feature is worse than one merely out of date.

also fixed was now
--force no limits stated applies to categories 2–5; 1 and 6 refused with a reason
command tour no iqforge train added — the rest of the README and CONTRIBUTING both use it
roadmap boxes no --group-by, no CUDA, no frequency-aware labelling all three listed, ticked per reality

On the status block: it still says 0.4.0, deliberately. __version__, CITATION.cff and the newest tag all agree, and docs/release-notes/v0.5.0.md says on its first line that it is an unpublished draft. Changing it to 0.5.0 would make the README claim a release that does not exist. What it now adds is that main carries unreleased work, so a reader can tell which parts are ahead of the wheel they installed.

2. ROADMAP (baa2602)

"Repeat the leakage measurement on a real recording" sat unchecked as though nothing had happened. Half of it shipped: the stride sweep ran on DASH7 and LoRaIQ, both tables are in artifacts/, and LoRaIQ at 7/8 overlap is the first individually significant real-data result here (+9.6 pp ± 2.7, t = 3.5).

Now two entries — the overlap half ticked with its numbers and its remaining limit (intermediate overlaps unresolved at 15 seed pairs), and the open item narrowed to the SNR half, pointing at docs/leakage-real-snr.md where the seed count and SNR list are locked before the run. The parity-gate re-measurement is added as its own finished entry.

The versioning footnote named 0.3.x / 0.4.0 as things to cut; 0.4.0 is out.

3. methodology reader's guide (5957658)

1100+ lines, nine sections, no contents. A question → section table, because "Measurement 1 — accuracy inflation against SNR" does not tell someone who wants to know how much a leaky split invents that this is their section. It also flags §7 (mistakes shipped) and §9 (what the numbers do not support) as worth reading regardless.

Anchors verified against GitHub's slug rule rather than assumed — my first checker collapsed consecutive spaces and flagged two correct anchors as dead. The em dash in those headings leaves a double space, and GitHub makes one hyphen per space without collapsing.

4. Repo sweep (24b0db3)

  • No dead relative links anywhere in *.md, docs/, .github/ — checked mechanically.
  • Version strings are consistent at 0.4.0 across README, CITATION.cff, ROADMAP and docs/publishing.md.
  • One expired statement: docs/phase5-phase6-checklist.md reports readiness as of 2026-08-19 and says no next-version release note exists. It does now. The file is a dated record, so correcting the line in place would falsify it — a superseded note at the top says the phase has since been executed and re-verified, names the line most likely to mislead, and points at CHANGELOG [Unreleased] and ROADMAP for current state.
  • README.md:304 "It does not train anything" was checked and is correct — it describes iqforge audit, which does not.

Verification

ruff check .            All checks passed!
ruff format --check .   82 files already formatted
pytest -q -rs           385 passed, 1 skipped
git diff --stat         README.md, ROADMAP.md, docs/methodology.md, docs/phase5-phase6-checklist.md

Reported, not changed

docs/leakage-real-snr.md still says the SNR grid "has not been run". A completed 180-run SNR sweep exists uncommitted in a sibling worktree, so from this repository's point of view the statement is still true. Committing that result is a decision for you, not a docs fix.

🤖 Generated with Claude Code

emrefbulut and others added 4 commits September 14, 2026 12:59
Four things, one of which was actively misleading.

measure-leakage was advertised as running "at split seed 42 and train seed 0".
That is the Phase 5 regression, described as if it were the design: the
command measures 15 seed pairs, and the single pair was the bug that this
project has spent the last stretch removing. A README that documents the
defect as the feature is worse than one that is merely out of date.

--force was described without its limits. It applies to categories 2-5, which
are inferences about what the recordings mean; categories 1 and 6 say no
measurement can be constructed, and are refused with a reason.

The command tour listed info, inspect, build, stats, audit and
measure-leakage, but not train -- which the rest of the README and
CONTRIBUTING both use.

The roadmap checklist was missing --group-by (three schemes, including
collection) and opt-in CUDA, both shipped, and did not list frequency-aware
labelling even though the known-limitations section points at it as the fix.

The status block keeps saying 0.4.0, because that is the released version:
__version__, CITATION.cff and the newest tag all agree, and
docs/release-notes/v0.5.0.md says on its first line that it is an unpublished
draft. What it now adds is that main carries unreleased work, so a reader can
tell which parts of this file are ahead of the wheel they installed.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Repeat the leakage measurement on a real recording" sat unchecked in Now as
though nothing had happened. Half of it has shipped: the stride sweep ran on
DASH7 and LoRaIQ, both tables are in artifacts/, and LoRaIQ at 7/8 overlap is
the first individually significant real-data result in this repository. A
roadmap that does not show that makes the remaining work look larger than it
is and the finished work invisible.

It is now two entries. The overlap half is ticked, with the numbers and the
limit that still applies -- the intermediate overlaps are unresolved at 15
seed pairs. The open item narrows to the SNR half, and points at
docs/leakage-real-snr.md, where the seed count and SNR list are locked before
the run so the result cannot pick its own stopping rule.

The parity-gate re-measurement is added as a separate finished entry, since
it is the thing that says the published tables still come out of the shipped
command.

The versioning footnote named 0.3.x / 0.4.0 as things to cut; 0.4.0 is out. It
now says where unreleased work lives and that the release notes for the next
version are marked drafts until a tag exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The document is over 1100 lines across nine numbered sections and had no
contents at all -- the only way in was the top. Most readers arrive with one
question, and the section that answers it is rarely the first.

The table maps a question to a section rather than listing titles, because
"Measurement 1 -- accuracy inflation against SNR" does not tell someone who
wants to know how much a leaky split invents that this is their section.

It also points out the two sections worth reading regardless of whether the
measurements apply to your data: 7, the mistakes this project shipped, and 9,
what the numbers do not support.

Anchors verified against GitHub's slug rule rather than assumed. The first
attempt used a checker that collapsed consecutive spaces, which flagged two
correct anchors as dead -- the em dash in those two headings leaves a double
space, and GitHub turns each space into its own hyphen without collapsing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The file carries "Date checked: 2026-08-19" and reports readiness as of that
date. One of its lines -- "no next-version draft file yet" -- is now false,
because docs/release-notes/v0.5.0.md exists.

Correcting the line in place would falsify a dated record, so the body is left
exactly as written and a note at the top says the phase has since been
executed and re-verified, names the line most likely to mislead, and points at
CHANGELOG [Unreleased] and ROADMAP for current state.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@emrefbulut
emrefbulut merged commit bd32714 into fix/gate-semantics-and-provenance Sep 14, 2026
5 checks passed
emrefbulut added a commit that referenced this pull request Sep 14, 2026
The stacked PRs #18, #19 and #20 were merged at the same moment, so only #18
reached main -- #19 merged into fix/gate-semantics-and-provenance and #20 into
docs/refresh, their own bases. Everything from those two is therefore sitting
on this branch and main is still on 0.4.0. That was my mistake in stacking
three deep instead of retargeting; this merge is the fix.

One conflict, in CHANGELOG.md. main carried the entry describing
docs/release-notes/v0.5.0.md as an unpublished draft, which commit 6491f27 on
this branch had already rewritten into "the four places that carry a version
agree again" -- the draft label was the state this release ends. Kept this
branch's side, which also carries three entries main does not have.

Checked rather than assumed, because a merge across branches that both edited
the test files could silently drop one side: #17's CUDA comparability tests
and #16's forced-measurement width test are both present afterwards, and the
suite goes from 385 to 388 passing, which is main's three additions arriving
rather than anything being lost.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant