Bring the docs refresh and the 0.5.0 version bump onto main - #21
Merged
Merged
Conversation
The gate was documented in two lines, neither of which said what it checks. "The published tables and the parity gate stay on CPU" is a policy, not a contract, and a reader had no way to tell whether a pass meant "ran with the same settings" or "produced the same numbers". Those are different claims and only one of them is true of this gate. It is the stronger one. compare_cell compares test_accuracy, train_accuracy, train_windows and test_windows by exact equality, row by row, matched by (strategy, split seed, train seed) rather than by position -- on top of the run count and the seed-pair set. Demonstrated against the real rows of leakage_real_stride_runs.json: one test_accuracy moved by 1e-12 fails the cell. What it does not compare is now written down too, because an undocumented exclusion reads as coverage. It ignores the environment block, and it ignores stride / noise_sigma / snr_db, which select the cell rather than being measured by it -- a wrong stride changes the window counts, and those are compared. The environment exclusion is load-bearing rather than missing. Because the gate does not compare environments, a re-run on a different library stack is a real difference between the two runs, and equality across it is a result. methodology 8 now records one: leakage_real_stride_runs.json was produced on 2026-08-11 under sigmf 1.11.1 -- inferred from the tripwire not recording 1.12.0 until eight days later, and labelled as an inference because the artifact carries environment: null -- and three cells of it came back bit-identical when re-measured under sigmf 1.13.0 on a torch, numpy and scipy that also moved. Stated at the width it supports: one table, three cells, one direction of upgrade, on CPU. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
--force overrode every refuse category, including the two where there is nothing to overrule. Categories 2, 3, 4 and 5 are inferences: this tool deciding what a shared timestamp, a 43-second gap, a separable axis or an audit LEAK probably means. Someone who knows their own recordings can be right where the inference is wrong, and that is what the flag is for -- the overridden category stays in the header so the block cannot be pasted as a clean run. Categories 1 and 6 are not inferences. Category 1 means the reader could not open the files; there is nothing to measure and forcing produces a crash rather than a questionable number. Category 6 means build would refuse the split, so the paired measurement cannot be constructed at all. Accepting --force on either is accepting a request that cannot be fulfilled, and then failing somewhere further in with a message about something else -- which is exactly the shape of failure the refuse path exists to prevent. They now stay REFUSED with exit code 1, and the report says so rather than merely omitting the FORCED header: `forced refused. category 1 ... cannot be overridden`. The JSON carries force_refused alongside forced and forced_past. Silence would have been indistinguishable from the flag not being passed. Mutations, each red on the test that should catch it: emptying STRUCTURAL_CATEGORIES lets the structural refusals be forced again; filling it with every category breaks the four that must stay forcible; dropping the report line makes the refusal silent. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The WOULD MEASURE path printed
started no. This version of the command stops before training
unconditionally, and on a machine with torch the measurement it denied
followed four lines later in the same block. A refuse path whose report
cannot be trusted about its own behaviour has no standing to be trusted
about the recordings.
The line is now derived rather than fixed. The CLI decides before the block
is rendered whether training will follow -- torch present, and a folder of
recordings rather than a built dataset -- and passes the obstacle to decide()
when there is one. `started` then reads `yes. Both arms train after this
block and the result follows below`, or `no.` followed by the actual reason,
or `no. Refused above; nothing was built and nothing was trained`.
The reason is passed in rather than inferred inside decide(), because the
decision genuinely does not know whether its caller intends to train.
Guessing is what produced the wrong sentence.
Three other places asserted the same stale thing and are corrected: the
module docstring, the default WOULD MEASURE reason, and the WorkEstimate
docstring. The JSON gains `trains` and `no_train_reason` so a machine reader
does not have to parse the sentence.
CHANGELOG: "No training CLI yet." and "This version does not train." were
both false and are rewritten, the second one also picking up the --force
restriction from the previous commit.
Mutations: hardcoding the old sentence back fails two tests; making the CLI
stop passing the reason fails the torch-free test. That second mutation
initially passed, because on this machine every run trains and "yes" is right
by accident -- so the branch is now forced open by patching _torch_available
rather than left unreachable.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… row writer measure-leakage --format json had two payload builders and only one was updated when measurement_schema was introduced. The single-cell path emitted it and serialised rows through _run_row; the sweep path emitted neither, and repeated _run_row's nine fields by hand. runs_from_payload treats a missing measurement_schema as "unversioned, assume compatible". That is right for payloads written before the field existed, and it meant a sweep payload written today was indistinguishable from one of those -- so sweep payloads would be exactly the ones that cannot be told apart when schema 2 arrives. The hand-rolled copy is the same shape of duplication that dropped the seed lists from one of three near-identical measurement calls. A field added to _run_row would not have reached the sweep payload. The sweep payload also gains `forced`, which the single-cell payload already carried. A reader should not have to know which mode produced a file to know whether --force was used. Closes #3. Mutations: removing the schema key fails two tests; hand-rolling the rows again fails the third. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three shipped changes had no entry. The most consequential one had none at all: the commit that restored the 15 seed pairs touched seven files and not this one, so the single change that fixed the sample-size regression -- and added the guard that stops a published grid shrinking again -- was visible only as a passing mention inside the parity-gate entry. Added: - The seed-pair restoration, the --split-seeds / --train-seeds flags, guard_artifact_rows, and the check_environment hole that made the guard ineffective on exactly the files it was meant to protect. - measurement_schema in the JSON payload, with the reason it exists: the same one read_manifest already has. - docs/release-notes/v0.5.0.md being labelled an unpublished draft, which is what makes __version__, CITATION.cff, the CHANGELOG and the notes agree. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The item was unchecked and its own body said "`--group-by collection` is now shipped, so the first half of that is done". A checkbox that disagrees with the paragraph under it teaches a reader to stop trusting the checkboxes. It was one item covering two things: read the SigMF field, and decide whether the field needs an extension. The first shipped -- `collection` is the third scheme beside `path:` and `csv:`, with a test, and a recording that declares no collection stays its own unit. The second has not started, and is the one with a bar to clear: use it on a public dataset and record with evidence what `core:collection` could not express. They are now two items, ticked according to what is true of each. The reasoning note and the sigmf-python#159 / SigMF#233 precedent stay attached to the half they inform, which is the proposal rather than the scheme. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
measure-leakage prints `category 4 ceiling (methodology 6.4)` and a reader follows that into §6. §6 did not contain the word "category" anywhere. The numbering matched for the four eliminated datasets by construction and nothing held it there, and past four it does not match at all -- which mattered more than the missing word: category 5 is a refusal, a structural leak audit named §6.5 is LoRaIQ, the one dataset in that section that was NOT eliminated Same digit, opposite meaning, and nothing warned a reader following a citation. §6 now opens with a cross-reference table covering all six categories, marks the two that have no case here and what they cite instead, states the 5 / 6.5 collision explicitly, and notes which categories --force can override. A test pins all three documents together: for every member of the Category enum, SPEC and methodology must each carry a table row with that number and that name, and the collision warning must still be there. Mutations: renumbering a category in the code alone fails it, deleting one methodology row fails it, softening the warning to "Note." fails it. The assertions compare with whitespace removed. These documents are hard-wrapped and "There is deliberately no category for it" already straddles two lines -- an exact-substring assertion passed only until the paragraph was re-wrapped. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
test_loraiq_pattern_is_not_refused_when_grouped asserted "stops before training" in its torch-free branch. That branch is unreachable on a machine with torch, so the full local suite passed after the output changed and CI's test (3.11) and test (3.12) jobs are what caught it. That is precisely the failure this PR describes one commit earlier -- an assertion whose coverage depends silently on the environment -- and I walked into it while fixing it. Worth saying rather than quietly amending. The branch now asserts what the report actually prints, the torch branch additionally pins `started yes.`, and the old sentence is asserted absent on both paths so neither can drift back. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Four things, one of which was actively misleading. measure-leakage was advertised as running "at split seed 42 and train seed 0". That is the Phase 5 regression, described as if it were the design: the command measures 15 seed pairs, and the single pair was the bug that this project has spent the last stretch removing. A README that documents the defect as the feature is worse than one that is merely out of date. --force was described without its limits. It applies to categories 2-5, which are inferences about what the recordings mean; categories 1 and 6 say no measurement can be constructed, and are refused with a reason. The command tour listed info, inspect, build, stats, audit and measure-leakage, but not train -- which the rest of the README and CONTRIBUTING both use. The roadmap checklist was missing --group-by (three schemes, including collection) and opt-in CUDA, both shipped, and did not list frequency-aware labelling even though the known-limitations section points at it as the fix. The status block keeps saying 0.4.0, because that is the released version: __version__, CITATION.cff and the newest tag all agree, and docs/release-notes/v0.5.0.md says on its first line that it is an unpublished draft. What it now adds is that main carries unreleased work, so a reader can tell which parts of this file are ahead of the wheel they installed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Repeat the leakage measurement on a real recording" sat unchecked in Now as though nothing had happened. Half of it has shipped: the stride sweep ran on DASH7 and LoRaIQ, both tables are in artifacts/, and LoRaIQ at 7/8 overlap is the first individually significant real-data result in this repository. A roadmap that does not show that makes the remaining work look larger than it is and the finished work invisible. It is now two entries. The overlap half is ticked, with the numbers and the limit that still applies -- the intermediate overlaps are unresolved at 15 seed pairs. The open item narrows to the SNR half, and points at docs/leakage-real-snr.md, where the seed count and SNR list are locked before the run so the result cannot pick its own stopping rule. The parity-gate re-measurement is added as a separate finished entry, since it is the thing that says the published tables still come out of the shipped command. The versioning footnote named 0.3.x / 0.4.0 as things to cut; 0.4.0 is out. It now says where unreleased work lives and that the release notes for the next version are marked drafts until a tag exists. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The document is over 1100 lines across nine numbered sections and had no contents at all -- the only way in was the top. Most readers arrive with one question, and the section that answers it is rarely the first. The table maps a question to a section rather than listing titles, because "Measurement 1 -- accuracy inflation against SNR" does not tell someone who wants to know how much a leaky split invents that this is their section. It also points out the two sections worth reading regardless of whether the measurements apply to your data: 7, the mistakes this project shipped, and 9, what the numbers do not support. Anchors verified against GitHub's slug rule rather than assumed. The first attempt used a checker that collapsed consecutive spaces, which flagged two correct anchors as dead -- the em dash in those two headings leaves a double space, and GitHub turns each space into its own hyphen without collapsing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The file carries "Date checked: 2026-08-19" and reports readiness as of that date. One of its lines -- "no next-version draft file yet" -- is now false, because docs/release-notes/v0.5.0.md exists. Correcting the line in place would falsify a dated record, so the body is left exactly as written and a note at the top says the phase has since been executed and re-verified, names the line most likely to mislead, and points at CHANGELOG [Unreleased] and ROADMAP for current state. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Raise the version to 0.5.0
src/iqforge/__init__.py is the single source: pyproject declares
dynamic = ["version"] and [tool.hatch.version] reads this file, so nothing
else in the tree carries a version string to keep in step.
Note for whoever runs this next. After the bump an editable install still
reported 0.4.0 from importlib.metadata, because uv sync had no reason to
rebuild the package metadata -- pyproject itself did not change. That is the
0.2.0 failure exactly, and the fix is to reinstall the package rather than to
trust the sync:
uv sync --group dev --extra torch --reinstall-package iqforge
__version__, importlib.metadata and `iqforge version` agree afterwards, and
tests/test_packaging.py passes. No tag here; tagging is a separate, manual
step.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Close [Unreleased] as [0.5.0], after filling three gaps in it
Twenty entries move under a dated 0.5.0 heading and Unreleased is left empty.
Three changes from this cycle had shipped without an entry and are added
first, because rolling up an incomplete section would have published the gap:
- the refuse categories now carrying the same numbers in code, SPEC and
methodology, including the category 5 / 6.5 collision that the cross-
reference table exists to head off
- what PARITY_GATE_PASSED asserts, which is the numbers rather than the
configuration
- the README / ROADMAP / methodology refresh, including the README having
documented the single-seed-pair regression as if it were the design
One item from the brief does not belong here. --group-by collection is
already in the [0.4.0] section: it shipped in that release, and moving it
would claim it twice and misdate it.
The date is today's. If the tag lands on a different day, this line is the
one to correct.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Rewrite the v0.5.0 release notes for what the release actually contains
The draft covered the Phase 5 migration and nothing else: three bullets of
highlights, a file list, and the gate description. Twenty entries went into
[0.5.0], and the draft named perhaps four of them. It was also written in a
checklist voice rather than the one the previous notes use.
Rewritten in the house style, with the headline where 0.4.0 put its own: the
correctness problem first. 0.4.0 opened with a bug it had been shipping since
0.1.0; this one opens with a migration that reduced every published
measurement to a sample size of one and a gate that approved it.
The "unpublished draft" banner is gone, because the version is now 0.5.0 in
__init__.py and the CHANGELOG section is dated. Tagging remains manual.
Coverage checked entry by entry against the [0.5.0] section rather than by
eye -- which caught --balance-by and measurement_schema missing on the first
pass, and a checker of mine that matched "manifest_schema" and reported
measurement_schema as present when it was not.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Point the citation at 0.5.0
version and date-released both move. The comment above date-released explains
why the date is pre-filled rather than added after tagging: doing it
afterwards put the correct value in a commit AFTER the tag, so the tagged tree
shipped a citation with no date, and 0.2.0 went two days that way.
That reasoning still applies, so the date here is the intended tag date
(2026-09-14). If the tag slips, correct this line before pushing it.
Validated as YAML and checked against __version__ rather than eyeballed.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
* Follow the bump through every place that named the old version
Raising __version__ made seven statements false that had been true an hour
earlier. Found by sweeping for "0.4.0" after the bump rather than by
remembering which files mention a version.
- docs/publishing.md: the tag, release and rollback commands, and the
"a burned number" example, all still said v0.4.0. Someone following that
page would have tagged the wrong version.
- CHANGELOG link refs: [Unreleased] compared v0.4.0...HEAD and there was no
[0.5.0] ref for the heading that now exists.
- CHANGELOG: the entry describing v0.5.0.md as an unpublished draft was about
a state this release ends. Reworded to what shipped -- the four places that
carry a version agreeing -- rather than deleted, since the drift it records
is the point.
- README status block said "latest release 0.4.0" plus a note about main
carrying unreleased work, which is no longer true now that [Unreleased] is
empty. It now names 0.5.0 and, in its place, the one thing a reader of an
older build needs to know: between the Phase 5 migration and this release
measure-leakage ran one seed pair, so any figure it produced carried +/- 0.0.
- README PyPI release list, and the ROADMAP versioning footnote.
- docs/phase5-phase6-checklist.md: the superseded note I added earlier said
v0.5.0.md was labelled a draft because no tag existed. It is the release
notes now.
Every surviving "0.4.0" was checked individually and is correct: the [0.4.0]
heading and link ref, the CHANGELOG entry describing the state that was fixed,
and the release notes comparing this release to the previous one.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
The stacked PRs #18, #19 and #20 were merged at the same moment, so only #18 reached main -- #19 merged into fix/gate-semantics-and-provenance and #20 into docs/refresh, their own bases. Everything from those two is therefore sitting on this branch and main is still on 0.4.0. That was my mistake in stacking three deep instead of retargeting; this merge is the fix. One conflict, in CHANGELOG.md. main carried the entry describing docs/release-notes/v0.5.0.md as an unpublished draft, which commit 6491f27 on this branch had already rewritten into "the four places that carry a version agree again" -- the draft label was the state this release ends. Kept this branch's side, which also carries three entries main does not have. Checked rather than assumed, because a merge across branches that both edited the test files could silently drop one side: #17's CUDA comparability tests and #16's forced-measurement width test are both present afterwards, and the suite goes from 385 to 388 passing, which is main's three additions arriving rather than anything being lost. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Brings #19 and #20 onto
main. They are already merged as pull requests, but into their own stacked bases rather than intomain, so none of their content reached it.Why this PR exists
#18, #19 and #20 were merged at the same moment. Only #18 had
mainas its base:GitHub retargets a stacked PR when its base merges, but with all three going in together there was no gap for that to happen. The result:
mainis on0.4.0with no[0.5.0]section, whiledocs/refreshcarries everything.That was my error — stacking three deep instead of retargeting each PR to
main. This is the correction, and it is one level, not a stack.What lands
13 commits: the version bump to
0.5.0, the CHANGELOG roll-up, the rewritten release notes,CITATION.cff,docs/publishing.md, the README/ROADMAP/methodology refresh, and the Bölüm C items from #18's branch that were not in #18 itself.src/iqforge/__init__.py0.4.0→0.5.0(the single source)CHANGELOG.md[0.5.0] — 2026-09-14, empty[Unreleased], link refsdocs/release-notes/v0.5.0.mdCITATION.cffdate-releaseddocs/publishing.mdv0.5.0README.md,ROADMAP.md,docs/methodology.md,docs/phase5-phase6-checklist.mdThe merge, checked rather than assumed
mainand this branch both editedtests/test_preflight.py, andmainalone added totests/test_cli_output.py. A merge across that can silently drop one side, so it was verified before committing:test_cli_output.py)test_preflight.py)--force/started/ category testsThe increase is
main's three additions arriving, not anything being lost.One conflict, in
CHANGELOG.md.maincarried the entry describingdocs/release-notes/v0.5.0.mdas an unpublished draft; commit6491f27on this branch had already rewritten it, because the draft label describes the state this release ends. This branch's side was kept — it also carries three entriesmaindoes not have.Verification
Before you tag
CITATION.cffdate-releasedand the[0.5.0]heading both say 2026-09-14, pre-filled deliberately — the comment inCITATION.cffrecords that adding the date after tagging once shipped0.2.0with a dateless citation for two days. If the tag lands on another day, correct both lines first.No tag is pushed here.
🤖 Generated with Claude Code