Skip to content

Documentation round: correct the seed-grid contract in SPEC, and add convention 9 - #23

Open
emrefbulut wants to merge 4 commits into
mainfrom
docs/refresh-round2
Open

emrefbulut wants to merge 4 commits into
mainfrom
docs/refresh-round2

Conversation

@emrefbulut

Copy link
Copy Markdown
Owner

Documentation only; no source file is touched. Four commits, one per file.

The find worth leading with

SPEC.md said, in the file that is supposed to be the authority:

Default operating point is split_seed=42 and train_seed=0.

That is the Phase 5 regression written down as the contract — the same sentence the README carried and that 0.5.0 exists to correct. The real default is 5 split seeds × 3 training seeds = 15 pairs.

--split-seeds and --train-seeds were not in SPEC at all, which is the wrong omission to make: they are the flags that decide sample size, and this release exists largely because that number changed once without anyone noticing.

Checked mechanically rather than by reading — every --flag on audit, build, measure-leakage and train is now mentioned somewhere in SPEC. That also turned up build's long form --output, documented only as -o.

README — two limitations the audit report names and the README did not

"Known limitations" is what someone reads before deciding whether this tool fits their recording. It listed two. The audit report names two more on every run:

  • audit cannot tell you a measurement is possible, only that it is impossible. It trains nothing, so it rules out the trivial case and cannot find the SNR band where a model is partly right — the only band where inflation is visible. The report prints NOT CHECKED task difficulty; the README now says so first.
  • Physical independence is outside what any of these checks can reach. Two recordings sharing no samples, no air time and no timestamp can still be near-duplicates. This is the limit that eliminated DASH7 ds_indoor — two 8-second captures 43 seconds apart, same antenna and position — after it passed every count-based test.

Also fixed: a roadmap line pointing at "the first known limitation below" when that section is above it.

ROADMAP — one item was listed twice

"Verification with an own hardware capture" (Now item 2) and "Hardware capture on one device" (Next) are the same work, and nothing said so, making the open list look one item longer than it is. The Next entry now cross-references it and keeps its own reason for existing.

CONTRIBUTING — convention 9, from this week

The eight conventions were each written after breaking one. The ninth is "a claim repeated is not a claim checked", and the example is the __original_version error corrected in #22: written once, restated in four more files, and five files agreeing looked like corroboration when it was one edit copied. One gh api call settled it.

The entry names the cheap sources this repo actually has — gh api for what a PR says, artifacts/ for what a number is, git log -S for when a line appeared — and asks that a correction state what the old version claimed rather than quietly overwriting it. The PR template listed eight conventions and now lists nine.

Checked and found correct — no commit

Not everything needed changing, and saying so is part of the report:

checked result
README status block 0.5.0, "On PyPI" — verified against PyPI's API: ['0.1.0'…'0.5.0'], latest 0.5.0, tag v0.5.0 exists
README quickstart audit, train, --group-by collection all present and correct (collection takes no colon; path:/csv: do)
README roadmap boxes every ticked item verified, incl. GROUP_SCHEMES == ('path','csv','collection')
methodology §2/§3 numbers every inflation row re-derived from artifacts/*.md — all four tables match
methodology section numbering test_the_three_documents_agree_on_the_category_numbers passes
methodology sigmf section never carried the __original_version or "two maintainers" claims
methodology open questions current
dead relative links none, across *.md, docs/, .github/
stale 0.4.0 references four remain, each a legitimate historical reference
the regression sentence grep for it across the repo returns nothing

Verification

ruff check .            All checks passed!
ruff format --check .   82 files already formatted
pytest -q               388 passed, 1 skipped

🤖 Generated with Claude Code

emrefbulut and others added 4 commits September 14, 2026 14:57
"Known limitations" is the section someone reads before deciding whether this
tool fits their recording, and it listed two. The audit report itself names two
more every time it runs, and neither appeared here.

The first is what `audit` cannot do. It trains nothing, so it can rule out the
trivial case -- one axis already classifying the recordings -- and it cannot
find the opposite end, the SNR band where a model is partly right, which is the
only band where inflation is visible at all. The report prints NOT CHECKED for
task difficulty rather than implying coverage; the README now says the same
thing before a user starts.

The second is physical independence. Two recordings can share no samples, no
air time and no timestamp and still be near-duplicates, and no index arithmetic
finds that. It is worth its own entry because it is the limit that eliminated a
dataset which passed every count-based test: DASH7 ds_indoor, two 8-second
captures 43 seconds apart, same antenna and position, separate recorder
invocations. --group-by is the answer and it needs the user to already know
which recordings belong together.

Also corrected: the roadmap line pointed at "the first known limitation below"
while that section is above it.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Verification with an own hardware capture" is Now item 2 and "Hardware
capture on one device" is the first unchecked entry under Next. They are the
same work, and nothing said so, which makes the open list look one item longer
than it is.

The Next entry now points at Now item 2 and keeps its own reason for being
there: it is parallel to the public-file track rather than blocked by it.

Everything else in the file was checked against reality rather than reread.
The unchecked items are all genuinely open -- the SNR half of the real-capture
measurement, frequency-aware labelling, the core:collection question, users,
and a docs site -- and the versioning note is correct now that 0.5.0 is tagged
and on PyPI.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…et it

SPEC said:

    Default operating point is split_seed=42 and train_seed=0.

That is the Phase 5 regression written down as the contract -- the same
sentence the README carried, in the file that is supposed to be the
authority. The default grid is 5 split seeds x 3 training seeds = 15 pairs,
which is the grid every published table used.

--split-seeds and --train-seeds were not in SPEC at all, which is the wrong
omission to make: they are the flags that decide the sample size, and this
release exists largely because that number was once changed without anyone
noticing. The signature now lists them with their defaults, and the text says
the pair count is printed with the result and that a single pair reports no
interval rather than +/- 0.0.

Checked mechanically rather than by reading: every --flag on audit, build,
measure-leakage and train is now mentioned somewhere in SPEC. That also turned
up build's long form --output, which existed only as -o here.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ecked

The eight conventions were each written after breaking one. This is the ninth,
and the example is from the same week.

A note claimed upstream had named an attribute __original_version and that it
shipped under a different name, so measuring the release beat reading the diff.
Written once, then restated in CHANGELOG.md, docs/release-notes/v0.5.0.md,
ROADMAP.md and a comment in tests/test_io.py. One API call settled it: the
string appears nowhere upstream and the pull request had named
_declared_version since the day it opened. The lesson each copy carried was
wrong too, and a second claim in the same paragraph -- "two maintainers
approved it" -- was one maintainer.

Five files agreeing looked like corroboration. It was one edit, copied.

The convention is to go back to the source rather than to the first file, and
the entry names the cheap sources this repository actually has: gh api for what
a pull request says, artifacts/ for what a number is, git log -S for when a
line appeared. It also asks that a correction state what the old version
claimed instead of quietly overwriting it, which is convention 6 applied to
documents.

The pull request template listed eight conventions and now lists nine.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant