Documentation round: correct the seed-grid contract in SPEC, and add convention 9 - #23
Open
emrefbulut wants to merge 4 commits into
Open
emrefbulut wants to merge 4 commits into
emrefbulut wants to merge 4 commits into
Conversation
"Known limitations" is the section someone reads before deciding whether this tool fits their recording, and it listed two. The audit report itself names two more every time it runs, and neither appeared here. The first is what `audit` cannot do. It trains nothing, so it can rule out the trivial case -- one axis already classifying the recordings -- and it cannot find the opposite end, the SNR band where a model is partly right, which is the only band where inflation is visible at all. The report prints NOT CHECKED for task difficulty rather than implying coverage; the README now says the same thing before a user starts. The second is physical independence. Two recordings can share no samples, no air time and no timestamp and still be near-duplicates, and no index arithmetic finds that. It is worth its own entry because it is the limit that eliminated a dataset which passed every count-based test: DASH7 ds_indoor, two 8-second captures 43 seconds apart, same antenna and position, separate recorder invocations. --group-by is the answer and it needs the user to already know which recordings belong together. Also corrected: the roadmap line pointed at "the first known limitation below" while that section is above it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
"Verification with an own hardware capture" is Now item 2 and "Hardware capture on one device" is the first unchecked entry under Next. They are the same work, and nothing said so, which makes the open list look one item longer than it is. The Next entry now points at Now item 2 and keeps its own reason for being there: it is parallel to the public-file track rather than blocked by it. Everything else in the file was checked against reality rather than reread. The unchecked items are all genuinely open -- the SNR half of the real-capture measurement, frequency-aware labelling, the core:collection question, users, and a docs site -- and the versioning note is correct now that 0.5.0 is tagged and on PyPI. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…et it
SPEC said:
Default operating point is split_seed=42 and train_seed=0.
That is the Phase 5 regression written down as the contract -- the same
sentence the README carried, in the file that is supposed to be the
authority. The default grid is 5 split seeds x 3 training seeds = 15 pairs,
which is the grid every published table used.
--split-seeds and --train-seeds were not in SPEC at all, which is the wrong
omission to make: they are the flags that decide the sample size, and this
release exists largely because that number was once changed without anyone
noticing. The signature now lists them with their defaults, and the text says
the pair count is printed with the result and that a single pair reports no
interval rather than +/- 0.0.
Checked mechanically rather than by reading: every --flag on audit, build,
measure-leakage and train is now mentioned somewhere in SPEC. That also turned
up build's long form --output, which existed only as -o here.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ecked The eight conventions were each written after breaking one. This is the ninth, and the example is from the same week. A note claimed upstream had named an attribute __original_version and that it shipped under a different name, so measuring the release beat reading the diff. Written once, then restated in CHANGELOG.md, docs/release-notes/v0.5.0.md, ROADMAP.md and a comment in tests/test_io.py. One API call settled it: the string appears nowhere upstream and the pull request had named _declared_version since the day it opened. The lesson each copy carried was wrong too, and a second claim in the same paragraph -- "two maintainers approved it" -- was one maintainer. Five files agreeing looked like corroboration. It was one edit, copied. The convention is to go back to the source rather than to the first file, and the entry names the cheap sources this repository actually has: gh api for what a pull request says, artifacts/ for what a number is, git log -S for when a line appeared. It also asks that a correction state what the old version claimed instead of quietly overwriting it, which is convention 6 applied to documents. The pull request template listed eight conventions and now lists nine. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Documentation only; no source file is touched. Four commits, one per file.
The find worth leading with
SPEC.mdsaid, in the file that is supposed to be the authority:That is the Phase 5 regression written down as the contract — the same sentence the README carried and that
0.5.0exists to correct. The real default is 5 split seeds × 3 training seeds = 15 pairs.--split-seedsand--train-seedswere not in SPEC at all, which is the wrong omission to make: they are the flags that decide sample size, and this release exists largely because that number changed once without anyone noticing.Checked mechanically rather than by reading — every
--flagonaudit,build,measure-leakageandtrainis now mentioned somewhere in SPEC. That also turned upbuild's long form--output, documented only as-o.README — two limitations the audit report names and the README did not
"Known limitations" is what someone reads before deciding whether this tool fits their recording. It listed two. The audit report names two more on every run:
auditcannot tell you a measurement is possible, only that it is impossible. It trains nothing, so it rules out the trivial case and cannot find the SNR band where a model is partly right — the only band where inflation is visible. The report printsNOT CHECKED task difficulty; the README now says so first.ds_indoor— two 8-second captures 43 seconds apart, same antenna and position — after it passed every count-based test.Also fixed: a roadmap line pointing at "the first known limitation below" when that section is above it.
ROADMAP — one item was listed twice
"Verification with an own hardware capture" (Now item 2) and "Hardware capture on one device" (Next) are the same work, and nothing said so, making the open list look one item longer than it is. The Next entry now cross-references it and keeps its own reason for existing.
CONTRIBUTING — convention 9, from this week
The eight conventions were each written after breaking one. The ninth is "a claim repeated is not a claim checked", and the example is the
__original_versionerror corrected in #22: written once, restated in four more files, and five files agreeing looked like corroboration when it was one edit copied. Onegh apicall settled it.The entry names the cheap sources this repo actually has —
gh apifor what a PR says,artifacts/for what a number is,git log -Sfor when a line appeared — and asks that a correction state what the old version claimed rather than quietly overwriting it. The PR template listed eight conventions and now lists nine.Checked and found correct — no commit
Not everything needed changing, and saying so is part of the report:
0.5.0, "On PyPI" — verified against PyPI's API:['0.1.0'…'0.5.0'], latest0.5.0, tagv0.5.0existsaudit,train,--group-by collectionall present and correct (collectiontakes no colon;path:/csv:do)GROUP_SCHEMES == ('path','csv','collection')artifacts/*.md— all four tables matchtest_the_three_documents_agree_on_the_category_numberspasses__original_versionor "two maintainers" claims*.md,docs/,.github/0.4.0referencesgrepfor it across the repo returns nothingVerification
🤖 Generated with Claude Code