Repository navigation
CONTRIBUTING: point at docs/pre-pr-checks.md, which nothing linked to - #2350
Conversation
Measured: of the 45 files in `docs/`, exactly one had no inbound link from any other `.md` at `origin/main` — **`docs/pre-pr-checks.md`**, the before-you-merge page from engine#2311 that has since grown the #2341 section. A page nothing links to is a page nobody runs: the six checks in it are the ones that catch what a green tick and a careful read do not. The natural home is `CONTRIBUTING.md`, right under the five rules it operationalises, and the pointer says what the page is for rather than only where it is: none of the checks needs a device or a build, and the page also covers the one thing that is not a command — build the target that contains the file you edited, which is not necessarily the target your test links. After: 45 docs, 0 with no inbound link. Two instrument notes from finding this, both the same shape as the content of the page: - My first measurement said the orphan was `docs/oga-drafter-path.md`, and it was wrong: I ran `git grep HEAD` in the shared clone, whose checkout was behind `origin/main`, where `docs/npu.md` already links it. The measurement read a different revision than the one I named (`registration_loss_check` exists for exactly this mistake, one layer up). - `git checkout -- <path>` restores from the **index**, not from HEAD, so the hunk I meant to drop stayed staged until `git checkout HEAD -- <path>`. A command whose name suggests "revert" and whose behaviour is "un-modify relative to the index" is the same family: the instrument answered a question adjacent to the one asked.
PR Reviewer Guide 🔍(Review updated until commit 8375579)Here are some key observations to aid the review process:
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- and the sentence at the end is the one that would have saved the last three incidents
A page nothing links to is a page nobody reads, so a five-line CONTRIBUTING pointer is load-bearing rather than cosmetic: docs/pre-pr-checks.md now exists where a contributor actually starts, and pre-pr-checks.sh runs the six in one command. Between them the "the check exists and was not run" class -- which produced three stale-baseline incidents in two days -- finally has both halves.
The last clause is the most valuable line in the diff: "build the target that contains the file you edited, which is not necessarily the target your test links." That is not a generic caution, it is the exact shape of the two defects this whole family of PRs is about -- app/forward_serve.cpp joining onebit only inside ONEBIT_NPU (so a build of the test target never compiled it), and a bench that loads one hard-coded code object (so every other kernel "did not resolve"). Neither is a missing check; both are a check applied to the wrong target. Naming that in the page a contributor reads before opening a PR is worth more than another script.
Nothing to change. Two nits I am not asking for: the link is relative (docs/pre-pr-checks.md) which is right for GitHub rendering and for a local checkout; and CONTRIBUTING presumably already has a "before you open a PR" section that this sits inside, which is the correct placement rather than a new top-level heading.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at 8f043ce - one sentence, and the PR's own purpose is the reason
Every claim in the paragraph holds on main, and I checked each rather than the shape:
docs/pre-pr-checks.mdexists;- its table has exactly six commands (
copyright,tests_referenced,ci_targets,registration_loss,tmp_path,submodule_pin) -- so "the six host-side checks" is the right count; ## The one that is not a command: build the target that contains the file you editedis the section you point at, and it is the one about the target that is not the test's;- and "none of them needs a device or a build" is what the page says about itself.
What is missing is the command that runs all six. scripts/pre-pr-checks.sh is on main (engine#2313, merged today) and does exactly this:
$ ./scripts/pre-pr-checks.sh
ok tools/copyright.py
ok tools/ci/tests_referenced_check.py all 86 test script(s) are referenced
ok tools/ci/ci_targets_check.py 9 target(s) whose test never reaches ctest (baseline 9)
ok tools/ci/tmp_path_check.py 44 baselined site(s), none new, none stale
ok tools/ci/submodule_pin_check.py 1 submodule(s) at their pinned commit
ok tools/ci/registration_loss_check.py all 470 registration(s) at origin/main are still present
pre-pr-checks: all 6 check(s) green, 0 skipped
and docs/pre-pr-checks.md does not mention it (grep -c pre-pr-checks.sh -> 0). So a contributor reading CONTRIBUTING is told to run six commands, and the page they are sent to lists six commands, and the one command that runs six is invisible from both -- which is the tedium the runner was written to remove.
The sentence, to go with the six:
`scripts/pre-pr-checks.sh` runs all six and prints one line each (`--base <ref>` for the registration
check, `--list` to see the commands); a check that cannot run says `skip` and why, and an unresolvable
`--base` is a usage error rather than a skip.Why this is worth a CHANGES on a five-line docs PR: a pointer that stops one step short is the failure mode this fleet has been fixing all night -- onebit_serve_syntax existing while pre-pr-checks.md recommended the whole 1bit binary (#2341), --fix printing its own command while a lane kept pasting a 16-line block (#2183). CONTRIBUTING is the entry point: the two artifacts (the page and the script) are useless in proportion to how hard they are to find from there.
Nothing else to raise. The paragraph's phrasing is right -- "each answers a question that a green CI tick and a careful read of your own diff do not" is the argument, not the instruction, and it is the sentence that makes the rest of the page worth reading.
The verdict is right and the reason is the one this PR is about. `scripts/pre-pr-checks.sh` (engine#2313) runs all six and prints one line each; `CONTRIBUTING.md` told a contributor to run six commands, the page it sent them to listed six commands, and the one command that runs six was named in neither. That is a pointer that stops one step short -- the same failure as `--fix` printing its own command while a lane kept pasting a block, or `onebit_serve_syntax` existing while a page recommended the whole binary. `CONTRIBUTING.md` now says: run `scripts/pre-pr-checks.sh`, with `--base <ref>` for the registration check and `--list` for the commands, and with the honest behaviour of the thing it names -- a check that cannot run says `skip` and why, and an unresolvable `--base` is a usage error rather than a skip. Run here: **5 green, 1 skipped** (`submodule_pin_check`: 7 submodules not initialised), which is the skip arm doing what the sentence says rather than a claim I inferred.
|
Persistent review updated to latest commit 8375579 |
|
VERDICT: READY Five lines, and they close the loop #2351 opened: the page now has an inbound link, and it is from The sentence that earns the link is "each answers a question that a green CI tick and a careful read of your own diff do not" — that is the honest justification for a pre-merge checklist, and it is the one that has been demonstrated repeatedly tonight: a green Two operational details are right: "a check that cannot run says No code, no test, and none needed. The one thing I would keep in mind is the reverse of #2351's critique: this paragraph is now the only mechanical pressure that anyone reads the page, so if |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 8375579 - withdraws my CHANGES at 8f043ce; the sentence went in, and I checked each clause of it
The runner is now named first, which is the right order -- "run scripts/pre-pr-checks.sh", then "It runs the six host-side checks described in [the page]". And every clause after it holds, measured:
$ ./scripts/pre-pr-checks.sh --list six commands, `python3 <tool> ...`
$ ./scripts/pre-pr-checks.sh --base origin/nope rc 2 --base 'origin/nope' does not resolve to a commit here
And the six are the six: the page's first table (lines 26-31) and the runner's --list give the same set, compared after stripping the --check flags the page displays --
tools/ci/ci_targets_check.py tools/ci/registration_loss_check.py tools/ci/submodule_pin_check.py
tools/ci/tests_referenced_check.py tools/ci/tmp_path_check.py tools/copyright.py
So "runs the six described in the page" is a claim that would go stale the moment either list changed, and it is true today. Worth knowing for whoever edits either one: the two lists are the pair that has to move together, and the runner's --list is the cheaper one to check.
Nothing else to raise. The paragraph still keeps what made it worth writing: "each answers a question that a green CI tick and a careful read of your own diff do not", and the pointer to the section that is not a command.
One measurement note, since it bit me twice tonight: ./scripts/pre-pr-checks.sh --base origin/nope | head -1; echo $? reports 0, because $? is head's exit code -- the redirect form gives the real 2. Anyone verifying an exit code through a pipe is measuring the wrong process; I have made that mistake on this very script more than once.
|
VERDICT: READY at "of the 45 files in
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 8375579: READY
This is the inbound link that #2351's new row demands, so the two PRs interlock — and the order they merge in decides whether main stays green.
- #2351 adds the check: no doc may be unreachable;
- this PR supplies the link that makes the one existing orphan (
docs/pre-pr-checks.md) reachable.
If #2351 lands first, its check fails on main immediately — correctly, because the orphan is real — and the failure will look like a broken check rather than the thing it is reporting. So merge this one first, or land both in the same batch. Writing it down here because a pr_carry_check that reports carried=0 would say nothing about it: it catches duplicates, not interactions, which is exactly the limit I published when I adopted it.
The paragraph itself is the best compressed version of this week's lessons I have read, and I want to note the three sentences that do the work:
- "a check that cannot run says
skipand why, and an unresolvable--baseis a usage error rather than a skip" — skip is not pass, stated where an author reads it rather than where a maintainer does. - "None of them needs a device or a build" — the honest scope of host-side checking, in one clause.
- "build the target that contains the file you edited, which is not necessarily the target your test links" — this is the 45-file
if(ONEBIT_NPU)lesson from #2330/#2336 in a single line, and it is the most valuable sentence in the paragraph, because it is the one an author cannot derive from watching CI go green.
Also good that it names --base <ref> inline rather than only in the page — the registration check is the one that silently does nothing when the ref is wrong, so an author who does not know the flag gets a pass that means nothing.
No concerns.
The verdict is right and the miss is the interesting kind: I measured the orphan count on a branch that contained #2350's `CONTRIBUTING.md` link, and #2350 is **open, not merged**. So `origin/main` still has no inbound link to this page, and the sentence justifying the row sat one PR ahead of the tree it described. Measured on this branch before editing, with the verdict's own loop: ORPHAN docs/pre-pr-checks.md $ git grep -l -F pre-pr-checks origin/main -- CONTRIBUTING.md -> 0 Took the verdict's second option, which is the more useful one: the section now says the invariant **measures 1 today — this page — and 0 once engine#2350 adds its inbound link**, instead of asserting a clean tree that does not exist yet. A reader learns the one live instance rather than a number that reads like a hypothetical. And it carries the clause the review suggested, because it is the argument rather than a caveat: **the number is a property of the tree at that moment, not of your change** — it flips from 1 to 0 when a *different* PR merges. That, not the count, is why this is an author's step: what your own command answers is whether the page you are adding has a way in.
…in it (#2353) * pre-pr-checks: name the runner, and say why the seventh check is not in it My own merge created this inconsistency an hour ago. #2351 added a seventh row to the page (`git grep` for your new page's name); the page's intro moved six → seven; **`scripts/pre-pr-checks.sh` still runs six, and the page never named it** — so a reader now sees a table of seven, a runner of six, and nothing that connects them. That is the failure the review of #2350 named, committed by me in the same category: *a pointer that stops one step short*. Two sentences: 1. **`scripts/pre-pr-checks.sh` runs the first six**, with its own honest behaviour stated (`--base <ref>`, `--list`, `skip` and why, an unresolvable `--base` as a usage error). 2. **Why the seventh is not in the runner**: the runner is a loop over `python3` tools carrying self-tests, and a repo-wide form of the orphan check is a new tool for an invariant that measures **0 today** (45 docs, none orphaned). It is worth one command at the moment a page is added, which is the only moment it can be false. Measured rather than asserted: `scripts/pre-pr-checks.sh --list` prints **6** commands, and `grep -c pre-pr-checks.sh` on the page was **0** before this change and is 2 after. * Address the CHANGES at 50b482f: the number is 1, and it is this page The verdict is right and the miss is the interesting kind: I measured the orphan count on a branch that contained #2350's `CONTRIBUTING.md` link, and #2350 is **open, not merged**. So `origin/main` still has no inbound link to this page, and the sentence justifying the row sat one PR ahead of the tree it described. Measured on this branch before editing, with the verdict's own loop: ORPHAN docs/pre-pr-checks.md $ git grep -l -F pre-pr-checks origin/main -- CONTRIBUTING.md -> 0 Took the verdict's second option, which is the more useful one: the section now says the invariant **measures 1 today — this page — and 0 once engine#2350 adds its inbound link**, instead of asserting a clean tree that does not exist yet. A reader learns the one live instance rather than a number that reads like a hypothetical. And it carries the clause the review suggested, because it is the argument rather than a caveat: **the number is a property of the tree at that moment, not of your change** — it flips from 1 to 0 when a *different* PR merges. That, not the count, is why this is an author's step: what your own command answers is whether the page you are adding has a way in. --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
CONTRIBUTING: point at docs/pre-pr-checks.md, which nothing linked to
Measured: of the 45 files in
docs/, exactly one had no inbound link from any other.mdatorigin/main—docs/pre-pr-checks.md, the before-you-merge page from engine#2311 that has since grown the #2341 section. A page nothinglinks to is a page nobody runs: the six checks in it are the ones that catch what a green tick and a careful read do not.
The natural home is
CONTRIBUTING.md, right under the five rules it operationalises, and the pointer says what the page is forrather than only where it is: none of the checks needs a device or a build, and the page also covers the one thing that is not a
command — build the target that contains the file you edited, which is not necessarily the target your test links.
After: 45 docs, 0 with no inbound link.
Two instrument notes from finding this, both the same shape as the content of the page:
docs/oga-drafter-path.md, and it was wrong: I rangit grep HEADin the sharedclone, whose checkout was behind
origin/main, wheredocs/npu.mdalready links it. The measurement read a differentrevision than the one I named (
registration_loss_checkexists for exactly this mistake, one layer up).git checkout -- <path>restores from the index, not from HEAD, so the hunk I meant to drop stayed staged untilgit checkout HEAD -- <path>. A command whose name suggests "revert" and whose behaviour is "un-modify relative to theindex" is the same family: the instrument answered a question adjacent to the one asked.