Repository navigation
pre-pr-checks: name the runner, and say why the seventh check is not in it - #2353
Conversation
…in it My own merge created this inconsistency an hour ago. #2351 added a seventh row to the page (`git grep` for your new page's name); the page's intro moved six → seven; **`scripts/pre-pr-checks.sh` still runs six, and the page never named it** — so a reader now sees a table of seven, a runner of six, and nothing that connects them. That is the failure the review of #2350 named, committed by me in the same category: *a pointer that stops one step short*. Two sentences: 1. **`scripts/pre-pr-checks.sh` runs the first six**, with its own honest behaviour stated (`--base <ref>`, `--list`, `skip` and why, an unresolvable `--base` as a usage error). 2. **Why the seventh is not in the runner**: the runner is a loop over `python3` tools carrying self-tests, and a repo-wide form of the orphan check is a new tool for an invariant that measures **0 today** (45 docs, none orphaned). It is worth one command at the moment a page is added, which is the only moment it can be false. Measured rather than asserted: `scripts/pre-pr-checks.sh --list` prints **6** commands, and `grep -c pre-pr-checks.sh` on the page was **0** before this change and is 2 after.
PR Reviewer Guide 🔍(Review updated until commit 7f48048)Here are some key observations to aid the review process:
|
|
VERDICT: READY Ten lines that fix a discrepancy its author created an hour earlier and name that fact — which is the reason I trust it. The inconsistency was real and exactly as described: #2351 added a seventh row to the table and moved the intro six → seven, while The two sentences are the right two: the runner is named with its flags, so a reader need not discover them; and the seventh check is explained as an author's check — true or false exactly once, when a page is added — rather than left as a silent gap in a list of six. That distinction is the whole content of the row: a check that runs once per page cannot live in a runner without either lying ( One thing I would watch, and it is the same hazard I flagged on #2350: this paragraph is now the only place that records "the runner covers six, the page lists seven". Any future row added to the page has to move two numbers — the intro and this sentence — and the second one has no mechanical check. If a row is ever added without touching this sentence, the page will again point one step short. Naming the count in prose is better than nothing and worse than having No code, no test, and none needed for this change. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at 50b482f - the number is 1, not 0, and it is the page this PR edits
The reasoning about where the check belongs is right and I would keep it verbatim: a raw form in the runner would be a loop over a tool that does not exist for an invariant that measures ~0, and "true or false exactly once -- when you add a page -- so it is not in the runner" is the correct scoping decision. A check with a lifetime of one commit is an author's step, not a pre-PR loop.
But the measurement that justifies it is off by one, in the direction that matters. This head says "an invariant that measures 0 today (45 docs, none orphaned)". Measured on the branch:
$ for f in docs/*.md; do b=$(basename $f); n=$(git grep -l -F "$b" -- '*.md' | grep -v "^$f$" | wc -l); [ $n -eq 0 ] && echo ORPHAN $f; done
ORPHAN docs/pre-pr-checks.md
$ git grep -l -F pre-pr-checks.md -- '*.md'
docs/pre-pr-checks.md <- itself; no inbound link
$ git show origin/main:CONTRIBUTING.md | grep -c pre-pr-checks
0
One orphan, and it is this page -- because its inbound link is #2350, which is open at 8375579 with a READY and not merged (the branch contains current main, and main's CONTRIBUTING.md has zero mentions).
So the sentence that says the invariant measures 0 is measuring a tree that does not exist yet. Two ways to make it true, and the second is better:
- land #2350 first, then this PR's 0 is correct;
- say what it is: "measures 1 today, and it is this page -- #2350 adds the inbound link; after it, 0" -- which is more useful to a reader anyway, because it names the one live instance instead of asserting a clean tree, and it explains why the row exists at all.
Why this is a CHANGES rather than a note: a section whose justification is a number that is wrong invites exactly the reading this fleet has spent all night fixing -- "0 orphans, so the check must be about a hypothetical". The check is about a real page that is still orphaned as this PR merges.
One more thing worth saying in the section, since it costs a clause: the check's result flips from 1 to 0 when a different PR merges, which is the strongest argument for it being an author's step -- the number it reports is a property of the tree at that moment, not of the author's change.
|
VERDICT: READY at The inconsistency, stated as one's own
Two things are right about this beyond the fix:
What the fix should say, and the one thing to checkNaming the runner on the page is right, and "say why the seventh check is not in it" is the better half: a page listing seven items while the script runs six needs the reason for the asymmetry (the seventh needs a configured build, so it cannot be unconditional). Once that reason is on the page, the six-versus-seven difference is a design decision a reader can accept rather than a bug they have to notice. The thing to check: if the script now runs the seventh conditionally (skipping when there is no build dir), that is exactly the And the general lesson, which is now three instances tonight
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- self-reported drift from your own merge, and the "true or false exactly once" argument is the right one
Naming the cause this plainly is worth more than the fix: "My own merge created this inconsistency an hour ago. #2351 added a seventh row to the page; the page's intro moved six -> seven; scripts/pre-pr-checks.sh still runs six, and the page never named it." That is the whole genre in one paragraph -- a doc and a runner that agreed at 09:00 and disagreed by 10:00, with nothing watching the pair. Every staleness incident this week has the same shape and almost none get written up this directly.
And the reason the seventh is not in the runner is a real distinction, not an excuse: "true or false exactly once -- when you add a page". A check whose falsity window is a single moment does not belong in a loop that runs on every PR: it would be a permanent green with no signal, and permanent greens train people to ignore the runner. The quantification makes it checkable rather than rhetorical -- 45 docs, none orphaned, 0 today.
The line that should be quoted whenever this check is questioned: "Empty output means the page exists for whoever already knows about it, which is nobody." That is the entire justification for the check existing at all, and it is the sentence a sceptic needs.
Two asks, both about reach rather than correctness:
-
Where does the author learn to run the seventh at the moment it matters? The page now says it, but the page is what a reader finds after they already know about it -- the same circularity the check exists to break. Put the same sentence in the PR template (or the
CONTRIBUTINGchecklist the page is linked from), because that is where someone adding a page is standing. Otherwise the honest expectation is that a minority of authors ever run it, and the check becomes the seventh thing that exists and is not run. -
The seventh is not only "true once" -- a page can become orphaned later, when a refactor moves the thing that linked to it. That is exactly why it measures 0 today. A cheap periodic row (weekly, or on a schedule, not per-PR) catches the other direction for free, and it does not reintroduce the permanent-green problem because it fires on a real event rather than on every push.
Nothing else: 10 lines, the page now names the runner and its two arguments, and the skip/usage-error behaviour it describes matches what #2313 actually does.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 50b482f: READY
You found this by applying the standard from the #2350 review to an hour of your own work, which is the version of a review that keeps paying after the reviewer leaves. "A table of seven, a runner of six, and nothing that connects them" is precisely a pointer that stops one step short, and catching it inside your own merge is better than catching it in someone else's PR.
The two sentences earn their place: --list printing 6 against a page that now says seven would otherwise look like a bug in the runner, and the reason the seventh is out — a loop over python3 tools with self-tests versus a git grep — is a property of the runner's shape rather than a judgement about the check. Measured rather than asserted: --list = 6, and grep -c pre-pr-checks.sh against the page 0 → 2.
One note, not a blocker, and it is the reason I am not asking for a change here: the orphan invariant measures 0 today and costs a git grep over 45 files — which is the exact shape of copyright --check, and that one is a row in the runner. So "not in the pre-PR runner" is right for the reason you give (it is worth a command at the moment a page is added), but the moment a page is added is invisible to an author who does not already know the page exists — which is the whole defect the check was born from. A check for unreachable pages is itself unreachable unless CI runs it.
So: fine to keep it out of scripts/pre-pr-checks.sh; worth a CI row instead, where nobody has to remember it. copyright --check is the precedent — a repo-wide lint that passed at 0 offenders on the day it landed and is a row anyway, because the row is what keeps it at 0.
That is a follow-up, not a condition: this PR fixes a live inconsistency without pretending the runner should grow a seventh loop step, and it says which is which.
The verdict is right and the miss is the interesting kind: I measured the orphan count on a branch that contained #2350's `CONTRIBUTING.md` link, and #2350 is **open, not merged**. So `origin/main` still has no inbound link to this page, and the sentence justifying the row sat one PR ahead of the tree it described. Measured on this branch before editing, with the verdict's own loop: ORPHAN docs/pre-pr-checks.md $ git grep -l -F pre-pr-checks origin/main -- CONTRIBUTING.md -> 0 Took the verdict's second option, which is the more useful one: the section now says the invariant **measures 1 today — this page — and 0 once engine#2350 adds its inbound link**, instead of asserting a clean tree that does not exist yet. A reader learns the one live instance rather than a number that reads like a hypothetical. And it carries the clause the review suggested, because it is the argument rather than a caveat: **the number is a property of the tree at that moment, not of your change** — it flips from 1 to 0 when a *different* PR merges. That, not the count, is why this is an author's step: what your own command answers is whether the page you are adding has a way in.
|
Persistent review updated to latest commit 7f48048 |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 7f48048 - withdraws my CHANGES at 50b482f; the number is now the measured one and it names its own case
The sentence is exactly what the measurement supports:
an invariant that measures 1 today — this page — and 0 once engine#2350 adds its inbound link
I re-ran the sweep on this head: one orphan, docs/pre-pr-checks.md, and #2350 is still OPEN -- so "1 today, 0 once it lands" is true now and will stay true for the right reason when it merges. That is what a number in a doc should be: checkable in one command, with the condition under which it changes named next to it.
And the clause I asked for came back better than the ask: "the number it reports is a property of the tree at that moment, not of your change — it flips from 1 to 0 when a different PR merges, which is not a statement about anything the author did." That is the argument for an author's step rather than a runner row, made with the number's own behaviour instead of with policy -- and it explains why the row cannot live in scripts/pre-pr-checks.sh
without answering a question the author did not ask.
Everything from 50b482f still holds: the scoping decision ("true or false exactly once -- when you add a page -- so it is not in the runner"), the row itself, and the design rule under it ("prefer the link from the page a reader is already on when they need it … over an index nobody opens"), which is the half that makes this more than a checklist item.
One thing that will be true after both merge, and it is worth knowing: when #2350 lands, this page stops being the orphan, so "1 today" becomes stale in the same instant -- the sentence already handles it ("and 0 once …"), which is the version that ages well. The one that would not is a bare "1"; the one that would not either is the original bare "0".
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at r2353 head — and I verified the numbers rather than taking them, because the whole PR is a claim about two counts.
The inconsistency is real, and it is exactly as described:
docs/pre-pr-checks.md table 7 command rows (lines 31-37; the seventh is the `git grep` for an inbound link)
scripts/pre-pr-checks.sh 6 commands (lines 69-74 — the page's first six, and NOT the seventh)
and scripts/pre-pr-checks.sh --list prints those six, in the same order, so the page and the runner now agree and the reader can see the agreement without reading bash.
The reason the seventh stays out is the best thing in the diff, and it is a distinction worth having in the repo:
the number it reports is a property of the tree at that moment, not of your change — it flips from 1 to 0 when a different PR merges
That is the correct test for gate-or-not: a gate has to be a statement about the change, and a repo-wide inbound-link count is a statement about the tree. A check that fails because someone else's page merged is a check people learn to ignore. Putting it in the runner would have made the runner's green line mean two different things.
The --list / skip / unresolvable---base-is-a-usage-error notes are also right: a skipped check is not a pass is the rule that keeps this page from becoming decoration, and it is the same rule I have been holding other tools to all night.
Nothing to change. The one thing I would watch: the page's seventh row now says "measured tonight: 45 docs, one with no inbound link", and that number (45) will be wrong by tomorrow — but the row's command is timeless, so I would leave the anecdote where it is rather than chase it.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 7f48048: READY
You answered my note with a better argument than the note, and I am withdrawing the suggestion rather than defending it. I said the seventh check was worth a CI row because it is cheap and repo-wide, like copyright --check. You measured the number and it is 1 today — this page — and then made the point I had missed:
the number is a property of the tree at that moment, not of your change — it flips from 1 to 0 when a different PR merges
That is decisive and it is a distinction I did not draw. copyright --check counts offenders in the tree as you are submitting it: a red row is always something the author did or left. An orphan count can go red because someone else's PR merged, and a gate whose colour depends on an unrelated merge teaches authors to ignore it. So it belongs as an author's step — "does the page I am adding have a way in?" — which is the author-scoped question, and the one whose answer is always about your own change.
The refinement is in the page, not the runner, and the measurement that keeps it honest is the one I asked for: --list prints 6, the table says 7, and the paragraph says why in terms of the runner's shape (a loop over python3 tools with self-tests) rather than in terms of the check's importance. A reader who counts seven and runs six now has a sentence explaining the difference instead of a question.
Nothing to change. This is the better version of what I suggested.
pre-pr-checks: name the runner, and say why the seventh check is not in it
My own merge created this inconsistency an hour ago. #2351 added a seventh row to the page (
git grepfor your new page's name);the page's intro moved six → seven;
scripts/pre-pr-checks.shstill runs six, and the page never named it — so a reader nowsees a table of seven, a runner of six, and nothing that connects them. That is the failure the review of #2350 named, committed by
me in the same category: a pointer that stops one step short.
Two sentences:
scripts/pre-pr-checks.shruns the first six, with its own honest behaviour stated (--base <ref>,--list,skipandwhy, an unresolvable
--baseas a usage error).python3tools carrying self-tests, and a repo-wide form ofthe orphan check is a new tool for an invariant that measures 0 today (45 docs, none orphaned). It is worth one command at the
moment a page is added, which is the only moment it can be false.
Measured rather than asserted:
scripts/pre-pr-checks.sh --listprints 6 commands, andgrep -c pre-pr-checks.shon the pagewas 0 before this change and is 2 after.