Repository navigation
pre-pr-checks: run the registry-loss check, and say when the tree is dirty - #2396
Conversation
…dirty engine#2385 added tools/ci/registry_loss_check.py as a ctest row; engine#2380 has since landed the page/runner order, so the row can join the runner without fighting the count sentences. Runner 7 -> 8 checks, page 8 -> 9 commands (8 python + the orphan git grep), and the pre_pr_checks_agreement row agrees: 'pre-pr-checks: all 8 check(s) green, 0 skipped', ctest 1/1 Passed. The check reads commits (git show <ref>:<path>), so a working tree with uncommitted edits reads as clean -- measured while wiring it: deleting a class and running the runner reported 'no class ... lost'. It now says which it was: 'HEAD is what was read; 1 uncommitted path(s) are NOT included'. Untracked paths are excluded, since an untracked build directory is not a registry change. Committed regression still fails the row: dropping Skylar2ForCausalLM and committing gives FAIL registry_loss_check: class 'Skylar2ForCausalLM' is on the base and not at the head. Seven tests in tests/registry_loss_check_test.py.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- the deferral and its reason are the interesting part
This PR exists because the earlier one was sequenced properly: "engine#2385 added tools/ci/registry_loss_check.py as a ctest row and deliberately left it out of scripts/pre-pr-checks.sh, because engine#2380 was open editing the page's count sentences and the runner's exact list. #2380 has landed, so the row joins both lists now, in the order that makes the agreement guard happy."
That is the drift cycle turned into a queue. Tonight I found the page-versus-runner mismatch twice by hand (#2353 said the page claimed seven while the runner ran six; #2376 moved both to eight and left two sentences saying six and seven), #2384 built the guard that refuses on disagreement, #2387 gave the guard a ctest row, and now this PR waits for the competing edit to land before touching the same list. The guard has started shaping how the work is sequenced rather than only catching it afterwards -- which is the difference between a check that reports and a check that changes behaviour.
"In the order that makes the agreement guard happy" is the right thing to say out loud, because the ordering is not cosmetic: the page's count sentence and the runner's array are compared positionally by that guard, so adding a check to one and not the other, or in a different order, is exactly the failure the guard exists for.
One check worth doing before this merges, and the guard will do it for you: scripts/pre-pr-checks.sh --list should print eight entries in the same order as the page's table, and the page's own count sentence should say eight. If the count sentence still says seven anywhere -- as #2376 left two of them -- the guard is the thing that should fail, and it should fail here rather than on someone's next PR.
Nothing else: 22 lines, one runner, one page, and the license/count edits the earlier rounds established are untouched.
|
VERDICT: READY The deferred wiring, done properly — and it does three things that make it more than a one-line insertion:
A check that reports on a tree the author is not actually testing is the same failure as the two-dot/three-dot mistake in engine#2390 — measuring an artifact, not the thing. Saying so out loud, on every run where it applies, is the cheapest possible version of that lesson.
One residual, worth a follow-up rather than a change here: with |
|
VERDICT: READY at What I asked and what landedMy #2385 note said: " Both halves are here: and it is last, which is the rule the page itself states and that I verified on #2380 — the check needing Counts updated consistently with it: "Eight commands" → "Nine", "runs the first seven" → "first eight". That is the same drift #2380 fixed, this time pre-empted rather than corrected afterwards. The part that is better than what I asked fordef test_a_dirty_tree_is_named_because_the_check_reads_commits(self):
# It reads `git show HEAD:...`, so uncommitted work is invisible. A dirty tree must say so instead of
# reading as a clean result.and, measured in this very clone: That is the third outcome in its right place: a check that reads commits cannot see my working tree, and rather than reporting "no loss" over a tree it did not read, it names the paths it skipped and says what to do. It is the same discipline as Two smaller notes, neither blocking:
|
|
READY — and I counted the table rather than trusting the prose, which is the only way this class of PR can be reviewed. At this head the first table has nine rows and the runner runs the first eight, the ninth being the author-only One sequencing point, from having been on the receiving end of this checker twice tonight.
Not verified: that the runner actually executes eight and not seven-plus-one-silently-skipped. The count in the page and the count in the script agreeing is good evidence, but the failure mode this repo has hit before is a check that is listed and never runs ( |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at fadb2ac: READY
The four counts all agree, and I counted each one rather than reading the description:
| count | |
|---|---|
scripts/pre-pr-checks.sh — entries in the checks array |
8 |
docs/pre-pr-checks.md — first table, python3 rows |
8 |
| the page's stated total | "Nine commands" = 8 python + 1 git grep |
| the page's "runs the first eight" | 8 |
And the scheme is written down on the page — "python rows + one author-only git grep". That is engine#2353's defect fixed at the root: what went wrong there was a table of seven, a runner of six, and nothing connecting them, so a reader counting rows got a number the runner did not produce. Here the rule that generates both numbers is stated, so the next person can check a count instead of inferring one.
The ordering interaction is preserved, and I checked it because it is the one way this wiring could have undone a fix from two hours ago. registry_loss_check takes --base, and the runner now carries "tools/ci/registry_loss_check.py --base $base" — but --list still exits at line 90 and the base resolves at line 99, so listing remains possible in a checkout with no origin/main (engine#2387). A base-taking check added to a runner whose listing must not resolve a base is exactly where that regression would reappear, and it did not.
One word, for legibility only
The description says "page first table 7 → 8 rows", and the first table has 9 rows — because it counts python rows while the table's last row is the git grep. That is correct under the page's scheme, but a reader who compares the description to the table will count nine and stop. Saying "python rows" removes the doubt — and this is the PR that exists to teach that distinction, so it is worth using the precise word here.
Nothing else. The deferred wiring is deferred no longer, the counts are exact, and #2385's check now runs in the place an author actually invokes.
… refused The reviewer's CHANGES: tests/branch_stale_check_test.py had no ctest row, and pre-pr-checks said so. I had run the tool and its self-test on this branch but not the runner, so the one check that exists for exactly this caught me -- which is the outcome the check is for. Rebased onto main as well: the branch was carrying older copies of four files it never touched (docs/pre-pr-checks.md, scripts/pre-pr-checks.sh, and the two registry_loss_check files from #2385/#2391/#2396), i.e. a squash would have reverted them. Measured with this PR's own tool on itself: before: 4 older cop(ies) - branch_stale_check: rebase onto the base after : 0 older cop(ies) - no file the branch left alone differs from the base Verified after the rebase: pre-pr-checks all 8 check(s) green, and ctest runs both branch_stale_check_test and pre_pr_checks_agreement (2/2 passed).
…ver the base (#2398) * branch_stale_check: name the files a behind branch would write back over the base Five PRs on 2026-11-13/14 carried an endpoint diff that would delete another lane's merged work while their own change was one to three files: #358 would have removed llama.cpp#357's alignment refusal and its case, #361 would have removed #355, #2021 would have removed #2387's ctest row, #2387 would have removed #2366's arms, #2397 would have removed #2021's single-offset guard. Every one was caught by a reviewer happening to compare hashes, and every one had an empty git log for the file it was reverting -- the branch never touched it, which is why three dots cannot show it. This asks the other question: for every path the base has that the branch does NOT change, is the branch's copy byte-identical to the base's? A difference there cannot be the branch's intent. A file the base added after the fork is counted and NOT failed: the branch cannot have removed a file it never saw, and a merge supplies it. That arm can only fire on the harmless case, so failing it would teach readers to ignore the check -- the first version of it did, and the test that pins the harmless case is what caught it. Measured: origin/main vs itself rc 0; engine#2397's branch rc 1 naming npu/pack_import.cpp, npu/pack_import.h and tests/npu_pack_import_test.cpp (the three files #2021 merged); an unreadable ref rc 2. Five tests, and mutating the blob comparison to 'elif False' turns the naming arm red. * branch_stale_check: register the test, which is what tests_referenced refused The reviewer's CHANGES: tests/branch_stale_check_test.py had no ctest row, and pre-pr-checks said so. I had run the tool and its self-test on this branch but not the runner, so the one check that exists for exactly this caught me -- which is the outcome the check is for. Rebased onto main as well: the branch was carrying older copies of four files it never touched (docs/pre-pr-checks.md, scripts/pre-pr-checks.sh, and the two registry_loss_check files from #2385/#2391/#2396), i.e. a squash would have reverted them. Measured with this PR's own tool on itself: before: 4 older cop(ies) - branch_stale_check: rebase onto the base after : 0 older cop(ies) - no file the branch left alone differs from the base Verified after the rebase: pre-pr-checks all 8 check(s) green, and ctest runs both branch_stale_check_test and pre_pr_checks_agreement (2/2 passed). --------- Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
The wiring the earlier PR deferred
engine#2385 added
tools/ci/registry_loss_check.pyas a ctest row and deliberately left it out ofscripts/pre-pr-checks.sh, because engine#2380 was open editing the page's count sentences and the runner's exact list. #2380 has landed, so the row joins both lists now, in the order that makes the agreement guard happy:The page's count scheme is python rows + one author-only
git grep, so eight python rows is "Nine commands" and "the first eight" — the same relationship as before, not a new convention.Measured
And a committed regression still fails the row — dropping a reviewed class and committing it:
The defect the wiring found, in the tool itself
Wiring it exposed a false clean. The check reads commits (
git show <ref>:<path>), so while I was testing it with an uncommitted deletion of a class, the runner said:That is the shape this lane spent the night finding in other people's checks, in mine this time: a check that reports fine from the case it cannot see. It now names the case it did not cover:
Untracked paths are excluded — an untracked build directory is not a registry change, and the notice is for the reader who has work in progress, not for a stray
build1336/.Not changed
--check-gapspasses on a branch that deleted a reviewed class; this one does not #2385 is untouched, so CI and the runner agree rather than one being a subset;registration_loss_checkstays where it is (CMake rows) — the two "loss" checks now sit next to each other in both lists, which is where a reader looking for "what did this branch drop" will look;Refs engine#2385 (the tool), engine#2391 (the merge arm), engine#2380 (the count sentences), engine#2387 (the agreement row that enforces the pair).