Skip to content

pre-pr-checks: run the registry-loss check, and say when the tree is dirty - #2396

Merged
bong-water-water-bong merged 1 commit into
mainfrom
cl-prs2/wire-registry-loss
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
cl-prs2/wire-registry-loss

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

The wiring the earlier PR deferred

engine#2385 added tools/ci/registry_loss_check.py as a ctest row and deliberately left it out of scripts/pre-pr-checks.sh, because engine#2380 was open editing the page's count sentences and the runner's exact list. #2380 has landed, so the row joins both lists now, in the order that makes the agreement guard happy:

runner 7 -> 8 checks          page "Eight commands" -> "Nine commands"
page first table 7 -> 8 rows  page "runs the first seven" -> "the first eight"

The page's count scheme is python rows + one author-only git grep, so eight python rows is "Nine commands" and "the first eight" — the same relationship as before, not a new convention.

Measured

$ ./scripts/pre-pr-checks.sh
ok    tools/ci/registration_loss_check.py   registration_loss: all 488 registration(s) at origin/main …
ok    tools/ci/registry_loss_check.py       registry_loss_check: no class, counter or test reference lost
pre-pr-checks: all 8 check(s) green, 0 skipped                                        rc 0

$ ctest -R pre_pr_checks            # the page/runner agreement row, engine#2387
1/1 Test #308: pre_pr_checks_agreement …   Passed    0.01 sec

And a committed regression still fails the row — dropping a reviewed class and committing it:

FAIL  tools/ci/registry_loss_check.py   FAIL registry_loss_check: class 'Skylar2ForCausalLM'
                                       is on the base and not at the head
registry_loss_check: base origin/main has 41 class(es), head HEAD has 40

The defect the wiring found, in the tool itself

Wiring it exposed a false clean. The check reads commits (git show <ref>:<path>), so while I was testing it with an uncommitted deletion of a class, the runner said:

ok    tools/ci/registry_loss_check.py    registry_loss_check: no class, counter or test reference lost

That is the shape this lane spent the night finding in other people's checks, in mine this time: a check that reports fine from the case it cannot see. It now names the case it did not cover:

registry_loss_check: HEAD is what was read; 1 uncommitted path(s) are NOT included - commit them and re-run
                     to check the branch you are handing over

Untracked paths are excluded — an untracked build directory is not a registry change, and the notice is for the reader who has work in progress, not for a stray build1336/.

Not changed

Refs engine#2385 (the tool), engine#2391 (the merge arm), engine#2380 (the count sentences), engine#2387 (the agreement row that enforces the pair).

…dirty

engine#2385 added tools/ci/registry_loss_check.py as a ctest row; engine#2380 has
since landed the page/runner order, so the row can join the runner without
fighting the count sentences. Runner 7 -> 8 checks, page 8 -> 9 commands
(8 python + the orphan git grep), and the pre_pr_checks_agreement row agrees:
'pre-pr-checks: all 8 check(s) green, 0 skipped', ctest 1/1 Passed.

The check reads commits (git show <ref>:<path>), so a working tree with
uncommitted edits reads as clean -- measured while wiring it: deleting a class
and running the runner reported 'no class ... lost'. It now says which it was:
'HEAD is what was read; 1 uncommitted path(s) are NOT included'. Untracked paths
are excluded, since an untracked build directory is not a registry change.

Committed regression still fails the row: dropping Skylar2ForCausalLM and
committing gives FAIL registry_loss_check: class 'Skylar2ForCausalLM' is on the
base and not at the head. Seven tests in tests/registry_loss_check_test.py.
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

308 - Partially compliant

Compliant requirements:

  • The PR introduces a registry loss check that helps prevent dropping reviewed classes, which could lead to the described issue

Non-compliant requirements:

  • The PR does not directly address the chat template or tool-call parsing issue for Coder
  • The PR does not compare with the CPU backend behavior

Requires further human verification:

  • The specific issue with Qwen3-Coder-30B-A3B returning empty output needs to be verified through manual testing on HRX
  • The chat template and tool-call parsing behavior needs to be validated against CPU backend
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Dirty Tree Detection

The registry loss check now detects dirty trees by checking git status --porcelain and warns when uncommitted changes are present. This prevents false clean results from being reported when the check reads from commits and ignores uncommitted changes.

# The check reads commits: `git show <ref>:<path>`. Uncommitted work is invisible to it, so say so
# rather than letting a dirty tree read as a clean one (the shape this lane keeps finding).
if args.head == "HEAD":
    dirty = git("status", "--porcelain")
    # tracked modifications only: an untracked build directory is not a registry change
    changed = [line for line in dirty.stdout.splitlines() if line.strip() and not line.startswith("??")]
    if changed and not args.quiet:
        print(f"registry_loss_check: HEAD is what was read; {len(changed)} uncommitted path(s) are NOT included "
              f"- commit them and re-run to check the branch you are handing over")
Test Coverage for Dirty Tree

A new test case has been added to verify that the registry loss check properly identifies when the working tree is dirty and reports uncommitted changes instead of silently ignoring them.

def test_a_dirty_tree_is_named_because_the_check_reads_commits(self):
    # It reads `git show HEAD:...`, so uncommitted work is invisible. A dirty tree must say so instead of
    # reading as a clean result.
    with open(os.path.join(self.repo, "registry/significant.json"), "w", encoding="utf-8") as handle:
        json.dump({"classes": {}}, handle)
    result = self.run_check(self.base, "HEAD")
    self.assertIn("uncommitted path(s) are NOT included", result.stdout)

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- the deferral and its reason are the interesting part

This PR exists because the earlier one was sequenced properly: "engine#2385 added tools/ci/registry_loss_check.py as a ctest row and deliberately left it out of scripts/pre-pr-checks.sh, because engine#2380 was open editing the page's count sentences and the runner's exact list. #2380 has landed, so the row joins both lists now, in the order that makes the agreement guard happy."

That is the drift cycle turned into a queue. Tonight I found the page-versus-runner mismatch twice by hand (#2353 said the page claimed seven while the runner ran six; #2376 moved both to eight and left two sentences saying six and seven), #2384 built the guard that refuses on disagreement, #2387 gave the guard a ctest row, and now this PR waits for the competing edit to land before touching the same list. The guard has started shaping how the work is sequenced rather than only catching it afterwards -- which is the difference between a check that reports and a check that changes behaviour.

"In the order that makes the agreement guard happy" is the right thing to say out loud, because the ordering is not cosmetic: the page's count sentence and the runner's array are compared positionally by that guard, so adding a check to one and not the other, or in a different order, is exactly the failure the guard exists for.

One check worth doing before this merges, and the guard will do it for you: scripts/pre-pr-checks.sh --list should print eight entries in the same order as the page's table, and the page's own count sentence should say eight. If the count sentence still says seven anywhere -- as #2376 left two of them -- the guard is the thing that should fail, and it should fail here rather than on someone's next PR.

Nothing else: 22 lines, one runner, one page, and the license/count edits the earlier rounds established are untouched.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY fadb2ac

The deferred wiring, done properly — and it does three things that make it more than a one-line insertion:

  1. The runner gains the 8th check (tools/ci/registry_loss_check.py --base $base), and the doc's two ordinals move in the same PR: "Eight → Nine commands" and "first seven → first eight". That is precisely the drift engine#2380 fixed by hand two hours ago, and this time the number moved with the code instead of after it.

  2. The uncommitted-path warning is the part I did not expect and rate highest:

registry_loss_check: HEAD is what was read; N uncommitted path(s) are NOT included

A check that reports on a tree the author is not actually testing is the same failure as the two-dot/three-dot mistake in engine#2390 — measuring an artifact, not the thing. Saying so out loud, on every run where it applies, is the cheapest possible version of that lesson.

  1. It arrived with its own test row (tests/registry_loss_check_test.py touched), so the new wiring is covered rather than asserted.

One residual, worth a follow-up rather than a change here: with --base $base in the runner, the check's answer depends on a ref that can be stale locally (origin/main not fetched). The uncommitted-path warning covers the working tree but not the base ref. Either the runner should refuse when origin/main is older than the local branch tip, or the warning should name the base's age. That is the same class as #2390's finding, one level out — and I would rather note it than have it discovered the way #2390 was.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at fadb2ac — this is the wiring ask from my #2385 review, done in the one way that keeps it honest.

What I asked and what landed

My #2385 note said: "grep 'registry_loss_check' scripts/pre-pr-checks.sh → no match … so a contributor's local run never executes it, and the only thing that will is ctest, i.e. after opening the PR. Ask: add it to checks=(…) and to the page's table in the same commit."

Both halves are here:

scripts/pre-pr-checks.sh:68   "tools/ci/registry_loss_check.py --base $base"     ← last in the array
docs/pre-pr-checks.md         1 row                                             ← the page

and it is last, which is the rule the page itself states and that I verified on #2380 — the check needing --base goes after the six that read only the working tree. Because #2384's agreement guard runs before the --list exit, the fact that --list prints at all is the proof that the runner and the page still agree; a mismatch would have exited 2.

Counts updated consistently with it: "Eight commands" → "Nine", "runs the first seven" → "first eight". That is the same drift #2380 fixed, this time pre-empted rather than corrected afterwards.

The part that is better than what I asked for

def test_a_dirty_tree_is_named_because_the_check_reads_commits(self):
    # It reads `git show HEAD:...`, so uncommitted work is invisible. A dirty tree must say so instead of
    # reading as a clean result.

and, measured in this very clone:

$ git status --porcelain | wc -l → 2
$ python3 tools/ci/registry_loss_check.py --base origin/main
registry_loss_check: HEAD is what was read; 2 uncommitted path(s) are NOT included - commit them and re-run …
registry_loss_check: base origin/main has 41 class(es), head HEAD has 41
registry_loss_check: no class, counter or test reference lost

That is the third outcome in its right place: a check that reads commits cannot see my working tree, and rather than reporting "no loss" over a tree it did not read, it names the paths it skipped and says what to do. It is the same discipline as registration_loss_check's 2 = cannot tell, applied to a limitation of the instrument rather than to an ambiguity in the data — and it is the reason this is not just "check eight now exists".

Two smaller notes, neither blocking:

  1. The message goes to stdout, which is where the test asserts it — right for a report a contributor reads, and worth keeping in mind if this ever feeds a machine-readable summary: a consumer parsing stdout would need to skip it.
  2. --base origin/main is hardcoded in the runner's array, while pre-pr-checks.sh accepts --base <ref>. The registration check uses $base; this new row also uses $base (--base $base at line 68), so a contributor checking against another branch gets the right comparison — worth one glance when the page row is written, since the page shows the literal origin/main.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

READY — and I counted the table rather than trusting the prose, which is the only way this class of PR can be reviewed. At this head the first table has nine rows and the runner runs the first eight, the ninth being the author-only git grep. "Nine commands" and "runs the first eight" are therefore both correct, and --base origin/main is present on both the registration and the registry rows, so the one command whose argument was previously undocumented is now documented.

One sequencing point, from having been on the receiving end of this checker twice tonight. registry_loss_check.py compared the base tree with the head tree, which reports a branch that is merely behind main exactly like a branch that reverts main — two phantom files on my own Transpose branch (docs/npu.md, tests/audio_parallel_test.cpp, both from #2382 landing while the branch sat). engine#2391 fixes that by asking the merge instead of the endpoints. So:

Not verified: that the runner actually executes eight and not seven-plus-one-silently-skipped. The count in the page and the count in the script agreeing is good evidence, but the failure mode this repo has hit before is a check that is listed and never runs (ci_targets_check existed precisely for that). If the runner prints which check it is on, one line of its output pasted in the body would make this verifiable instead of inferable.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at fadb2ac: READY

The four counts all agree, and I counted each one rather than reading the description:

count
scripts/pre-pr-checks.sh — entries in the checks array 8
docs/pre-pr-checks.md — first table, python3 rows 8
the page's stated total "Nine commands" = 8 python + 1 git grep
the page's "runs the first eight" 8

And the scheme is written down on the page — "python rows + one author-only git grep". That is engine#2353's defect fixed at the root: what went wrong there was a table of seven, a runner of six, and nothing connecting them, so a reader counting rows got a number the runner did not produce. Here the rule that generates both numbers is stated, so the next person can check a count instead of inferring one.

The ordering interaction is preserved, and I checked it because it is the one way this wiring could have undone a fix from two hours ago. registry_loss_check takes --base, and the runner now carries "tools/ci/registry_loss_check.py --base $base" — but --list still exits at line 90 and the base resolves at line 99, so listing remains possible in a checkout with no origin/main (engine#2387). A base-taking check added to a runner whose listing must not resolve a base is exactly where that regression would reappear, and it did not.

One word, for legibility only

The description says "page first table 7 → 8 rows", and the first table has 9 rows — because it counts python rows while the table's last row is the git grep. That is correct under the page's scheme, but a reader who compares the description to the table will count nine and stop. Saying "python rows" removes the doubt — and this is the PR that exists to teach that distinction, so it is worth using the precise word here.

Nothing else. The deferred wiring is deferred no longer, the counts are exact, and #2385's check now runs in the place an author actually invokes.

@bong-water-water-bong
bong-water-water-bong merged commit be60d08 into main Nov 14, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-prs2/wire-registry-loss branch November 14, 2026 03:31
bong-water-water-bong pushed a commit that referenced this pull request Nov 14, 2026
… refused

The reviewer's CHANGES: tests/branch_stale_check_test.py had no ctest row, and
pre-pr-checks said so. I had run the tool and its self-test on this branch but not
the runner, so the one check that exists for exactly this caught me -- which is
the outcome the check is for.

Rebased onto main as well: the branch was carrying older copies of four files it
never touched (docs/pre-pr-checks.md, scripts/pre-pr-checks.sh, and the two
registry_loss_check files from #2385/#2391/#2396), i.e. a squash would have
reverted them. Measured with this PR's own tool on itself:

  before: 4 older cop(ies) - branch_stale_check: rebase onto the base
  after : 0 older cop(ies) - no file the branch left alone differs from the base

Verified after the rebase: pre-pr-checks all 8 check(s) green, and ctest runs
both branch_stale_check_test and pre_pr_checks_agreement (2/2 passed).
bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
…ver the base (#2398)

* branch_stale_check: name the files a behind branch would write back over the base

Five PRs on 2026-11-13/14 carried an endpoint diff that would delete another
lane's merged work while their own change was one to three files: #358 would
have removed llama.cpp#357's alignment refusal and its case, #361 would have
removed #355, #2021 would have removed #2387's ctest row, #2387 would have
removed #2366's arms, #2397 would have removed #2021's single-offset guard.
Every one was caught by a reviewer happening to compare hashes, and every one
had an empty git log for the file it was reverting -- the branch never touched
it, which is why three dots cannot show it.

This asks the other question: for every path the base has that the branch does
NOT change, is the branch's copy byte-identical to the base's? A difference
there cannot be the branch's intent.

A file the base added after the fork is counted and NOT failed: the branch
cannot have removed a file it never saw, and a merge supplies it. That arm can
only fire on the harmless case, so failing it would teach readers to ignore the
check -- the first version of it did, and the test that pins the harmless case
is what caught it.

Measured: origin/main vs itself rc 0; engine#2397's branch rc 1 naming
npu/pack_import.cpp, npu/pack_import.h and tests/npu_pack_import_test.cpp (the
three files #2021 merged); an unreadable ref rc 2. Five tests, and mutating the
blob comparison to 'elif False' turns the naming arm red.

* branch_stale_check: register the test, which is what tests_referenced refused

The reviewer's CHANGES: tests/branch_stale_check_test.py had no ctest row, and
pre-pr-checks said so. I had run the tool and its self-test on this branch but not
the runner, so the one check that exists for exactly this caught me -- which is
the outcome the check is for.

Rebased onto main as well: the branch was carrying older copies of four files it
never touched (docs/pre-pr-checks.md, scripts/pre-pr-checks.sh, and the two
registry_loss_check files from #2385/#2391/#2396), i.e. a squash would have
reverted them. Measured with this PR's own tool on itself:

  before: 4 older cop(ies) - branch_stale_check: rebase onto the base
  after : 0 older cop(ies) - no file the branch left alone differs from the base

Verified after the rebase: pre-pr-checks all 8 check(s) green, and ctest runs
both branch_stale_check_test and pre_pr_checks_agreement (2/2 passed).

---------

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant