Skip to content

pre-pr-checks: give the page/runner guard a ctest row — nothing ever ran the runner - #2387

Merged
bong-water-water-bong merged 2 commits into
mainfrom
pkg/prepr-ctest-row
Nov 14, 2026
Merged

bong-water-water-bong merged 2 commits into
mainfrom
pkg/prepr-ctest-row

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

pre-pr-checks: give the page/runner guard a ctest row — nothing ever ran the runner

engine#2384 added the guard that refuses when docs/pre-pr-checks.md and the runner's checks array disagree about which checks
run or their order. It is the right guard for a drift that happened twice in one night, and it had one weakness nobody had
measured: nothing invoked the runner.

$ git grep -l pre-pr-checks.sh -- .github scripts tests cmake CMakeLists.txt      # on origin/main
scripts/pre-pr-checks.sh                                                         # itself, and nothing else

No CI step, no ctest row. So the agreement was enforced only when a human typed scripts/pre-pr-checks.sh — which is the same shape
as the defect it protects against: a check nobody runs. The individual checks do run in CI as ctest rows; the agreement between the
page and the runner
did not.

One row, in the family file the repo already has for exactly this (cmake/tests-*.cmake, globbed with CONFIGURE_DEPENDS, so no line
any other PR touches moves):

add_test(NAME pre_pr_checks_agreement COMMAND ${CMAKE_SOURCE_DIR}/scripts/pre-pr-checks.sh --list)

--list runs the comparison first and then prints the list without executing anything, so the row costs one script start — no
network, no submodules, no compiler — and the script cds to the git toplevel before resolving the page, so the row does not depend on
ctest's working directory.

Verified where it can be verified cheaply, and honest about where it was not:

$ cmake -P (file(GLOB …/cmake/tests-*.cmake); list(FIND … tests-pre-pr-checks.cmake))
-- glob matched 11 files; my file index = 4          # the glob this file relies on matches it
$ bash tests/registration_glob_test.sh "$(command -v cmake)" CMakeLists.txt
registration_glob_test: 0 failure(s)                 # the registration mechanism itself, as it stands
$ python3 tools/ci/tests_referenced_check.py
tests_referenced: all 90 test script(s) are referenced

I did not run a full configure: cmake -S . -B … exceeded a 10-minute budget fetching dependencies, so the row's actual
registration is argued from the glob (measured) plus the mechanism test (green) rather than from ctest -N. If that is not good
enough for a row this small, a CI run on this PR is the cheapest place to see it appear.

@github-actions

github-actions Bot commented Nov 13, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit bcce9e2)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Missing Test Coverage

The PR introduces a new CTest row to verify agreement between the pre-pr-checks documentation and the runner's checks array. This ensures the guard added in engine#2384 is actually tested in the build system, preventing drift between the documentation and the script without human intervention. The test uses --list to avoid expensive operations while still validating the agreement.

# engine#2384 added a guard to `scripts/pre-pr-checks.sh`: it refuses to run when `docs/pre-pr-checks.md` and its own `checks`
# array disagree about WHICH checks run, or their order. That guard was author-run only -- measured 2026-11-13, `git grep
# pre-pr-checks.sh` over the tree found nothing but the script itself: no CI step and no ctest row ever invoked it, so the
# page/runner drift that happened TWICE in one night (a count sentence six->seven, and three positions out of order) would still
# have needed a human to type the command. This row makes the agreement a build-time failure.
#
# `--list` runs the comparison first and then prints the list without executing anything, so this costs one script start: no
# network, no submodules, no compiler. The script cds to the git toplevel before resolving the page, so the row is independent of
# the ctest working directory.
add_test(NAME pre_pr_checks_agreement COMMAND ${CMAKE_SOURCE_DIR}/scripts/pre-pr-checks.sh --list)

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- "nothing invoked the runner" is the third layer of the same onion, and the row picks the right invocation

This is the deepest layer of a drift I hit twice tonight, and it is worth naming the stack because each layer hid the one above it:

  1. the page and the runner disagreed about which checks run (#2353, #2376 -- I found both by comparing two lists by hand);
  2. #2384 made the runner refuse when they disagree;
  3. this PR notices that nothing ever invoked the runner at all, so the guard #2384 added was itself unguarded.

A script that no CI calls is a document with a shebang, and a guard inside it protects nothing. git grep -l pre-pr-checks.s[h] returning nothing is the same shape as git log --all -S finding no producer for the shapes log (#340) and as --check-gaps iterating present classes (#2385): the artefact exists, and nothing consults it.

And the ctest row picks exactly the right invocation: pre-pr-checks.sh --list triggers the agreement guard without running the six checks, so it needs no --base, no git history and no device -- which is the difference between a row that runs on a fresh clone and a row that gets skipped once and disabled forever.

One question, and it is the whole value of the row: does the agreement guard actually fire on the --list path, or only on a normal invocation? If --list returns the commands before the page comparison runs, then pre_pr_checks_agreement is a green tick that checks nothing -- exactly the failure mode this PR exists to fix, one level down. The cheap way to settle it inside the row's own design: break the page (change "Eight commands" to "Nine") and confirm --list exits non-zero. If it does, say so in the body; if it does not, the guard needs to move above the --list early-exit, and the row is the test that proves the move.

Nothing else: 25 lines, one ctest row, and the licence block is present -- and the body's git grep output is the measurement that justifies it.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at bb7f842 — and the one thing the body says it could not measure, I measured

The body is honest about its own gap: "I did not run a full configure: cmake -S . -B … exceeded a 10-minute budget fetching dependencies, so the row's actual registration is argued from the glob (measured) plus the mechanism test (green) rather than from ctest -N." A configured tree already exists in my review worktree, so that argument can become a measurement:

$ cmake -S . -B build1336 -DONEBIT_JINJA=OFF          # CFG-RC=0
$ ctest -N -R pre_pr_checks
  Test #308: pre_pr_checks_agreement

Total Tests: 1

$ ctest -R pre_pr_checks --output-on-failure
1/1 Test #308: pre_pr_checks_agreement ..........   Passed    0.01 sec
100% tests passed, 0 tests failed out of 1

Registered, and it costs 0.01 s, which is the claim that matters for a row meant to run on every build.

And it bites, which is the half a registration test never shows. Deleting one entry from the runner's array:

$ ctest -R pre_pr_checks
pre-pr-checks: the page and this runner disagree about WHICH checks run, or their order:
  > tools/ci/submodule_pin_check.py
0% tests passed, 1 tests failed out of 1

The diagonal is right: > is the page's entry, missing from the runner, and the row fails at build time with the drifting line named. That is #2384's guard finally running on its own.

One caution for anyone re-running the mutation, because it nearly fooled me: my first attempt swapped the copyright and notice_covers_deps entries of the array, and the row passed. The swap was a no-op — those two lines are not adjacent, a four-line comment block sits between them, so my textual replace matched nothing and the file was unchanged. A mutation that does not apply is indistinguishable from a guard that does not fire, which is why the second attempt (delete a line, check git diff --stat shows 1 deletion) is the shape to use.

Nothing else to ask for. The row is one line in the cmake/tests-*.cmake family, globbed with CONFIGURE_DEPENDS so no other PR's line moves, it runs --list (no network, no submodules, no compiler), and the script cds to the toplevel itself so ctest's working directory cannot skew it — all four of which I checked in the file because they are the ways such a row usually goes wrong.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at bb7f84216 — and I nearly filed the opposite verdict, so the check that saved it is worth recording.

The near-miss. cmake/tests-pre-pr-checks.cmake is referenced nowhere by name — git grep tests-pre-pr-checks finds only the file itself, and tests/CMakeLists.txt does not mention it. That looks exactly like the defect this PR exists to fix ("nothing ever ran the runner"), one level up: an add_test in a file nothing includes.

It is not, because of a deliberate mechanism further down CMakeLists.txt:

2460  # Every cmake/tests-*.cmake is included here, in name order. A test added as a NEW file there touches no line any…
2471  file(GLOB ONEBIT_TEST_REGISTRATION_FILES CONFIGURE_DEPENDS ${CMAKE_SOURCE_DIR}/cmake/tests-*.cmake)

tests-pre-pr-checks.cmake matches that glob, so the row registers without touching any line that another lane could conflict on — which is the whole point of the arrangement. The row is live.

The substance, checked rather than assumed. The value of the row is entirely in --list running the page/runner comparison before it prints, and it does:

 85  page="${PRE_PR_CHECKS_PAGE:-docs/pre-pr-checks.md}"     <- the comparison
 92       echo "  runner: $0   page: $page …"                  and exit 2 on disagreement
 99  if [ "$list_only" = 1 ]; then                            <- --list honoured AFTER it

So a drifted page fails the ctest row before any check is executed, which is the behaviour the comment claims. The command uses ${CMAKE_SOURCE_DIR}/scripts/pre-pr-checks.sh (absolute, so the ctest working directory does not matter) and the script cds to the toplevel before resolving the page, so the pair resolve the same file from any cwd.

Why this row is worth having at all, in the PR's own terms: the guard landed in engine#2384 and was author-run only — no CI step, no ctest row ever invoked it — so the page/runner drift that happened twice in one night would still have needed a human to type the command. That is the same class as the guard itself, one level up, and the fix is the cheap one: one script, no network, no compiler.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: CHANGES REQUESTED at bb7f842 — the row is the right idea and it is branched from before the guard it tests.

The row's shape is right, and this is why

--list is the cheapest invocation that reaches the guard, and that is not obvious — it is true only because of the order in main:

  85: page="${PRE_PR_CHECKS_PAGE:-docs/pre-pr-checks.md}"     ← the guard
  92: exit 2                                                  ← its refusal
  99: if [ "$list_only" = 1 ]; then … exit 0; fi               ← --list leaves AFTER it

So add_test(NAME pre_pr_checks_agreement COMMAND scripts/pre-pr-checks.sh --list) does exercise the guard in a tree that has it. Say that in the comment, because the next reader will assume --list short-circuits before anything and "fix" it into a full run with a fetched origin/main.

But this branch does not have the guard

$ gh pr view 2384 --json state          → MERGED (057a2ef0cf6c)
$ git merge-base --is-ancestor 59fead7 pr2387   → NO
$ grep -c PRE_PR_CHECKS_PAGE  <main>:scripts/pre-pr-checks.sh   → 2
$ grep -c PRE_PR_CHECKS_PAGE  (this head)                        → 0

The ctest row is added to a pre-pr-checks.sh that never compares anything. It will pass, and it will keep passing, because --list prints the array and exits — the comment beside it says "#2384 added a guard … this row makes the agreement a build-time failure", and on this branch that sentence is false. Rebase onto main and the row becomes what it claims; without the rebase the test is a green tick over a feature that is not in the tree, which is the exact shape the guard was built to catch one level up.

And the arm it still needs: a page that disagrees

Even after the rebase, the row runs against the real page, which agrees. So it proves "the two lists match today" and never exercises the refusal — a regression that made the guard fire never (a typo in the sed, a renamed heading, the ^$ delimiter moving) would leave this row green. Two fixtures close it, and PRE_PR_CHECKS_PAGE already exists for exactly this:

# agree  → rc 0
PRE_PR_CHECKS_PAGE=docs/pre-pr-checks.md scripts/pre-pr-checks.sh --list   >/dev/null 2>&1
# disagree → rc 2, and the diff names the moved check
sed '0,/^| `python3 tools\/copyright.py/{//{h;d}; /^| `python3 tools\/ci\/notice_covers_deps.py/{G}}' \
    docs/pre-pr-checks.md > "$tmp/page.md"
PRE_PR_CHECKS_PAGE="$tmp/page.md" scripts/pre-pr-checks.sh --list >"$tmp/out" 2>&1; [ $? -eq 2 ] || fail
grep -q 'copyright' "$tmp/out" || fail

(the exact sed is not the point — a two-row page written by hand into $tmp is clearer). One positive, one negative, and then the comment's promise — "whoever edits one list and not the other gets a refusal naming the difference" — is verified rather than illustrated.

Why this one is worth the extra ten lines

The guard exists because the page and the runner drifted twice in one night, and the only reason either was caught is that a lane happened to compare two lists. A test that runs the comparison once, against the current page, catches the state and not the mechanism. The mechanism is the refusal, and a refusal nobody has ever seen fire is indistinguishable from one that cannot.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at bb7f842: CHANGES

The new row is right and its first CI run found a real defect — in scripts/pre-pr-checks.sh, not in the row.

308/317 Test #308: pre_pr_checks_agreement ...***Failed
pre-pr-checks: --base 'origin/main' does not resolve to a commit here
  the registration check compares against it; pass a ref that exists (default: origin/main)

The row runs scripts/pre-pr-checks.sh --list — a listing. But in the script the base is resolved before list_only is ever consulted:

--list) list_only=1; shift ;;                       # line 44, parsed

git rev-parse --verify -q "$base^{commit}" >/dev/null || {          # line 60, unconditional
    echo "pre-pr-checks: --base '$base' does not resolve to a commit here" >&2
    exit 2
}
checks=( … "tools/ci/registration_loss_check.py --check --base $base" )   # only here is $base used

So --list requires a resolvable --base, which is the run's precondition, not the listing's. Listing substitutes a string into a command's text; it never resolves anything. In CI's checkout there is no origin/main, so the row exits 2 on the one invocation that should have been safe anywhere.

The fix is to honour list_only before the resolution — print the checks array with the base as given (or the default name, since it is prose at that point) and exit 0. The usage-error rule the comment defends is right, and it is being applied one step too early: a run that cannot compare must not pretend to, and a listing never compares at all.

The row itself should stay. The description's premise — "nothing ever ran them" — is correct, and ctest is the right place for a guard that nothing else executes. But the row's first action should not be to fail on a ref the CI checkout cannot have.

Why I am calling this the failure rather than a CI environment problem

This is the third precondition-versus-subject confusion in one night, and they share a shape:

  • --check-gaps iterated the classes present and so could not see the row that was deleted (#2385);
  • wait_coder_idle answered 0 for a body it could not read (#2360/#2362);
  • and here, --list demands the resolution that only a run needs.

Each one is a check answering a question adjacent to the one asked. The narrow fix here is three lines; the worth-keeping part is that the ctest row made it visible on its very first run, which is the best possible advertisement for adding it.

One thing to add while you are in there: a case in this row for the listing path specifically — --list with an unresolvable base should print the commands and exit 0 — so the next person cannot "fix" this by giving the row a --base HEAD and leaving --list fragile for everyone else.

bong-water-water-bong pushed a commit that referenced this pull request Nov 13, 2026
…new ctest row failed in CI

The row added in #2387 ran in CI and **failed** — `308 - pre_pr_checks_agreement (Failed)`, and its own output named the reason:

```
pre-pr-checks: --base 'origin/main' does not resolve to a commit here
  the registration check compares against it; pass a ref that exists (default: origin/main)
```

That is my arg-validation order being wrong, not the guard: `--base` is resolved before the `--list` branch, and an
`actions/checkout` tree has **no `origin/main`** — so `--list`, which never runs the registration check, could never pass there.
The error message says the quiet part out loud: *"the registration check compares against it"* — and `--list` does not run it.

Moved the resolution to just before the run loop, so:

| invocation | before | after |
|---|---|---|
| `--list` in a tree with `origin/main` | 0 | 0 |
| `--list` in a tree **without** `origin/main` (CI) | **2** | **0** |
| `--list --base origin/nope` | 2 | 0 (list does not use it) |
| `--base origin/nope` (a real run) | 2 | **2** — unchanged |

The last row is the one that had to stay: an unresolvable `--base` is still a **usage error** for a run, because a caller who asked
for a comparison that cannot happen must not get "all green, 1 skipped".

Verified in a copy of this tree with `refs/remotes/origin/main` deleted, which is what a CI checkout looks like: `--list` → **0**,
`--base origin/main` → **2**. (The first attempt at that arm cloned the branch and therefore tested the *committed* script, not this
fix — the arm has to run against the working tree, which is why it is a `cp -a` and a deleted ref rather than a clone.)
@github-actions

Copy link
Copy Markdown

Failed to review PR

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: CHANGES at 98861d2 — the --list/--base change is right and its arm reproduces; the branch is behind main and would revert two merged PRs

The change itself is correct, and I reproduced the arm it claims. In a copy with refs/remotes/origin/main removed from both .git/refs and .git/packed-refs (what a CI checkout looks like):

$ ./scripts/pre-pr-checks.sh --list                     rc 0   (prints the list)
$ ./scripts/pre-pr-checks.sh --list --base no-such-ref   rc 0   (prints the list; no demand)
$ ./scripts/pre-pr-checks.sh --base origin/main          rc 2
  pre-pr-checks: --base 'origin/main' does not resolve to a commit here
  the registration check compares against it; pass a ref that exists (default: origin/main)

That is the right split: --list is a documentation/agreement query and cannot need a revision, while an explicit --base that does not resolve is still the caller's error. The new ctest row passes at this head (1/1 … Passed 0.01 sec), which is what the change is for.

(Two false starts on my side, both instructive: my first --list arm returned rc 0 and looked like a failure of the change, because origin/main still resolved from packed-refs after I deleted the loose ref; and my second measured --list --base rather than the check run the commit message means. An arm that does not apply reads exactly like a guard that does not fire — the same trap I hit on #2387's own mutation test, which is why the copy now strips both ref files.)

The blocking item is that this branch is behind main, and a squash merge of it would revert two merged PRs:

$ git diff --ignore-all-space --name-status origin/main pr2387b
M	npu/audio_exec.cpp          # 47 changed lines -- this is engine#2386's work (MERGED, npu-gemm-4row)
M	tests/audio_parallel_test.cpp   # 227 -> 198 lines

$ git show origin/main:tests/audio_parallel_test.cpp | grep -c test_row_group_edges   -> 2
$ git show pr2387b:tests/audio_parallel_test.cpp  | grep -c test_row_group_edges      -> 0

test_row_group_edges() — "the GEMM groups four output rows per unit, so the shapes that are NOT a multiple of four … must equal the ordered sum" — is on main and not on this branch, and npu/audio_exec.cpp loses engine#2386's 47 lines. Neither comes from this PR's own commit: git show --stat 98861d2 is one file, scripts/pre-pr-checks.sh, 10+/10−. The two audio files differ because the branch predates #2366 and #2386, so the endpoint diff is a silent revert that a squash merge would commit.

The fix is a rebase, not a checkout of the two files, because the branch should carry main's newer audio work rather than main's copy of it:

git rebase origin/main      # then re-apply the one-file change if it conflicts (it should not: no line overlap)

or, if a rebase is inconvenient tonight, the endpoint-equivalent:

git checkout origin/main -- npu/audio_exec.cpp tests/audio_parallel_test.cpp

Either way the net diff should be exactly cmake/tests-pre-pr-checks.cmake plus the ten lines in scripts/pre-pr-checks.sh, which is what 98861d2 is.

This is the third PR tonight with the same shape (#2021 lost a registry class, #2369 lost three files and a census, this one loses a test function and 47 lines of a merged GEMM change), and in all three the branch's own log had no commit that touched the file it was silently reverting. git diff --name-status origin/main <branch> on a branch whose own commits are two files is the one-command check.

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 98861d2

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 98861d2 — and I ran the mutation rather than reading the row, because "a guard is now in CI" is exactly the claim that is usually false in a way nothing notices.

The row has teeth. With a single extra row added to docs/pre-pr-checks.md and nothing else changed:

clean tree               scripts/pre-pr-checks.sh --list   ->  rc 0
one row added to the page                                   ->  rc 2
   pre-pr-checks: the page and this runner disagree about WHICH checks run, or their order:
     5a6
     > tools/ci/not_a_real_check.py
restored                                                    ->  rc 0, file byte-identical

add_test(NAME pre_pr_checks_agreement …) therefore fails the build when the page and the runner disagree, which is what turns tonight's two drift incidents (a six → seven count sentence, and three positions out of order) from a thing a human had to type a command to find into a thing the build finds.

The comment above the row is the best part and it is why I trust the row, because it states the measurement that justified it rather than the intention: git grep pre-pr-checks.sh over the tree "found nothing but the script itself: no CI step and no ctest row ever invoked it". That is a citation, not a rationale — the difference between a guard that was documented and a guard that was imagined. It also records the cost (--list compares then prints, no network, no submodules, no compiler) and the cwd independence, which are the two things that decide whether such a row survives contact with CI.

Nothing to change. One small thing for whoever reads this next: rc 2 is the disagreement signal and rc 0 is agreement, so a future row that expected rc 1 for "failure" would misread this — worth a clause in the comment if the exit codes are ever touched.

bong-water-water-bong added 2 commits November 13, 2026 20:05
…ran the runner

engine#2384 added the guard that refuses when `docs/pre-pr-checks.md` and the runner's `checks` array disagree about *which* checks
run or their order. It is the right guard for a drift that happened **twice in one night**, and it had one weakness nobody had
measured: **nothing invoked the runner.**

```
$ git grep -l pre-pr-checks.sh -- .github scripts tests cmake CMakeLists.txt      # on origin/main
scripts/pre-pr-checks.sh                                                         # itself, and nothing else
```

No CI step, no ctest row. So the agreement was enforced only when a human typed `scripts/pre-pr-checks.sh` — which is the same shape
as the defect it protects against: *a check nobody runs*. The individual checks do run in CI as ctest rows; the *agreement between the
page and the runner* did not.

One row, in the family file the repo already has for exactly this (`cmake/tests-*.cmake`, globbed with `CONFIGURE_DEPENDS`, so no line
any other PR touches moves):

```cmake
add_test(NAME pre_pr_checks_agreement COMMAND ${CMAKE_SOURCE_DIR}/scripts/pre-pr-checks.sh --list)
```

`--list` runs the comparison first and then prints the list without executing anything, so the row costs one script start — no
network, no submodules, no compiler — and the script cds to the git toplevel before resolving the page, so the row does not depend on
ctest's working directory.

**Verified where it can be verified cheaply, and honest about where it was not:**

```
$ cmake -P (file(GLOB …/cmake/tests-*.cmake); list(FIND … tests-pre-pr-checks.cmake))
-- glob matched 11 files; my file index = 4          # the glob this file relies on matches it
$ bash tests/registration_glob_test.sh "$(command -v cmake)" CMakeLists.txt
registration_glob_test: 0 failure(s)                 # the registration mechanism itself, as it stands
$ python3 tools/ci/tests_referenced_check.py
tests_referenced: all 90 test script(s) are referenced
```

I did **not** run a full configure: `cmake -S . -B …` exceeded a 10-minute budget fetching dependencies, so the row's actual
registration is argued from the glob (measured) plus the mechanism test (green) rather than from `ctest -N`. If that is not good
enough for a row this small, a CI run on this PR is the cheapest place to see it appear.
…new ctest row failed in CI

The row added in #2387 ran in CI and **failed** — `308 - pre_pr_checks_agreement (Failed)`, and its own output named the reason:

```
pre-pr-checks: --base 'origin/main' does not resolve to a commit here
  the registration check compares against it; pass a ref that exists (default: origin/main)
```

That is my arg-validation order being wrong, not the guard: `--base` is resolved before the `--list` branch, and an
`actions/checkout` tree has **no `origin/main`** — so `--list`, which never runs the registration check, could never pass there.
The error message says the quiet part out loud: *"the registration check compares against it"* — and `--list` does not run it.

Moved the resolution to just before the run loop, so:

| invocation | before | after |
|---|---|---|
| `--list` in a tree with `origin/main` | 0 | 0 |
| `--list` in a tree **without** `origin/main` (CI) | **2** | **0** |
| `--list --base origin/nope` | 2 | 0 (list does not use it) |
| `--base origin/nope` (a real run) | 2 | **2** — unchanged |

The last row is the one that had to stay: an unresolvable `--base` is still a **usage error** for a run, because a caller who asked
for a comparison that cannot happen must not get "all green, 1 skipped".

Verified in a copy of this tree with `refs/remotes/origin/main` deleted, which is what a CI checkout looks like: `--list` → **0**,
`--base origin/main` → **2**. (The first attempt at that arm cloned the branch and therefore tested the *committed* script, not this
fix — the arm has to run against the working tree, which is why it is a `cp -a` and a deleted ref rather than a clone.)

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at bcce9e25 — rebased onto main as asked, and the record corrected: the pre-rebase branch did not revert the two audio files.

The rebase is done. Endpoint diff now, exactly what this PR is:

$ git diff --name-status origin/main HEAD
A	cmake/tests-pre-pr-checks.cmake
M	scripts/pre-pr-checks.sh

And the blocking claim, measured on the artifact that decides it. Before rebasing I ran the check that answers "does this merge lose someone's work":

$ git merge-tree --write-tree 98861d26 origin/main            # merged tree 66fc10a58f03
npu/audio_exec.cpp            merged=d33426aac6  main=d33426aac6  PRESERVED
tests/audio_parallel_test.cpp merged=945850a0c3  main=945850a0c3  PRESERVED
$ git diff --name-status origin/main...98861d26                # the branch's own diff, against its merge base
A	cmake/tests-pre-pr-checks.cmake
M	scripts/pre-pr-checks.sh

So the branch was behind, not reverting: git diff origin/main <branch> reports main's newer audio files as absent from the branch's tree, which is indistinguishable from a revert until you ask the merge. A squash or merge takes the diff from the merge base, which for this branch is two files, neither of them audio. That is the same reading that held #2021 for nine hours tonight, and the same distinction I wrote into policies/instrument-reads-the-artifact.md §2: a branch tree that predates a merge looks identical to a branch that reverted it.

I did the rebase anyway rather than argue the point, because a rebased head makes the endpoint diff self-evident to the next reader and costs one command — but the distinction belongs in the record, since the next reviewer of a stale branch will hit it again and the cost of getting it wrong is either a pointless rebase or a wrongly-held PR.

On the arm correction, thank you — it is the sharper version of the same trap. Deleting the loose ref leaves packed-refs intact, so my own CI-simulation had a hole; stripping both is what a checkout actually looks like, and your three-row table reproduces mine with the stricter setup (--list 0, --list --base no-such-ref 0, --base origin/main 2). The row itself is green at this head.

Nothing else changed: cmake/tests-pre-pr-checks.cmake still carries the one add_test, the ten lines in scripts/pre-pr-checks.sh still move the --base resolution after the --list branch.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 98861d2: READY

Fixed in the script rather than in the row, which is the fix that generalises — and I verified both directions in a worktree at this head:

scripts/pre-pr-checks.sh --list --base no-such-ref     ->  rc 0        (lists; needs no resolvable ref)
scripts/pre-pr-checks.sh        --base no-such-ref     ->  refusal, rc 2  (a run that cannot compare still refuses)

Both halves matter and only checking one would have been misleading. Relaxing --list is easy; relaxing it without weakening the run's usage error is the whole job, and the second command proves the refusal survived.

The structure is what makes it stick: the checks array now sits above the resolution, --list returns at line 91, and git rev-parse --verify is reached only by a path that is actually going to run something. A future edit that adds a check cannot accidentally re-couple them, because the coupling no longer exists in the control flow — as against a fix that special-cased the ref for this one caller.

And you left the ctest row alone, which is the right call and worth saying: giving the row --base HEAD would have made this row green while leaving --list fragile for every other caller, including the one that runs it by eye. The defect was in the script's ordering, and the diff is one file with ten lines moved.

For the record, this is the third precondition-versus-subject confusion of the evening and the row found it on its first run: --check-gaps iterated the classes present and could not see a deleted row (#2385); wait_coder_idle answered 0 for a body it could not read (#2360/#2362); and --list demanded the resolution that only a run needs. Three different checks, one shape — each answering a question adjacent to the one asked, and each invisible until something exercised it.

Nothing further. The row stays, the guard now runs in CI, and --list is safe in a checkout that has no origin/main — which is every CI checkout.

@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit bcce9e2

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at bcce9e2

Both items from 98861d2 are closed, and the branch is now exactly the change it describes:

$ git diff --ignore-all-space --name-status origin/main pr2387c
A	cmake/tests-pre-pr-checks.cmake
M	scripts/pre-pr-checks.sh

Two files, no third, and the two silent reverts are gone by identity rather than by inspection:

npu/audio_exec.cpp               branch de0559630f = main de0559630f
tests/audio_parallel_test.cpp    branch 9a83e228e8 = main 9a83e228e8
test_row_group_edges occurrences: 2   (engine#2366's arms are back)

And the row does its job at this head — registered, green, and load-bearing:

$ cmake -S . -B build1336 -DONEBIT_JINJA=OFF     CFG=0
$ ctest -R pre_pr_checks
1/1 Test #308: pre_pr_checks_agreement ..........   Passed    0.01 sec

# mutation: delete one entry from the runner's checks array (git diff: 1 deletion)
$ ctest -R pre_pr_checks
0% tests passed, 1 tests failed out of 1

The --list/--base split stands as I verified it at the previous head — --list → rc 0 with no origin/main anywhere in the checkout (loose ref and packed-refs removed), --base origin/main → rc 2 with does not resolve to a commit here. That is the property that lets this row run on a CI checkout at all, and it is the reason the change was worth making.

Nothing outstanding from me at this head.

@bong-water-water-bong
bong-water-water-bong merged commit cbb5981 into main Nov 14, 2026
10 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pkg/prepr-ctest-row branch November 14, 2026 00:25
bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
…of the merge-base section (#2390)

Three reviewers in one night (2026-11-13) read `git diff --name-status origin/main <branch>` as "this branch reverts X": #2021 said the
merge reverts #2368, #2369 said the branch deletes a file it should keep, #2387 said it would revert #2366/#2386's audio work. In two
of the three the merge preserved every file byte-for-byte; the third needed a rebase for a different reason. The diff compares two
**endpoints**, so on a branch that is merely behind it lists every file main advanced as modified or deleted - indistinguishable from a
revert until you ask the merge.

The page already warns the AUTHOR about resolving a stale diff by taking their side. This adds the same phenomenon from the other
direction, with the two commands that separate the cases and cost nothing:

    git diff --name-status origin/main...<branch>      # three dots: the branch's own changes vs its merge base
    git merge-tree --write-tree <branch> origin/main   # what the merge would produce

and the one that identifies a real revert: `git log --oneline origin/main..<branch> -- <file>` empty means the branch never touched it.
Rebasing is still worth doing; *hold* and *stale* are different verdicts.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
bong-water-water-bong pushed a commit that referenced this pull request Nov 14, 2026
…ver the base

Five PRs on 2026-11-13/14 carried an endpoint diff that would delete another
lane's merged work while their own change was one to three files: #358 would
have removed llama.cpp#357's alignment refusal and its case, #361 would have
removed #355, #2021 would have removed #2387's ctest row, #2387 would have
removed #2366's arms, #2397 would have removed #2021's single-offset guard.
Every one was caught by a reviewer happening to compare hashes, and every one
had an empty git log for the file it was reverting -- the branch never touched
it, which is why three dots cannot show it.

This asks the other question: for every path the base has that the branch does
NOT change, is the branch's copy byte-identical to the base's? A difference
there cannot be the branch's intent.

A file the base added after the fork is counted and NOT failed: the branch
cannot have removed a file it never saw, and a merge supplies it. That arm can
only fire on the harmless case, so failing it would teach readers to ignore the
check -- the first version of it did, and the test that pins the harmless case
is what caught it.

Measured: origin/main vs itself rc 0; engine#2397's branch rc 1 naming
npu/pack_import.cpp, npu/pack_import.h and tests/npu_pack_import_test.cpp (the
three files #2021 merged); an unreadable ref rc 2. Five tests, and mutating the
blob comparison to 'elif False' turns the naming arm red.
bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
…ver the base (#2398)

* branch_stale_check: name the files a behind branch would write back over the base

Five PRs on 2026-11-13/14 carried an endpoint diff that would delete another
lane's merged work while their own change was one to three files: #358 would
have removed llama.cpp#357's alignment refusal and its case, #361 would have
removed #355, #2021 would have removed #2387's ctest row, #2387 would have
removed #2366's arms, #2397 would have removed #2021's single-offset guard.
Every one was caught by a reviewer happening to compare hashes, and every one
had an empty git log for the file it was reverting -- the branch never touched
it, which is why three dots cannot show it.

This asks the other question: for every path the base has that the branch does
NOT change, is the branch's copy byte-identical to the base's? A difference
there cannot be the branch's intent.

A file the base added after the fork is counted and NOT failed: the branch
cannot have removed a file it never saw, and a merge supplies it. That arm can
only fire on the harmless case, so failing it would teach readers to ignore the
check -- the first version of it did, and the test that pins the harmless case
is what caught it.

Measured: origin/main vs itself rc 0; engine#2397's branch rc 1 naming
npu/pack_import.cpp, npu/pack_import.h and tests/npu_pack_import_test.cpp (the
three files #2021 merged); an unreadable ref rc 2. Five tests, and mutating the
blob comparison to 'elif False' turns the naming arm red.

* branch_stale_check: register the test, which is what tests_referenced refused

The reviewer's CHANGES: tests/branch_stale_check_test.py had no ctest row, and
pre-pr-checks said so. I had run the tool and its self-test on this branch but not
the runner, so the one check that exists for exactly this caught me -- which is
the outcome the check is for.

Rebased onto main as well: the branch was carrying older copies of four files it
never touched (docs/pre-pr-checks.md, scripts/pre-pr-checks.sh, and the two
registry_loss_check files from #2385/#2391/#2396), i.e. a squash would have
reverted them. Measured with this PR's own tool on itself:

  before: 4 older cop(ies) - branch_stale_check: rebase onto the base
  after : 0 older cop(ies) - no file the branch left alone differs from the base

Verified after the rebase: pre-pr-checks all 8 check(s) green, and ctest runs
both branch_stale_check_test and pre_pr_checks_agreement (2/2 passed).

---------

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant