From 07b9cc3fee76f0fbe4433e37e2446e7aeab47416 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Fri, 13 Nov 2026 19:15:19 -0400 Subject: [PATCH 1/2] =?UTF-8?q?pre-pr-checks:=20give=20the=20page/runner?= =?UTF-8?q?=20guard=20a=20ctest=20row=20=E2=80=94=20nothing=20ever=20ran?= =?UTF-8?q?=20the=20runner?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit engine#2384 added the guard that refuses when `docs/pre-pr-checks.md` and the runner's `checks` array disagree about *which* checks run or their order. It is the right guard for a drift that happened **twice in one night**, and it had one weakness nobody had measured: **nothing invoked the runner.** ``` $ git grep -l pre-pr-checks.sh -- .github scripts tests cmake CMakeLists.txt # on origin/main scripts/pre-pr-checks.sh # itself, and nothing else ``` No CI step, no ctest row. So the agreement was enforced only when a human typed `scripts/pre-pr-checks.sh` — which is the same shape as the defect it protects against: *a check nobody runs*. The individual checks do run in CI as ctest rows; the *agreement between the page and the runner* did not. One row, in the family file the repo already has for exactly this (`cmake/tests-*.cmake`, globbed with `CONFIGURE_DEPENDS`, so no line any other PR touches moves): ```cmake add_test(NAME pre_pr_checks_agreement COMMAND ${CMAKE_SOURCE_DIR}/scripts/pre-pr-checks.sh --list) ``` `--list` runs the comparison first and then prints the list without executing anything, so the row costs one script start — no network, no submodules, no compiler — and the script cds to the git toplevel before resolving the page, so the row does not depend on ctest's working directory. **Verified where it can be verified cheaply, and honest about where it was not:** ``` $ cmake -P (file(GLOB …/cmake/tests-*.cmake); list(FIND … tests-pre-pr-checks.cmake)) -- glob matched 11 files; my file index = 4 # the glob this file relies on matches it $ bash tests/registration_glob_test.sh "$(command -v cmake)" CMakeLists.txt registration_glob_test: 0 failure(s) # the registration mechanism itself, as it stands $ python3 tools/ci/tests_referenced_check.py tests_referenced: all 90 test script(s) are referenced ``` I did **not** run a full configure: `cmake -S . -B …` exceeded a 10-minute budget fetching dependencies, so the row's actual registration is argued from the glob (measured) plus the mechanism test (green) rather than from `ctest -N`. If that is not good enough for a row this small, a CI run on this PR is the cheapest place to see it appear. --- cmake/tests-pre-pr-checks.cmake | 25 +++++++++++++++++++++++++ 1 file changed, 25 insertions(+) create mode 100644 cmake/tests-pre-pr-checks.cmake diff --git a/cmake/tests-pre-pr-checks.cmake b/cmake/tests-pre-pr-checks.cmake new file mode 100644 index 000000000..1263f91f7 --- /dev/null +++ b/cmake/tests-pre-pr-checks.cmake @@ -0,0 +1,25 @@ +# Copyright 2026 bong-water-water-bong +# SPDX-License-Identifier: Apache-2.0 +# +# Licensed under the Apache License, Version 2.0 (the "License"); +# you may not use this file except in compliance with the License. +# You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. + +# engine#2384 added a guard to `scripts/pre-pr-checks.sh`: it refuses to run when `docs/pre-pr-checks.md` and its own `checks` +# array disagree about WHICH checks run, or their order. That guard was author-run only -- measured 2026-11-13, `git grep +# pre-pr-checks.sh` over the tree found nothing but the script itself: no CI step and no ctest row ever invoked it, so the +# page/runner drift that happened TWICE in one night (a count sentence six->seven, and three positions out of order) would still +# have needed a human to type the command. This row makes the agreement a build-time failure. +# +# `--list` runs the comparison first and then prints the list without executing anything, so this costs one script start: no +# network, no submodules, no compiler. The script cds to the git toplevel before resolving the page, so the row is independent of +# the ctest working directory. +add_test(NAME pre_pr_checks_agreement COMMAND ${CMAKE_SOURCE_DIR}/scripts/pre-pr-checks.sh --list) From bcce9e2588f32d922210e429342a7a9b56e2f5e2 Mon Sep 17 00:00:00 2001 From: bong-water-water-bong Date: Fri, 13 Nov 2026 19:46:02 -0400 Subject: [PATCH 2/2] pre-pr-checks: `--list` must not demand a `--base`, which is why the new ctest row failed in CI MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The row added in #2387 ran in CI and **failed** — `308 - pre_pr_checks_agreement (Failed)`, and its own output named the reason: ``` pre-pr-checks: --base 'origin/main' does not resolve to a commit here the registration check compares against it; pass a ref that exists (default: origin/main) ``` That is my arg-validation order being wrong, not the guard: `--base` is resolved before the `--list` branch, and an `actions/checkout` tree has **no `origin/main`** — so `--list`, which never runs the registration check, could never pass there. The error message says the quiet part out loud: *"the registration check compares against it"* — and `--list` does not run it. Moved the resolution to just before the run loop, so: | invocation | before | after | |---|---|---| | `--list` in a tree with `origin/main` | 0 | 0 | | `--list` in a tree **without** `origin/main` (CI) | **2** | **0** | | `--list --base origin/nope` | 2 | 0 (list does not use it) | | `--base origin/nope` (a real run) | 2 | **2** — unchanged | The last row is the one that had to stay: an unresolvable `--base` is still a **usage error** for a run, because a caller who asked for a comparison that cannot happen must not get "all green, 1 skipped". Verified in a copy of this tree with `refs/remotes/origin/main` deleted, which is what a CI checkout looks like: `--list` → **0**, `--base origin/main` → **2**. (The first attempt at that arm cloned the branch and therefore tested the *committed* script, not this fix — the arm has to run against the working tree, which is why it is a `cp -a` and a deleted ref rather than a clone.) --- scripts/pre-pr-checks.sh | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-) diff --git a/scripts/pre-pr-checks.sh b/scripts/pre-pr-checks.sh index fe7686484..7d9e8f29a 100755 --- a/scripts/pre-pr-checks.sh +++ b/scripts/pre-pr-checks.sh @@ -53,16 +53,6 @@ done top=$(git rev-parse --show-toplevel 2>/dev/null) || { echo "pre-pr-checks: not inside a git work tree" >&2; exit 2; } cd "$top" || { echo "pre-pr-checks: cannot enter $top" >&2; exit 2; } -# A `--base` that does not resolve is a USAGE error, not a skip: the caller asked for a comparison that -# cannot happen, and a run that prints "all 5 check(s) green, 1 skipped" then exits 0 is the shape this -# script exists to stop (review of engine#2313). A skip is for a check that cannot run in this -# environment; an argument that cannot be resolved is the caller's to fix. -git rev-parse --verify -q "$base^{commit}" >/dev/null || { - echo "pre-pr-checks: --base '$base' does not resolve to a commit here" >&2 - echo " the registration check compares against it; pass a ref that exists (default: origin/main)" >&2 - exit 2 -} - # The checks, in the order docs/pre-pr-checks.md lists them: cheapest and most common first, the two # that need a revision last. checks=( @@ -101,6 +91,16 @@ if [ "$list_only" = 1 ]; then exit 0 fi +# A `--base` that does not resolve is a USAGE error, not a skip: the caller asked for a comparison that +# cannot happen, and a run that prints "all 5 check(s) green, 1 skipped" then exits 0 is the shape this +# script exists to stop (review of engine#2313). A skip is for a check that cannot run in this +# environment; an argument that cannot be resolved is the caller's to fix. +git rev-parse --verify -q "$base^{commit}" >/dev/null || { + echo "pre-pr-checks: --base '$base' does not resolve to a commit here" >&2 + echo " the registration check compares against it; pass a ref that exists (default: origin/main)" >&2 + exit 2 +} + ok=0; failed=0; skipped=0; detail="" for c in "${checks[@]}"; do name=${c%% *}