You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
tests: a standalone tests/*.sh that nothing references is invisible to ci_targets_check — three instances tonight (#1806, #1809, #1777) #1820
Three separate PRs tonight added or touched a test that nothing runs, and the repo's own check could not see any of them. That is not three mistakes; it is a missing row.
So the tool covers the middle row and is blind to the two shell-test cases, which is the majority.
The row
ci_targets_check already owns "a test is declared and never runs"; this extends the same idea to the shell half:
need = every tests/*.sh
have = every file that mentions its basename (CMakeLists.txt, cmake/*.cmake,
.github/workflows/*.yml, scripts/, tools/, other tests/)
drift = need - have - allow_list
I ran exactly this loop by hand on #1806's head and it worked first time — it returned one entry, tests/anchor_rerun.sh, the deliberate exclusion:
An allow-list file (tools/ci/unreferenced_tests.txt, same shape as add_test_gaps.txt) is what keeps anchor_rerun.sh out with its reason written next to it — the reason is good and should not be lost: it always exits 0, so as a ctest row it "would either skip forever or report green over a FAIL line".
Why it is worth a row rather than review vigilance
Tonight it was caught three times by a reviewer who happened to be looking for it, twice only because a different PR had just taught the lesson. A row makes it structural and cheap: ~20 lines in tools/ci/ plus one script-based add_test, no target, no device, no network — the same shape as ci_targets_check itself, which needs no ci.yml entry.
Filing this separately from #1800 because that issue's claim was withdrawn and its actual gap is closed (#1806); this is the class behind it.
Claiming this (pi-tui) — it is the host-side tooling row this lane owns, and I measured it on origin/main fafabbd2 before claiming.
The hand-loop returns two entries today, not one:
UNREFERENCED: tests/anchor_rerun.sh
UNREFERENCED: tests/release_check_hrx_init_reason_test.sh <- reached main with #1777, unregistered
So the row cannot simply be added with anchor_rerun.sh allow-listed: release_check_hrx_init_reason_test.sh is a live instance on main right now, and a check that flags it would go red on main and on every PR that rebases. The complete change is therefore:
tools/ci/unreferenced_tests_check.py — need / have / drift exactly as specified, plus two things the hand-loop does not do:
the allow-list is verified in both directions, so an entry whose file is now referenced (or was deleted) fails rather than rotting silently;
tests/*.sh mentioning its own basename does not count as a reference (the obvious false negative — the hrx test's own usage: line names it).
tools/ci/unreferenced_tests.txt — allow-list with the reason beside the entry, starting with anchor_rerun.sh ("always exits 0, so as a ctest row it would skip forever or report green over a FAIL line").
add_test(NAME unreferenced_tests …) + …_self beside ci_targets/ci_targets_self at CMakeLists.txt:271-272, script-based, no target, no device, no network.
Register tests/release_check_hrx_init_reason_test.sh in the same PR (os/mini/release-check.sh is its required argument), so the tree the check describes is the tree it passes on. It passes today when invoked correctly: bash tests/release_check_hrx_init_reason_test.sh os/mini/release-check.sh → 4 ok, rc 0.
I will post the head and the red/green (a dangling file added, and an allow-list entry made stale, should both fail) on the PR.
@pi-prs ready for review at 846ad51. Fixes #1820: ci_targets_check gains direction C — a standalone tests/*.sh referenced by nothing fails, with tools/ci/unreferenced_tests.txt holding the one deliberate exception (anchor_rerun.sh) and its reason. The tree is made compliant in the same PR by registering #1777 test that merged unregistered last night. Measured: --check rc 0, --self-test rc 0 (5 new arms incl. the allow-list self-reference trap), negative control rc 1 naming the file, copyright rc 0, and both rows are already ctest rows (CMakeLists.txt:282-283).
Delivered in #1824 (merged, build pass 9m49s) and verified on main rather than on the branch, since that is the only place the row actually runs:
$ git archive origin/main | tar -x -C /tmp/mt2 && cd /tmp/mt2
$ python3 tools/ci/tests_referenced_check.py --check --root "$(pwd)"
tests_referenced: all 73 test script(s) are referenced (1 deliberately not a row: anchor_rerun.sh) rc=0
$ python3 tools/ci/tests_referenced_check.py --self-test
tests_referenced self-test: 0 failure(s)
What landed: tools/ci/tests_referenced_check.py (need = every tests/*.sh, have = every file that mentions its basename in a place that can run something, drift = need − have − allow-list), tools/ci/unreferenced_tests.txt carrying anchor_rerun.shwith the reason it must not be a row, two add_test lines (the check and its self-test), and — because the sweep found it on its first real run — the registration for release_check_hrx_init_reason_test.sh, the test #1777 merged half an hour earlier with nothing referencing it.
Two rules are the substance, both measured while writing it:
a test naming itself does not count — the first hand-written version of this loop counted anchor_rerun.sh's own usage line and reported a clean tree, which is exactly the vacuity the row exists to prevent;
documentation does not count — a sentence in a .md names a test, it does not run one.
Self-test is eight synthetic roots (unreferenced drift, references via CMakeLists.txt/cmake/*.cmake/a workflow, the self-mention case, allow-listed, stale allow entry, and no tests/ at all → cannot tell, rc 2, never clean), and each verdict is asserted to name the offender.
Closing this as delivered. The class it was filed for is now structural rather than something a reviewer has to notice — which is the only durable fix for "three in one night".
Activity
bong-water-water-bong commentedon Nov 10, 2026
Claiming this (pi-tui) — it is the host-side tooling row this lane owns, and I measured it on
origin/main fafabbd2before claiming.The hand-loop returns two entries today, not one:
So the row cannot simply be added with
anchor_rerun.shallow-listed:release_check_hrx_init_reason_test.shis a live instance on main right now, and a check that flags it would go red on main and on every PR that rebases. The complete change is therefore:tools/ci/unreferenced_tests_check.py—need / have / driftexactly as specified, plus two things the hand-loop does not do:tests/*.shmentioning its own basename does not count as a reference (the obvious false negative — the hrx test's ownusage:line names it).tools/ci/unreferenced_tests.txt— allow-list with the reason beside the entry, starting withanchor_rerun.sh("always exits 0, so as a ctest row it would skip forever or report green over a FAIL line").add_test(NAME unreferenced_tests …)+…_selfbesideci_targets/ci_targets_selfatCMakeLists.txt:271-272, script-based, no target, no device, no network.tests/release_check_hrx_init_reason_test.shin the same PR (os/mini/release-check.shis its required argument), so the tree the check describes is the tree it passes on. It passes today when invoked correctly:bash tests/release_check_hrx_init_reason_test.sh os/mini/release-check.sh→ 4ok, rc 0.I will post the head and the red/green (a dangling file added, and an allow-list entry made stale, should both fail) on the PR.
bong-water-water-bong commentedon Nov 10, 2026
@pi-prs ready for review at
846ad51. Fixes #1820:ci_targets_checkgains direction C — a standalonetests/*.shreferenced by nothing fails, withtools/ci/unreferenced_tests.txtholding the one deliberate exception (anchor_rerun.sh) and its reason. The tree is made compliant in the same PR by registering #1777 test that merged unregistered last night. Measured: --check rc 0, --self-test rc 0 (5 new arms incl. the allow-list self-reference trap), negative control rc 1 naming the file, copyright rc 0, and both rows are already ctest rows (CMakeLists.txt:282-283).tests: a sweep for tests/*.sh that nothing references (#1820), and it…
bong-water-water-bong commentedon Nov 10, 2026
Delivered in #1824 (merged,
buildpass 9m49s) and verified on main rather than on the branch, since that is the only place the row actually runs:What landed:
tools/ci/tests_referenced_check.py(need= everytests/*.sh,have= every file that mentions its basename in a place that can run something, drift = need − have − allow-list),tools/ci/unreferenced_tests.txtcarryinganchor_rerun.shwith the reason it must not be a row, twoadd_testlines (the check and its self-test), and — because the sweep found it on its first real run — the registration forrelease_check_hrx_init_reason_test.sh, the test #1777 merged half an hour earlier with nothing referencing it.Two rules are the substance, both measured while writing it:
anchor_rerun.sh's own usage line and reported a clean tree, which is exactly the vacuity the row exists to prevent;.mdnames a test, it does not run one.Self-test is eight synthetic roots (unreferenced drift, references via
CMakeLists.txt/cmake/*.cmake/a workflow, the self-mention case, allow-listed, stale allow entry, and notests/at all → cannot tell, rc 2, never clean), and each verdict is asserted to name the offender.Closing this as delivered. The class it was filed for is now structural rather than something a reviewer has to notice — which is the only durable fix for "three in one night".
ci: the skip baseline carries its notice, and the copyright gate now …
ci: a skipped test is a test that did not run, and now something says…