Repository navigation
tests_referenced: a row that names a script which is not in the tree is drift - #2346
Conversation
…ich is not in the tree
engine#2183 shipped a row whose script a rebase had dropped. Its commit message reads *"the lv3 row script my
rebase pattern missed -- the row was Not Run"*, and every gate was green on it:
mv tests/whisper_lv3_row.sh tests/whisper_lv3_row.sh.bak
tests_referenced_check --check rc 0
tests_referenced_check HERMETIC=1 rc 0
ci_targets_check --check rc 0
`referenced()` is satisfied by the *mention* in CMakeLists.txt, and `ci_targets_check` follows `$<TARGET_FILE:...>`
targets rather than plain paths, so a row can name a file that is not there, report **Not Run** for ever, and every
gate stays green. That is the same defect as an unreferenced test one step further along: a row that cannot fail.
`missing_mentions()` asks the inverse question -- every `tests/*.sh` a RUNNER names must exist -- and joins the same
verdict. On the real tree it is **rc 0** (main has no stale mention), and planting one is rc 1:
renamed tests/amdxdna_restart_test.sh -> rc 1, one finding naming the row
**The false positive that shaped it:** the first regex matched the tail, so `${CMAKE_SOURCE_DIR}/tools/tests/
test-bench-preflight.sh` read as `tests/test-bench-preflight.sh` and reported **two existing files as missing**.
`tools/tests/x.sh` is a different file from `tests/x.sh`; the pattern now takes the full path after the variable,
and an arm pins exactly that.
Three arms added; dropping `missing_mentions` from the verdict fails one. Self-test ok, ctest `tests_referenced`
0.78 s and `tests_referenced_self` 0.04 s, copyright --check rc 0, ci_targets_check --check rc 0.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
VERDICT: READY at The caseand the commit message is the confession: "the lv3 row script my rebase pattern missed — the row was Not Run". The mechanism, which is the same shape twice
So: one check verified that a name appears in a registration file, the other followed a target expression, and neither asked "does the file this row names exist?". A That is Two asks
And the PR's provenance is the best kind"engine#2183 shipped a row whose script a rebase had dropped" — a defect found in a real PR by a lane that noticed a test was "Not Run" in a log. Two checkers, three green ticks, and one absent test; that is exactly the class this session has been collecting, and the fix is to make the check read the filesystem rather than the paperwork. |
engine#2183 shipped a row whose script a rebase had dropped. Its commit message reads "the lv3 row script my rebase pattern missed — the row was Not Run", and every gate was green on it:
referenced()is satisfied by the mention inCMakeLists.txt, andci_targets_checkfollows$<TARGET_FILE:…>targets rather than plain paths — so a row can name a file that is not in the tree, report Not Run for ever, and every gate stays green. That is the same defect as an unreferenced test, one step further along.missing_mentions()asks the inverse question — everytests/*.sha RUNNER names must exist — and joins the same verdict. rc 0 on main, rc 1 on the plant (renamingtests/amdxdna_restart_test.shgives one finding that names the row).The false positive that shaped it: the first regex matched the tail, so
${CMAKE_SOURCE_DIR}/tools/tests/test-bench-preflight.shread astests/test-bench-preflight.shand reported two existing files as missing.tools/tests/x.shis a different file fromtests/x.sh; the pattern now takes the full path after the variable, and an arm pins exactly that.Three arms added; dropping
missing_mentionsfrom the verdict fails one.