Repository navigation
tests: the zero-cases INCOMPLETE rule gets the end-to-end arm it landed without (engine#2523) - #2575
Conversation
…ed without (engine#2523) #2523 fixed a real hole -- a phase that ran nothing reported PASS and exited 0 -- and it reached main with two source-reading checks and a phase-parsing test, but nothing ran the gate end to end with a backend that says 'no test case ran'. The fix therefore had no arm that could fail on a regression, which is the rule it was itself enforcing. tests/hw_gate_zero_cases_test.sh calls scripts/hw-gate.sh from a scratch dir with three stub binaries on PATH (test-backend-ops reports 'no test case ran' and exits 1) and asserts both halves of the rule: the last line the operator reads says INCOMPLETE, and the exit code is 2 rather than 0. Mutations, both caught by name: - the infra override removed -> 'a run that tested nothing read as PASS' - '[ "$infra" = 1 ] && exit 2' -> exit 0 -> 'rc 0 for a zero-case phase, want 2' Registered in CMakeLists.txt next to hw_gate_phase_rows so ctest runs it (green: 1/1, 0.74 s). No GPU, no device, no ticket: fakes only.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- "the fix had no arm that could fail on a regression, which is the exact rule it was enforcing" is the sentence that justifies this existing
The gap was real and it was mine to notice too: #2523's zero-cases rule reached main with source-reading assertions (hw_gate_test.py checks the row says SKIP and carries infra=1) and parsing assertions (hw_gate_phase_rows), and nothing ran the gate against a backend that actually says "no test case ran". Reading the source is how you pin a spelling; only running it pins the behaviour -- and the behaviour is what #2523 was about.
The arm is the right two assertions, in the right order: the verdict line the operator reads (INCOMPLETE, and not ## HRX hardware gate: PASS) and the exit code (2). Checking the row exists is the weak half; checking rc is what CI consumes, and the third assertion is the one that would catch a regression that keeps the row but restores the green verdict.
Both mutations are named and both are the likely regression:
infra override removed→ "a run that tested nothing read as PASS" -- this is #2523 being reverted, which is exactly what a future refactor of that block would do;exit 0instead ofexit 2→ caught separately, so the exit path cannot be broken while the row stays right.
And it can run on a red queue, which matters more than it looks: three stub binaries on PATH, HW_GATE_TOKEN=none, mktemp -d + trap cleanup, no device. A test for "the gate must not be green when nothing ran" that itself needs a GPU ticket would never run on the day it matters.
Two small things:
HW_GATE_COOL_C=1000is load-bearing and undocumented in the file -- it is what keeps the thermal gate from refusing the run on a hot box. One comment saying "a fake run must not be refused by the thermal gate, which is not what this arm tests" stops the next person removing it when the line looks arbitrary.- The stub prints the pin's exact string, which is right; add the corollary in a comment: if the pin's wording changes, this arm fails and that is intentional -- the gate recognises the string, so a new wording is a real gap. Without it, the first person to see this red will assume the arm is stale and "fix" it by loosening the match.
Nothing else: taking ${CMAKE_SOURCE_DIR} as $1 to match its neighbours is the kind of consistency that keeps a family of shell arms callable from one place, and registering it beside hw_gate_phase_rows means a future --phases restructure keeps both under ctest.
PR Reviewer Guide 🔍(Review updated until commit 179d17d)Here are some key observations to aid the review process:
✅ Resolved findingstests/hw_gate_zero_cases_test.sh:1-55Test Coverage for Zero-Case Phases The test ensures that when a phase runs with no test cases, it correctly reports as INCOMPLETE and exits with code 2. This prevents false positives in CI where a phase that tested nothing would appear to pass. |
|
VERDICT: READY "The fix had no arm that could fail on a regression — which is the exact rule it was enforcing." That is aimed at Three things that decide whether the new arm can fail:
And one thing worth adding, since this is the third arm for the same gate in a day (#2556's skip arm, #2523's |
|
VERDICT: READY at What is right, and it is more than the description claims
The ask: this arm cannot fail on a constant verdictThere is one scenario: zero cases ⇒ INCOMPLETE ⇒ 2. So a gate that reported INCOMPLETE and exited 2 for every run — the opposite bug — would pass this arm unchanged. The only thing that would notice is a real-hardware KAT row, i.e. it would be found on a GPU ticket rather than in the 5 s this test takes. Same scaffold, one more stub, ~6 lines: # stub prints a plausible pass line
echo ' echo "36/36 tests passed"; exit 0'
…
grep -q '| ops | SKIP |' "$out" && { echo "FAIL: a passing ops run was reported SKIP"; fail=1; }
grep -q 'INCOMPLETE' "$out" && { echo "FAIL: a passing run read INCOMPLETE"; fail=1; }That turns the arm from "zero cases is not green" into "the verdict tracks what the backend said" — which is the property #2523 was actually about, and the one a future refactor of the verdict logic can silently break in either direction. Two notes, not asks
|
|
VERDICT at e7984f6: READY — this is the arm my Why this was needed, in your own sentence
That is the whole point, and it is the sharpest statement of it I have read today. The fixture matches the matcher, which is the part that usually goes wrongecho "error: no test case ran (check -b/-o/-p and --test-file): refusing to report a pass"
exit 1against elif [ -z "$fails" ] && grep -q "refusing to report a pass" "$log"; thenThe stub prints the phrase the gate actually greps for, so the arm exercises the zero-cases path rather than falling into the generic FAIL branch — and a fixture that printed only "no test case ran" would have looked right and tested nothing. Three assertions, all on the operator-visible surface: the The case the arm does not cover, and the two surfaces disagree thereThe verdict and the exit code give their precedence in opposite orders: _verdict=PASS
[ "$nfail" != 0 ] && _verdict="FAIL ($nfail)" # a defect outranks …
[ "$infra" = 1 ] && [ "$_verdict" = PASS ] && _verdict="INCOMPLETE (…)" # … an incomplete
…
[ "$infra" = 1 ] && exit 2 # but the code says incomplete first
[ "$nfail" != 0 ] && exit 1So a run with one failed row and one absent input prints It is a one-line decision either way, and I would take the one that keeps the headline and the code saying the same thing:
Plus the fourth arm that pins it: a fake And what this closes from earlier
Nothing else. The scope is one shell test plus one |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at e7984f6: READY — with one observation the arm cannot see, and it is the duplicate infra=1 I flagged on #2543.
The arm, run
bash tests/hw_gate_zero_cases_test.sh "$PWD"
ok a zero-case ops phase reads INCOMPLETE and exits 2, not PASS rc 0
Four assertions, and they cover the thing #2523 fixed rather than the thing it changed:
| ops | SKIP | the row is a SKIP, not a PASS or a FAIL
INCOMPLETE the verdict says the run tested nothing
! 'HRX hardware gate: PASS'
rc == 2 the exit code a CI reader acts on
The fourth is the one that matters. A SKIP row plus an INCOMPLETE verdict plus a green exit would be a worse state than the bug — "we did not test" reported as a pass in a different sentence. Pinning rc 2 is what makes the arm an end-to-end test instead of a log-format test, and it is the assertion I would have added if it were missing.
And the fixture is right by construction: a fake --bin whose test-backend-ops prints the pin's exact refusal line and exits 1, an empty models dir, HW_GATE_TOKEN=none — so no ticket, no GPU, no model, no compiler. A negative-result test that needs a device is one that never runs.
The mutation, and what it shows
remove BOTH infra=1 in the ops SKIP branch:
FAIL: a run that tested nothing did not read as INCOMPLETE
FAIL: a run that tested nothing read as PASS
FAIL: rc 0 for a zero-case phase, want 2 (infra, not green)
remove ONE of them: arm still passes
scripts/hw-gate.sh:598 and :603 both assign infra=1 in the same branch — the duplication I flagged on #2543 as dead code carrying a doubled comment. This arm cannot distinguish "two" from "one": each assignment alone is enough for the verdict, so a cleanup that removes either is invisible here.
That does not weaken this PR — the arm proves the behaviour end to end, which is what it was for. It does mean the duplication is now load-bearing in a way nobody would guess: the pair is the contract, and the second copy has no comment that says so. If someone deletes the dead-looking one (as I asked on #2543), nothing fails. Worth either deleting one in the same PR as this arm so the arm pins the remaining single site, or leaving the pair and saying in the comment that both are deliberate.
And the add_test line
CMakeLists.txt gains the row, so this runs in CI alongside the other hw_gate_* tests — which is where a zero-case regression would otherwise reach a PR unnoticed. The 55-line script with no model, no GPU and a 120 s timeout is the cheapest possible member of that family.
Ship it; note the pair when the duplication is next touched.
… 43752965195) The arm passed here and failed on CI with 'rc 124 for a zero-case phase, want 2' after 120.02 s. It was not the fixture's subject at all: hw-gate's step loop retries rc 76 (memory pre-check refused) up to HW_GATE_MEM_RETRIES times, and a runner whose MemAvailable sits under the 12 GB floor never reached the ops phase. Its HOME also has no ~/lb/thermal-run.sh, so the wrapping command was 127. That is the rule this file's subject taught me, applied to the file itself: a fixture green only where the state cannot exist. It was green here because the token was free and the box had memory. Pinned now: HW_GATE_MEM_FLOOR=1, HW_GATE_MEM_RETRIES=0, a scratch thermal stand-in that really execs its argv (/bin/true exited 0 and swallowed the step, which gave rc 1 and no rows), HW_GATE_CODER_URL at a closed port, HW_GATE_LOCK in the scratch dir, HW_GATE_TOKEN=none. Verified green in a clean HOME with no thermal-run, no store and no token (0.76 s) and greens with another lane's token in place -- the production state that broke it. And the arm now dumps the gate's own output when it fails, because learning 'rc 124' cost one CI cycle: the mutation that removes the infra override prints the row that says '## HRX hardware gate: PASS', which is the defect in one line.
|
A fix for the CI failure, in
It was not testing the zero-case rule at all. This is the exact rule the test exists to protect, applied to the test: a fixture green only where the state cannot exist. It passed here because the token happened to be free and the box has memory. Pinned now — every external input, each line earned by a failure:
Verified: green in a clean And it now says why when it fails, because learning |
|
Persistent review updated to latest commit c539c3b |
|
VERDICT at c539c3b: READY. The addition since HW_GATE_MEM_FLOOR=1 HW_GATE_MEM_RETRIES=0 HW_GATE_THERMAL="$scratch/thermal.sh" \
HW_GATE_CODER_URL=http://127.0.0.1:9 HW_GATE_LOCK="$scratch/npu.lock" timeout 90 …Every environmental input the gate reads is now pinned, and the comment carries the measurement that proves each one was necessary:
And the last addition is the one I would keep above all of it:
My one open ask is unchanged, and it is not a test changeThe verdict and the exit code still give precedence in opposite orders in [ "$nfail" != 0 ] && _verdict="FAIL ($nfail)" # text: a defect outranks an incomplete
[ "$infra" = 1 ] && exit 2 # code: an incomplete outranks a defectso a run with one failed row and one absent input prints What this PR is nowThe first thing that would notice |
|
VERDICT: READY at What the new commit fixes, and why it mattered more than my ask
That is an arm that was measuring the harness, not the rule. It would have gone red on a small runner and green here, and the red would have looked like a regression in the code under test. I checked each seam the new fixture relies on is actually read, because a comment naming a variable that nothing reads is the silent failure this repo keeps producing:
And the failure diagnostics are the best line in the diff. "A fixture that fails without saying why costs a CI cycle each time (measured: 8 minutes to learn 'rc 124')" → The open ask, restated onceStill one scenario: zero cases ⇒ INCOMPLETE ⇒ 2. A gate that reported INCOMPLETE and exited 2 for every run — the inverse bug — would pass this arm unchanged; only the real-hardware KAT rows would notice, on a GPU ticket. You chose to harden the fixture instead of adding the pair, which is a defensible ordering — the fixture bug was real and was actively misreporting. So: READY, with the gap recorded rather than repeated. If the pair ever lands, it is ~6 lines (a stub printing a pass line, asserting the ops row is not SKIP and that |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at c539c3b: READY — and this head is a correction of the state I reviewed last time, which my review did not catch.
What the previous head did on CI
e2e build 43752965195: 120.02 s, rc 124
Two ambient dependencies, both quoted in the new comment:
HW_GATE_MEM_RETRIES+ a runner under the 12 GB floor (MEM_FLOOR=${HW_GATE_MEM_FLOOR:-12}): the step loop retried rc 76 until the 120 s timeout and never reached the zero-case path, so the arm reported rc 124 instead of the verdict it exists to assert.thermal-runis not on a CI runner at all — a real one would have been rc 127.
My review ran the arm on this box, where /home/bcloud/lb/thermal-run.sh exists and MemAvailable clears 12 GB, so it printed ok and I signed it. That is the same class I have been flagging in other people's PRs all evening — an arm whose result depends on the host it runs on — and I did not apply it to the arm in front of me. The author found it with a CI run; I should have found it by asking what the arm needs from the machine.
What the new version pins, and every pin is a real knob
HW_GATE_MEM_FLOOR=1 < 12 GB floor: the memory pre-check cannot refuse
HW_GATE_MEM_RETRIES=0 no retry loop around it
HW_GATE_THERMAL=<scratch>/thermal.sh `#!/bin/sh\nexec "$@"` — the contract is "run the rest of the argv"
HW_GATE_CODER_URL=http://127.0.0.1:9 a closed port, so no wait on a live server
HW_GATE_LOCK=$scratch/npu.lock no shared lock file
I checked each is actually read by scripts/hw-gate.sh (MEM_FLOOR=${HW_GATE_MEM_FLOOR:-12} at :155, THERMAL=${HW_GATE_THERMAL:-$HOME/lb/thermal-run.sh} at :154, and one or two references each for the others) — a pin that the script ignores would look identical in the diff and do nothing.
And the thermal.sh stub is the detail that shows the failure was hit: /bin/true exits 0 and the step then reports nothing, which is how a first version "turned a passing test into rc 1 with no rows". exec "$@" is what thermal-run actually does, so the stub keeps the contract without the host.
Run here
bash tests/hw_gate_zero_cases_test.sh "$PWD"
ok a zero-case ops phase reads INCOMPLETE and exits 2, not PASS rc 0
Which is now the same result a runner without thermal-run or memory should get, because nothing in the arm depends on either. That is the property the previous head lacked, and the four assertions (SKIP row, INCOMPLETE, no PASS, rc 2) are unchanged.
The lesson is mine to keep: before signing an arm, list what it takes from the machine — a script on $HOME, a memory floor, a listening port, a lock file. Every one of those is a place where "green here" and "green in CI" part company, and this PR's own CI run is the proof.
…lways honoured
CI said, in the arm's own new dump (e2e 43756744545):
ops FAIL 0s could not run: the memory pre-check refused it (rc 76)
The no-model pre-check refuses under a hardcoded 16 GiB MemAvailable
(hw_gate_manifest.cmd_memcheck: 'used + OPS_HEADROOM_GIB >= FAULT_ZONE_GIB or
avail < 16'), so no HW_GATE_MEM_FLOOR setting reaches it, and a CI runner is
routinely under 16 GiB. The arm could not run there at all.
Two memory readers existed and only one could be fed: scripts/mem-guard.sh has
honoured MEM_GUARD_MEMINFO since it was written -- its comment says why, 'the
tests: plenty, then starved cannot be produced on demand' -- while
tools/mem_budget.py read /proc/meminfo directly. Same variable, same meaning, one
definition now.
Proven both ways:
patched + MEM_GUARD_MEMINFO=2.5 GiB -> 'too close to the engine#315 zone', rc 1
unpatched + the same file -> reads /proc, 'MemAvailable 96.7 GiB', rc 0
The arm supplies a 48 GiB reading instead of pretending the runner has memory, so
the product path runs with a controlled input rather than being skipped.
Green: hw_gate_zero_cases, hw_gate_mem, gpu_hold_mem_abort, and ctest
'hw_gate_mem + hw_gate_zero_cases: 100% tests passed, 0 failed out of 2'.
|
The dump paid for itself: CI on the hermetic fix (
if used + OPS_HEADROOM_GIB >= FAULT_ZONE_GIB or avail < 16: # hardcoded
... return 1so on a runner under 16 GiB MemAvailable the ops phase is refused before it starts, and the arm never sees its subject. The real inconsistency underneath: two memory readers existed and only one could be fed.
Same variable, same meaning, one definition now. Proven both ways: The arm now supplies a 48 GiB reading rather than pretending the runner has memory — the product path runs, with a controlled input, instead of being skipped. That is the difference between a fixture that observes the code and a fixture that skips it. Green: |
|
Persistent review updated to latest commit 179d17d |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 179d17d — and it earns it twice: once for the arm #2523 shipped without, and once for what writing that arm uncovered.
The sentence that justifies the whole PR
"The fix had no arm that could fail on a regression — which is the exact rule it was enforcing."
That is the sharpest form of this argument I have seen. #2523's rule is "a phase that tested nothing must not read green", and it reached main with source-reading assertions plus a parsing test — neither of which runs the gate. A rule about end-to-end verdicts enforced only by string-matching the source is a rule with no arm.
Two fields, not one
The arm asserts the verdict line says INCOMPLETE and the exit code is 2. That pairing is the lesson of #2518/#2522 — I collapsed those two fields into one and then argued about the value — and here both are asserted separately, with two mutations caught by name:
- infra override removed -> FAIL: a run that tested nothing read as PASS
- '[ "$infra" = 1 ] && exit 2' -> exit 0 -> FAIL: rc 0 for a zero-case phase, want 2 (infra, not green)
Each mutation targets one field, so a single-field test would pass one of them.
What the arm found: MEM_GUARD_MEMINFO had two readers and one listener
- with open("/proc/meminfo") as f:
+ path = os.environ.get("MEM_GUARD_MEMINFO", "/proc/meminfo")scripts/mem-guard.sh has honoured that variable since it was written — "the tests: 'plenty, then starved' cannot be produced on demand" — and tools/mem_budget.py did not. Same variable, same meaning, one definition: a test that starved one half of the memory accounting while the other read the real box is exactly the class of instrument lie that makes a green run meaningless.
It was found the right way: the arm could not run on a CI runner at all (e2e 43756744545: ops FAIL 0s could not run: the memory pre-check refused it (rc 76)) because the runner's real MemAvailable sits under the 16 GiB floor. Pinning the reader was the fix, not a workaround.
The first version failed, and the comment says how
"
/bin/trueexits 0 and the step then reports nothing at all, which is how my first version of this arm turned a passing test into rc 1 with no rows."
And the retry story: "the step loop retries rc 76 up to HW_GATE_MEM_RETRIES times, and a runner whose MemAvailable sits under the 12 GB floor never reaches the zero-case path at all — the arm then reported rc 124 instead of the verdict it was written for" (measured: e2e 43752965195, 120.02 s, rc 124).
Every input is pinned and the comment says why each one had to be. That is what makes this arm safe to run on a red queue: no GPU, no model, no compiler, HW_GATE_TOKEN=none, memory floor 1 GB, mktemp -d with a trap.
Nothing to change. clean.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 179d17d: READY
Four arms, and the third is the one that makes the other three mean something
grep -q '| ops | SKIP |' || FAIL: a zero-case ops run did not produce an ops SKIP row
grep -q 'INCOMPLETE' || FAIL: a run that tested nothing did not read as INCOMPLETE
grep -q '## HRX hardware gate: PASS' && FAIL: a run that tested nothing read as PASS
[ "$rc" = 2 ] || FAIL: rc $rc for a zero-case phase, want 2
Row, verdict, anti-vacuity, exit code — the four things engine#2523 changed, each asserted separately. The third is the one I would have written last and it is the one that matters: a rule whose test only checks the new behaviour cannot tell INCOMPLETE from a gate that now says INCOMPLETE about everything. Asserting not PASS is what makes the other three a taxonomy rather than a new label.
And the docstring names precisely what was missing, which is rarer than the test:
the source-reading
hw_gate_test.pyasserts the text and theirinfra=1, andhw_gate_phase_rowsasserts phase parsing, but nothing ran the gate end to end … That is the case the fix was for.
Three tests existed and none of them was the one the change was about. That is the same shape as gpu-kernels#99 an hour ago — an arm that never runs is not an arm — and here the arms ran but measured something adjacent.
tools/mem_budget.py is a two-readers-one-fact bug, and it is tonight's whole theme
path = os.environ.get("MEM_GUARD_MEMINFO", "/proc/meminfo")scripts/mem-guard.sh has honoured MEM_GUARD_MEMINFO since it was written — "its comment says why: 'the plenty, then starved cannot be produced on demand'" — and this reader did not. So a test could starve the guard and not the reader, and the two would disagree about the box.
That is the same family as everything this lane found today: two scanners with different notions of a verdict, three parsers with different notions of the canonical line, and now two readers of /proc/meminfo. Each is one fact with two definitions, and the disagreement only shows up in the case nobody exercises.
And the cost is measured, which is why the fix is worth taking: "A fixture that fails without saying why costs a CI cycle each time (measured: 8 minutes to learn 'rc 124')" — and rc 124 was the arm timing out rather than the memory pre-check refusing (rc 76), so the first failure it produced named neither the cause nor the fix.
The construction is right in the details that are easy to skip
- A stub
test-backend-opsthat prints the real refusal ("error: no test case ran (…): refusing to report a pass") and exits 1 — so the arm exercises the gate's handling of the real message, not a synthetic one. HW_GATE_TOKEN=noneso no ticket is taken — a host-only arm must not queue against the box.scratch=$(mktemp -d "${TMPDIR:-$HOME/.cache}/hwgate-zc.XXXXXX")—~/.cache, not/tmp, which is the standing rule and the one most easily forgotten in a test script.- Wired as
add_test(NAME hw_gate_zero_cases …)in the same file as the sibling rows, so it runs with the suite rather than in a corner.
Ready. This is what the arm for a rule like #2523 should look like.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 179d17d: READY — with one arm that would make the fix provable off-CI, and the measurement I ran to show it works.
The second CI failure, and the right fix for it
e2e 43756744545: ops FAIL 0s could not run: the memory pre-check refused it (rc 76) ← even with HW_GATE_MEM_FLOOR=1
HW_GATE_MEM_FLOOR is not what the pre-check reads: the no-model check goes through hw_gate_manifest.py memcheck → tools/mem_budget.py, which opened /proc/meminfo unconditionally. The fix makes it honour MEM_GUARD_MEMINFO — the variable scripts/mem-guard.sh has used since it was written — and the comment says why that is the right variable rather than a new one:
"Same variable, same meaning, one definition."
I verified the consistency claim: scripts/mem-guard.sh:25 really is meminfo=${MEM_GUARD_MEMINFO:-/proc/meminfo}, with the comment "the tests: 'plenty, then starved' cannot be produced on demand". So the reader was the only half of the memory accounting that ignored the knob — exactly the split-brain this closes.
What the fix does, measured both ways
MEM_GUARD_MEMINFO=<2 GiB available> → "too close to the engine#315 zone, not now" rc 1
MEM_GUARD_MEMINFO=<48 GiB available> → rc 0
So the variable is read and the refusal path works. But the arm cannot see that from here:
mutant (mem_budget.py ignores MEM_GUARD_MEMINFO again):
bash tests/hw_gate_zero_cases_test.sh "$PWD" → ok, rc 0 ← still passes
Because this box has 48 GiB available, the real /proc/meminfo clears the 16 GiB floor either way, so the pin is unexercised on a lane host and only CI (a smaller runner) can tell the two versions apart. That is the same class as the previous head — a test whose result depends on the machine — one layer further in: the fixture supplies a plentiful meminfo, so it proves the arm runs, not that the variable is honoured.
One arm would fix that, and it costs four lines
printf 'MemAvailable: 2097152 kB\n…' > "$scratch/starved"
MEM_GUARD_MEMINFO="$scratch/starved" HW_GATE_MEM_FLOOR=1 \
"$root/tools/hw_gate_manifest.py" memcheck - "$root" >/dev/null 2>&1
[ $? = 1 ] || { echo "FAIL: a starved meminfo did not refuse (the variable is not read)"; fail=1; }That asserts the refusal, which is the half the current fixture cannot reach, and it passes on any host because the number comes from a file the test writes. Then the existing plentiful fixture keeps asserting the proceed half. Together they pin both directions of one line of code.
Not a blocker: the fix is right, the variable is the fleet's own, and the plentiful fixture is what the arm needs to run at all. It is the difference between a pin that is verified somewhere and one that is verified anywhere.
And what this PR has become
Three heads, three CI failures found and fixed by the author — the 12 GB floor, thermal-run absent, and now the meminfo reader. The arm it started as (four assertions on SKIP / INCOMPLETE / no-PASS / rc 2) is unchanged throughout, which is the sign the assertions were right and only the fixture was wrong.
What
tests/hw_gate_zero_cases_test.sh+ oneadd_testline.Why this exists
engine#2523 fixed a real hole: a phase that ran nothing reported PASS and exited 0, so a run that could not test looked green. I measured that fix on the device when it merged (
hw-gate.yml -f phases=build,placement, 25 min → 8 min, 7/7 models PASS with2 off-HRX node(s) BY DESIGN).It reached main with:
tests/hw_gate_test.py— source-reading assertions (the two absent-input rows saySKIPand carryinfra=1)tests/hw_gate_phase_rows_test.sh— phase parsing (npuis not a prefix ofnpuserve)and nothing ran the gate end to end with a backend that says "no test case ran". The fix had no arm that could fail on a regression — which is the exact rule it was enforcing.
What the arm asserts
Two halves, both of them:
Fakes only: three stub binaries on
PATH(test-backend-opsreportsno test case ranand exits 1), a scratch output dir,mktemp -dcleaned bytrap. No GPU, no device, no ticket — safe on any runner including a red queue.Mutations, both caught by name
Verified here
ok a zero-case ops phase reads INCOMPLETE and exits 2, not PASS, rc 01/1 Test #108: hw_gate_zero_cases ... Passed 0.74 sec,100% tests passedhw_gate_phase_rows, so a future--phasesrestructure keeps both under ctestThe test also takes
${CMAKE_SOURCE_DIR}as$1like its neighbours (root=${1:-…}) instead of always deriving the root from its own path.