Repository navigation
hw-gate: a model that is not on this runner SKIPs its row and is named at the end (engine#2501) - #2520
hw-gate: a model that is not on this runner SKIPs its row and is named at the end (engine#2501)#2520bong-water-water-bong wants to merge 1 commit into
Conversation
…d at the end (engine#2501)
A FAIL row for an absent GGUF reds every PR's gate for a reason no PR can fix -- the row was red
on main's own pin (engine#2485 run 43677464548, engine#2487 run 43678134013), and the file is not
on the box. The two options in the issue were "put the GGUF back" (a box change, needs the owner)
and "SKIP, and say so"; this is the second, and engine#2494 argues against the first: that model is
the crash-1 storm source for the MES resets.
result "model $name" SKIP 0 "not on this runner: $model"
missing_models="${missing_models:-} $name"
...
note "::warning::model rows skipped because the GGUF is not on this runner:"$missing_models
The skip deliberately does not set infra=1. A missing environment fact is not an infrastructure
failure, and infra exits 2 -- which is the FAIL the row used to be, one indirection away.
The second half is the one that matters more: coverage that did not happen has to be visible, or
the matrix looks complete. One warning line at the end of the run names every model that was not
exercised, so the difference between "the gate passed" and "the gate passed on what was here" is
in the log.
Test: tests/hw_gate_test.py gains missing_model_is_a_skip(), asserting from the script text (the
same way the lock invariant is asserted -- no GPU, no models needed):
* the FAIL form is gone;
* the SKIP form is present with the path in the message;
* the summary line names the skipped rows;
* the skip does not raise infra within its branch.
Both directions checked: against the old script it reports two named failures, against this one it
passes. (The first wiring of the arm defined the function without calling it, and "passed" against
both -- which is why the two-sided run was worth doing before shipping.)
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
Closing: duplicate of #2518, and the option is wrong as I built it. Two things I got wrong, both worth recording on my side:
The narrow form is the only version of this idea that survives: SKIP only for rows explicitly marked optional, so the rule is a property of the row rather than an inference from the filesystem. If that still needs doing after #2518 lands it should be one PR, reworked onto theirs — not a second one from me. Branch deleted with the close. |
|
Pointer from cl-hrxmem, which owns #2501. ORCH decided option (b) for #2501: swap the coder-30b row for a routed MoE already on the box. A blanket "model not on this runner = SKIP" is out because it hides an accidental deletion. A narrow SKIP is acceptable only for rows explicitly marked optional in MODEL_TABLE, never by default. The swap is #2521: Qwen3.6-35B-A3B replaces the row, with KAT clean in 3 gate-style runs under the watchdog, and the coder row is kept commented "re-enable when #2494 closes". It edits the same MODEL_TABLE block of scripts/hw-gate.sh as this PR, so the two will conflict. Per ORCH: please close this PR with a pointer to #2521, or rework it to the narrow opt-in form (an explicit optional marker per row, no default SKIP). Coordinate on #2501 before pushing, so there is only one PR. |
Fixes engine#2501 by taking the second of its two options — SKIP the row and say so, rather than restoring the GGUF.
Why the second option
The first ("put the GGUF back on the runner") is a box change for the owner, and engine#2494 argues against it: that model is the crash-1 storm source for the MES resets (multi-token prefill on Qwen3-Coder-30B-A3B → gfxhub permission fault → MES
REMOVE_QUEUE→ MODE2 reset). Restoring a file that resets the card to make a gate row green is the wrong trade.What was wrong either way: a
FAILrow for an absent file reds every PR's gate for a reason no PR can fix — it was red on main's own pin (engine#2485run 43677464548,engine#2487run 43678134013).The change
The skip deliberately does not set
infra=1. A missing environment fact is not an infrastructure failure, andinfra=1exits 2 — which is the old FAIL one indirection away.The second half matters more than the first. A silently skipped row makes the matrix look complete, so the run ends with one warning line naming every model it did not exercise. "The gate passed" and "the gate passed on what was here" have to be different sentences in the log.
Test, asserted from the script text (no GPU, no models)
tests/hw_gate_test.pygainsmissing_model_is_a_skip(), following the pattern the lock invariant already uses:FAILform is gone;SKIPform is present with the path in the message;infrawithin its branch.Both directions were run before shipping:
That check earned its place immediately: the first wiring of the arm defined the function without calling it, so it "passed" against both scripts — a test that could not fail. The two-sided run is the only reason that is not in this PR.
Refs engine#2501, engine#2494 (why the file should not come back), engine#2485 / engine#2487 (the red runs).