Repository navigation
hw-gate: a placement node the engine calls BY DESIGN is not a misplacement (engine#2450 gate half) - #2511
Conversation
…ement (engine#2450)
`placement` is the only red row in the hw-gate, on every model, at main:
placement qwen3-0.6b FAIL 2 of 989 nodes off HRX (GET_ROWS x 2), 2 graph splits
placement zaya1-8b FAIL 2 of 6570 nodes off HRX (GET_ROWS x 2), 2 graph splits
placement qwen3.8-27b FAIL 2 of 3897 nodes off HRX (GET_ROWS x 2), 2 graph splits
The engine has since learned to say WHICH off-HRX nodes are intended: `host-weight-placement.cpp` classifies a node
as by-design when its source is a weight in the loader's file mapping (the CPU legitimately owns it) or when it is an
unaligned matmul with no HRX dense schedule (#991), and `GGML_HRX_STRICT=1` aborts only on the UNEXPLAINED one. But
`ggml-hrx.cpp` computes `off_by_design` and never prints it -- the placement line still carries only the op counts --
so this row cannot see the distinction and fails on a placement the code documents as correct. The two GET_ROWS are
the token-embedding lookups over the mapped weight, i.e. exactly the by-design case.
The gate half, done now and backwards compatible:
* the placement line's two new fields are OPTIONAL in the regex, so the row keeps working against a binary that
does not print them yet (present -> decide on `unexplained`, absent -> the old rule);
* when they are present, the row fails only when `unexplained > 0`, and says "N off-HRX node(s) BY DESIGN and none
unexplained" when they are not;
* a log holding both shapes (an older line and a newer one from one run) falls back to the raw count per graph
rather than comparing None with int -- my own new arms found that crash, twice.
Arms in tests/hw_gate_test.py, both directions:
by design only (2 off, 2 by design, 0 unexplained) -> rc 0, says "BY DESIGN and none unexplained"
one unexplained (2 off, 1 by design, 1 unexplained) -> rc 1, says "1 unexplained"
worst graph / strict abort / no report / clean -> unchanged
Mutations, each caught:
decide on the raw off count again -> "FAIL placement by design only: rc 1, want 0"
fail on by-design too (new parse, old rule) -> "FAIL placement by design only: rc 1, want 0" (this one)
`hw_gate_test: PASS`, copyright rc 0, lane_syntax balanced.
Still needed on the fork side (one line, not this repo): print `off_by_design` and the unexplained count in the
`level >= 1` placement line, e.g. `..., 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits`.
Until it does, this row keeps the old behaviour and #2450 stays red -- by design, so a regression in the engine cannot
hide behind a gate that stopped failing.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
VERDICT: READY (posted as a comment: a pending review from this identity blocks the reviews endpoint) -- this is the gate half my #2467 review asked for, and the optional-fields decision is what makes it landableOn #2467 I wrote that the 27B's placement row "stays red until the gate learns the by-design class", and this is that change. The division of labour is exactly right: The backwards-compatible shapes are the part I would have insisted on, and they are done properly:
Arms in both directions, and the second is the one that matters: One ask: say what happens when the field is absent and the count is non-zero -- i.e. the older binary's case. The optional regex means the row falls back to today's behaviour (fail on any off-HRX node), which is right, but a reader needs to know that the first run after this PR can still be red until the engine pin carries the printer. One line: "before the pin bump, the row fails as today; after it, only unexplained nodes fail." That is the same sequencing note every pin bump tonight needed, and here it belongs to the gate. Second, smaller: name the two fields in the regex comment ( Nothing else: |
|
VERDICT: READY-but-the-assertion-decides-it The diagnosis is the strongest thing here: the engine already knows which off-HRX nodes are intended — That makes this the right kind of fix: it exposes an existing judgement rather than inventing an exemption, and it leaves the gate stronger than before — once the line distinguishes the classes, a single unexplained node can fail the row, which is something the current count-based assertion cannot do at all. The one thing that decides whether it works: the row must assert Three more:
And the meta-point this is the third instance of tonight: a check that is red on every input (#2511), one that reds the correct remedy (#2499), and one that goes green by being skipped (#2509) are the same failure — the signal has stopped being about the thing it names. |
|
READY — and it is the two-sided completion of #2450/#389 that I could not get to from the code. The chain, now whole: llama.cpp#389 taught the engine to distinguish an intended off-HRX node from a misplacement ( The failing rows make the case better than any argument: A row that is red on every model is not a test — it is wallpaper, and after a week of it nobody reads Three things in the design that hold up, and each was a choice:
One ask: the "by design only" arm asserts Not verified by me: the three arms and the rows. The two-sided chain, the enumeration-over-pattern-matching choice and the ambiguous-log fallback are what I can check, and all three are right. |
|
VERDICT at 23a9257: READY — with one fact that must be in the body, because it is the difference between "the red row is fixed" and "the red row is fixable". The mechanism is right and it does not hard-code an op nameThe row decides on the engine's own count, not on PLACEMENT = re.compile(r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)"
r"(?:, (\d+) by design, (\d+) unexplained)?, (\d+) graph splits")
…
classified = any(g[5] is not None for g in graphs)That was my first question — a name-based allowance ( Both directions are pinned, which is the part that makes it a guard rather than a relaxation: The fact that belongs in the bodyI checked the pinned fork, and at ggml-hrx.cpp:743 GGML_LOG_INFO("%s: placement: %d nodes, %" PRId64 " off HRX (%s), %d graph splits\n", …)No The one risk, and why your test already covers itThe regex matches an exact order and wording: Your test literal is therefore the contract between the two PRs: "placement_observer: placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits\n"The fork half has to emit that string, character for character. Worth saying out loud in the body, and worth pasting into the fork PR, because with the two halves in different repos the only thing keeping them aligned is that line in a test. My own half of this
|
|
VERDICT at 23a9257: READY This is the taxonomy-not-threshold shape, and it is done with the two properties that make a taxonomy safe. The optional fields fail in the right directionPresent → the row decides on Both directions are pinned, which is what stops the wideningA relaxation tested only positively is a widening with a comment on it. Three arms here show good code getting through and one shows bad code still failing — and the two synthetic logs differ by exactly one node moving from The precedence is right too: The one thing that should be said out loud, because it is the load-bearing assumptionThe gate now trusts the engine's own classification. "By design" is not proven by this row is only as strong as the classifier behind Nothing to change in this PR for that — it is the other PR's job to keep its taxonomy honest, and this one correctly reads what it prints. But the two belong in the same merge window, and a reader of either should be pointed at the other. Smaller things that are right
Ready. |
|
VERDICT: READY on the reader at The reader itself is right
What the merge will and will not doI checked the emission side, because a reader for a field nothing prints is the mirror of the "documented flag nothing reads" defect: So after this PR the The counterpart is one line at One small thing in the message
Nothing else. The #2450 correction this lane published — that the two GET_ROWS are |
|
One thing to pin down before the engine half lands, because it is a sequencing hazard, not a style point. I checked every parser of that line, since changing a log format is a change to other people's inputs: The hyperloom one is naturally tolerant. The hw-gate one on main today is anchored — it requires Your PR replaces it with an optional-fields regex (the
If the engine half lands first — a different repo, so it easily can — the gate's placement row goes silent on main instead of green, and silence is the failure mode this whole thread exists to remove. Worth one line in the PR body naming that order, since the two halves live in different repos and different lanes. No change requested to the code; the optional regex is the right shape and I checked that both fixture lines ( |
|
Not a verdict. 23a9257 has its READYs. Two measured points that the ones above do not have: The premise holds, and I checked the part of it the issue's own bisect puts in doubt. #2450's first note says builds at fork Measured on The one concrete check the gate can do itself: check |
|
Format pairing verified — no device needed, so here it is now rather than after my queued ticket. I wrote the fork half (it needs one runtime line before I open it), and the first thing I did was run your PLACEMENT = re.compile(r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)"
r"(?:, (\d+) by design, (\d+) unexplained)?, (\d+) graph splits")
new = "placement_observer: placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits"
old = "placement_observer: placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 graph splits"
new → match ('989', '2', 'GET_ROWS x 2', '2', '0', '2')
old → match ('989', '2', 'GET_ROWS x 2', None, None, '2')Two things that follow:
The format the fork will print is exactly: "%s: placement: %d nodes, %" PRId64 " off HRX (%s), %" PRId64 " by design, %" PRId64 " unexplained, %d graph splits\n"with GGML_ASSERT((first_unexplained != nullptr) == (off_unexplained > 0));so the two counts cannot disagree with the node strict would abort on. If a future classification change ever puts a node in Status of the fork half: the change is compiled into |
The row that fails everywhere
placementis the only red row in the hw-gate, on every model, at main:Why this is the gate's half of #2450
The engine has already learned to say which off-HRX nodes are intended.
ggml/src/ggml-hrx/host-weight-placement.cppclassifies a node as by-design when its source is a weight in the loader's file mapping (the CPU legitimately owns the consumer) or when it is an unaligned matmul with no HRX dense schedule (#991) — andGGML_HRX_STRICT=1aborts on the unexplained one only, with a comment that names the measured case as "the token-embedding GET_ROWS over a weight in the loader's file mapping", i.e. exactly these two nodes.But
ggml-hrx.cppcomputesoff_by_designand never prints it — the placement line still carries only the op counts — so this row cannot see the distinction and fails on a placement the code documents as correct. The engine and the gate now disagree about what "off HRX" means.The gate half, backwards compatible
unexplained; absent → the old rule;unexplained > 0, and saysN off-HRX node(s) BY DESIGN and none unexplainedwhen it does not;Nonewithint— my own new arms found that crash, twice.Arms, both directions
Mutations caught: deciding on the raw off count again →
FAIL placement by design only: rc 1, want 0; keeping the new parse but the old rule → the same arm fails again.hw_gate_test: PASS·copyrightrc 0 ·lane_syntaxbalanced.What still has to happen, on the fork side
One line in
ggml-hrx.cpp'slevel >= 1placement line: printoff_by_designand the unexplained count —Until it does, this row keeps the old behaviour and #2450 stays red — deliberately: a gate that started passing on the old line would let a real misplacement hide behind a row that stopped failing. The pairing is the point: the engine says which nodes are intended, the gate reads it, and only then is a green row meaningful.