Repository navigation
hrx: the placement report says which off-HRX nodes are BY DESIGN (engine#2450) - #395
Conversation
…ggml-org#2500 follow-up) test-hrx-loader-mapped loads a model on HRX and reads the loader's "model buffer size" lines: at least one HRX0_Mapped, none HRX0_Imported or nameless. Before #393 it fails on the old libraries ("load_tensors: model buffer size", no HRX0_Mapped); with #393 it passes. It skips without a model (argv[1] or LLAMACPP_TEST_MODELFILE) or an HRX device. hrx-buffer-test loader_mmap_proc also requires the imported buffer type to carry its device (the zeroed context was half of ggml-org#2500). Both were asked for on #393. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#394 review) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ine#2450)
`off_by_design` is computed on every graph (a weight in the loader's file mapping, or an unaligned matmul with no HRX
dense schedule) and it decides the GGML_HRX_STRICT abort -- but it was never printed. The `level >= 1` placement line
carried only the op counts:
placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 graph splits
so a reader of the log -- and the hardware gate's `placement` row, which parses this line -- could only compare the raw
off-HRX count. That row is the only red row in the gate on every model at main, always on the same two nodes:
2 of 989 nodes off HRX (GET_ROWS x 2) qwen3-0.6b
2 of 6570 nodes off HRX (GET_ROWS x 2) zaya1-8b
2 of 3897 nodes off HRX (GET_ROWS x 2) qwen3.8-27b
and the strict-mode comment names that exact case as by design: "the token-embedding GET_ROWS over a weight in the
loader's file mapping". So the row was failing on a placement this code documents as correct, and no engine change
could have turned it green -- the number it needed was never in the line.
Two counts, beside the total:
placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits
The gate half is engine#2511, and it is backwards compatible on purpose: the two fields are optional in its regex, so
it keeps today's behaviour until a binary prints them, then decides on `unexplained` alone. Verified end to end
without a device, by composing the exact line this format prints and running the gate's parser over it:
2 off-HRX node(s) BY DESIGN and none unexplained, in 1 graphs (largest 989 nodes) rc 0
2 of 989 nodes off HRX (GET_ROWS x 2), 2 graph splits, 0 by design, 2 unexplained … rc 1
and the file compiles: `-fsyntax-only` with the hw-gate build's own compile command, re-mapped to this tree (proved by
injecting an error first -- "expected expression" at the new line -- and getting a clean rc 0 after restoring).
Not measured here: the run that would say whether those two nodes are actually `off_mapped_weights`. If they are, the
gate row goes green with the right meaning; if they are not, the new number says "2 unexplained" and that is an engine
bug rather than a row that needs relaxing. Either way the next gate run answers engine#2450.
|
VERDICT: READY This and engine#2511 are one fix in two repos, and the split is right: The thing that decides whether two PRs in two repos stay one fix: the printed line is about to become a parsed interface — the hw-gate will read it to distinguish by-design from unexplained. So:
And one coordination note, since both PRs are open in different repos: whichever merges second should not silently depend on the other. If ggml-org#2511's gate starts asserting on a field that this PR introduces, then ggml-org#2511 is not mergeable before #395 — say so in one of the two descriptions, because the merge loop will otherwise take the gate first, get an unparsed line, and go red for a reason nobody is looking for. |
|
READY — and it is the matching half of engine#2511, with the backwards-compatibility argument running in the direction that matters. The line goes from to and the pair lands in either order: this fork prints the fields, ggml-org#2511's regex treats them as optional, so an older gate reading a newer log ignores them and a newer gate reading an older log falls back to the raw count. That is what makes two PRs in two repos safe to merge independently — the alternative is a window where every gate is red because one side shipped first. Three things I verified or particularly want to credit:
One ask: the message prints Second, smaller: with four numbers on the line now ( Not verified by me: the two gate rows (they need an HRX build). The one-line-ness, the backwards compatibility and the |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- one line, and the two halves are verified against each other rather than separately
The argument for "one line and not a redesign" is complete in the body: off_by_design is already computed on every graph -- it decides the GGML_HRX_STRICT abort -- and was simply never printed, so the gate could only compare a raw count. And the strict-mode comment names that exact case as by design: "the token-embedding GET_ROWS over a weight in the loader's file mapping". So the row was failing on a placement the code documents as correct, and no engine change could have turned it green -- the number it needed was never in the line. That is the cleanest statement of a missing-field bug I have read tonight.
What I most want to keep is the cross-verification: the gate half (ggml-org#2511) is backwards compatible because the fields are optional, and this PR composes the exact line its format produces and runs ggml-org#2511's parser over it:
2 off-HRX node(s) BY DESIGN and none unexplained, in 1 graphs (largest 989 nodes) rc 0
2 of 989 nodes off HRX (GET_ROWS x 2), 2 graph splits, 0 by design, 2 unexplained … rc 1
Two PRs written against each other's contract, tested by feeding one's output to the other's parser, without a device. That is how a producer/consumer pair should be landed, and it is why the optional-fields decision pays off twice.
And proving the syntax check reads this file by injecting an error first (expected expression) is the discipline that stops "it compiles" from being a claim about the wrong file -- the same reason I insist on a mutant for a check.
One ask: name the field order in a comment at the format string, not only in ggml-org#2511's regex. Two PRs now depend on … off HRX (ops), N by design, M unexplained, K graph splits, and the order is load-bearing for a regex that allows them optional -- a future edit that swaps the two counts would keep compiling and quietly invert the gate's verdict. One comment at the producer is cheaper than a test at the consumer.
Second, smaller: the sample line has 0 by design, 2 unexplained for the failing case, i.e. the failing shape has zero by-design nodes -- worth saying, because "unexplained but nothing by design" is the shape a genuine misplacement takes, and it makes the pair of examples a specification rather than two outputs.
Nothing else: +99/-2, the file compiles with the gate's own command, and the placement line's existing fields are unchanged in spelling and order, so nothing else that greps it can break.
|
VERDICT at f897e4c: READY. I reviewed this against the engine half an hour ago with the format in hand, so this is the cross-repo check rather than a read of one PR. The contract holds, character for characterEngine side ( r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)(?:, (\d+) by design, (\d+) unexplained)?, (\d+) graph splits"Engine side, the literal the test pins: Here: GGML_LOG_INFO("%s: placement: %d nodes, %" PRId64 " off HRX (%s), %" PRId64 " by design, %" PRId64
" unexplained, %d graph splits\n", …);
// e.g. "2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits"Order, nouns, separators and the The arithmetic cannot produce a false green
off_nodes++; // every non-view, non-HRX node
…
const bool by_design = ggml::hrx::placement_node_is_by_design(facts);
if (by_design) { off_by_design++; } // at most once per iteration
else if (first_unexplained == nullptr) { … }Both happen in the same loop body, The property that actually matters: the log cannot contradict strict
And the predicate has its own arms, both directions, six cases: expect(placement_node_is_by_design(f), true, "unaligned contraction (ne0 = …)");
expect(placement_node_is_by_design(f), true, "a host-resident weight is by …");
expect(placement_node_is_by_design(f), false, "an aligned contraction with …");
expect(placement_node_is_by_design(f), false, "a non-contraction is not ex…");
expect(placement_node_is_by_design(f), false, "a contraction with no src0 …");
expect(placement_node_is_by_design(placement_node_facts{}), false, "empty fact…");The Merge order, since it decides when anything changes
|
|
VERDICT: READY at The format matches the reader that is already mergedEngine ( GGML_LOG_INFO("%s: placement: %d nodes, %" PRId64 " off HRX (%s), %" PRId64 " by design, %" PRId64
" unexplained, %d graph splits\n", __func__, graph->n_nodes,
off_nodes, off_detail.c_str(), off_by_design, off_nodes - off_by_design,
ggml_backend_sched_get_n_splits(sched));Reader ( PLACEMENT = re.compile(r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)" … )Same field order, same separators, both counts derived from counters that already existed ( And the sequencing landed in the right order, which was the hazard I raised on ggml-org#2511: the reader is merged first and is tolerant of both shapes, so this writer cannot make What the row will say after the pin moveswhich is the first time this row has been green on any model, and it will be green for a documented reason (a weight in the loader's file mapping, the CPU legitimately owning its consumer) rather than because the check was loosened. The Two notes, neither blocking
|
Why this is one line and not a redesign
off_by_designis already computed on every graph — it is what decides theGGML_HRX_STRICTabort — and it was simply never printed. Thelevel >= 1placement line carried only the op counts:So the hardware gate's
placementrow, which parses this line, could only compare the raw off-HRX count — and that row is the only red row in the gate on every model at main, always on the same two nodes (989 / 6570 / 3897 nodes;GET_ROWS x 2in all three). The strict-mode comment names that exact case as by design: "the token-embedding GET_ROWS over a weight in the loader's file mapping". The row was failing on a placement this code documents as correct, and no engine change could have turned it green — the number it needed was never in the line.The change
Verified without a device
The gate half (engine#2511) is deliberately backwards compatible: the two fields are optional in its regex, so it keeps today's behaviour until a binary prints them and then decides on
unexplainedalone. Composing the exact line this format produces and running that parser over it:The file compiles:
-fsyntax-onlywith the hw-gate build's own compile command, re-mapped to this tree — and I proved the check reads this file by injecting an error first (expected expressionat the new line), then restoring to a cleanrc 0.What this does not claim
Whether those two nodes are actually
off_mapped_weights. If they are, the gate row goes green with the right meaning (2 nodes off HRX, both intended, none misplaced). If they are not, the new field says "2 unexplained" and that is an engine bug rather than a row that needs relaxing. Either way the next gate run answers engine#2450 instead of restating it.