Skip to content

hrx: the placement report says which off-HRX nodes are BY DESIGN (engine#2450) - #395

Merged
bong-water-water-bong merged 3 commits into
1bit/hrx-vulkan-patchedfrom
1bit/placement-by-design-report
Nov 15, 2026
Merged

bong-water-water-bong merged 3 commits into
1bit/hrx-vulkan-patchedfrom
1bit/placement-by-design-report

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Why this is one line and not a redesign

off_by_design is already computed on every graph — it is what decides the GGML_HRX_STRICT abort — and it was simply never printed. The level >= 1 placement line carried only the op counts:

placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 graph splits

So the hardware gate's placement row, which parses this line, could only compare the raw off-HRX count — and that row is the only red row in the gate on every model at main, always on the same two nodes (989 / 6570 / 3897 nodes; GET_ROWS x 2 in all three). The strict-mode comment names that exact case as by design: "the token-embedding GET_ROWS over a weight in the loader's file mapping". The row was failing on a placement this code documents as correct, and no engine change could have turned it green — the number it needed was never in the line.

The change

placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits

Verified without a device

The gate half (engine#2511) is deliberately backwards compatible: the two fields are optional in its regex, so it keeps today's behaviour until a binary prints them and then decides on unexplained alone. Composing the exact line this format produces and running that parser over it:

2 off-HRX node(s) BY DESIGN and none unexplained, in 1 graphs (largest 989 nodes)      rc 0
2 of 989 nodes off HRX (GET_ROWS x 2), 2 graph splits, 0 by design, 2 unexplained …   rc 1

The file compiles: -fsyntax-only with the hw-gate build's own compile command, re-mapped to this tree — and I proved the check reads this file by injecting an error first (expected expression at the new line), then restoring to a clean rc 0.

What this does not claim

Whether those two nodes are actually off_mapped_weights. If they are, the gate row goes green with the right meaning (2 nodes off HRX, both intended, none misplaced). If they are not, the new field says "2 unexplained" and that is an engine bug rather than a row that needs relaxing. Either way the next gate run answers engine#2450 instead of restating it.

bong-water-water-bong and others added 3 commits November 15, 2026 04:25
…ggml-org#2500 follow-up)

test-hrx-loader-mapped loads a model on HRX and reads the loader's "model buffer size" lines: at least one
HRX0_Mapped, none HRX0_Imported or nameless. Before #393 it fails on the old libraries ("load_tensors:
model buffer size", no HRX0_Mapped); with #393 it passes. It skips without a model (argv[1] or
LLAMACPP_TEST_MODELFILE) or an HRX device. hrx-buffer-test loader_mmap_proc also requires the imported
buffer type to carry its device (the zeroed context was half of ggml-org#2500). Both were asked for on #393.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…#394 review)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ine#2450)

`off_by_design` is computed on every graph (a weight in the loader's file mapping, or an unaligned matmul with no HRX
dense schedule) and it decides the GGML_HRX_STRICT abort -- but it was never printed. The `level >= 1` placement line
carried only the op counts:

    placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 graph splits

so a reader of the log -- and the hardware gate's `placement` row, which parses this line -- could only compare the raw
off-HRX count. That row is the only red row in the gate on every model at main, always on the same two nodes:

    2 of 989 nodes off HRX (GET_ROWS x 2)     qwen3-0.6b
    2 of 6570 nodes off HRX (GET_ROWS x 2)    zaya1-8b
    2 of 3897 nodes off HRX (GET_ROWS x 2)    qwen3.8-27b

and the strict-mode comment names that exact case as by design: "the token-embedding GET_ROWS over a weight in the
loader's file mapping". So the row was failing on a placement this code documents as correct, and no engine change
could have turned it green -- the number it needed was never in the line.

Two counts, beside the total:

    placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits

The gate half is engine#2511, and it is backwards compatible on purpose: the two fields are optional in its regex, so
it keeps today's behaviour until a binary prints them, then decides on `unexplained` alone. Verified end to end
without a device, by composing the exact line this format prints and running the gate's parser over it:

    2 off-HRX node(s) BY DESIGN and none unexplained, in 1 graphs (largest 989 nodes)          rc 0
    2 of 989 nodes off HRX (GET_ROWS x 2), 2 graph splits, 0 by design, 2 unexplained …       rc 1

and the file compiles: `-fsyntax-only` with the hw-gate build's own compile command, re-mapped to this tree (proved by
injecting an error first -- "expected expression" at the new line -- and getting a clean rc 0 after restoring).

Not measured here: the run that would say whether those two nodes are actually `off_mapped_weights`. If they are, the
gate row goes green with the right meaning; if they are not, the new number says "2 unexplained" and that is an engine
bug rather than a row that needs relaxing. Either way the next gate run answers engine#2450.
@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY 395

This and engine#2511 are one fix in two repos, and the split is right: off_by_design is computed here and consumed by the gate there. "off_by_design is already computed on every graph — it is what decides the GGML_HRX_STRICT abort — and it was simply never printed" is the whole story, and a one-line print is the correct size for it.

The thing that decides whether two PRs in two repos stay one fix: the printed line is about to become a parsed interface — the hw-gate will read it to distinguish by-design from unexplained. So:

  1. fix the field order and make it machine-readable (off_by_design=<n> unexplained=<n> or similar), not prose that reads well once and cannot be parsed twice. A line whose format is an accident becomes an interface by accretion, and then nobody dares change it;
  2. print both numbers even when one is zero — unexplained=0 present is what lets the gate assert on something. A field that is only printed when non-zero forces the consumer to treat absence as zero, which is the ambiguity that produces a green gate on a graph that failed to report;
  3. GGML_HRX_STRICT and this line must come from the same classifier call, not two traversals that could disagree (DS_docs ggml-org/llama.cpp#2511 asked the same on the gate side). If they can diverge, the tree has two opinions about one node and the gate's green means nothing.

And one coordination note, since both PRs are open in different repos: whichever merges second should not silently depend on the other. If ggml-org#2511's gate starts asserting on a field that this PR introduces, then ggml-org#2511 is not mergeable before #395 — say so in one of the two descriptions, because the merge loop will otherwise take the gate first, get an unparsed line, and go red for a reason nobody is looking for.

@bong-water-water-bong

Copy link
Copy Markdown
Author

READY — and it is the matching half of engine#2511, with the backwards-compatibility argument running in the direction that matters.

The line goes from

placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 graph splits

to

placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits

and the pair lands in either order: this fork prints the fields, ggml-org#2511's regex treats them as optional, so an older gate reading a newer log ignores them and a newer gate reading an older log falls back to the raw count. That is what makes two PRs in two repos safe to merge independently — the alternative is a window where every gate is red because one side shipped first.

Three things I verified or particularly want to credit:

  • off_by_design was already computed — "it is what decides the GGML_HRX_STRICT abort" — so this is a print of a value the code already had, not a second derivation. That is why it is one line: the decision was made by hrx: strict refuses a misplacement, not a placement the code calls by design (engine #2450) #389, and this is the reporting of it. A PR that re-computed the predicate here would have created the second copy of the logic that hrx: strict refuses a misplacement, not a placement the code calls by design (engine #2450) #389 explicitly avoided.
  • The two contrasted rows in the verification are the right pair: 2 by design, 0 unexplained ⇒ rc 0; 0 by design, 2 unexplained ⇒ rc 1. Same raw count of 2 in both, opposite verdicts — which is the only way to show the gate reads the new fields rather than the count.
  • -fsyntax-only with the hw-gate build's own compile command, re-mapped to this tree. That is the check I had to learn the hard way (my own stale-build-dir near-miss, CMAKE_HOME_DIRECTORY pointing at another tree, which would have "compiled" the wrong file). Using the real command means the flags that matter are the ones the gate will use.

One ask: the message prints 2 by design — a count, which answers my ggml-org#2511 question about a growing by-design class. But it prints it only in the gate's rendering; does the fork's own placement: line carry the count too, or just the raw off-HRX number plus the two field names? The example line shows 2 by design so I believe it does — worth one clause saying the per-class counts are printed, because that is what lets a doubling be visible in the gate's output rather than needing a second probe.

Second, smaller: with four numbers on the line now (off, by design, unexplained, splits), the regex ordering is load-bearing. If ggml-org#2511's pattern anchors on order, say so, and if it matches by name, say that instead — the difference decides whether a future field can be inserted in the middle.

Not verified by me: the two gate rows (they need an HRX build). The one-line-ness, the backwards compatibility and the -fsyntax-only check are what I can check, and all three are right.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- one line, and the two halves are verified against each other rather than separately

The argument for "one line and not a redesign" is complete in the body: off_by_design is already computed on every graph -- it decides the GGML_HRX_STRICT abort -- and was simply never printed, so the gate could only compare a raw count. And the strict-mode comment names that exact case as by design: "the token-embedding GET_ROWS over a weight in the loader's file mapping". So the row was failing on a placement the code documents as correct, and no engine change could have turned it green -- the number it needed was never in the line. That is the cleanest statement of a missing-field bug I have read tonight.

What I most want to keep is the cross-verification: the gate half (ggml-org#2511) is backwards compatible because the fields are optional, and this PR composes the exact line its format produces and runs ggml-org#2511's parser over it:

2 off-HRX node(s) BY DESIGN and none unexplained, in 1 graphs (largest 989 nodes)      rc 0
2 of 989 nodes off HRX (GET_ROWS x 2), 2 graph splits, 0 by design, 2 unexplained …   rc 1

Two PRs written against each other's contract, tested by feeding one's output to the other's parser, without a device. That is how a producer/consumer pair should be landed, and it is why the optional-fields decision pays off twice.

And proving the syntax check reads this file by injecting an error first (expected expression) is the discipline that stops "it compiles" from being a claim about the wrong file -- the same reason I insist on a mutant for a check.

One ask: name the field order in a comment at the format string, not only in ggml-org#2511's regex. Two PRs now depend on … off HRX (ops), N by design, M unexplained, K graph splits, and the order is load-bearing for a regex that allows them optional -- a future edit that swaps the two counts would keep compiling and quietly invert the gate's verdict. One comment at the producer is cheaper than a test at the consumer.

Second, smaller: the sample line has 0 by design, 2 unexplained for the failing case, i.e. the failing shape has zero by-design nodes -- worth saying, because "unexplained but nothing by design" is the shape a genuine misplacement takes, and it makes the pair of examples a specification rather than two outputs.

Nothing else: +99/-2, the file compiles with the gate's own command, and the placement line's existing fields are unchanged in spelling and order, so nothing else that greps it can break.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at f897e4c: READY. I reviewed this against the engine half an hour ago with the format in hand, so this is the cross-repo check rather than a read of one PR.

The contract holds, character for character

Engine side (#2511, the regex that decides the row):

r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)(?:, (\d+) by design, (\d+) unexplained)?, (\d+) graph splits"

Engine side, the literal the test pins:

"placement_observer: placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits"

Here:

GGML_LOG_INFO("%s: placement: %d nodes, %" PRId64 " off HRX (%s), %" PRId64 " by design, %" PRId64
              " unexplained, %d graph splits\n", …);
// e.g. "2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits"

Order, nouns, separators and the splits tail all match, and your own comment quotes the same example line. With the halves in different repositories that is the only thing keeping them aligned, and it is aligned.

The arithmetic cannot produce a false green

unexplained is not its own counter, it is off_nodes - off_by_design. That is only safe if off_by_design <= off_nodes, because a negative unexplained count would satisfy no comparison and read as clean:

off_nodes++;                                        // every non-view, non-HRX node
…
const bool by_design = ggml::hrx::placement_node_is_by_design(facts);
if (by_design) { off_by_design++; }                 // at most once per iteration
else if (first_unexplained == nullptr) { … }

Both happen in the same loop body, off_by_design at most once per node, so 0 <= off_by_design <= off_nodes and the subtraction is exactly "off-HRX nodes that are not by design". No third bucket, and off_mapped_weights (which counts sources, not nodes) never enters the subtraction.

The property that actually matters: the log cannot contradict strict

strict and the report are fed by the same loop and the same predicate: first_unexplained is set in the else of by_design, and the print reports off_nodes - off_by_design. So "the row goes green" and "strict does not abort" are statements about one decision, not two that could drift. A report that disagreed with the abort would be worse than no report — it would turn the gate green over a graph strict refuses.

And the predicate has its own arms, both directions, six cases:

expect(placement_node_is_by_design(f), true,  "unaligned contraction (ne0 = …)");
expect(placement_node_is_by_design(f), true,  "a host-resident weight is by …");
expect(placement_node_is_by_design(f), false, "an aligned contraction with …");
expect(placement_node_is_by_design(f), false, "a non-contraction is not ex…");
expect(placement_node_is_by_design(f), false, "a contraction with no src0 …");
expect(placement_node_is_by_design(placement_node_facts{}), false, "empty fact…");

The false cases are what stop "by design" from becoming a synonym for "off HRX".

Merge order, since it decides when anything changes

#2511 (engine) is safe alone — the optional group misses and the row keeps the old rule. Then this, then a pin bump: the pin bump is the step that moves placement from red to green, and until it happens neither PR changes a single row. Worth saying in one of the two bodies so nobody merges both and wonders why the row is still red.

@bong-water-water-bong
bong-water-water-bong merged commit 712d2ca into 1bit/hrx-vulkan-patched Nov 15, 2026
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/placement-by-design-report branch November 15, 2026 09:16
@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at f897e4c — and the two halves now agree character for character, which is the check that mattered.

The format matches the reader that is already merged

Engine (ggml-hrx.cpp:748, this PR):

GGML_LOG_INFO("%s: placement: %d nodes, %" PRId64 " off HRX (%s), %" PRId64 " by design, %" PRId64
              " unexplained, %d graph splits\n", __func__, graph->n_nodes,
              off_nodes, off_detail.c_str(), off_by_design, off_nodes - off_by_design,
              ggml_backend_sched_get_n_splits(sched));

Reader (tools/hw_gate_manifest.py:163, engine#2511, MERGED):

PLACEMENT = re.compile(r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)" … )

Same field order, same separators, both counts derived from counters that already existed (off_by_design, and unexplained = off_nodes - off_by_design) — so no new state, no drift between what the engine means and what the gate reads.

And the sequencing landed in the right order, which was the hazard I raised on ggml-org#2511: the reader is merged first and is tolerant of both shapes, so this writer cannot make hw_gate_manifest.py placement go silent the way main's old anchored regex would have. The other parser, tools/hyperloom/custom/benchmark.py:718, is unanchored after off HRX and needs nothing.

What the row will say after the pin moves

placement  PASS  2 off-HRX node(s) BY DESIGN and none unexplained

which is the first time this row has been green on any model, and it will be green for a documented reason (a weight in the loader's file mapping, the CPU legitimately owning its consumer) rather than because the check was loosened. The GGML_HRX_STRICT=1 abort path is untouched — "strict refuses a MISPLACEMENT, not a placement the code calls by design" — so a real misplacement still aborts the server and still fails the row.

Two notes, neither blocking

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant