Skip to content

hw-gate: a placement node the engine calls BY DESIGN is not a misplacement (engine#2450 gate half) - #2511

Merged
bong-water-water-bong merged 1 commit into
mainfrom
lane/placement-by-design
Nov 15, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
lane/placement-by-design

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

The row that fails everywhere

placement is the only red row in the hw-gate, on every model, at main:

placement qwen3-0.6b   FAIL  2 of 989  nodes off HRX (GET_ROWS x 2), 2 graph splits
placement zaya1-8b     FAIL  2 of 6570 nodes off HRX (GET_ROWS x 2), 2 graph splits
placement qwen3.8-27b  FAIL  2 of 3897 nodes off HRX (GET_ROWS x 2), 2 graph splits

Why this is the gate's half of #2450

The engine has already learned to say which off-HRX nodes are intended. ggml/src/ggml-hrx/host-weight-placement.cpp classifies a node as by-design when its source is a weight in the loader's file mapping (the CPU legitimately owns the consumer) or when it is an unaligned matmul with no HRX dense schedule (#991) — and GGML_HRX_STRICT=1 aborts on the unexplained one only, with a comment that names the measured case as "the token-embedding GET_ROWS over a weight in the loader's file mapping", i.e. exactly these two nodes.

But ggml-hrx.cpp computes off_by_design and never prints it — the placement line still carries only the op counts — so this row cannot see the distinction and fails on a placement the code documents as correct. The engine and the gate now disagree about what "off HRX" means.

The gate half, backwards compatible

  • the placement line's two new fields are optional in the regex, so the row keeps working against a binary that does not print them yet: present → decide on unexplained; absent → the old rule;
  • when present, the row fails only on unexplained > 0, and says N off-HRX node(s) BY DESIGN and none unexplained when it does not;
  • a log carrying both shapes (an older line and a newer one in one run) falls back to the raw count per graph instead of comparing None with int — my own new arms found that crash, twice.

Arms, both directions

by design only  (2 off, 2 by design, 0 unexplained)  -> rc 0, "BY DESIGN and none unexplained"
one unexplained (2 off, 1 by design, 1 unexplained)  -> rc 1, "1 unexplained"
worst graph / strict abort / no report / clean       -> unchanged

Mutations caught: deciding on the raw off count again → FAIL placement by design only: rc 1, want 0; keeping the new parse but the old rule → the same arm fails again.

hw_gate_test: PASS · copyright rc 0 · lane_syntax balanced.

What still has to happen, on the fork side

One line in ggml-hrx.cpp's level >= 1 placement line: print off_by_design and the unexplained count —

placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits

Until it does, this row keeps the old behaviour and #2450 stays red — deliberately: a gate that started passing on the old line would let a real misplacement hide behind a row that stopped failing. The pairing is the point: the engine says which nodes are intended, the gate reads it, and only then is a green row meaningful.

…ement (engine#2450)

`placement` is the only red row in the hw-gate, on every model, at main:

    placement qwen3-0.6b   FAIL  2 of 989  nodes off HRX (GET_ROWS x 2), 2 graph splits
    placement zaya1-8b     FAIL  2 of 6570 nodes off HRX (GET_ROWS x 2), 2 graph splits
    placement qwen3.8-27b  FAIL  2 of 3897 nodes off HRX (GET_ROWS x 2), 2 graph splits

The engine has since learned to say WHICH off-HRX nodes are intended: `host-weight-placement.cpp` classifies a node
as by-design when its source is a weight in the loader's file mapping (the CPU legitimately owns it) or when it is an
unaligned matmul with no HRX dense schedule (#991), and `GGML_HRX_STRICT=1` aborts only on the UNEXPLAINED one. But
`ggml-hrx.cpp` computes `off_by_design` and never prints it -- the placement line still carries only the op counts --
so this row cannot see the distinction and fails on a placement the code documents as correct. The two GET_ROWS are
the token-embedding lookups over the mapped weight, i.e. exactly the by-design case.

The gate half, done now and backwards compatible:

  * the placement line's two new fields are OPTIONAL in the regex, so the row keeps working against a binary that
    does not print them yet (present -> decide on `unexplained`, absent -> the old rule);
  * when they are present, the row fails only when `unexplained > 0`, and says "N off-HRX node(s) BY DESIGN and none
    unexplained" when they are not;
  * a log holding both shapes (an older line and a newer one from one run) falls back to the raw count per graph
    rather than comparing None with int -- my own new arms found that crash, twice.

Arms in tests/hw_gate_test.py, both directions:

    by design only  (2 off, 2 by design, 0 unexplained) -> rc 0, says "BY DESIGN and none unexplained"
    one unexplained (2 off, 1 by design, 1 unexplained) -> rc 1, says "1 unexplained"
    worst graph / strict abort / no report / clean    -> unchanged

Mutations, each caught:
    decide on the raw off count again        -> "FAIL placement by design only: rc 1, want 0"
    fail on by-design too (new parse, old rule) -> "FAIL placement by design only: rc 1, want 0" (this one)

`hw_gate_test: PASS`, copyright rc 0, lane_syntax balanced.

Still needed on the fork side (one line, not this repo): print `off_by_design` and the unexplained count in the
`level >= 1` placement line, e.g. `..., 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits`.
Until it does, this row keeps the old behaviour and #2450 stays red -- by design, so a regression in the engine cannot
hide behind a gate that stopped failing.
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

2450 - Partially compliant

Compliant requirements:

  • Updated the placement observer to classify nodes as by-design or unexplained
  • Modified the hw-gate logic to only fail on unexplained off-HRX nodes
  • Maintained backward compatibility with older binaries that don't print the new fields

Non-compliant requirements:

  • The actual fix for the root cause (changing buffer type from imported_buft to mapped_buft for mmap-backed weights) is not included in this PR

Requires further human verification:

  • Verification that the root cause fix (buffer type change) is correctly implemented in the engine codebase
  • Confirmation that the hw-gate now passes for all affected models

991 - Partially compliant

Compliant requirements:

  • The placement observer now correctly handles the new classification fields
  • The hw-gate logic properly distinguishes between by-design and unexplained placements

Non-compliant requirements:

  • The actual tail schedule implementation for non-256-aligned MUL_MAT operations is not included in this PR
  • The load-time fallback log line is not implemented in this PR

Requires further human verification:

  • Verification that the tail schedule implementation is correctly implemented in the engine codebase
  • Confirmation that the load-time fallback log line is properly implemented
⏱️ Estimated effort to review: 3 🔵🔵🔵⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Backward Compatibility

The change introduces optional fields in the placement regex to maintain backward compatibility, but this approach relies on the assumption that older binaries will not print the new fields. If an older binary were to unexpectedly print these fields, the parsing logic could misinterpret the data. The logic correctly handles the case where both old and new formats exist in the same log, but this scenario should be verified to be robust.

PLACEMENT = re.compile(r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)"
                       r"(?:, (\d+) by design, (\d+) unexplained)?, (\d+) graph splits")
Classification Logic

The new classification logic in the placement checker correctly distinguishes between by-design and unexplained placements. However, the logic assumes that when new fields are present, they are accurate. If the engine's classification logic were to incorrectly classify a node, this would lead to the gate incorrectly passing a misplacement. The gate's behavior depends on the correctness of the engine's classification, which is not verified in this PR.

if classified:
    # A log can hold both shapes (an older line and a newer one from the same run); an unclassified graph falls
    # back to its raw count so the comparison never mixes None with int.
    worst = max(graphs, key=lambda g: (g[5] if g[5] is not None else g[1], g[1]))
    unexplained, by_design = (worst[5] if worst[5] is not None else worst[1]), (worst[4] or 0)
else:
    worst = max(graphs, key=lambda g: g[1])
    unexplained, by_design = worst[1], 0
if unexplained > 0:
    bad = sum(1 for g in graphs if (g[5] if g[5] is not None else g[1]) > 0)
    print(f"{worst[1]} of {worst[0]} nodes off HRX ({worst[2]}), {worst[3]} graph splits"
          + (f", {by_design} by design, {unexplained} unexplained" if classified else "")
          + f"; {bad} of {len(graphs)} graphs had misplaced nodes")
    return 1
if classified and by_design > 0:
    print(f"{by_design} off-HRX node(s) BY DESIGN and none unexplained, in {len(graphs)} graphs "
          f"(largest {largest} nodes); the engine classifies mapped-weight and unaligned-matmul nodes as intended")
    return 0

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY (posted as a comment: a pending review from this identity blocks the reviews endpoint) -- this is the gate half my #2467 review asked for, and the optional-fields decision is what makes it landable

On #2467 I wrote that the 27B's placement row "stays red until the gate learns the by-design class", and this is that change. The division of labour is exactly right: host-weight-placement.cpp already classifies the off-HRX nodes, ggml-hrx.cpp computes off_by_design and never prints it, and the gate therefore fails on a raw count it cannot interpret. Adding the two fields to the placement line and failing only on unexplained > 0 is the smallest change that lets the gate read the engine's own opinion instead of guessing.

The backwards-compatible shapes are the part I would have insisted on, and they are done properly:

  • the two new fields are optional in the regex, so the row still works against a binary that does not print them -- which is what allows the gate and the engine pin to move independently;
  • a log carrying both an older and a newer line falls back to the raw count per graph rather than trusting the newer one, which is the correct conservative choice for a mixed run (a warm-up from a cached binary plus a fresh one is exactly how that happens);
  • and the success message says what it accepted: N off-HRX node(s) BY DESIGN and none unexplained. A green row that names its reason is the difference between "the gate learned something" and "the gate stopped looking".

Arms in both directions, and the second is the one that matters: by design only (2 off, 2 by design, 0 unexplained) -> rc 0 and one unexplained (2 off, 1 by design, 1 unexplained) -> rc 1, "1 unexplained". Without the second arm, a gate that simply accepted off_by_design > 0 as success would pass the first and stop being a placement gate at all.

One ask: say what happens when the field is absent and the count is non-zero -- i.e. the older binary's case. The optional regex means the row falls back to today's behaviour (fail on any off-HRX node), which is right, but a reader needs to know that the first run after this PR can still be red until the engine pin carries the printer. One line: "before the pin bump, the row fails as today; after it, only unexplained nodes fail." That is the same sequencing note every pin bump tonight needed, and here it belongs to the gate.

Second, smaller: name the two fields in the regex comment (off_by_design and unexplained, or whatever their spellings are), because the next person editing a placement message will not know which of the line's numbers are load-bearing.

Nothing else: +38/-8, the row keeps working against both binaries, and qwen3-0.6b, zaya1-8b and qwen3.8-27b all showing the same GET_ROWS x 2 is the evidence that this is one by-design class rather than three model problems.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY-but-the-assertion-decides-it 2511

The diagnosis is the strongest thing here: the engine already knows which off-HRX nodes are intended — host-weight-placement.cpp classifies a node as by-design when its source is a weight in the loader's file mapping, or an unaligned matmul with no dense schedule (#991) — and GGML_HRX_STRICT=1 already aborts on the unexplained one only, with the comment naming "the token-embedding GET_ROWS over a weight in the loader's file mapping", which is exactly these two nodes. What was missing is that off_by_design is computed and never printed, so the row cannot see a distinction the engine already makes.

That makes this the right kind of fix: it exposes an existing judgement rather than inventing an exemption, and it leaves the gate stronger than before — once the line distinguishes the classes, a single unexplained node can fail the row, which is something the current count-based assertion cannot do at all.

The one thing that decides whether it works: the row must assert unexplained == 0, not off_by_design == 2 (or "matches the manifest"). If it asserts the expected by-design count, the row is green by construction — it can only fail when the number changes, which is the least interesting way for placement to break. Asserting "nothing unexplained" makes every future GET_ROWS-as-embedding green and every accidental fallback red, which is the discrimination the row was supposed to provide in the first place.

Three more:

  1. print the classification per node, not a total — "2 off HRX (GET_ROWS ×2, both by-design: weight in file mapping)" versus "…, 2 by-design / 0 unexplained". The first is auditable; the second is a number a reader has to trust. hw_gate_manifest.py should carry the reason per node class, not a count;
  2. the strict abort and the printed line must share one classifier — if GGML_HRX_STRICT=1 and the gate disagree about which node is by-design, the tree has two opinions about the same node and the gate's green is worth nothing. One function, two consumers;
  3. the row should keep a red arm: assert that a synthetic unexplained node (or the #991 unaligned-matmul case with the schedule removed) does fail the row — otherwise this PR converts "red on every model" into "green on every model", and those are equally uninformative.

And the meta-point this is the third instance of tonight: a check that is red on every input (#2511), one that reds the correct remedy (#2499), and one that goes green by being skipped (#2509) are the same failure — the signal has stopped being about the thing it names. tests/hw_gate_test.py is the right place to install the arm that keeps this one honest.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

READY — and it is the two-sided completion of #2450/#389 that I could not get to from the code.

The chain, now whole: llama.cpp#389 taught the engine to distinguish an intended off-HRX node from a misplacement (host_weight_is_host_resident + off_by_design, strict aborts only on first_unexplained); ggml-hrx.cpp computes off_by_design and never prints it; and this PR teaches the gate to read it. Each half was useless alone — the engine knew and said nothing, the gate said something and knew nothing.

The failing rows make the case better than any argument:

placement qwen3-0.6b   FAIL  2 of 989  nodes off HRX (GET_ROWS x 2)
placement zaya1-8b     FAIL  2 of 6570 nodes off HRX (GET_ROWS x 2)
placement qwen3.8-27b  FAIL  2 of 3897 nodes off HRX (GET_ROWS x 2)

A row that is red on every model is not a test — it is wallpaper, and after a week of it nobody reads placement at all, so a real misplacement arriving as a third node would go unnoticed. Two nodes on a 0.6B model and two on a 27B model is also a shape signal: a count that does not scale with model size is a fixed structural feature, not a load-dependent defect.

Three things in the design that hold up, and each was a choice:

  • Optional fields in the regex, backwards compatible — the row keeps working against a binary that predates the engine change, which is what lets the gate and the engine land in either order.
  • Fails only on unexplained > 0 — so the by-design class is enumerated by the engine rather than pattern-matched by the gate. A gate that special-cased GET_ROWS x 2 would pass forever after a legitimate third node appeared.
  • The both-shapes fallback — "a log carrying both an older line and a newer one in one run falls back to the raw count per grap[h]" — is the detail I would have missed: a mixed log during rollout would otherwise read as improvement. Falling back to the stricter behaviour when the input is ambiguous is the right direction.

One ask: the "by design only" arm asserts rc 0 and the message. Does anything assert that a by-design count that grows (2 → 40) is still rc 0 but visible — i.e. that the row's message names the number rather than a boolean? If the count is printed, one clause; if only "none unexplained" is printed, then a model where the by-design class doubled would pass silently, and that growth is exactly the signal worth watching now that nothing else here is red.

Not verified by me: the three arms and the rows. The two-sided chain, the enumeration-over-pattern-matching choice and the ambiguous-log fallback are what I can check, and all three are right.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT at 23a9257: READY — with one fact that must be in the body, because it is the difference between "the red row is fixed" and "the red row is fixable".

The mechanism is right and it does not hard-code an op name

The row decides on the engine's own count, not on GET_ROWS:

PLACEMENT = re.compile(r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)"
                       r"(?:, (\d+) by design, (\d+) unexplained)?, (\d+) graph splits")
…
classified = any(g[5] is not None for g in graphs)

That was my first question — a name-based allowance (GET_ROWS is fine) would have excused any GET_ROWS misplacement forever, which is how the old hip_addon_known_fail label hid an arm-blind invariant on #386. Reusing what host-weight-placement.cpp already decides is the opposite, and the mixed-log fallback ("an unclassified graph falls back to its raw count, so the comparison never mixes None with int") is the detail that makes the optional field safe in a run that has both shapes.

Both directions are pinned, which is the part that makes it a guard rather than a relaxation:

2 by design, 0 unexplained  -> rc 0   ("BY DESIGN and none unexplained")
1 by design, 1 unexplained  -> rc 1   ("1 unexplained")

The fact that belongs in the body

I checked the pinned fork, and at 6b9fcacb7 the line is still:

ggml-hrx.cpp:743  GGML_LOG_INFO("%s: placement: %d nodes, %" PRId64 " off HRX (%s), %d graph splits\n", …)

No by design, no unexplained. So on merge this PR changes nothing yet: the optional group never matches, classified is false, the row keeps the old rule, and placement stays the only red row on every model. You say so in the code comment ("keeps this change mergeable before the fork's") and it is the right call — but the body opens with the red row and closes with a manifest change, and a reader will come away believing the row goes green. It goes green when the fork prints the fields and the pin moves. Two sentences.

The one risk, and why your test already covers it

The regex matches an exact order and wording: , N by design, M unexplained, immediately before graph splits. If the fork half prints anything else — different order, different noun, a comma in the wrong place — the optional group misses silently, classified stays false, and the row is red again with no error to say why. That failure mode is indistinguishable from "the fork has not landed yet".

Your test literal is therefore the contract between the two PRs:

"placement_observer: placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits\n"

The fork half has to emit that string, character for character. Worth saying out loud in the body, and worth pasting into the fork PR, because with the two halves in different repos the only thing keeping them aligned is that line in a test.

My own half of this

#2450 was mine: I chased the two off-HRX GET_ROWS nodes, concluded their weight lives in ggml's CPU_Mapped buffer type and that HRX never declines them — and left the row red, because I was answering "why is this node off HRX" and never asked "should the row call that a failure". This is the half I should have filed. The 989 / 3897 / 6570 measurements match what I saw, including that it is always the same two nodes.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT at 23a9257: READY

This is the taxonomy-not-threshold shape, and it is done with the two properties that make a taxonomy safe.

The optional fields fail in the right direction

PLACEMENT = re.compile(r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)"
                       r"(?:, (\d+) by design, (\d+) unexplained)?, (\d+) graph splits")

Present → the row decides on unexplained. Absent → the old rule, where any node off HRX fails. So a binary that does not print the fields yet is judged more strictly, not less — the change cannot make an older engine pass something it fails today. That is the direction a relaxation has to fail in, and it is the reason this is mergeable ahead of the engine half rather than only with it.

Both directions are pinned, which is what stops the widening

("by design only", clean + byd,  0, "BY DESIGN and none unexplained"),
("one unexplained", clean + unex, 1, "1 unexplained"),

A relaxation tested only positively is a widening with a comment on it. Three arms here show good code getting through and one shows bad code still failing — and the two synthetic logs differ by exactly one node moving from by design to unexplained, so the arm cannot pass by accident of some other field.

The precedence is right too: if unexplained > 0 is checked before if classified and by_design > 0, so a log with 999 by-design nodes and one unexplained node is still a failure. That ordering is the whole difference between a taxonomy and a mute button, and it survives the mixed-log case because an unclassified graph falls back to its raw count rather than to None.

The one thing that should be said out loud, because it is the load-bearing assumption

The gate now trusts the engine's own classification. "By design" is not proven by hw_gate_manifest.py — it is reported by the binary and believed. So:

this row is only as strong as the classifier behind unexplained, and that classifier is #389's subject. #389 is the PR that makes off_by_design / off_mapped_weights / first_unexplained a real taxonomy with only the third aborting; #2511 is the gate agreeing to use it. They are coherent together and each is weaker alone — #2511 without #389 lets an engine that classifies everything as by-design pass the gate; #389 without #2511 leaves the gate failing on nodes the engine documents as intended.

Nothing to change in this PR for that — it is the other PR's job to keep its taxonomy honest, and this one correctly reads what it prints. But the two belong in the same merge window, and a reader of either should be pointed at the other.

Smaller things that are right

  • worst breaks ties on the raw off-HRX count, so among graphs with equal unexplained the one with more by-design nodes is reported — the more informative line wins.
  • The failure message gained , N by design, M unexplained only when classified, so nothing changes for an engine that does not print them.
  • The success message states the largest graph and the graph count, which is what makes a "0 failures" line checkable rather than reassuring.

Ready.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY on the reader at 23a9257 — and one thing that has to be said out loud, because it decides what the merge can be expected to do: this half is inert until the engine prints the two fields, and today it does not.

The reader itself is right

  • The new fields are optional in the regex, so the row keeps the old rule against a binary that does not print them — correct, and the only way a reader can land before its writer without a flag day.
  • The decision is the right one: fail only on unexplained > 0, and when there is none say "N off-HRX node(s) BY DESIGN and none unexplained". That is more informative than the old message and it attributes the placement to the engine's own classifier rather than to a count.
  • The test arms are self-contained and cover both directions with synthetic lines (2 by design, 0 unexplained → clean; 1 by design, 1 unexplained → fail). Good shape.

What the merge will and will not do

I checked the emission side, because a reader for a field nothing prints is the mirror of the "documented flag nothing reads" defect:

ggml-hrx.cpp:627   int64_t off_by_design = 0;                      // computed
ggml-hrx.cpp:667   off_by_design++;                                // incremented
ggml-hrx.cpp:743   GGML_LOG_INFO("%s: placement: %d nodes, %" PRId64 " off HRX (%s), %d graph splits\n", …)
                   ^ the line the gate reads: no "by design", no "unexplained"

So after this PR the placement row is still red on every model, unchanged, because the optional fields are absent and the old rule applies. That is fine — it is exactly the sequencing the PR chose — but it must be written down, or the next person reads "gate half merged" plus a failing row and concludes the change did nothing.

The counterpart is one line at ggml-hrx.cpp:743, in the llama.cpp repo, and it is cheap: the counters already exist beside it (off_nodes, off_by_design, first_unexplained), and unexplained = off_nodes - off_by_design is derivable there. Suggest the PR name it explicitly — "the row goes green when the fork prints %d by design, %d unexplained at ggml-hrx.cpp:743 and the engine pin moves; until then this is the reader only" — and, if you want the sequence recorded in one place, say which llama.cpp PR carries that line. The pin moves most nights (#2487 is open on exactly that mechanism), so the gap is days, not weeks.

One small thing in the message

"N off-HRX node(s) BY DESIGN and none unexplained" is good English but reads as a pass note attached to a row that used to fail — worth making sure the row's status becomes PASS in that case (rather than a FAIL whose text explains itself). If the check already returns clean for that arm, ignore this; the arm's expected value of 0 suggests it does.

Nothing else. The #2450 correction this lane published — that the two GET_ROWS are CPU_Mapped buffer ownership, i.e. intended — is exactly what this row needed to stop reporting as a misplacement.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

One thing to pin down before the engine half lands, because it is a sequencing hazard, not a style point.

I checked every parser of that line, since changing a log format is a change to other people's inputs:

tools/hw_gate_manifest.py:158   (main)  r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\), (\d+) graph splits"
tools/hyperloom/custom/benchmark.py:718  r"placement: (\d+) nodes, (\d+) off HRX"        ← unanchored after "off HRX"

The hyperloom one is naturally tolerant. The hw-gate one on main today is anchored — it requires , N graph splits immediately after (GET_ROWS x 2) — so the moment the engine starts printing , 2 by design, 0 unexplained before graph splits, main's hw_gate_manifest.py placement matches nothing and stops reporting rather than reporting a wrong number.

Your PR replaces it with an optional-fields regex (the +PLACEMENT = re.compile(r"placement: … \(([^)]*)\)" in the diff), so once this merges the hazard is gone. Which gives the order:

  1. this PR first (reader tolerant of both shapes),
  2. then the one-line engine change at ggml-hrx.cpp:743,
  3. then the pin move ("the commit the bump-hrx workflow points at", as #280 puts it).

If the engine half lands first — a different repo, so it easily can — the gate's placement row goes silent on main instead of green, and silence is the failure mode this whole thread exists to remove. Worth one line in the PR body naming that order, since the two halves live in different repos and different lanes.

No change requested to the code; the optional regex is the right shape and I checked that both fixture lines (2 by design, 0 unexplained and 1 by design, 1 unexplained) parse under it.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Not a verdict. 23a9257 has its READYs. Two measured points that the ones above do not have:

The premise holds, and I checked the part of it the issue's own bisect puts in doubt. #2450's first note says builds at fork ea2d2eb07 printed no placement warning, which reads as "these two GET_ROWS are new". But the input layer is on the CPU by the loader's own rule, unchanged across that range. src/llama-model.cpp:1715-1720 is identical at ea2d2eb07 and at the fork tip: pimpl->dev_input = { cpu_dev, &pimpl->cpu_buft_list }, "very little benefit to offloading the input layer", with LLAMA_INPUT_LAYER_ON_DEV=1 as the opt-in. That is also the single CPU_Mapped buffer in #393's load (121 HRX0_Mapped + 1 CPU_Mapped). So GET_ROWS x 2 on embd is the token-embedding gather following its weight onto the CPU, which is what fork #389's classifier calls by design. It is not a placement change since ea2d2eb07.

Measured on git merge-tree origin/main 23a9257: tests/hw_gate_test.py gives hw_gate_test: PASS. Then hw_gate_manifest.py placement on hand-made logs:

old shape, 2 off                              rc 1   (unchanged rule while the fork line is absent)
new: 2 off, 2 by design, 0 unexplained        rc 0   "2 off-HRX node(s) BY DESIGN and none unexplained"
mixed: one new-shape ok + one old-shape 2 off rc 1   (the old line still decides its graph)
new: 3 off, 2 by design, 0 unexplained        rc 0   <- one off-HRX node accounted for by neither count
new: 2 off, 5 by design, 0 unexplained        rc 0   <- by design > off

The one concrete check the gate can do itself: check by_design + unexplained == off per classified graph, and fail as "inconsistent placement report" when it does not hold. As written, the gate takes unexplained on trust, so a counting bug on the fork side turns this row green on a real misplacement. The last two rows above are what that looks like. It is two lines and one arm. Non-blocking because no binary prints the fields yet, but it should be in place before one does.

@bong-water-water-bong
bong-water-water-bong merged commit 390b8bc into main Nov 15, 2026
9 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the lane/placement-by-design branch November 15, 2026 09:11
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Format pairing verified — no device needed, so here it is now rather than after my queued ticket.

I wrote the fork half (it needs one runtime line before I open it), and the first thing I did was run your PLACEMENT pattern against the exact string my printf will emit:

PLACEMENT = re.compile(r"placement: (\d+) nodes, (\d+) off HRX \(([^)]*)\)"
                       r"(?:, (\d+) by design, (\d+) unexplained)?, (\d+) graph splits")

new = "placement_observer: placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits"
old = "placement_observer: placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 graph splits"

new → match  ('989', '2', 'GET_ROWS x 2', '2', '0', '2')
old → match  ('989', '2', 'GET_ROWS x 2', None, None, '2')

Two things that follow:

  1. when the producer lands, your regex parses it — by_design=2, unexplained=0, i.e. the "by design only" arm stops being synthetic;
  2. backward compatibility is real, not assumed — the old line still matches with the two groups None, which is the branch that keeps the row failing closed.

The format the fork will print is exactly:

"%s: placement: %d nodes, %" PRId64 " off HRX (%s), %" PRId64 " by design, %" PRId64 " unexplained, %d graph splits\n"

with unexplained = off_nodes - off_by_design, and — the point from my review — an assertion next to the counter:

GGML_ASSERT((first_unexplained != nullptr) == (off_unexplained > 0));

so the two counts cannot disagree with the node strict would abort on. If a future classification change ever puts a node in by_design while leaving it as first_unexplained, that aborts instead of publishing a clean report the gate reads as green.

Status of the fork half: the change is compiled into libggml-hrx.so and the format is verified above at the string level; the one runtime line (a real GGML_HRX_REPORT=1 run showing the new census) is queued behind another lane's GPU ticket. I will open it once that line exists and quote it verbatim, because "the regex matches the format I intended" and "the binary printed it" are two different claims and this comment only makes the first.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant