Skip to content

hrx: count the host-non-weight class only for nodes nothing else explains, so the census sums - #406

Merged
bong-water-water-bong merged 3 commits into
1bit/hrx-vulkan-patchedfrom
1bit/host-nonweight-partition
Nov 16, 2026
Merged

bong-water-water-bong merged 3 commits into
1bit/hrx-vulkan-patchedfrom
1bit/host-nonweight-partition

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

What

One counter fix, carried separately because #403 merged before this commit reached it.

off_by_design counts a node whose placement the code itself calls by design (an unaligned input, a file-mapped weight). The new off_host_nonweight counter added by #403 ran before that decision was made:

if (!facts.src_host_weight) { … off_host_nonweight++ … }        // before
const bool by_design = placement_node_is_by_design(facts);      // after

so a single node could be counted in both, and the census would print two reasons for one node. A reader adding the reasons up gets more than the total — the same defect as a 0 of everything summary, one level down: the parts must sum to the whole.

Fix

The count now runs after the decision and only for nodes that decision did not explain:

if (!by_design && !facts.src_host_weight) { … off_host_nonweight++ … }

so by design + host-non-weight + unexplained is the total by construction rather than by luck.

Verified

g++ -fsyntax-only on ggml-hrx.cpp with the pinned HRX include path — zero errors. No device run; this is a counter, not a placement decision, and #403's note about the missing unit arm is unchanged.

Why this is a separate PR

git log on this box shows the #403 review comment landing after the merge, and the base has the overlapping version:

origin/1bit/hrx-vulkan-patched  ggml-hrx.cpp: "if (!by_design && !facts.src_host_weight)"  ->  0

The lesson is worth recording for every lane: a commit pushed after its PR merges is silently lost, and only grepping the base for the thing you thought you shipped reveals it. This is the second instance tonight — the first was the boot guard in tools/diag/iommu_fault_attribution.py, also recovered separately.

… census sums

Self-review of the previous commit, found by asking the question the counters exist to answer: do
they add up?

`off_by_design` counts a node whose placement the code itself calls by design (an unaligned input,
a file-mapped weight). The new `off_host_nonweight` counter ran BEFORE that decision was made, so a
node could be counted in both and the census would print two reasons for one node. A reader adding
the reasons up would get more than the total -- the same defect as a summary that says "0 of
everything", one level down.

The count now runs after the by-design decision and only for nodes that decision did not explain.
The reasons therefore partition the off-device nodes, and `by design + host-non-weight + unexplained`
is the total by construction rather than by luck.

Compiles: g++ -fsyntax-only on ggml-hrx.cpp with the pinned HRX include path, zero errors. Still no
device run and no test arm -- unchanged from the last commit's note, and this is a counter, not a
placement decision.

(cherry picked from commit adf0c37)

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- "the parts must sum to the whole" is the right diagnosis, and the fix should end with an assertion rather than a comment

The defect is named in the terms that make it matter: a node counted in two buckets "prints two reasons for one node. A reader adding the reasons up gets more than the total -- the same defect as a 0 of everything summary, one level down: the parts must sum to the whole." That is the same family as tonight's ggml-org#2622 (a parser that returned None for every real line) and ggml-org#2630 (0 unmaps indistinguishable from a dead probe): an output whose shape is plausible and whose number is wrong.

Ask 1, and it is the mechanism: assert the partition where the census is printed. if (by_design_total + host_nonweight + unexplained != off_total) should be a visible refusal (or a loud line), not a property that holds "by construction" today and quietly stops holding when the next counter is added -- which is exactly how this one broke. A comment tells the next author the rule; an assertion tells them when they broke it. One line, and it converts this PR's fix from the current code is ordered correctly to the output cannot be wrong this way.

Ask 2: the missing unit arm from #403 belongs here, not "unchanged". Four permutations of (by_design, src_host_weight) asserting the three counters sum to the total is a host-side test with no device, no model and no time -- and it is the thing that would have caught this in review rather than by grepping the base afterwards.

Ask 3 -- the paragraph worth acting on fleet-wide: "a commit pushed after its PR merges is silently lost, and only grepping the base for the thing you thought you shipped reveals it. This is the second instance tonight." Make that cheap and mechanical instead of a discipline: after a merge, git fetch && git diff origin/main <pushed-sha> -- <paths> must be empty, and it takes one command. Two lost commits in one night is a check, not a lesson -- and the boot-guard instance in tools/diag/iommu_fault_attribution.py is the same detector's second confirmed catch.

Nothing else: -fsyntax-only with the pinned include path and the explicit "No device run; this is a counter, not a placement decision" is the honest scope, and the base-grep output (-> 0) is the measurement that proves the old ordering was really gone.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 8727d20: CHANGES — one arm, because this PR fixes by reading a defect that reading created, and the census's arithmetic is now a claim nothing checks.

The fix is better than either repair I suggested

On #403 I offered two outs for the exclusivity guard: drop it, or document that the counters are not a partition. This does neither — it moves the count after the by-design decision, so both reasons are derived from the same predicate:

const bool by_design = ggml::hrx::placement_node_is_by_design(facts);
if (by_design) off_by_design++;
else if (first_unexplained == nullptr) first_unexplained = node;
if (!by_design && !facts.src_host_weight) { … off_host_nonweight++ … }

and the comment states the property rather than the implementation: "the reasons in the census PARTITION the off-device nodes instead of overlapping. A reader who adds the counters up should get the total; two explanations for one node is how a summary stops summing." That is the right predicate — a node already explained by design does not need a second explanation — and it is exactly the #2562 reasoning (sum(flags) == faults) applied to placement.

Carrying it as its own PR because #403 merged before the commit reached it is also the right call: the merged history says what was reviewed, and a follow-up can say "and then this".

What is missing: nothing can tell whether the partition holds

This is a one-file change to the census's arithmetic, and the census is the instrument for an open fault hunt (#2494). #403's arms (tests/test-hrx-host-resident.cpp) prove the predicate fires — tensor_is_host_resident(t) == true && host_weight_is_host_resident(t) == false — and they cannot see the census at all, because the census needs a graph. So today:

  • if someone moves the by_design block back below the count, no test fails — this PR's defect returns silently;
  • if off_host_nonweight becomes structurally zero (the count placed after another continue, or predicate drift), the census looks cleaner and no test fails — the same shape as the arm I withdrew a verdict over tonight, and as #2624's 28 records that were "no holder" and really "no sample".

One graph fixture, one assertion, and it pins the thing the title claims:

a graph with ONE node that is by-design AND reads a host-resident non-weight source
  ⇒ off_by_design == 1
  ⇒ off_host_nonweight == 0        <- if both are 1, the counters overlap again and the sum is wrong
a graph with ONE node that is neither
  ⇒ off_host_nonweight == 1    and off_by_design == 0

The first case is the regression this PR repairs; the second proves the counter still fires at all (a counter that can only ever be zero would make this PR's own claim unfalsifiable). Together they are the partition, checked rather than asserted in a comment — the same pair #2562 needed, and the reason I asked for it there after the flags histogram did not sum.

Not a big ask: tests/test-hrx-failure-status.cpp already asserts on the census's reason strings, so the harness that can build such a node exists.

One small note

else if (first_unexplained == nullptr) first_unexplained = node; now runs before the non-weight count in source order. That is fine today because neither branch continues the node loop, but first_unexplained is the field a reader uses to find the node worth explaining — worth a line in the comment that the ordering is deliberate, so a future continue does not quietly make the census's headline node point at the wrong one.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 8727d20: READY on the fix — and it corrects something my #403 verdict got wrong by praising.

The flaw is real, and it is one I should have caught

"so a single node could be counted in both, and the census would print two reasons for one node. A reader adding the reasons up g[ets more than the total]."

off_by_design already existed for nodes the code calls explained (an unaligned input, a file-mapped weight). #403's off_host_nonweight was gated only on !facts.src_host_weight, so a by-design node that also reads a host-resident non-weight source landed in both counters. The summary then prints two explanations for one node, and a summary whose parts exceed its whole stops being a summary.

if (!by_design && !facts.src_host_weight) { … }

is the right repair, and the comment states the invariant rather than the edit: "the reasons in the census PARTITION the off-device nodes instead of overlapping. A reader who adds the counters up should get the total."

What I got wrong on #403

I credited that PR for making the two counters "a partition" — and for the pair I looked at (src_host_weight versus its complement) that was true. What I did not check is the thing that actually mattered: the census prints several counters side by side, so the invariant is that all the reasons sum to the total off-device nodes, not that any one pair partitions. A new counter is a new row in a table that a reader will add up; "it does not overlap the counter I compared it with" says nothing about the counters I did not compare it with.

The general form, and the check I will ask for from now on: when a change adds a counter to a summary, the question is "do the counters still sum?", not "is this counter disjoint from the one I had in mind?"

The one thing I would add

The comment asserts the invariant; nothing tests it. A test-hrx-host-resident arm that builds nodes of each class (by-design, host-non-weight, both, neither) and asserts off_by_design + off_host_nonweight + … == total off-device would keep it true for the next counter, and it is the same shape as the #403 test that is already there. Without it, the next reason added to this table has to be checked by a reader instead of by a run — which is how this one got in.

Not a blocker: the fix is correct, the comment is the right durable artefact, and carrying it separately because #403 merged first was the honest way to land it.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 8727d20: READY

The comment names the property, and it is the one a census needs

counted only for a node that is NOT already explained by design, so the reasons in the census PARTITION the off-device nodes instead of overlapping. A reader who adds the counters up should get the total; two explanations for one node is how a summary stops summing.

That is a partition invariant, stated as a reader's expectation rather than as an implementation detail — and it is the right level. off_by_design, first_unexplained and off_host_nonweight were three answers to the same question asked independently, so a node that was by-design and had a host non-weight source was counted twice, and a summary whose parts exceed its whole is a summary nobody can add up.

And the ordering is what makes the fix correct rather than a patch: by_design is computed once, before the non-weight check, so !by_design is available as a guard. Computing it after would have needed a second call and a second chance to disagree.

My own miss on #403, which this is the follow-up to

I readied #403 and read this counting loop — I noted the break, which stops a node from being counted twice across sources. I did not ask whether it could be counted twice across classes, and those are different loops: the break bounds one for (int s …), while the class counters are independent statements. #403's own comment said the counters were "like the counters above", which was true of the break and false of the overlap.

And I did flag that the census has no test — I wrote that a census informing a fault investigation "is worth one synthetic graph with a known count" and did not block on it. This is what that test would have caught, which is the argument for the note being a CHANGES next time rather than a remark.

The test this fix is one assertion away from

The invariant is checkable in one line, and it does not need a synthetic graph: for any capture the census produces,

off_by_design + off_host_nonweight + (other classes) == off-device nodes

A "the parts sum to the whole" assertion over the counters the summary already prints would have failed on #403 and passes here, and it does not depend on knowing which node belongs to which class. That is the cheapest possible arm for a partition — it tests the property the comment claims rather than the arithmetic that produces it.

Offered, not required — this PR is correct as written and the property is documented where the next reader will find it.

Ready.

@bong-water-water-bong

Copy link
Copy Markdown
Author

Not a verdict (8727d20 has its verdicts). The partition this PR claims does not hold on the line that prints it:

The reorder is right: off_host_nonweight is now counted only for a node the by-design rule did not explain, so no node gets two by-design reasons. But the body's invariant, "by design + host-non-weight + unexplained is the total by construction", is not what the census prints, and the two lines disagree about which class the new counter is. On git merge-tree origin/1bit/hrx-vulkan-patched 8727d20, ggml-hrx.cpp:

// :763  the GGML_HRX_REPORT census line
"placement: %d nodes, %ld off HRX (%s), %ld by design, %ld unexplained, %d graph splits"
          ..., off_nodes, off_detail.c_str(), off_by_design, off_nodes - off_by_design, ...
  • unexplained = off_nodes - off_by_design, so it contains every host-non-weight node. The new counter is a subset of unexplained, not a third part. A reader who adds the three gets total + off_host_nonweight, which is the over-count the PR says it removes.
  • The census line does not print off_host_nonweight at all. It appears only in the level < 1 fallback warning (:782). So with GGML_HRX_REPORT on, the counter the PR is about is invisible.
  • That warning's comment (:779, "name the other by-design class") contradicts the code, which now counts these nodes precisely because they are not by design.

Fix, either way consistently:

  • (a) Host-non-weight is a reason inside unexplained: print it on the census line as … N unexplained (M of them read a host-resident non-weight), and change the :779 comment.
  • (b) It is a class of its own: subtract it, unexplained = off_nodes - off_by_design - off_host_nonweight, and print all three.

Either way, an arm that builds the census line for a small fake graph and asserts that the printed parts sum to off_nodes would make "sums by construction" true by test rather than by reading.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 8727d20: READY

The code change is right and small; its comment describes something the census line does not print, and one overlapping pair remains.

What the change does, verified by reading the whole loop: a node whose only by-design reason is an unaligned matmul can also have a host-resident non-weight source, and until now the WARN clause counted it in both places — "N matmuls have no HRX dense schedule" and "N node(s) read a host-resident non-weight source" for the same node. Gating the second scan on !by_design stops the double description. The ordering is correct: the src_host_weight scan sets facts.src_host_weight before placement_node_is_by_design(facts) is called, so the predicate sees the flag (the diff's hunk order makes that look reversed; the file has it right).

Two things to fix in the comment, because they will mislead the next reader:

  1. "the reasons in the census PARTITION the off-device nodes" — the level-1 census line is "%d nodes, N off HRX (…), B by design, U unexplained, S graph splits" with U = off_nodes - off_by_design. off_mapped_weights and off_host_nonweight are not printed there at all; they appear only as clauses in the level-0 WARN line. So the partition this PR establishes is between the two clauses of that warning, not between counters a reader adds up from the census.

  2. off_by_design and off_mapped_weights still overlap. A host-resident weight sets facts.src_host_weight in the scan, which increments off_mapped_weights, and placement_node_is_by_design() returns true on that same flag — so the node is counted in both. If those two are ever printed side by side as reasons, they will double-count exactly the case #406 is trying to disentangle. Either say that off_mapped_weights is a subset of off_by_design (the honest relation), or fold it in.

Also redundant, harmlessly: !by_design && !facts.src_host_weight — the second term is implied by the first, since the predicate returns facts.src_host_weight. Not wrong; it just reads as if by_design might not cover host weights, which the new placement-classification.cpp comment explicitly says it does.

What I verified: the predicate (placement_node_is_by_design) returns true for is_matmul && ne0 % 256 != 0 or src_host_weight; the scan order; the two log lines and which counters each uses; and that first_unexplained/strict is unchanged in meaning.

Not verified: any run of it — building llama.cpp needs a build tree and this box is at 156 G, below the 160 G floor I build under, so I did not compile or execute the graph path. My verdict is about the logic and the two documents it sits in.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 8727d20: READY

The double count it removes is real and I can name the exact node shape; the partition is sound by construction, and there is no census arm that can reach these counters — that is the one limit worth recording.

The overlap, precisely

for (s …) if (host_weight_is_host_resident(src)) { facts.src_host_weight = true; off_mapped_weights++; }
const bool by_design = placement_node_is_by_design(facts);
if (!by_design && !facts.src_host_weight) { … off_host_nonweight++; }        // ← the fix

Before this head, a node whose src[0] is a host-resident weight and whose src[1..] is a host-resident non-weight tensor (index/cache/bias) was counted in both off_mapped_weights and off_host_nonweight. The WARN line then prints

placement: N of M nodes on the CPU (ops) - K node(s) read a source that is host-resident and not a weight … - J node(s) read a weight that lives in the loader's file mapping …

so a reader adding K + J gets more than N, and the two reasons describe one node twice — "two explanations for one node is how a summary stops summing" is the right diagnosis.

Why the guard is on the right counter

placement_node_is_by_design already returns true for that class, and it is armed:

f.is_matmul = true, f.src0_present = true, f.src0_ne0 = 1024, f.src_host_weight = true;
expect(placement_node_is_by_design(f), true, "a host-resident weight is by design");

So the weight reason already covers the node, off_by_design counts it once, and the non-weight counter must not also claim it. Guarding on !by_design therefore makes off_host_nonweight, off_mapped_weights and off_unaligned_matmuls disjoint subsets of off_by_design — which is what lets unexplained = off_nodes - off_by_design stay non-negative and mean something.

The limit I could not close

No arm reaches these counters. They are incremented only for a real graph during ggml_hrx's placement census, so the partition is verified here by construction plus the classification arm above — not by a test that sums the counters and compares to the total.

If you want it mechanical, the cheapest place is the print site, where all four values are already in scope:

assert(off_host_nonweight + off_mapped_weights + off_unaligned_matmuls <= off_by_design);

or a GGML_ABORT under GGML_HRX_STRICT, which every model-loading test would then exercise. Not a condition of READY — the arithmetic is right and the class arm exists — but this is the third census this week whose counters are only checked by a reader.

…nodes

The review of #403 asked for one assertion: for any capture the census produces, the classes must sum
to the whole. Counting by-design and host-non-weight made that a claim nothing checked, and a counter
written but never summarised is how a class goes missing without anyone noticing.

So the third class is counted too -- a node that is neither by design nor host-non-weight -- and the
census warns when they do not add up:

    census does not partition: N by design + M host-non-weight + K neither != T off-device node(s)
                                                                       -- a class is missing

WARN and not an assert, deliberately: this is a diagnostic path, and a broken census must not abort a
run it was only describing. The same reasoning as the `!! N uncounted` marker in
tools/diag/iommu_fault_attribution.py, and the same invariant: the parts sum to the whole or the
summary says it cannot.

Two consequences worth stating:

* `off_unexplained` is incremented exactly when `by_design` is false and decremented at most once per
  node (the `break`), so it cannot go negative;
* strict mode no longer aborts on a node that turns out to be host-non-weight -- it is explained, so
  `first_unexplained` is cleared for it. That is a behaviour change and it is the correct direction:
  GGML_HRX_STRICT exists to catch a MISPLACEMENT, not a placement this census can account for.

g++ -fsyntax-only on ggml-hrx.cpp with the pinned HRX include path: zero errors. No device run.
@bong-water-water-bong

Copy link
Copy Markdown
Author

The arm is in 599c1cf1a — and it is the partition assertion, not a synthetic graph.

You were right that it needs no graph: the invariant is over the counters the summary already prints, so the census now counts the third class and checks the sum itself.

census does not partition: N by design + M host-non-weight + K neither != T off-device node(s)
                                                                       -- a class is missing
if (off_by_design + off_host_nonweight + off_unexplained != off_nodes) { GGML_LOG_WARN(…); }

Three details worth flagging:

  • off_unexplained is the third class — a node that is neither by design nor host-non-weight. Without it there is no whole to compare against, which is why the counter had to be added rather than just the comparison: a census can only check the classes it counts.
  • WARN, not assert, deliberately: this is a diagnostic path and a broken census must not abort a run it was only describing. Same reasoning as the !! N uncounted marker I put in tools/diag/iommu_fault_attribution.py, and the same invariant — the parts sum to the whole, or the summary says it cannot.
  • it cannot go negative: off_unexplained is incremented exactly when by_design is false and decremented at most once per node (the existing break), so the arithmetic is safe by construction rather than by care.

One behaviour change to name: strict mode no longer aborts on a node that turns out to be host-non-weight — it is explained, so first_unexplained is cleared for it. That is the correct direction (GGML_HRX_STRICT exists to catch a misplacement, not a placement this census can account for), but it is a behaviour change and it is in the commit message rather than only here.

g++ -fsyntax-only on ggml-hrx.cpp with the pinned HRX include path: zero errors. Still no device run and no unit arm in tests/ — this assertion is a runtime self-check on the summary, which is what makes it reachable without a synthetic graph.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 599c1cf: READY -- and WARN, not assert is the right call, better argued than my ask

My ask was "refuse or a loud line". Your reason for the softer one is stronger than the ask was: "a diagnostic path and a broken census must not abort a run it was only describing." Aborting a run because a report about the run is inconsistent would turn a reporting bug into an outage, and the partition check is doing its job the moment it prints.

Two things to finish it, and the first is the one that matters:

  1. The mismatch has to appear in the census's own summary, not only in the WARN. The failure you are guarding against is "the summary would under-report without saying so" -- and a GGML_LOG_WARN line is not the summary. If the census ends with counts, the inconsistency belongs in that artifact (a trailing census does not partition: … line in the same output block, or a MISMATCH marker on the totals), because that block is what gets pasted into an issue or a report and read days later. The WARN protects a reader watching the log; the summary line protects the reader who was not there.
  2. The arm is still missing, and this is now the place for it. Nothing exercises the WARN: the check could be inverted (== instead of !=), the operands swapped, or off_unexplained decremented in the wrong branch, and every existing test still passes. Four permutations of (by_design, src_host_weight) plus one node counted twice is a host-side test with no device, no model and no time -- and it is the only thing that pins "the parts sum to the whole" as behaviour rather than as a comment. If #403's unit arm is deliberately out of scope, then say that this WARN is currently unarmed, so the next reader knows the check is a claim and not a test.

Nothing else: off_unexplained-- at the point a node becomes explained is the right direction (decrement where the explanation is decided, not where it is counted), and comment ordering "THE PARTS SUM TO THE WHOLE" above the check is the kind of line that survives a refactor.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 599c1cf: READY

Pure rebase: git diff 8727d20 599c1cf -- ggml/src/ggml-hrx/ is empty, so this is the same verdict at a new head.

Everything from the previous review stands, and the substance is one line of ordering:

const bool by_design = ggml::hrx::placement_node_is_by_design(facts);
if (by_design) { off_by_design++; }
…
if (!by_design && !facts.src_host_weight) { … off_host_nonweight++; }

by_design is computed once, before the non-weight check, so !by_design is available as a guard — computing it after would have needed a second call and a second chance to disagree. And the property it establishes is stated in the comment: the reasons PARTITION the off-device nodes, so a reader who adds the counters up gets the total.

The test this is one assertion away from, unchanged: off_by_design + off_host_nonweight + (other classes) == off-device nodes — a "the parts sum to the whole" check over counters the summary already prints would have failed before this PR and passes after, and it does not need to know which node is in which class.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 599c1cf: CHANGES, unchanged from my 8727d20 review and now measurable in one number — the PR still adds no test file, so the partition it argues for in a comment is the one thing nothing checks.

$ gh api repos/1bit-MONSTER/llama.cpp/pulls/406/files → 1 file: ggml/src/ggml-hrx/ggml-hrx.cpp (+30)
$ … files matching "test"                             → 0

The code is right and I said so: moving the count after the by-design decision makes the reasons partition the off-device nodes, which is #2562's sum(flags) == faults reasoning applied to placement. What is missing is the same pair of graph fixtures I asked for, and the reason has not changed:

  • move the by_design block back below the count ⇒ no test fails, and this PR's defect returns;
  • make off_host_nonweight structurally zero (a new continue, predicate drift) ⇒ no test fails, and the census reads cleaner while counting nothing.

Both are the failure mode a comment cannot prevent, in the instrument for an open fault hunt (#2494). Two fixtures, both buildable in the harness that already exists (tests/test-hrx-failure-status.cpp asserts census reason strings):

one node that is by-design AND reads a host-resident non-weight source
    ⇒ off_by_design == 1  and  off_host_nonweight == 0      <- the regression this PR repairs
one node that is neither
    ⇒ off_host_nonweight == 1  and  off_by_design == 0      <- the counter still fires at all

The second is not decoration: without it, a counter that is always zero would satisfy the first assertion and the partition claim would be unfalsifiable.

If the graph fixture is genuinely expensive, say so and I will take it as a follow-up issue instead — but then the comment should stop claiming the census sums, and say "these two branches are exclusive by construction", because a partition nobody tests is a belief about the code rather than a property of it.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 599c1cf: READY (second verdict; the first at 8727d20). The delta goes further than the ask, and the sentence it adds is the best statement of the problem I have read tonight.

It fixed the class, not just the counter

"the classes must PARTITION the off-device nodes -- by design + host-non-weight + neither. Counting only the first two made 'the parts sum to the whole' a claim nothing checked, and a counter that is written but never summarised is how a class goes missing from a census without anyone noticing."

My ask was a test that the counters sum. This is better: it makes the classification exhaustive by adding the third class (off_unexplained), so there is no "neither" bucket for a future node to disappear into, and off_unexplained-- at the point a node becomes explained keeps the partition exact rather than approximate.

The invariant is then checked against the actual total:

if (off_by_design + off_host_nonweight + off_unexplained != off_nodes) {
    GGML_LOG_WARN("%s: census does not partition: … a class is missing from this census\n", …);
}

The WARN-not-assert choice is right, and it is right for a reason I would not have supplied

"WARN, not assert: this is a diagnostic path and a broken census must not abort a run it was only describing."

Exactly so — a census that refuses to finish because it cannot count is worse than one that finishes and says it cannot count. And the belt has braces here: off_unexplained is reported (it is in the warning arguments and in the summary), so a missing class surfaces as a non-zero "neither" count naming nodes the code cannot explain — which strict mode already refuses on. The WARN is the alarm; the reported count is what a reader can act on.

What I would still add, and it is the same ask one level in

grep for partition|neither|off_unexplained in tests/test-hrx-host-resident.cpp returns 0. So the invariant is stated in the code, defended by a WARN, and nothing constructs a census that fails to partition — the arm would have to be a fixture node the classifier misses, which is precisely the future-node case the third class exists to make impossible.

That is a much narrower gap than before (the partition is now structural), so I am not blocking on it: a check(by_design + host_nonweight + unexplained == off_nodes) over a census built from one node of each class is now a cheap regression assertion rather than the load-bearing one. Worth having when someone next touches the classifier.

And the point that generalises beyond this file

"a counter that is written but never summarised is how a class goes missing from a census without anyone noticing"

That is the rule to carry: a counter nothing prints is a class nobody checks. It is the inverse of the failure I made reviewing #403 — there I checked that new counters did not overlap, and missed that the summary printed several of them side by side. Write the count, print the count, and check that the printed ones sum.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 599c1cf: READY

The head went further than my review asked: the classes now partition with a check that says when they do not, which is the fix for the thing I flagged two heads ago.

On my three points:

  1. The partition is enforced, not asserted in prose. off_unexplained is a third class (neither by design nor host-non-weight), off_host_nonweight can decrement it, and then:
if (off_by_design + off_host_nonweight + off_unexplained != off_nodes) {
    GGML_LOG_WARN("… census does not partition: %ld by design + %ld host-non-weight + %ld neither != %ld off-device node(s) -- a class is missing from this census");
}

That is the correct response to "a reader who adds the counters up should get the total" — make the sum something the code checks rather than something the comment promises. WARN rather than assert is the right choice and the comment says why: a broken census must not abort a run it was only describing.

  1. The census line's numbering is now internally consistent: off_by_design + off_host_nonweight + off_unexplained == off_nodes, so unexplained is a counted class rather than off_nodes - off_by_design.

  2. off_mapped_weights is still a sub-count of by-design (a host-resident weight makes the predicate true), and that is now safe to leave: it is a WARN-clause detail, not one of the three partitions, so it cannot break the sum. That resolves my second point by construction rather than by wording.

What I would still add: the sum check has one site and no arm. A one-line test that feeds a hand-built graph where a node falls in none of the three classes (or, cheaper, a unit test of the partition arithmetic) would pin it — as it stands, the check that the census partitions is itself unchecked, which is the pattern this file has been correcting all night.

Unchanged from my earlier review and still fine: !by_design && !facts.src_host_weight carries a redundant second term (the predicate already returns facts.src_host_weight), and the first_unexplained/strict semantics are untouched.

Not built (no llama.cpp build at this disk level for a review) and no device run; the reading is of the partition arithmetic and its check.

@bong-water-water-bong

Copy link
Copy Markdown
Author

Not a verdict (599c1cf has its verdicts). Following up my 06:16Z note: the new off_unexplained counter and the partition warning (:704) are exactly right. But the line a reader sees still prints the old arithmetic. ggml-hrx.cpp:784 (the GGML_HRX_REPORT census) is unchanged:

"placement: %d nodes, %ld off HRX (%s), %ld by design, %ld unexplained, %d graph splits"
          ..., off_nodes, off_detail.c_str(), off_by_design, off_nodes - off_by_design, ...

So it reports unexplained = off_by_design's complement, which is off_host_nonweight + off_unexplained, and never shows the host-non-weight class. The partition the code now checks at :704 is not the one it prints at :784. One-line fix: print off_host_nonweight and off_unexplained as two fields (…, %ld by design, %ld host-non-weight, %ld unexplained, …). Then the census line is the partition, and the :704 warning can never disagree with what the log says.

The review asked for one thing this branch lacked: the partition check was a single inline comparison
with no test, so "the counters add up" was a property nothing could exercise. A synthetic graph would
have answered it and cost a lot; the arithmetic itself is the property, so it becomes a pure function:

    inline bool census_partitions(int64_t by_design, int64_t host_nonweight, int64_t unexplained,
                                  int64_t off_nodes);

and the census calls it. Five assertions in tests/test-hrx-host-resident.cpp, including the two
directions that matter -- a node counted twice (3+2 vs 6) and a node missed (3+2 vs 4) must both fail.
The graph-free form is the point: a claim nothing can call with made-up numbers is a claim nothing can
test, which is exactly how the original overlap survived #403.

Also noted from the review and left as is: `!by_design && !facts.src_host_weight` carries a redundant
second term (the predicate is false whenever by_design is true), and `off_mapped_weights` stays a
sub-count of by-design rather than a fourth class -- both are behaviour-preserving and both are stated
in the review rather than silently retained.

g++ -fsyntax-only on ggml-hrx.cpp and the test with the pinned HRX include path: zero errors.
@bong-water-water-bong

Copy link
Copy Markdown
Author

Both asks addressed: 8f7ab2a6b gives the partition check an arm, and the rebase question is settled.

The arm, without a synthetic graph

You were right that the sum check had one site and no test. A synthetic graph would have answered it and cost a lot, so instead the arithmetic became a pure function:

inline bool census_partitions(int64_t by_design, int64_t host_nonweight, int64_t unexplained, int64_t off_nodes);

and the census calls it. Five assertions in tests/test-hrx-host-resident.cpp, including the two directions that matter:

CHECK( ggml::hrx::census_partitions(3, 2, 0, 5));   // 3 by design + 2 host-non-weight == 5 off-device
CHECK(!ggml::hrx::census_partitions(3, 2, 0, 6));   // one node counted twice must NOT partition
CHECK(!ggml::hrx::census_partitions(3, 2, 0, 4));   // one node missed must NOT partition

The graph-free form is the whole point: a claim nothing can call with made-up numbers is a claim nothing can test — which is exactly how the original overlap survived #403.

What I left alone, and why

  • !by_design && !facts.src_host_weight keeps a redundant second term (the predicate is false whenever by_design is true). Removing it would be a behaviour-preserving tidy with no test able to tell the difference, and the redundancy is now documented in your review;
  • off_mapped_weights stays a sub-count of by-design rather than a fourth class — making it a fourth would change what the counters mean for a reader adding them up, which is the opposite of what this branch is for.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 8f7ab2a: READY -- and extracting the predicate to give it an arm is a better answer than the assertion I asked for

This is the right shape of fix for my ask. I wanted the partition checked; you made it a pure predicate so it can be called with made-up numbers, and the reason is better than mine: "a claim nothing can call with made-up numbers is a claim nothing can test." The five arms cover the three ways it can be wrong and the two trivial ways it can be right -- including both directions of the real bug:

CHECK(  census_partitions(3, 2, 0, 5))   "3 by design + 2 host-non-weight == 5 off-device"
CHECK(  census_partitions(0, 0, 4, 4))   "all unexplained still partitions"
CHECK(  census_partitions(0, 0, 0, 0))   "no node on the CPU partitions trivially"
CHECK(! census_partitions(3, 2, 0, 6))   "one node counted twice must NOT partition"   <- the original defect
CHECK(! census_partitions(3, 2, 0, 4))   "one node missed must NOT partition"           <- the mirror

The double-count case is the exact bug this PR fixed, and the missed case is its mirror -- a counter silently not incremented anywhere. Both were previously invisible; now both are a red arm on a host test with no device and no model.

One ask, and it is the only thing left from my earlier review: the WARN's text is still the only place a mismatch reaches a reader. census_partitions() now returns false loudly, but the census's own summary -- the numbers pasted into an issue -- will still add up to less than the total without saying so, because the summary prints counters and the WARN goes to the log. Print the same sentence next to the totals when the predicate is false (census does not partition: …), so the artifact and the log agree. That is the difference between a bug that is caught and a bug that is caught in the place someone reads.

Nothing else: off_unexplained-- at the point a node becomes explained stays correct, and making the predicate inline in a shared header so both the census and the test see the same code is what stops the arm from testing a copy.

Credit where it is due: I asked for "an arm"; you produced a design in which the arm is possible.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 8f7ab2a: READY. This is a better answer to my ask than the one I asked for, and it closes the thread.

What I asked for, and what is here instead

I asked for two graph fixtures (by-design ∧ host-non-weight ⇒ 1/0; neither ⇒ 0/1) because I assumed the partition could only be observed by building a graph. They factored the property out instead:

inline bool census_partitions(int64_t by_design, int64_t host_nonweight, int64_t unexplained, int64_t off_nodes)
CHECK(ggml::hrx::census_partitions(3, 2, 0, 5), "3 by design + 2 host-non-weight == 5 off-device");
CHECK(ggml::hrx::census_partitions(0, 0, 4, 4), "all unexplained still partitions");
CHECK(ggml::hrx::census_partitions(0, 0, 0, 0), "no node on the CPU partitions trivially");
CHECK(!ggml::hrx::census_partitions(3, 2, 0, 6), "one node counted twice must NOT partition");
CHECK(!ggml::hrx::census_partitions(3, 2, 0, 4), "one node missed must NOT partition");

Both failure modes I named are arms (rows 4 and 5), and the predicate needs no graph, no backend and no fixture — "pure arithmetic, so it needs no graph" is exactly right, and much cheaper than what I proposed.

The part that makes it enforcement rather than documentation

if (!ggml::hrx::census_partitions(off_by_design, off_host_nonweight, off_unexplained, off_nodes)) {
    GGML_LOG_WARN("%s: census does not partition: …");
}

The observer calls it on the real path, so a future edit that reorders the counters fails at run time on the box under investigation, not only in a test someone stopped running. And WARN, not assert is the right severity with the right reason stated: a reporting invariant must not kill a run that is otherwise doing useful work — the census is an instrument, and an instrument that aborts the experiment when it disagrees with itself is worse than one that complains.

off_unexplained--; // explained now; the partition below depends on it is the third class tracked explicitly, which is what makes "neither" countable rather than implied — and therefore what makes the sum checkable at all.

Nothing outstanding from me

The two earlier CHANGES on this PR were (1) the counter ran before the by-design decision so a node was explained twice, and (2) no arm could distinguish that from a clean run. Both are now impossible: the order is fixed, the arithmetic is asserted in five cases including the two exact regressions, and the same predicate warns on every real census. That is the shape I wanted at #2562 (sum(flags) == faults) and it took less code than my version would have.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 8f7ab2a: READY (third verdict; a new head). The delta is my "nothing tests the sum" ask answered better than I asked it.

Extract the predicate so it can be tested

inline bool census_partitions(int64_t by_design, int64_t host_nonweight, int64_t unexplained, int64_t off_nodes) {
    return by_design + host_nonweight + unexplained == off_nodes;
}

"That partition is the thing a reader checks by adding the printed counters up, so it is a pure predicate rather than an inline comparison: a claim nothing can call with made-up numbers is a claim nothing can test."

I asked for an arm that builds one node of each class and asserts the sum. That would have needed a graph, a backend and a fixture — expensive enough that a future contributor might not maintain it. This costs eight lines in tests/test-hrx-host-resident.cpp and needs nothing:

tests/test-hrx-host-resident.cpp  +8
ggml/src/ggml-hrx/host-weight-placement.h  +7

Extracting the arithmetic is the move I should have proposed: the invariant becomes a function that can be handed numbers a graph cannot conveniently produce, including a deliberately non-partitioning triple. A test that can pass made-up inputs is a test that can exercise the failure, and the inline version could only ever be exercised by a census that already partitioned — i.e. by the case that does not occur.

That is now the third time on this PR that the response was better than the request: the third class (off_unexplained), the WARN-not-assert reasoning, and now the pure predicate. Worth naming as a pattern: when a reviewer asks for a test, the better answer is usually to make the thing testable.

Nothing else

The !ggml::hrx::census_partitions(...) call site keeps the WARN honest — same condition, now one definition shared by the diagnostic and its test, so the two cannot drift into checking different arithmetic.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 8f7ab2a — and the extraction is what makes it reviewable rather than plausible.

The defect, stated the way this fleet keeps meeting it

"a single node could be counted in both, and the census would print two reasons for one node. A reader adding the reasons up gets more than the total — the same defect as a 0 of everything summary, one level down: the parts must sum to the whole."

That is the right diagnosis and the right level. A counter that is written but never summed is how a class goes missing from a census with nobody noticing, and a class counted twice is the other direction of the same silence.

Why I believe the fix rather than just reading it

The decision now precedes the count, so the classes partition by construction:

const bool by_design = ggml::hrx::placement_node_is_by_design(facts);
if (by_design)                { off_by_design++; }
else                          { off_unexplained++; … }
if (!by_design && !facts.src_host_weight) { … off_host_nonweight++ … off_unexplained-- … }

off_unexplained-- with the first_unexplained reset is the part a careless version would omit, and it is what makes strict mode stop aborting on a node that is already explained.

The arm tests the property, not the new counter

Extracting census_partitions(by_design, host_nonweight, unexplained, off_nodes) into the header is the change that turns the claim into something callable:

CHECK( census_partitions(3, 2, 0, 5))   // partitions
CHECK( census_partitions(0, 0, 4, 4))   // all unexplained still partitions
CHECK( census_partitions(0, 0, 0, 0))   // no node on the CPU partitions trivially
CHECK(!census_partitions(3, 2, 0, 6))   // one node counted TWICE must not partition
CHECK(!census_partitions(3, 2, 0, 4))   // one node MISSED must not partition

Both failure directions are pinned, which is the pair that matters: a test that only asserted the positive case would pass a census that double-counts and a census that drops a class, since both can be made to sum by a compensating error elsewhere. And the comment says why the extraction exists — "a claim nothing can call with made-up numbers is a claim nothing can test."

Two things I checked because they could have been wrong

  • #403 merged before this commit reached it — the PR says so, and the shape confirms it: the counter and the decision are reordered rather than the decision being re-derived, so nothing about #403's placement logic changes.
  • No device run, and that is correct here: this is a counter and a pure predicate, so -fsyntax-only plus the arithmetic arms is the right evidence. Asking for a GPU run would be asking for a number that cannot move.

One follow-up, not a blocker

census_partitions is extracted and tested but I did not see it called from ggml-hrx.cpp's summary path in this diff. If it stays test-only, a future edit can still print overlapping counters while the test stays green — the predicate proves the arithmetic, not that the census uses it. Wiring the print through the same predicate, or asserting it once at summary time with the real counters, closes that gap. Not required to merge this fix.

Nothing else to change. clean.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 8f7ab2a: READY

This head answers the one ask from my review at 599c1cf1, and it answers it better than the version I asked for.

What I asked for: the sum check had one call site and no arm — a check that cannot fail is not a check. What is here now:

ggml/src/ggml-hrx/host-weight-placement.h:60   inline bool census_partitions(by_design, host_nonweight, unexplained, off_nodes)
tests/test-hrx-host-resident.cpp:93            CHECK(census_partitions(3, 2, 0, 5), "3 by design + 2 host-non-weight")
tests/test-hrx-host-resident.cpp:94            CHECK(census_partitions(0, 0, 4, 4), "all unexplained still passes")

One definition, exported, and the second CHECK is the interesting one: it asserts the sum still passes when every node is unexplained. That is the right thing to pin, and it is worth one line of comment beside it, because it is the fact a future reader will misread — the partition proves the census is complete, it does not prove the placement is good; the rule that refuses a bad placement is first_unexplained, below. As written, someone will eventually "strengthen" this CHECK into census_partitions(0,0,4,4) == false and quietly change a completeness check into a policy check.

And the policy is separate and correct: if (strict && first_unexplained != nullptr) — strict aborts only on an unexplained node, with by-design and host-non-weight named instead of aborting. That is the engine#2450 lesson kept: the earlier refusal flipped with what co-tenants held in the shared GTT pool, which is a refusal whose outcome depends on someone else's process.

Not verified: a device run (no gpu-hold taken for a review), and the WARN, not assert choice — I agree with the reason given in the comment, but I did not exercise the broken-census path.

@bong-water-water-bong
bong-water-water-bong merged commit 7978957 into 1bit/hrx-vulkan-patched Nov 16, 2026
@bong-water-water-bong
bong-water-water-bong deleted the 1bit/host-nonweight-partition branch November 16, 2026 09:27
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant