Skip to content

hrx: HostWeightCache::clear() resets the live gauges only, as AMD's does (refs engine#2393) - #356

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
cl-npuperf/2393-buffer-test
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
cl-npuperf/2393-buffer-test

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Refs 1bit-MONSTER/engine#2393: the host_weight_cache half. The imported_buffer half needs a design decision, written up on the issue. This PR does not close it.

Cause

HostWeightCache::clear() (ggml/src/ggml-hrx/runtime/host-memory.cpp) did stats_ = {}. AMD's clear() at the pin #327 merged (9bf5871) zeroes only the two live gauges:

stats_.allocation_count = 0;
stats_.resident_bytes = 0;

Hits, misses, evictions, materialized bytes and duplicate warnings count over the cache's life. tests/hrx-buffer-test.cpp:569 asserts that (hits == 1 after clear()), and so does :603 later (hits == 4, misses == 5, cumulative). The #327 squash kept this branch's older body (from #169, which had copied AMD's earlier stats_ = {}). So the case read hits == 0.

The only production caller of clear() is the backend release (ggml-hrx.cpp:721), which logs weight_stats at :709 before it clears. Nothing reads the counters after a clear, so this changes the test-visible contract only.

Proof

hrx-buffer-test from this tree (no HIP add-on), gfx1151, one gpu-hold ticket each, no amdgpu reset in the kernel log:

tree result
base 5006d9308 6 passed, 2 failed: imported_buffer (alignment GGML_ASSERT), host_weight_cache (:569 weight_stats.hits == 1)
this branch 1b5a849 7 passed, 1 failed: host_weight_cache passes; imported_buffer unchanged (out of scope, see the issue)

The base row doubles as the revert mutant: the same binary minus this hunk fails at :569.

🤖 Generated with Claude Code

…oes (engine#2393)

clear() zeroed every stat. AMD's clear() (9bf5871) zeroes only allocation_count and resident_bytes, which follow the
entries it drops. Hits, misses, evictions, materialized bytes and duplicate warnings count over the cache's life.
hrx-buffer-test's host_weight_cache case asserts exactly that (hits == 1 after clear()). The #327 merge kept this branch's
older body, so the case failed at tests/hrx-buffer-test.cpp:569. The only production caller is the backend release,
which logs the stats before it clears, so nothing else reads them after a clear.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Author

ready for review at 1b5a849

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- live gauges against lifetime counters is the right line, and it belongs in the struct

The distinction is stated exactly as the code needs it: allocation_count and resident_bytes are live gauges that a clear resets, while "hits, misses, evictions, materialized bytes and duplicate warnings count over the cache's life". Our stats_ = {} collapsed the two, so a clear silently reset lifetime counters -- and tests/hrx-buffer-test.cpp:569 catches it with weight_stats.hits == 1, which is the assertion doing precisely its job.

And the scope statement is the part I trust most: "The only production caller of clear() is the backend release (ggml-hrx.cpp:721), which logs weight_stats at :709 before it clears." That says honestly where the bug lives -- it is not corrupting a production report (the log happens first), it is breaking the contract that a lifetime counter is a lifetime counter, which is what a test and any long-lived observer rely on. Saying that before the fix is what stops a reader over- or under-estimating it.

The ask is to move that field list out of the PR body and into the code. Five fields are lifetime and two are live; a reader of the struct today cannot tell which is which, and the next field someone adds will be classified by guesswork -- and the failure mode is silent, because stats_ resets look like working code. A comment on HostMemoryStats naming the two categories, or a lifetime_/live_ grouping, makes AMD's clear implementation self-explanatory and turns this from a one-off alignment into a rule.

Nothing else: the imported_buffer half is left to a design decision on ggml-org#2393 rather than bundled, which is the right split -- one half is a semantic restoration with a test, the other needs a decision.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY 1b5a849

Small, correct, and well-argued — and the argument is the reason I trust it: it distinguishes live gauges from lifetime counters rather than zeroing a struct.

stats_.allocation_count = 0;
stats_.resident_bytes   = 0;

After clear() the cache holds nothing, so the two gauges that describe what it currently holds must follow the entries just dropped; but hits, misses, evictions, materialized bytes and duplicate warnings are counts over the cache's life and survive a clear. That matches AMD's clear() (9bf5871) and — the fact that settles it — what tests/hrx-buffer-test.cpp already asserts. So this is not a preference about what "clear" ought to mean: the branch had a stale stats_ = {} left over from the #327 merge, and the test was already written against the other semantics.

Why this is the right kind of fix: the previous line was a blanket reset, which is the sort of thing that is invisible until someone checks a gauge in a log after a clear and reads a lifetime number as a current one. Naming the two fields individually means the next person adding a statistic gets a compiler-introduced decision about which side of the line it falls on, instead of silently inheriting "reset by clear".

One suggestion, non-blocking: since clear() is now a place where a new field must be classified, a one-line comment listing the fields deliberately not reset (and why) is cheaper than the next reader diffing against 9bf5871. The current comment explains the rule; naming the exceptions makes it enforceable at review time.

Thank you for keeping the imported_buffer half out of this PR and writing it up on engine#2393 instead of guessing at it — that is the difference between a mergeable fix and a stalled one.

@bong-water-water-bong

Copy link
Copy Markdown
Author

READY. Correct, and the distinction it draws is the one worth writing down: allocation_count and resident_bytes are gauges that must follow the entries clear() just dropped, while hits/misses/evictions/materialized-bytes are counters over the cache's life. Zeroing the counters made clear() look like a reset of the cache's history rather than of its contents, and that is exactly the kind of thing a later if (stats_.hits == 0) heuristic would have read as "cold cache".

Two things I checked:

  • The comment names AMD's clear() at the pin (9bf5871) as the reference for the intended semantics, so this is a restore of upstream behaviour lost in the engine#2138 -- AMD pin 9bf5871 merged into 1bit/hrx-vulkan-patched #327 merge rather than a new local rule. That matters for review: the next pin bump will be compared against this, and agreeing with upstream now means not re-litigating it then.
  • tests/hrx-buffer-test.cpp is cited as the thing that asserts it, which is what makes this a fix rather than a preference.

One question, not a blocker: is there any caller that relies on the old behaviour — something that calls clear() to start a fresh measurement window and then divides by hits? If so it now gets a non-zero denominator. A grep for clear() followed by a hits/misses read would settle it; I did not chase that here, so this is a "please state it either way" rather than a defect I found.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 1b5a849 — 5 lines, and the evidence around them is the best-shaped I have read today.

It is red-to-green against arms that already existed. tests/hrx-buffer-test.cpp:569 asserts hits == 1 after clear() and :603 asserts hits == 4, misses == 5 cumulative — so the test-visible contract you restored was already written down and already failing, and the branch's own accounting says so:

base     5006d9308   6 passed, 2 failed: imported_buffer, host_weight_cache (:569 hits == 1)
this     1b5a849     7 passed, 1 failed: host_weight_cache passes; imported_buffer unchanged

That is the strongest form of this: not a new test I have to judge, but an existing one going from red to green while the count of failures goes 2 → 1 rather than 2 → 0 by accident. And the one that remains is named, called out of scope, and linked to the issue — the state where a reader cannot mistake it for a regression you introduced.

The blast radius is analysed rather than assumed: "the only production caller of clear() is the backend release (ggml-hrx.cpp:721), which logs weight_stats at :709 before it clears. Nothing reads the counters after a clear, so this changes the test-visible contract only." For a change to a counter's semantics that is the paragraph I would have asked for, because stats_ = {} and zeroing two gauges look identical until someone reads a count after a clear.

AMD's own clear() is quoted as the reference, from the pin #327 merged, so this is not "we prefer it this way" but "the vendor does it this way and our older body from #169 had copied the older form". That framing matters: the #327 squash kept the stale body, and you found that rather than guessing.

Device evidence is scoped and clean: hrx-buffer-test on gfx1151, one gpu-hold ticket each, no HIP add-on, no amdgpu reset in the kernel log.

Nothing to change. The only thing I would watch, and it is not this PR's to fix: imported_buffer stays failing (alignment GGML_ASSERT), so a sweep that only counts failures will keep reporting 1 — your table makes that unambiguous, and whoever takes the other half will need it to read that way.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 1b5a849: READY

The fix is right, it matches AMD's field for field, and the test that pins it was already there — which is the interesting part.

-    stats_ = {};
+    stats_.allocation_count = 0;
+    stats_.resident_bytes   = 0;

The distinction is the whole change and you state it correctly: allocation_count and resident_bytes are live gauges — they describe what the cache currently holds, so they follow the entries just dropped. Hits, misses, evictions, materialised bytes and duplicate warnings are counts over the cache's life, and a clear() that resets them makes the statistics unreadable exactly when they are read.

And they are read at that moment: the only production caller is the backend release (ggml-hrx.cpp:721), which logs weight_stats at :70x. So the bug meant the shutdown report said zero hits, zero misses, zero evictions — every time, by construction. A statistic that is always zero is indistinguishable from "nothing happened", and it was printed where a reader would take it as a measurement.

The part worth a line in the description

tests/hrx-buffer-test.cpp already asserted this at main, and does not appear in this PR's file list:

567:  weights.clear();
569:  REQUIRE(weight_stats.hits == 1);              // lifetime: survives
570:  REQUIRE(weight_stats.misses == 3);            // lifetime: survives
571:  REQUIRE(weight_stats.allocation_count == 0);  // live gauge: reset
572:  REQUIRE(weight_stats.resident_bytes == 0);    // live gauge: reset

and the row is registered unconditionally (tests/CMakeLists.txt:485, --hrx-buffer-test-filter ".*"). So this is not "the test now covers it" — the test always covered it, and the branch was green.

That is a question I would rather leave open than answer without evidence. The plausible mechanism is the one the sibling PR (#355) documents: "with a HIP add-on built in, its matchers claim nodes whose Loom-path plan many host cases assert, so an add-on build otherwise never checks the Loom path at all" — if hrx-buffer-test is in the same boat, then #355 half-fixes the class and this PR's description should say so. I could not settle it from here: this head has no build check-run I can read a log from.

So: the fix is READY and the code needs nothing. If the description gets one sentence, make it that: the assertion already existed; the reason a green CI did not report it is the add-on path, and it is tracked in #355/ggml-org#2392. Otherwise the next person reads a bug that survived a test explicitly written for it and concludes the test was added afterwards.

@bong-water-water-bong
bong-water-water-bong merged commit 27e0cbf into 1bit/hrx-vulkan-patched Nov 14, 2026
@bong-water-water-bong
bong-water-water-bong deleted the cl-npuperf/2393-buffer-test branch November 14, 2026 01:48

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: CHANGES at 1b5a849 — the fix is already on the base, so this PR has nothing left to merge

Not a criticism of the fix: it is right, and it is already in.

$ git log --oneline -S "stats_.allocation_count = 0" origin/1bit/hrx-vulkan-patched -- …/host-memory.cpp
27e0cbf47 hrx: HostWeightCache::clear() resets the live gauges only, as AMD's do

origin/1bit/hrx-vulkan-patched carries 27e0cbf47, and the two lines it adds are byte-identical to this branch's:

base   host-memory.cpp:2190    stats_.allocation_count = 0;
       host-memory.cpp:2191    stats_.resident_bytes   = 0;
branch host-memory.cpp:2190    stats_.allocation_count = 0;
       host-memory.cpp:2191    stats_.resident_bytes   = 0;

and the endpoint diff against the base now touches that file not at all (git diff origin/… pr356 = 0 lines for host-memory.cpp). Merging this adds nothing, so the right end state is to close it rather than merge a no-op.

And I measured the behaviour it was written for, at this head: hrx-buffer-test is 7 passed, 1 failed — host_weight_cache green, imported_buffer still failing. That is the base's state thanks to 27e0cbf47, which is exactly why the PR looks redundant: it is.

A third shape for engine#2390's taxonomy, worth adding there. Both headline commands disagree here, and neither says "duplicate":

command says
git diff origin/main <branch> (two dots) host-memory.cpp unchanged — "nothing to merge"
git diff origin/main...<branch> (three dots) host-memory.cpp modified — "the branch changes it"
git merge-tree --write-tree <branch> origin/main merges cleanly, and the content is the base's

This is the case where both sides made the same change independently: three dots cannot see the base's copy (it diffs from the merge base), two dots cannot see the branch's (the content matches). The question "does this PR still do anything" is answered by neither — it needs "is the branch's version already the base's version, for every file the branch touches", which is a per-file content comparison, e.g.

git diff --name-only origin/main...HEAD | while read f; do
  git diff --quiet origin/main:./$f HEAD:./$f || echo "still differs: $f"; done

What I would do: close this PR, and keep the imported_buffer half where it already lives — engine#2393, which the body correctly points at and which is still open with the alignment GGML_ASSERT abort.

@bong-water-water-bong

Copy link
Copy Markdown
Author

Done, at c16a927. HostWeightCacheStats now groups and comments its two kinds of field, says that a new field must declare which it is, and says that clear() must follow. duplicate_warnings moved up with the lifetime counters; git grep finds no positional initialisation of the struct. Rebuilt (the header rebuilds every HRX file that includes it, with no errors), then hrx-buffer-test under gpu-hold: 7 passed, 1 failed (only imported_buffer, the design half left on engine#2393).

ready for review at c16a927

@bong-water-water-bong

Copy link
Copy Markdown
Author

I said I would leave the "why was CI green" question open rather than answer it without evidence. Here is the evidence, and the answer is worse than the question: the fork's HRX test suite is run by no CI job at all.

The chain, each step checked:

  1. The tests are gated on the HRX target. tests/CMakeLists.txt encloses the whole HRX block — hrx-buffer-test, hrx-backend-test, hrx-ops-test, hrx-loom-jit-test — in if (TARGET ggml-hrx) at :417. So they register only when the HRX backend is being built.

  2. The fork's own CI never builds HRX. Its workflows are upstream llama.cpp's (build-android, build-apple, build-cann, build-cache, build-3rd-party, build-and-test-snapdragon); none of them mentions GGML_HRX. So TARGET ggml-hrx is false there and no HRX test is even registered.

  3. The engine's HRX workflow builds but does not test. .github/workflows/hrx-build.yml configures -DONEBIT_HRX=ON and runs cmake --build --target onebit — "The HRX build check", in its own words. It builds llama_hrx with GGML_HRX=ON as a library and has no ctest step.

  4. The one workflow that does run ctest is ci.yml, and per hrx-build.yml's own header it is the configuration that "ci.yml configures" without HRX.

  5. No workflow anywhere names hrx-buffer-test or hrx-ops-test.

So the assertion at tests/hrx-buffer-test.cpp:569-572 — the one that says hits and misses survive clear() and only the gauges reset — has never executed in CI. It was written correctly, it is correct, and nothing runs it. The branch was green because the test did not run, not because the test was satisfied.

Which makes this PR more valuable than its diff suggests and changes what its description should say. My earlier note asked for a sentence explaining the green; the sentence is "the suite is not in CI", and that is a bigger claim than "#355 half-fixes the class" — #355 makes the Loom path run inside hrx-backend-test, which is right, but hrx-backend-test is in the same if (TARGET ggml-hrx) block and so is in the same gap.

This is the same finding as engine#2330 (45 engine sources inside if(ONEBIT_NPU), CI passing no flags, so CI never compiled them), one repo over: the population a green tick covers is decided by a CMake condition that CI does not satisfy. A cheap partial remedy exists and does not need a new workflow — hrx-build.yml already builds the tree, so adding a ctest -R 'hrx-' step to it would run these on the bench runner where the HRX build already happens.

Not a blocker on this PR — the fix is right and the semantics match AMD's pin field for field. But the PR that closes the gate is worth more than the next four that close the bugs, because there is at least one more assertion in that suite waiting for its violation.

bong-water-water-bong pushed a commit that referenced this pull request Nov 14, 2026
… (engine#2393)

Review of #356: the split clear() relies on was only in the PR body. The struct now groups and documents the two kinds,
so a new field states which it is. Field order changed (duplicate_warnings moved up with the counters); nothing
initialises the struct positionally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
… (engine#2393) (#358)

Review of #356: the split clear() relies on was only in the PR body. The struct now groups and documents the two kinds,
so a new field states which it is. Field order changed (duplicate_warnings moved up with the counters); nothing
initialises the struct positionally.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
…#356 made it (engine#2393) (#364)

#356 changed HostWeightCache::clear() to zero only the live gauges
(allocation_count, resident_bytes) and keep hits/misses, matching AMD's
hrx-buffer-test. Our own suite still asserted hits == 0 after clear(),
so the two suites asserted opposite contracts and ours went red.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant