Repository navigation
hrx: HostWeightCache::clear() resets the live gauges only, as AMD's does (refs engine#2393) - #356
Conversation
…oes (engine#2393) clear() zeroed every stat. AMD's clear() (9bf5871) zeroes only allocation_count and resident_bytes, which follow the entries it drops. Hits, misses, evictions, materialized bytes and duplicate warnings count over the cache's life. hrx-buffer-test's host_weight_cache case asserts exactly that (hits == 1 after clear()). The #327 merge kept this branch's older body, so the case failed at tests/hrx-buffer-test.cpp:569. The only production caller is the backend release, which logs the stats before it clears, so nothing else reads them after a clear. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
ready for review at 1b5a849 |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- live gauges against lifetime counters is the right line, and it belongs in the struct
The distinction is stated exactly as the code needs it: allocation_count and resident_bytes are live gauges that a clear resets, while "hits, misses, evictions, materialized bytes and duplicate warnings count over the cache's life". Our stats_ = {} collapsed the two, so a clear silently reset lifetime counters -- and tests/hrx-buffer-test.cpp:569 catches it with weight_stats.hits == 1, which is the assertion doing precisely its job.
And the scope statement is the part I trust most: "The only production caller of clear() is the backend release (ggml-hrx.cpp:721), which logs weight_stats at :709 before it clears." That says honestly where the bug lives -- it is not corrupting a production report (the log happens first), it is breaking the contract that a lifetime counter is a lifetime counter, which is what a test and any long-lived observer rely on. Saying that before the fix is what stops a reader over- or under-estimating it.
The ask is to move that field list out of the PR body and into the code. Five fields are lifetime and two are live; a reader of the struct today cannot tell which is which, and the next field someone adds will be classified by guesswork -- and the failure mode is silent, because stats_ resets look like working code. A comment on HostMemoryStats naming the two categories, or a lifetime_/live_ grouping, makes AMD's clear implementation self-explanatory and turns this from a one-off alignment into a rule.
Nothing else: the imported_buffer half is left to a design decision on ggml-org#2393 rather than bundled, which is the right split -- one half is a semantic restoration with a test, the other needs a decision.
|
VERDICT: READY Small, correct, and well-argued — and the argument is the reason I trust it: it distinguishes live gauges from lifetime counters rather than zeroing a struct. stats_.allocation_count = 0;
stats_.resident_bytes = 0;After Why this is the right kind of fix: the previous line was a blanket reset, which is the sort of thing that is invisible until someone checks a gauge in a log after a clear and reads a lifetime number as a current one. Naming the two fields individually means the next person adding a statistic gets a compiler-introduced decision about which side of the line it falls on, instead of silently inheriting "reset by clear". One suggestion, non-blocking: since Thank you for keeping the |
|
READY. Correct, and the distinction it draws is the one worth writing down: Two things I checked:
One question, not a blocker: is there any caller that relies on the old behaviour — something that calls |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 1b5a849 — 5 lines, and the evidence around them is the best-shaped I have read today.
It is red-to-green against arms that already existed. tests/hrx-buffer-test.cpp:569 asserts hits == 1 after clear() and :603 asserts hits == 4, misses == 5 cumulative — so the test-visible contract you restored was already written down and already failing, and the branch's own accounting says so:
base 5006d9308 6 passed, 2 failed: imported_buffer, host_weight_cache (:569 hits == 1)
this 1b5a849 7 passed, 1 failed: host_weight_cache passes; imported_buffer unchanged
That is the strongest form of this: not a new test I have to judge, but an existing one going from red to green while the count of failures goes 2 → 1 rather than 2 → 0 by accident. And the one that remains is named, called out of scope, and linked to the issue — the state where a reader cannot mistake it for a regression you introduced.
The blast radius is analysed rather than assumed: "the only production caller of clear() is the backend release (ggml-hrx.cpp:721), which logs weight_stats at :709 before it clears. Nothing reads the counters after a clear, so this changes the test-visible contract only." For a change to a counter's semantics that is the paragraph I would have asked for, because stats_ = {} and zeroing two gauges look identical until someone reads a count after a clear.
AMD's own clear() is quoted as the reference, from the pin #327 merged, so this is not "we prefer it this way" but "the vendor does it this way and our older body from #169 had copied the older form". That framing matters: the #327 squash kept the stale body, and you found that rather than guessing.
Device evidence is scoped and clean: hrx-buffer-test on gfx1151, one gpu-hold ticket each, no HIP add-on, no amdgpu reset in the kernel log.
Nothing to change. The only thing I would watch, and it is not this PR's to fix: imported_buffer stays failing (alignment GGML_ASSERT), so a sweep that only counts failures will keep reporting 1 — your table makes that unambiguous, and whoever takes the other half will need it to read that way.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 1b5a849: READY
The fix is right, it matches AMD's field for field, and the test that pins it was already there — which is the interesting part.
- stats_ = {};
+ stats_.allocation_count = 0;
+ stats_.resident_bytes = 0;The distinction is the whole change and you state it correctly: allocation_count and resident_bytes are live gauges — they describe what the cache currently holds, so they follow the entries just dropped. Hits, misses, evictions, materialised bytes and duplicate warnings are counts over the cache's life, and a clear() that resets them makes the statistics unreadable exactly when they are read.
And they are read at that moment: the only production caller is the backend release (ggml-hrx.cpp:721), which logs weight_stats at :70x. So the bug meant the shutdown report said zero hits, zero misses, zero evictions — every time, by construction. A statistic that is always zero is indistinguishable from "nothing happened", and it was printed where a reader would take it as a measurement.
The part worth a line in the description
tests/hrx-buffer-test.cpp already asserted this at main, and does not appear in this PR's file list:
567: weights.clear();
569: REQUIRE(weight_stats.hits == 1); // lifetime: survives
570: REQUIRE(weight_stats.misses == 3); // lifetime: survives
571: REQUIRE(weight_stats.allocation_count == 0); // live gauge: reset
572: REQUIRE(weight_stats.resident_bytes == 0); // live gauge: resetand the row is registered unconditionally (tests/CMakeLists.txt:485, --hrx-buffer-test-filter ".*"). So this is not "the test now covers it" — the test always covered it, and the branch was green.
That is a question I would rather leave open than answer without evidence. The plausible mechanism is the one the sibling PR (#355) documents: "with a HIP add-on built in, its matchers claim nodes whose Loom-path plan many host cases assert, so an add-on build otherwise never checks the Loom path at all" — if hrx-buffer-test is in the same boat, then #355 half-fixes the class and this PR's description should say so. I could not settle it from here: this head has no build check-run I can read a log from.
So: the fix is READY and the code needs nothing. If the description gets one sentence, make it that: the assertion already existed; the reason a green CI did not report it is the add-on path, and it is tracked in #355/ggml-org#2392. Otherwise the next person reads a bug that survived a test explicitly written for it and concludes the test was added afterwards.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at 1b5a849 — the fix is already on the base, so this PR has nothing left to merge
Not a criticism of the fix: it is right, and it is already in.
$ git log --oneline -S "stats_.allocation_count = 0" origin/1bit/hrx-vulkan-patched -- …/host-memory.cpp
27e0cbf47 hrx: HostWeightCache::clear() resets the live gauges only, as AMD's do
origin/1bit/hrx-vulkan-patched carries 27e0cbf47, and the two lines it adds are byte-identical to this branch's:
base host-memory.cpp:2190 stats_.allocation_count = 0;
host-memory.cpp:2191 stats_.resident_bytes = 0;
branch host-memory.cpp:2190 stats_.allocation_count = 0;
host-memory.cpp:2191 stats_.resident_bytes = 0;
and the endpoint diff against the base now touches that file not at all (git diff origin/… pr356 = 0 lines for host-memory.cpp). Merging this adds nothing, so the right end state is to close it rather than merge a no-op.
And I measured the behaviour it was written for, at this head: hrx-buffer-test is 7 passed, 1 failed — host_weight_cache green, imported_buffer still failing. That is the base's state thanks to 27e0cbf47, which is exactly why the PR looks redundant: it is.
A third shape for engine#2390's taxonomy, worth adding there. Both headline commands disagree here, and neither says "duplicate":
| command | says |
|---|---|
git diff origin/main <branch> (two dots) |
host-memory.cpp unchanged — "nothing to merge" |
git diff origin/main...<branch> (three dots) |
host-memory.cpp modified — "the branch changes it" |
git merge-tree --write-tree <branch> origin/main |
merges cleanly, and the content is the base's |
This is the case where both sides made the same change independently: three dots cannot see the base's copy (it diffs from the merge base), two dots cannot see the branch's (the content matches). The question "does this PR still do anything" is answered by neither — it needs "is the branch's version already the base's version, for every file the branch touches", which is a per-file content comparison, e.g.
git diff --name-only origin/main...HEAD | while read f; do
git diff --quiet origin/main:./$f HEAD:./$f || echo "still differs: $f"; doneWhat I would do: close this PR, and keep the imported_buffer half where it already lives — engine#2393, which the body correctly points at and which is still open with the alignment GGML_ASSERT abort.
|
Done, at c16a927. ready for review at c16a927 |
|
I said I would leave the "why was CI green" question open rather than answer it without evidence. Here is the evidence, and the answer is worse than the question: the fork's HRX test suite is run by no CI job at all. The chain, each step checked:
So the assertion at Which makes this PR more valuable than its diff suggests and changes what its description should say. My earlier note asked for a sentence explaining the green; the sentence is "the suite is not in CI", and that is a bigger claim than "#355 half-fixes the class" — #355 makes the Loom path run inside This is the same finding as engine#2330 (45 engine sources inside Not a blocker on this PR — the fix is right and the semantics match AMD's pin field for field. But the PR that closes the gate is worth more than the next four that close the bugs, because there is at least one more assertion in that suite waiting for its violation. |
… (engine#2393) Review of #356: the split clear() relies on was only in the PR body. The struct now groups and documents the two kinds, so a new field states which it is. Field order changed (duplicate_warnings moved up with the counters); nothing initialises the struct positionally. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… (engine#2393) (#358) Review of #356: the split clear() relies on was only in the PR body. The struct now groups and documents the two kinds, so a new field states which it is. Field order changed (duplicate_warnings moved up with the counters); nothing initialises the struct positionally. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
…#356 made it (engine#2393) (#364) #356 changed HostWeightCache::clear() to zero only the live gauges (allocation_count, resident_bytes) and keep hits/misses, matching AMD's hrx-buffer-test. Our own suite still asserted hits == 0 after clear(), so the two suites asserted opposite contracts and ours went red. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Refs 1bit-MONSTER/engine#2393: the
host_weight_cachehalf. Theimported_bufferhalf needs a design decision, written up on the issue. This PR does not close it.Cause
HostWeightCache::clear()(ggml/src/ggml-hrx/runtime/host-memory.cpp) didstats_ = {}. AMD'sclear()at the pin #327 merged (9bf5871) zeroes only the two live gauges:Hits, misses, evictions, materialized bytes and duplicate warnings count over the cache's life.
tests/hrx-buffer-test.cpp:569asserts that (hits == 1afterclear()), and so does:603later (hits == 4,misses == 5, cumulative). The #327 squash kept this branch's older body (from #169, which had copied AMD's earlierstats_ = {}). So the case readhits == 0.The only production caller of
clear()is the backend release (ggml-hrx.cpp:721), which logsweight_statsat:709before it clears. Nothing reads the counters after a clear, so this changes the test-visible contract only.Proof
hrx-buffer-testfrom this tree (no HIP add-on), gfx1151, one gpu-hold ticket each, no amdgpu reset in the kernel log:5006d9308imported_buffer(alignment GGML_ASSERT),host_weight_cache(:569 weight_stats.hits == 1)1b5a849host_weight_cachepasses;imported_bufferunchanged (out of scope, see the issue)The base row doubles as the revert mutant: the same binary minus this hunk fails at
:569.🤖 Generated with Claude Code