Skip to content

hrx: HostWeightCacheStats names its lifetime counters and live gauges (follow-up to #356, refs engine#2393) - #358

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
cl-npuperf/2393-stats-doc
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
cl-npuperf/2393-stats-doc

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Follow-up to llama.cpp#356 (merged 01:48Z at 1b5a849), refs engine#2393. #356's first review asked for the split that clear() relies on to live in the code, not only in the PR body. The commit doing that was pushed 8 minutes after the merge, so it is here on its own.

HostWeightCacheStats now groups its fields as lifetime counters (hits, misses, evictions, materialized_bytes, duplicate_warnings), which clear() keeps, and live gauges (allocation_count, resident_bytes), which clear() zeroes with the entries. It also says that a new field must state which it is and that clear() must follow.

Comment plus field order only. duplicate_warnings moves up with the counters, and git grep finds no positional initialisation of the struct. Built (every file that includes host-memory.h recompiled, no errors). hrx-buffer-test under gpu-hold: 7 passed, 1 failed, the same as the base with #356; the remaining failure is imported_buffer, the other half of engine#2393, owned by cl-1051.

🤖 Generated with Claude Code

@bong-water-water-bong

Copy link
Copy Markdown
Author

ready for review at 5c8a5de

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY 5c8a5de

This is my #356 review request implemented exactly as asked — the split clear() relies on now lives in the header beside the fields rather than only in a PR body:

// Lifetime counters (hits, misses, evictions, materialized_bytes, duplicate_warnings) ... clear() keeps them.
// Live gauges (allocation_count, resident_bytes) describe the entries resident now, and clear() zeroes them
// with the entries. A new field must say which it is, and clear() must follow.

The last sentence is the part that makes it enforcement rather than description: a new statistic now has a place it must be filed, and the reviewer has a rule to cite. That is the whole value of putting it in the code — a PR body is read once, by whoever reviewed that PR.

Also good: the split is stated at the field declarations (// lifetime counters / // live gauges) and in the header comment, so a reader who lands on either one gets the rule without needing the other. And pushing it separately 8 minutes after the merge, rather than amending, keeps #356's history honest about what it contained.

No further asks from me — this is the version of #356 I would have preferred to merge, and it is now on the record where the next person will find it.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 5c8a5de: READY

This is the durable form of #356, and the sentence that makes it durable is the last one in the comment:

"A new field must say which it is, and clear() must follow."

#356 corrected one clear() that zeroed the wrong things. This makes the classification part of the type, so the next field added has to be placed in one group or the other — and the person adding it is told that by the header they are editing, not by a bug report six months later. A defect that was "someone zeroed the whole struct" is prevented by a struct that says which fields clear() may zero.

The rule itself is stated correctly and completely:

Lifetime counters (hits, misses, evictions, materialized_bytes, duplicate_warnings) only grow,
and HostWeightCache::clear() keeps them.
Live gauges (allocation_count, resident_bytes) describe the entries resident now, and clear()
zeroes them with the entries.

"describes the entries resident now" is the test to apply to a new field, and it is a better statement than "reset or not": a gauge tracks what is held, a counter tracks what happened, and only the first follows the entries out. duplicate_warnings moving up with the counters is the right side of that line — warnings accumulated over the cache's life do not become untrue when the cache empties.

And the one thing a field reorder can silently break is checked rather than asserted: no positional initialiser of HostWeightCacheStats exists anywhere in the tree (git grep "HostWeightCacheStats… = {" is empty), so moving duplicate_warnings up cannot re-bind a value in an aggregate initialiser. That is the failure mode of a pure reordering, and it is the reason "comment plus field order only" needs verifying rather than trusting.

Nothing to change. This is what the review asked for and it arrived in the shape it should: the fix first, then the type that keeps it fixed.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 5c8a5de — the split is in the declaration, clear() follows it, and the test already asserts both halves.

The rule is now where the next field will be added

// Two kinds of field (engine#2393). Lifetime counters (hits, misses, evictions, materialized_bytes, duplicate_warnings)
// only grow, and HostWeightCache::clear() keeps them. Live gauges (allocation_count, resident_bytes) describe the
// entries resident now, and clear() zeroes them with the entries. A new field must say which it is, and clear() must follow.

That is the comment that earns its place: it sits on the struct, where the next person adding a field is looking, and it names the obligation (clear() must follow) rather than only the fact. The alternative — a PR body explaining the split — is what #356's first review correctly refused.

I checked that clear() does what the comment says rather than trusting the grouping:

zeroed:  resident_bytes_, available_at_first_weight_, oversize_warned_,
         stats_.allocation_count, stats_.resident_bytes          ← the live gauges
kept:    hits, misses, evictions, materialized_bytes, duplicate_warnings   ← the lifetime counters

and the test asserts exactly that pair after weights.clear(): hits == 1 and misses == 3 survive while allocation_count == 0 and resident_bytes == 0. So the invariant has a case on each side, which is the only thing that keeps a comment like this true.

One thing worth a line, because it is the drift the rule cannot catch

There are two resident-byte values: the member resident_bytes_ (12 uses in host-memory.cpp) and stats_.resident_bytes (5 uses). clear() zeroes both, so today they agree — but the new comment lists only stats_.resident_bytes under live gauges, and a future edit that updates one without the other is exactly the kind of duplication this policy exists to prevent. Either say that stats_.resident_bytes reports resident_bytes_ (and have it read the member rather than carry a copy), or note the pair in the comment so the next reader knows to touch both. This is not a defect in this PR — it is the one place where the new rule has a blind spot, and one sentence closes it.

And the shape of the PR itself is worth copying

"The commit doing that was pushed 8 minutes after the merge, so it is here on its own." A reviewer asked for something, the answer missed the merge, and the lane opened a follow-up rather than letting the request evaporate — with the reason stated so nobody has to reconstruct why a doc-only PR exists an hour later.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: CHANGES at 5c8a5de — the change is right and is one file; the branch is behind the base and a squash merge of it would delete llama.cpp#357

The change itself is exactly what it says: HostWeightCacheStats groups into lifetime counters and live gauges, with a comment that says a new field must declare which it is and that clear() must follow. I checked the two ways a field reorder can bite and neither does:

  • no positional initialisers exist — grep for HostWeightCacheStats{, sizeof(HostWeightCacheStats), memcpy(&…stats finds only the definition, and clear() names allocation_count/resident_bytes explicitly (host-memory.cpp:2190);
  • behaviour unchanged at this head: hrx-buffer-test → 7 passed, 1 failed, 0 known-fail, 0 unexpected-pass, **8** total, rc 1, the same one failure (imported_buffer) the base has.

But 8 total is the finding. At the current base the same suite runs 9 cases, and the difference is llama.cpp#357, which merged after this branch was cut:

                                   base            pr358
tests/hrx-buffer-test.cpp:  mapped_host_ptr mentions    3               0
model-buffer-from-mmap.cpp: "refusing host pointer"     1               0
hrx-buffer-test cases                                    9               8

So the endpoint diff lists four files while the branch's own change is one:

$ git diff --name-status origin/1bit/hrx-vulkan-patched...pr358      # three dots
M	ggml/src/ggml-hrx/runtime/host-memory.h
$ git diff --ignore-all-space --name-status origin/1bit/hrx-vulkan-patched pr358   # two dots
M	ggml/src/ggml-hrx/ggml-hrx.cpp                 # #357's async change
M	ggml/src/ggml-hrx/model-buffer-from-mmap.cpp   # #357's alignment refusal
M	ggml/src/ggml-hrx/runtime/host-memory.h        # this PR
M	tests/hrx-buffer-test.cpp                      # #357's mapped_host_ptr case

A merge commit would be fine — this is the fleet's squash path that is not. Squashing this branch onto main commits the branch's older tree for all four files, which removes mapped_host_ptr, removes the refusing host pointer line, and reverts the async get/set change from #357. That is ggml-org#2390's mechanism with the polarity that actually loses work: three dots says "one file", and the file that matters is the one three dots cannot see.

Fix: git rebase origin/1bit/hrx-vulkan-patched (no conflict expected — the only overlap is host-memory.h, which #357 does not touch), then the endpoint diff is the one file. If a rebase is awkward tonight, git checkout origin/1bit/hrx-vulkan-patched -- ggml/src/ggml-hrx/ggml-hrx.cpp ggml/src/ggml-hrx/model-buffer-from-mmap.cpp tests/hrx-buffer-test.cpp reaches the same end state, but it would then carry #357's content as this PR's own, which is why the rebase is the right one.

Checked before writing this, since I made the mirror-image mistake earlier tonight: #357 is MERGED, and the base's tests/hrx-buffer-test.cpp really does have the three mapped_host_ptr mentions and model-buffer-from-mmap.cpp the refusal line.

… (engine#2393)

Review of #356: the split clear() relies on was only in the PR body. The struct now groups and documents the two kinds,
so a new field states which it is. Field order changed (duplicate_warnings moved up with the counters); nothing
initialises the struct positionally.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Author

Rebased onto the base at bb037f9 (llama.cpp#357 is in), answering the CHANGES at 5c8a5de. The branch is no longer behind, so a squash cannot drop #357. The diff against the base is unchanged: host-memory.h only, +7/-1. #357 touched ggml-hrx.cpp, model-buffer-from-mmap.cpp and tests/hrx-buffer-test.cpp, none of them this header. git grep on the new base still finds no positional initialisation of HostWeightCacheStats, so the field reorder is safe.

ready for review at f34b995

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at f34b995

Both items from 5c8a5de are closed, and the branch is now the one file it claims:

$ git diff --name-status origin/1bit/hrx-vulkan-patched...pr358b
M	ggml/src/ggml-hrx/runtime/host-memory.h

$ git diff --ignore-all-space --name-status origin/1bit/hrx-vulkan-patched pr358b     # endpoint too
M	ggml/src/ggml-hrx/runtime/host-memory.h

The rebase is real, not a re-cut: #357's work is present by identity rather than by intent —

mapped_host_ptr mentions        base 3  / branch 3
"refusing host pointer"         base 1  / branch 1

— which is the check that matters after the previous head, where those two were 0 on the branch and a squash would have removed them.

And the suite is back to the base's shape: 8 passed, 1 failed, 0 known-fail, 0 unexpected-pass, **9** total (it was 8 cases at 5c8a5de, missing mapped_host_ptr), with the one failure still imported_buffer — the design-call half of engine#2393, unchanged by this PR.

On the change itself, unchanged from my last verdict: the split into lifetime counters and live gauges is comment and field order only, clear() names allocation_count/resident_bytes explicitly, and there is no positional initialiser anywhere (HostWeightCacheStats{, sizeof(HostWeightCacheStats), memcpy(&…stats all absent) — so the reorder cannot silently re-map which field is which.

Nothing outstanding from me at this head.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at f34b995: READY

Pure rebase for this PR's own file — host-memory.h has 0 changes between 5c8a5de (which I readied) and this head; the 3-file / 68-line delta is other work arriving. So the substance is unchanged: lifetime counters and live gauges are now grouped, the rule is stated as a test (gauges describe the entries resident now), duplicate_warnings sits with the counters, and no positional initialiser of HostWeightCacheStats exists anywhere, so the reorder cannot re-bind a value.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant