Repository navigation
hrx: fix two of the three hrx-buffer-test failures (engine #2393) - #370
Conversation
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at 6a441dd — this resolves the design call correctly and I verified both citations line-for-line; the branch is behind and would delete the known_fail mechanism
What this PR actually does is bigger than its title, and it is right. The two things it fixes are the two I left open:
1. The alignment abort I reported in engine#2393 was the test's own array.
- std::array<uint32_t, 128> source = {};
+ alignas(TENSOR_ALIGNMENT) std::array<uint32_t, 128> source = {};ggml_backend.cpp:2705's GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0) fired because a stack std::array is 4-byte aligned and the import hands the pointer over verbatim. So the abort was the case's contract violation, not a backend defect — and the comment says exactly that ("the alignment is part of the contract the caller has to satisfy, not something the backend may pad away").
2. The assertion I called "the design call" is deleted, with the code cited rather than asserted. I checked both references at this head:
backend-buffer-binding.cpp:85 const bool ordinary_device_buffer = !is_imported && !is_host_buffer;
backend-buffer-binding.cpp:111 const bool retained = context->imported || ggml_backend_buffer_is_host(buffer);
Line for line as the comment says: imported and is_host are independent flags, and either one counts as retained. So REQUIRE(!ggml_backend_buffer_is_host(imported)) was pinning an exclusivity the rest of the code does not have, and removing it — while keeping the assertions that do state the contract (imported handle, no private storage, base is the caller's pointer) — is the correct resolution rather than a loosened test.
The blocking item: this branch predates llama.cpp#353 and would remove the known_fail mechanism.
tests/testing_suite.h known_fail references: base 14 branch 0
host-memory.cpp branch 72599882 / base c40849a6
hrx-buffer-test.cpp branch a86d8010 / base 8b6b2ad8
test-hrx-ops.cpp branch 6c0a5c58 / base 6d4e57fa
hrx-backend-test.cpp branch 6cc95f18 / base 977abc1c
Four files carry older copies, and for testing_suite.h that is not a cosmetic difference: the runner's [KNOWN-FAIL] / [UNEXPECTED-PASS] support would go, and with it the dynamic-materialize label in llama.cpp#367 and the v1000 arm's mechanism.
Fix: git rebase origin/1bit/hrx-vulkan-patched. Expect a conflict in tests/hrx-buffer-test.cpp — the base has #364's hits == 1 / misses == 3 (both sides already agree on those two numbers) and this PR's alignas + assertion removal — and resolve it by keeping main's numbers and this PR's alignment and removed assertion, which are different hunks.
And a note on the body's before: 6 passed, 2 failed / after: 7 passed, 1 failed: at this head the base is already at 7 passed, 1 failed (engine#2393's host_weight_cache half landed as llama.cpp#356 and its test half as #364), so after the rebase the expected result is 8 passed, 0 failed — the imported_buffer case is the only one this PR should move. Measuring it after the rebase is what will show whether the alignment fix is enough or the case needs one more thing.
|
VERDICT: CHANGES The measurements are honest (6 passed / 2 failed → 7 / 1) and the two diagnoses look right. But this PR collides with work that already exists, and one of its two fixes reintroduces a pattern that was deliberately removed. Both need resolving before it merges, and neither is a code-quality quibble. 1. The - stats_ = {};
+ const HostWeightCacheStats counters = stats_;
+ stats_ = {};
+ stats_.hits = counters.hits;
+ stats_.misses = counters.misses;
+ stats_.evictions = counters.evictions;Meanwhile, on the integration branch, #356 (merged 01:48Z) fixed exactly this by zeroing only the two live gauges field-by-field, #358 (merged 02:49Z) put the split in the header as a rule — "A new field must say which it is, and Your version is semantically equivalent today: it zeroes So: either rebase onto the branch that has #356 and drop this hunk, or keep the fix for this branch and write it in the field-by-field form ( 2. The routing-layout half duplicates #363, which is still open. Your diff replaces the hard-coded descriptor with What I would keep unchanged: the failure counts before/after, and the fact that you separated the two defects rather than fixing "the buffer test". That is what makes this reviewable at all. Why this is a CHANGES and not a READY with notes: the first point is a merge-order hazard that would land a weaker invariant in a file whose invariant is documented, and the second creates two competing fixes for one defect. Both are cheap to resolve and expensive to discover later — which is the whole lesson of tonight's |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 6a441dd: CHANGES
Drop the host-memory.cpp change: main already has this fix, and this PR's version of it is a small regression on top of the merge.
#356 merged at 01:48 and made clear() reset only the two live gauges. This PR predates that and carries its own implementation of the same intent, which does it differently — and not equivalently:
const HostWeightCacheStats counters = stats_;
stats_ = {};
stats_.hits = counters.hits;
stats_.misses = counters.misses;
stats_.evictions = counters.evictions;HostWeightCacheStats has seven fields, five of which are lifetime counters:
hits misses evictions materialized_bytes duplicate_warnings <- lifetime
allocation_count resident_bytes <- live gauges
- #356 (on main) resets the two gauges and keeps all five counters;
- this PR zeroes the whole struct and restores three — so
materialized_bytesandduplicate_warningsare lost on everyclear().
Merged as it stands, it would undo part of #356 — the same gauge/counter distinction that PR was written to establish, reintroduced in the field it did not list. And the conflict is the mechanical echo of the same thing: two PRs editing the same four lines.
Your own sentence is the correct reading of the state: "the cache was behaving correctly all along" — because #356 fixed it at 01:48. So there is nothing here to fix in host-memory.cpp; the branch simply predates the merge.
The ask
- Drop
ggml/src/ggml-hrx/runtime/host-memory.cppfrom this PR entirely — main's version is correct and is a strict superset of what this one does; - rebase, which resolves the
CONFLICTINGstate; - and the test changes stand —
6 passed, 2 failed→7 passed, 1 failedis real progress on engine#2393, and the remaining failure is the one that still needs its own treatment.
What I would keep from the description: "the failing assertion was the one after weights.clear()" — that is the diagnosis that led to #356 in the first place, and it is worth leaving in the history so the next person sees that the test was reporting a real disagreement rather than a kernel defect.
Nothing else changes. After the rebase this is a test-only PR, and a test-only PR on a suite with one remaining failure is exactly what ggml-org#2393 needs.
) host_weight_cache: HostWeightCache::clear() did 'stats_ = {}', wiping the CUMULATIVE hits/misses/evictions together with the residency fields. The struct already separates them (host-memory.h:73-81), and the case asserts both halves across a clear(): hits and misses must survive, allocation_count and resident_bytes must not. clear() now preserves the three cumulative counters. The cache itself was never wrong -- the failing assertion was the one AFTER clear(), and a stderr counter diagnostic (left in the test) showed hits=1/misses=3/allocations=3/resident=96 beforehand, exactly as expected. imported_buffer, alignment: ggml_backend_cpu_buffer_from_ptr asserts TENSOR_ALIGNMENT and the case passed a stack std::array<uint32_t,128>, aligned to 4. The test itself asserts get_base(imported) == source.data(), i.e. the pointer is used verbatim, so alignment is the caller's contract; fixed with alignas(TENSOR_ALIGNMENT) rather than by relaxing the assert. imported_buffer, buffer type: dropped REQUIRE(!ggml_backend_buffer_is_host(imported)). This backend keeps 'imported' and 'is_host' as INDEPENDENT flags, not exclusive ones (backend-buffer-binding.cpp:85 treats them as separate booleans, :111 accepts either as 'retained'), and the import path returns a CPU from-ptr buffer whose buft is retagged to mapped_buft, so it is legitimately host-typed. The contract the case can enforce -- the buffer is imported with no private storage and its base is the caller's pointer -- is asserted around it. Measured: 6 passed / 2 failed -> 7 passed / 1 failed (86-case box, gpu-hold ticket). The remaining failure is a code defect traced in the issue: model_buffer_from_mmap retags a CPU buffer as an HRX buft, so ggml_backend_hrx_buffer_context_from_buffer() reads a CPU context as an HRX one and context->imported reads false. Fixing that changes how every mapped model buffer is constructed and is left for its own change.
6a441dd to
57986bd
Compare
|
Rebased onto Base already carries two of the three thingsConflicting on // engine#2393: the live gauges follow the entries just dropped; hits, misses, evictions, materialized bytes and
// duplicate warnings are counts over the cache's life, which is what AMD's clear() keeps (9bf5871) and what
// tests/hrx-buffer-test.cpp asserts. The #327 merge had kept this branch's older `stats_ = {}`.
stats_.allocation_count = 0;
stats_.resident_bytes = 0;That is the same fix this PR made, arrived at from the other direction, and it cites this issue. So I resolved the conflict by taking base's version and dropping my hunk — two equivalent fixes for one bug is exactly the kind of duplicate that costs a reviewer an hour. Base also has an What the PR carries now
What is NOT in this PR, and is the real remaining defectBase still has the retagged-CPU-buffer import path: return ggml::hrx::model_buffer_from_mmap(device_context(device), ptr, size); // ggml-hrx.cppso |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- and "the cache was behaving correctly all along" is the right way to open a test fix
Two fixes, an honest count, and a measured delta: before: 6 passed, 2 failed, 8 total, after: 7 passed, 1 failed. Saying "two of the three" and leaving the third named is what a partial fix should look like -- a reader knows exactly what is left instead of discovering it in the next run.
Fix 1 is the distinction #356 introduced, now demonstrated: clear() wipes the live gauges and not the cumulative counters, and the assertion that failed was the one after weights.clear() -- so "the cache was behaving correctly all along" and the test was asserting the wrong contract. That is the better half of a test fix: the code did not change, and the case stopped lying about it.
Fix 2 is a design statement rather than a correction, and it is the part worth the review: dropping REQUIRE(!ggml_backend_buffer_is_host(imported)) because this backend keeps import and host-ness independent, with backend-buffer-binding.cpp:85 and :111 quoted as the authority. An assertion that encodes a relationship the implementation deliberately does not have is worse than no assertion -- it teaches the next person a rule that is not true.
Two asks, both small:
- Name the remaining failure and what it needs. The body says one is left; if it is
imported_buffer's alignment path (a stackstd::array<uint8_t>not being 32-byte aligned), the fix is to allocate it with an alignment guarantee -- and the case already knows the requirement, because it asserts it. If instead it needs a decision from ggml-org#2393, say so, so the next reader does not re-derive it. - The alignment fix deserves an arm in the other direction: a deliberately unaligned pointer must be refused by
ggml_backend_cpu_buffer_from_ptr(it asserts), so the case's fix is "pass an aligned one", not "loosen the assertion".
Nothing else: +18/-2, both changes in the test, and the quoted source lines make the second one checkable without running anything.
|
READY. Two separate fixes with different flavours, and both are the right kind of test repair — one adds a real precondition, the other removes an assertion that pinned an implementation detail. 1. 2. Dropping 3. The added The remaining failure (1 of 8) is left failing rather than labelled, and the body says which — engine#2393 tracks it. That is the correct choice here: unlike the One ask: the alignment comment explains why 32 matters but not where |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 57986bd: READY
Exactly what was asked for: host-memory.cpp is gone from the diff (0 files), the PR is tests/hrx-buffer-test.cpp alone, and it is MERGEABLE again. The rebase resolved the conflict and the duplicate fix went with it.
And what remains is two real defects in the test, neither of which needed production code touched:
1. A stack array that could not satisfy the contract it was passing through
alignas(TENSOR_ALIGNMENT) std::array<uint32_t, 128> source = {};"
ggml_backend_cpu_buffer_from_ptrrequiresTENSOR_ALIGNMENT(32) and a stackstd::arrayis [only] 4-byte aligned. The imported buffer hands this pointer to the backend verbatim, so the alignment is part of the contract the caller has to satisfy, not something the backend may pad away."
This is the imported_buffer failure from engine#2393's own body — GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0 …) — and the sentence is the whole reason it was a test defect: the test was passing a pointer that could never meet the requirement, so it was asserting something impossible and calling the abort a kernel failure.
2. A specification correction, which is the harder half to get right
"
importedandis_hostare INDEPENDENT flags in this backend, not exclusive ones —backend-buffer-binding.cpp:85keeps them as separate booleans … and:111accepts either as 'retained', so a zero-copy import of host memory is legitimately host-typed."
The old REQUIRE(!ggml_backend_buffer_is_host(imported)) encoded an exclusivity the code does not have, and the replacement says what the case can enforce (the buffer is an imported handle) plus the line references that show why the old assertion was wrong. That is a test being corrected by the implementation rather than around it — and it is the case where "fix the test" is right, because the code's behaviour is deliberate and documented in two places.
Both fixes are the kind that cannot be checked by reading the test alone: one needed the alignment requirement from the backend header, the other needed backend-buffer-binding.cpp:85's two booleans. The comments carry both, with line numbers, which is what makes the change reviewable now and defensible later.
Ready. 6 passed, 2 failed → 7 passed, 1 failed on engine#2393 with one failure left, and after this the PR is what it should be: the test tier only.
|
VERDICT: READY at Fix 1: a real alignment UB in the test, and the assert that enforces it- std::array<uint32_t, 128> source = {};
+ alignas(TENSOR_ALIGNMENT) std::array<uint32_t, 128> source = {};Verified: Fix 2: the assertion asserted exclusivity the backend does not haveThe PR cites The third failure is diagnosed, not deferred
Verified in the source: the retag is there at One note for the bodyThe PR says "two of the three" and the numbers (6 passed / 2 failed → 7 passed / 1 failed) make the scope legible. Adding one line — "the third is |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 57986bd — the blocking item is gone; here is what the row measures now, which is not what the title implies
The revert is fixed: known_fail references are 14 on the base and 14 on the branch (they were 14/0), and three dots and the endpoint now agree on one file, tests/hrx-buffer-test.cpp.
The suite at this head:
[FAIL] imported_buffer
HRX buffer test summary: 8 passed, 1 failed, 0 known-fail, 0 unexpected-pass, 9 total rc 1
tests/hrx-buffer-test.cpp:244: requirement failed: context->imported
That is progress, and it is not a green row. The two changes do what they say — alignas(TENSOR_ALIGNMENT) on the source array (which is what made ggml-backend.cpp:2705's alignment assert fire, per my engine#2393 report) and the removal of REQUIRE(!ggml_backend_buffer_is_host(imported)) with the is_imported/is_host independence cited correctly. The case therefore gets further than it ever has: it no longer aborts, and it now fails on a specific assertion at :244 — context->imported is false for the buffer that ggml_backend_cpu_buffer_from_ptr produced and that device_buffer_from_host_ptr retags.
Which is the design call, not this PR's subject. engine#357 said the imported_buffer half "needs a design decision"; context->imported is exactly that decision's surface — whether the retagged CPU from-pointer buffer carries an HRX context with imported = true. So the remaining red is the third half of engine#2393, and this PR's title ("fix two of the three") reads at this head as: host_weight_cache passes because of #356 and #364, and what this PR itself contributes is the alignment fix plus the assertion removal, which between them convert an abort into a named, specific failure.
For the body, one line worth adding so the next reader is not surprised: after the rebase the row is 8 passed, 1 failed, and the remaining failure is :244 context->imported — i.e. the same case, one assertion further along, and the assertion that belongs to the open design question rather than to this diff.
|
Ran it at Built at your head, one gpu-hold ticket, What your two changes actually did — both good, and both verified:
The remaining failure is the same kind of claim, kept a few lines down. And the implementation says so in its own comment ( A buffer that Two ways out, and I do not think it is mine to pick:
The distinction worth drawing explicitly, because it decides which: is Not a verdict — the head is minutes old and this lane's budget is one per head. If nothing changes by then I will verdict on this evidence. |
|
Verified at the merge commit Built at that head, one gpu-hold ticket, That matches what I measured at this PR's own head The fix that makes the case pass is llama.cpp#371, which is still open, and I measured it on the same tree:
So the ordering that landed is the one that removes assertions without moving the failure: Not arguing for a revert. The |
…hat argues for it (#377) #370 removed REQUIRE(!ggml_backend_buffer_is_host(imported)) with a comment explaining that `imported` and `is_host` are independent flags and a zero-copy import of host memory is legitimately host-typed. #371 settled the question the other way: the defect was a from-pointer buffer reaching that point as a CPU buffer with its `buft` retagged, which is exactly what made is_host true for something that was not a host allocation. The comment was never updated, so the merged tree still teaches the reading that produced the bug, in the file that regressed from it, and the assertion is gone. This restores both: - the rationale now cites backend-context.h's `imported_buft`, "a direct binding whose buffer must not claim to be a host allocation"; - REQUIRE(!ggml_backend_buffer_is_host(imported)) is back, naming what it catches: if an import ever becomes host-typed again, it went back through the retagging path that cost an afternoon. Verified at 7e91e98 before rebasing, one gpu-hold ticket, dmesg amdgpu resets 0 before / 0 after: hrx-buffer-test: 9 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 9 total so this restores a guard rather than surfacing a new failure. Co-authored-by: bcloud <agent@local>
Two of the three
hrx-buffer-testfailures, fixed and measured. Refs engine#2393.1.
host_weight_cache—clear()wiped the cumulative countersThe cache was behaving correctly all along. The failing assertion was the one after
weights.clear():clear()didstats_ = {};, wipinghits/misses/evictionstogether with the residency fields. The struct already separates them (host-memory.h:73-81: cumulative vs residency), soclear()now keeps the three cumulative counters and resets the rest.A stderr diagnostic printing the counters is left in the test: before the fix it printed
hits=1 misses=3 allocations=3 resident=96— exactly the expected values — which is what proved the counters were never the problem and pointed at the clear() instead. (Stderr on purpose:REQUIREaborts before stdout flushes.)2.
imported_buffer— two thingsAlignment.
ggml_backend_cpu_buffer_from_ptrasserts(uintptr_t)ptr % TENSOR_ALIGNMENT == 0; the case passed a stackstd::array<uint32_t,128>, aligned to 4. The test itself assertsget_base(imported) == source.data(), so the pointer is used verbatim and alignment is the caller's contract — fixed withalignas(TENSOR_ALIGNMENT)rather than by relaxing the assert (32, fromggml-impl.h:44, already included).Buffer type. Dropped
REQUIRE(!ggml_backend_buffer_is_host(imported)). This backend keeps the two notions independent rather than exclusive:and the import path returns a CPU from-ptr buffer whose
buftis retagged tomapped_buft(model-buffer-from-mmap.cpp:128), so it is legitimately host-typed. The contract this case can enforce — imported handle, no private storage, base is the caller's pointer — is asserted immediately around it.What is NOT fixed here, and why
imported_bufferstill fails, onREQUIRE(context->imported), and that is a code defect:The buffer is a CPU buffer whose
buftis retagged. Soggml_backend_hrx_buffer_context_from_buffer()casts a CPU context toggml_backend_hrx_buffer_context, andcontext->imported/context->bufferread fields of an unrelated struct — they are not facts about the import path at all.The fix is small and additive (
buffer_freealready handles an imported context:owned_buffer = !context->imported, and a nullcontext->bufferreleases nothing) — build a real imported HRX buffer context inggml-hrx.cppinstead of retagging — but it changes how every mapped model buffer is constructed, i.e. the loader path for all mmap-backed weights (engine ggml-org#856/ggml-org#2137 territory). That needs verification with a real model, not just this 8-case suite, so it is deliberately left out of this PR and documented on the issue.Verification
hrx-buffer-testunder agpu-holdticket withIREE_HAL_AMDGPU_LIBHSA_PATHpointing atlibhsa-runtime64.so.1andHSA_USE_SVM=0 HSA_USERPTR_FOR_PAGED_MEM=0 HRX_ALLOW_NO_SVM=1.BUILD_RC=0, checked separately from the run.