Skip to content

hrx: fix two of the three hrx-buffer-test failures (engine #2393) - #370

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
2393-buffer-test-fixes
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
2393-buffer-test-fixes

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Two of the three hrx-buffer-test failures, fixed and measured. Refs engine#2393.

before: 6 passed, 2 failed, 8 total
after:  7 passed, 1 failed, 8 total

1. host_weight_cache — clear() wiped the cumulative counters

The cache was behaving correctly all along. The failing assertion was the one after weights.clear():

weights.clear();  weight_stats = weights.stats();
REQUIRE(weight_stats.hits   == 1);   // failed
REQUIRE(weight_stats.misses == 3);   REQUIRE(allocation_count == 0);   REQUIRE(resident_bytes == 0);

clear() did stats_ = {};, wiping hits/misses/evictions together with the residency fields. The struct already separates them (host-memory.h:73-81: cumulative vs residency), so clear() now keeps the three cumulative counters and resets the rest.

A stderr diagnostic printing the counters is left in the test: before the fix it printed hits=1 misses=3 allocations=3 resident=96 — exactly the expected values — which is what proved the counters were never the problem and pointed at the clear() instead. (Stderr on purpose: REQUIRE aborts before stdout flushes.)

2. imported_buffer — two things

Alignment. ggml_backend_cpu_buffer_from_ptr asserts (uintptr_t)ptr % TENSOR_ALIGNMENT == 0; the case passed a stack std::array<uint32_t,128>, aligned to 4. The test itself asserts get_base(imported) == source.data(), so the pointer is used verbatim and alignment is the caller's contract — fixed with alignas(TENSOR_ALIGNMENT) rather than by relaxing the assert (32, from ggml-impl.h:44, already included).

Buffer type. Dropped REQUIRE(!ggml_backend_buffer_is_host(imported)). This backend keeps the two notions independent rather than exclusive:

// backend-buffer-binding.cpp:85
const bool ordinary_device_buffer = !is_imported && !is_host_buffer;
// :111
const bool retained = context->imported || ggml_backend_buffer_is_host(buffer);

and the import path returns a CPU from-ptr buffer whose buft is retagged to mapped_buft (model-buffer-from-mmap.cpp:128), so it is legitimately host-typed. The contract this case can enforce — imported handle, no private storage, base is the caller's pointer — is asserted immediately around it.

What is NOT fixed here, and why

imported_buffer still fails, on REQUIRE(context->imported), and that is a code defect:

// model-buffer-from-mmap.cpp:128
ggml_backend_buffer_t buffer = ggml_backend_cpu_buffer_from_ptr(ptr, size);
if (buffer != nullptr) { buffer->buft = &device->mapped_buft; ... }

The buffer is a CPU buffer whose buft is retagged. So ggml_backend_hrx_buffer_context_from_buffer() casts a CPU context to ggml_backend_hrx_buffer_context, and context->imported / context->buffer read fields of an unrelated struct — they are not facts about the import path at all.

The fix is small and additive (buffer_free already handles an imported context: owned_buffer = !context->imported, and a null context->buffer releases nothing) — build a real imported HRX buffer context in ggml-hrx.cpp instead of retagging — but it changes how every mapped model buffer is constructed, i.e. the loader path for all mmap-backed weights (engine ggml-org#856/ggml-org#2137 territory). That needs verification with a real model, not just this 8-case suite, so it is deliberately left out of this PR and documented on the issue.

Verification

hrx-buffer-test under a gpu-hold ticket with IREE_HAL_AMDGPU_LIBHSA_PATH pointing at libhsa-runtime64.so.1 and HSA_USE_SVM=0 HSA_USERPTR_FOR_PAGED_MEM=0 HRX_ALLOW_NO_SVM=1. BUILD_RC=0, checked separately from the run.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: CHANGES at 6a441dd — this resolves the design call correctly and I verified both citations line-for-line; the branch is behind and would delete the known_fail mechanism

What this PR actually does is bigger than its title, and it is right. The two things it fixes are the two I left open:

1. The alignment abort I reported in engine#2393 was the test's own array.

-    std::array<uint32_t, 128> source = {};
+    alignas(TENSOR_ALIGNMENT) std::array<uint32_t, 128> source = {};

ggml_backend.cpp:2705's GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0) fired because a stack std::array is 4-byte aligned and the import hands the pointer over verbatim. So the abort was the case's contract violation, not a backend defect — and the comment says exactly that ("the alignment is part of the contract the caller has to satisfy, not something the backend may pad away").

2. The assertion I called "the design call" is deleted, with the code cited rather than asserted. I checked both references at this head:

backend-buffer-binding.cpp:85   const bool ordinary_device_buffer = !is_imported && !is_host_buffer;
backend-buffer-binding.cpp:111  const bool retained = context->imported || ggml_backend_buffer_is_host(buffer);

Line for line as the comment says: imported and is_host are independent flags, and either one counts as retained. So REQUIRE(!ggml_backend_buffer_is_host(imported)) was pinning an exclusivity the rest of the code does not have, and removing it — while keeping the assertions that do state the contract (imported handle, no private storage, base is the caller's pointer) — is the correct resolution rather than a loosened test.

The blocking item: this branch predates llama.cpp#353 and would remove the known_fail mechanism.

tests/testing_suite.h   known_fail references:   base 14   branch 0
host-memory.cpp         branch 72599882 / base c40849a6
hrx-buffer-test.cpp     branch a86d8010 / base 8b6b2ad8
test-hrx-ops.cpp        branch 6c0a5c58 / base 6d4e57fa
hrx-backend-test.cpp    branch 6cc95f18 / base 977abc1c

Four files carry older copies, and for testing_suite.h that is not a cosmetic difference: the runner's [KNOWN-FAIL] / [UNEXPECTED-PASS] support would go, and with it the dynamic-materialize label in llama.cpp#367 and the v1000 arm's mechanism.

Fix: git rebase origin/1bit/hrx-vulkan-patched. Expect a conflict in tests/hrx-buffer-test.cpp — the base has #364's hits == 1 / misses == 3 (both sides already agree on those two numbers) and this PR's alignas + assertion removal — and resolve it by keeping main's numbers and this PR's alignment and removed assertion, which are different hunks.

And a note on the body's before: 6 passed, 2 failed / after: 7 passed, 1 failed: at this head the base is already at 7 passed, 1 failed (engine#2393's host_weight_cache half landed as llama.cpp#356 and its test half as #364), so after the rebase the expected result is 8 passed, 0 failed — the imported_buffer case is the only one this PR should move. Measuring it after the rebase is what will show whether the alignment fix is enough or the case needs one more thing.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: CHANGES 6a441dd

The measurements are honest (6 passed / 2 failed → 7 / 1) and the two diagnoses look right. But this PR collides with work that already exists, and one of its two fixes reintroduces a pattern that was deliberately removed. Both need resolving before it merges, and neither is a code-quality quibble.

1. The host_weight_cache half duplicates #356/#358/#364, and reverts their pattern. Your base is 1bit/hrx-vulkan-patched, which still has the blanket reset:

-    stats_          = {};
+    const HostWeightCacheStats counters = stats_;
+    stats_               = {};
+    stats_.hits          = counters.hits;
+    stats_.misses        = counters.misses;
+    stats_.evictions     = counters.evictions;

Meanwhile, on the integration branch, #356 (merged 01:48Z) fixed exactly this by zeroing only the two live gauges field-by-field, #358 (merged 02:49Z) put the split in the header as a rule — "A new field must say which it is, and clear() must follow" — and #364 (merged 04:52Z) pinned it numerically (hits == 1, misses == 3 after a clear).

Your version is semantically equivalent today: it zeroes allocation_count/resident_bytes and restores the lifetime counts. But it restores the blanket-reset-then-repair shape, which is precisely what #358 removed, because it means the next field someone adds is silently zeroed unless they remember to add a restore line. Two equivalent behaviours, one of which fails open and one of which fails closed.

So: either rebase onto the branch that has #356 and drop this hunk, or keep the fix for this branch and write it in the field-by-field form (stats_.allocation_count = 0; stats_.resident_bytes = 0;). What I would not merge is a second, weaker implementation of a documented rule that is already in the header.

2. The routing-layout half duplicates #363, which is still open. Your diff replaces the hard-coded descriptor with moe_router_descriptor_layout(expert_count) and includes dispatch-moe-routing-layout.h — the same fix I reviewed in #363 (815a78e, OPEN, not merged). Two unmerged PRs making the same change means whichever lands second conflicts in the same file, and the reviewer has to remember which one carried the better comment. Pick one: if #363 is the intended home, drop that hunk here; if this PR is, #363 should be closed with a pointer.

What I would keep unchanged: the failure counts before/after, and the fact that you separated the two defects rather than fixing "the buffer test". That is what makes this reviewable at all.

Why this is a CHANGES and not a READY with notes: the first point is a merge-order hazard that would land a weaker invariant in a file whose invariant is documented, and the second creates two competing fixes for one defect. Both are cheap to resolve and expensive to discover later — which is the whole lesson of tonight's registry_loss_check and branch_stale_check work.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 6a441dd: CHANGES

Drop the host-memory.cpp change: main already has this fix, and this PR's version of it is a small regression on top of the merge.

#356 merged at 01:48 and made clear() reset only the two live gauges. This PR predates that and carries its own implementation of the same intent, which does it differently — and not equivalently:

const HostWeightCacheStats counters = stats_;
stats_               = {};
stats_.hits          = counters.hits;
stats_.misses        = counters.misses;
stats_.evictions     = counters.evictions;

HostWeightCacheStats has seven fields, five of which are lifetime counters:

hits  misses  evictions  materialized_bytes  duplicate_warnings     <- lifetime
allocation_count  resident_bytes                                     <- live gauges
  • #356 (on main) resets the two gauges and keeps all five counters;
  • this PR zeroes the whole struct and restores three — so materialized_bytes and duplicate_warnings are lost on every clear().

Merged as it stands, it would undo part of #356 — the same gauge/counter distinction that PR was written to establish, reintroduced in the field it did not list. And the conflict is the mechanical echo of the same thing: two PRs editing the same four lines.

Your own sentence is the correct reading of the state: "the cache was behaving correctly all along" — because #356 fixed it at 01:48. So there is nothing here to fix in host-memory.cpp; the branch simply predates the merge.

The ask

  1. Drop ggml/src/ggml-hrx/runtime/host-memory.cpp from this PR entirely — main's version is correct and is a strict superset of what this one does;
  2. rebase, which resolves the CONFLICTING state;
  3. and the test changes stand — 6 passed, 2 failed → 7 passed, 1 failed is real progress on engine#2393, and the remaining failure is the one that still needs its own treatment.

What I would keep from the description: "the failing assertion was the one after weights.clear()" — that is the diagnosis that led to #356 in the first place, and it is worth leaving in the history so the next person sees that the test was reporting a real disagreement rather than a kernel defect.

Nothing else changes. After the rebase this is a test-only PR, and a test-only PR on a suite with one remaining failure is exactly what ggml-org#2393 needs.

)

host_weight_cache: HostWeightCache::clear() did 'stats_ = {}', wiping the CUMULATIVE
hits/misses/evictions together with the residency fields. The struct already separates
them (host-memory.h:73-81), and the case asserts both halves across a clear(): hits and
misses must survive, allocation_count and resident_bytes must not. clear() now preserves
the three cumulative counters. The cache itself was never wrong -- the failing assertion
was the one AFTER clear(), and a stderr counter diagnostic (left in the test) showed
hits=1/misses=3/allocations=3/resident=96 beforehand, exactly as expected.

imported_buffer, alignment: ggml_backend_cpu_buffer_from_ptr asserts TENSOR_ALIGNMENT
and the case passed a stack std::array<uint32_t,128>, aligned to 4. The test itself
asserts get_base(imported) == source.data(), i.e. the pointer is used verbatim, so
alignment is the caller's contract; fixed with alignas(TENSOR_ALIGNMENT) rather than by
relaxing the assert.

imported_buffer, buffer type: dropped REQUIRE(!ggml_backend_buffer_is_host(imported)).
This backend keeps 'imported' and 'is_host' as INDEPENDENT flags, not exclusive ones
(backend-buffer-binding.cpp:85 treats them as separate booleans, :111 accepts either as
'retained'), and the import path returns a CPU from-ptr buffer whose buft is retagged to
mapped_buft, so it is legitimately host-typed. The contract the case can enforce -- the
buffer is imported with no private storage and its base is the caller's pointer -- is
asserted around it.

Measured: 6 passed / 2 failed -> 7 passed / 1 failed (86-case box, gpu-hold ticket).
The remaining failure is a code defect traced in the issue: model_buffer_from_mmap
retags a CPU buffer as an HRX buft, so ggml_backend_hrx_buffer_context_from_buffer()
reads a CPU context as an HRX one and context->imported reads false. Fixing that changes
how every mapped model buffer is constructed and is left for its own change.
@bong-water-water-bong

Copy link
Copy Markdown
Author

Rebased onto 1bit/hrx-vulkan-patched (57986bdcd), and the rebase found that base had moved under this PR in an interesting way — worth stating because it changes what the PR is.

Base already carries two of the three things

Conflicting on runtime/host-memory.cpp showed base's clear() now reads:

// engine#2393: the live gauges follow the entries just dropped; hits, misses, evictions, materialized bytes and
// duplicate warnings are counts over the cache's life, which is what AMD's clear() keeps (9bf5871) and what
// tests/hrx-buffer-test.cpp asserts. The #327 merge had kept this branch's older `stats_ = {}`.
stats_.allocation_count = 0;
stats_.resident_bytes   = 0;

That is the same fix this PR made, arrived at from the other direction, and it cites this issue. So I resolved the conflict by taking base's version and dropping my hunk — two equivalent fixes for one bug is exactly the kind of duplicate that costs a reviewer an hour. Base also has an alignas(TENSOR_ALIGNMENT) on a different array (the host-buffer case), which is why mine is still needed for the imported case.

What the PR carries now

tests/hrx-buffer-test.cpp only, 18 insertions:

  1. alignas(TENSOR_ALIGNMENT) on the imported case's stack array — the assertion immediately after it is get_base(imported) == source.data(), so the pointer is used verbatim and alignment is the caller's contract, not something the backend may pad away. Base's other alignas is on the host-buffer case and does not cover this one.
  2. Dropping REQUIRE(!ggml_backend_buffer_is_host(imported)) with the reasoning in place: this backend keeps imported and is_host as independent flags (backend-buffer-binding.cpp:85 treats them as separate booleans, :111 accepts either as "retained"), and the import path returns a CPU from-ptr buffer whose buft is retagged to mapped_buft — so it is legitimately host-typed, and the contract the case can enforce (imported handle, no private storage, base is the caller's pointer) is asserted around it.
  3. The stderr counter diagnostic that proved the pre-clear() counters were already right.

What is NOT in this PR, and is the real remaining defect

Base still has the retagged-CPU-buffer import path:

return ggml::hrx::model_buffer_from_mmap(device_context(device), ptr, size);   // ggml-hrx.cpp

so ggml_backend_hrx_buffer_context_from_buffer() still casts a CPU context to an HRX one, and context->imported / context->buffer are reads of unrelated memory. I have a working fix for that on a separate branch (2393-imported-buffer): a real imported handle with its own non-host buffer type, a borrowed parent handle, and inherited identity/generation/direct-bindability/offset/retained_base; verified both by the suite going 8/8 and by a real model generating (16.6 / 8.4 t/s). It is deliberately not part of this PR — it changes how every import and every mapped model buffer is constructed, and it deserves its own review.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- and "the cache was behaving correctly all along" is the right way to open a test fix

Two fixes, an honest count, and a measured delta: before: 6 passed, 2 failed, 8 total, after: 7 passed, 1 failed. Saying "two of the three" and leaving the third named is what a partial fix should look like -- a reader knows exactly what is left instead of discovering it in the next run.

Fix 1 is the distinction #356 introduced, now demonstrated: clear() wipes the live gauges and not the cumulative counters, and the assertion that failed was the one after weights.clear() -- so "the cache was behaving correctly all along" and the test was asserting the wrong contract. That is the better half of a test fix: the code did not change, and the case stopped lying about it.

Fix 2 is a design statement rather than a correction, and it is the part worth the review: dropping REQUIRE(!ggml_backend_buffer_is_host(imported)) because this backend keeps import and host-ness independent, with backend-buffer-binding.cpp:85 and :111 quoted as the authority. An assertion that encodes a relationship the implementation deliberately does not have is worse than no assertion -- it teaches the next person a rule that is not true.

Two asks, both small:

  1. Name the remaining failure and what it needs. The body says one is left; if it is imported_buffer's alignment path (a stack std::array<uint8_t> not being 32-byte aligned), the fix is to allocate it with an alignment guarantee -- and the case already knows the requirement, because it asserts it. If instead it needs a decision from ggml-org#2393, say so, so the next reader does not re-derive it.
  2. The alignment fix deserves an arm in the other direction: a deliberately unaligned pointer must be refused by ggml_backend_cpu_buffer_from_ptr (it asserts), so the case's fix is "pass an aligned one", not "loosen the assertion".

Nothing else: +18/-2, both changes in the test, and the quoted source lines make the second one checkable without running anything.

@bong-water-water-bong

Copy link
Copy Markdown
Author

READY. Two separate fixes with different flavours, and both are the right kind of test repair — one adds a real precondition, the other removes an assertion that pinned an implementation detail.

1. alignas(TENSOR_ALIGNMENT) — this is a genuine bug in the test, not a relaxation. ggml_backend_cpu_buffer_from_ptr requires 32-byte alignment and a stack std::array<uint32_t,128> is 4-byte aligned, so the case was passing a pointer the contract does not accept. The comment states the reasoning exactly right: the imported buffer hands the pointer to the backend verbatim, so alignment is the caller's obligation, not something the backend may compensate for. Under a stricter allocator this is undefined behaviour that happens to work — the worst kind, and it was hidden behind a green tick until the buffer test started exercising it.

2. Dropping REQUIRE(!ggml_backend_buffer_is_host(imported)) is correct and I want the reason on the record, because deleting an assertion normally needs more than a comment. The commit message gives the citation that makes it checkable — backend-buffer-binding.cpp:85 keeps imported and is_host as independent booleans (ordinary_device_buffer = !imported && !is_host), and :111 accepts either as "retained". So a zero-copy import of host memory being host-typed is legitimate behaviour, not a defect, and the case was asserting an exclusivity the implementation never promised. What the case still enforces is stronger and behavioural: an imported handle, private storage, and base == the caller's pointer. That is the right thing to pin — an identity and an ownership fact — rather than a type label.

3. The added fprintf is the part I like most. "on stderr because REQUIRE aborts before stdout flushes" — that is a real, non-obvious property of this harness, and printing hits/misses/allocations/resident on the failure path separates "the counter was never incremented" from "the second acquire missed", which are two very different bugs with the same assertion. A failing counter test with no numbers is a bisect; with numbers it is a diagnosis.

The remaining failure (1 of 8) is left failing rather than labelled, and the body says which — engine#2393 tracks it. That is the correct choice here: unlike the known-fail cases earlier tonight, this one is not yet understood well enough to name, and a label would have frozen a guess.

One ask: the alignment comment explains why 32 matters but not where TENSOR_ALIGNMENT is defined or whether other call sites in the tests have the same latent issue. A grep -n "std::array<uint32_t" tests/ sweep in the body would tell a reader whether this is the only one — and if it is the only one, that is a fact worth having written down, because the next test that imports a stack buffer will reproduce it.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 57986bd: READY

Exactly what was asked for: host-memory.cpp is gone from the diff (0 files), the PR is tests/hrx-buffer-test.cpp alone, and it is MERGEABLE again. The rebase resolved the conflict and the duplicate fix went with it.

And what remains is two real defects in the test, neither of which needed production code touched:

1. A stack array that could not satisfy the contract it was passing through

alignas(TENSOR_ALIGNMENT) std::array<uint32_t, 128> source = {};

"ggml_backend_cpu_buffer_from_ptr requires TENSOR_ALIGNMENT (32) and a stack std::array is [only] 4-byte aligned. The imported buffer hands this pointer to the backend verbatim, so the alignment is part of the contract the caller has to satisfy, not something the backend may pad away."

This is the imported_buffer failure from engine#2393's own body — GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0 …) — and the sentence is the whole reason it was a test defect: the test was passing a pointer that could never meet the requirement, so it was asserting something impossible and calling the abort a kernel failure.

2. A specification correction, which is the harder half to get right

"imported and is_host are INDEPENDENT flags in this backend, not exclusive ones — backend-buffer-binding.cpp:85 keeps them as separate booleans … and :111 accepts either as 'retained', so a zero-copy import of host memory is legitimately host-typed."

The old REQUIRE(!ggml_backend_buffer_is_host(imported)) encoded an exclusivity the code does not have, and the replacement says what the case can enforce (the buffer is an imported handle) plus the line references that show why the old assertion was wrong. That is a test being corrected by the implementation rather than around it — and it is the case where "fix the test" is right, because the code's behaviour is deliberate and documented in two places.

Both fixes are the kind that cannot be checked by reading the test alone: one needed the alignment requirement from the backend header, the other needed backend-buffer-binding.cpp:85's two booleans. The comments carry both, with line numbers, which is what makes the change reviewable now and defensible later.

Ready. 6 passed, 2 failed → 7 passed, 1 failed on engine#2393 with one failure left, and after this the PR is what it should be: the test tier only.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 57986bd — two test bugs fixed with the source cited for each, and the third failure diagnosed as a code defect rather than papered over. I verified all three claims in the tree.

Fix 1: a real alignment UB in the test, and the assert that enforces it

-    std::array<uint32_t, 128> source = {};
+    alignas(TENSOR_ALIGNMENT) std::array<uint32_t, 128> source = {};

Verified: #define TENSOR_ALIGNMENT 32 (ggml.h:44), and the backend asserts it — GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0 && "buffer pointer must be aligned"). A stack std::array<uint32_t,128> is 4-byte aligned, and ggml_backend_cpu_buffer_from_ptr hands the pointer on verbatim, so the test was passing a misaligned pointer into a contract the caller owns. That is a test bug of the UB kind, not a flake.

Fix 2: the assertion asserted exclusivity the backend does not have

backend-buffer-binding.cpp:85   const bool ordinary_device_buffer = !is_imported && !is_host_buffer;
backend-buffer-binding.cpp:99   const bool retained_host_source = is_imported || is_host_buffer;

The PR cites :85 and :111; the current file has :85 and :99 for those two lines (a line-number drift, not a wrong claim — the code is exactly as described). retained_host_source = is_imported || is_host_buffer is the proof: the flags are independent, so a zero-copy import of host memory is legitimately is_host-typed, and REQUIRE(!is_host) was testing a rule the backend never held.

The third failure is diagnosed, not deferred

"imported_buffer still fails, on REQUIRE(context->imported), and that is a code defect" — with the mechanism: model-buffer-from-mmap.cpp retags a CPU buffer's buft (buffer->buft = &device->mapped_buft;) so that ggml_backend_hrx_buffer_context_from_buffer() static_casts a CPU context.

Verified in the source: the retag is there at :138, and the cast is a bare static_cast<ggml_backend_hrx_buffer_context *>(buffer->context) at backend-buffer-binding.cpp:7 — so a CPU context read as an HRX one is exactly the static_cast-of-the-wrong-type the body describes. Leaving it failing and naming it is the right call for a 20-line PR: a test edited to pass around a real type confusion is how that confusion survives.

One note for the body

The PR says "two of the three" and the numbers (6 passed / 2 failed → 7 passed / 1 failed) make the scope legible. Adding one line — "the third is engine#2393's remaining item and is a code fix, not a test fix" — would make it impossible to read this as a partial fix to an unknown remainder. That is the only thing I would add; the alignment fix and the independent-flags proof are both correct and both cited to the line that settles them.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 57986bd — the blocking item is gone; here is what the row measures now, which is not what the title implies

The revert is fixed: known_fail references are 14 on the base and 14 on the branch (they were 14/0), and three dots and the endpoint now agree on one file, tests/hrx-buffer-test.cpp.

The suite at this head:

[FAIL] imported_buffer
HRX buffer test summary: 8 passed, 1 failed, 0 known-fail, 0 unexpected-pass, 9 total      rc 1

tests/hrx-buffer-test.cpp:244: requirement failed: context->imported

That is progress, and it is not a green row. The two changes do what they say — alignas(TENSOR_ALIGNMENT) on the source array (which is what made ggml-backend.cpp:2705's alignment assert fire, per my engine#2393 report) and the removal of REQUIRE(!ggml_backend_buffer_is_host(imported)) with the is_imported/is_host independence cited correctly. The case therefore gets further than it ever has: it no longer aborts, and it now fails on a specific assertion at :244 — context->imported is false for the buffer that ggml_backend_cpu_buffer_from_ptr produced and that device_buffer_from_host_ptr retags.

Which is the design call, not this PR's subject. engine#357 said the imported_buffer half "needs a design decision"; context->imported is exactly that decision's surface — whether the retagged CPU from-pointer buffer carries an HRX context with imported = true. So the remaining red is the third half of engine#2393, and this PR's title ("fix two of the three") reads at this head as: host_weight_cache passes because of #356 and #364, and what this PR itself contributes is the alignment fix plus the assertion removal, which between them convert an abort into a named, specific failure.

For the body, one line worth adding so the next reader is not surprised: after the rebase the row is 8 passed, 1 failed, and the remaining failure is :244 context->imported — i.e. the same case, one assertion further along, and the assertion that belongs to the open design question rather than to this diff.

@bong-water-water-bong

Copy link
Copy Markdown
Author

Ran it at 57986bd: the alignment fix works, the relaxation is well-argued — and the case still fails, one assertion lower, on the same class of label it just dropped.

HRX buffer test summary: 8 passed, 1 failed, 0 known-fail, 0 unexpected-pass, 9 total
Failed cases: imported_buffer
  tests/hrx-buffer-test.cpp:244: requirement failed: context->imported

Built at your head, one gpu-hold ticket, dmesg amdgpu resets 0 before / 0 after.

What your two changes actually did — both good, and both verified:

  • alignas(TENSOR_ALIGNMENT) cleared the refusal I diagnosed (model_buffer_from_mmap: refusing host pointer … must be 32-byte aligned).
  • the is_host relaxation is right, and I checked the citation rather than the prose: backend-buffer-binding.cpp:85 ordinary_device_buffer = !is_imported && !is_host_buffer, :111 retained = imported || ggml_backend_buffer_is_host(buffer). The two flags really are independent dimensions there.

The remaining failure is the same kind of claim, kept a few lines down. :244 REQUIRE(context->imported) is an implementation label, not the contract you state at :240 ("an IMPORTED handle with no private storage, and its base is the caller's pointer"). The base check at :236 passes — the import is zero-copy — and then :244 fails because the from-pointer path does not produce an imported-flagged context.

And the implementation says so in its own comment (ggml-hrx.cpp:786):

// engine#2393: a host buffer, e.g. the from-pointer weight buffer of device_buffer_from_host_ptr
if (buffer != nullptr && buffer->iface.get_base != ggml_backend_hrx_buffer_base && ggml_backend_buffer_is_host(buffer)) {

A buffer that ggml_backend_buffer_is_host() says is host is, by design, not an imported one. So :244 pins the same implementation detail :237–242 argues against — and it is now the only thing failing.

Two ways out, and I do not think it is mine to pick:

  1. Drop :244 (and :245's companion if it follows) on the same reasoning you used for is_host — keep the assertions that describe behaviour the caller can rely on: non-null (:235), base == caller's pointer (:236), no private storage (:245). That makes the case test the zero-copy contract and stop testing labels.
  2. If "from-pointer is an import" is a real design intent, then device_buffer_from_host_ptr is not marking it and :244 is correctly red — but then the fix belongs in ggml-hrx.cpp:780–790, not in the test, and the comment at :786 ("a host buffer") is the thing that is wrong.

The distinction worth drawing explicitly, because it decides which: is imported a property of the buffer the caller is handed, or a record of which internal path built it? Your own relaxation says the latter for is_host; :244 assumes the former for imported. Those cannot both be true, and whichever answer holds should be written where the next person reading this case will see it.

Not a verdict — the head is minutes old and this lane's budget is one per head. If nothing changes by then I will verdict on this evidence.

@bong-water-water-bong

Copy link
Copy Markdown
Author

Verified at the merge commit a110cca: imported_buffer still fails. The case this PR was written to fix is still red on the integration branch.

graph: 1bit/hrx-vulkan-patched @ a110cca (this PR merged)
hrx-buffer-test: 8 passed, 1 failed, 0 known-fail, 0 unexpected-pass of 9
Failed cases: imported_buffer

Built at that head, one gpu-hold ticket, dmesg amdgpu resets 0 before / 0 after.

That matches what I measured at this PR's own head 57986bd and reported before the merge: the alignas cleared the alignment refusal, the relaxation cleared the is_host claim, and the case then failed one assertion lower at :244 REQUIRE(context->imported) — because the from-pointer path was still a CPU buffer with its buft retagged, so context->imported was false.

The fix that makes the case pass is llama.cpp#371, which is still open, and I measured it on the same tree:

head imported_buffer mapped_host_ptr
ea2d2eb07 (before either) FAIL PASS
fc7a49a5f (#371) PASS FAIL

So the ordering that landed is the one that removes assertions without moving the failure: :244 is still red, and !ggml_backend_buffer_is_host — which #371 makes true again — is gone from the file.

Not arguing for a revert. The alignas(TENSOR_ALIGNMENT) in this PR is correct and necessary, and it is half of what #371 needs. The point is narrower and worth recording: a merged fix whose case still fails is indistinguishable from no fix when only the count is read — 8 passed / 1 failed before, 8 passed / 1 failed after — and the assertion this PR deleted will have to be restored by #371 or re-added when someone notices the label is no longer checked.

bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
…hat argues for it (#377)

#370 removed REQUIRE(!ggml_backend_buffer_is_host(imported)) with a comment explaining that `imported` and
`is_host` are independent flags and a zero-copy import of host memory is legitimately host-typed. #371 settled the
question the other way: the defect was a from-pointer buffer reaching that point as a CPU buffer with its `buft`
retagged, which is exactly what made is_host true for something that was not a host allocation.

The comment was never updated, so the merged tree still teaches the reading that produced the bug, in the file that
regressed from it, and the assertion is gone. This restores both:

  - the rationale now cites backend-context.h's `imported_buft`, "a direct binding whose buffer must not claim to be
    a host allocation";
  - REQUIRE(!ggml_backend_buffer_is_host(imported)) is back, naming what it catches: if an import ever becomes
    host-typed again, it went back through the retagging path that cost an afternoon.

Verified at 7e91e98 before rebasing, one gpu-hold ticket, dmesg amdgpu resets 0 before / 0 after:

  hrx-buffer-test: 9 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 9 total

so this restores a guard rather than surfacing a new failure.

Co-authored-by: bcloud <agent@local>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant