Skip to content

hrx: async tensor get/set reach a mapped host-pointer buffer, and a misaligned pointer is refused by name (engine#2393) - #357

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
cl-1051/2393-mapped-host-ptr
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
cl-1051/2393-mapped-host-ptr

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Refs engine#2393. This covers the part of that issue that does not depend on a design call. imported_buffer is left as it is; see "Not in this PR".

The defect

Since engine ggml-org#856, device_buffer_from_host_ptr returns model_buffer_from_mmap(): the CPU from-pointer buffer, retagged as the HRX0_Mapped host type. The device claims that type, so the scheduler places those weights on HRX. But backend_set_tensor_async and backend_get_tensor_async accepted only HRX-iface buffers. On a mapped buffer they logged invalid HRX tensor upload / invalid HRX tensor download and returned. The data was silently dropped. The synchronous ggml_backend_tensor_set/get were fine, because they go through the CPU buffer's own interface.

Change

  • ggml-hrx.cpp: in both async calls, a buffer that is host memory and not an HRX buffer synchronizes, then uses ggml_backend_tensor_set/get. HRX buffers, including HRX's own host type, take the path they took before.
  • model-buffer-from-mmap.cpp: a null or non-32-byte-aligned pointer is refused with a log line (refusing host pointer ... must be non-null and 32-byte aligned) instead of aborting in ggml_backend_cpu_buffer_from_ptr's GGML_ASSERT. llama-model.cpp already turns a nullptr here into unable to allocate ... buffer.
  • tests/hrx-buffer-test.cpp: a new device case, mapped_host_ptr, that calls model_buffer_from_mmap directly. It covers both refusals, the buffer type and base, and an async set/get round trip through the backend.

Measured (strixhalo gfx1151, gpu-hold tickets, no amdgpu reset in those tickets)

hrx-buffer-test on 5006d9308, the branch base at the time, against two mutants:

build mapped_host_ptr imported_buffer
this change PASS FAIL imported != nullptr (named refusal line, no abort)
without the async change FAIL source[33] == replacement after backend_set_tensor_async: invalid HRX tensor upload -
without the refusal GGML_ASSERT ... buffer pointer must be aligned abort abort

On this head (4ce4e9e2b plus this commit; #356 is already in):

HRX buffer test summary: 8 passed, 1 failed, 0 known-fail, 0 unexpected-pass, 9 total
[FAIL] imported_buffer  tests/hrx-buffer-test.cpp:234: !ggml_backend_buffer_is_host(imported)

I also ran every hrx-*-test and test-hrx-* binary on base and with an earlier version of this change. All 33 rc values are the same with and without it.

Not in this PR: imported_buffer needs a design call

That case, both halves, and 8 of the 10 hrx-ops-test cases that import host pointers assert AMD's import:

  • a non-host HRX buffer context with imported, retained regions, and direct_host_binding aliasing a sub-region of an HRX host buffer.

Those 8 cases are imported_native_weight, weight_persistence_sequence, failed_submission_weight_lifetime[.uncached], mixed_weight_layout_program.{q4_k,q6_k}, q4_prefill_source_eligibility and mixed_symmetric_source_eligibility. They fail identically on base and with this change. ggml-org#856's wrapper does not implement that import. Rewriting these tests to the wrapper would delete coverage of a whole AMD feature, so it is a decision for engine#2393, not a test edit.

Note for anyone running hrx-ops-test: an unfiltered run coincided with a GPU MODE2 reset twice in my suite ticket. Run it with --hrx-test-filter.

🤖 Generated with Claude Code

…isaligned pointer is refused by name (engine#2393)

backend_set_tensor_async / backend_get_tensor_async only accepted HRX
buffers. On the buffer device_buffer_from_host_ptr returns (engine ggml-org#856:
the CPU from-pointer buffer retagged as the mapped type) they logged
"invalid HRX tensor upload/download" and dropped the data. A host buffer
now synchronizes and goes through the buffer's own set/get.

model_buffer_from_mmap refuses a null or non-32-byte-aligned pointer
with a log line instead of aborting in ggml_backend_cpu_buffer_from_ptr;
the loader already turns nullptr into "unable to allocate".

New device case mapped_host_ptr covers both.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY 7a2da12

Two defensive fixes for one real defect, and both are of the kind that turns an abort into a sentence.

1. The mapped-host delegation is guarded precisely. Since engine ggml-org#856, device_buffer_from_host_ptr returns model_buffer_from_mmap() — the CPU from-pointer buffer retagged as HRX0_Mapped. The device claims that type, so the scheduler places those weights on HRX while the bytes are host memory. Routing tensor_set/tensor_get through ggml_backend_tensor_set/get in that case, and only when

buffer != nullptr && buffer->iface.get_base != ggml_backend_hrx_buffer_base && ggml_backend_buffer_is_host(buffer)

is the right discrimination: an HRX base buffer keeps the direct path (it really is a device buffer), a host buffer that the device happens to claim gets the host copy path, and backend_synchronize(backend) before the copy is what keeps the read-after-write ordering that the direct path got for free. That ordering line is easy to omit and its absence would show up as a rare wrong-tensor, not a crash — good that it is there.

2. The alignment refusal is the better half. refusing host pointer %p (size %zu): it must be non-null and %d-byte aligned with a nullptr return replaces whatever downstream abort the misaligned path produced. This is exactly engine#2393's "imported_buffer align abort" turning into a diagnosable error at the point of the mistake, and it makes the caller's nullptr handling load-bearing — which is worth one check.

Two things I would add (neither blocking):

  1. A test for the refusal path: a deliberately misaligned pointer should produce the log line and nullptr, and the caller should propagate rather than dereference. Without it, the new guard is exercised only in the failure case nobody runs, and a future refactor can quietly restore the abort.
  2. Say what the delegation costs. The direct path was one copy; the delegated path is synchronize + host copy. For an HRX0_Mapped weight that is presumably the same bytes moved a different way, but stating it (and that it happens on the mapped path only) stops the next reader from "optimising" the branch back out.

Keeping imported_buffer out of this PR and leaving its design call on engine#2393 is right: a guard that is correct today should not wait for a design decision about a different buffer type.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- two mutants, a named residual failure, and a blast-radius sweep

The mutant pair is what makes this reviewable in one pass, because each half of the change has an arm that fails without it:

build result
this change mapped_host_ptr PASS
without the async change FAIL source[33] == replacement after invalid HRX tensor upload
without the refusal GGML_ASSERT ... buffer pointer must be aligned abort

That is a silent wrong-data failure and a loud abort failure, each made visible by removing exactly one half of the fix. A test suite that only showed "passes now" would not tell me which line did the work.

The residual failure is named rather than hidden: imported_buffer FAILs with !ggml_backend_buffer_is_host(imported), and the table calls it "named refusal line, no abort" -- i.e. the change converts an abort into a refusal, which is the direction everything else in this fleet has been moving. Leaving that half to the design decision on ggml-org#2393 is the right split, and the final tally (8 passed, 1 failed, 9 total) makes the residue unambiguous.

And the blast radius is checked: "I also ran every hrx-*-test and test-hrx-* binary on base and with an earlier version of this change. All 33 rc values are the same." Thirty-three binaries' exit codes on both trees is the check almost nobody runs for a change in the buffer layer, and it is exactly where a retagging fix could quietly break a sibling.

One ask, small: source[33] is a magic index in an assertion. It is presumably the byte just past a 32-byte alignment boundary -- which is the byte a mis-sized copy would leave stale -- but a reader cannot tell that from 33. Either a named constant or a comment saying why that byte is the tell would make the mutant's message self-explanatory, which matters because that string is what a future failure will show.

Nothing else: the refusal path logs the pointer and the alignment requirement, which is the difference between a refusal that can be acted on and one that only says no.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 7a2da12: READY

Both halves are right, and the two details that make each of them correct are the ones a reader skims.

The refusal names the requirement instead of restating it

if (ptr == nullptr || size == 0 || reinterpret_cast<uintptr_t>(ptr) % TENSOR_ALIGNMENT != 0) {
    GGML_LOG_ERROR("%s: refusing host pointer %p (size %zu): it must be non-null and %d-byte aligned\n", …);
    return nullptr;
}

TENSOR_ALIGNMENT and not a literal 32 — the requirement now lives in one place, so a change to the tensor layout's alignment cannot leave this check asserting an old number. Compare the router descriptor two repos over, where a hard-coded 9 sat next to a shared layout function that said 7. And the message carries the pointer, the size, and the requirement, which is the difference between a refusal that starts an investigation and one that ends it.

size == 0 refused too — a zero-size mapped buffer is a buffer that cannot be read, and returning it would push the failure to the first access.

The async fallback's ordering is the load-bearing line

backend_synchronize(backend);
ggml_backend_tensor_set(tensor, data, offset, size);
return;

The synchronise before the host access is what makes it correct: HRX work may still be in flight for that buffer, and a CPU read or write that starts before the queue drains sees a state that is about to change. Same class as fflush before the fork in the test harness — the sequencing is the fix, not the branch.

And the condition is narrow in both directions — not an HRX buffer and is host memory — so a host buffer that is an HRX buffer stays on the HRX path. A one-sided test would have diverted HRX-host buffers into a CPU copy and quietly cost throughput on the case the backend was built for.

One thing to record, and it is not about this PR

The new case is mapped_host_ptr, a device case calling model_buffer_from_mmap directly — the right shape. But I verified tonight that the fork's HRX test suite is run by no CI job at all: tests/CMakeLists.txt:417 gates the whole block on if (TARGET ggml-hrx), the fork's CI never builds HRX, and the engine's hrx-build.yml builds without a ctest step. So this case — like the assertion in #356 — will run when someone runs it by hand, and not otherwise.

That belongs on the board (it is there, 21:58Z) rather than in this description; I mention it because a device case is the least likely to be run by accident, and it is the one whose value most depends on the gate being closed. The cheap remedy stands: hrx-build.yml already has the tree built on the bench runner, so ctest -R 'hrx-' is a few lines.

Nothing to change in the code.

@bong-water-water-bong
bong-water-water-bong merged commit bb037f9 into 1bit/hrx-vulkan-patched Nov 14, 2026
@bong-water-water-bong
bong-water-water-bong deleted the cl-1051/2393-mapped-host-ptr branch November 14, 2026 02:08

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 7a2da12

I measured this one myself, including the mutant — and the refusal is load-bearing for two cases, not one.

$ hrx-buffer-test            # at this head, gpu-hold ticket
[PASS]    4/9 mapped_host_ptr
[FAIL]    3/9 imported_buffer
HRX buffer test summary: 8 passed, 1 failed, 0 known-fail, 0 unexpected-pass, 9 total

and the failure that is left is the one this PR deliberately leaves:

tests/hrx-buffer-test.cpp:234: requirement failed: !ggml_backend_buffer_is_host(imported)

Compare that with what engine#2393 recorded before this PR: GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0 && "buffer pointer must be aligned") failed in ggml-backend.cpp:2705, an abort with a detached inferior and no pointer value anywhere in the output. Now the failure is a named requirement that says what is wrong — the imported pointer yields a host buffer, and the case says it should not — which is exactly the design call this PR scopes out. A wrong number one can read beats an assertion that names the rule but not the value.

The mutant, which is the half of the body's table I could re-run: with the alignment clause removed from the refusal

-  if (ptr == nullptr || size == 0 || reinterpret_cast<uintptr_t>(ptr) % TENSOR_ALIGNMENT != 0) {
+  if (ptr == nullptr || size == 0) {
[FAIL]    3/9 imported_buffer
[FAIL]    4/9 mapped_host_ptr
ggml/src/ggml-backend.cpp:2705: GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0 …) failed
HRX buffer test summary: 7 passed, 2 failed, 0 known-fail, 0 unexpected-pass, 9 total

Two cases go red and the original abort comes back, so the refusal converts an abort into a named refusal and the new mapped_host_ptr case is the arm that would notice its removal. The body says the mutant breaks imported_buffer; it breaks both, which makes the new case more load-bearing than the PR claims — worth the one line in the comment.

The async change (ggml-hrx.cpp: a host, non-HRX buffer synchronizes and then uses the plain path instead of an HRX upload) is the one I did not mutate: reverting it needs the old binary and a device run of the model path, and the mapped_host_ptr case covers its observable half (source[33] == replacement was the failure the body records). I have no reason to doubt it — the evidence given is a before/after pair on this exact case — but it is their measurement, not mine, and I am saying which is which.

Refs engine#2393 (both halves named there; imported_buffer's design call is the one still open).

bong-water-water-bong added a commit that referenced this pull request Nov 14, 2026
… refusal that #371 removed (engine#2393) (#373)

#371 deleted both halves of #357 while adding the imported buffer type:
- backend_set/get_tensor_async again dropped data on a mapped (retagged
  CPU) buffer with "invalid HRX tensor upload/download";
- model_buffer_from_mmap again aborted in ggml_backend_cpu_buffer_from_ptr's
  GGML_ASSERT on a null or misaligned pointer, so mapped_host_ptr aborts.

model_buffer_from_mmap still returns the retagged CPU buffer after #371,
so both are needed. Restored as #357 had them, plus the ggml-impl.h
include the refusal uses.

Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant