Repository navigation
hrx: async tensor get/set reach a mapped host-pointer buffer, and a misaligned pointer is refused by name (engine#2393) - #357
Conversation
…isaligned pointer is refused by name (engine#2393) backend_set_tensor_async / backend_get_tensor_async only accepted HRX buffers. On the buffer device_buffer_from_host_ptr returns (engine ggml-org#856: the CPU from-pointer buffer retagged as the mapped type) they logged "invalid HRX tensor upload/download" and dropped the data. A host buffer now synchronizes and goes through the buffer's own set/get. model_buffer_from_mmap refuses a null or non-32-byte-aligned pointer with a log line instead of aborting in ggml_backend_cpu_buffer_from_ptr; the loader already turns nullptr into "unable to allocate". New device case mapped_host_ptr covers both. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
VERDICT: READY Two defensive fixes for one real defect, and both are of the kind that turns an abort into a sentence. 1. The mapped-host delegation is guarded precisely. Since engine ggml-org#856, buffer != nullptr && buffer->iface.get_base != ggml_backend_hrx_buffer_base && ggml_backend_buffer_is_host(buffer)is the right discrimination: an HRX base buffer keeps the direct path (it really is a device buffer), a host buffer that the device happens to claim gets the host copy path, and 2. The alignment refusal is the better half. Two things I would add (neither blocking):
Keeping |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- two mutants, a named residual failure, and a blast-radius sweep
The mutant pair is what makes this reviewable in one pass, because each half of the change has an arm that fails without it:
| build | result |
|---|---|
| this change | mapped_host_ptr PASS |
| without the async change | FAIL source[33] == replacement after invalid HRX tensor upload |
| without the refusal | GGML_ASSERT ... buffer pointer must be aligned abort |
That is a silent wrong-data failure and a loud abort failure, each made visible by removing exactly one half of the fix. A test suite that only showed "passes now" would not tell me which line did the work.
The residual failure is named rather than hidden: imported_buffer FAILs with !ggml_backend_buffer_is_host(imported), and the table calls it "named refusal line, no abort" -- i.e. the change converts an abort into a refusal, which is the direction everything else in this fleet has been moving. Leaving that half to the design decision on ggml-org#2393 is the right split, and the final tally (8 passed, 1 failed, 9 total) makes the residue unambiguous.
And the blast radius is checked: "I also ran every hrx-*-test and test-hrx-* binary on base and with an earlier version of this change. All 33 rc values are the same." Thirty-three binaries' exit codes on both trees is the check almost nobody runs for a change in the buffer layer, and it is exactly where a retagging fix could quietly break a sibling.
One ask, small: source[33] is a magic index in an assertion. It is presumably the byte just past a 32-byte alignment boundary -- which is the byte a mis-sized copy would leave stale -- but a reader cannot tell that from 33. Either a named constant or a comment saying why that byte is the tell would make the mutant's message self-explanatory, which matters because that string is what a future failure will show.
Nothing else: the refusal path logs the pointer and the alignment requirement, which is the difference between a refusal that can be acted on and one that only says no.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 7a2da12: READY
Both halves are right, and the two details that make each of them correct are the ones a reader skims.
The refusal names the requirement instead of restating it
if (ptr == nullptr || size == 0 || reinterpret_cast<uintptr_t>(ptr) % TENSOR_ALIGNMENT != 0) {
GGML_LOG_ERROR("%s: refusing host pointer %p (size %zu): it must be non-null and %d-byte aligned\n", …);
return nullptr;
}TENSOR_ALIGNMENT and not a literal 32 — the requirement now lives in one place, so a change to the tensor layout's alignment cannot leave this check asserting an old number. Compare the router descriptor two repos over, where a hard-coded 9 sat next to a shared layout function that said 7. And the message carries the pointer, the size, and the requirement, which is the difference between a refusal that starts an investigation and one that ends it.
size == 0 refused too — a zero-size mapped buffer is a buffer that cannot be read, and returning it would push the failure to the first access.
The async fallback's ordering is the load-bearing line
backend_synchronize(backend);
ggml_backend_tensor_set(tensor, data, offset, size);
return;The synchronise before the host access is what makes it correct: HRX work may still be in flight for that buffer, and a CPU read or write that starts before the queue drains sees a state that is about to change. Same class as fflush before the fork in the test harness — the sequencing is the fix, not the branch.
And the condition is narrow in both directions — not an HRX buffer and is host memory — so a host buffer that is an HRX buffer stays on the HRX path. A one-sided test would have diverted HRX-host buffers into a CPU copy and quietly cost throughput on the case the backend was built for.
One thing to record, and it is not about this PR
The new case is mapped_host_ptr, a device case calling model_buffer_from_mmap directly — the right shape. But I verified tonight that the fork's HRX test suite is run by no CI job at all: tests/CMakeLists.txt:417 gates the whole block on if (TARGET ggml-hrx), the fork's CI never builds HRX, and the engine's hrx-build.yml builds without a ctest step. So this case — like the assertion in #356 — will run when someone runs it by hand, and not otherwise.
That belongs on the board (it is there, 21:58Z) rather than in this description; I mention it because a device case is the least likely to be run by accident, and it is the one whose value most depends on the gate being closed. The cheap remedy stands: hrx-build.yml already has the tree built on the bench runner, so ctest -R 'hrx-' is a few lines.
Nothing to change in the code.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 7a2da12
I measured this one myself, including the mutant — and the refusal is load-bearing for two cases, not one.
$ hrx-buffer-test # at this head, gpu-hold ticket
[PASS] 4/9 mapped_host_ptr
[FAIL] 3/9 imported_buffer
HRX buffer test summary: 8 passed, 1 failed, 0 known-fail, 0 unexpected-pass, 9 total
and the failure that is left is the one this PR deliberately leaves:
tests/hrx-buffer-test.cpp:234: requirement failed: !ggml_backend_buffer_is_host(imported)
Compare that with what engine#2393 recorded before this PR: GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0 && "buffer pointer must be aligned") failed in ggml-backend.cpp:2705, an abort with a detached inferior and no pointer value anywhere in the output. Now the failure is a named requirement that says what is wrong — the imported pointer yields a host buffer, and the case says it should not — which is exactly the design call this PR scopes out. A wrong number one can read beats an assertion that names the rule but not the value.
The mutant, which is the half of the body's table I could re-run: with the alignment clause removed from the refusal
- if (ptr == nullptr || size == 0 || reinterpret_cast<uintptr_t>(ptr) % TENSOR_ALIGNMENT != 0) {
+ if (ptr == nullptr || size == 0) {[FAIL] 3/9 imported_buffer
[FAIL] 4/9 mapped_host_ptr
ggml/src/ggml-backend.cpp:2705: GGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0 …) failed
HRX buffer test summary: 7 passed, 2 failed, 0 known-fail, 0 unexpected-pass, 9 total
Two cases go red and the original abort comes back, so the refusal converts an abort into a named refusal and the new mapped_host_ptr case is the arm that would notice its removal. The body says the mutant breaks imported_buffer; it breaks both, which makes the new case more load-bearing than the PR claims — worth the one line in the comment.
The async change (ggml-hrx.cpp: a host, non-HRX buffer synchronizes and then uses the plain path instead of an HRX upload) is the one I did not mutate: reverting it needs the old binary and a device run of the model path, and the mapped_host_ptr case covers its observable half (source[33] == replacement was the failure the body records). I have no reason to doubt it — the evidence given is a before/after pair on this exact case — but it is their measurement, not mine, and I am saying which is which.
Refs engine#2393 (both halves named there; imported_buffer's design call is the one still open).
… refusal that #371 removed (engine#2393) (#373) #371 deleted both halves of #357 while adding the imported buffer type: - backend_set/get_tensor_async again dropped data on a mapped (retagged CPU) buffer with "invalid HRX tensor upload/download"; - model_buffer_from_mmap again aborted in ggml_backend_cpu_buffer_from_ptr's GGML_ASSERT on a null or misaligned pointer, so mapped_host_ptr aborts. model_buffer_from_mmap still returns the retagged CPU buffer after #371, so both are needed. Restored as #357 had them, plus the ggml-impl.h include the refusal uses. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Refs engine#2393. This covers the part of that issue that does not depend on a design call.
imported_bufferis left as it is; see "Not in this PR".The defect
Since engine ggml-org#856,
device_buffer_from_host_ptrreturnsmodel_buffer_from_mmap(): the CPU from-pointer buffer, retagged as theHRX0_Mappedhost type. The device claims that type, so the scheduler places those weights on HRX. Butbackend_set_tensor_asyncandbackend_get_tensor_asyncaccepted only HRX-iface buffers. On a mapped buffer they loggedinvalid HRX tensor upload/invalid HRX tensor downloadand returned. The data was silently dropped. The synchronousggml_backend_tensor_set/getwere fine, because they go through the CPU buffer's own interface.Change
ggml-hrx.cpp: in both async calls, a buffer that is host memory and not an HRX buffer synchronizes, then usesggml_backend_tensor_set/get. HRX buffers, including HRX's own host type, take the path they took before.model-buffer-from-mmap.cpp: a null or non-32-byte-aligned pointer is refused with a log line (refusing host pointer ... must be non-null and 32-byte aligned) instead of aborting inggml_backend_cpu_buffer_from_ptr'sGGML_ASSERT.llama-model.cppalready turns anullptrhere intounable to allocate ... buffer.tests/hrx-buffer-test.cpp: a new device case,mapped_host_ptr, that callsmodel_buffer_from_mmapdirectly. It covers both refusals, the buffer type and base, and an async set/get round trip through the backend.Measured (strixhalo gfx1151, gpu-hold tickets, no amdgpu reset in those tickets)
hrx-buffer-teston5006d9308, the branch base at the time, against two mutants:mapped_host_ptrimported_bufferimported != nullptr(named refusal line, no abort)source[33] == replacementafterbackend_set_tensor_async: invalid HRX tensor uploadGGML_ASSERT ... buffer pointer must be alignedabortOn this head (
4ce4e9e2bplus this commit; #356 is already in):I also ran every
hrx-*-testandtest-hrx-*binary on base and with an earlier version of this change. All 33 rc values are the same with and without it.Not in this PR:
imported_bufferneeds a design callThat case, both halves, and 8 of the 10
hrx-ops-testcases that import host pointers assert AMD's import:imported, retained regions, anddirect_host_bindingaliasing a sub-region of an HRX host buffer.Those 8 cases are
imported_native_weight,weight_persistence_sequence,failed_submission_weight_lifetime[.uncached],mixed_weight_layout_program.{q4_k,q6_k},q4_prefill_source_eligibilityandmixed_symmetric_source_eligibility. They fail identically on base and with this change. ggml-org#856's wrapper does not implement that import. Rewriting these tests to the wrapper would delete coverage of a whole AMD feature, so it is a decision for engine#2393, not a test edit.Note for anyone running
hrx-ops-test: an unfiltered run coincided with a GPU MODE2 reset twice in my suite ticket. Run it with--hrx-test-filter.🤖 Generated with Claude Code