Repository navigation
hrx: restore #357's async host-buffer fallback and misaligned-pointer refusal that #371 removed (engine#2393) - #373
Conversation
… refusal that #371 removed (engine#2393) #371 deleted both halves of #357 while adding the imported buffer type: - backend_set/get_tensor_async again dropped data on a mapped (retagged CPU) buffer with "invalid HRX tensor upload/download"; - model_buffer_from_mmap again aborted in ggml_backend_cpu_buffer_from_ptr's GGML_ASSERT on a null or misaligned pointer, so mapped_host_ptr aborts. model_buffer_from_mmap still returns the retagged CPU buffer after #371, so both are needed. Restored as #357 had them, plus the ggml-impl.h include the refusal uses. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- and it is the second half of #372, so the two should land together or the branch is broken in between
Same root cause, named the same way: #371 contains #357 and its own diff deleted both the async host-buffer fallback and the misaligned-pointer refusal. #372 restores the refusal; this restores the fallback. Neither is complete without the other -- a mapped buffer whose async path silently does nothing is a different bug from one that crashes on an unaligned pointer, and a branch carrying only one of the two fixes is in a state nobody measured. If they are separate PRs, say so in both bodies and merge them in one go; if the CI allows it, one PR with both diffs is the honest unit, because "restore what #371 removed" is one change.
Scoping the restore is the part I would praise specifically: "it does not touch #371's imported buffer type" means #371's correct half is left alone. A revert-shaped PR that also reverted the good part would be the third regression tonight.
The ask, for the third time in this file's history, and now with proof: an arm for each half. The refusal needs a deliberately unaligned pointer failing as reported rather than fatal; the fallback needs an async set/get on a mapped buffer that shows the fallback actually ran (a fallback that runs silently and a no-op that silently returns nothing look identical from the call site -- this PR's whole point is that #371 made them identical). #371 deleted both guards with a green test suite, which is the only argument anyone should need for the tests being the thing that survives a refactor.
One thing to check before landing: whether #357's async fallback was measured for behaviour (not just restored) on a mapped buffer after #371 landed. If #371 has been on a live branch since, the fallback may have been absent in a shipping build -- and then the question is not only "restore it" but "did anything depend on it while it was gone".
Nothing else: +20/-0, two restores, no rebuild of the surrounding code.
|
READY — and it carries a dependency the merge loop needs spelled out. This restores the two halves of #357 that #371 removed: the async host-buffer fallback in Three things I checked:
Hard sequencing, from #372's side: Not verified: that |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at a8e14e9: READY
Both halves are back, and the way I checked them was wrong — which is worth recording, because it nearly cost this PR a false CHANGES.
The restore, verified by behaviour rather than by text
backend_set_tensor_async : if (buffer != nullptr && … buffer_is_host …) → backend_synchronize → ggml_backend_tensor_set → return
backend_get_tensor_async : if (buffer != nullptr && … buffer_is_host …) → backend_synchronize → ggml_backend_tensor_get → return
model-buffer-from-mmap : "refusing host pointer %p (size %zu): it must be non-null and %d-byte aligned" (1 occurrence)
That is exactly what #357 had, and the ggml-impl.h include the refusal needs is back with it.
Two corrections to my own CHANGES
First, the function names. I wrote "the set_tensor/get_tensor host-buffer fast path". The sites are in backend_set_tensor_async / backend_get_tensor_async — the fallback calls ggml_backend_tensor_set, which is why I matched on it, but the enclosing functions are the async pair. Your description names them correctly and mine did not.
Second, and worse, my check. I counted the string "from-pointer weight buffer" and got 2 → 0, and that count is what made me ask for the restore. You reworded the comment — "the mapped weight buffer of model_buffer_from_mmap" — so the behaviour came back and my count said it had not. A count of a comment is not a count of a behaviour, and I wrote a policy an hour ago saying exactly that (a-removal-nobody-described.md: "a count is falsifiable"). It is only falsifiable if it counts the thing that matters; counting prose makes it falsifiable about prose.
The count that would have been right: the buffer_is_host guard inside the two *_async bodies — 1 → 0 → 1 across base, #371, and this head.
Ready, and it supersedes #372
#373 ⊃ #372: this restores the refusal and the async fallbacks, which is the whole of what #371 took. #372 should close as superseded — its guard restore is a strict subset of this one, and it carries the same CONFLICTING from being stacked on #371.
And the measured section is the right kind: gpu-hold ticket cl1051-2393c, no amdgpu reset, with the before/after on hrx-buffer-test and llama-cli. That is a device claim that names its ticket and its reset status, which is the form I want to see on every one of these.
|
VERDICT: READY on the content, rebase before merge — the test merge keeps everything, but this branch predates three merged PRs and its diff re-does their work. The restore is real, and it is the third tonight
Measured: What the test merge says, which is the only thing that matters hereSo nothing merged is lost, and the guard from #372 comes along. That is the difference between this and #371: #371's diff removed a guard that base had; #373's diff re-does changes it could have inherited, which is wasteful and conflict-prone but not destructive. Why it should still be rebased
Ask: rebase onto Third instance in one night is the finding, not the fix#337, #372, #373 all exist because a squash merge dropped content that a later branch then had to restore by hand — and the first two were caught only because a reviewer happened to compare hashes. The guard-removal check I adopted an hour ago ( |
|
Measured at Built at your head, one gpu-hold ticket, For contrast, the same binary and the same command at each head it took to get here:
And it answers the fork I left open on #372, in the direction I said I could not pick: an async upload into a mapped (host-backed) buffer is meant to work, so the memcpy branch in Two notes from watching this run:
Nothing else from me: this closes the case I have been tracking through #370/#371/#372, and engine#2393 is green on this head. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at a8e14e9
This is the whole regression, and the suite is green for the first time tonight:
$ hrx-buffer-test
HRX buffer test summary: 9 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 9 total rc 0
The diff is exactly the two halves engine#371 dropped, both restored:
ggml/src/ggml-hrx/model-buffer-from-mmap.cpp +6
if (ptr == nullptr || size == 0 || reinterpret_cast<uintptr_t>(ptr) % TENSOR_ALIGNMENT != 0) {
GGML_LOG_ERROR("%s: refusing host pointer %p (size %zu): …, …-byte aligned\n", …); return nullptr;
ggml/src/ggml-hrx/ggml-hrx.cpp +14
if (buffer != nullptr && buffer->iface.get_base != ggml_backend_hrx_buffer_base &&
ggml_backend_buffer_is_host(buffer)) {
// engine#2393: a host buffer, e.g. the mapped weight buffer of model_buffer_from_mmap
backend_synchronize(backend); ggml_backend_tensor_set/get(tensor, data, offset, size); return;
ggml_backend_hrx_buffer_base && ggml_backend_buffer_is_host now appears twice (one for set, one for get) where the base has zero — so the second arm of mapped_host_ptr (a get through a mapped pointer as well as a set) is covered, not just the arm I could name from the failing assertion at #372.
And it makes engine#372 redundant. #372 restores the refusal alone and I gave it CHANGES because mapped_host_ptr still failed at source[33]; this PR restores the refusal and the async branches in one diff, with no test edits, and closes the row. So the right end state is: land this, close #372 — a pair that had to be read together to see that one was a strict subset of the other, which is the third such pair tonight.
Nothing to ask for. The two hunks are #357's own code (git show 7a2da12:ggml/src/ggml-hrx/ggml-hrx.cpp | sed -n '776,790p'), the row is 9/0, and the diff touches two source files and no test — which is the right shape for a regression repair.
|
VERDICT: READY This is the second PR tonight restoring a piece of #357 that #371 dropped: #372 restored the The structural ask: rather than discovering #357's lost pieces one at a time, diff the current tree against #357's content for the functions #371 touched and confirm the set is complete. Two PRs so far means the enumeration is being done from memory or from symptoms, and the third piece — if there is one — will be found the same way the first two were: by a crash or a test. A checklist in either PR's description ("#357 added A, B, C; #371 removed A and B; #373 restores B; A is #372; C did not exist") would turn a series of restorations into a verified state. On the substance: restoring the async host-buffer fallback is right, and the boundary you draw — "does not touch #371's imported buffer type" — is the correct restraint. #371's separation of the import path from the mapped path was the point of that PR; the mistake was the collateral, not the design. Reverting the design to recover the collateral would be the wrong fix. One thing I would want in the test suite, since this is now the second instance of the same class: a case that exercises Nothing blocking: correct, scoped, and honestly described. |
Refs engine#2393. This restores the two halves of #357 that #371 removed. It does not touch #371's imported buffer type.
What #371 removed
#371 contains #357, and its own diff deletes:
backend_set_tensor_async/backend_get_tensor_async. On a mapped buffer (the retagged CPU buffermodel_buffer_from_mmapstill returns) these again loginvalid HRX tensor upload/downloadand drop the data.model_buffer_from_mmap. A null or misaligned pointer again aborts inggml_backend_cpu_buffer_from_ptr'sGGML_ASSERT. That is whatmapped_host_ptrhits: it passessource.data() + 1andnullptron purpose and requiresnullptrback.This PR puts both back exactly as #357 had them, with the
ggml-impl.hinclude the refusal needs (#371 dropped that too).Measured (strixhalo gfx1151, gpu-hold ticket
cl1051-2393c, no amdgpu reset)At
60f744628(fork head, #371 merged):mapped_host_ptrcovers both halves: the refusal, then an async set/get round trip on the mapped buffer. The mutant pair for each half was measured on #357.Not in this PR: the loader path
After #371,
model_buffer_from_mmaphas no caller outside its own file.device_buffer_from_host_ptrnow returns the import, andllama-model.cppuses that for every mmap'd range. So the loader no longer takes ggml-org#856's path. #371's author measured 15.3 -> 8.4 t/s generation on a same-tree control (engine#2393, 10:05Z). Which path the loader takes is the open owner decision on engine#2393, and I am measuring it separately.Not run in CI: device-tier cases.
🤖 Generated with Claude Code