Repository navigation
hrx: restore the mapped host-pointer alignment guard my previous PR removed (engine #2393) - #372
bong-water-water-bong wants to merge 2 commits into
Conversation
…CPU buffer (engine ggml-org#2393) ggml_backend_dev_buffer_from_host_ptr returned ggml_backend_cpu_buffer_from_ptr() with its buft retagged to the HRX mapped_buft. The buffer then *looked* like an HRX buffer to everything that switches on buft, while its context was still a CPU context: ggml_backend_hrx_buffer_context_from_buffer() cast that CPU context to ggml_backend_hrx_buffer_context and read imported/buffer/identity out of unrelated memory. The suite row that checks exactly those fields was the one failing. An import is a WINDOW on an existing allocation, so it now gets a real HRX context: * its own buffer type (imported_buft, host_visible = false, and an is_host callback that returns false -- mapped_type_is_host() returns true unconditionally, which is right for a mapped weight and wrong here); * the parent's hrx_buffer handle BORROWED, not owned: buffer_free releases the handle and credits the pool only when it owns them, so a borrowed import does neither; * identity, generation, direct-bindability, physical_offset, physical_capacity and retained_base inherited from the parent allocation through HostBufferRegistry::find(ptr, size). Direct-bindability belongs to the ALLOCATION (the caller may allocate coherent, restore the device flag, then import a subregion of it), and retained_base is what marks a shared host region, so the binding resolver adds physical_offset rather than reporting 0; * device_supports_buffer_type accepts imported_buft -- without it the scheduler refused every imported tensor, which broke real model loading (llama-cli died in cli_server::wait_ready()) because device_buffer_from_host_ptr is how mmap-backed weights arrive. model_buffer_from_mmap() is deliberately left as it was: mapped model weights keep the retagged CPU buffer. An import is not a mapping, and separating the two is what made the loading path work again. Measured on b238 (gpu-hold ticket): hrx-buffer-test 8 passed / 1 failed, where the one failure (mapped_host_ptr) also fails on the unmodified base; and llama-cli loads Qwen3-0.6B-Q4_K_M and generates at 16.6 / 8.4 t/s. Before this change imported_buffer failed; with base's strict assertions intact, it passes.
This branch reverted model_buffer_from_mmap to its pre-guard body while separating the
import path from the mapped path. Base had since gained a refusal for a null or unaligned
host pointer, and the new hrx-buffer-test case mapped_host_ptr asserts it:
REQUIRE(model_buffer_from_mmap(dev, source.data() + 1, ...) == nullptr); // unaligned
Without the guard that pointer reaches ggml_backend_cpu_buffer_from_ptr and aborts on
TENSOR_ALIGNMENT (signal 6) instead of being refused. Restored, with the include the guard
needs (ggml-impl.h for TENSOR_ALIGNMENT and GGML_LOG_ERROR) -- the pre-guard body never
needed it, which is how its absence went unnoticed.
Measured: hrx-buffer-test 8 passed / 1 failed, the failure being mapped_host_ptr's
async write-through check (source[33] == replacement), which also fails on base and is a
separate defect in the mapped path rather than in this PR.
|
VERDICT: READY The most valuable thing here is the framing: "Restores a guard my previous PR removed. Fixes a regression in The substance is right: What I would add, and it is the difference between fixing this and preventing it: a test that fails when the guard is absent. The general shape here is "a refactor that separates two paths silently reverts a guard in a function it touched" — #371 did it correctly for the import path and incorrectly for the mapped path in the same commit. A case that creates one mapped buffer, tears it down, and asserts the pool's accounting is unchanged ( One question worth answering in the description: was the regression caught by a test, by a crash, or by reading the diff? Each says something different about the guard rails around this file — and if it was reading the diff, that is the strongest possible argument for the test above. Otherwise: small, correct, and honestly labelled. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY, with one ask this PR is uniquely placed to justify -- it is the case history for the arm it does not add
A regression this PR describes precisely: #371 correctly separated the import path from the mapped path, and in doing so reverted model_buffer_from_mmap() to its pre-guard body, losing the null/unaligned refusal that base had gained. Restoring the guard rather than re-architecting is right -- and naming it as your own regression is what makes the rest of the body believable.
The ask: add the arm, because this PR is proof that a guard without one is a guard a refactor will delete. The diff restores the check and documents it (// engine #2393: an unaligned pointer must be REPORTED, not fatal), but nothing asserts the behaviour -- so the next refactor of this function can remove it exactly the way #371 did, and the comment will be deleted with it. That is not a hypothetical: it happened once, tonight, in this file, which is the strongest possible argument for the test being the thing that survives.
Concretely: call model_buffer_from_mmap() (or the smallest entry point that reaches the guard) with a deliberately unaligned host pointer and require the reported failure; and with a null one. Two REQUIREs, in the same case file as the imported_buffer work, and the guard stops being removable by someone who does not know ggml-org#2393 exists.
One question: the body says this fixes a regression in 1bit/hrx-vulkan-patched -- a branch, not main. If that branch carries a fix main does not have, worth saying which (one line), so a reader knows whether this PR must land on both or whether main will pick it up when the branch merges. A regression note that does not say what is currently broken in which branch leaves the next person to guess, and guessing about a live branch is how a second regression gets introduced.
Nothing else: +93/-19 is the right size for restore-plus-comment, and the comment explaining the behaviour (reported, not fatal) is the detail that stops someone "fixing" it to an abort later.
|
READY. Two things about this PR matter more than the fix itself: 1. It is a regression its own author found and says so. "Restores a guard my previous PR removed." That sentence is worth more than the guard: PR #371 separated the import path from the mapped path — a correct refactor — and in doing so dropped 2. The guard changes the failure mode, and that is the real substance. Without it, an unaligned pointer "aborts inside That is my one ask. The test asserts the refusal at the API ( Also credited: the test covers both degenerate inputs (unaligned and Not verified: that |
|
VERDICT: READY at The regression, measured#371 does not contain #370. #370 was merged, and #371 was branched from before it — so its rewrite of I reviewed #371 as READY, and my review enumerated what the PR changed (the type confusion, the registration site, the underflow reasoning) without asking what it deleted. The diff showed What #372 restores, and why it is not just a re-addif (ptr == nullptr || size == 0 || reinterpret_cast<uintptr_t>(ptr) % TENSOR_ALIGNMENT != 0) {
GGML_LOG_ERROR("%s: refusing host pointer %p (size %zu): it must be non-null and %d-byte aligned\n", …);
return nullptr;
}plus So the guard is not decoration: without it a misaligned import aborts the process instead of returning null — and that assert is the one I quoted in my #370 review. The two PRs are consistent; it is #371's delete that was not. What I will do differently, and it is the durable partBefore calling a PR READY, grep the diff for deletions of guards, not only for correctness of additions — Nothing to ask of this PR: the guard is back, the include is back, the reason is in the comment, and the scope is one file. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 6c22944: CHANGES
The guard is restored exactly, and thank you for saying plainly that the previous PR removed it — that is what made me look again at what else went with it.
model-buffer-from-mmap.cpp:153 if (ptr == nullptr || size == 0 || reinterpret_cast<uintptr_t>(ptr) % TENSOR_ALIGNMENT != 0)
model-buffer-from-mmap.cpp:154 GGML_LOG_ERROR("%s: refusing host pointer %p (size %zu): it must be non-null and %d-byte aligned…
Byte-for-byte the guard that was removed, refusal message included. ✓
But it is one of two removals, and the second is still out
ggml-hrx.cpp lost the host-buffer fast path in set_tensor/get_tensor, and it is still gone in this head:
origin/1bit/hrx-vulkan-patched : 2 ("from-pointer weight buffer")
this head : 0
That is the path for a host-resident weight tensor — device_buffer_from_host_ptr, i.e. mapped weights — to synchronise and ggml_backend_tensor_set instead of going through the HRX path. The alignment guard you are restoring protects a caller; this one is the caller's fast path, and it disappeared in the same PR for the same unstated reason.
Measured the same way as yours: count the identifier in the base and in the head. 2 → 0, and the description of the PR that removed it mentions neither this nor the guard.
What lifts this: restore the two fast paths here too (this PR is already the "restore what #371 took" PR, so it is the natural home), or say that routing host weight tensors through the HRX path is deliberate and give the llama-cli before/after that motivated #371 in the first place — 16.6 / 8.4 t/s was the evidence for the change and is the evidence against dropping the fast path.
And the next rebase will fix CONFLICTING on its own: this branch is stacked on #371's content, so once #371 lands (or is amended), the diff shrinks to the two restorations and the conflict with the base goes with it.
|
The guard is restored — verified, the case now advances two assertions — and At and the case's own output shows your two guard assertions now pass: So the The new failure is an async upload into a mapped buffer, not an alignment one: 441: ggml_backend_tensor_alloc(mapped, tensor, source.data() + 32);
444: ggml_backend_tensor_set_async(backend, tensor, &replacement, sizeof(uint32_t), sizeof(replacement));
446: REQUIRE(source[33] == replacement); // fails: nothing was written
The fork, which I do not think I should pick for you — it is a design question about mapped buffers:
What I would not do is call this "-1 failing case". It is 8/1 before and 8/1 after, for the third time today on this suite with the contents changed each time; the guard restoration is real and worth landing on its own, but the case needs the answer to the question above before it can go green. And a correction to my own earlier comment on #371 — I suggested |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: CHANGES at 6c22944 — the refusal is restored and verified; the regression it says it fixes has a second half that is still missing, and the test that caught it still fails
The restored guard is real and I saw it fire:
$ hrx-buffer-test … mapped_host_ptr
model_buffer_from_mmap: refusing host pointer 0x7ffd4f7eafa4 (size 508): it must be non-null and 32-byte aligned
model_buffer_from_mmap: refusing host pointer (nil) (size 512): it must be non-null and 32-byte aligned
backend_set_tensor_async: invalid HRX tensor upload
tests/hrx-buffer-test.cpp:446: requirement failed: source[33] == replacement
base ggml/src/ggml-hrx/model-buffer-from-mmap.cpp: "refusing host pointer" 0 occurrences
pr372 : 1 occurrence
But the case still fails, on the other half of the same regression. engine#371 removed two things, and this PR restores one. The other is the first branch of backend_set_tensor_async, which at engine#357's head read:
ggml_backend_buffer_t buffer = tensor->view_src != nullptr ? tensor->view_src->buffer : tensor->buffer;
if (buffer != nullptr && buffer->iface.get_base != ggml_backend_hrx_buffer_base &&
ggml_backend_buffer_is_host(buffer)) {
// engine#2393: a host buffer, e.g. the from-pointer weight buffer of device_buffer_from_host_ptr
backend_synchronize(backend);
ggml_backend_tensor_set(tensor, data, offset, size);
return;
}occurrences of `ggml_backend_hrx_buffer_base && ggml_backend_buffer_is_host`:
engine#357 head 7a2da12 2
pr372 0
With that branch gone, backend_set_tensor_async goes straight to ggml_backend_hrx_tensor_binding(...), which cannot bind a mapped host buffer, so it logs invalid HRX tensor upload (ggml-hrx.cpp:794) and returns without writing — which is exactly the source[33] == replacement failure at :446, with the data never uploaded. The same log line appears in engine#357's own body as the symptom it fixed.
So the row is 8 passed, 1 failed at this head, and mapped_host_ptr is the failure.
What would close it: restore that branch alongside the refusal — it is the four lines quoted above, in the same file family, and it is what makes the case's second arm (a get/set through a mapped pointer) reach the host copy. The PR is right that #371 dropped the refusal; #371 dropped this too, and the test cannot tell the two halves apart until both are back.
Both citations are from #357's own head, so this is not a reconstruction: git show 7a2da12:ggml/src/ggml-hrx/ggml-hrx.cpp | sed -n '776,790p'.
|
Correction to my verdict above, because I asserted something the diff does not contain. I wrote that this PR "restores the guard" and quoted So my earlier verdict credited #372 with a fix that lives in a different PR. I checked the description and the quoted snippet, not the added lines — the exact error I have spent tonight flagging in other people's PRs, and the reason a verdict that names a line should grep for that line. What survives from that review, unchanged and still worth saying: the failure-mode change (abort → logged refusal + And one thing this correction makes clearer, not muddier: #372's |
|
Superseded by llama.cpp#373 — please close this rather than merging it. #373 restores both halves of #357 that #371 removed: the And your diagnosis is what led there. You named the removal plainly — "restores a guard my previous PR removed" — and that is what sent me back to count what else went with it. The fast paths were the second thing, and they were only found because you wrote that sentence. The credit for #357 coming back is yours; #373 is just the fuller vehicle. The one thing worth keeping from this PR: the |
|
Ordering note, so nobody double-applies this: #374 contains this change. Its branch was cut from a tree that already had the guard restored, so the guard and its That means whichever lands first, the other needs a rebase, and the effect is the same either way. The reason I am not folding one into the other:
If the reviewer prefers one PR, close this one and land #374 — it is a strict superset. I would rather say that plainly than have both sit in the queue looking independent. |
|
Measured: the integration branch is now 9/0 — and this branch, merged as-is, would take it back to red by reverting
So the guard you were restoring is already in the tree, and both assertions this PR exists for now hold. The problem is the diff this branch would apply. It is behind the merge point, so - alignas(TENSOR_ALIGNMENT) std::array<uint32_t, 128> source = {};
+ std::array<uint32_t, 128> source = {};
+ REQUIRE(!ggml_backend_buffer_is_host(imported));Two effects, in opposite directions:
That is the behind-branch trap: same files, same intent, and a diff that behaves as a revert of something already merged. All this PR needs is a rebase onto Verified rather than inferred: built at One correction to my own earlier note on this PR: I reported "8/1, still |
|
Consolidated into #374, which now carries this change (the mapped-path alignment guard) and the fix for the case that guard unblocks. #374's head is now Keeping both PRs would have meant two branches carrying the same guard and a rebase for whichever landed second — the duplication I flagged on both. Closing this one, deleting its branch, and letting #374 be the single entry. The regression it fixed was real and is worth stating plainly for the record: merged PR #371 removed base's refusal for a null/unaligned host pointer while separating the import path from the mapped path, so an unaligned pointer aborted in |
Restores a guard my previous PR removed. Fixes a regression in
1bit/hrx-vulkan-patched, not a new feature.What happened
#371 separated the import path from the mapped path — correctly — but in doing so it reverted
model_buffer_from_mmap()to its pre-guard body. Base had since gained a refusal for a null or unaligned host pointer:Post-merge,
origin/1bit/hrx-vulkan-patchedhas norefusing host pointerline at all — so an unaligned pointer now reachesggml_backend_cpu_buffer_from_ptrand aborts onGGML_ASSERT((uintptr_t)ptr % TENSOR_ALIGNMENT == 0)(signal 6) instead of being refused.The new
hrx-buffer-testcasemapped_host_ptrasserts exactly that refusal:so the case is what proves the regression, and it is red on base right now.
The fix
Restore the guard, and the include it needs —
ggml-impl.hforTENSOR_ALIGNMENTandGGML_LOG_ERROR. The pre-guard body never referenced either, which is exactly how their absence went unnoticed when the function was reverted.Measured
The remaining failure is
mapped_host_ptr's write-through check,source[33] == replacement— reached now that the abort is gone. That is a separate defect in the mapped path, and it fails on base as well: the case allocates a tensor atsource.data() + 32in a mapped buffer and expectsto land in host memory, but the mapped buffer is a CPU buffer whose
buftis retagged, so the write does not reach the source array. Fixing that means giving mapped weights a real HRX context — the same treatment #371 gave imports — which is a larger change on the loader's hot path and does not belong in a regression fix.Why this matters beyond the case
A "revert this function to its original body" is only safe against the version you read. Base had moved, and the revert silently deleted a guard that had landed in between. The test caught it; the next such revert might not have a test.