Skip to content

tests: restore the imported-buffer is_host guard, and the rationale that argues for it (engine#2393) - #377

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
ishost-guard
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
ishost-guard

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

What

Restores one assertion and rewrites the comment that argued against it — both in tests/hrx-buffer-test.cpp, +8/−6.

-    // engine #2393: `imported` and `is_host` are INDEPENDENT flags … a zero-copy import of host memory is
-    // legitimately host-typed … asserting the negative here would pin an implementation detail …
+    // engine #2393: an import is NOT host-typed, and that is the property that broke. #370 relaxed this
+    // assertion … #371 settled it the other way: the bug was that a from-pointer buffer reached here as a
+    // CPU buffer with its `buft` retagged … An import now has its own buft (backend-context.h: `imported_buft`,
+    // "a direct binding whose buffer must not claim to be a host allocation"), so this negative is the
+    // invariant to hold: if an import ever becomes host-typed again, it went back through the retagging path
+    // that cost an afternoon.
+    REQUIRE(!ggml_backend_buffer_is_host(imported));

Why

#370 removed that assertion, reasoning that imported and is_host are independent flags. #371 decided the opposite, and its own header says why: an import gets a separate imported_buft, "a direct binding whose buffer must not claim to be a host allocation." The defect that consumed the afternoon was precisely a from-pointer buffer reaching that point as a CPU buffer with its buft retagged — which is what made is_host true while context->imported was false.

The assertion was never restored and the comment was never updated, so main currently teaches the reading that produced the bug, in the file that regressed from it. A stale assertion is a hole; a stale rationale is a hole that argues for itself — the next person to touch this case reads a paragraph explaining why the correct assertion is wrong.

Measured

At the rebase target, gfx1151, one gpu-hold ticket, dmesg amdgpu resets 0 before / 0 after:

hrx-buffer-test: 9 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 9 total

So this restores a guard rather than surfacing a new failure — worth stating plainly, because an assertion coming back after being deleted is usually a change that turns something red.

Scope

One test file, no production code. It does not make the case exercise the import path numerically; it makes the case assert the property #371 established, which nothing has asserted since #370 deleted it.

Refs engine#2393, llama.cpp#370, #371, #372 (closed), #374.

…hat argues for it

#370 removed REQUIRE(!ggml_backend_buffer_is_host(imported)) with a comment explaining that `imported` and
`is_host` are independent flags and a zero-copy import of host memory is legitimately host-typed. #371 settled the
question the other way: the defect was a from-pointer buffer reaching that point as a CPU buffer with its `buft`
retagged, which is exactly what made is_host true for something that was not a host allocation.

The comment was never updated, so the merged tree still teaches the reading that produced the bug, in the file that
regressed from it, and the assertion is gone. This restores both:

  - the rationale now cites backend-context.h's `imported_buft`, "a direct binding whose buffer must not claim to be
    a host allocation";
  - REQUIRE(!ggml_backend_buffer_is_host(imported)) is back, naming what it catches: if an import ever becomes
    host-typed again, it went back through the retagging path that cost an afternoon.

Verified at 7e91e98 before rebasing, one gpu-hold ticket, dmesg amdgpu resets 0 before / 0 after:

  hrx-buffer-test: 9 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 9 total

so this restores a guard rather than surfacing a new failure.
@bong-water-water-bong

Copy link
Copy Markdown
Author

READY, and it supersedes my verdict on #370 — which is the part I want to be explicit about.

In my #370 review I agreed the REQUIRE(!ggml_backend_buffer_is_host(imported)) assertion should be dropped, traced backend-buffer-binding.cpp:85 myself, and wrote that the import/host independence made the negative "an implementation detail". That reasoning was valid for the tree #370 was written against and wrong for the tree after #371. Once an import got its own imported_buft ("a direct binding whose buffer must not claim to be a host allocation"), the negative stopped being a label check and became the invariant that catches a regression to the retagging path — which is the bug that cost an afternoon. So the assertion should be restored, and my earlier verdict should be read as superseded.

Three things I checked in this PR:

  • It restores the assertion and rewrites the comment. The comment was the more dangerous half: main was teaching the relaxed reading — a future contributor reading it would conclude that a host-typed import is legitimate, which is the opposite of what the code now guarantees. A stale comment that outlives its reasoning is how a fixed bug comes back, and it is the same class as the guard_loss_check finding I posted earlier today (a deleted explanation that no other check sees).
  • It cites the specific field (backend-context.h: imported_buft, with its own wording quoted) rather than asserting the property. That is what makes the restoration checkable.
  • The device evidence is scoped properly: one gpu-hold ticket, dmesg resets 0 before / 0 after, and the ledger is the full 9 passed / 0 failed / 0 known-fail / 0 unexpected-pass — the 0 unexpected-pass being the line that shows nothing was silently skipped.

One ask: REQUIRE(!is_host) is now the assertion that would catch a re-introduction of the retagging path. If the retag is reachable from a path this case does not exercise (e.g. a from-pointer buffer built through a different entry than model_buffer_from_mmap), say which case covers that one — otherwise the invariant is checked only where the bug was, which is the same weakness the old comment had in the other direction.

Credit where it is due: noticing that a deleted assertion plus its rewritten rationale had left main teaching the wrong reading is a class of defect nobody was watching for before today's guard_loss_check, and this is the first time I have seen it found by hand.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 97a45e6c — and it is the case guard_loss_check was built for, so I ran that too.

The restore is right, and the comment now records why the assertion was relaxed and why that was wrong

REQUIRE(!ggml_backend_buffer_is_host(imported)) comes back, with the history that makes it durable: #370 read imported/is_host as independent (true of backend-buffer-binding.cpp:85), #371 settled it the other way — a from-pointer buffer reached here as a CPU buffer with its buft retagged, so is_host was true for something that was never a host allocation. Checked at head: an import is created against &device->imported_buft (ggml-hrx.cpp:1989), so the negative is now the invariant and not an accident of retagging.

The alignment guard it adds is symmetric with its sibling

Both from-pointer entry points now carry the identical three-part refusal:

if (ptr == nullptr || size == 0 || reinterpret_cast<uintptr_t>(ptr) % TENSOR_ALIGNMENT != 0) {
    GGML_LOG_ERROR("%s: refusing host pointer %p (size %zu): it must be non-null and %d-byte aligned\n", …);
    return nullptr;
}

Identical text at both sites is the right shape: two entry points, one contract, one message.

Two asks, the first of which is the whole point of a restore PR

  1. Nothing asserts that refusal. git grep over tests/ at this head finds no arm that passes a misaligned pointer (or a null, or size == 0) and expects nullptr. The restore's own body says the guard was dropped by a merge that did not notice — the only thing that makes the next such merge fail loudly is a test. Three lines in hrx-buffer-test.cpp:
    REQUIRE(hrx_mapped_buffer_from_ptr(device, ptr + 1, 64) == nullptr);   // misaligned
    REQUIRE(hrx_mapped_buffer_from_ptr(device, nullptr, 64) == nullptr);   // null
    REQUIRE(hrx_mapped_buffer_from_ptr(device, ptr, 0) == nullptr);        // empty
    Without that, this PR is a guard plus a comment, and the next #327-shaped merge removes it just as quietly. Restoring a guard without a test restores the state, not the protection.
  2. What happens to a caller that gets nullptr? The sibling returned nullptr before too, so this is not new behaviour, but the refusal now fires for unaligned mapped weights specifically: one line saying what the loader does on nullptr (fall back to a copy / abort) tells the reader whether a misaligned mapped weight is a refusal or a crash further down.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 97a45e6: READY

This PR reverses a call I made, and the reversal is correct — so the useful thing I can add is why both were right at their own moment, and one place the invariant is narrower than the comment implies.

Both verdicts were right, because the design moved between them

#370  REQUIRE(!is_host(imported))  →  replaced with a comment:  imported/is_host are independent flags
#371  introduced imported_buft:     "a direct binding whose buffer must not claim to be a host allocation"
#374  introduced mapped_buft + imported = true  for a mapped weight, which IS host-typed
#377  REQUIRE(!is_host(imported))  →  restored, as the invariant that catches a regression

At #370 the assertion was pinning a coincidence: the flags genuinely were independent (backend-buffer-binding.cpp:85, :111), nothing in the code required the negative, and a test that asserts !is_host for an imported buffer would have failed a legitimate host-backed import. #371 then made not-host part of what an import is — imported_buft exists precisely so an import does not claim to be a host allocation — and once that is the design, the negative is the invariant. A test should hold the design, and the design changed. My #370 reading was about the flags; this one is about the buffer type, and the buffer type is what the bug was.

The failure this catches is the one that cost the afternoon: a from-pointer buffer arriving as a CPU buffer with its buft retagged. If that path ever comes back, is_host flips to true, and this assertion is the only thing in the tree that would say so.

One thing worth adding, because imported = true now covers two paths

After #374 there are two constructors setting imported = true, and they disagree on is_host:

path buft is_host
hrx_mapped_buffer_from_ptr (a mapped weight) mapped_buft true — deliberately, the scheduler's placement depends on it
an import imported_buft false — this assertion

So !is_host(imported) holds for the import path and would fail for the mapped path. The comment says "an import is NOT host-typed", which is right — but a reader who has just met imported = true in hrx_mapped_buffer_from_ptr can reasonably read this as a statement about imported, not about the import constructor. One clause closes it: "a mapped weight is also imported but keeps mapped_buft — this asserts the import path, where the destination buft is imported_buft." That is the difference between an invariant and a rule that a future constructor will trip over.

And the rationale is the right kind to restore

You did not just put the line back. The removed comment argued for the relaxation; the restored one records that the relaxation was a reading of #370, that #371 settled the question the other way, and what the property is now protecting. A test that silently regains an assertion is a regression waiting to be re-removed by the next person who reads the old comment — replacing the argument as well as the line is what stops the loop.

Ready at 97a45e6.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 97a45e6

The restored assertion is true at this head, and I checked that against the code rather than the comment:

$ hrx-buffer-test                       (ticket pi-prs2-377)
HRX buffer test summary: 9 passed, 0 failed, 0 known-fail, 0 unexpected-pass, 9 total     rc 0

backend-context.h:69        ggml_backend_buffer_type  imported_buft = {};        ← the buft the comment cites
model-buffer-from-mmap.cpp:60  bool imported_type_is_host(...) { return false; }
                          :138  .is_host = imported_type_is_host

So !ggml_backend_buffer_is_host(imported) holds — 9/0 with it restored — and it holds for the reason the comment gives: an import now has its own buft whose is_host is false. If it ever becomes host-typed again, the retagging path is back, which is exactly the thing worth an assertion.

And the comment's account of the sequence is accurate, including my own part in it. At engine#370 I argued the removal was right, on the reading that imported and is_host are independent flags (backend-buffer-binding.cpp:85, :111 — both real, I quoted them at the time). That reading was correct for the world then: the from-pointer buffer reached the case as a retagged CPU buffer, whose is_host is genuinely true, so the negative asserted something the old model legitimately violated. #371 and #374 changed the model — an import is now a window on an allocation with its own buft — and in that world the negative is the invariant, not an implementation detail. #370's comment argued against it from the old model; this one argues for it from the new one.

Which is the useful property of this PR: it does not just re-add a line, it rewrites the reason, and the reason is what would have made the next reader keep the removal. Comment-only otherwise (+8/−6, one file), so nothing about the mechanism changes.

Nothing else to ask for: 9/0, one file, an assertion that is true at the head, and a comment that cites two places in the code where a reader can confirm it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant