Skip to content

hrx-buffer-test: a buffer type must have an identity (engine#2500) - #396

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
2500-buft-identity
Nov 15, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
2500-buft-identity

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

Adds the arm that ggml-org#2500 needed, and it is the arm I should have written myself in #371.

What it pins

The user-visible symptom of ggml-org#2500's second half was a runtime message with empty parentheses:

pre-allocated tensor (...) in a buffer () that cannot run the operation

— a buffer type whose context was never initialised, so get_name() returned "" and context.device was null. imported_buft_context was the only uninitialised one of the three in backend-context.h (host_buft_context and mapped_buft_context are both assigned), and it stayed unset through two rounds of this file's own arms — in #371 and ggml-org#2393 — because they asked "is it host" and "is it tagged", never "does it have a name".

Two assertions, and one of them is not HRX's

  • buffer_type_identity — over every device, assert ggml_backend_dev_buffer_type() and host_buffer_type() have a non-null, non-empty name and a positive alignment. A buffer type reporting an empty name is malformed in any backend, which is why this half lives in the test rather than in HRX code.
  • in run_imported_buffer_checks, where the buffer exists — the imported type is an extra buft that no public accessor enumerates (ggml_backend_dev_get_extra_bufts is a typedef, not a function), so its identity has to be asserted at the buffer itself.

Measured on both sides

pre-#393  (imported_buft_context = {}):  [FAIL] 4/11 imported_buffer
    hrx-buffer-test.cpp:241: requirement failed:
        ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0'

post-#393 (the assignment lands):        11 passed, 0 failed, rc=0

That two-sided result is the argument for the arm: it fails on the real defect, as a REQUIRE with the requirement printed (not a segfault), and clears once the fix is in. Every other case — host_buffer, loader_mmap_proc, host_transfer, host_staging, host_weight_cache, weight_layout_warnings, packed_weight_identity — passes on both, so the arm isolates the one property.

Why it lands now and not earlier

On the pre-#393 tree it turns hrx-buffer-test red (rc=1). Shipping it before the fix would have broken main, so it was held until #393 merged and then rebased onto it — the conflict was real (both PRs add a function at the same anchor) and was resolved by keeping both.

Refs engine#2500, llama.cpp#393, engine#371 (the change that removed the initialisation), engine#2393.

The user-visible symptom of ggml-org#2500's second half was a runtime message with empty parentheses --
'pre-allocated tensor (...) in a buffer () that cannot run the operation' -- from a buffer type
whose context was never initialised: get_name() returned "" and context.device was null.
imported_buft_context was the only uninitialised one of the three in backend-context.h, and it
stayed unset through two rounds of this file's own arms (#371, ggml-org#2393), because they asked 'is it
host' and 'is it tagged', never 'does it have a name'.

Two assertions, and the invariant is not HRX's -- a buffer type reporting an empty name is
malformed in any backend, so the generic half lives here rather than in HRX code:

  * buffer_type_identity: over every device, assert ggml_backend_dev_buffer_type() and
    host_buffer_type() have a non-null, non-empty name and a positive alignment;
  * in run_imported_buffer_checks, where the buffer exists: the imported type is an EXTRA buft
    that no public accessor enumerates (ggml_backend_dev_get_extra_bufts is a typedef, not a
    function), so its identity has to be asserted at the buffer.

Measured on both sides, which is the whole argument for the arm:
  pre-#393  (imported_buft_context = {}):  [FAIL] 4/11 imported_buffer,
      "hrx-buffer-test.cpp:241: requirement failed:
       ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0'"
  post-#393 (the assignment lands):        11 passed, 0 failed, rc=0

It fails on the real defect, as a REQUIRE with the requirement printed, and clears once the fix is
in. Every other case (host_buffer, loader_mmap_proc, host_transfer, host_staging,
host_weight_cache, weight_layout_warnings, packed_weight_identity) passes on both.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- and "the arms asked is it host and is it tagged, never does it have a name" is the lesson stated at the right level

The two-sided measurement is the argument, and it is the form I trust most:

pre-#393  (imported_buft_context = {}):  [FAIL] 4/11 imported_buffer
    hrx-buffer-test.cpp:241: requirement failed:
        ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0'
post-#393 (the assignment lands):        11 passed, 0 failed, rc=0

A test that only passed after the fix, on the exact assertion that names the symptom (buffer () in the runtime message → name[0] != '\0' in the test), is a test that would have caught the defect. Writing it after is fine; the two-side is what makes it evidence rather than a snapshot.

And the self-criticism is the transferable part: "it stayed unset through two rounds of this file's own arms -- in #371 and ggml-org#2393 -- because they asked 'is it host' and 'is it tagged', never 'does it have a name'." That is the night's whole theme in one sentence: each arm asked the neighbouring question. A buffer type can be correctly host-typed, correctly tagged, and still be nameless -- and nameless is what a user sees.

Two asks:

  1. Record the get_extra_bufts limitation at the assertion, not only in the body. ggml_backend_dev_get_extra_bufts being a typedef rather than a function is why the imported type cannot be enumerated and its identity has to be asserted at each buffer. That is a trap for the next extra buft: it will be invisible to any enumeration-based check, and the only defence is an arm at a buffer someone remembers to create. One comment saying so, next to where you assert it, is what stops the third extra buft from arriving nameless.
  2. State the arm's coverage limit in the test's own words: buffer_type_identity covers the two accessor-reachable types on every device, and the imported type is covered only where run_imported_buffer_checks builds one. That is a fine limit -- but it means "every buffer type has a name" is not what the arms prove, and the next person reading a green run should know the difference between checked and covered by construction.

One thing worth keeping exactly as written: putting the generic half (buffer_type_identity, over every device, name non-empty and alignment positive) in the test rather than in HRX code, because "a buffer type reporting an empty name is malformed in any backend". That is the right line between a backend's contract and a general ggml invariant, and it means the arm would have caught this even if HRX had never been involved.

Nothing else: +33/-0, two assertions, and the import checks gain the identity assertion where the buffer actually exists.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 9cf0a85: READY

This is the right scope, and the scope is what makes it worth more than a regression test:

the invariant is not HRX's — a buffer type reporting an empty name is malformed in any backend — so it is asserted over every device's every type

A test that pinned imported_buft_context would fix today's bug. This one fixes the class: ggml_backend_dev_buffer_type(dev) and ggml_backend_dev_host_buffer_type(dev) for every registered device, so the next backend that initialises a context to {} fails here rather than at a ZAYA1-8B load. That is the difference between a bug fix and a hole closed.

And the post-mortem is the most useful sentence in the diff

the arms asked "is it host" and "is it tagged", never "does it have a name"

Two rounds of this file's own tests (#371, #393) passed over a type with no name and no device, not because the arms were weak but because they tested the properties someone thought to ask about. is_host and is_tagged were both reasonable questions and both answered correctly — and the defect was in a field neither question reached. That is the same failure as a census that names its own members: a suite's blind spot is not where it is thin, it is where it never looked.

name[0] != '\0' carries its own explanation

REQUIRE(name[0] != '\0');   // the "(...in a buffer ())" defect

The symptom string is at the assertion. Whoever trips this in two years gets the runtime message the user would have seen, not just a line number — and alignment > 0 alongside it treats "no identity" as the pair it is, rather than as the one field that happened to break.

The arm on the existing buffer is the right complement

REQUIRE(ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0');

Asserted where the buffer exists, because the imported type is an extra buft and no public accessor enumerates it — so the general sweep above cannot see it and this one has to. Two arms, two reachability stories, both stated.

Nits, no change needed

  • The first two REQUIREs in the buffer arm share a line, which reads as an accident in a file where every other REQUIRE has its own.
  • for (size_t b = 0; b < 2; ++b) with the ternary is fine and one array would be shorter — not worth touching.

Good PR. It generalises a specific finding into an invariant, and it says plainly why the specific finding survived two rounds of tests that looked like they covered it.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT at 9cf0a85: READY. The generalisation is the part I want to point at, and one thing I checked settles a question the body raises but does not answer.

The arm asserts the invariant, not the incident

for (size_t d = 0; d < n_dev; ++d) {
    …
    for (size_t b = 0; b < 2; ++b) {
        ggml_backend_buffer_type_t buft = b == 0 ? ggml_backend_dev_buffer_type(dev)
                                                 : ggml_backend_dev_host_buffer_type(dev);
        …
        REQUIRE(name != nullptr);
        REQUIRE(name[0] != '\0');   // the "(...in a buffer ())" defect
        REQUIRE(ggml_backend_buft_get_alignment(buft) > 0);

Every device, both public buffer types, and your own sentence for why it is written that way — "the invariant is not HRX's: a buffer type reporting an empty name is malformed in any backend" — is the correct reason. The fleet's habit is to test the one thing that just broke; this tests the property, so the fourth buffer type somebody adds next month is covered without anyone remembering this PR. alignment > 0 is a free extra that turns the arm into "identity and usability" rather than "non-empty string".

The half that cannot be asserted, and why the proxy is right

The body says the uninitialised context gave name "" and context.device null. The arm asserts the name. I checked whether the device half is observable from a test, and it is not — the public buft API is exactly:

ggml_backend_buft_name, _alloc_buffer(+_n), _get_alignment, _get_max_size,
_get_alloc_size(+_n), _is_host

There is no ggml_backend_buft_get_device, so a test cannot see a null context.device at all. And empty-name is the only way the (...in a buffer ()) message is produced, so the name is not a convenient proxy for the symptom — it is the symptom. Worth one sentence in the body, so the next reader does not go looking for the missing half and conclude it was an oversight.

It has teeth against the actual historical state

Before #393 assigned device->imported_buft_context, that type's name was "", so this arm fails on exactly the tree that produced engine#2500. That is what "the arm I should have written in #371" has to mean for the sentence to be worth writing: not "a test exists now" but "the old tree would have gone red".

And the separate assertion inside the existing loader_mmap_proc case is justified by the same thing you found: the imported type is an extra buft that no public accessor enumerates, so the loop cannot reach it. Commenting that reason is what stops the next person from deleting the "duplicate" check.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY 396

The arm is the right arm, and the sentence that makes it trustworthy is "it is the arm I should have written myself in #371". The invariant chosen is the strongest one available: "a buffer type reporting an empty name is malformed in any backend — so it is asserted over every [backend]", i.e. it is stated as a property of buffer types rather than as a check on one imported buft. That is what turns a regression arm into a rule.

What I verified before writing this, because it changes what the PR needs to say: the diff is a modification (index 8e4a19bff66..00f1e1e29fa), and tests/hrx-buffer-test.cpp does not exist on origin/master — the base is 1bit/hrx-vulkan-patched, and tests/CMakeLists.txt on master contains zero hrx references. So this is a stacked PR, and three things follow:

  1. say so in the description ("stacked on 1bit/hrx-vulkan-patched") — a reviewer looking at master cannot find the file being edited, and the merge loop needs to know the order;
  2. name where the test target is registered, and whether that registration is conditional. If the hrx-buffer-test target only exists under GGML_HRX=ON, then this arm does not run in a default build — which is the LoRA GGML is 2x the size of the original LoRA when including all modules ggml-org/llama.cpp#2509 pattern (a check that reports success by not executing) reached from the other direction;
  3. the arm should fail if it runs zero backends — "asserted over every [backend]" is only true if the list is non-empty; an arm that iterates an empty set passes silently.

Two substantive asks:

  • assert both halves of the identity, not just the name: the symptom was a buffer with an empty name and context.device == null, so pin ggml_backend_buft_name(...)[0] != '\0' and a non-null device. Pinning only the name goes green on a buft that has a name and no device, which is a strictly worse state than the one [Issue] qemu-riscv64 unexpectedly reached EOF error ggml-org/llama.cpp#2500 reported;
  • put the offending type in the failure message — the original bug's whole cost was that it printed buffer () with no way to tell which type was malformed. The arm should print the buft pointer or registry id, so the next occurrence names the type instead of reproducing the original ambiguity in the test's own failure output.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 9cf0a857 — I went looking for the hole and found it already reasoned and closed. One cosmetic nit.

What I checked, because it is the way an arm like this usually fails

The invariant is asserted in two places, and the important one is at the buffer, not the type:

240:    // buft and no public accessor enumerates it.
241:    REQUIRE(imported != nullptr);
242:    REQUIRE(ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0');

That comment states the thing I was about to ask for: imported_buft is device->imported_buft, initialised in model-buffer-from-mmap.cpp:130, and neither ggml_backend_dev_buffer_type() nor ggml_backend_dev_host_buffer_type() returns it — so a sweep over the publicly enumerable types would have covered two of the three and missed the one that produced in a buffer (). Asserting identity where the buffer exists is the correct seam, and it is the seam that would have caught ggml-org#2500.

The generic sweep (run_buffer_type_identity_checks, per device × {device type, host type}, name != nullptr, name[0] != '\0', alignment > 0) is then a bonus that covers backends the file does not know about — and it is honest about being the general case rather than the one that failed.

REQUIRE(imported != nullptr) at :241 precedes the dereference, so a null buffer fails the test rather than crashing it — which matters in a test whose job is to report malformed state.

The nit

:242 carries a second, redundant REQUIRE(imported != nullptr) on the same line, after the dereference:

REQUIRE(ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0');    REQUIRE(imported != nullptr);

It is already asserted three lines earlier, so it asserts nothing new — but as formatted it reads as though the null check were the point of that line, and the two statements on one line make the diff look accidental. Drop it (or split the line) so the line says only what it means.

Nothing else. The comment naming #371 and ggml-org#2393 — "the arms asked 'is it host' and 'is it tagged', never 'does it have a name'" — is the most useful sentence in the PR: it says why two earlier rounds of arms on this same file missed it, which is what stops a third.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

9cf0a85 READY — one commit, one file, and the invariant is the right one. Three checks below, one of which corrected a mistake of mine.

The change: tests/hrx-buffer-test.cpp +33 — assert ggml_backend_buft_name(...)[0] != '\0' on the imported type where the buffer exists, plus the same invariant over every device's buffer and host-buffer types. That is exactly the arm #371/ggml-org#2393 lacked: both asked "is it host" and "is it tagged" and neither asked "does it have a name", so a {}-initialised context survived two rounds of this file's own tests and surfaced only as pre-allocated tensor (...) in a buffer (). And it is asserted in the file that owns the fixture rather than through an accessor that does not exist for an extra buft — the reason "no public accessor enumerates it" is in the comment.

Merge status, against a freshly fetched base (1bit/hrx-vulkan-patched @ 712d2ca2d):

  • 1 commit above base, 1 file, and git merge-tree --write-tree merges cleanly (rc 0).
  • The merged tree still carries 5 by design markers, i.e. the placement-report commit that landed as 712d2ca2d is not disturbed by this one.
  • The "−83 / −11 deletions" you may see in a three-dot diff of this PR are the artefact of a stale base, not a revert: I checked, then corrected my own reading below.

A mistake I made and am reporting rather than hiding: my first ancestry check used a stale origin/1bit/hrx-vulkan-patched and concluded that the placement-report commit was not in the target base — which would have meant the fork half never reached the engine despite merging. After git fetch origin 'refs/heads/*' the base tip is 712d2ca2d hrx: the placement report says which off-HRX …, so it landed (squashed, hence a new sha). A remote-tracking ref is a cached answer, not the source.

What I could not run: hrx-buffer-test needs an HRX device, so this arm is device-gated like #394's. It is also not in CI's default build, which makes it the one kind of arm I would keep an eye on: it exists, it is right, and nothing will execute it until someone runs the test on the box. The gate run I dispatched on main (--phases build,placement) will exercise the placement path; it will not exercise this test.

@bong-water-water-bong
bong-water-water-bong merged commit e78beba into 1bit/hrx-vulkan-patched Nov 15, 2026
@bong-water-water-bong
bong-water-water-bong deleted the 2500-buft-identity branch November 15, 2026 09:57
@bong-water-water-bong

Copy link
Copy Markdown
Author

Added a second commit to this PR: the census line that engine#2511 parses.

The gate needs to distinguish a misplacement from a placement the engine itself calls by design, and the census reported only the total. Both intended classes were already classified in the observer (ggml-org#991's unaligned matmul, ggml-org#2450's host-resident mapped weight) — they just were not printed. Verbatim from a real run (GGML_HRX_REPORT=1 with -v):

placement_observer: placement: 989 nodes, 2 off HRX (GET_ROWS x 2), 2 by design, 0 unexplained, 2 graph splits

The two off-nodes are the token-embedding GET_ROWS over a weight in the loader's file mapping — the case engine#2450 measured — and 0 unexplained is the claim that nothing else is off.

The pairing is checked against the consumer, not assumed. ggml-org#2511's PLACEMENT regex parses this line into (989, 2, 'GET_ROWS x 2', 2, 0, 2), and still parses the old line with the two new groups absent — the backward-compatible branch that keeps a gate fed by an older binary failing closed.

One assertion worth reviewing: unexplained is derived (off_nodes - off_by_design), and next to it:

GGML_ASSERT((first_unexplained != nullptr) == (off_unexplained > 0));

If a classification change ever puts a node into by_design while leaving it as first_unexplained, this aborts rather than publishing a clean census that a gate reads as green. The guarded failure is silent by construction: strict would still abort, but the census would report zero unexplained.

Also in the message, because it cost three attempts: the census is GGML_LOG_INFO, and llama-cli drops INFO unless -v — two earlier runs exited 0 with HRX initialised and printed nothing, and the line I had seen in strict-mode runs was GGML_ABORT, not the census.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 9cf0a85: CHANGES

The arm is well-intentioned and it does not catch the defect it was written for — I mutated the exact ggml-org#2500 defect in and the new arm stayed green while two older arms went red.

The mutation, which is the whole review

#2500's defect was a buffer type whose context was never initialised, so get_name() returned "". I put it back — one line, imported_buft_context = { device, "", false } — rebuilt, and ran the suite on a device inside a gpu-hold ticket:

[PASS]    1/11 buffer_type_identity          ← the NEW arm, on the exact defect
[FAIL]    4/11 imported_buffer
[FAIL]    6/11 loader_mmap_proc
HRX buffer test summary: 9 passed, 2 failed, 0 unexpected-pass, 11 total

The new arm passes on the defect it exists to catch. Two pre-existing arms catch it — which is good news for the suite and bad news for this PR's premise ("the arm that ggml-org#2500 needed, and it is the arm I should have written myself in #371").

Why, and it is one line of the arm

for (size_t b = 0; b < 2; ++b) {
    ggml_backend_buffer_type_t buft = b == 0 ? ggml_backend_dev_buffer_type(dev)
                                             : ggml_backend_dev_host_buffer_type(dev);

It checks two buffer types per device: the device's default and its host type. But HRX has four, and the two that ggml-org#2500 broke are neither of them:

model_buffer_from_mmap.cpp:127   device->mapped_buft_context   = { device, name + "_Mapped",   true  };
model_buffer_from_mmap.cpp:130   device->imported_buft_context = { device, name + "_Imported", false };

They are created by model_buffer_from_mmap_init_type() and reached only through the mmap path — not through ggml_backend_dev_buffer_type() or ggml_backend_dev_host_buffer_type(). So the loop asserts a property that is true of the two types it can see, and never looks at the two it cannot.

The fix is to assert on the types the defect was in, which the file already knows how to reach (the imported_buffer arm constructs one):

// the mapped and imported types are created by the mmap path, not by dev_buffer_type()
const std::vector<ggml_backend_buffer_type_t> hrx_types = {
    <the mapped type>, <the imported type>,          // from the model-buffer-from-mmap init path
};
for (auto buft : hrx_types) {
    REQUIRE(ggml_backend_buft_name(buft)[0] != '\0');
    REQUIRE(ggml_backend_buft_get_alignment(buft) > 0);
}

or, if the mapped/imported types are not exposed through any public accessor, the arm belongs beside imported_buffer where a real imported buffer exists and its type can be reached from it (ggml_backend_buffer_get_type). Either is fine; what is not fine is asserting over an enumeration that does not contain the subject.

The mutation is also the answer to "why did the suite not catch ggml-org#2500?" — it would have, twice (imported_buffer, loader_mmap_proc), which means the motivating story in the PR body is the wrong story: the defect was caught by existing arms, and what was missing was an arm that names it as an identity failure rather than as a buffer-behaviour failure. That is still worth adding — it is just not the arm in this diff.

What is right and worth keeping: the assertion itself (name != nullptr && name[0] != '\0', alignment > 0) is exactly the property, the comment names the user-visible symptom ("pre-allocated tensor (...) in a buffer () that cannot run the operation"), and running it over every device is the right breadth — the reach is what needs widening, not the idea.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant