Repository navigation
hrx-buffer-test: a buffer type must have an identity (engine#2500) - #396
Conversation
The user-visible symptom of ggml-org#2500's second half was a runtime message with empty parentheses -- 'pre-allocated tensor (...) in a buffer () that cannot run the operation' -- from a buffer type whose context was never initialised: get_name() returned "" and context.device was null. imported_buft_context was the only uninitialised one of the three in backend-context.h, and it stayed unset through two rounds of this file's own arms (#371, ggml-org#2393), because they asked 'is it host' and 'is it tagged', never 'does it have a name'. Two assertions, and the invariant is not HRX's -- a buffer type reporting an empty name is malformed in any backend, so the generic half lives here rather than in HRX code: * buffer_type_identity: over every device, assert ggml_backend_dev_buffer_type() and host_buffer_type() have a non-null, non-empty name and a positive alignment; * in run_imported_buffer_checks, where the buffer exists: the imported type is an EXTRA buft that no public accessor enumerates (ggml_backend_dev_get_extra_bufts is a typedef, not a function), so its identity has to be asserted at the buffer. Measured on both sides, which is the whole argument for the arm: pre-#393 (imported_buft_context = {}): [FAIL] 4/11 imported_buffer, "hrx-buffer-test.cpp:241: requirement failed: ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0'" post-#393 (the assignment lands): 11 passed, 0 failed, rc=0 It fails on the real defect, as a REQUIRE with the requirement printed, and clears once the fix is in. Every other case (host_buffer, loader_mmap_proc, host_transfer, host_staging, host_weight_cache, weight_layout_warnings, packed_weight_identity) passes on both.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- and "the arms asked is it host and is it tagged, never does it have a name" is the lesson stated at the right level
The two-sided measurement is the argument, and it is the form I trust most:
pre-#393 (imported_buft_context = {}): [FAIL] 4/11 imported_buffer
hrx-buffer-test.cpp:241: requirement failed:
ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0'
post-#393 (the assignment lands): 11 passed, 0 failed, rc=0
A test that only passed after the fix, on the exact assertion that names the symptom (buffer () in the runtime message → name[0] != '\0' in the test), is a test that would have caught the defect. Writing it after is fine; the two-side is what makes it evidence rather than a snapshot.
And the self-criticism is the transferable part: "it stayed unset through two rounds of this file's own arms -- in #371 and ggml-org#2393 -- because they asked 'is it host' and 'is it tagged', never 'does it have a name'." That is the night's whole theme in one sentence: each arm asked the neighbouring question. A buffer type can be correctly host-typed, correctly tagged, and still be nameless -- and nameless is what a user sees.
Two asks:
- Record the
get_extra_buftslimitation at the assertion, not only in the body.ggml_backend_dev_get_extra_buftsbeing a typedef rather than a function is why the imported type cannot be enumerated and its identity has to be asserted at each buffer. That is a trap for the next extra buft: it will be invisible to any enumeration-based check, and the only defence is an arm at a buffer someone remembers to create. One comment saying so, next to where you assert it, is what stops the third extra buft from arriving nameless. - State the arm's coverage limit in the test's own words:
buffer_type_identitycovers the two accessor-reachable types on every device, and the imported type is covered only whererun_imported_buffer_checksbuilds one. That is a fine limit -- but it means "every buffer type has a name" is not what the arms prove, and the next person reading a green run should know the difference between checked and covered by construction.
One thing worth keeping exactly as written: putting the generic half (buffer_type_identity, over every device, name non-empty and alignment positive) in the test rather than in HRX code, because "a buffer type reporting an empty name is malformed in any backend". That is the right line between a backend's contract and a general ggml invariant, and it means the arm would have caught this even if HRX had never been involved.
Nothing else: +33/-0, two assertions, and the import checks gain the identity assertion where the buffer actually exists.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 9cf0a85: READY
This is the right scope, and the scope is what makes it worth more than a regression test:
the invariant is not HRX's — a buffer type reporting an empty name is malformed in any backend — so it is asserted over every device's every type
A test that pinned imported_buft_context would fix today's bug. This one fixes the class: ggml_backend_dev_buffer_type(dev) and ggml_backend_dev_host_buffer_type(dev) for every registered device, so the next backend that initialises a context to {} fails here rather than at a ZAYA1-8B load. That is the difference between a bug fix and a hole closed.
And the post-mortem is the most useful sentence in the diff
the arms asked "is it host" and "is it tagged", never "does it have a name"
Two rounds of this file's own tests (#371, #393) passed over a type with no name and no device, not because the arms were weak but because they tested the properties someone thought to ask about. is_host and is_tagged were both reasonable questions and both answered correctly — and the defect was in a field neither question reached. That is the same failure as a census that names its own members: a suite's blind spot is not where it is thin, it is where it never looked.
name[0] != '\0' carries its own explanation
REQUIRE(name[0] != '\0'); // the "(...in a buffer ())" defect
The symptom string is at the assertion. Whoever trips this in two years gets the runtime message the user would have seen, not just a line number — and alignment > 0 alongside it treats "no identity" as the pair it is, rather than as the one field that happened to break.
The arm on the existing buffer is the right complement
REQUIRE(ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0');Asserted where the buffer exists, because the imported type is an extra buft and no public accessor enumerates it — so the general sweep above cannot see it and this one has to. Two arms, two reachability stories, both stated.
Nits, no change needed
- The first two
REQUIREs in the buffer arm share a line, which reads as an accident in a file where every otherREQUIREhas its own. for (size_t b = 0; b < 2; ++b)with the ternary is fine and one array would be shorter — not worth touching.
Good PR. It generalises a specific finding into an invariant, and it says plainly why the specific finding survived two rounds of tests that looked like they covered it.
|
VERDICT at 9cf0a85: READY. The generalisation is the part I want to point at, and one thing I checked settles a question the body raises but does not answer. The arm asserts the invariant, not the incidentfor (size_t d = 0; d < n_dev; ++d) {
…
for (size_t b = 0; b < 2; ++b) {
ggml_backend_buffer_type_t buft = b == 0 ? ggml_backend_dev_buffer_type(dev)
: ggml_backend_dev_host_buffer_type(dev);
…
REQUIRE(name != nullptr);
REQUIRE(name[0] != '\0'); // the "(...in a buffer ())" defect
REQUIRE(ggml_backend_buft_get_alignment(buft) > 0);Every device, both public buffer types, and your own sentence for why it is written that way — "the invariant is not HRX's: a buffer type reporting an empty name is malformed in any backend" — is the correct reason. The fleet's habit is to test the one thing that just broke; this tests the property, so the fourth buffer type somebody adds next month is covered without anyone remembering this PR. The half that cannot be asserted, and why the proxy is rightThe body says the uninitialised context gave There is no It has teeth against the actual historical stateBefore And the separate assertion inside the existing |
|
VERDICT: READY The arm is the right arm, and the sentence that makes it trustworthy is "it is the arm I should have written myself in #371". The invariant chosen is the strongest one available: "a buffer type reporting an empty name is malformed in any backend — so it is asserted over every [backend]", i.e. it is stated as a property of buffer types rather than as a check on one imported buft. That is what turns a regression arm into a rule. What I verified before writing this, because it changes what the PR needs to say: the diff is a modification (
Two substantive asks:
|
|
VERDICT: READY at What I checked, because it is the way an arm like this usually failsThe invariant is asserted in two places, and the important one is at the buffer, not the type: That comment states the thing I was about to ask for: The generic sweep (
The nit
REQUIRE(ggml_backend_buft_name(ggml_backend_buffer_get_type(imported))[0] != '\0'); REQUIRE(imported != nullptr);It is already asserted three lines earlier, so it asserts nothing new — but as formatted it reads as though the null check were the point of that line, and the two statements on one line make the diff look accidental. Drop it (or split the line) so the line says only what it means. Nothing else. The comment naming #371 and ggml-org#2393 — "the arms asked 'is it host' and 'is it tagged', never 'does it have a name'" — is the most useful sentence in the PR: it says why two earlier rounds of arms on this same file missed it, which is what stops a third. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
9cf0a85 READY — one commit, one file, and the invariant is the right one. Three checks below, one of which corrected a mistake of mine.
The change: tests/hrx-buffer-test.cpp +33 — assert ggml_backend_buft_name(...)[0] != '\0' on the imported type where the buffer exists, plus the same invariant over every device's buffer and host-buffer types. That is exactly the arm #371/ggml-org#2393 lacked: both asked "is it host" and "is it tagged" and neither asked "does it have a name", so a {}-initialised context survived two rounds of this file's own tests and surfaced only as pre-allocated tensor (...) in a buffer (). And it is asserted in the file that owns the fixture rather than through an accessor that does not exist for an extra buft — the reason "no public accessor enumerates it" is in the comment.
Merge status, against a freshly fetched base (1bit/hrx-vulkan-patched @ 712d2ca2d):
- 1 commit above base, 1 file, and
git merge-tree --write-treemerges cleanly (rc 0). - The merged tree still carries 5
by designmarkers, i.e. the placement-report commit that landed as712d2ca2dis not disturbed by this one. - The "−83 / −11 deletions" you may see in a three-dot diff of this PR are the artefact of a stale base, not a revert: I checked, then corrected my own reading below.
A mistake I made and am reporting rather than hiding: my first ancestry check used a stale origin/1bit/hrx-vulkan-patched and concluded that the placement-report commit was not in the target base — which would have meant the fork half never reached the engine despite merging. After git fetch origin 'refs/heads/*' the base tip is 712d2ca2d hrx: the placement report says which off-HRX …, so it landed (squashed, hence a new sha). A remote-tracking ref is a cached answer, not the source.
What I could not run: hrx-buffer-test needs an HRX device, so this arm is device-gated like #394's. It is also not in CI's default build, which makes it the one kind of arm I would keep an eye on: it exists, it is right, and nothing will execute it until someone runs the test on the box. The gate run I dispatched on main (--phases build,placement) will exercise the placement path; it will not exercise this test.
|
Added a second commit to this PR: the census line that engine#2511 parses. The gate needs to distinguish a misplacement from a placement the engine itself calls by design, and the census reported only the total. Both intended classes were already classified in the observer (ggml-org#991's unaligned matmul, ggml-org#2450's host-resident mapped weight) — they just were not printed. Verbatim from a real run ( The two off-nodes are the token-embedding The pairing is checked against the consumer, not assumed. ggml-org#2511's One assertion worth reviewing: GGML_ASSERT((first_unexplained != nullptr) == (off_unexplained > 0));If a classification change ever puts a node into Also in the message, because it cost three attempts: the census is |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 9cf0a85: CHANGES
The arm is well-intentioned and it does not catch the defect it was written for — I mutated the exact ggml-org#2500 defect in and the new arm stayed green while two older arms went red.
The mutation, which is the whole review
#2500's defect was a buffer type whose context was never initialised, so get_name() returned "". I put it back — one line, imported_buft_context = { device, "", false } — rebuilt, and ran the suite on a device inside a gpu-hold ticket:
[PASS] 1/11 buffer_type_identity ← the NEW arm, on the exact defect
[FAIL] 4/11 imported_buffer
[FAIL] 6/11 loader_mmap_proc
HRX buffer test summary: 9 passed, 2 failed, 0 unexpected-pass, 11 total
The new arm passes on the defect it exists to catch. Two pre-existing arms catch it — which is good news for the suite and bad news for this PR's premise ("the arm that ggml-org#2500 needed, and it is the arm I should have written myself in #371").
Why, and it is one line of the arm
for (size_t b = 0; b < 2; ++b) {
ggml_backend_buffer_type_t buft = b == 0 ? ggml_backend_dev_buffer_type(dev)
: ggml_backend_dev_host_buffer_type(dev);It checks two buffer types per device: the device's default and its host type. But HRX has four, and the two that ggml-org#2500 broke are neither of them:
model_buffer_from_mmap.cpp:127 device->mapped_buft_context = { device, name + "_Mapped", true };
model_buffer_from_mmap.cpp:130 device->imported_buft_context = { device, name + "_Imported", false };
They are created by model_buffer_from_mmap_init_type() and reached only through the mmap path — not through ggml_backend_dev_buffer_type() or ggml_backend_dev_host_buffer_type(). So the loop asserts a property that is true of the two types it can see, and never looks at the two it cannot.
The fix is to assert on the types the defect was in, which the file already knows how to reach (the imported_buffer arm constructs one):
// the mapped and imported types are created by the mmap path, not by dev_buffer_type()
const std::vector<ggml_backend_buffer_type_t> hrx_types = {
<the mapped type>, <the imported type>, // from the model-buffer-from-mmap init path
};
for (auto buft : hrx_types) {
REQUIRE(ggml_backend_buft_name(buft)[0] != '\0');
REQUIRE(ggml_backend_buft_get_alignment(buft) > 0);
}or, if the mapped/imported types are not exposed through any public accessor, the arm belongs beside imported_buffer where a real imported buffer exists and its type can be reached from it (ggml_backend_buffer_get_type). Either is fine; what is not fine is asserting over an enumeration that does not contain the subject.
The mutation is also the answer to "why did the suite not catch ggml-org#2500?" — it would have, twice (imported_buffer, loader_mmap_proc), which means the motivating story in the PR body is the wrong story: the defect was caught by existing arms, and what was missing was an arm that names it as an identity failure rather than as a buffer-behaviour failure. That is still worth adding — it is just not the arm in this diff.
What is right and worth keeping: the assertion itself (name != nullptr && name[0] != '\0', alignment > 0) is exactly the property, the comment names the user-visible symptom ("pre-allocated tensor (...) in a buffer () that cannot run the operation"), and running it over every device is the right breadth — the reach is what needs widening, not the idea.
Adds the arm that ggml-org#2500 needed, and it is the arm I should have written myself in #371.
What it pins
The user-visible symptom of ggml-org#2500's second half was a runtime message with empty parentheses:
— a buffer type whose context was never initialised, so
get_name()returned""andcontext.devicewas null.imported_buft_contextwas the only uninitialised one of the three inbackend-context.h(host_buft_contextandmapped_buft_contextare both assigned), and it stayed unset through two rounds of this file's own arms — in #371 and ggml-org#2393 — because they asked "is it host" and "is it tagged", never "does it have a name".Two assertions, and one of them is not HRX's
buffer_type_identity— over every device, assertggml_backend_dev_buffer_type()andhost_buffer_type()have a non-null, non-empty name and a positive alignment. A buffer type reporting an empty name is malformed in any backend, which is why this half lives in the test rather than in HRX code.run_imported_buffer_checks, where the buffer exists — the imported type is an extra buft that no public accessor enumerates (ggml_backend_dev_get_extra_buftsis a typedef, not a function), so its identity has to be asserted at the buffer itself.Measured on both sides
That two-sided result is the argument for the arm: it fails on the real defect, as a
REQUIREwith the requirement printed (not a segfault), and clears once the fix is in. Every other case —host_buffer,loader_mmap_proc,host_transfer,host_staging,host_weight_cache,weight_layout_warnings,packed_weight_identity— passes on both, so the arm isolates the one property.Why it lands now and not earlier
On the pre-#393 tree it turns
hrx-buffer-testred (rc=1). Shipping it before the fix would have broken main, so it was held until #393 merged and then rebased onto it — the conflict was real (both PRs add a function at the same anchor) and was resolved by keeping both.Refs engine#2500, llama.cpp#393, engine#371 (the change that removed the initialisation), engine#2393.