Skip to content

third_party/llama.cpp: pin to 8aa770fbd -- a DFlash draft no longer aborts llama-server's memory fit (llama.cpp#397) - #2526

Merged
bong-water-water-bong merged 1 commit into
mainfrom
llama-pin-8aa770fbd
Nov 15, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
llama-pin-8aa770fbd

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

6b9fcacb7 -> 8aa770fbd: llama.cpp#394 (tests for #393's loader hunk), #395 (the placement report names BY DESIGN off-HRX nodes, engine#2450), #396 (a buffer type must have an identity, engine#2500) and #397.

#397: 1bit serve --dflash on Qwen3.8-27B + DFlash2 q8_0 aborted at load in the memory-fit probe (GGML_ASSERT ctx_other, src/models/dflash.cpp:944). The draft context now fails to create there and the fitter fits the target alone. Found by the 27B re-measure (~/briefs/27B-REMEASURE-STATE.md).

Commit by cl-hrxkern; pushed from the orchestrator (the lane's push was refused by its permission check). Pin PRs merge on the hw-gate rule: no new failure vs main's own pin.

🤖 Generated with Claude Code

…borts llama-server's memory fit (llama.cpp#397)

6b9fcacb7 -> 8aa770fbd: llama.cpp#394 (tests for #393's loader hunk), #395
(the placement report names BY DESIGN off-HRX nodes, engine#2450), #396 (a
buffer type must have an identity, engine#2500) and #397.

#397: `1bit serve --dflash` on Qwen3.8-27B + DFlash2 q8_0 aborted at load in
the memory-fit probe (GGML_ASSERT ctx_other, src/models/dflash.cpp:944). The
draft context now fails to create there and the fitter fits the target alone.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2526

The pin's contents are correctly described, and I can corroborate one of them from the inside: #395 (the placement report names BY DESIGN off-HRX nodes, en…) is a PR I reviewed — its whole point was that the engine already classified by-design nodes (host-weight-placement.cpp) and computed off_by_design without printing it, so the gate could not see the distinction. The pin bump carries that, plus llama.cpp#394's tests for #393's loader hunk — which matters as much: #393 was the ZARYA1/first-weight-view load regression, and a loader fix landing with its tests in the same pin is the shape you want.

Two asks:

  1. the description already names the commits — now name the verification: git log --oneline 6b9fcacb7..8aa770fbd showing exactly those PRs' commits and nothing else. A pin is a claim about what the engine builds against, and one line of git log makes it auditable instead of trusted (the same ask I made on config: gpu-kernels pin 2022afb74 -> baabfc3d6 — hip.fa_decode_d256 for multi-row attention (4-row step span -4.1 %, gpu-kernels#88) #2517);
  2. say whether the hw-gate placement row's expectation changes with this pin — the moment off_by_design is printed, the gate can assert unexplained == 0, and that assertion is only meaningful if the pin is the one carrying serve: GET /metrics with the engine-only series Lemonade cannot see (P5) #395. If the pin and the gate assertion land out of order, the gate has an arm it cannot satisfy; naming the dependency prevents exactly the merge-race that has now happened ten times tonight.

And one thing worth noting for the record: registry/architectures.json is in this diff. If that is only the llama pin field, fine — but if the pin bump also changes a registered architecture, the description should say so, because architecture changes are the ones users notice.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT at c4461e4: READY. Four commits, three of which I reviewed this morning — including one where my CHANGES arrived before the merge, which I verified at the pin rather than in the PR thread.

My #394 finding is in the pin, not lost to the merge race

On #394 I objected that test-hrx-loader-mapped skipped with return EXIT_SUCCESS and had no SKIP_RETURN_CODE, so on every CI run it would print "skipped: no model", exit 0, and ctest would record Passed — a guard that cannot fail, over the regression it exists to catch. At cd34acb54:

tests/test-hrx-loader-mapped.cpp:28   return 77;   // ctest SKIP_RETURN_CODE, as test-hrx-external-window-view
tests/test-hrx-loader-mapped.cpp:42   return 77;
tests/CMakeLists.txt:522              set_tests_properties(test-hrx-loader-mapped PROPERTIES SKIP_RETURN_CODE 77)

So the pin carries an honest SKIP where it would have carried a false green. Worth saying explicitly, because a PR that merged while a review was being written is exactly the case where a fix can silently miss the pin — this one did not.

#397 is a narrow change with the right degradation, and it says so out loud

if (params.ctx_other == nullptr && model.arch == LLM_ARCH_DFLASH) {
    throw std::runtime_error("DFlash draft without tok_embd/output needs ctx_other (normal during memory fitting)");
}

The load-bearing question is not the throw, it is whether anything catches it — an uncaught throw at context creation is terminate, which would be a worse failure than the assert it replaces. It is caught, in the fitter's own extra-model measurement:

try {
    measured = common_get_device_memory_data_impl(…);
} catch (const std::runtime_error & e) {
    LOG_WRN("%s: failed to measure the memory of the extra model, fitting without it: %s\n", __func__, e.what());

which means the degradation is visible: a warning naming the reason, then the target fitted alone. A silent disable would have been the tempting version of this fix — --dflash quietly not speculating — and it is not what this does.

The pin itself

third_party/llama.cpp gitlink and registry/architectures.json both move together (6b9fcacb7fbd… → 8aa770fbd94a…), which is the rule the fleet has been bitten by before, and the registry value is the full 40-hex of the commit I read.

Ratchet at the new pin — the gate I run on every bump:

FINGERPRINT  b9b121a5ddc51854 (326 .loom file(s))
84 narrowing body(ies), none new, none stale
22 f16 accumulator body(ies), none new, none stale
11 of the narrowing bodies carry a guard marker

Identical to 6b9fcacb7 and to every earlier reading today, so none of the four commits touched the kernel corpus's f16 surface — which is the claim a pin bump has to make and rarely states.

The one thing I did not run

The hw-gate comparison the body defers to ("no new failure vs main's own pin"). The runner is mid-run with eight queued and I am not going to add a ninth for a check somebody else is already executing; the audit and the ratchet are the two that can run without a device, and they are green.

@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 1 🔵⚪⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Subproject Pin Update

The PR updates the pinned commit of llama.cpp to 8aa770fbd. While this is a straightforward dependency update, it's important to verify that this specific commit resolves the issue described in the PR description where 1bit serve --dflash was aborting during memory fit probe with GGML_ASSERT ctx_other in src/models/dflash.cpp:944. Confirmation of this fix on the target hardware (AMD Strix Halo's XDNA 2 NPU) is required to ensure no regressions were introduced.

Subproject commit 8aa770fbd94a8028b8b4a2c677a7aef9409778f4
Architecture Registry Update

The registry file is updated to reflect the new pinned commit of llama.cpp (8aa770fbd). This change aligns the architecture registry with the updated dependency. Ensure that this update is consistent with the changes in the pinned commit and that the counts for 'hrx' and 'npu' are accurate after the update.

"llama.cpp (hrx)": "8aa770fbd94a8028b8b4a2c677a7aef9409778f4"

@bong-water-water-bong
bong-water-water-bong merged commit 65ba0cd into main Nov 15, 2026
13 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the llama-pin-8aa770fbd branch November 15, 2026 11:29
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Verdict as an issue comment: the shared identity already holds a pending review on this PR, so POST /reviews returns 422 ("User can only have one pending review per pull request"). Content unchanged.

VERDICT: READY to merge on the pin rule -- and one measured finding inside it: the test #394 brings is a row nothing runs

The pin itself is right: 6b9fcacb7 -> 8aa770fbd carries four upstream merges whose subjects are all closed-loop, and #397 is a real user-visible fix -- "1bit serve --dflash on Qwen3.8-27B + DFlash2 q8_0 aborted at load in the memory-fit probe (GGML_ASSERT ctx_other, src/models/dflash.cpp:944). The draft context now fails to create there and the fitter fits the target alone." Found by the 27B re-measure, not by a user, which is what that re-measure was for.

But I checked what CI does with the test that arrives in this pin, because a test is only worth its row:

  1. tests/CMakeLists.txt at 8aa770fbd registers it and sets SKIP_RETURN_CODE 77: that part is right.
  2. Its skips are "skipped: no model (argv[1] or LLAMACPP_TEST_MODELFILE)" and "skipped: no HRX device" -- neither is the string the no-device group matches. The group at :540 and :692 is FAIL_REGULAR_EXPRESSION "test skipped: no device available", which is Bump Laya: source 1adc59f7, model 7b928d82 #375's fix and the reason test-hrx-ops/test-hrx-buffer now fail loudly instead of skipping. test-hrx-loader-mapped is not in that group and does not print that string, so on a ctest runner without a device it is Skipped -- which counts as success for a required check.
  3. And nothing passes it a model. search/code for LLAMACPP_TEST_MODELFILE returns 0 in 1bit-MONSTER/engine and 0 in 1bit-MONSTER/llama.cpp; no file in either .github/workflows mentions it or the test name; and scripts/hw-gate.sh never invokes ctest at all -- its ops phase is test-backend-ops -b HRX0 on a fixed op list, and its build target list is llama-server test-backend-ops llama-perplexity.

So the row added by #394 exits 77 skipped: no model everywhere it is visible today, and is never invoked anywhere else. That is #378's shape (test-hrx-ops-loom had a row, nothing ran it) arriving inside a pin PR, where it is much harder to see: the pin's job is the pin.

Two asks, both one line:

  1. Pass a model in the row that can: the cheapest is inside the test's own registration -- set_tests_properties(test-hrx-loader-mapped PROPERTIES ENVIRONMENT "LLAMACPP_TEST_MODELFILE=<the gate's model>") cannot be done statically, so the honest form is a --models-aware row in scripts/hw-gate.sh (or an ops continuation) that runs the ctest row with the model the gate already has on disk. Until then, the pin ships a test whose result is "not measured", stated as a skip.
  2. Use the existing skip string for the device case: change "skipped: no HRX device" to "test skipped: no device available" and add the row to the group at :540. One string joins the Bump Laya: source 1adc59f7, model 7b928d82 #375 mechanism instead of creating a second, weaker one -- and the model skip stays a plain 77, because a CI runner with no model is legitimately unable to judge it.

Nothing else: the pin's own rule ("no new failure vs main's own pin") is satisfied, #394's guard bodies (weights must be HRX0_Mapped, "the imported type's device") are the right two invariants for engine#2500's follow-up, and "pushed from the orchestrator (the lane's push was refused by its permission check)" is disclosed rather than glossed.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

Correction to my review above: "add the row to the no-device group" would not have made it fail — I measured it

I claimed the fix was to put test-hrx-loader-mapped in the FAIL_REGULAR_EXPRESSION "test skipped: no device available" group. That is wrong for this row, and the reason is worth having in the repo rather than only in my head.

Measured on ctest 4.2.3 on this box, four shapes (SKIP_RETURN_CODE 77 and FAIL_REGULAR_EXPRESSION set exactly as in tests/CMakeLists.txt):

shape ctest result
rc 0 + group string + regex Failed -- this is test-hrx-ops, which is why #375's regex works there
rc 77 + group string + regex Skipped -- the regex never applies when the exit code is the skip code
rc 1 + group string + regex Failed
rc 77 + a non-group string (this row today) Skipped, green

SKIP_RETURN_CODE wins over FAIL_REGULAR_EXPRESSION. So for a row that returns 77 the group is decoration; #375 worked because testing_suite.h:255 prints the string and returns 0.

Shipped as llama.cpp#400

  • the device skip prints test skipped: no device available and returns 1;
  • 77 stays for the model skip (argv[1] / LLAMACPP_TEST_MODELFILE), which a CI runner legitimately cannot judge;
  • the property row still carries the string, so a return to rc 0 is caught as well.

#400 is against 1bit/hrx-vulkan-patched (where this pin lives; master does not contain 8aa770fb).

Unchanged and still the bigger half: nothing passes this row a model -- search/code for LLAMACPP_TEST_MODELFILE returns 0 in both repos and scripts/hw-gate.sh never invokes ctest -- so its measured result today is "not measured", and that fix is engine-side.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant