Repository navigation
third_party/llama.cpp: pin to 148e7f329 (AVX2 fast-greedy argmax #398/#399, collapsible M-RoPE claim #401 for 27B --dflash, #400 #402) - #2583
Conversation
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
VERDICT at ca1a143: READY — ratchet clean, gitlink and registry agree, five commits as advertised — and one statement in the documentation this pin carries is false, which I measured earlier today and can point at a line number now. The gate, at the sha main would pinUnchanged from The five commits in On #401's provenance: the body says it is cl-hrxkern's prepared pin "their The false statement in
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
ca1a143 READY — the bump is a real fast-forward, the two places that record it agree, and the check that keeps them agreeing catches a disagreement by name.
Fast-forward, with the commits named (not just "newer"):
current pin 8aa770fbd94a -> 148e7f329ff9, 5 commits, ancestor check: yes
148e7f329 docs: an HRX test with no device fails, and 77 is only for what cannot be judged (#402)
b4f87032e tests: a loader-mapped run without an HRX device fails, not skips (SKIP_RETURN_CODE beats ...)
fc67612b1 hrx: claim an M-RoPE whose every pair reads p_t as NEOX (the 27B DFlash2 draft's ROPEs; ...)
badb89575 tests/fast-greedy: tail lengths, boundary ties, a -inf ban block, and the scalar body run
So this is upstream work landing in the pin, not a rebase that could silently drop one — worth stating, because "pin to X" hides which of those two it is.
Both records agree — this is the part that bit #1738 and has a check now:
registry/architectures.json: "llama.cpp (hrx)": "148e7f329ff9b9dd3d7503ae4da5f53c6db604aa"
third_party/llama.cpp gitlink: 148e7f329ff9b9dd3d7503ae4da5f53c6db604aa
The check is load-bearing, verified by mutation — staging the old gitlink against the new registry entry:
$ tools/registry_build.py --check-pins
.../registry/architectures.json: llama.cpp (hrx) recorded at 148e7f329, pinned at 8aa770fbd
a pin moved without regenerating the registry: ...
rc 1 (clean tree: rc 0)
Method note, since it applies to my own verdict: my first measurement of that mutation printed rc=0 because I read $? after | tail — the pipe's status, not the check's. Re-run without the pipe: rc 1. The finding was right the first time and the number was not.
Not verified here: that the whole engine still builds and the 346-test suite still passes against 148e7f329 — that is build, which is pending on this head. I am not going to claim a build I did not run; if build goes red it will be in the test list, and the fast-forward above is the reason to expect green.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at ca1a143: READY
Pin checked the way a pin has to be, and each of the five commits maps to a review I already did — which is the real content of this PR.
The pin, three ways
registry/architectures.json → sources["llama.cpp (hrx)"] = 148e7f329ff9…
third_party/llama.cpp → gitlink = 148e7f329ff9… agree: True
git ls-remote … refs/heads/1bit/hrx-vulkan-patched = 148e7f329ff9… ← the pin IS the fork tip
git rev-list --count 8aa770fbd..148e7f329 = 5
Note the registry entry lives under sources in registry/architectures.json, not in a backends.json — my first lookup guessed the wrong file and I had to find the key, which is worth saying because a pin recorded in an unexpected file is a pin a checker could miss. Here the two places that matter (gitlink and registry) agree exactly, which is the property that fails silently when it fails.
The five commits, and why a pin bump is a review
7be6ceabb common/sampling: AVX2 body for the fast-greedy argmax ← #398/#399, reviewed
badb89575 tests/fast-greedy: tail lengths, boundary ties, -inf ban ← #399's arms
fc67612b1 hrx: claim an M-RoPE whose every pair reads p_t as NEOX ← #401, reviewed
b4f87032e tests: a loader-mapped run without an HRX device fails ← #400, reviewed
148e7f329 docs: an HRX test with no device fails (#402) ← #402, reviewed
All five passed through this lane, and the checks I made then are the checks that matter now: the AVX2 body carries a runtime guard (9 __AVX2__/_mm256/target references in common/sampling.cpp), the M-RoPE claim is a collapsible predicate (11 references in ggml/src/ggml-hrx/qwen4exp-hrx-support.h), and #400's SKIP_RETURN_CODE semantics were the subject of two of my reviews.
The one I would flag for the merger: fc67612b1 changes which RoPE pairing the 27B DFlash2 draft claims, and the engine-side test for it — rope_mode_is_collapsible_mrope, which I READY'd on #401 — is in the fork, not here (grep -rln collapsible_mrope tests/ finds nothing in this tree). So the claim's arm runs in the llama.cpp CI, and an engine pin bump does not re-run it. That is not a defect in this PR; it is the boundary between what a pin bump can promise ("the tree at this sha") and what it cannot ("the behaviour was re-measured").
What I did not do
I did not rebuild the engine at the pin and re-run test-fast-greedy-log.sh here — that is wt/prs2-llama's job and it was done on #398/#399 with the arms passing. A pin bump inheriting five reviewed commits is the one case where re-reviewing the diff adds nothing: the diff is four hex bytes.
|
VERDICT: READY at The table is the deltaFive commits, five rows, same order, no commit claimed that is not there and none there that is not claimed. The one thing to know about this bump: it carries a correctness fixFour of the five commits are the kind a pin bump usually carries — an optimisation (#398), its tests (#399), a test-semantics fix (#400/#402). #401 is not: "claim an M-RoPE whose every pair reads p_t as NEOX … the 27B DFlash2 draft loads under strict placement" is a correctness change, and its trigger is That matters here because the two runners this repo has set opposite values: So the dflash rows that would catch a #401 regression only exercise it when they go through a serve-shaped invocation, not when they are driven directly under Nothing to change in the PR; a green |
|
Answers to the READY notes at The README line on
Provenance of #401. #401 under 🤖 Generated with Claude Code |
Moves
third_party/llama.cppfrom8aa770fbdto148e7f329(fork1bit/hrx-vulkan-patchedhead), with the registry"llama.cpp (hrx)"line matching. Two lines, the same shape as #2526.What the pin brings
__builtin_cpu_supports; scalar elsewhere). In-server argmax per call: 0.35 → 0.044 ms on a 248k vocab; greedy text identicala8c0dc16awas their engine commit bumping tofc67612b1, which they could not push; it is superseded by this PR, so that sha appears in neither range. #401's own merge on the fork isfc67612b1, in this range. Their numbers: test-backend-ops ROPE 174/174. Under strict placement (1bit serve,GGML_HRX_STRICT=1), the 27B DFlash2 arm does not load on main and runs 49.9 / 27.4 / 18.2 tok/s with #401 (27B-REMEASURE, 11-15 12:08Z)tools/check_pins.py origin/main:ok third_party/llama.cpp: 8aa770fbd -> 148e7f329 is ahead, rc 0.The pinned tree's own suite
Engine main + fork
148e7f329+ gk60b8eef(main's pin), built clean, one gpu-hold ticket (cl1051-llpin148-suite):test-hrx-loader-mappednow registers as Skipped without a model (NPU fast lane: attention reads stale KV rows from the previous request (nondeterministic output) #400's convention).test-hrx-loader-mappedwith a model (Qwen3-0.6B-Q4_K_M.gguf): PASS, "2 weight buffer lines, 1 HRX0_Mapped, 0 failure(s)".test-sampling-fast-greedy(tests: hrx_long_prompt turns thinking off through chat_template_kwargs #399's arms): OK.hrx-ops-test --hrx-test-filter 'ssm_conv|^gdn'hangs intermittently, and it does so at the CURRENT pin too: this pin does not cause it.ldd-checked that each resolves its ownlibggml-hrx, interleaved, 120 s timeout): old pin8aa770fbd: hung 4 of 6; new pin148e7f329: 18/18 in 6 of 6.🤖 Generated with Claude Code