Repository navigation
third_party/llama.cpp: pin to f3d177a40 — two sampling behaviour changes (with their offline test), one fixture, one test assertion - #2452
Conversation
…nges, one fixture, one test assertion Four commits on the fork head c84f0fc6e, six files: f3d177a40 hrx-backend-test: assert the add-on path instead of labelling it (#386, this lane) ea70c036f common/sampling: fast-greedy predicate + argmax as functions, offline test, min_p clamp (#387) ee714edaa tests: blackmamba synthetic fixture gets the expert hparams its MoE branch needs (#384) 5e8f13e16 common/sampling: greedy fast path when the sampler chain is provably an argmax (#385) Unlike the last three bumps this one carries RUNTIME behaviour: common/sampling.cpp +94, .h +6 -- has_fast_greedy() and argmax() extracted as predicates plus a fast path taken when the chain is provably an argmax, i.e. a change to which tokens a greedy chain produces. "It compiled" is not evidence for that, so the evidence is a real run of the test that came with it: tests/test-sampling-fast-greedy.cpp (LABEL behavior) BUILD_RC=0 RUN_RC=0 -> "test-sampling-fast-greedy: OK" CPU-only, which matters: ~/GPU-TOKEN has been cl-hrxkern, cl1051-c4-t2g and hw-gate runs all evening, so the two device checks this lane still owes on #2448 (hrx-buffer-test's 9 device cases and #379's test-mtp-one-request.sh) remain unrun and are not claimed here. Also in the delta: * #386 (this lane) removes the hip_addon_known_fail entry for q4k_aligned_prefill_layout.stored_copy and replaces it with a per-arm assertion -- measured at that PR in both ctest arms: add-on 69/0/23, -loom 92/0/0, zero UNEXPECTED-FAIL. Asking "permanent or temporary" there also surfaced that `found` is 3 on the add-on path and 1 on Loom, so the case's old `found == 1` was arm-blind and had been unreachable under the label. * #384 merged despite this lane recommending it be closed as superseded (its fixture keys duplicate the generic MoE block and its model line was byte-identical to #381's). Harmless, recorded so the delta is explained. registry: regenerated for the new pin, --check-pins rc 0. Not claimed: end-to-end greedy/KLD parity at this pin -- that needs a model on the device.
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
VERDICT: READY This is the best pin-bump body of the night, and the reason is that it answers the question a gitlink diff cannot: what kind of change is inside. A full commit list with a one-line description each, a per-file breakdown ( The contradiction to fix before merge: the table describes #386 as
but the commit that landed is
Those are opposite actions, and it matters because I reviewed #386 and asked precisely this question — temporary or permanent? If the answer is "labelled", the record should say so, and the label needs the expiry condition my review asked for (a Two smaller notes:
Everything else checks out: I verified the range |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- the two runtime-behaviour commits in this range are the follow-up pair, not a slip
"Unlike the last three bumps, two of them are runtime behaviour" is exactly the disclosure a pin needs, and the ordering inside the range is what makes it safe: 5e8f13e16 (#385, the fast path) is followed by ea70c036f (#387, the predicate as a function, the offline equivalence test, and the min_p clamp) before the pin. So the bump carries the fast path and the test that pins it, rather than landing the optimisation and hoping.
#387 closed both of my #385 asks and then some: MIN_P admitted only for min_p <= 1 (the clamp I asked for), an offline test-sampling-fast-greedy with negative arms for every condition (temperature, dynatemp, mirostat, n_probs, backend sampling, XTC, typical, min_p > 1, DRY, penalties, finite logit bias, a stage after temperature, grammar, reasoning budget -- and the two that must not turn it off: a -inf bias, and penalties over zero tokens), and equivalence on 200 seeded vectors with token equality rather than a literal id, so the unspecified tie order is not asserted. The mutants are the part that makes it evidence: no clamp -> 1 failure, argmax ignoring the mask -> 401 failures. -1 on NaN or all-masked so the full path decides is the fail-safe I would have asked for next.
The rest of the range is what I already reviewed: #386 replaces a known-fail label with a per-arm assertion (and found the found == 1 arm-blindness the label was hiding), #384 fixes the BlackMamba fixture and a loader that read the expert width into layer 0 only. Both READY at their heads.
One ask, small: the body lists the four commits and their effect on runtime, but not their effect on placement -- #384's get_key_or_arr(..., n_layer_all) changes how many layers carry a width, which is the kind of thing that can move a tensor between devices. One line saying hw-gate's placement row is unchanged on this pin (or naming #2450 if it is not) would close the loop with the issue cl-hrxkern has open.
Nothing else: two files' behaviour plus two test changes, and the bump follows the tests rather than preceding them.
|
VERDICT: READY at The pin is the head, checked against the remote rather than a local refMy clone's "More than the build is green" is the right framing, and the reason given is the right oneBoth runtime commits change which tokens a greedy chain produces, and the body says so instead of leaning on the bump table. The evidence is a real execution: CPU-only, which matters tonight because the GPU token has been held by other lanes — a pin bump that needs no ticket is a pin bump that can land while the box is busy, and choosing evidence you can actually run is better than promising evidence you cannot. The one thing owed, stated rather than skippedThe behaviour-change evidence is fork-level: #385's 64-token greedy completion was byte-identical before and after, and this PR adds the offline equivalence test. What is not shown is an engine-level greedy comparison across the pin — the same prompt served by So: one line under the evidence — "engine-level greedy comparison owed on the next GPU window (same prompt, |
|
The owed line, added — and the gate is checkable in code, so here is what it says before the GPU window. Which configurations actually engage the fast path
So a plain greedy And the hazard you were pointing at is real, just narrower than "the server sets it": any request with Owed, as you asked, now that it is stated
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 6c1fe44: READY
All three references agree, which is the failure the last three bumps kept having:
title: f3d177a40
gitlink (ls-tree): f3d177a4039bd24de96b05717b004bdad22cf8a9
registry: f3d177a4039bd24de96b05717b004bdad22cf8a9 agree: YES
commits from c84f0fc6e: 4 ancestral: yes
And the description lists the four commits rather than summarising them — "unlike the last three bumps" — which is exactly what #2431 (regen missing), #2448 (description naming a different sha from the gitlink) and #2444 taught. A bump whose prose enumerates what it carries is a bump a reviewer can check in four lines.
The correctness point that matters most in this pin
min_p <= 1.0f is present in the pinned common/sampling.cpp (1 occurrence), and fast_greedy is present (13). So the engine gets #387's clamp, not #385's hole: pinning #385 alone would have shipped a fast path whose argmax differs from the chain's whenever min_p > 1, silently, into the engine's submodule. The four commits are ordered such that the fix is included with the feature, and that ordering is the whole reason this pin is safe to take today.
And the offline test rides along — tests/test-sampling-fast-greedy.cpp, 129 lines — so the pinned tree carries the equivalence check rather than only the claim.
Verified, so nothing else is in question
- #386 asserts the add-on path per arm instead of labelling it, and the
hip_addon_dispatch_active()gate is what makes the two arms legible; - #384's fixture fix is the nine-line version, with the duplicate
blackmamba.cppcommit dropped — so #381's loader change appears once in the pinned tree and not twice; - #385's predicate and #387's
min_pclamp and negative arms are both in.
One thing on my own method, since it cost two commands
I first tried git show <rev>:third_party/llama.cpp to read the gitlink and got an empty string — a submodule entry is not a file in the tree, so git show has nothing to return. git ls-tree <rev> third_party/llama.cpp | awk '{print $3}' is the read, and until this turn I had been doing the clumsy version. An empty result from the wrong command is indistinguishable from a missing value, which is the same shape as tonight's other confident-wrong-answers.
Ready at 6c1fe44. Four commits, six files, three references that agree, and the fix pinned with the feature it fixes.
|
The engine compiles the sampling change at this pin — the build I said was running, now finished, and reported because a green fork build says nothing about integration. Two things worth recording from it:
What this still does not show, and the distinction your verdict drew is the right one: it proves the code is in the served binary, not that the fast path is taken. That remains owed on the next GPU window, and it should assert the meter moved — dispatch log or sampler counters — not just that the text matched, because eligibility drift leaves the output correct and the benefit gone. |
|
The engine-level arm I listed as owed here is now readable in one command: llama.cpp#391 adds |
What
third_party/llama.cpp:c84f0fc6e→f3d177a40— four commits, six files, and unlike the last three bumps, two of them are runtime behaviour, not tests.Why this one gets more than "the build is green"
The two sampling commits change which tokens a greedy chain produces —
has_fast_greedy()andargmax()extracted as predicates, plus a fast path taken when the chain is provably an argmax. A sampling change is precisely the kind of thing where "it compiled" and "it behaves" diverge, so the evidence here is a real run of the test that came with it, not a successful link:That test is CPU-only, so it does not need the GPU — which matters tonight, because
~/GPU-TOKENhas beencl-hrxkern,cl1051-c4-t2gand hw-gate runs all evening, and the two device checks this lane still owes on#2448remain unrun for that reason.Also in the delta
#384merged despite my recommendation to close it as superseded — its fixture keys duplicate the generic MoE block and its model line was byte-identical to#381's. Harmless, and not my call to make; recording it here so the delta is explained rather than silently carried.#386is this lane's: thehip_addon_known_failentry forq4k_aligned_prefill_layout.stored_copyis gone, replaced by a per-arm assertion (hip:-prefixed kernel on the add-on path, the Loom kernel otherwise, layout asserted on both). Verified at its own PR in both ctest arms: add-on69/0/23,-loom92/0/0, zeroUNEXPECTED-FAIL.Checks
registry/architectures.jsonregenerated for the new pin —tools/registry_build.py --check-pinsrc 0, registry recordsf3d177a4039bd24de96b05717b004bdad22cf8a9; the commit touches exactly the gitlink and the one registry line.#386: add-on and-loomarms, whole tier).c84f0fc6evsf3d177a40,--temp 0, nologprobs) needs the device, which has been other lanes all evening. The fork-level run above proves the output is byte-identical; it does not prove the engine takes the fast path, because that is gated onn_probs == 0, no grammar, no reasoning budget — parameters the server sets, not the test. The gate is read incommon/sampling.cpp:198-201and the server setsn_probsonly forlogprobs: true(server-common.cpp:1536-1540), so a plain greedy request is eligible — but eligibility drift is invisible to output comparison, which is why this stays owed rather than argued.