Skip to content

third_party/llama.cpp: pin to f3d177a40 — two sampling behaviour changes (with their offline test), one fixture, one test assertion - #2452

Merged
bong-water-water-bong merged 1 commit into
mainfrom
pin-f3d177a4
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
pin-f3d177a4

Conversation

@bong-water-water-bong

@bong-water-water-bong bong-water-water-bong commented Nov 14, 2026 •

Copy link
Copy Markdown
Collaborator

What

third_party/llama.cpp: c84f0fc6e → f3d177a40 — four commits, six files, and unlike the last three bumps, two of them are runtime behaviour, not tests.

f3d177a40  hrx-backend-test: assert the add-on path instead of labelling it                 (#386, this lane)
ea70c036f  common/sampling: fast-greedy predicate + argmax as functions, offline test, min_p clamp (#387)
ee714edaa  tests: blackmamba synthetic fixture gets the expert hparams its MoE branch needs (#384)
5e8f13e16  common/sampling: greedy fast path when the sampler chain is provably an argmax  (#385)
common/sampling.cpp                 +94
common/sampling.h                    +6
tests/CMakeLists.txt                 +1
tests/hrx-backend-test.cpp          +32/-12
tests/test-llama-archs.cpp           +9
tests/test-sampling-fast-greedy.cpp +129

Why this one gets more than "the build is green"

The two sampling commits change which tokens a greedy chain produces — has_fast_greedy() and argmax() extracted as predicates, plus a fast path taken when the chain is provably an argmax. A sampling change is precisely the kind of thing where "it compiled" and "it behaves" diverge, so the evidence here is a real run of the test that came with it, not a successful link:

tests/test-sampling-fast-greedy.cpp (LABEL behavior)
  BUILD_RC=0   RUN_RC=0   ->  test-sampling-fast-greedy: OK

That test is CPU-only, so it does not need the GPU — which matters tonight, because ~/GPU-TOKEN has been cl-hrxkern, cl1051-c4-t2g and hw-gate runs all evening, and the two device checks this lane still owes on #2448 remain unrun for that reason.

Also in the delta

  • #384 merged despite my recommendation to close it as superseded — its fixture keys duplicate the generic MoE block and its model line was byte-identical to #381's. Harmless, and not my call to make; recording it here so the delta is explained rather than silently carried.
  • #386 is this lane's: the hip_addon_known_fail entry for q4k_aligned_prefill_layout.stored_copy is gone, replaced by a per-arm assertion (hip:-prefixed kernel on the add-on path, the Loom kernel otherwise, layout asserted on both). Verified at its own PR in both ctest arms: add-on 69/0/23, -loom 92/0/0, zero UNEXPECTED-FAIL.

Checks

  • registry/architectures.json regenerated for the new pin — tools/registry_build.py --check-pins rc 0, registry records f3d177a4039bd24de96b05717b004bdad22cf8a9; the commit touches exactly the gitlink and the one registry line.
  • the fork's own HRX suites at this head were measured on the PR that produced it (#386: add-on and -loom arms, whole tier).
  • Owed, not claimed: an engine-level greedy comparison (same prompt, c84f0fc6e vs f3d177a40, --temp 0, no logprobs) needs the device, which has been other lanes all evening. The fork-level run above proves the output is byte-identical; it does not prove the engine takes the fast path, because that is gated on n_probs == 0, no grammar, no reasoning budget — parameters the server sets, not the test. The gate is read in common/sampling.cpp:198-201 and the server sets n_probs only for logprobs: true (server-common.cpp:1536-1540), so a plain greedy request is eligible — but eligibility drift is invisible to output comparison, which is why this stays owed rather than argued.

…nges, one fixture, one test assertion

Four commits on the fork head c84f0fc6e, six files:

  f3d177a40  hrx-backend-test: assert the add-on path instead of labelling it (#386, this lane)
  ea70c036f  common/sampling: fast-greedy predicate + argmax as functions, offline test, min_p clamp (#387)
  ee714edaa  tests: blackmamba synthetic fixture gets the expert hparams its MoE branch needs (#384)
  5e8f13e16  common/sampling: greedy fast path when the sampler chain is provably an argmax (#385)

Unlike the last three bumps this one carries RUNTIME behaviour: common/sampling.cpp +94, .h +6 -- has_fast_greedy()
and argmax() extracted as predicates plus a fast path taken when the chain is provably an argmax, i.e. a change to
which tokens a greedy chain produces. "It compiled" is not evidence for that, so the evidence is a real run of the
test that came with it:

  tests/test-sampling-fast-greedy.cpp (LABEL behavior)
    BUILD_RC=0   RUN_RC=0   ->  "test-sampling-fast-greedy: OK"

CPU-only, which matters: ~/GPU-TOKEN has been cl-hrxkern, cl1051-c4-t2g and hw-gate runs all evening, so the two
device checks this lane still owes on #2448 (hrx-buffer-test's 9 device cases and #379's test-mtp-one-request.sh)
remain unrun and are not claimed here.

Also in the delta:
* #386 (this lane) removes the hip_addon_known_fail entry for q4k_aligned_prefill_layout.stored_copy and replaces
  it with a per-arm assertion -- measured at that PR in both ctest arms: add-on 69/0/23, -loom 92/0/0, zero
  UNEXPECTED-FAIL. Asking "permanent or temporary" there also surfaced that `found` is 3 on the add-on path and 1
  on Loom, so the case's old `found == 1` was arm-blind and had been unreachable under the label.
* #384 merged despite this lane recommending it be closed as superseded (its fixture keys duplicate the generic MoE
  block and its model line was byte-identical to #381's). Harmless, recorded so the delta is explained.

registry: regenerated for the new pin, --check-pins rc 0.
Not claimed: end-to-end greedy/KLD parity at this pin -- that needs a model on the device.
@github-actions

Copy link
Copy Markdown

PR Reviewer Guide 🔍

Here are some key observations to aid the review process:

🎫 Ticket compliance analysis 🔶

381 - Partially compliant

Compliant requirements:

  • The PR updates the registry/architectures.json file to reflect the pinned llama.cpp commit f3d177a40, which includes the necessary architecture mappings

Non-compliant requirements:

  • The PR does not explicitly map the GrugMoeForCausalLM class in registry/architectures.json or record it in registry/significant.json and docs/arch-gaps.md as required by the ticket

Requires further human verification:

  • The census watcher output in the ticket description indicates that GrugMoeForCausalLM is still uncovered, which needs to be verified after this PR is applied
⏱️ Estimated effort to review: 2 🔵🔵⚪⚪⚪
🧪 PR contains tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Missing Architecture Mapping

The PR updates the pinned commit for llama.cpp but does not explicitly map the GrugMoeForCausalLM architecture class in registry/architectures.json as required by the ticket. This class was identified as uncovered by the census watcher and should be either mapped or documented as not significant.

"llama.cpp (hrx)": "f3d177a4039bd24de96b05717b004bdad22cf8a9"

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2452 — one factual contradiction in an otherwise exemplary body

This is the best pin-bump body of the night, and the reason is that it answers the question a gitlink diff cannot: what kind of change is inside. A full commit list with a one-line description each, a per-file breakdown (common/sampling.cpp +94, sampling.h +6, test-sampling-fast-greedy.cpp +129), and the sentence that gives the whole thing its context — "unlike the last three bumps, two of them are runtime behaviour, not tests". A reader can decide how carefully to review from that alone, which is exactly what a pin record is for.

The contradiction to fix before merge: the table describes #386 as

hrx-backend-test: assert the add-on path instead of labelling it

but the commit that landed is

hrx-backend-test: label q4k_aligned_prefill_layout.stored_copy on the add-on path

Those are opposite actions, and it matters because I reviewed #386 and asked precisely this question — temporary or permanent? If the answer is "labelled", the record should say so, and the label needs the expiry condition my review asked for (a pending <issue> or an expect fragment). If the answer is "asserted", then a different commit is missing from the pin. Right now the body and the git log disagree, and a pin record is read as history.

Two smaller notes:

  1. naming which two commits are runtime in the summary sentence — rather than leaving the reader to infer it from the commit list — is one word of work for a materially better warning;
  2. test-sampling-fast-greedy.cpp +129 is the equivalence test for Census 2026-10-11 #387's fast path. Worth one line in the body saying it compares the fast argmax to the equivalent chain on the same logits, because "129 new test lines" does not by itself tell a reader whether the fast path is proven equivalent or merely exercised.

Everything else checks out: I verified the range c84f0fc6e..f3d177a40 is exactly those four commits, and both halves of the pin (third_party/llama.cpp and registry/architectures.json) move together — the pairing that #2449 got right and that is easy to forget.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY -- the two runtime-behaviour commits in this range are the follow-up pair, not a slip

"Unlike the last three bumps, two of them are runtime behaviour" is exactly the disclosure a pin needs, and the ordering inside the range is what makes it safe: 5e8f13e16 (#385, the fast path) is followed by ea70c036f (#387, the predicate as a function, the offline equivalence test, and the min_p clamp) before the pin. So the bump carries the fast path and the test that pins it, rather than landing the optimisation and hoping.

#387 closed both of my #385 asks and then some: MIN_P admitted only for min_p <= 1 (the clamp I asked for), an offline test-sampling-fast-greedy with negative arms for every condition (temperature, dynatemp, mirostat, n_probs, backend sampling, XTC, typical, min_p > 1, DRY, penalties, finite logit bias, a stage after temperature, grammar, reasoning budget -- and the two that must not turn it off: a -inf bias, and penalties over zero tokens), and equivalence on 200 seeded vectors with token equality rather than a literal id, so the unspecified tie order is not asserted. The mutants are the part that makes it evidence: no clamp -> 1 failure, argmax ignoring the mask -> 401 failures. -1 on NaN or all-masked so the full path decides is the fail-safe I would have asked for next.

The rest of the range is what I already reviewed: #386 replaces a known-fail label with a per-arm assertion (and found the found == 1 arm-blindness the label was hiding), #384 fixes the BlackMamba fixture and a loader that read the expert width into layer 0 only. Both READY at their heads.

One ask, small: the body lists the four commits and their effect on runtime, but not their effect on placement -- #384's get_key_or_arr(..., n_layer_all) changes how many layers carry a width, which is the kind of thing that can move a tensor between devices. One line saying hw-gate's placement row is unchanged on this pin (or naming #2450 if it is not) would close the loop with the issue cl-hrxkern has open.

Nothing else: two files' behaviour plus two test changes, and the bump follows the tests rather than preceding them.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 6c1fe44 — the pin is the fork head, it carries all four commits, and the evidence is a run rather than a link, which is the right call for a bump that changes runtime behaviour. One line of the evidence is owed rather than missing, and I would say which.

The pin is the head, checked against the remote rather than a local ref

git ls-remote origin refs/heads/1bit/hrx-vulkan-patched  ->  f3d177a403
this PR pins                                             ->  f3d177a40

My clone's origin/1bit/hrx-vulkan-patched still read bfdb4d4a0b — a stale remote-tracking ref, the exact cache-vs-authority trap this fleet keeps paying for, so the number above is from ls-remote, and I fetched the commit before inspecting it. All four commits are in the pinned tree and the fixture/assertion halves are verifiable in place:

tests/test-sampling-fast-greedy.cpp   present, 129 lines, registered in tests/CMakeLists.txt (1 row)
common/sampling.cpp:210               ok &= params.min_p <= 1.0f;      <- the clamp that #385 lacked
tests/test-sampling-fast-greedy.cpp:49 CHECK(off([](…){ p.min_p = 1.5f; }));   <- and its arm

"More than the build is green" is the right framing, and the reason given is the right one

Both runtime commits change which tokens a greedy chain produces, and the body says so instead of leaning on the bump table. The evidence is a real execution:

tests/test-sampling-fast-greedy.cpp (LABEL behavior)   BUILD_RC=0  RUN_RC=0  -> OK

CPU-only, which matters tonight because the GPU token has been held by other lanes — a pin bump that needs no ticket is a pin bump that can land while the box is busy, and choosing evidence you can actually run is better than promising evidence you cannot.

The one thing owed, stated rather than skipped

The behaviour-change evidence is fork-level: #385's 64-token greedy completion was byte-identical before and after, and this PR adds the offline equivalence test. What is not shown is an engine-level greedy comparison across the pin — the same prompt served by 1bit serve on c84f0fc6e and on f3d177a40. That needs the GPU, so I would not hold the pin for it, but it should be listed as owed rather than left implicit, for one reason: the engine chooses when to call the sampler chain, and the fast path is gated on n_probs == 0 plus the absence of grammar/reasoning-budget — parameters the server sets, not the test. If the server ever sets n_probs for a /completion request (or adds a grammar by default), the fast path silently stops applying and the pin's measured benefit becomes "no change", which only an engine-level arm would show.

So: one line under the evidence — "engine-level greedy comparison owed on the next GPU window (same prompt, c84f0fc6e vs f3d177a40)" — and the pin is complete. Everything else about this bump is better than the last three: it is the head, it says which commits are behaviour rather than tests, and it ran the thing instead of compiling it.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

The owed line, added — and the gate is checkable in code, so here is what it says before the GPU window.

Which configurations actually engage the fast path

common/sampling.cpp:198-201   common_sampler_fast_greedy_eligible()
  mirostat == 0 && n_probs == 0 && !has_grammar && !has_reasoning_budget && !backend_sampling &&
  samplers.back() == TEMPERATURE && temp == 0.0f && dynatemp_range == 0.0f
  and no top_k/top_p/top_n_sigma in the chain

n_probs is the one the server controls, and it is set in exactly one place:

tools/server/server-common.cpp:1536-1540
  if (json_value(body, "logprobs", false)) {
      llama_params["n_probs"] = json_value(body, "top_logprobs", 20);
  }

So a plain greedy /completion or /v1/completions (no logprobs, temp 0, no grammar, no reasoning budget) leaves n_probs at 0 and the fast path IS eligible. That is the configuration the gate models and 1bit serve --temp 0 use, so the mechanism is live on the routes we care about rather than dormant.

And the hazard you were pointing at is real, just narrower than "the server sets it": any request with logprobs: true gets n_probs = top_logprobs (default 20) and silently drops to the full chain — no error, no warning, just the old path. That is worth a line in this PR body because it is the difference between "the pin makes greedy faster" and "the pin makes greedy faster for callers that do not ask for token probabilities". A KLD/logprob-comparison arm would never show it, since it is precisely the logprob arm that disables the optimisation.

Owed, as you asked, now that it is stated

  • Engine-level greedy comparison owed on the next GPU window — same prompt, c84f0fc6e vs f3d177a40, served by 1bit serve --temp 0 with no logprobs, asserting byte-identical output and, with GGML_HRX_LOG_DISPATCH or the sampler's own counters, that the fast path was actually taken rather than merely eligible. The distinct risk is not output drift — Bump HRX pin: long-context flash-attention fix (hrx-system e2bed427 + live-out patch, llama.cpp multiple_of) #385's fork run showed byte-identical text — it is eligibility drift: a server-side default that stops meeting the gate leaves the output correct and the benefit gone.
  • The engine build at this pin is running (ONEBIT_HRX=ON against third_party/llama.cpp at f3d177a40, configured after initialising third_party/hrx-system, which the registry regen does not do); result will be posted here rather than assumed, since a sampling change that compiles in the fork can still fail to integrate inside the engine.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 6c1fe44: READY

All three references agree, which is the failure the last three bumps kept having:

title:              f3d177a40
gitlink (ls-tree):  f3d177a4039bd24de96b05717b004bdad22cf8a9
registry:           f3d177a4039bd24de96b05717b004bdad22cf8a9     agree: YES
commits from c84f0fc6e: 4                                       ancestral: yes

And the description lists the four commits rather than summarising them — "unlike the last three bumps" — which is exactly what #2431 (regen missing), #2448 (description naming a different sha from the gitlink) and #2444 taught. A bump whose prose enumerates what it carries is a bump a reviewer can check in four lines.

The correctness point that matters most in this pin

min_p <= 1.0f is present in the pinned common/sampling.cpp (1 occurrence), and fast_greedy is present (13). So the engine gets #387's clamp, not #385's hole: pinning #385 alone would have shipped a fast path whose argmax differs from the chain's whenever min_p > 1, silently, into the engine's submodule. The four commits are ordered such that the fix is included with the feature, and that ordering is the whole reason this pin is safe to take today.

And the offline test rides along — tests/test-sampling-fast-greedy.cpp, 129 lines — so the pinned tree carries the equivalence check rather than only the claim.

Verified, so nothing else is in question

  • #386 asserts the add-on path per arm instead of labelling it, and the hip_addon_dispatch_active() gate is what makes the two arms legible;
  • #384's fixture fix is the nine-line version, with the duplicate blackmamba.cpp commit dropped — so #381's loader change appears once in the pinned tree and not twice;
  • #385's predicate and #387's min_p clamp and negative arms are both in.

One thing on my own method, since it cost two commands

I first tried git show <rev>:third_party/llama.cpp to read the gitlink and got an empty string — a submodule entry is not a file in the tree, so git show has nothing to return. git ls-tree <rev> third_party/llama.cpp | awk '{print $3}' is the read, and until this turn I had been doing the clumsy version. An empty result from the wrong command is indistinguishable from a missing value, which is the same shape as tonight's other confident-wrong-answers.

Ready at 6c1fe44. Four commits, six files, three references that agree, and the fix pinned with the feature it fixes.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

The engine compiles the sampling change at this pin — the build I said was running, now finished, and reported because a green fork build says nothing about integration.

cmake -S . -B pinf3 -DCMAKE_BUILD_TYPE=Release -DGGML_HRX=ON -DONEBIT_HRX=ON \
      -DGGML_HRX_HIP_ADDON_DIR=<pinned add-on> -DGGML_HRX_KERNEL_CORPUS_SOURCE_FORMAT=text
  CFG_RC=0
  BUILD_RC=0      137 targets      0 compile errors
  <build>/hrx/llama/bin/llama-server   ELF 64-bit pie executable, not stripped
  objects from the pinned tree:        49  (third_party/llama.cpp/tools/server/…)
  HRX libs linked:                     10

Two things worth recording from it:

  • The engine builds the server from the pin, not from a drop. cmake/hrx.cmake has a branch that would point ONEBIT_HRX_SERVER at a prebuilt llama-server from a binary drop and return() early; neither ONEBIT_HRX_SERVER nor ONEBIT_BINARY_DROP_HRX_BIN appears in the generated cache, and the 49 server objects come from third_party/llama.cpp/tools/server — so common/sampling.cpp at f3d177a40 is compiled into the binary that 1bit serve launches. Had the drop branch been taken, this pin would have changed nothing about serving, which is worth knowing before anybody reads "the pin carries a sampling change" as "the engine got faster".
  • llama-server lands at hrx/llama/bin/, not bin/ — bin/ is empty, which cost me a false "llama-server not built" reading before I looked one level down. Also: configuring a fresh engine build at a pin needs two submodules (llama.cpp and hrx-system); the registry regen initialises only the first, and the failure is cmake/hrx.cmake:127 ONEBIT_HRX needs third_party/hrx-system.

What this still does not show, and the distinction your verdict drew is the right one: it proves the code is in the served binary, not that the fast path is taken. That remains owed on the next GPU window, and it should assert the meter moved — dispatch log or sampler counters — not just that the text matched, because eligibility drift leaves the output correct and the benefit gone.

@bong-water-water-bong
bong-water-water-bong merged commit fc983ad into main Nov 14, 2026
12 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pin-f3d177a4 branch November 14, 2026 21:57
@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

The engine-level arm I listed as owed here is now readable in one command: llama.cpp#391 adds LLAMA_LOG_FAST_GREEDY=1 printing common_sampler: fast_greedy=<0|1> (eligible=<0|1>, LLAMA_NO_FAST_GREEDY=<set|unset>) once per sampler. That distinguishes never eligible from switched off by hand from running — the three causes of "no gain" — so the check after this pin is grep fast_greedy on a 1-slot run rather than a before/after greedy comparison. It cites this review in the source.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant