Skip to content

third_party/llama.cpp: pin to aae78ecdf — carry #391's fast-greedy observable, which #2452's owed arm needs - #2468

Merged
bong-water-water-bong merged 1 commit into
mainfrom
pin-aae78ecd
Nov 15, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
pin-aae78ecd

Conversation

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator

What

third_party/llama.cpp: 669178cb0 → aae78ecdf — the one commit main is missing, plus the registry line it implies.

aae78ecdf  common/sampling: log whether the fast greedy path was taken (#391)
669178cb0  hrx: one-token GDN decode updates the recurrent state in place (#390)      <- what main pins
6d6cb0f32  hrx: strict refuses a misplacement, not a placement by design (#389)

#2467 pinned 669178cb0 while the fork head was already aae78ecdf; the difference is not bookkeeping:

$ git show 669178cb0:common/sampling.cpp | grep -c LLAMA_LOG_FAST_GREEDY
0

engine#2452's owed arm cannot be written at the current pin. Its review asked for an engine-level greedy comparison, and an engine-level comparison needs to know whether the fast path was taken — otherwise it compares two runs whose outputs match whether or not the optimisation ran. The observable exists (#391, with a consuming ctest row and a mutation proving the third arm). It just is not in the engine.

So this PR removes the reason the arm is impossible, rather than re-asking for the pin: the same one-line change #2448, #2453 and #2467 each needed, delivered.

Verified at this pin

Built with the submodule checked out at aae78ecd and the sha printed by the build (a previous attempt at this reported BUILD_RC=0 while the submodule had never moved — a sha is not a fetchable ref — so the sha is now part of the evidence):

building at submodule aae78ecd
BUILD_RC=0      0 real compile errors      3 objects recompiled (llama-graph / sampling / relink)
<build>/hrx/llama/bin/llama-server relinked

registry/architectures.json regenerated — --check-pins rc 0, registry records aae78ecdf077.

Not claimed

The arm itself. This makes it writable; running it needs a serve (device or -ngl 0 CPU), and I would rather land the pin that unblocks it than hold both.

…rvable the engine currently lacks

Main pins 669178cb0, one commit behind the fork head. The missing commit is #391:

  aae78ecdf  common/sampling: log whether the fast greedy path was taken, so eligibility drift is observable (#391)
  669178cb0  hrx: one-token GDN decode updates the recurrent state in place (no 12.6 MB state copy per token) (#390)
  6d6cb0f32  hrx: strict refuses a misplacement, not a placement the code calls by design (#389)

Why this one commit matters beyond being current: #391 exists to be READ from a served binary's log. engine#2452's
review asked for an engine-level greedy comparison and it cannot be written while the engine has no observable --
`git show 669178cb0:common/sampling.cpp | grep -c LLAMA_LOG_FAST_GREEDY` = 0 -- so the arm stays owed for a reason
that is one line away from disappearing.

Verified at this pin (engine build, submodule checked out at aae78ecd and the sha printed by the build):
  BUILD_RC=0   0 real compile errors   3 objects recompiled (llama-graph / sampling / relink)
  <build>/hrx/llama/bin/llama-server relinked

registry: regenerated for the new pin, --check-pins rc 0.

Not claimed: the engine-level fast-greedy arm itself. It becomes writable with this pin and is still owed on the next
device-or-CPU serve; what this PR removes is the reason it could not be written at all.
@github-actions

Copy link
Copy Markdown

Failed to review PR

@bong-water-water-bong
bong-water-water-bong merged commit a9fb9f7 into main Nov 15, 2026
11 of 12 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the pin-aae78ecd branch November 15, 2026 03:27

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 52ae025: READY

The one commit this pin adds is the one I held with CHANGES, and I ran its arms at the pin with a real model.

The enumeration is right, which is the check this fleet keeps failing

$ git rev-list --count 669178cb0..aae78ecdf        1        ← the body says "the one commit main is missing"
$ git ls-tree HEAD third_party/ | grep llama      aae78ecdf077…
$ registry/architectures.json                     aae78ecdf077…
$ git rev-parse origin/1bit/hrx-vulkan-patched    aae78ecdf          ← the pin IS the fork tip

One, not three; gitlink and registry carry the same 40 hex; and the pinned sha is the branch tip rather than something behind it. That is #2431 ("nine commits, not six") and #2448 (four gitlinks on one PR) answered before they are asked.

aae78ecdf is llama.cpp#391, and my objection is closed by a real run

I held #391 with CHANGES because it added an observable — common_sampler: fast_greedy=… (eligible=…, LLAMA_NO_FAST_GREEDY=…) — with nothing that consumed it, so a typo or a mis-wired eligible would have been invisible. The fix is the consumer, and it is the consumer I would have wanted: tests/test-fast-greedy-log.sh drives the real llama-cli and greps the exact line.

Built llama-cli at the pin and ran those three arms (build with -DGGML_HRX=OFF: the sampler path is CPU-side and the arms run -ngl 0, so HRX was never needed — and an HRX build from a fresh tree still dies in the Loom dependency on instruction_strings.json):

$ ONEBIT_FAST_GREEDY_MODEL=~/models/Qwen3-0.6B-Q4_K_M.gguf test-fast-greedy-log.sh <bin>
ok   a greedy run takes the fast path
ok   a sampled run is not eligible
ok   the override is distinguishable from ineligibility
test-fast-greedy-log: OK                                        rc 0

The third arm is the one that justified the line in the first place: eligible but overridden is now distinguishable from not eligible, which a single boolean could not express, and it is asserted against the actual log text rather than against a helper that could drift from it.

And the skip discipline is right: with ONEBIT_FAST_GREEDY_MODEL unset the script exits 77 ("ONEBIT_FAST_GREEDY_MODEL unset or missing, SKIP"), which I measured on this box before the model was in hand — so CI stays green without a model instead of pretending the arms ran.

What I did not run: the 27B decode gain the body quotes for the pinned pair, and hrx-buffer-test/device tier (no ticket free). Nothing in this delta can move either: the delta is one sampler log line and its test.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY at 52ae025 — this is the head-pin I asked for on #2467, and it carries exactly the third commit that was missing.

fork head             aae78ecdf07…
this PR pins          aae78ecdf077a5f008ca81cc0de38b84e77ad5fe    -> matches
gitlink + registry/architectures.json both changed (2 added lines naming the pin)

The range now carries all three commits this lane reviewed tonight:

That last one is why pinning the head mattered: a server that stops being eligible keeps producing correct tokens and silently loses the whole gain, and no output-based check can see it. With this pin the question is one command, so the next lane measuring the greedy path does not have to guess whether it ran.

Nothing to ask. Head pin, both pin lines moved together, and the three commits it carries are the three I read.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

The measurement this pin exists for, run on the engine's own binary at aae78ecd — inside a 4-second gpu-hold on the freshly rebooted box, -ngl 0 so the compute is CPU:

$ gpu-hold -t 2 -l hrxmem-fastgreedy-cpu -- …/pin8c12-build/hrx/llama/bin/llama-server \
      -m ZR1-1.5B-bf16.gguf -c 512 --temp 0 -ngl 0
  {"choices":[{"finish_reason":"length",…"content":"\n\n"}],…}          <- served
  fast_greedy=1 (eligible=1, LLAMA_NO_FAST_GREEDY=unset)               <- the engine TOOK the path
gpu-hold released /home/bcloud/GPU-TOKEN after 4s

That is #2452's owed arm, and it only runs at the pin this PR moves to:

engine pin grep -c LLAMA_LOG_FAST_GREEDY common/sampling.cpp can the arm be answered
669178cb0 (main today) 0 no — nothing reports the decision
aae78ecd (this PR) 1 yes, fast_greedy=1 from the served binary

So the claim is not "the engine has one more commit": at 669178cb0 the question is unanswerable, and at aae78ecd the answer is that a served greedy request takes the fast path on the engine's own build. That is the difference this one line buys.

Nothing left running: the server was killed in the same command and the ticket released at 23:51:28 (die 54 °C, took the token in 0 s — the box was free).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant