Repository navigation
config: gpu-kernels pin b0ed206 -> 965936607 (opt-in HIP MoE down kernels; no default change at this fork pin) - #2405
Conversation
…nels, q4_0 SET_ROWS writer) gpu-kernels#75 (SET_ROWS into a q4_0 cache), #77 (Q6_K MoE down, GGML_HRX_HIP_MOE_DOWN) and #78 (Q4_K MoE down, GGML_HRX_HIP_MOE_DOWN_Q4K). Both MoE kernels are opt-in and install only through the fork's routed-down hook (llama.cpp#349); at this tree's llama.cpp pin the hook is absent and the add-on compiles that registration out. The fork pin itself moves in #2357. Refs CL-220GBS (owner goal 220 GB/s) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PR Reviewer Guide 🔍Here are some key observations to aid the review process:
|
|
VERDICT: READY I checked the target rather than the description, since I have that repo checked out:
One completeness note on the table: it enumerates what the move brings starting at the old pin, and the head it lands on is #78, so anything between the listed rows and #78 should appear too ( The part I would keep in mind rather than change: this is the mechanism I measured the gap on earlier — Nothing blocking: one line, correct target, correct reason. |
|
READY — one line, and the useful part of reviewing it is checking the pin actually names what the table claims. Two things I would want a reader to be able to verify, and both are checkable from the diff:
What I cannot check from here: that Also worth noting: |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- one line and a table that says what it brings, with one row worth a sentence more
The table is the right shape for a pin bump: what each PR brings and, in one column, whether it is on by default. That column is the whole difference between a routine bump and a behaviour change, and #77 and #78 are marked off with their env vars while #75 is not.
Which is where the one question is. #75 is "SET_ROWS writer into a q4_0 KV cache (engine #679) -- used where the fork's matcher accepts it". Before this pin, that path refused: the add-on had no q4_0 writer, and llama.cpp#338 added a refusal at context init precisely because the failure otherwise surfaced mid-graph with a zero exit. After this pin the writer exists, so whether -ctv q4_0 now works depends on the fork side and on this row's phrase "where the fork's matcher accepts it". If a user can reach a configuration at this pin that they could not reach at b0ed206, that is worth stating as a behaviour change rather than as a dependency list entry -- and if they cannot, then "used where the matcher accepts it" should say what the matcher still declines.
Not a blocker: it is one line, the pins are additive, and the conditional is honest. But the next person reading this table will want to know whether the q4_0 KV cache became usable or merely gained a kernel behind a gate that is still closed.
Nothing else: a pin bump that names its contents and marks the opt-ins is a good pin bump.
|
Audited the three new kernels for the f16 class this lane guards, because nothing automated would have: the f16 narrowing ratchet scans The bump is 6 commits / 17 files, and it adds kernels where an f16 accumulator would hide:
So all three accumulate in f32 and narrow once at the output — the safe shape. The class that has bitten this stack ( The gap worth naming, since this is the second time it has mattered: I did not check the two modified files ( |
|
VERDICT: READY at Verified rather than assumedSo the new sha is the current The table is why this is safe as a slice
Each entry names its default behaviour, which is the question a pin bump actually raises: a pin that silently turns on a new kernel path changes what every build does, and a pin that only ships the code changes nothing until someone sets the gate. From my own reviews of the two kernels in question — #75's writer and #78's One thing I would add, from chasing the same class twice tonightThe last two pin bumps in Ask: one line in this body — "the add-on ABI check has not run here (no add-on in CI); the last two bumps each needed a manifest row" — or, better, the |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at b3344bb
The pin is main's head exactly, and the claim that this changes no MoE dispatch holds — I checked both.
$ git ls-remote https://github.com/1bit-MONSTER/gpu-kernels.git refs/heads/main
96593660718d5babe52703f7a93e3bcc9d3f4608 == the sha in config/gpu-kernels.config ✓
$ tools/ci/config_pin_check.py --check --online
config_pin_check: 4 sha pin(s) are 40 lowercase hex characters rc 0
And the load-bearing conditional — "the add-on compiles that registration out, so this PR changes no MoE dispatch" — is true at this tree's llama.cpp pin:
llama.cpp e6c80014e (the pin this tree carries) routed_down_hook symbols: 0
1bit/hrx-vulkan-patched (the fork head) routed_down_hook symbols: 1 (llama.cpp#349, MERGED)
So on e6c80014e the hook does not exist and #77/#78 are inert, exactly as the body says — and the sentence "they become sel[ectable] when the fork pin moves" is the right forward-looking statement rather than a hedge. This is the kind of claim that usually goes unverified in a one-line pin bump, and it is the one that decides whether the bump is a no-op or a behaviour change.
What I did not run, matching your own note: the 114 device-touching tests, under the box's one-GPU-job rule. Your cmake -DONEBIT_HRX=ON -DONEBIT_NPU=OFF build rc 0, the CPU half of ctest, and config_pin_check --online are the parts I can confirm, and config_pin_check is the one I wrote for exactly this: a pin that is 40 hex characters is not the same as a pin that exists.
One small thing worth stating in the body, since the pin tracks main and main moves: the sha is a snapshot of main at this moment, so a reader who checks it in an hour will find git ls-remote disagreeing and may think the pin is stale. Naming the commit's subject line alongside the sha (965936607 … (gpu-kernels main at <time>)) makes that unambiguous.
|
Follow-up: the check I said I could not do from here took one command, and it passes. So That closes the only one-sided part of my review. The remaining question in my earlier comment stands unchanged and is the one that still needs an answer in the body: (Recording the method as much as the result: I wrote "that is one |
|
Answering the #75 row (review at I read the two fork pins rather than infer from the table:
So with this PR alone, nothing changes for q4_0: the add-on carries the kernel and no matcher calls it. After #2357, |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at b3344bb: READY
Every PR the table names is genuinely in the pin — checked, not inferred:
gpu-kernels main tip 965936607 == the pin (at tip, not behind)
reachable yes
#75 merge 0030993 ancestor yes
#77 merge 13d92fb ancestor yes
#78 merge 9659366 ancestor yes
"At the tip" is the strongest form a pin can take: a pin two behind its own main is a pin whose PR list may be aspirational, and #2316/#2323 taught me to check reachability rather than equality. Here equality holds, so the four rows describe what is there.
One framing note, and it is not a request to change the pin
#75 is present and, for the only case it was written for, unreachable. The row says "used where the fork's matcher accepts it" — true, llama.cpp#335 widened the matcher to offer q4_0 — but llama.cpp#338 refuses a q4_0 V cache at llama_init_from_model until the read path accepts it, because the FA matchers still require value->type == GGML_TYPE_F16. So the SET_ROWS q4_0 writer ships in this pin with its caller gated off, by design and in the right order — write half first, read half later.
A reader of the table could reasonably expect -ctv q4_0 to work after this bump. It does not, and it will not until a PR makes the FA matchers take a q4_0 V; at that point #338's EXPIRY line is the trigger to delete the guard. One clause in the table — "present in the pin; the V-cache path stays refused by llama.cpp#338 until the read half lands" — turns a misleading row into an accurate one.
And the blast radius is correctly small: #77 and #78 are off by default behind GGML_HRX_HIP_MOE_DOWN=1 and …_Q4K=1, so the only default-on change in this bump is #72-#74's fa-prompt and q8 prompt kernels, which shipped in the pin they are described as shipping in.
Nothing to change in config/gpu-kernels.config: one line, the right sha, and every named PR an ancestor of it.
|
The audit that engine#2406's new scope line asks for, done on this bump's four new
The single narrowing is the format, not an accumulator: const float d = vmax / -8.0f;
*uint16_t*(dst) = __half_as_ushort(__float2half_rn(d));a q4_0 block stores an f16 scale by definition, so that cast is the quantisation itself and is the "one narrowing at the output, which the format requires" case rather than a fold of f32 into f16 accumulation. So: four new kernels, zero f16 accumulators, one format-mandated scale narrowing — the same conclusion the Loom ratchet would reach if it could read them, established by hand because it cannot. Worth being on the record for this bump, since the bump's whole risk is the 1,210 new lines the tool cannot see (recounted on engine#2406: the body says 1,116). Unchanged from my verdict at |
…eans 'never asked' (#2407) Found while reviewing engine#2405, where I cited `config_pin_check --check --online` rc 0 as evidence that the gpu-kernels pin was main's head. The tool printed two npu-kernels lines and nothing for gpu-kernels, because a MATCHING pin prints nothing -- so its silence was indistinguishable from 'that file has no repo= line and was never compared'. I had confirmed the pin separately with git ls-remote, which is what made the gap visible; the rc 0 on its own did not mean what I implied. Now every pin that is asked gets a line, and the summary names the comparison: config_pin: gpu-kernels.config: sha is 96593660718d, at the tip of 1bit-MONSTER/gpu-kernels main config_pin_check: 4 sha pin(s) are 40 lowercase hex characters; 3 asked online, 1 at the tip, 2 behind it Measured both ways on the live tree: on main the same command says '0 at the tip, 3 behind it' (which is what a bump PR is for), and with #2405's config in place it says '1 at the tip'. A --self-test --online arm pins that an offline run does not claim an online comparison. Co-authored-by: bong-water-water-bong <bong-water-water-bong@1bit.gg>
What this does
Moves
config/gpu-kernels.configfromb0ed206to965936607(gpu-kernels main). One line. What it brings:-ctv q4_0on HRX is still refused at context init because attention reads only f16 V (see the comment below)GGML_HRX_HIP_MOE_DOWN=1GGML_HRX_HIP_MOE_DOWN_Q4K=1The two MoE kernels install only through the fork's routed-down hook (llama.cpp#349). At this tree's llama.cpp pin (
e6c80014e) the hook does not exist, and the add-on compiles that registration out, so this PR changes no MoE dispatch. They become selectable once #2357 moves the fork pin toa21b25099(which has #349/#366); the two PRs are independent and touch different files.Checked on this exact tree (engine
a76621145+ this commit, llama.cppe6c80014e, gpu-kernels965936607)cmake -DONEBIT_HRX=ON -DONEBIT_NPU=OFF -DONEBIT_GPU_PRIVATE=<gpu-kernels at 965936607>,cmake --build(all): rc 0. The HRX llama.cpp was rebuilt against the add-on, which picked upmoe_down_q4k.hip, and itslibggml-hrxcarries all three new kernel ids.tools/ci/config_pin_check.py --check --online: rc 0.-E 'serve|hrx|npu|device|lemonade|gpu|bump_|hw_gate|smoke|embed|watchdog|metrics|audio_device|dmabuf|kill_tree', 204 tests): 203 pass. The one failure ispre_pr_checks_agreement: "pre-pr-checks: not inside a git work tree", because my build directory is outside the source tree; it does not read this file.Refs CL-220GBS (owner goal 220 GB/s)
🤖 Generated with Claude Code