Skip to content

config/gpu-kernels.config: pin to 60b8eef (gk#97: kqt_mm_q4k -7-8 % at 4 rows, bit-exact) - #2572

Merged
bong-water-water-bong merged 1 commit into
mainfrom
cl-1051/gk-pin-97
Nov 15, 2026
Merged

bong-water-water-bong merged 1 commit into
mainfrom
cl-1051/gk-pin-97

Conversation

@bong-water-water-bong

@bong-water-water-bong bong-water-water-bong commented Nov 15, 2026 •

Copy link
Copy Markdown
Collaborator

Moves config/gpu-kernels.config from baabfc3 to 60b8eef (gpu-kernels main, which ends with #97).

What the pin brings

gk PR change
#97 a pipelined 4-token Q4_K path for kqt_mm_q4k / kqt_swiglu_q4k_q4k. Bit-exact; kqt_mm_q4k is −7–8 % per dispatch in-graph at 4 rows on Qwen3.8-27B (C4-STREAMS); swiglu flat. GEAK run wf_a078f173-1c0
#93 a K >= 8192 batch under 256 tokens takes the plain m64 tile for every format
#91 Q4_0 in the dense-prefill kernels (DP_Q4_0)
#92 a per-object check of kernel ids against the registry
#94, #96 dense-prefill-bench fails on a run with no timing line; README rc
#82, #89 docs and comments

The pinned tree's own suite

Engine main 21a186d9 + fork 8aa770fbd (main's pin) + gk 60b8eef, built clean. Run in one gpu-hold ticket (cl1051-gkpin97-suite, 18:19–18:37Z, no device fault):

  • fork ctest: 9/111 fail. 8 of them are the same names that fail at the current pin, measured on this box with the same script for config: gpu-kernels pin 2022afb74 -> baabfc3d6 — hip.fa_decode_d256 for multi-row attention (4-row step span -4.1 %, gpu-kernels#88) #2517:
    • test-tokenizers-ggml-vocabs, test-kv-zero, test-llama-archs, test-llama-context, hrx-backend-test-loom, test-hrx-fa-transposed-value, test-hrx-mul-mat-id-k32, test-hrx-fa-masked-v.
    • The 9th, test-hrx-loader-mapped, was Not Run. It is new at this fork pin (serve: /health follows the backend; progress watchdog for a hung backend (P3) #394), and the engine build does not build its target. Built by name and run in its own ticket: PASS. Ctest skips it with no model (exit 77), so it was run directly on Qwen3-0.6B-Q4_K_M.gguf in ticket cl1051-gkpin97-loader2: "2 weight buffer lines, 1 HRX0_Mapped, 0 failure(s)", rc 0, no device fault.
  • hrx-ops-test ssm_conv|gdn: 18/18.
  • engine ctest (NPU tests excluded): 3/256 fail, the same three as at the current pin:
    • laya_gguf_route (known);
    • token_rate_floor_device (Timeout: it takes its own gpu-hold ticket, which queues behind the suite's);
    • gpu_hold_child_identity (inherits the suite ticket's env; it passed outside a ticket on 11-15).

No new failure. The kernel code embedded in this build's library is #97's (kquant_tokens_q8 .text 77a716e8af70).

🤖 Generated with Claude Code

…_K kqt path, + #91 #92 #93 #94 #96)

The pinned tree's suite (engine main + fork 8aa770fbd + gk 60b8eef) shows no new failure against the current pin.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

github-actions Bot commented Nov 15, 2026 •

Copy link
Copy Markdown

PR Reviewer Guide 🔍

(Review updated until commit 9cea2d8)

Here are some key observations to aid the review process:

⏱️ Estimated effort to review: 1 🔵⚪⚪⚪⚪
🧪 No relevant tests
🔒 No security concerns identified
⚡ Recommended focus areas for review

Pin Update

The PR updates the pinned commit for the gpu-kernels repository from baabfc3 to 60b8eef. This change aligns with the described improvements in gk#97, including a pipelined 4-token Q4_K path for kqt_mm_q4k and kqt_swiglu_q4k_q4k with bit-exact results and performance improvements. The update also incorporates other kernel improvements from gk#91, #92, #93, #94, #96, #82, and #89. Ensure that the new pin is thoroughly tested on the target hardware (Strix Halo XDNA 2 NPU) to validate the claimed performance gains and correctness.

sha=60b8eef58487a774109284f6942a53dfa7fcd12c

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY 2572 — pin range verified against the commits

I enumerated the range rather than reading the table, git log --oneline baabfc3..60b8eef in gpu-kernels (8 commits):

60b8eef58 kquant_tokens: a pipelined 4-token Q4_K path for kqt_mm_q4k / kqt_swiglu_q4k_q…   (#97)
4d695e0f6 README item 3: the bench returns rc 4 now, not rc 0                                  (#96)
ddb4a3253 dense-prefill-bench: a run that prints no timing line fails                          (#94)
263e44f09 hip-hrx: a K >= 8192 batch under 256 tokens takes the plain m64 tile for every…      (docs)
82ec45d8a tools: check every object's kernel ids against the registry, per object              (#92)
de38cd20e hip-hrx: Q4_0 in the dense-prefill kernels (DP_Q4_0) -- Q4_0 prefill x2.95 on …      (#91)
074b76146 docs: the four things that break a kernel measurement (each cost turns in #76)       (#82)
4cc4d8d52 hip-hrx: say why hip.fa_decode_d256 has two n_kv thresholds (review of #88)          (#89)

So the pin is exactly the eight I expect, and two of them are behavioural, which is the part that needs a sentence in the description:

  1. serve --device hrx: default the context to 32768 so HRX0 keeps flash attention (#95) #97 is a kernel body change (kqt_mm_q4k / kqt_swiglu_q4k_q4k, T≤4). A pin that moves a decode kernel must say the engine's own suite ran against it — test-backend-ops -o MUL_MAT -b HRX0 (or the gate's KAT rows) with bad=0 — because a pin bump is where a kernel change becomes the fleet's kernel change. If that run happened, quote the count; if it has not, the pin is asserting an unverified body.
  2. 1bit route: print the scorer's load and decision times #91 brings Q4_0 dense-prefill (DP_Q4_0, "x2.95") — a new format on the prefill path, so the same question applies, plus: its 11 new kernel ids were hash-verified when I reviewed it, so the registry side is covered.
  3. Model registry and the daily HF census (PORTING step 5) #94 + Census 2026-09-25 #96 change an exit code (rc 4 for an absent measurement) and README, PORTING: the Laya router landed as an opt-in (#90, #91) #92 adds a check to tools/check.sh. Nothing in the engine consumes those, but anything in the engine that shells out to the bench and tests rc == 0 will now see a new distinct failure — worth a grep before the pin lands, since 3 and 4 used to be unambiguous and one of them is new.

And one thing to keep as written: baabfc3 → 60b8eef with "which ends with #97" is exactly the right form — an endpoint plus the PR it corresponds to is checkable in one command, unlike "newer than the last pin".

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT at 9cea2d8: READY — the audit is green at the new sha and, more usefully, it is unchanged despite 298 new lines of kernel code.

The gate, measured at the sha main would pin

hip_f16_audit --dir <gpu-kernels @60b8eef> --expect-sha 60b8eef     rc 0
AUDITED      60b8eef58487a774109284f6942a53dfa7fcd12c
69 kernel file(s), no f16 accumulator; 62 narrowing(s)

Identical totals to the outgoing pin (baabfc3: 69 / 62 / 0), while the diff against it is

kquant_tokens_q8.hip       +233     #97's pipelined 4-token Q4_K body
hrx_dense_prefill.hip       +26     #91's DP_Q4_0 path
hrx_kernel_aliases.h        +13
hrx_kernel_ids.h            +26
hrx_kernel_names.txt        +11

So +298 lines of kernel code introduce no new f16→f32 narrowing and no f16 accumulator. That is the claim a Q4_K/DP_Q4_0 change has to make and rarely states: the pipelined body keeps f32 accumulation, which is what "bit-exact against the old body" implies and what this audit independently confirms. 62 narrowings before and after is the specific number — a rewrite that had introduced a (half) cast in the new prefetch or select path would have shown up as 63.

The one thing to run after this lands

hrx_kernel_names.txt gains 11 names — a registry claim that eleven kernels exist. Today's #98 work (check-kernel-ids now failing on header entries that no object carries, my review) is exactly the tool for that claim, and it is the natural acceptance step for this pin:

check-kernel-ids --objects <built hip-code-objects> --header <this add-on>/kernels/hrx_kernel_ids.h

On the tree measured in #98's docstring a matching header reports 0 of 465; after +11 names it should report 0 of 476. If it reports fewer, one of the new DP_Q4_0 kernels is not being emitted — which is a silent loss until someone asks, and precisely the direction #98 adds. Worth one line in the body so the next reader runs it rather than assuming.

What this pin does not touch

The Loom corpus and its ratchet are in the fork, not here: a config/gpu-kernels.config bump cannot move FINGERPRINT b9b121a5ddc51854. The audit above is the add-on half; the loom half is unchanged by construction, which is worth saying so nobody re-runs it looking for a difference.

And the provenance

The body names GEAK run wf_a078f173-1c0 and the per-dispatch figure (kqt_mm_q4k −7–8 % in-graph at 4 rows, swiglu flat) with the scope attached — an in-graph dispatch percentage, not an end-to-end tok/s. Both of those are the details that make a pin reviewable rather than a leap of faith, and they match what I reviewed on #97 itself.

@bong-water-water-bong

Copy link
Copy Markdown
Collaborator Author

VERDICT: READY on the pin at 9cea2d8 — it names the right commit and it is the only place the sha lives. The one piece of evidence still outstanding is the one this PR cannot supply: the engine's own build, which is pending.

The pin identity, checked against the repo rather than the description

pin 60b8eef   = 2026-11-15T18:03:26Z  "kquant_tokens: a pipelined 4-token Q4_K path …"
main head     = 60b8eef58            (same commit)
gk#97         = MERGED, merge commit 60b8eef58

The pin is gpu-kernels#97's merge commit and the branch head — not a commit that merely contains the fix, and not one 1–5 commits short of it. That is the case I have had to reject three times tonight, so it is worth stating plainly when it holds: this pin is exactly the change it claims to pin.

And the sha lives in one place. baabfc3d… appears in exactly one file on main (config/gpu-kernels.config) and 60b8eef… in exactly one file in this PR — no second copy left behind, which is the failure #2487 taught me to check (a pin and its registry line moving independently).

The diff is the single line it says it is: sha=baabfc3d6eae… → sha=60b8eef5848….

What this PR cannot tell us, and the row that can

A pin bump is the one change whose correctness cannot be judged from its own diff: the kernel-level evidence lives in gpu-kernels (#97's 40/40 against a CPU reference at T=1..8, the bit-identical guards on the neighbouring kernels, the 5 ABBA rounds), and what remains is whether this tree still passes with the new kernels. That is build — 340 ctest rows including the KAT rows that exercise exactly these kernels — and it is pending at this head.

So: merge on green build, and treat a red one as information about the pin rather than about the PR's shape. If it does go red, the most likely rows are the KAT ones, and the first thing to compare is #97's claim at the shapes the KAT rows use (its own sweep covered T = 1..8 across five Q4_K shapes; a KAT row that lands outside those shapes would be a coverage gap in the bench, not necessarily a wrong kernel).

One note on the pairing

#97 changing kquant_tokens_q8.hip and this PR pinning it in the same evening is the right order: the kernel landed with bit-exactness evidence in its own repo, and the engine takes it as a discrete, revertable step. Worth keeping that shape for the next one — a pin that arrives in the same PR as a kernel edit would make the two unreviewable together.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 9cea2d8: READY

Pin checked the way a pin has to be: it is the gk main tip, the bump is 8 commits, the sha is well-formed, and nothing else in the tree still names the old one.

config/gpu-kernels.config:  repo=1bit-MONSTER/gpu-kernels
                            sha=baabfc3d6e… -> 60b8eef5848…
git ls-remote … refs/heads/main          -> 60b8eef58487a774109284f6942a53dfa7fcd12c   ← the pin IS the tip
git rev-list --count baabfc3d6..60b8eef58 -> 8
python3 tools/ci/config_pin_check.py --check  -> 4 sha pin(s) are 40 lowercase hex characters
grep -rn <old sha>  -> only the config line this PR replaces

What the 8 commits carry, and why that matters for a review

60b8eef58  kquant_tokens: a pipelined 4-token Q4_K path (GEAK)                     ← #97, reviewed
4d695e0f6  README item 3: the bench returns rc 4 now, not rc 0                     ← #96
ddb4a3253  dense-prefill-bench: a run that prints no timing line fails             ← #94, reviewed
263e44f09  hip-hrx: K >= 8192 batch under 256 tokens takes the plain m64 tile      ← #93
82ec45d8a  tools: check every object's kernel ids against the registry             ← #92, reviewed
de38cd20e  hip-hrx: Q4_0 in the dense-prefill kernels (x2.95 on a 27B, KLD 1…)     ← #91
074b76146  docs: the four things that break a kernel measurement                   ← #82
4cc4d8d52  hip-hrx: why hip.fa_decode_d256 has two n_kv thresholds                 ← #88, reviewed

So this pin is the first time #91's Q4_0 prefill (claimed ×2.95 on a 27B) enters the engine tree — it is the one item in the list I have not reviewed, and a pin bump is where an unreviewed kernel change arrives. That is not a blocker for this PR (the pin is a deliberate pointer, and the change was reviewed in gk), but it is worth naming: the pin's performance claim now affects a number a lane could quote from this repo.

The rest are the ones I did review and they arrive as a set: the rc 4 bench guard (#94/#96), the per-object id check (#92), and #97's −7–8 % at 4 rows with the bit-exactness argument. A pin bump's review is mostly "do I know what is in it", and here the answer is yes for seven of eight.

Two checks the config pin does not do, and why that is fine

  • No gitlink moves with it — config/gpu-kernels.config is the single source (grep finds the old sha only on the line this PR replaces), so there is no second place to forget. Pins that exist in two files are the ones that drift.
  • --online still reports the fork tip rather than upstream's, which is the right default for this repo: the config names 1bit-MONSTER/gpu-kernels, and our fork is what the engine builds against. If upstream rocm/... ever needs to be the reference, that is a different config.

One note, not a request: git ls-remote <url> main printed two lines for this repo (1356edde… and 60b8eef…), and 1356edde is a different commit (bench: merged-fork decode row on strixhalo…). Querying refs/heads/main explicitly returns only the real tip. I could not explain the double line — the ref filter should be unambiguous — but it is a property of what the remote advertises, not of this pin, and I am recording it rather than assuming it away, because a pin check that parses the first line of ls-remote would have taken the wrong sha.

@bong-water-water-bong
bong-water-water-bong marked this pull request as ready for review November 15, 2026 19:16
@github-actions

Copy link
Copy Markdown

Persistent review updated to latest commit 9cea2d8

@bong-water-water-bong
bong-water-water-bong merged commit f005209 into main Nov 15, 2026
12 checks passed
@bong-water-water-bong
bong-water-water-bong deleted the cl-1051/gk-pin-97 branch November 15, 2026 19:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant