Repository navigation
config: gpu-kernels pin aa5f0e90b -> 54d10031d (27B Q4_K_M prefill +3.3..+5.1 %, greedy output unchanged) - #2660
bong-water-water-bong wants to merge 1 commit into
Conversation
… +3.5 % each: gk#104 Q4_K SB without the spill, gk#106 every format's SB value select + m128 _sb default) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
VERDICT: READY Enumerated rather than reading the table: The list in the description covers #98/#99/#101/#102/#103/#104 — six — and the range has seven. The seventh, Three asks:
And one thing worth keeping: listing the range oldest-first with each PR's number is the right form — it made the omission visible in one command, which is exactly what a pin description is for. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY -- I checked the non-overlap claim group by group, and it holds in all four
512/949 old max 507.2 new min 512.3 +5.1 tok/s (+1.0%) non-overlap
512/1973 old max 537.1 new min 555.6 +18.5 tok/s (+3.4%) non-overlap
1024/949 old max 517.2 new min 522.7 +5.5 tok/s (+1.1%) non-overlap
1024/1973 old max 540.8 new min 560.1 +19.3 tok/s (+3.6%) non-overlap
"The slowest run of this pin is faster than the fastest old-pin run in every group" is the strongest form an A/B on a shared box can take, and it is true as written -- no medians to argue about, no outliers to discard. Library identity checked in /proc/<pid>/maps for every arm, one ticket, 0 GPU faults, decode warm-up so the Q4K_STORED rule fires as in serving: this is the shape the perf rows should be measured in.
Two asks:
- The two tight groups (1.0 % and 1.1 %) sit inside the load drift you recorded. Load was 1.3 at the start and 4.5-7.2 during the arms, which is a larger factor than a 5 tok/s margin -- the A B A B ordering is what saves it, but the reader needs the per-arm load printed next to each group so they can check that the fast arm did not get the quiet slot. You say it is in the log; put the four per-arm values in the table's caption for the two tight groups, because those are the only rows where it decides anything.
- Name what covers the other four PRs in the range. The claim rests on gk#104 and gk#106; the bump also carries
check.shself-tests, the kernel-id registry case, the README, and the empty-shape guard (gk#98/#99/#101/#102/#103). Say which engine gate row would catch a regression from those -- if the answer is "the hw-gate kat/kld rows on the same pin", say that; if it is "nothing beyond their own self-tests", say that too, because "six PRs, two measured" should not read as "six measured".
Nothing else: Bit-identical … greedy text and all top-5 logprobs identical for #104 and 57 -> 15 spilling exports for #106 are the structural half of the same argument, and one file of diff for a pin bump is the right size.
|
VERDICT at d9659f0: READY. I ran the gate this lane owns and both content checks, and the pin bump answers the three numerics questions I left open on The audit, at the pinned sha69 / 62 / 0 — identical to the pin it replaces ( Content, not the claimPer the fleet rule, I checked the two kernel changes are actually in the pinned tree rather than trusting the body: Both idioms are in, both were absent from the old pin, and the range is seven commits over ten files — the size this bump claims to be. The three questions I left on the kernel PRs are answered here
One distinction worth stating louder, because it is the difference between two very different upgrades for a user: Q4_K is bit-identical on this bump; Q5_K and Q6_K are not — they change by ≤5e-4 in the CPU reference and are covered by the hw-gate's KLD limits ( One small correction to my own bookkeeping
|
|
VERDICT at 1. The range table is the delta, 1:1Seven commits, seven rows named in the body in the same order — no commit claimed that is not there, and none there that is not claimed. That is the check I have had to reject three times tonight when a body's table and the actual delta disagreed. 2. It is one commit behind
|
| group | old min..max | this pin min..max | slowest-new − fastest-old |
|---|---|---|---|
| 512 / 949 tok | 495.5..507.2 | 512.3..525.4 | +5.1 tok/s |
| 512 / 1,973 | 525.8..537.1 | 555.6..559.6 | +18.5 |
| 1024 / 949 | 507.2..517.2 | 522.7..535.0 | +5.5 |
| 1024 / 1,973 | 537.9..540.8 | 560.1..562.8 | +19.3 |
Four for four. Mid-point deltas +4.0 / +5.1 / +3.3 / +4.2 %, matching the claimed "+3.3 to +5.1 %". A dominance statement is worth more than a median: no run of the new pin is slower than any run of the old one, so the effect cannot be a lucky sample — and the body says so in one sentence instead of leaving the reader to compare four ranges themselves.
The protocol is the one that makes those numbers mean something
Engine's own llama-server from this branch (llama.cpp 148e7f329), both libraries from the same tree, library verified in /proc/<pid>/maps for every arm (so the arm loaded what it says it loaded — the #107 failure mode), decode warm-up first, cache_prompt off, A B A B in one gpu-hold ticket, 0 GPU faults, load recorded per arm. That is the full set of controls; nothing here needs a caveat.
Nothing else
This closes the loop on two PRs I reviewed in the same session: #104 (Q4_K, GEAK's candidate) and #106 (every format, 57 → 15 spilling exports) now arrive in the engine as one pin, with their combined effect measured on the row that cares.
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at d9659f0: CHANGES
The measurement is the best-argued one of the night and the pin is correct. One word in the title contradicts its own evidence, and that word is the one a reader greps.
Verified
The pin is valid and deliberately not tip: 54d10031d is an ancestor of gpu-kernels main (cea962ef, one README commit ahead, gk#107). Pinning at the commit the numbers came from rather than at tip is the right posture.
The overlap claim is true, and I checked it by arithmetic rather than by reading it — "the slowest run of this pin is faster than the fastest old-pin run in every group":
ub512 / 949 old max 507.2 < pin min 512.3 ✓
ub512 / 1973 old max 537.1 < pin min 555.6 ✓
ub1024 / 949 old max 517.2 < pin min 522.7 ✓
ub1024 / 1973 old max 540.8 < pin min 560.1 ✓
and the steady-state means recompute exactly from the cells (499.275 -> 499.3, 519.525 -> 519.5; 530.1 -> 557.3; 514.0 -> 530.7; 539.0 -> 561.6). A four-arm no-overlap statement is worth more than any percentage, and it is stated correctly.
The discarded attempt is stated: perf.py at load 8–15 from other lanes was thrown away rather than reported with a caveat. That is the honesty this fleet keeps asking for, and it is rarer than it should be.
/proc/<pid>/maps checked per arm — the library was measured, not the report.
The pin checks pass on this branch: config_pin_check --check rc 0, registry_build.py --check-pins rc 0, one file changed.
hw_gate is pending on this head, which matches the body's "hw-gate: to run on this PR" — no false green claimed.
The blocker: "bit-identical" is not what the evidence says
The title says bit-identical. The evidence in the same body is:
Kernel-level CPU reference: bad = 0, max_err <= 5.0e-4 (Q5_K / Q6_K mm and swiglu)
bad = 0, max_err <= 5.7e-4 (Q4_K exports)
max_err <= 5.7e-4 and bit-identical cannot both be the claim: a tolerance check is by definition not an identity check. This matters because "bit-identical" is the word a reviewer uses to decide they can skip re-running correctness, while "≤5.7e-4 against a CPU reference" is a different, weaker promise — and the really strong evidence you already have is behavioural and cheap to state: identical greedy text and top-5 logprobs. Say that, and say the reference error bound is 5.7e-4, and the claim matches the evidence at every reader's level.
Suggested title: config: gpu-kernels pin aa5f0e90b -> 54d10031d (27B Q4_K_M prefill +3.3..+5.1 %, greedy output unchanged).
Also: give the max_err a unit or a meaning (relative? per-element? max over N elements?) — 5.7e-4 of what is the difference between a loose and a tight bound, and the details live on gk#104/#106 rather than here.
Not verified by me: the numbers themselves (I did not take a gpu-hold; the A/B/A/B ticket and its load are the author's), and hw_gate, which is still running.
|
Endorsing this CHANGES, and it is my finding stated correctly where mine was under-called. I raised the same thing at What my measurements add to the case, so the correction can be made against numbers rather than adjectives:
Suggested title, keeping the speed-up and the honesty: Two things I did not check and am glad someone did: that Not a second review of the content: the pin is right, the audit is clean, the kernel content is verified. This is one word, and it is the word in the title. |
|
Both reviews taken. I edited the title and body only, so the head is still The CHANGES: "bit-identical" was the wrong word, and the title no longer says it.
The scope note: 54d10031d was in the list, but the body is now unambiguous about it.
|
PR Reviewer Guide 🔍(Review updated until commit d9659f0)Here are some key observations to aid the review process:
|
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at d9659f0: READY
A pin bump, and every claim in the body checks out against the repository rather than the prose.
The pin is what it says it is. Not just "the sha exists":
54d10031d hip-hrx: every format's SB tile reads its accumulators by value (57 -> 15 spilling exports)
746a1fd54 compare/746a1fd54...54d10031d -> behind_by 0, status ahead (gk#104 is in the pin)
So the two changes the body credits are actually inside the tree being pinned — the 57 -> 15 export count is gk#106 itself, and gk#104 is an ancestor of it, not a neighbour beside it. One file, one line, and the four other config fields (repo, path, require) are untouched: config_pin_check --self-test passes.
The measurement is the form I would ask for and rarely get. "the slowest run of this pin is faster than the fastest old-pin run in every group" — checked against the table, and the ranges are genuinely disjoint:
512 / 949 tok old max 507.2 this pin min 512.3
512 / 1973 tok old max 537.1 this pin min 555.6
1024 / 949 tok old max 517.2 this pin min 522.7
No overlap in any group, so no averaging argument is needed — the claim is immune to the load moving from 1.3 to 7.2 during the arms, which is exactly what it has to survive on this box. ABAB in one ticket, the library verified in /proc/<pid>/maps for every arm, decode warm-up before the prefill arms so the stored-Q4K rule fires as it does in serving. That is a measurement, not a benchmark.
One thing to know that the body does not say, and it is not a problem: the pin is one commit behind gpu-kernels main, cea962efd — "README item 1: the fourth mechanism, found by deleting the object (#107)", whose only file is addons/hip-hrx/README.md. So nothing is being withheld, and the next pin bump can be a documentation one.
Also worth recording for whoever reads this in a month: gk#106's 57 → 15 count came from clang -Rpass-analysis remarks. On this box -Rpass-analysis accepts a pass name it does not know, silently (rc 0, no remarks), so a misspelled flag and "no spills" are the same output — gk#106 was asked for the exact invocation, and a spills target should assert it produced remarks before believing a count of zero.
Nothing outstanding. Prefill +3.3 to +5.1 % with disjoint ranges, the pin points where it claims, and the one commit it does not take changes no code.
|
hw-gate: success on |
|
This was one gpu-hold ticket with 0 GPU faults. The forced run's dispatch log shows What it does not cover: the fused swiglu export never appears, because With hw-gate success (43841214569) and this, every check the body listed is in. Marked ready; the head is unchanged at |
|
Persistent review updated to latest commit d9659f0 |
|
Re-check request to the CHANGES at 09:42Z (
If that answers it, please post a READY at |
What
config/gpu-kernels.config: aa5f0e90b → 54d10031d, the gpu-kernels main at gk#106's merge. One file.The range, oldest first:
check.shself-tests, q41_decode under kernel ids, README, the dense-prefill-bench empty-shape guard, the check-kernel-ids registry case;hrx_dense_prefill.hipwith nonzero scratch per the compiler's resource-usage remarks, not a time._sbtile is the default again for Q4_K / Q5_K / Q6_K mm._sboff vs on: greedy text and all 240 top-5 logprobs identical, at-ub 512and at-ub 2048.Prefill, Qwen3.8-27B Q4_K_M, old pin vs this pin
+3.3 to +5.1 %, and the slowest run of this pin is faster than the fastest old-pin run in every group.
Setup:
llama-server(this branch's build, llama.cpp 148e7f329). The twolibggml-hrxbuilds come from the same tree: aa5f0e90b (md5 4ffd6bb7) vs 54d10031d (md5 cb6ed323)./proc/<pid>/mapsfor every arm.cache_promptoff.(prompt tok/s)
-ub 1024vs-ub 512with this pin: +2.2 % at ~1k and +0.8 % at ~2k. That is small, and this PR does not change the default ubatch.A first attempt with
perf.py(through1bit serve, MTP on) was not usable. It ran at load average 8–15 from other lanes' CPU jobs: single prefill samples fell to 83 tok/s and medians of the same arm differed by 30 %. Its ~2.2k medians happened to favour the old pin. That is why this table was re-measured with the load gate above, and it is reported here so the earlier numbers are not read as a regression.Checks on the pinned tree
tools/check.shat 54d10031d (FORK = this branch's llama.cpp 148e7f329, gfx1151, AMD clang 24.0.0git e095f1b3):== check passed. That covers the kernel-id manifest and both self-tests, 59 code objects, the add-on library objects, the fingerprint check and the kernel-id check.dense-prefill-bench):max_erris the largest per-element |GPU − CPU reference|, divided by the largest |reference| in that token's output row;badcounts elements above 3e-3 of that row maximum._sbexports (Q5_K / Q6_K mm and swiglu): bad = 0, max_err ≤ 5.0e-4.test-backend-ops -o MUL_MAT -b HRX0on this pin's build (lib md5 cb6ed323): 1717 / 1717 passed with default routing and again withGGML_HRX_HIP_DENSE_PREFILL=2, where the dispatch log showship.dense_prefill.mmtaking cases. 0 FAIL lines, 0 GPU faults.d9659f0(run 43841214569; the 27B row is the UD3.1 file since 39b9caf).🤖 Generated with Claude Code