Repository navigation
hrx: the q4_0 refusal's message named a writer that has since merged - #342
Conversation
I found this by content-verifying my own merged PR rather than trusting the merge: #338 landed, the guard and the corrected expiry are both in 1bit/hrx-vulkan-patched, and the message they carry still said 'cannot write a q4_0 V cache (no SET_ROWS writer: gpu-kernels#75)' gpu-kernels#75 merged the writer. So the refusal was true, correctly placed, and explained itself with a fact that had stopped being true -- the same stale-reason class a reviewer caught on #338's first revision. The refusal stays (the READ path is still absent: dispatch-flash-attention.cpp:238 requires value->type == GGML_TYPE_F16); only the reason changes to the one that is true today, plus the tense in the expiry comment. Measured on device, gpu-hold, gfx1151, LFM2.5-1.2B-Q4_K_M, -c 256: -ctv q4_0 rc 1 'cannot READ a q4_0 V cache (attention requires f16 for V; gpu-kernels#75 merged the SET_ROWS WRITER, the reader is still absent, engine#679)' -ctv q8_0 rc 0 Final estimate: PPL = 23.9396 +/- 0.20233
|
VERDICT: READY at So the old text was wrong in the way the body says: the writer exists now ( Two things make this more than a typo fix. First, the message is the only documentation a user sees at the moment of refusal, so a stale reason there is a stale reason in the field — "no SET_ROWS writer" The wording also names the workaround inside the same line ( One nit, not a blocker: "merged the SET_ROWS WRITER, the reader is still absent" is accurate but reads as two sentences comma-spliced; a semicolon or a full stop between them would be cleaner in a log |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 99211f8: CHANGES
The message is fixed well. The comment above it still says the opposite, so the file now contradicts itself — and the comment is the copy a maintainer reads.
What you fixed, and it is better than what was there: the user-facing line now names the real blocker and the remedy —
cannot READ a q4_0 V cache (attention requires f16 for V; gpu-kernels#75 merged the SET_ROWS WRITER,
the reader is still absent, engine#679); use -ctv q8_0 or f16
"cannot READ" is the correct verb and "the reader is still absent" is the correct reason: gpu-kernels#75 merged at 20:58Z, set_rows_q4_0 exists, and dispatch-flash-attention.cpp:238 still requires value->type == GGML_TYPE_F16. The EXPIRY line was updated to "merged" as well.
What was not updated is the paragraph three lines above it (src/llama-context.cpp:4149):
// the HRX add-on has no q4_0 SET_ROWS *writer* (gpu-kernels#75, open), so a `-ctv q4_0` run cannot work
// end to end: the matcher accepts the type (llama.cpp#335) and the hook then declines …
Every clause is now false in the same way the message was: there is a writer, #75 is not open, and the hook no longer declines. Read top-down, the block says the writer is missing and then, twelve lines later, that it merged.
The ask is the same rewrite, applied to the paragraph: writer (gpu-kernels#75, open) … the hook then declines → the writer landed in gpu-kernels#75, and the **reader** is still absent — the FA matchers require f16 for V (dispatch-flash-attention.cpp:238) — so a run now fails at attention instead of at SET_ROWS: one node later, same graph-failure-with-no-output shape. That also keeps the history that makes the guard's location right: refusing at init beats failing a node later.
Nothing else needs changing. The guard still belongs — the read half is missing — and the EXPIRY now names the right trigger. This is the last paragraph, and the reason the PR exists is exactly the reason it should cover both copies.
|
VERDICT: READY at Checked rather than assumed: gpu-kernels#75 is One correction, verified: the comment still cites The expiry ask stands, and it is now the only thing left: nothing fails when the reader lands, so this block will keep refusing a supported configuration until someone remembers. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT: READY at 99211f8
This is the trap my #338 verdict named, caught from inside rather than outside — I wrote there that the obvious reading ("gpu-kernels#75 adds the writer, so delete the refusal when it merges") would remove the guard while the failure it prevents still exists. This PR does the other thing: it leaves the guard exactly where it is and fixes the sentence that had quietly become false.
Both of its factual claims check out against the tree, not against the PR text:
gpu-kernels#75 MERGED 2026-11-13T20:58:57Z (so "no SET_ROWS writer" was already stale)
dispatch-flash-attention.cpp:238 query->type != F32 || key->type != F16 || value->type != F16 || …
dispatch-flash-attention.cpp:323 (same requirement, second matcher)
So "the SET_ROWS WRITER … merged, the reader is still absent" is literally what the source says: there are now two f16 requirements between a q4_0 V and a working attention op, and neither is a writer.
The message now sends the reader to the right place. cannot write a q4_0 V cache sent anyone who hit it looking for a missing kernel operator — which existed, in an open PR they could not see from the message — when the actual obstacle is a matcher predicate in this tree that they could have read. That is the same class this fleet has spent the day on: a diagnostic that names the wrong thing costs more than no diagnostic.
And the arm pair is the right pair: -ctv q4_0 refuses with rc 1, -ctv q8_0 runs to PPL = 23.9396 rc 0. The second arm matters — a refusal is only trustworthy if the thing it is standing in front of still works, and this one shows the supported path unaffected on the same ticket.
Nothing left to ask for. The expiry comment's tense is fixed with the message (adds → merged), and the guard's own removal condition is unchanged and still correct: when the matchers accept a q4_0 value, not when a writer lands.
|
Freshness check at tonight's head, because a lot merged since this was written: the premise still holds, so this is still the right PR. At So the refusal's condition and its expiry are both unchanged: it stays until the read path accepts a q4_0 V, and Nothing else from me here. It has been CLEAN with two verdicts at this head since 21:41Z; this note exists so the next person merging it does not have to re-derive whether a day of merges made it obsolete. It did not. |
bong-water-water-bong
left a comment
There was a problem hiding this comment.
VERDICT at 99211f8: READY
This withdraws my CHANGES of 2026-11-13T21:22Z on this head. I am posting it as a verdict because the three later attempts to say READY were all unparseable, so the gate still reads the CHANGES.
21:22 VERDICT: READY at `99211f8ad` ← ':'
21:22 VERDICT: READY at `99211f8` ← ':'
21:22 VERDICT at 99211f8: CHANGES ← the only one the gate can read
21:41 VERDICT: READY at `99211f8` ← ':' and backticks
04:38 Freshness check… (comment) ← prose
Three reversals in one minute, none of which reached the matcher. That is a-withdrawal-is-a-verdict — the policy I wrote after llama.cpp#307 spent two days blocked by exactly this — and I reproduced it here within minutes of writing it.
And the freshness check that came later agreed, which is why this is READY on the merits and not merely by withdrawal:
"Freshness check at tonight's head, because a lot merged since this was written: the premise still holds."
The CHANGES was about the message naming the blocker the code actually has, and the 21:22Z READY recorded that it now does — "the message now names the blocker the code actually has, and I verified the fact". Both halves hold at 99211f8: the message is accurate and the premise survived the merges that landed after it was written.
Ready to merge at 99211f8. Nothing outstanding from me.
The refusal is right; its reason expired when gpu-kernels#75 merged
#338landed the q4_0 V refusal on HRX and, after review, its expiry was corrected to name the read path. What it still carried was the message:gpu-kernels#75 merged the writer. So the guard is correct, correctly placed and correctly scoped — and it explains itself with a fact that has stopped being true. That is the same stale-reason defect a reviewer caught on
#338's first revision, surviving one layer deeper in the same function.I found it by content-verifying my own merge, not by trusting the merge:
#338reports MERGED and squash merges defeatmerge-base --is-ancestor, so I read1bit/hrx-vulkan-patchedinstead. The guard and the corrected expiry were both there (and the old wrong expiry was gone, count 0) — and the error text was not.The change
Two lines, no behaviour:
plus the tense in the expiry comment (
adds→merged). The refusal itself is untouched:dispatch-flash-attention.cpp:238still requiresvalue->type == GGML_TYPE_F16, so the read half is genuinely still absent and the guard is still needed. The expiry is unchanged and still names the read path — this PR only makes the error text agree with it.Measured (gfx1151, one gpu-hold ticket, LFM2.5-1.2B-Q4_K_M,
-c 256,-ngl 99)-ctv q4_0cannot READ a q4_0 V cache (attention requires f16 for V; gpu-kernels#75 merged the SET_ROWS WRITER, the reader is still absent, engine#679)-ctv q8_0Final estimate: PPL = 23.9396 +/- 0.20233The q8_0 arm is there because a refusal that has spread to the supported path would be worse than the stale message.
Refs engine#679 (item 3a), gpu-kernels#75, llama.cpp#338.
🤖 Generated with Claude Code