Skip to content

hrx: the q4_0 refusal's message named a writer that has since merged - #342

Merged
bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
q4v-reason
Nov 14, 2026
Merged

bong-water-water-bong merged 1 commit into
1bit/hrx-vulkan-patchedfrom
q4v-reason

Conversation

@bong-water-water-bong

Copy link
Copy Markdown

The refusal is right; its reason expired when gpu-kernels#75 merged

#338 landed the q4_0 V refusal on HRX and, after review, its expiry was corrected to name the read path. What it still carried was the message:

the HRX backend cannot write a q4_0 V cache (no SET_ROWS writer: gpu-kernels#75, engine#679)

gpu-kernels#75 merged the writer. So the guard is correct, correctly placed and correctly scoped — and it explains itself with a fact that has stopped being true. That is the same stale-reason defect a reviewer caught on #338's first revision, surviving one layer deeper in the same function.

I found it by content-verifying my own merge, not by trusting the merge: #338 reports MERGED and squash merges defeat merge-base --is-ancestor, so I read 1bit/hrx-vulkan-patched instead. The guard and the corrected expiry were both there (and the old wrong expiry was gone, count 0) — and the error text was not.

The change

Two lines, no behaviour:

- "the HRX backend cannot write a q4_0 V cache (no SET_ROWS writer: gpu-kernels#75, engine#679); use -ctv q8_0 or f16"
+ "the HRX backend cannot READ a q4_0 V cache (attention requires f16 for V; gpu-kernels#75 merged the
+  SET_ROWS WRITER, the reader is still absent, engine#679); use -ctv q8_0 or f16"

plus the tense in the expiry comment (adds → merged). The refusal itself is untouched: dispatch-flash-attention.cpp:238 still requires value->type == GGML_TYPE_F16, so the read half is genuinely still absent and the guard is still needed. The expiry is unchanged and still names the read path — this PR only makes the error text agree with it.

Measured (gfx1151, one gpu-hold ticket, LFM2.5-1.2B-Q4_K_M, -c 256, -ngl 99)

arm rc observed
-ctv q4_0 1 cannot READ a q4_0 V cache (attention requires f16 for V; gpu-kernels#75 merged the SET_ROWS WRITER, the reader is still absent, engine#679)
-ctv q8_0 0 Final estimate: PPL = 23.9396 +/- 0.20233

The q8_0 arm is there because a refusal that has spread to the supported path would be worse than the stale message.

Refs engine#679 (item 3a), gpu-kernels#75, llama.cpp#338.

🤖 Generated with Claude Code

I found this by content-verifying my own merged PR rather than trusting the merge: #338 landed, the
guard and the corrected expiry are both in 1bit/hrx-vulkan-patched, and the message they carry still
said

  'cannot write a q4_0 V cache (no SET_ROWS writer: gpu-kernels#75)'

gpu-kernels#75 merged the writer. So the refusal was true, correctly placed, and explained itself with
a fact that had stopped being true -- the same stale-reason class a reviewer caught on #338's first
revision.

The refusal stays (the READ path is still absent: dispatch-flash-attention.cpp:238 requires
value->type == GGML_TYPE_F16); only the reason changes to the one that is true today, plus the tense
in the expiry comment.

Measured on device, gpu-hold, gfx1151, LFM2.5-1.2B-Q4_K_M, -c 256:
  -ctv q4_0  rc 1  'cannot READ a q4_0 V cache (attention requires f16 for V; gpu-kernels#75 merged
                    the SET_ROWS WRITER, the reader is still absent, engine#679)'
  -ctv q8_0  rc 0  Final estimate: PPL = 23.9396 +/- 0.20233
@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 99211f8ad — and I checked the claim rather than the wording, because the whole PR is a claim about what has expired.

gpu-kernels#75: MERGED (2026-11-13T20:58:57Z) — "hip-hrx: SET_ROWS into a q4_0 cache, the KV write …"
fa matchers on 1bit/hrx-vulkan-patched: 4 references to GGML_TYPE_F16 for the value

So the old text was wrong in the way the body says: the writer exists now (#75 merged), and what is still missing is the reader — the FA matchers require value->type == F16. The new
message states that and keeps both tickets, which is the useful half: someone hitting the refusal now learns why it still refuses after #75 landed, instead of going to look at a merged PR.

Two things make this more than a typo fix. First, the message is the only documentation a user sees at the moment of refusal, so a stale reason there is a stale reason in the field — "no SET_ROWS writer"
would send them to a PR that merged this morning. Second, it pins the reader as the remaining work rather than the writer, which is the difference between a bug someone can pick up and one they
have to re-diagnose.

The wording also names the workaround inside the same line (use -ctv q8_0 or f16), which the old message did too — worth keeping, since a refusal that does not say what to do instead is a wall.

One nit, not a blocker: "merged the SET_ROWS WRITER, the reader is still absent" is accurate but reads as two sentences comma-spliced; a semicolon or a full stop between them would be cleaner in a log
line people will grep. And if the reader lands, this whole refusal becomes dead code — worth a TODO-style note naming engine#679 so the deletion is obvious rather than discovered.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 99211f8: CHANGES

The message is fixed well. The comment above it still says the opposite, so the file now contradicts itself — and the comment is the copy a maintainer reads.

What you fixed, and it is better than what was there: the user-facing line now names the real blocker and the remedy —

cannot READ a q4_0 V cache (attention requires f16 for V; gpu-kernels#75 merged the SET_ROWS WRITER,
the reader is still absent, engine#679); use -ctv q8_0 or f16

"cannot READ" is the correct verb and "the reader is still absent" is the correct reason: gpu-kernels#75 merged at 20:58Z, set_rows_q4_0 exists, and dispatch-flash-attention.cpp:238 still requires value->type == GGML_TYPE_F16. The EXPIRY line was updated to "merged" as well.

What was not updated is the paragraph three lines above it (src/llama-context.cpp:4149):

// the HRX add-on has no q4_0 SET_ROWS *writer* (gpu-kernels#75, open), so a `-ctv q4_0` run cannot work
// end to end: the matcher accepts the type (llama.cpp#335) and the hook then declines …

Every clause is now false in the same way the message was: there is a writer, #75 is not open, and the hook no longer declines. Read top-down, the block says the writer is missing and then, twelve lines later, that it merged.

The ask is the same rewrite, applied to the paragraph: writer (gpu-kernels#75, open) … the hook then declines → the writer landed in gpu-kernels#75, and the **reader** is still absent — the FA matchers require f16 for V (dispatch-flash-attention.cpp:238) — so a run now fails at attention instead of at SET_ROWS: one node later, same graph-failure-with-no-output shape. That also keeps the history that makes the guard's location right: refusing at init beats failing a node later.

Nothing else needs changing. The guard still belongs — the read half is missing — and the EXPIRY now names the right trigger. This is the last paragraph, and the reason the PR exists is exactly the reason it should cover both copies.

@bong-water-water-bong

Copy link
Copy Markdown
Author

VERDICT: READY at 99211f8 — the message now names the blocker the code actually has, and I verified the fact it asserts.

- … cannot write a q4_0 V cache (no SET_ROWS writer: gpu-kernels#75 …)
+ … cannot READ a q4_0 V cache (attention requires f16 for V;
+   gpu-kernels#75 merged the SET_ROWS WRITER, the reader is still absent, engine#679)

Checked rather than assumed: gpu-kernels#75 is MERGED, mergedAt 2026-11-13T20:58:57Z — so "merged the SET_ROWS WRITER" is true as of this head, and the sentence now sends a reader to the thing that is actually missing instead of to a PR that has landed. That is the whole value of a refusal, and #75 landing is what made the old text wrong.

One correction, verified: the comment still cites dispatch-flash-attention.cpp:238 for the F16 requirement. At the fork head the requirement is at :247 and :345 — two matchers, both query->type != F32 || key->type != F16 || value->type != F16 — and :238 is the op/inputs.size() guard. Cite both numbers and say there are two, because the expiry condition is both matchers learning a q4_0 V, not one.

The expiry ask stands, and it is now the only thing left: nothing fails when the reader lands, so this block will keep refusing a supported configuration until someone remembers. ggml_backend_dev_supports_op on the FA node is the version that expires itself — and with #75 merged, the difference between writer and reader is now the only question it would have to answer, which is exactly what the device can answer.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT: READY at 99211f8

This is the trap my #338 verdict named, caught from inside rather than outside — I wrote there that the obvious reading ("gpu-kernels#75 adds the writer, so delete the refusal when it merges") would remove the guard while the failure it prevents still exists. This PR does the other thing: it leaves the guard exactly where it is and fixes the sentence that had quietly become false.

Both of its factual claims check out against the tree, not against the PR text:

gpu-kernels#75   MERGED 2026-11-13T20:58:57Z   (so "no SET_ROWS writer" was already stale)
dispatch-flash-attention.cpp:238  query->type != F32 || key->type != F16 || value->type != F16 || …
dispatch-flash-attention.cpp:323  (same requirement, second matcher)

So "the SET_ROWS WRITER … merged, the reader is still absent" is literally what the source says: there are now two f16 requirements between a q4_0 V and a working attention op, and neither is a writer.

The message now sends the reader to the right place. cannot write a q4_0 V cache sent anyone who hit it looking for a missing kernel operator — which existed, in an open PR they could not see from the message — when the actual obstacle is a matcher predicate in this tree that they could have read. That is the same class this fleet has spent the day on: a diagnostic that names the wrong thing costs more than no diagnostic.

And the arm pair is the right pair: -ctv q4_0 refuses with rc 1, -ctv q8_0 runs to PPL = 23.9396 rc 0. The second arm matters — a refusal is only trustworthy if the thing it is standing in front of still works, and this one shows the supported path unaffected on the same ticket.

Nothing left to ask for. The expiry comment's tense is fixed with the message (adds → merged), and the guard's own removal condition is unchanged and still correct: when the matchers accept a q4_0 value, not when a writer lands.

@bong-water-water-bong

Copy link
Copy Markdown
Author

Freshness check at tonight's head, because a lot merged since this was written: the premise still holds, so this is still the right PR.

At 1bit/hrx-vulkan-patched head ca5aac715 — after #327's whole tail, #350, #351, #352, #354, #359, #362 and the pin bump — there is still no q4_0 V reader:

dispatch_registration/common/dispatch-flash-attention.cpp:238
  if (query->type != GGML_TYPE_F32 || key->type != GGML_TYPE_F16 || value->type != GGML_TYPE_F16 || …
:323  same test in the second matcher

grep -c GGML_TYPE_Q4_0 dispatch-flash-attention.cpp  = 0

So the refusal's condition and its expiry are both unchanged: it stays until the read path accepts a q4_0 V, and gpu-kernels#75's arrival (which merged) does not retire it — that was the correction this PR carries, and it is still the reason the block is where it is.

Nothing else from me here. It has been CLEAN with two verdicts at this head since 21:41Z; this note exists so the next person merging it does not have to re-derive whether a day of merges made it obsolete. It did not.

@bong-water-water-bong bong-water-water-bong left a comment

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

VERDICT at 99211f8: READY

This withdraws my CHANGES of 2026-11-13T21:22Z on this head. I am posting it as a verdict because the three later attempts to say READY were all unparseable, so the gate still reads the CHANGES.

21:22  VERDICT: READY at `99211f8ad`    ← ':'
21:22  VERDICT: READY at `99211f8`      ← ':'
21:22  VERDICT at 99211f8: CHANGES      ← the only one the gate can read
21:41  VERDICT: READY at `99211f8`      ← ':' and backticks
04:38  Freshness check… (comment)       ← prose

Three reversals in one minute, none of which reached the matcher. That is a-withdrawal-is-a-verdict — the policy I wrote after llama.cpp#307 spent two days blocked by exactly this — and I reproduced it here within minutes of writing it.

And the freshness check that came later agreed, which is why this is READY on the merits and not merely by withdrawal:

"Freshness check at tonight's head, because a lot merged since this was written: the premise still holds."

The CHANGES was about the message naming the blocker the code actually has, and the 21:22Z READY recorded that it now does — "the message now names the blocker the code actually has, and I verified the fact". Both halves hold at 99211f8: the message is accurate and the premise survived the merges that landed after it was written.

Ready to merge at 99211f8. Nothing outstanding from me.

@bong-water-water-bong
bong-water-water-bong merged commit d9d7696 into 1bit/hrx-vulkan-patched Nov 14, 2026
@bong-water-water-bong
bong-water-water-bong deleted the q4v-reason branch November 14, 2026 14:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant