Skip to content
Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions src/llama-context.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -4153,7 +4153,7 @@ llama_context * llama_init_from_model(
// exit code (measured on Qwen3-0.6B at gpu-kernels b0ed206, 2026-11-13). Refuse here instead, where the
// message can name the fix. Gated on the device so every other backend keeps its q4_0 V support.
// EXPIRY: remove this whole block when the READ path accepts a q4_0 V -- not when the writer lands. gpu-kernels#75
// adds `set_rows_q4_0`, and the FA matchers still require value->type == GGML_TYPE_F16
// merged `set_rows_q4_0`, and the FA matchers still require value->type == GGML_TYPE_F16
// (dispatch-flash-attention.cpp:238), so a run after #75 fails one node later, at attention, which is the same
// graph-failure-with-no-output shape this refusal exists to prevent. Two halves, two conditions.
if (params.type_v == GGML_TYPE_Q4_0) {
Expand All @@ -4164,8 +4164,9 @@ llama_context * llama_init_from_model(
on_hrx = name != nullptr && std::strncmp(name, "HRX", 3) == 0;
}
if (on_hrx) {
LLAMA_LOG_ERROR("%s: the HRX backend cannot write a q4_0 V cache (no SET_ROWS writer: gpu-kernels#75, "
"engine#679); use -ctv q8_0 or f16\n", __func__);
LLAMA_LOG_ERROR("%s: the HRX backend cannot READ a q4_0 V cache (attention requires f16 for V; "
"gpu-kernels#75 merged the SET_ROWS WRITER, the reader is still absent, engine#679); "
"use -ctv q8_0 or f16\n", __func__);
return nullptr;
}
}
Expand Down