Upstream: ggml-org#28455
Problem: A default CUDA build accepts -fa on -ctk q5_1 -ctv q5_1 but silently schedules every FlashAttention node on CPU, causing a large slowdown.
Environment: upstream 6a1a922d2 (b10819), NVIDIA GB10, CUDA 13.0, Linux, default GGML_CUDA_FA_ALL_QUANTS=OFF, Gemma 4 12B Q4_K_M.
Reproduction:
./build/bin/llama-bench -m gemma-4-12B-it-Q4_K_M.gguf -fa on -ctk q5_1 -ctv q5_1 -p 512 -n 64 -ngl 99 -t 12 -r 3
GGML_SCHED_DEBUG=2 ./build/bin/llama-bench -v -m gemma-4-12B-it-Q4_K_M.gguf -fa on -ctk q5_1 -ctv q5_1 -p 32 -n 1 -ngl 99 -t 12 -r 1
Observed: q5_1/q5_1 measured 378.70 t/s pp512 and 9.35 t/s tg64; the scheduler trace assigned FLASH_ATTN to CPU. The q8_0/q8_0 control measured 1899.28 t/s and 27.05 t/s, with FLASH_ATTN assigned to CUDA0.
Likely cause: The default FA kernel set excludes q5_1, so CUDA rejects the op and the scheduler silently falls back to CPU. The default restricted kernel set was introduced by 9b596417a (ggml-org#7527).
Upstream: ggml-org#28455
Problem: A default CUDA build accepts
-fa on -ctk q5_1 -ctv q5_1but silently schedules every FlashAttention node on CPU, causing a large slowdown.Environment: upstream
6a1a922d2(b10819), NVIDIA GB10, CUDA 13.0, Linux, defaultGGML_CUDA_FA_ALL_QUANTS=OFF, Gemma 4 12B Q4_K_M.Reproduction:
Observed: q5_1/q5_1 measured 378.70 t/s pp512 and 9.35 t/s tg64; the scheduler trace assigned
FLASH_ATTNtoCPU. The q8_0/q8_0 control measured 1899.28 t/s and 27.05 t/s, withFLASH_ATTNassigned toCUDA0.Likely cause: The default FA kernel set excludes q5_1, so CUDA rejects the op and the scheduler silently falls back to CPU. The default restricted kernel set was introduced by
9b596417a(ggml-org#7527).