qwen4exp : enable sparse flash attention in build_attn_qsa

[sammcj]

Overview

Pass the indexer top-k width as n_kv_max in build_attn_qsa, so backends with a sparse FA path (#27970 CUDA, #28098 Metal) gather the selected KV rows instead of masking the full cache. Same call as the MLA sparse path in llama-graph.cpp.

The mask is set_rows over the top-k indices plus the causal mask, so each row has at most top_k->ne[0] finite entries, which is the bound ggml_flash_attn_ext_set_n_kv_max requires.

M5 Max, Qwen3.8-Flash-Next IQ4_XS, q8_0 KV, -ub 2048 -b 2048, llama-bench -fa 1 -r 2:

test	master	PR
pp2048 @ d65536	388.1 ± 3.6	702.2 ± 9.0
pp2048 @ d131072	340.0 ± 4.8	588.7 ± 1.6
tg32 @ d16384	35.0 ± 5.2	39.4 ± 0.2
tg32 @ d32768	34.5 ± 0.5	35.1 ± 0.3
tg32 @ d65536	28.3 ± 0.2	28.7 ± 0.1
tg32 @ d131072	20.6 ± 0.1	20.7 ± 0.1

llama-server cold prefill, three ~33k prompts: 664 -> 760 t/s. Greedy output over a 36k prompt matched master.

Measured on master + #27836 (MTP), which does not touch this path. CUDA not tested here. Its gate falls back to dense below 4096 KV, and CPU and Vulkan ignore the hint.

Additional information

Removes the TODO from #28098. The change is the one @am17an suggested in #27970 (comment), with the Metal numbers in #28098 (comment).

Requirements

• I have read and agree with the contributing guidelines
• AI usage disclosure: YES, Claude (Fable 5.1) ran the benchmarks, checked the mask bound, and drafted the patch. Reviewed and rewritten by me.

[am17an]

You need to measure it on way more devices before enabling this change, that was the definition to "when we're ready"

[sjordan0228]

M2 Ultra results for this change.

Setup: Apple M2 Ultra (192 GB), Metal. Qwen3.8-Flash-Next Q5_K_M (DevQuasar), F16 KV, -c 131072 -np 1 --kv-unified, FA auto, no speculative decoding. Binary is the #28213 head (beed2f78a, build 10815, base 1548a240e) with this PR's one-line change behind an env switch, so masked, sparse-FA and gather run on the same build with everything else identical. Confirmed the sparse path is active with -lv 5: all FA-vec pipelines compiled as ..._sparse=1_....

GGUF note: DevQuasar's Q5_K_M has qwen4exp.attention.compress_ratios set to all zeros (converted 2026-08-26, before #27742 landed). That disables QSA, so with that file none of these paths do anything. I patched shard 1 to [0,0,0,4]x12, which matches the HF config (indexer_compress_ratio: 4) and unsloth's conversion. The zeroed file doubles as a dense reference and is included below.

Method: three fixed prompts (about 2k, 32k and 128k tokens), cache_prompt: false, 512 tokens greedy, warm-up plus 3 runs, median. PP is prompt_tokens / TTFT on a cold full-prompt prefill. TG is decode tokens / (total - TTFT). Run-to-run noise on this machine is 0.1-0.2%.

path	2k TG	2k PP	32k TG	32k PP	128k TG	128k PP	128k TTFT
masked (master, n_kv_max = 0)	32.10	522	24.98	454	15.02	335	382 s
sparse-FA (this PR)	31.32	488	24.05	464	14.38	401	319 s
gather (#28213, reference)	32.02	522	23.62	451	14.84	332	386 s
dense (zeroed ratios, reference)	35.04	531	31.21	472	21.66	289	443 s

On this device the change is a prefill win and a small decode loss. Prefill: +20% over masked at 128k (cold 128k TTFT 382 s to 319 s), -2% at 32k, -7% at 2k where the top-k list covers every cell. Decode: -2.4% at 2k, -3.7% at 32k, -4.3% at 128k. No decode gain at any depth, unlike the M5 Max numbers above.

Separately: all three QSA variants decode within 4% of each other and about 31% below the dense reference at 128k on Metal, so the per-token cost at depth is in build_qsa_top_k, not the attention scan. TOP_K alone measures 198 us at 131072 cells in test-backend-ops perf, about 2.4 ms per token across 12 layers; the rest is the block-key re-pooling. Details on #28213.