WIP: Strix Halo Vulkan stack (FA KV-quant, mmid, dense GEMM, delta-net prefill, DSv4 sparse attention) - #13

Closed

Nathanw1014 wants to merge 115 commits into

staging/vulkan-stack-20260830

Commits on Aug 27, 2026

ci : build only the ggml-hip backend for windows-rocm release (#27753)
harkgill-amd
authored
3217633
server: add ctx-per-slot (--kv-unified-per-slot) (#24124)

bartowski1182
and
ngxson
authored
1844325
hex-unary: fix RMS_NORM_MUL weight-offset bugs for grouped/broadcast norms (#27798)
aparmp-quic
authored
83d855c
ggml-hexagon: add HTP unary ops for ABS and LOG (#27786)
cqderek
authored
e70802a
model: add DSpark support for Nemotron3.5 (#27804)

authored
ca3d5a3

Commits on Aug 28, 2026

tests : run test-save-load-state across all architectures (#27755)
ggerganov
authored
4e97ac8
metal : add fa-vec tunings for M4 Pro (#27824)
infinitewarp
authored
6d6b697
metal : add fa-vec tunings for M3 Max, M5 and M5 Pro (#27863)
ggerganov
authored
8963a9b
sycl: bind the f16 KV cache in place for the oneDNN SDPA path (#27468)
Titaniumtown
authored
be87620
sycl: use TILE for quantized KV decode on BMG (#26689)
johnkarlhill
authored
d077b4c
convert: prevent ndarray conversion in LazyChunkedTensor (#27869)
ngxson
authored
b19cbe9
OpenVINO: Update OV to 2026.3.1, whisper.cpp support, Qwen3.5 on NPU, and new ops (#27843)

authored
511f9c1
metal : add fa-vec tunings for M4 (#27875)
Strongtut
authored
f5e85d4
context : disable non-fused GDN and LID ops (#27877)
ggerganov
authored
8663224
Vulkan: add hoisting support for row IDs and expert count in shaders (#26686)

ravel7524
and
jeffbolznv
authored
90c26fc
ggml : fix conv_transpose_2d for multiple batches (#26132)

tekinertekin
and
ggerganov
authored
a43c398
vulkan: fix missing view-alias dependencies in ggml_vk_graph_optimize (#27812)

Eric-A-Stalee
and
jeffbolznv
authored
b387ddf
model: qwen4exp: reduce number of graph splits (#27880)
ngxson
authored
6fe7498
bench: add --tensor-read-lazy (#27881)
ngxson
authored
50f068f
snapdragon: python SDK setup (Windows) (#27903)

kurquhar
and
Kristopher Urquhart
authored
d7bd3bf

Commits on Aug 29, 2026

Commits on Aug 30, 2026

ggml: allow passing alloc dependencies in graph_optimize (#27301)
am17an
authored
57291f2
memory : copy Hadamard matrix to k_rot tensor only if it has buffer assigned to prevent crashes during context shift of unquantized K cache (#27967)

authored
bdf3955
metal : fix null-pipeline crash for F16 src1 mul_mat/mul_mat_id (#25648)
QuintinShaw
authored
d882575
sycl: split long rows in TOP_K instead of one work-group per row (#27847)
Titaniumtown
authored
370cb12
hexagon: support for device discovery and create sessions on demand (#27785)

authored
9e54e68
rpc : fix pre-rdma macOS versions (#27815)
ryan5rdx
authored
2bf0415
vendor : update cpp-httplib to 0.54.0 (#27919)
cabelo
authored
dc7aecf
metal : Add fa-vec tuning for M3 Pro (#27963)
addianto
authored
b8b743c
metal: add fa-vec tunings for M3 Ultra (#27999)
ngladitz
authored
093adb2
rpc: fix apple rdma error spew on teardown (#27908)
ryan5rdx
authored
742347b
ggml : add ggml_backend_op_alloc_size_may_expand, use it in RPC (#27960)
ggerganov
authored
73f56d1
common: rename --tensor-read-lazy to --lazy-mode, add -lzm shorthand (#27969)
ggerganov
authored
bebc935
experiment: dequant-once FA scratch for all KV quant types (q4_0/q4_1/q5_0/q5_1/iq4_nl)
Nathanw1014
committed
816c7bd
vulkan : gate FA dequant-once scratch on device-local capacity
Nathanw1014
committed
0aac213
vulkan: contiguize strided f16 KV for FA prefill (GGML_VK_FA_KV_CONTIG, env-gated)

Nathanw1014
and
claude
committed
c8230e7
tests: dense-permuted K/V option + Strix FA prefill perf/probe cases

Nathanw1014
and
claude
committed
40aaa49
vulkan: enable the f16 KV contiguize pass by default (GGML_VK_FA_KV_CONTIG=0 opts out)

Nathanw1014
and
claude
committed
e7a0e58
vulkan: single source of truth for native FA K/V types + non-native hard-gate

Nathanw1014
and
claude
committed
bf54e45
vulkan: hoist the coopmat1 FA P-fragment load out of the hsv_tile loop
Nathanw1014
committed
66e7348
tests: dense-permuted iq4_nl FA cases (dequant-once route coverage)

Nathanw1014
and
claude
committed
6a6896c
vulkan: store coopmat1 FA Psh query-major so the GEMM2 A load vectorizes
Nathanw1014
committed
306eecf
vulkan: bill the FA K/V contiguize pass on its own perf-logger line
Nathanw1014
committed
48407ef
vulkan: pin a 32-wide subgroup for coopmat1 FA where narrowing is free
Nathanw1014
committed
d7124e9
vulkan: scale the FA MMQ dot product in fp32 before narrowing

Nathanw1014
and
claude
committed
c4444f9
vulkan: enable the coopmat1 FA wave32 narrowing rule by default

Nathanw1014
and
claude
committed
a94369d
vulkan: mul_mat_id per-expert-n tile selection (Stage 2a, env-gated)

Nathanw1014
and
claude
committed
41d17f1
vulkan: mul_mat_id small-tile shape probes (env-gated)

Nathanw1014
and
claude
committed
7f0c243
vulkan: mul_mat_id taller medium tile probe (GGML_VK_MMID_M128, env-gated)

Nathanw1014
and
claude
committed
afedbab
vulkan: mmid wave32 probe (GGML_VK_MMID_WAVE32, env-gated)

Nathanw1014
and
claude
committed
7722eb4
vulkan: mul_mat_id f16-B probe (GGML_VK_MMID_F16B, env-gated)

Nathanw1014
and
claude
committed
0986850
vulkan: guard mmid f16-B path on pipeline existence (Q2_0 fallback)
Nathanw1014
committed
633ab28
tests: cover MMQ tile boundaries in MUL_MAT and MUL_MAT_ID

Nathanw1014
and
claude
committed
bc235d8
vulkan: create coopmat2 mul_mat_id pipelines with the real param count

Nathanw1014
and
claude
committed
3df1fd5
vulkan: enable the Strix mmid tile gates by default

Nathanw1014
and
claude
committed
266166f
vulkan : optional f16 B operand for quantized MUL_MAT on coopmat1
Nathanw1014
committed
fcf2f38
vulkan : add auto mode to GGML_VK_DENSE_F16B
Nathanw1014
committed
74b3efa
vulkan: run the quantised dense coopmat pipelines at wave32
Nathanw1014
committed
bc3638c
vulkan: tune the coopmat matmul LDS pad per path on RADV
Nathanw1014
committed
beea09e
vulkan: gate the RADV coopmat pad 2 on driver >= 25.3
Nathanw1014
committed
73608d2
vulkan: four env-gated Strix Halo prefill fixes for delta-net MoE

Nathanw1014
and
claude
committed
b9e43c5
vulkan: reject ne[3] > 1 in the mul_mat_id scale epilogue

Nathanw1014
and
claude
committed
7a1eb6d
vulkan : default the transposed-concat path on
Nathanw1014
committed
0d542b8
vulkan: flush pending compute ctx before perf logger timestamps
Nathanw1014
committed
ce93d29
vulkan: bound command buffers by memory traffic, not just flops
Nathanw1014
committed
8a8c218
ggml: cut backend splits on the input constant, not the grown capacity

Nathanw1014
and
claude
committed
9125bb2
vulkan : take the copy path for bulk UMA reads from uncached mappings
Nathanw1014
committed
e4615cc
vulkan: DeepSeek V4 lightning indexer kernels + indexed sparse FA

gaetan-puleo
authored and
Nathanw1014
committed
69e3052
vulkan: harden the sparse-FA shader and document the top-k API

Nathanw1014
and
claude
committed
aa04dcc
tests: sparse top-k FA parity + perf coverage (V4 CSA shape)

Nathanw1014
and
claude
committed
5046bf6
vulkan: gather-to-compact sparse decode FA for DeepSeek V4 top-k selection

Nathanw1014
and
claude
committed
8a15795
vulkan: fused DeepSeek V4 hyper-connection ops (HC pre / comb / post)

Nathanw1014
and
claude
committed
a27636c
llama: keep DeepSeek lightning-indexer key cache f16 under quantized -ctk
Nathanw1014
committed
631fd48
llama: contiguize grouped o-proj input for small multi-token batches (DSv4)
Nathanw1014
committed
0e71642
vulkan: accelerate DeepSeek V4 sparse prefill FA

Mushoz
authored and
Nathanw1014
committed
aca6680
vulkan: split sparse prefill attention

Mushoz
authored and
Nathanw1014
committed
10c5457
vulkan: tile sparse prefill scratch

Mushoz
authored and
Nathanw1014
committed
aa5085f
vulkan: reuse sparse FA probability fragments

Mushoz
authored and
Nathanw1014
committed
8a4a20b
vulkan: cache sparse FA masks per key block

Mushoz
authored and
Nathanw1014
committed
13bedfe
vulkan: fix DeepSeek V4 sparse split attention with multiple sequences

Nathanw1014
and
claude
committed
2c79376
vulkan: harden the sparse FA split path and drop its debug scaffolding

Nathanw1014
and
claude
committed
9c950c5
test-backend-ops: cover sparse top-k FA with more than one sequence

Nathanw1014
and
claude
committed
09f6a7d
vulkan: let sparse FA query tiling and multiple sequences coexist

Nathanw1014
and
claude
committed
932245a
vulkan: parallelize DSV4 Lightning Indexer prefill

Mushoz
authored and
Nathanw1014
committed
ff7ac3d
vulkan: extend DeepSeek V4 gather-to-compact to small batches

Nathanw1014
and
claude
committed
93f18c3
vulkan: record the resource limits behind the two DSV4 prefill kernels

Nathanw1014
and
claude
committed
4c35864
vulkan: deduplicated union for DeepSeek V4 small-batch decode

Nathanw1014
and
claude
committed
7532a73
vulkan: gate the DeepSeek V4 small-batch union on the measured union size

Nathanw1014
and
claude
committed
0ff9e37
vulkan: measure the V4 union on real draft tokens, and correct the fixture calibration

Nathanw1014
and
claude
committed
ce9fef7
vulkan: default the DeepSeek V4 small-batch union on

Nathanw1014
and
claude
committed
7dfe8c8
vulkan: let the DeepSeek V4 small-batch gather serve quantised K/V

Nathanw1014
and
claude
committed
8851c37
vulkan: dequantise q8_0 K/V inside the DeepSeek V4 small-batch gather

Nathanw1014
and
claude
committed
6b7c998
vulkan: decode q4_0 in the V4 gather too, with the real element mapping

Nathanw1014
and
claude
committed
2cd621b
vulkan : restore the unrolled row copy in the DSV4 gather shaders
Nathanw1014
committed
7d45323
vulkan: decode quantised K/V inside the DeepSeek V4 per-token gather

Nathanw1014
and
claude
committed
14f7c55
vulkan: dequantise the cache for the DeepSeek V4 sparse prefill

Nathanw1014
and
claude
committed
a2b7675
vulkan: use small Lightning Indexer CM for batches 4-15

pepuscz
authored and
Nathanw1014
committed
ca50763
vulkan: hoist the Lightning Indexer K fragments out of the head loop

Nathanw1014
and
claude
committed
1ca0f9a
vulkan: route the whole small-batch Lightning Indexer window to the decode CM shader

Nathanw1014
and
claude
committed
06b89c3
vulkan: hoist the Lightning Indexer K fragments out of the CM head loop

Nathanw1014
and
claude
committed
849aca6
vulkan: support arbitrary Lightning Indexer head counts via specialization constant

Nathanw1014
and
claude
committed
e85148e
vulkan: N_HEAD spec constant for the scalar-64 lightning indexer
Nathanw1014
committed
46b7ed7
Merge branch 'vulkan/dsv4' into stage/strix-stacked
Nathanw1014
committed
b19d076
Merge branch 'vulkan/fa-coopmat1-micro' into stage/strix-stacked
Nathanw1014
committed
d24b48c
Merge branch 'vulkan/mmid-stack' into stage/strix-stacked
Nathanw1014
committed
3f32113
Merge branch 'vulkan/dense-gemm' into stage/strix-stacked
Nathanw1014
committed
4366d29
Merge branch 'vulkan/deltanet-prefill' into stage/strix-stacked
Nathanw1014
committed
f34e16a
Merge branch 'vulkan/strix-misc' into stage/strix-stacked
Nathanw1014
committed
a6e38dd