feat(BACKEND-VULKAN-TQ1_0): TQ1_0 ternary keep-quant matmul, MoE, and rope shaders for Vulkan - #2248

Open

phantomic12 wants to merge 25 commits into

mudler:mainfrom

phantomic12:row/BACKEND-VULKAN-TQ1_0

Commits on Sep 10, 2026

spec(BACKEND-VULKAN-TQ1_0): TQ1_0 ternary keep-quant matmul for maple MoE
phantomic12
committed
c068571
feat(BACKEND-VULKAN-TQ1_0): TQ1_0 ternary keep-quant matmul, MoE, and rope shaders for Vulkan
phantomic12
committed
1373178
perf(BACKEND-VULKAN-TQ1_0): optimize 6 keep-quant shaders — 5.1→9.4 tok/s (1.85x)
phantomic12
committed
f612da5
perf(BACKEND-VULKAN-TQ1_0): multi-column workgroups + packed reads — 9.4→10.4 tok/s
phantomic12
committed
8b127c2
perf(BACKEND-VULKAN-TQ1_0): fix barrier placement + multi-column dev shaders — 10.4→11.9 tok/s
phantomic12
committed
ddd7c58
fix(BACKEND-VULKAN): enable GEMV for prefill (M>1) and flush heavy dispatches
phantomic12
committed
701a38e
fix(BACKEND-VULKAN-TQ1_0): chunk large dispatches to prevent hangcheck crashes at 8k context
phantomic12
committed
15f00d3
feat(BACKEND-VULKAN-TQ1_0): add TQ1_0/TQ2_0 dtype, CPU dequant, and quant traits
phantomic12
committed
f4f6c33
perf(BACKEND-VULKAN-TQ1_0): cache weight blocks in shared memory and add on-device MoE fast path
phantomic12
committed
fffb216
fix(BACKEND-VULKAN-TQ1_0): use mainline ggml type ids 34/35 for TQ1_0/TQ2_0
phantomic12
committed
cde72b4
fix(BACKEND-VULKAN-TQ1_0): maple model correctness and gather_k for grouped kernels
phantomic12
committed
a56fa13
fix(BACKEND-VULKAN-TQ1_0): review fixes — shaderFloat64 gate, barrier safety, fence errors, N param fix
phantomic12
committed
31f513e
perf(BACKEND-VULKAN-TQ1_0): byte-at-a-time trit extraction for non-dev shaders
phantomic12
committed
68d722e
test(BACKEND-VULKAN): update SPIR-V module count for vt_matmul_tiled
phantomic12
committed
04ee518
bench(BACKEND-VULKAN-TQ): add TQ1_0/TQ2_0 keep-quant microbenchmark
phantomic12
committed
e2c64ce
perf(BACKEND-VULKAN-IDOT): dotPacked4x8EXT for TQ1_0/TQ2_0 shaders
phantomic12
committed
2e35bd9
fix(BACKEND-VULKAN-TQ): gate IDOT dispatch on nb even to prevent OOB reads
phantomic12
committed
8fd6dd5
fix(BACKEND-VULKAN-TQ1_0): rebase fixes — GgufLoadPolicy device arg, brace, test sweep
phantomic12
committed
0a85058
perf(BACKEND-VULKAN-TQ): cache MoE weight blocks in shared memory
phantomic12
committed
5f5d39c
perf(BACKEND-VULKAN-TQ): cache weights + subgroup amax in grouped dev shaders
phantomic12
committed
9b72c0d
perf(BACKEND-VULKAN-TQ): add grouped + MoE TQ microbenchmark
phantomic12
committed
ad1ef7a
perf(BACKEND-VULKAN-TQ): revert weight caches and subgroup amax
phantomic12
committed
40256ed
perf(BACKEND-VULKAN-TQ): subgroup amax + remove redundant col_reduce barrier
phantomic12
committed
7b261d0
test(BACKEND-VULKAN-TQ1_0): loader-to-forward reachability for maple TQ on Vulkan

yoav
authored and
phantomic12
committed
8368b1a
fix(BACKEND-VULKAN-TQ): CPU TQ vec_dot against Q8_K so the keep-quant oracle matches the Vulkan shader
phantomic12
committed
ff6905c

Navigation Menu

feat(BACKEND-VULKAN-TQ1_0): TQ1_0 ternary keep-quant matmul, MoE, and rope shaders for Vulkan - #2248

feat(BACKEND-VULKAN-TQ1_0): TQ1_0 ternary keep-quant matmul, MoE, and rope shaders for Vulkan#2248
phantomic12 wants to merge 25 commits into
mudler:mainfrom
phantomic12:row/BACKEND-VULKAN-TQ1_0

Commits on Sep 10, 2026

spec(BACKEND-VULKAN-TQ1_0): TQ1_0 ternary keep-quant matmul for maple MoE

feat(BACKEND-VULKAN-TQ1_0): TQ1_0 ternary keep-quant matmul, MoE, and rope shaders for Vulkan

perf(BACKEND-VULKAN-TQ1_0): optimize 6 keep-quant shaders — 5.1→9.4 tok/s (1.85x)

perf(BACKEND-VULKAN-TQ1_0): multi-column workgroups + packed reads — 9.4→10.4 tok/s

perf(BACKEND-VULKAN-TQ1_0): fix barrier placement + multi-column dev shaders — 10.4→11.9 tok/s

fix(BACKEND-VULKAN): enable GEMV for prefill (M>1) and flush heavy dispatches

fix(BACKEND-VULKAN-TQ1_0): chunk large dispatches to prevent hangcheck crashes at 8k context

feat(BACKEND-VULKAN-TQ1_0): add TQ1_0/TQ2_0 dtype, CPU dequant, and quant traits

perf(BACKEND-VULKAN-TQ1_0): cache weight blocks in shared memory and add on-device MoE fast path

fix(BACKEND-VULKAN-TQ1_0): use mainline ggml type ids 34/35 for TQ1_0/TQ2_0

fix(BACKEND-VULKAN-TQ1_0): maple model correctness and gather_k for grouped kernels

fix(BACKEND-VULKAN-TQ1_0): review fixes — shaderFloat64 gate, barrier safety, fence errors, N param fix

perf(BACKEND-VULKAN-TQ1_0): byte-at-a-time trit extraction for non-dev shaders

test(BACKEND-VULKAN): update SPIR-V module count for vt_matmul_tiled

bench(BACKEND-VULKAN-TQ): add TQ1_0/TQ2_0 keep-quant microbenchmark

perf(BACKEND-VULKAN-IDOT): dotPacked4x8EXT for TQ1_0/TQ2_0 shaders

fix(BACKEND-VULKAN-TQ): gate IDOT dispatch on nb even to prevent OOB reads

fix(BACKEND-VULKAN-TQ1_0): rebase fixes — GgufLoadPolicy device arg, brace, test sweep

perf(BACKEND-VULKAN-TQ): cache MoE weight blocks in shared memory

perf(BACKEND-VULKAN-TQ): cache weights + subgroup amax in grouped dev shaders

perf(BACKEND-VULKAN-TQ): add grouped + MoE TQ microbenchmark

perf(BACKEND-VULKAN-TQ): revert weight caches and subgroup amax

perf(BACKEND-VULKAN-TQ): subgroup amax + remove redundant col_reduce barrier

test(BACKEND-VULKAN-TQ1_0): loader-to-forward reachability for maple TQ on Vulkan

fix(BACKEND-VULKAN-TQ): CPU TQ vec_dot against Q8_K so the keep-quant oracle matches the Vulkan shader