From e2ffc11d470bd4d26965936c695eed556785542e Mon Sep 17 00:00:00 2001 From: Petr Vasilev Date: Mon, 31 Aug 2026 23:26:17 +0300 Subject: [PATCH] llama : enable recurrent-state rollback for qwen3next / qwen4exp qwen3next and qwen4exp are GDN-based hybrids with the same recurrent-state design as qwen35 / qwen35moe, which already support rollback. Adding them to llm_arch_supports_rs_rollback lets speculative decoding roll back the recurrent state natively (n_rs_seq widens the state to 1 + n_rs_seq groups) instead of falling back to SEQ_RM_TYPE_FULL, which serializes the whole recurrent state to host on every speculative round. Measured on AMD Strix Halo gfx1151 / Vulkan+RADV, Qwen3.8-Flash-Next Q4_K_M, -ctk q8_0 -ctv q8_0, temp 0, -np 1, MTP draft: no draft: 32.4 t/s MTP, SEQ_RM_TYPE_FULL: 6.2 t/s (host checkpoint every round, 5x loss) MTP, this change (RS): 43.6 t/s (+35% over no-draft; acceptance 0.89) Output stays greedy-equivalent to no-draft. Validated with -np 1 and short/medium generations; the widened-state rollback across deep rejections, long context and multi-slot could use a maintainer eye on the qwen4exp graph. --- src/llama-arch.cpp | 2 ++ 1 file changed, 2 insertions(+) diff --git a/src/llama-arch.cpp b/src/llama-arch.cpp index 5e61f61f7f0d..f2a156a97590 100644 --- a/src/llama-arch.cpp +++ b/src/llama-arch.cpp @@ -1098,6 +1098,8 @@ bool llm_arch_is_diffusion(const llm_arch & arch) { bool llm_arch_supports_rs_rollback(const llm_arch & arch) { switch (arch) { + case LLM_ARCH_QWEN3NEXT: + case LLM_ARCH_QWEN4EXP: case LLM_ARCH_QWEN35: case LLM_ARCH_QWEN35MOE: case LLM_ARCH_DEEPSEEK4: