Server forces full prompt re-processing on subsequent requests (SWA/recurrent memory error)

[asg-3d]

Name and Version

$ ./llama-server --version
version: 8769 (21a4933)
built with Clang 19.1.5 for Windows x86_64
(Also reproduced on version 8025 (bb96bfd))

Operating systems

Windows

GGML backends

CUDA

Hardware

Intel Core i5-12400F + NVIDIA GeForce RTX 5060 Ti (16GB VRAM)

Models

• Qwen3.5-35B-A3B (Q4_K_M) from: https://huggingface.co/lmstudio-community/Qwen3.5-35B-A3B-GGUF
• Gemma-4-26B-A4B (IQ4_XS) from: https://huggingface.co/unsloth/gemma-4-26B-A4B-it-GGUF

Problem description & steps to reproduce

The llama-server fails to reuse the KV cache for subsequent requests in the same chat session. On the second request to the same slot, it forces a full prompt re-processing, logging: forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory). As a result, the model "forgets" the conversation history.

This occurs on different models (Qwen 3.5 MoE, Gemma 4 MoE), different llama.cpp versions (b8025, b8769), and even with minimal context (e.g., 45 tokens). The issue is reproducible with direct curl requests, ruling out client-side problems.

Steps to reproduce:

1. Start llama-server with Qwen3.5-35B-A3B or Gemma-4-26B-A4B.
2. Send first request:
   curl -X POST http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\": \"qwen3.5-35b-a3b-128k\", \"messages\": [{\"role\": \"user\", \"content\": \"My name is Tester. Remember it.\"}], \"temperature\": 0.6, \"max_tokens\": 50}"
3. Send second request:
   curl -X POST http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\": \"qwen3.5-35b-a3b-128k\", \"messages\": [{\"role\": \"user\", \"content\": \"What is my name?\"}], \"temperature\": 0.6, \"max_tokens\": 50}"
4. Observe server logs and model response.

Expected behavior:
Server restores context from KV cache; model answers "Tester".

Actual behavior:
Server logs forcing full prompt re-processing...; model responds that it does not know the name.

First Bad Commit

Unknown. The issue is present in both b8025 (month-old) and b8769 (latest). It likely predates both versions.

Relevant log output

# Command used to start the server (example for Qwen 3.5):
.\llama-server.exe --models-preset models.ini --host 127.0.0.1 --port 8080 --temp 1.0 --top-p 1.0 --min-p 0.01 --top-k 0 --reasoning-budget 0 --no-mmap --metrics

# First request (establish context):
curl -X POST http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\": \"qwen3.5-35b-a3b-128k\", \"messages\": [{\"role\": \"user\", \"content\": \"My name is Tester. Remember it.\"}], \"temperature\": 0.6, \"max_tokens\": 50}"

# Second request (check context):
curl -X POST http://127.0.0.1:8080/v1/chat/completions -H "Content-Type: application/json" -d "{\"model\": \"qwen3.5-35b-a3b-128k\", \"messages\": [{\"role\": \"user\", \"content\": \"What is my name?\"}], \"temperature\": 0.6, \"max_tokens\": 50}"

Server log output (second request, v8769)

[50224] slot update_slots: id  3 | task 24 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
[50224] slot update_slots: id  3 | task 24 | n_tokens = 0, memory_seq_rm [0, end)
[50224] slot update_slots: id  3 | task 24 | prompt processing progress, n_tokens = 12, batch.n_tokens = 12, progress = 0.750000
[50224] slot update_slots: id  3 | task 24 | prompt done, n_tokens = 18, batch.n_tokens = 18
[50224] slot print_timing: id  3 | task 24 |
[50224] prompt eval time =     317.59 ms /    18 tokens (   17.64 ms per token,    56.68 tokens per second)
[50224]        eval time =    2130.55 ms /    50 tokens (   42.61 ms per token,    23.47 tokens per second)
[50224]       total time =    2448.14 ms /    68 tokens
[50224] slot      release: id  3 | task 24 | stop processing: n_tokens = 67, truncated = 0

Model response (second request)

{"choices":[{"finish_reason":"length","index":0,"message":{"role":"assistant","content":"I don't know your name because I don't have access to your personal information unless you tell me.\n\nIf you'd like, you can tell me your name and I'll remember it for this conversation"}],"created":1776037749,"model":"qwen3.5-35b-a3b-128k","system_fingerprint":"b8769-21a493304","object":"chat.completion","usage":{"completion_tokens":50,"prompt_tokens":18,"total_tokens":68}}...

[asg-3d]

Final Update with Root Cause Confirmation:

After extensive testing and debugging, the root cause is confirmed and isolated.

Executive Summary:
The forcing full prompt re-processing error is exclusive to hybrid architectures (e.g., qwen35moe, gemma4moe) that utilize recurrent layers (SSM/Gated DeltaNet). Pure Transformer models (e.g., qwen2) are unaffected and perform KV-caching correctly. The intended mitigation (--checkpoint-every-nb) is currently non-functional in both release binaries and self-compiled builds, leaving hybrid models unusable for multi-turn conversations.

---

1. Root Cause Isolation: Hybrid vs. Pure Transformer

I conducted a controlled A/B test using the exact same llama-server build (b8791), models.ini configuration, and test prompts.

Test: A simple two-turn conversation.

• Turn 1: "My password is ABRACADABRA. Remember it."
• Turn 2: "What is my password?"

Results:

Model	Architecture	Server Log (Turn 2)	Result
Qwen3.5-35B-A3B	Hybrid (qwen35moe)	forcing full prompt re-processing... & memory_seq_rm [0, end)	Failed. Model forgot context.
Qwen2.5-Coder-14B	Transformer (qwen2)	n_tokens = 219, memory_seq_rm [219, end) & NO forced re-processing.	Passed. Model remembered "ABRACADABRA".

Conclusion: The bug is a direct consequence of how llama.cpp currently handles the recurrent state in hybrid models. Standard Transformer KV-caching works flawlessly.

---

2. The Mitigation (--checkpoint-every-nb) is Non-Functional

The intended mitigation for this exact issue was merged in PR #20087 via the --checkpoint-every-nb flag.

Problem: The flag is unreachable.

• Tested on: Official release binary b8218.
• Command: llama-server.exe ... --checkpoint-every-nb 3
• Result: error: invalid argument: --checkpoint-every-nb
• Confirmation: Other users in PR Hybrid model cache: add --checkpoint-every-nb #20087 have reported the same error even when compiling from the latest master branch (Hybrid model cache: add --checkpoint-every-nb #20087 (comment)).

This confirms that the failure is not a local setup issue but a deeper integration or implementation bug with the flag itself.

---

3. Recommendations

This issue is a critical blocker for using powerful hybrid models like Qwen3.5 in any multi-turn application (e.g., AiderDesk, OpenCode, Claude Code).

For Maintainers:

1. Priority: Investigate why the --checkpoint-every-nb flag is not recognized, as it is the primary user-facing mitigation.
2. Documentation: Until a fix is in place, a prominent warning in the README about known caching limitations with qwen35moe/gemma4moe architectures would save users significant debugging time.

For Other Users Finding This Issue:

• Workaround: Use a pure Transformer model for tasks requiring context retention (e.g., Qwen2.5-Coder-14B, DeepSeek-Coder-V2-Lite).
• Verification: If you believe you have a working hybrid setup, check your server logs for the line forcing full prompt re-processing. If present, your context is not being cached correctly.

[asg-3d]

Comprehensive Testing Results & Root Cause Analysis

After extensive testing of 8 different models across various architectures, I have identified the precise scope and nature of this bug.

Executive Summary

The forcing full prompt re-processing error and associated context loss is NOT a universal problem with hybrid/SSM models. It is highly specific to certain model architectures, primarily affecting qwen35moe (Qwen3.5-35B-A3B) and manifesting differently even within the same architecture family depending on model size.

Test Methodology

Controlled A/B test for all models:

• Turn 1: "My password is ABRACADABRA. Remember it."
• Turn 2: "What is my password?"
• Environment: llama-server build b8791, Windows 11, RTX 5060 Ti (16GB VRAM), identical models.ini configuration.

Complete Test Results

Model	Architecture	Cache Error in Logs	Context Retained	Notes
qwen3.5-35b-a3b	qwen35moe (MoE + SSM)	✅ YES	❌ NO	BUG CONFIRMED — complete context loss
qwen3.5-9b	qwen35 (Dense + SSM)	✅ YES	✅ YES	SAME ERROR, BUT WORKS!
qwen3-coder-next-80b	qwen3next (MoE + SSM)	❌ NO	✅ YES	Technically stable
gemma-4-26b-a4b-it	gemma4 (MoE + SWA)	✅ YES	✅ YES	SAME ERROR, BUT WORKS!
qwen3-coder-30b	qwen3moe (MoE, no SSM)	❌ NO	✅ YES	Perfectly stable
qwen3-32b	qwen3 (Dense, no SSM)	❌ NO	✅ YES	Perfectly stable (needs >16GB VRAM)
qwen2.5-coder-14b	qwen2 (Dense)	❌ NO	✅ YES	Perfectly stable
gpt-oss-20b	gpt-oss (MoE + SWA)	❌ NO	✅ YES (EN only)	Stable

Key Findings

1. The error message is not fatal in all cases.

   • qwen3.5-9b and gemma-4-26b-a4b-it both log the exact same forcing full prompt re-processing error and perform a full cache reset (memory_seq_rm [0, end)), yet both models successfully retain context.
   • qwen3.5-35b-a3b is the only model tested that both logs the error and loses context.

2. The bug is not inherent to SSM/GDN layers.

   • qwen3-coder-next-80b (which has SSM layers: ssm_d_conv = 4, ssm_d_state = 128) works flawlessly with no cache errors.
   • gemma-4-26b-a4b-it (which has SWA layers) works despite logging the error.

3. The --checkpoint-every-nb mitigation is non-functional.

   • Tested on official release b8218 and self-compiled builds.
   • Flag is unrecognized: error: invalid argument: --checkpoint-every-nb.

4. The problem appears specific to qwen35moe architecture implementation.

   • Both qwen3.5-9b (qwen35) and qwen3.5-35b-a3b (qwen35moe) log the error.
   • Only the MoE version (35B) fails to retain context.
   • The dense version (9B) works correctly despite the error.

Log Evidence

qwen3.5-35b-a3b (FAILS): Full cache reset, context lost

[50224] slot update_slots: forcing full prompt re-processing due to lack of cache data...
[50224] slot update_slots: n_tokens = 0, memory_seq_rm [0, end)

qwen3.5-9b (WORKS despite error): Same error, but context retained

[64179] slot update_slots: forcing full prompt re-processing due to lack of cache data...
[64179] slot update_slots: n_tokens = 0, memory_seq_rm [0, end)
[64179] slot update_slots: created context checkpoint 1 of 32 (pos_min = 321, pos_max = 321, n_tokens = 322)

Model correctly answered password question.

qwen2.5-coder-14b (WORKS perfectly): No error, proper cache reuse

[53209] slot update_slots: n_tokens = 219, memory_seq_rm [219, end)
[53209] slot update_slots: prompt processing done, n_tokens = 258, batch.n_tokens = 39

Recommendations for Maintainers

1. Investigate the difference between qwen35 and qwen35moe cache handling. The dense 9B version works; the MoE 35B version fails. The key difference may be in how expert weights or M-RoPE dimensions are checkpointed.

2. Fix the --checkpoint-every-nb flag recognition. This is the primary user-facing mitigation and it's currently unreachable.

3. Consider documenting known-good model architectures. Pure qwen2/qwen3 (dense) and qwen3moe (MoE without SSM) work flawlessly. Users can be directed to these as stable alternatives.

Workaround for Users

Stable models for context-heavy applications (AiderDesk, OpenCode, Claude Code):

• ✅ qwen3-coder-30b — Best balance of size, code quality, and stability for 16GB VRAM
• ✅ qwen2.5-coder-14b — Proven stable, excellent code performance
• ✅ qwen3-coder-next-80b — Technically stable, but large
• ✅ gemma-4-26b-a4b-it — Works despite log warnings

Verification: Check server logs for forcing full prompt re-processing. If present, test whether context is actually retained before assuming failure.

[ITankForCAD]

Hey, I dug into common/arg.cpp to find the cli flag. It has been renamed from --checkpoint-every-nb to -cpent or --checkpoint-every-n-tokens. Here is the flag definition from arg :

string_format("create a checkpoint every n tokens during prefill (processing), -1 to disable (default: %d)"

[asg-3d]

@ITankForCAD Thanks!

Workaround found for qwen35moe models:

llama-server flags:
--checkpoint-every-n-tokens 1024
--ctx-checkpoints 256

AiderDesk: disable Task Tools in agent profile.

Result: no forcing full prompt re-processing, context stable up to 262k tokens.
Tested on Qwen3.5-35B-A3B and Qwen3.6-35B-A3B with RTX 5060 Ti 16GB.

[ITankForCAD]

Result: no forcing full prompt re-processing, context stable up to 262k tokens. Tested on Qwen3.5-35B-A3B and Qwen3.6-35B-A3B with RTX 5060 Ti 16GB.

Can I ask what kind of performance you get out of that 5060ti on this model ? I'm thinking of upgrading my pc.

[asg-3d]

Can I ask what kind of performance you get out of that 5060ti on this model ? I'm thinking of upgrading my pc.

Qwen3.5 is 10-15% faster than Qwen3.6. Here are the results for Qwen3.6.

Context Size (tokens)	Scenario	Prompt Eval (t/s)	Generation (t/s)	File Processing Time	Note
~20k	Starting a dialogue, simple requests	~1400-1500	~20-21	< 1 sec	Easy mode, all in VRAM
~70k	Analysis of one large file	~1300-1350	~18-19	~50-60 sec	Comfortable work
~140k	Medium Project Analysis	~1250-1300	~16-18	~100-110 sec	Noticeable Slowdown
~220k	Large Project Analysis	~840-850	~14-15	~4.5 min	Active CPU Activity (Experts)
~250k	Extreme Mode	~800	~13-14	~5.5 min	Maximum Load
~262k	Physical Limit	~800	~13-14	—	Context Exceeded Error

Launch parameters:

.\llama-server.exe --models-preset models.ini --host 127.0.0.1 --port 8080 --reasoning-budget 0 --no-mmap --metrics --reasoning off

[MoE.Qwen3.6-35B-A3B.256k]
model = unsloth/Qwen3.6-35B-A3B-GGUF/Qwen3.6-35B-A3B-UD-IQ4_XS.gguf
n-gpu-layers = 99
ctx-size = 262144
parallel = 1
threads = 6
ubatch-size = 4096
batch-size = 4096
cache-type-k = q8_0
cache-type-v = q8_0
flash-attn = on
cont-batching = true
no-warmup = true
override-tensor = blk\.\d+\.(ffn_down_exps|ffn_gate_exps|ffn_up_exps)\.weight=CPU
checkpoint-every-n-tokens = 1024
ctx-checkpoints = 256
jinja = on
temperature = 0.7
top-p = 0.8
top-k = 20
min-p = 0.0
presence-penalty = 1.5
repeat-penalty = 1.0

[d1ffuser]

Very much appreciated — those flags improved performance immensly. And no errors so far.
(Qwen3.6-35B-A3B-UD-Q4_K_M, Apple)

[nilo85]

I experienced this issue with Qwen 3.6 27B (dense)

Took some help from Gemini to format my findings

Issue: Frequent Cache Invalidation / Forced Full Reprocessing on Hybrid/Recurrent Models (Qwen 3.6)
Description:
When using Qwen 3.6 (hybrid architecture with recurrent memory), the server frequently invalidates existing context checkpoints and forces a full prompt re-processing from token 0, even when the prefix matches. This appears to be related to the SWA/Recurrent memory safety checks in server-context.cpp.

Server Logs:

# Checking existing checkpoints against current task
[51305] slot update_slots: id  0 | task 8065 | Checking checkpoint with [86562, 86562] against 84986...
[51305] slot update_slots: id  0 | task 8065 | Checking checkpoint with [86058, 86058] against 84986...
[51305] slot update_slots: id  0 | task 8065 | Checking checkpoint with [85784, 85784] against 84986...
[51305] slot update_slots: id  0 | task 8065 | Checking checkpoint with [85619, 85619] against 84986...

# Cache Invalidation Triggered
[51305] slot update_slots: id  0 | task 8065 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)

# Purging existing state
[51305] slot update_slots: id  0 | task 8065 | erased invalidated context checkpoint (pos_min = 85619, pos_max = 85619, n_tokens = 85620, n_swa = 0, pos_next = 0, size = 149.626 MiB)
[51305] slot update_slots: id  0 | task 8065 | erased invalidated context checkpoint (pos_min = 85784, pos_max = 85784, n_tokens = 85785, n_swa = 0, pos_next = 0, size = 149.626 MiB)
[51305] slot update_slots: id  0 | task 8065 | erased invalidated context checkpoint (pos_min = 97568, pos_max = 97568, n_tokens = 97569, n_swa = 0, pos_next = 0, size = 149.626 MiB)

# Restarting from Token 0
[51305] slot update_slots: id  0 | task 8065 | n_tokens = 0, memory_seq_rm [0, end)
[51305] slot update_slots: id  0 | task 8065 | prompt processing progress, n_tokens = 2048, batch.n_tokens = 2048, progress = 0.020968
[51305] slot update_slots: id  0 | task 8065 | prompt processing progress, n_tokens = 4096, batch.n_tokens = 2048, progress = 0.041937
[51305] slot update_slots: id  0 | task 8065 | prompt processing progress, n_tokens = 8192, batch.n_tokens = 2048, progress = 0.083873
Environment:

Model: Qwen 3.6 27B (Hybrid/Recurrent)

Backend: llama.cpp (Intel Arc B70)

Context Size: 131,072

Notes: The server appears to create checkpoints correctly, but fails the pos_min validation check during subsequent requests, likely because recurrent memory layers do not report positional data in the way the current server logic expects.

[NikoBele1]

I’m experiencing similar issues with Qwen 3.6 27B using the latest llama.cpp server build. I send a request to the endpoint with image frames and ask for a description. On the first run, all images are encoded and a response is generated. On the second run, if I append just a single word to the text (which comes after my images), all images are re-encoded again. The logs state a forced full reprocessing.

I also tested Qwen 3.5 27B, since I had working prompt caching with that model before, but it showed the same behavior. I checked an older build (b8185), and in that version, prompt caching behaves correctly when modifying the text after the video frames.

So it seems newer builds changed something.

EDIT:
It is do_checkpoint = do_checkpoint && !has_mtmd; in tools/server/server-context.cpp. commenting it out brings back the previous behavior

[cvgro]

I had the same problem. The context was lost on "regenerate" in llamma.cpp server's ui. In my last tests, a few days ago, on b8905, the problem disappeared switching the browser from Firefox to Chrome. Could be a coincidence, but that session was without errors.

[walker0643]

Same here for Qwen 3.6 27B. Should the flags for qwen35moe, --checkpoint-every-n-tokens 1024 and --ctx-checkpoints 256, work for 27B dense?

[nilo85]

Using the --swa-full flag seems to have resolved the issue for me.

What model are you using?

EDIT: re-evaluating Qwen3.6 27B with it on again (was one variable before but not sure what I was troubleshooting when it was on)

EDIT2: did not fix the issue for me...

apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | Checking checkpoint with [12757, 12757] against 32...
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | Checking checkpoint with [12245, 12245] against 32...
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | Checking checkpoint with [8191, 8191] against 32...
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see https://github.com/ggml-org/llama.cpp/pull/13194#issuecomment-2868343055)
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | erased invalidated context checkpoint (pos_min = 8191, pos_max = 8191, n_tokens = 8192, n_swa = 0, pos_next = 0, size = 149.626 MiB)
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | erased invalidated context checkpoint (pos_min = 12245, pos_max = 12245, n_tokens = 12246, n_swa = 0, pos_next = 0, size = 149.626 MiB)
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | erased invalidated context checkpoint (pos_min = 12757, pos_max = 12757, n_tokens = 12758, n_swa = 0, pos_next = 0, size = 149.626 MiB)
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | erased invalidated context checkpoint (pos_min = 20949, pos_max = 20949, n_tokens = 20950, n_swa = 0, pos_next = 0, size = 149.626 MiB)
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | erased invalidated context checkpoint (pos_min = 29141, pos_max = 29141, n_tokens = 29142, n_swa = 0, pos_next = 0, size = 149.626 MiB)
apr 29 23:30:48 niklas-pc llama-cpp-multimodel[38440]: [45025] slot update_slots: id  0 | task 6644 | erased invalidated 

EDIT3:
Model does not support it it says =)
apr 29 23:15:08 niklas-pc llama-cpp-multimodel[38440]: [45025] srv load_model: swa_full is not supported by this model, it will be disabled

Model:
unsloth/Qwen3.6-27B-GGUF:UD-Q4_K_XL

[walker0643]

I should have said earlier, but same model here, unsloth/Qwen3.6-27B-GGUF:UD-Q4_K_XL