Misc. bug: Prompt cache serialization crashes on second request with SSM model on Vulkan

[task24]

Name and Version

• llama.cpp commit: 2b2cd57
• Version: master branch (2026-04-11)

Operating systems

• Linux

Modules affected

• libllama (core library)
• llama-server

Command line

llama-server -hf nvidia/NVIDIA-Nemotron-3-Nano-4B-GGUF

Problem description & steps to reproduce

1. Run llama-server with NVIDIA Nemotron-3 Nano 4B (SSM-based model)
2. Send a chat completion request — works fine
3. Send a second chat completion request
4. Server crashes during prompt cache serialization

Workaround: llama-server -hf nvidia/NVIDIA-Nemotron-3-Nano-4B-GGUF --cache-ram 0 (disabling prompt cache fixes it)

Environment

• GPU: AMD Radeon RX 7900 XTX
• Vulkan: 1.4.341
• CPU: Ryzen 16-core

Relevant log output

The crash occurs in ggml-backend.cpp:348 during llama_kv_cache::state_write_data

Error: GGML_ASSERT(tensor->data != NULL && "tensor not allocated") failed

Backtrace shows the failure during prompt cache save after first response completes successfully.

[jtmckinn]

Reproduced on ROCm/HIP (gfx1100) + Qwen3.6-27B - root cause identified + appears fixed by #22673

Reproducing environment: AMD RX 7900 XTX (gfx1100), ROCm 6.x, Qwen3.6-27B-IQ4_XS (hybrid SSM/attention - 49 Mamba layers + 16 attention layers), llama-server with --cache-ram 128 --spec-type mtp --draft-n-max 3. Crash: SIGABRT on second request, identical stack to yours (state_write ? GGML_ABORT in llama-memory-recurrent.cpp).

---

Root cause - two paths that produce dirty rs_idx at checkpoint time

The crash hit GGML_ABORT("recurrent state read/write is not supported with partial rollback") in state_write() when rs_idx[i] != 0.

Path A - EOS on the same step as an MTP token rejection:
MTP rejection calls set_rs_idx(), leaving rs_idx non-zero. If EOS fires on that same forward pass, no subsequent forward pass runs and rs_idx is never consumed/reset. On the next request, create_checkpoint() calls update_tgt ? state_write ? SIGABRT.

Path B - LCP slot reuse triggering the speculative checkpoint path:
A continuation request hits a high-similarity LCP match. seq_rm sets rs_idx for rollback. Before the consuming forward pass can reset it, the speculative decode path checks draft.size() > n_rs_seq, finds true, and calls ckpt.update_tgt ? state_write ? SIGABRT. This is a separate code path in server-context.cpp from create_checkpoint, also unguarded.

The --cache-ram 0 workaround works because it prevents any checkpoint from being created, so state_write is never reached.

---

Status vs current upstream

After inspecting current master: PR #22673 redesigned state_write() to use rs_idx as a rollback buffer offset (cell_id = rs_idx_cur * size + src) rather than treating any non-zero rs_idx as an error condition. The GGML_ABORT that triggered the crash no longer exists in the refactored code.

Our crash paths should not be reproducible on current master. If someone can confirm against a hybrid SSM model + MTP + --cache-ram > 0, this issue should be closeable.

Note: #21831 tracks the related symptom of forced full re-processing on every turn - that's what surfaces when the server falls back to re-prefill and may warrant a separate look.

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.