Eval bug: draft-mtp + --parallel > 1 causes cross-slot content contamination (not HIP-graphs related)

[jsaigou]

Summary

With --spec-type draft-mtp and --parallel N (N > 1), concurrent requests carrying clearly
distinct prompts can produce responses that drift into content belonging to a different
concurrent request's prompt/completion. Output stays valid UTF-8 with no garbage-token
signature — it reads as plausible, on-topic-adjacent text, not obviously broken output, which
makes it easy to miss in casual testing.

Reproduces identically with GGML_CUDA_DISABLE_GRAPHS=1 set, so it's unrelated to HIP graph
capture. Likely adjacent to #28019 (qwen4exp multi-seq rs-rollback corruption) — both are real
gaps in how MTP's still-draft code (originally from #27836) handles per-sequence state under
concurrent slots — but this reproduces the content-mixing symptom specifically, distinct from
that issue's logits-corruption report, and isn't architecture-specific in the way #28019 is
(not yet tested on a non-recurrent target model, see below).

Environment

• OS: Linux (Fedora-based), AMD Strix Halo (Ryzen AI Max+ 395, gfx1151), ROCm-TheRock 10.1
• Build: a downstream fork (jsaigou/llama.cpp, branch
  kintsugi) rebased onto upstream master at 458681e1d (2026-09-01), carrying only an
  unrelated cross-turn KV-cache patch plus the qwen4exp MTP integration from qwen4exp : add NextN/MTP draft head (--spec-type draft-mtp) for Qwen3.8-Flash-Next #27836 and its
  follow-ups (qwen4exp: follow up fixes #27941, qwen4exp: reduce the generation slowdown as context grows #27977, models: fix GDN normalization from max to rsqrt #28068) — no changes to the speculative-decode accept/verify loop
  itself relevant to this report
• Model: Qwen3.8-Flash-Next (qwen4exp architecture), UD-Q3_K_XL, with the
  drluoto/Qwen3.8-Flash-Next-MTP-GGUF
  Q8_0 draft sidecar

Steps to reproduce

Launch with:

llama-server --model Qwen3.8-Flash-Next-UD-Q3_K_XL.gguf \
  --model-draft mtp-Qwen3.8-Flash-Next-Q8_0.gguf \
  --spec-type draft-mtp --spec-draft-n-max 4 \
  --parallel 4 --flash-attn on \
  --cache-type-k q8_0 --cache-type-v q8_0

Fire 4 concurrent /v1/completions requests (temperature 0) with clearly distinct prompts,
e.g.:

• "def quicksort(arr):\n if len(arr) <= 1:\n return arr\n"
• "class Matrix:\n def __init__(self, data):\n"
• "The history of the Roman Empire began with"
• "In distributed systems, the CAP theorem states that"

Observed

Responses drift into content that only makes sense in a different slot's request. Examples
from one run:

• The quicksort-completion request's response trailed into: ... Caesar's assassination in 44 BC, and he is considered one of (content belonging to the Roman Empire request)
• The CAP-theorem request's response trailed into: a distributed data)\n def __arr) + right\n return quick_sort(arr) (content belonging to the quicksort request)
• A second run: the Roman-Empire request's response included arr = [38, 27, 43, 3, 8, 64, 21, 17, 52, 10]\n print("Orig — again, code from a different concurrent request

Expected

Each concurrent request's response should only ever reflect that request's own prompt/context.

Notes on isolation

• Reproduces identically with GGML_CUDA_DISABLE_GRAPHS=1 set — ruled out HIP graph capture as
  a contributing factor.
• Ruled out as a client-side test-harness artifact: requests were fired via
  ThreadPoolExecutor.submit(call, i, prompt) (arguments bound at submit time, not a late-bound
  closure over a loop variable), and response bodies were matched to requests via the HTTP
  response object returned by each future, not by array index.
• A control test with maximally-repetitive, low-information prompts (e.g. "write ROCKET forty
  times" vs. "write BANANA forty times", fired concurrently) showed zero contamination
  across several rounds — the effect needs enough content complexity/entropy to surface, which
  may be why it's easy to miss with a quick smoke test.
• Does not reproduce at --parallel 1 (single-stream), even under heavy multi-turn use with the
  same model/build.
• Not yet tested on a non-recurrent/non-hybrid target architecture with MTP — this report is
  scoped to what was actually observed (qwen4exp, a Gated-DeltaNet hybrid). If it turns out to be
  qwen4exp-specific, it may share a root cause with qwen4exp: multi-seq split replay corrupts recurrent state when rs rollback is enabled (why the arch is excluded from llm_arch_supports_rs_rollback?) #28019 (both point at qwen4exp : add NextN/MTP draft head (--spec-type draft-mtp) for Qwen3.8-Flash-Next #27836's still-draft
  code not correctly isolating per-sequence state — recurrent/SSM state in qwen4exp: multi-seq split replay corrupts recurrent state when rs rollback is enabled (why the arch is excluded from llm_arch_supports_rs_rollback?) #28019's case, and
  something in the draft/verify or batch-construction path here).

Suspected area

common_speculative_draft() / the accept-and-verify loop in tools/server/server-context.cpp
(added by #27836) — likely a shared buffer or per-sequence draft state that isn't correctly
keyed/isolated across concurrent slots when --parallel > 1.