server : do not re-verify replayed draft tokens after a checkpoint restore

[Nathanw1014]

Branch: server/no-reverify-replayed-draft (one commit, f8668dd35, cherry-picked with -x from
Nathanw1014/llama.cpp 9c5d899ff, 2026-08-22). Diff: +16/-3 in tools/server/server-context.cpp.

What breaks

With full-checkpoint rollback, a partial draft acceptance restores the pre-round state and re-decodes
the accepted tokens to rebuild it. That replay goes through the same verification as a fresh draft
(common_sampler_sample_and_accept_n). The synthetic-acceptance path already handles this
(server_sample_and_accept_synth takes spec_is_replay and accepts unconditionally); the real path
does not.

On backends where logits depend on batch shape (Vulkan), the replay batch has a different shape from
the verify batch that accepted those tokens, so a near-tie sample can flip. The re-verification then
rejects a token that was already accepted, restores the same checkpoint, replays again, and the slot
loops on one position without emitting anything. The process stays up, the log shows nothing at the
default verbosity, the GPU sits near 90% busy. On CPU and HIP the logits are batch-shape invariant, so
the replay always agrees and the loop never starts.

This is the "generation just stops on Vulkan, ROCm is fine" report.

How this fork reached it

• f169e0b (2026-09-01) removed DRAFT_MTP from need_n_rs_seq(). MTP targets then have
  n_rs_seq = 0, so every partial acceptance restores a checkpoint and replays.
• speculative : give MTP targets their recurrent rollback slots back #26 (c69f0e8, 2026-09-06) put MTP back on recurrent rollback slots. That closes the trigger for
  MTP on architectures in llm_arch_supports_rs_rollback() because n_rollback <= n_draft <= n_rs_seq,
  but the replay path itself is still wrong for anything that reaches it: any SEQ_RM_TYPE_FULL
  target, and RS targets whenever a rollback exceeds n_rs_seq.

Builds between those two commits stall with --spec-type draft-mtp on Vulkan. The same sequence
happened on the Strix Halo fork on 2026-08-21: the full-checkpoint change (Gaetan's f82e59263) landed,
v0.6.8 hung a few hundred tokens into long MTP generations, and the fix here is what closed it there
(the change was then re-applied on top of it and has shipped since v0.6.9).

Fix

When spec_is_replay is set, accept the replayed tokens as-is (they were accepted by the verification
that triggered the restore; the replay exists to rebuild state), sample only the continuation from
the final position, and fall through to the existing accounting, which already subtracts the replayed
target token from n_accepted.

On backends with batch-shape invariant logits the re-verification always agreed, so behaviour is
unchanged there: on the original fork this was verified bit-identical on CPU with and without the
change, 800-token greedy pair, 118 restore rounds.

Evidence on this fork

Radeon 8060S (gfx1151), Vulkan/RADV Mesa 26.3.0-devel, LLAMA_TRACE=1, streamed chat completions,
stall detector = fewer than 20 tokens in any 120 s window. Restore counts are restore checkpoint
lines in the server log. Raw logs: ~/strix-results/halobox-mtp-stall-20260908/ on the test box.

MoE target, the reporters' case. Ornith-1.5-35B-Q4_K_M (qwen35moe, embedded MTP head),
--spec-type draft-mtp --spec-draft-n-max 3 -c 32768 -ub 1024 -fa on, thinking off, 3 prompts x 3000 tokens
per sampler setting.

build	greedy	temp 0.7
c7af5c6 (pre-#26: MTP on full checkpoints)	livelock after 286 tokens: 102 normal restores, then accepted 0/1 draft tokens (restore checkpoint) 341,000 times in 107 min at 52 rounds/s, GPU 89-90% busy, output frozen	not reached (arm killed)
c7af5c6 + this fix	3/3 complete, 8153 tokens at 43-48 t/s through 2881 restores	3/3 complete, 3218 further restores, no stall

The loop is the mechanism in miniature: the replay re-decodes one target-sampled token as a batch of
one, re-verifies it against what a batch of two produced, rejects it, restores, repeats.

Dense target. Qwen3.8-27B UD-Q4_K_XL (qwen35, embedded MTP head), the issue #25 setup:
--spec-draft-adaptive --spec-draft-n-max 8 --spec-draft-n-min 3 --spec-draft-p-min 0.75 -c 32768 -ctk q8_0 -ctv q8_0 --temp 0.7 --top-p 0.8 --top-k 20 --presence-penalty 1.5, 6 prompts x 2500 tokens.

build	restores	result	decode t/s (6 prompts)
c7af5c6	658	no stall	10.4-11.8
c7af5c6 + fix	646	no stall	10.9-12.4
master 7449a0f	0	no stall	11.6-13.9
master + fix (this branch)	0	no stall, per-prompt speeds identical to master to 0.01 t/s	11.6-13.9

The dense target takes the replay path just as often but its replay re-verification never disagreed
in 15k tokens; the MoE target disagreed within 300. On master the recurrent rollback slots from #26
mean the replay path is never entered for MTP, so the fix is a no-op there, as the identical numbers show.

CPU check (--device none, greedy, 400 tokens, 27B). Run-to-run deterministic (pre-#26 twice: same
hash, same 102 restores). The fix changes the CPU output at the first restore: the guard keeps the token
the original verification sampled, the unguarded path re-samples it in the smaller replay batch and gets
a different token. Neither matches a no-speculation greedy run past ~90 tokens (unguarded diverges at
char 415, guarded at char 8), i.e. on this base CPU logits are not batch-shape invariant either, so
the "bit-identical on CPU" result from the original fork does not carry over and is not claimed here.
What the guard does is what upstream's own synthetic-acceptance replay path already does: repeat the
decisions of the verification that triggered the restore, instead of re-deciding them.

Notes for reviewers

• Upstream has the same code (tools/server/server-context.cpp, non-synth branch), so this is also an
  upstream candidate. Filed separately.
• Not a replacement for speculative : give MTP targets their recurrent rollback slots back #26: speculative : give MTP targets their recurrent rollback slots back #26 makes MTP take the cheaper rollback route; this makes the expensive
  route correct when it is taken.
• Reporters: please post llama-server --version output; a stall on a build at or after c69f0e8
  would be a different bug.

AI usage disclosure: yes. Investigation and port by an agent (Claude Fable 5.1) at Nathan Wilson's
direction; the original fix was written the same way on 2026-08-22.

🤖 Generated with Claude Code