server: keep the Qwen thinking checkpoint when tools are offered but unused — 14x on agentic turns (1.5s vs 23.2s)

[azamatgizzatullin]

Hi Andrea! Small server-side fix on the Qwen3.6 fork-main path, found while
driving ds4-server from a real agentic coding client (OpenCode) on a 24 GiB
M5 Pro.

One line in should_remember_thinking_checkpoint() costs every agentic turn a
full re-prefill. The predicate rejects requests that merely carry tool
definitions, while the call site already rejects turns that produced tool
calls. Agentic clients send the whole tool list on every request and answer with
text most turns, so the live checkpoint is never remembered and each turn
re-prefills from token 0.

Measured result (M5 Pro 24 GiB, Metal SSD streaming, Q4_K_S expert-major)

Identical 3-turn conversation, greedy, run twice. The only difference
between the two runs is the presence of the tools field; the model emits no
tool call in either run.

Turn	no tools	with tools, before	with tools, after
1 (cold)	20.8 s	22.2 s	21.2 s
2	1.6 s	23.2 s	1.5 s
3	1.8 s	24.8 s	1.7 s

Realistic agent scale — 14,310-token system prompt (repo listing), 4 tool
definitions, -c 65536, --ssd-streaming-cache-experts 1800:

Turn	prompt tokens	before	after
1 (cold prefill)	14,310	198.7 s	198.7 s
2	14,340	~150 s	1.6 s
3	14,372	~160 s	1.3 s

The cold first prefill is unchanged and expected — this only stops paying it
again on every subsequent turn. Answers stayed correct across the run
(220, team-3, 1h against the generated fixture). Swap stayed at 0.

Correctness (reuse path vs cold path, byte-identical)

Speed alone would not justify touching a cache guard, so the reuse path was
diffed against the cold path on identical inputs. Greedy (temperature=0,
reasoning_effort=none). The cold arm defeats reuse by interleaving an
unrelated request, which advances the live session so
thinking_live.live_tokens != live_pos and forces a full prefill.

Turn	reuse	cold	output
1	21.5 s	22.1 s	119 — identical
2	1.5 s	23.8 s	validate input 5 before processing and log result 5. — identical
3	1.6 s	25.0 s	validate input 10 before processing and log result 10. — identical

3/3 byte-identical.

make                                        # clean build, no new warnings
./build/metal-arm64/bin/ds4_test --server   # server: OK
make model-free-test                        # all green

--server is the suite covering request parsing, chat rendering, thinking
controls and KV disk-cache bookkeeping — the exact surface touched here.
test_thinking_checkpoint_remember_gate passes unchanged: its has_tools
assertion is written against DS4_CHAT_FORMAT_DEEPSEEK_V4, which this change
does not alter.

--logprob-vectors, --long-context and --tool-call-quality were not run:
they default to the DeepSeek V4 Flash artifact and vectors, which this
environment does not have. Happy to run any of them if you point at the expected
artifact for the Qwen path.

The change

ds4_server.c:10607:

-    if (!r || r->kind != REQ_CHAT || r->has_tools) return false;
+    if (!r || r->kind != REQ_CHAT) return false;
+    if (r->has_tools && format != DS4_CHAT_FORMAT_QWEN36) return false;

The call site (ds4_server.c:11951) already guards the sampled-tool-call case:

} else if (!parsed_calls.len &&
           should_remember_thinking_checkpoint(...)) {
    remember_thinking_checkpoint(...);
}

so r->has_tools inside the predicate only adds the broader tools offered,
tools not used case.

Why this is fail-closed rather than a widened trust boundary.
thinking_live_visible_prefix_prompt() requires, for Qwen specifically,
s->thinking_live.visible_tokens.len > 0 && ds4_tokens_starts_with(&req->prompt, &s->thinking_live.visible_tokens) on top of the byte-prefix match and
live_tokens == live_pos. If a tools-present prompt renders differently on the
next turn, that token-prefix check fails and the request falls back to ordinary
token/text/disk matching. Nothing reuses mismatched state silently — which is
also what the correctness table above measures.

Why Qwen is the only affected protocol. /v1/responses and /v1/messages
both reject Qwen3.6 at request validation ("Qwen3.6 currently supports only /v1/chat/completions"), so the Responses visible-transcript route and the
Anthropic tool-call-id route are unavailable here and there is no alternative
continuation path. DeepSeek deployments have both, which is likely why this has
not surfaced before.

Upstreamability classification

Model-specific research on a fork-main path. Per FORK_NOTES.md the
Qwen3.6-35B-A3B path is FORK MAIN and currently marked not mergeable upstream,
so the patch keeps the explicit DS4_CHAT_FORMAT_QWEN36 guard and stays
fork-scoped. No FORK_NOTES.md boundary change is needed.

The observation is not Qwen-specific though: on /v1/chat/completions the
predicate rejects "tools offered" while the call site already rejects "tools
used", for every chat format. If you consider the broader relaxation sound, the
DeepSeek arm needs its own correctness evidence — particularly the interaction
with canonicalize_tool_checkpoint() and exact DSML replay — which this
environment cannot produce. Flagging it rather than proposing it blind.

Open question

Is has_tools in this predicate intentional beyond the sampled-tool-call case
the call site already covers — e.g. guarding against the tool list changing
between turns? If so, a narrower condition (keying the checkpoint on the
rendered tool block, or comparing it) would preserve that intent while keeping
reuse alive for stable tool lists. Happy to redo it that way instead.

Environment

Base	6d5572d ("docs: refresh Qwen 16 GB production benchmark")
Build	metal-arm64, clean make, no other deltas
Hardware	Apple M5 Pro, 24 GiB unified memory, macOS 25.5.0
Backend	Metal, SSD streaming (DS4_QWEN_EXPERIMENTAL_METAL=1)
Model	Qwen3.6-35B-A3B-DS4-ExpertMajor-v1-Q4_K_S (native expert-major GGUF)
Planner	321 experts prefill / 1400–1800 decode, swap 0 throughout