From ef77b3ecf46083a468980d2b4cd57dd295046600 Mon Sep 17 00:00:00 2001 From: Azamat Date: Sat, 25 Jul 2026 08:45:16 +0300 Subject: [PATCH] server: keep the Qwen thinking checkpoint when tools are offered but unused should_remember_thinking_checkpoint() rejected any request carrying tool definitions, while the call site already rejects turns that produced tool calls (!parsed_calls.len). The predicate therefore also dropped the far more common "tools offered, tools not used" case, which is the normal shape of an agentic coding session: clients send the full tool list on every request and answer with text most turns. The live checkpoint was never remembered, so every turn re-prefilled the whole prompt. Scope the guard to non-Qwen formats. For Qwen the reuse path in thinking_live_visible_prefix_prompt() additionally proves an exact token prefix of the remembered visible tokens (ds4_tokens_starts_with) on top of the byte-prefix match and live_tokens == live_pos, so a tools-present prompt that renders differently fails closed into ordinary token/text/disk matching. Qwen is also the only affected protocol: /v1/responses and /v1/messages reject Qwen3.6 at request validation, so neither the Responses visible-transcript route nor the Anthropic tool-call-id route is available as an alternative. Measured on M5 Pro 24 GiB, Metal SSD streaming, Q4_K_S expert-major, identical 3-turn conversation differing only by the tools field: turns 2/3 go from 23.2s/24.8s to 1.5s/1.7s. At agent scale (14,310-token prompt, 4 tools) turns 2/3 go from ~150s to 1.6s/1.3s. Reuse-path output is byte-identical to the cold path on all three turns under greedy decode. ds4_test --server passes unchanged; test_thinking_checkpoint_remember_gate asserts has_tools against DS4_CHAT_FORMAT_DEEPSEEK_V4, which is untouched. Co-Authored-By: Claude Opus 5 --- ds4_server.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/ds4_server.c b/ds4_server.c index 730182708..d41e092be 100644 --- a/ds4_server.c +++ b/ds4_server.c @@ -10604,7 +10604,16 @@ static bool should_remember_thinking_checkpoint(const request *r, const thinking_state *thinking, const char *finish, ds4_chat_format format) { - if (!r || r->kind != REQ_CHAT || r->has_tools) return false; + if (!r || r->kind != REQ_CHAT) return false; + /* Tools offered but never used must not cost a full re-prefill. The call + * site already excludes turns that sampled tool calls (!parsed_calls.len), + * so this guard only covers "the request carried tool definitions". For + * Qwen the reuse path additionally proves an exact token prefix of the + * remembered visible tokens before continuing, so a tools-present prompt + * that renders differently fails closed into ordinary token/text/disk + * matching. Agentic clients (OpenCode) pass tools on every turn and answer + * with text most turns; without this the whole prompt is re-prefilled. */ + if (r->has_tools && format != DS4_CHAT_FORMAT_QWEN36) return false; if (r->prompt_preserves_reasoning) return false; /* Qwen's no-thinking generation prefix still contains a trusted * ... wrapper that the next canonical history omits. */