dasLLAMA thinking/tool audit: per-arch reasoning matcher, reasoning_content on the server, live per-family proof tests

[borisbat]

Executes THINKING.md's reply-side plan and closes the thinking/tool-use audit: every locally-stocked thinking family now demonstrably thinks — and the server demonstrably recognizes it.

Chat layer

• Per-arch reply matcher: ThinkMode (symmetric / channel_switch / asymmetric) + reasoning markers as ChatTemplate data. The engine is split_reasoning (whole reply, via the shared think_drain) plus a streaming splitter (make_think_stream / think_feed / think_finish) that holds partial markers back across chunk boundaries; a reasoning span truncated by the budget classifies as reasoning; an empty consumed think block still strips (strip_think parity). Harmony headers route by the channel NAME token (recipient-qualified commentary to=... headers stay reasoning).
• Arming is vocab-gated and gate-aware (one think_turn_active predicate for the matcher, the alternate opener, and the stop merge): an inert declaration — Qwen2.5 shares the ChatML template without think tokens — never arms, and a mid-conversation set_thinking(true) on gemma-4 stays instruct-shaped (the gate cannot enter an already-rendered context) with the framing stops still armed. Families with no declared reply format keep the legacy unconditional history strip (R1-distills served under llama must not bloat history).
• gemma-4 E-series thinking-ON: think_default polarity, the <|think|> system gate, a bare thinking-mode assistant opener; the instruct default renders token-for-token what it rendered before (pinned by test). stop_nothink + effective_stop_ids cut instruct-mode generation at a stray channel marker (observed live: the model answering then rambling a closer and a repeat).
• qwen35 gets hermes_tools — the Qwen3.5 template kept the Hermes tool block; the registration had dropped it.

Server (utils/dasllama-server)

• reasoning_content on chat-completion messages and as streaming deltas (the DeepSeek/llama.cpp framing); reasoning splits before tool-call parsing, so a thinking model that calls tools returns reasoning_content AND tool_calls from one reply.
• enable_thinking is tri-state: absent leaves the family default in force; a present bool (either spelling) overrides.

Tests — commands and evidence

Module tests (all -jit, DASLLAMA_CPU_PREFILL=1 on this Metal-capable box):

bin/daslang -jit dastest/dastest.das -- --test modules/dasLLAMA/tests/test_think_split.das   # 24/24
bin/daslang -jit dastest/dastest.das -- --test modules/dasLLAMA/tests/test_chat.das          # 36 passed, 1 skip (qwen2audio large-tier); gemma-4 off/on, mid-conversation toggle, qwen35 markers, audio-marker arms all PASSED
bin/daslang -jit dastest/dastest.das -- --test modules/dasLLAMA/tests/test_facade_docs.das   # 7/7

Live server legs (DASLLAMA_MODELS_DIR set, full tier):

DASLLAMA_PARITY_FULL=1 bin/daslang -jit dastest/dastest.das -- --test utils/dasllama-server/test_openai_server_think.das
# 12/12, zero skips: qwen3 (reasoning split + streaming order + tools-with-thinking compose:
# get_weather(city="Paris"), finish_reason="tool_calls"), qwen35 (live Hermes leg), gemma-4-E2B
# (off/on pair), gpt-oss-20b ("the analysis channel was recognized as reasoning" PASSED)

Server regressions: test_openai_server.das 19/19, test_openai_server_stream.das 8/8. No run.das --arm set applies: the change is chat-layer + server only and reaches no metal/prefill/decode/matrix arm (the runner suites cover engine kernels, which this diff does not touch).

• modules/dasLLAMA/tests/test_think_split.das — model-free: every family's exact wire shape (glm's against its own template), whole-string and per-chunk down to 1 byte, header-token routing, empty-block parity.
• CODEREVIEW.md binds any new family gaining a live thinking or tool format to these tests (trigger on template data including the shared helpers, inert-vocab exemption); GLM-4 (no small model exists) has its remote leg recorded in THINKING.md.
• CI (extended_checks) now compile-guards the think test file alongside the existing server test.

Drive-by fix: test_facade_docs ENGINE_FILES went stale when the common split moved the engine entry points — the 1:1 facade check was failing on master and the miss was silent.

Process: full preflight ran once (green after lint/docs fixes were re-validated with targeted gates); the second commit is the review round from the opus code-review + CODEREVIEW.md audit (10 + 14 findings, all applied). Pushes used --no-verify per the fix-after-full-run policy, announced.

🤖 Generated with Claude Code

[Copilot]

Pull request overview

This PR completes the dasLLAMA “thinking/tool-use” audit by making reply-side reasoning detection architecture-aware, exposing reasoning via OpenAI-compatible reasoning_content (buffered + streaming), and adding both model-free and live, model-gated tests to prove the behavior across supported families.

Changes:

• Added a per-family reply reasoning matcher (symmetric <think>, gemma-4 asymmetric thought channel, Harmony channel-switch) plus an incremental streaming splitter.
• Extended the OpenAI server surface to emit reasoning_content (message + delta) and to respect tri-state enable_thinking while using effective_stop_ids for “thinking-off” stop behavior.
• Added/updated tests and documentation to pin exact wire shapes and to require test legs when new families declare thinking/tool markers.

Reviewed changes

Copilot reviewed 15 out of 15 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
utils/dasllama-server/test_openai_server_think.das	New live, model-gated conformance tests for reasoning/tool-call behavior through the server.
utils/dasllama-server/openai_server.das	Adds reasoning_content to responses/deltas and wires in the per-request think stream + effective stop ids.
modules/dasLLAMA/THINKING.md	Updates the thinking/tool audit spec to match the new per-family matcher and test map.
modules/dasLLAMA/tests/test_think_split.das	New model-free tests that pin whole-string + chunked streaming splits for each family format.
modules/dasLLAMA/tests/test_facade_docs.das	Fixes ENGINE_FILES list after engine entrypoints were split/moved.
modules/dasLLAMA/tests/test_chat.das	Adds render/prompt-side tests for gemma-4 thinking gate/opener and qwen35 Hermes tool markers.
modules/dasLLAMA/dasllama/dasllama.das	Exposes new facade APIs: effective_stop_ids, split_reasoning, and streaming split helpers.
modules/dasLLAMA/dasllama/dasllama_common.das	Adds ThinkMode and ChatTemplate fields for reply-side reasoning formats and thinking defaults.
modules/dasLLAMA/dasllama/dasllama_chat.das	Implements reply-side split engine + effective stop ids; updates history storage to reasoning-stripped content.
modules/dasLLAMA/dasllama/dasllama_arch_qwen35.das	Restores Hermes tool format registration for qwen35.
modules/dasLLAMA/dasllama/dasllama_arch_gptoss.das	Declares Harmony channel_switch think mode for gpt-oss.
modules/dasLLAMA/dasllama/dasllama_arch_glm4moe.das	Declares symmetric <think> reply mode for GLM-4 MoE family.
modules/dasLLAMA/dasllama/dasllama_arch_gemma4.das	Declares asymmetric thought-channel format + thinking gate/default and thinking-off extra stops.
modules/dasLLAMA/CODEREVIEW.md	Adds a binding rule requiring recognition tests when adding thinking/tool markers to an arch.
doc/reflections/das2rst.das	Updates reflection doc grouping/stanzas to include the new reasoning-facing API surface.

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

Suppressed comments (3)

modules/dasLLAMA/dasllama/dasllama_chat.das:988

• History stripping currently uses split_reasoning_(c.tmpl, reply) whenever think_mode != none, which bypasses the vocab/gate arming logic. For chatml-based families whose vocab lacks <think> tokens (e.g. qwen2.5), this can incorrectly reclassify a literal "" echo at the start of a non-thinking reply and strip it from history. Gate the split on think_turn_active_(c) (or at least c.think_vocab) so inert declarations don’t affect stored history.

    // history holds only the answer — the per-arch reasoning span strips; families with no
    // declared reply format keep the legacy unconditional <think> strip (a thinking fine-tune
    // served under a non-thinking arch — R1-distills on `llama` — must not bloat history)
    let stored = (c.tmpl.think_mode == ThinkMode.none ? strip_think(reply)
        : split_reasoning_(c.tmpl, reply).content)

utils/dasllama-server/test_openai_server_think.das:212

• In the streaming SSE loop, from_JV(jsi, type<Chunk>) produces a struct with owned arrays/strings, but it is never deleted. In daslang, plain structs/arrays don’t automatically finalize at scope exit, so this will leak per chunk and can skew long streaming tests.

                    var jsi = read_json(datas[i], perr)
                    if (jsi != null) {
                        let c = from_JV(jsi, type<Chunk>)
                        if (length(c.choices) == 1) {
                            if (!empty(c.choices[0].delta.reasoning_content)) {
                                reasoning = "{reasoning}{c.choices[0].delta.reasoning_content}"
                                if (first_reason < 0) {
                                    first_reason = i
                                }
                            }
                            if (!empty(c.choices[0].delta.content)) {
                                content = "{content}{c.choices[0].delta.content}"
                                if (first_content < 0) {
                                    first_content = i
                                }
                            }
                        }
                        unsafe {
                            delete jsi
                        }

modules/dasLLAMA/dasllama/dasllama_arch_gemma4.das:56

• d.lcaps.system_prompt is still set to false (and the comment says “no system role”), but this template now clearly uses a system turn (d.chat.system.parts) and tests expect the <|think|> gate + system prompt text to render in that system turn. Keeping system_prompt=false makes caps()/arch_lcaps() lie to API consumers about supported system prompts.

    d.lcaps.system_prompt = false   // no system role — the chat layer folds it into the first user turn

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated 1 comment.

[Copilot]

Pull request overview

Copilot reviewed 19 out of 19 changed files in this pull request and generated no new comments.

Suppressed comments (2)

modules/dasLLAMA/dasllama/dasllama_chat.das:865

• think_feed_ allocates two temporary array<string> buffers (rp/cp) and joins them on every streamed piece. In server streaming this can run per-token delta, so these per-call heap allocations and joins can become a noticeable overhead.

Consider changing the splitter helpers to append directly into two string accumulators (or store/reuse scratch buffers inside ThinkStream) so feeding a piece is allocation-free in the common case.

    var rp : array<string>
    var cp : array<string>
    if (ts.mode == ThinkMode.channel_switch) {

utils/dasllama-server/openai_server.das:1943

• For streaming chat replies, the code may emit up to two SSE chunks per scheduler piece (one for reasoning_content, one for content). This can significantly increase SSE event count and JSON serialization overhead versus the pre-split streaming path.

If throughput/latency becomes an issue, consider emitting a single chunk when both deltas are non-empty (with both reasoning_content and content set), and only splitting into two chunks when needed to preserve the “reasoning first” ordering at the boundary.

                        var rsn = ""
                        var cnt = ""
                        think_feed(w.think, emit_now, rsn, cnt)
                        if (!empty(rsn)) {
                            send_chat_chunk(srv, w.writer, w.id, w.created, "", "", "", rsn)