Skip to content

[Bug]: acpRegistry runs complete in milliseconds with zero items while the agent keeps executing queued turns — thread silently unresponsive, all output dropped #36163

Open

Description

@chenweigao

Before submitting

Area

apps/server

Steps to reproduce

Observed on a long-lived thread (~640 items, 17 runs) driven by an ACP Registry provider. Reconstruction from logs:

  1. Thread runs on provider instance acpRegistry_kimi_code (Kimi Code CLI as ACP agent), model aone/qwen3.8-max, runtime mode full-access. A run is in flight (agent streaming LLM steps and tool calls).
  2. While that run is still active, send a steering message to the thread from the web UI.
  3. The agent ends the in-flight turn early (its session/prompt resolves "succeeded" mid-reasoning; the last reasoning item is truncated mid-sentence). T3 then immediately delivers the steering message as a fresh session/prompt — and from this point on, every session/prompt for the thread resolves in 6–42 ms regardless of content.
  4. Send further user messages to the thread. Each one creates a run that is marked completed within tens of milliseconds with zero provider items.

Expected behavior

  • A run stays open until the agent's turn for that prompt actually ends, and all provider output (reasoning, tool calls, assistant messages) is attached to the run and visible in the thread.
  • If a prompt cannot be delivered or is queued behind an in-flight turn, the run should show a queued/blocked/failed state — never completed with zero items and no error.

Actual behavior

  • Every run after the steering collision is marked status: completed in 37–83 ms (requestedAt→completedAt) with zero items. The thread reports activeRunId: null, empty queue, pendingRequestCount: 0 — it looks healthy and idle.
  • Meanwhile the agent session is fully alive: it consumes every queued prompt as a full turn at its turn boundaries (dozens of LLM steps per turn, tool calls, git commits, file writes over 15+ minutes). None of that output ever reaches the thread — all session/update notifications for those turns are dropped because no run is open to attach them to.
  • The agent-side session state.json lastPrompt field is updated with each new message (proof of delivery into the agent), and its updatedAt matches the agent's next turn start.
  • Net effect: the thread becomes permanently and silently unresponsive from the user's point of view, while the agent invisibly keeps receiving and executing every message. The user gets no indication anything is wrong.

Impact

Major degradation or frequent failure

Version or commit

t3 v0.1.9 (linux-x64), runtime service-state protocol 3, activeVersion 0.1.9

Environment

Linux x64 headless server host; T3 server + web UI. Provider: acpRegistry_kimi_code (Kimi Code CLI ACP agent), model aone/qwen3.8-max, options thinking=max mode=yolo. Affected thread 8596cad7-de97-40ef-b958-4848be7edff9, agent session session_da355b67-a123-4b67-b0d4-74f9cdf81170. All timestamps below are UTC on 2026-11-13.

Logs or stack traces

T3 provider event log (~/.t3/userdata/logs/provider/events.<threadId>.log) — the collision at 12:33:49 and every prompt afterwards acking in milliseconds:

12:31:43.358  session/prompt started                     (run 14)
12:33:49.860  user steering message recorded             (attached to run 14)
12:33:49.879  session/prompt succeeded                   (run 14 turn ended mid-reasoning;
                                                          last reasoning item truncated mid-sentence)
12:33:49.913  session/prompt started                     (steering redelivered as fresh prompt)
12:33:49.955  session/prompt succeeded                   (42 ms — yet the agent's turn 15 that this
                                                          prompt started kept running until 12:36:50)
12:34:53.921  session/prompt started   (run 15)
12:34:53.927  session/prompt succeeded                   (6 ms, zero items)
12:35:06.670  session/prompt started   (run 16)
12:35:06.676  session/prompt succeeded                   (6 ms, zero items)
12:44:58.611  session/prompt started   (run 17)
12:44:58.621  session/prompt succeeded                   (10 ms, zero items)

Agent-side session log (~/.kimi-code/sessions/<workspace-dir>/<session>/logs/kimi-code.log) — continuous turns across the same window; the agent never stopped and consumed each queued prompt at its turn boundary:

12:33:37.068  llm request   turnStep=14.5    (run 14 in flight; this step's response never
                                              completes — turn ended by the steering collision)
12:33:49.922  llm request   turnStep=15.1    (turn 15 starts 9 ms after the 42 ms prompt ack;
                                              steps 15.1–15.5 run until 12:36:50)
12:36:50.443  llm request   turnStep=16.1    (turn 16 = next queued prompt; 17 steps,
                                              runs until 12:47:42)
12:47:42.903  llm request   turnStep=17.1    (turn 17 = next queued prompt)
12:51:47.845  llm request   turnStep=18.2    (still executing ~7 min after run 17 was
                                              marked "completed" with zero items)

Agent session state.json: lastPrompt equals the run-17 message text; updatedAt = 12:47:42.893Z == turn 17.1 start (12:47:42.903Z) — queued prompts are consumed at turn boundaries.

T3 thread record (t3_thread_read): runs 15/16/17 status: completed, startedAt→completedAt = 37/54/83 ms, no assistant/reasoning/tool items; itemCount frozen at 644 while the agent executed turns 15–18 (dozens of LLM steps, one 6,285-token output step, a git commit, and dozens of generated files in the thread's worktree — all invisible in the thread).

Interpretation. Two faults compound:

  1. Agent side (primary): the ACP agent resolves session/prompt before the turn ends — immediately when the prompt is queued behind an in-flight turn, and (for the 12:33:49.913 prompt) ~40 ms into a turn that then ran for 3 more minutes. Per ACP semantics the request should stay open until the turn completes with a stop reason.
  2. T3 side: the acpRegistry driver treats that instant resolve as a completed run (zero items, no anomaly flagged), and afterwards drops the session's continuing session/update stream because no run is open to attach it to. T3 also keeps offering steering on this provider (cf. [Bug]: ACP providers offer Steer but always fail with "does not support active-run steering" #34267); the "end turn, then send steering as a fresh prompt" fallback is what broke the run↔turn mapping here. A sanity check — e.g. a "completed" run with zero items while the session is still emitting updates, or updates arriving for a session with no open run — would catch this and could re-attach or at least surface the desync instead of showing a healthy idle thread.

Once desynced, the thread never recovers: every subsequent message repeats the millisecond no-op run.

Workaround

None that restores visibility. Messages sent to the thread still reach and are executed by the agent (verified via agent-side logs and worktree effects), so the loop kept running — but the user sees nothing and gets no replies. We are monitoring the agent session log and the git worktree out-of-band to observe real progress. We deliberately avoided freshContext/session restart because the live agent session is still working in the same worktree, and a restart would create two concurrent sessions writing to it.

Activity

  1. chenweigao commented on Nov 13, 2026

    @chenweigao
    Author

    Follow-up data point: the reporter switched the thread's provider model, which starts a fresh agent session, and the thread immediately returned to normal (runs open, stream, and complete with items again). This confirms the desync is confined to the existing agent session and never self-heals within it — a new session is currently the only recovery. It also contradicts my 'Workaround: none' wording: switching the model (or anything else that forces a fresh session) does restore responsiveness, at the cost of the old session's in-memory turn context.

  2. juliusmarminge commented on Nov 13, 2026

    @juliusmarminge
    Member

    @chenweigao Which Kimi Code build were you running (kimi --version, and is it the public kimi-code from the ACP Registry or an internal or custom build)? The public 2.6.1 rejects a prompt sent while a turn is running ('another turn is already in progress') and keeps session/prompt open until the turn ends, and with it the steering flow works correctly on current main.

  3. daliboy commented on Nov 15, 2026

    @daliboy

    Additional evidence from a second installation, with a different trigger (background-task completion/stop rather than a user steering message).

    Environment:

    • Windows x64, T3 desktop; connected T3 server reports version 0.1.7.
    • T3-managed ACP Registry provider acpRegistry_kimi_code, model kimi-code/k3.1-pro.
    • Installed npm package @moonshot-ai/kimi-code version 2.6.1, loaded from T3's managed tools/kimi-code/2.6.1/npm/node_modules installation. No custom build or local patch was applied during this diagnosis.
    • SHA256 of installed dist/main.mjs: c2b313e725a6fae56b08c99fbff5a1bdd93e3ee79e21bb2039c135b7aeee9f21.

    Observed sequence reconstructed from T3's provider event log and the same Kimi session's agents/main/wire.jsonl (UTC, 2026-11-15):

    15:14:43.922  T3: session/prompt started
    15:53:34.296  Kimi: WaitFor(background test task, timeout=30)
    15:54:15.118  Kimi: TaskOutput
    15:54:37.327  Kimi: TaskStop (stopping a hung test task)
    15:54:37.817  Kimi: prompt.steered
    15:54:37.818  Kimi: step.end finishReason=tool_use
    15:54:37.818  Kimi: turn.ended reason=completed
    15:54:37.820  Kimi: prompt.completed reason=completed
    15:54:37.823  T3: session/prompt succeeded; delegated task reported completed
    15:54:45.487  Same Kimi session: Bash starts another targeted test
    15:54:46.854  That test completes successfully; another step begins
    16:03:31      New targeted-test evidence produced locally
    16:04:20      Offscreen-render command completes successfully
    16:07:40      Same session continues tool/step activity
    

    The completion delivered to the parent was only a progress message saying the agent would stop the hung test and isolate the tests. It was not a final handoff. The agent then actually continued working, while the parent/sidebar had received completed. This is not inferred from an idle Node process: subsequent tool calls, test results and generated evidence were inspected. This failure mode has recurred across several delegated sessions on this installation.

    This appears related to the lifecycle mismatch described here, but we have not established that it is the identical cause as the queued-prompt/zero-item scenario, or reproduced it on current T3 main. In our case the run initially streamed normally and later completed prematurely. We are also not claiming T3 invented the end event: Kimi emitted turn.ended/prompt.completed before its internal continuation.

    Could you investigate background-task notification/TaskStop continuation ownership in addition to user steering? Expected behavior is a truthful working/waiting state until the continuation's actual final handoff, or an explicit desynchronization state if the provider ends the ACP request while continuing work. A misleading completed state risks a parent starting a second writer in the same workspace.

    Potential regression coverage: background task completion/stop -> internal steering/continuation -> further tools -> actual final result; verify status visibility, result delivery and cancellation ownership across that transition.

    No raw prompts, reasoning, project source, credentials or personal filesystem paths are included. No automatic restart/model switch was performed while the writer was active. Avoiding background task handovers is only a proposed mitigation, not a verified fix.

  4. meywd commented on Nov 21, 2026

    @meywd

    Another data point, answering the version question above: this reproduces on Kimi Code 2.6.3 (the copy T3 manages from the ACP Registry, kimi-code/k3.1-pro), T3 v0.1.7 on macOS. Unlike the 2.6.1 behaviour described above, 2.6.3 did not refuse a prompt sent while a turn was running: it queued it and session/prompt came back with end_turn within about 8–14 ms.

    What set it off here was a turn Kimi started by itself after T3's run had already closed, with no prompt from T3 and no steering from the user.

    Timeline (UTC), from Kimi's agents/main/wire.jsonl and T3's server trace:

    05:14:06.056  Kimi  prompt.steered            (turn 13, background `npm start` task)
    05:14:26.651  Kimi  turn.ended 13 completed   → T3 shows the reply and closes the run (correct)
    05:14:26.652  Kimi  turn.prompt 14  promptId=task:bash-…:timed_out
                        (Kimi's own turn, set off by the background task timing out; no T3 run is open)
    05:16:51.525  Kimi  interaction.request (Bash approval)
    05:16:51.526  Kimi  interaction.resolved decision=rejected        ← 1 ms later, nobody clicked anything
    05:17:26.041  T3    user sends "and?" → session/prompt
    05:17:26.055  T3    end_turn received (~14 ms); KimiTurnOutcome.check then warns
                        "Kimi's session log has no outcome for this turn." → run completed with 0 items
    05:17:43.376  Kimi  turn.ended 14 completed   (its final summary never reached the thread)
    05:17:43.377  Kimi  turn.prompt 15 "and?"     (the queued message actually runs now)
    05:18:10.826  T3    user sends "hello" → end_turn after ~8 ms, run completed with 0 items
    05:18:17.516  Kimi  interaction.request (Bash approval) → rejected at .517
    05:18:31.387  Kimi  turn.ended 15 / turn.prompt 16 "hello"
    05:18:44.630  Kimi  turn.ended 16 (reply "Hello! …" also never reached the thread)
    

    Two things I haven't seen covered here or in #39031 / #36181:

    1. Turns Kimi starts on its own after T3's run has closed (turn.prompt with a task:<id>:timed_out prompt ID, separate from the turn that launched the task). From that point every reply arrives one turn late and is dropped, until the session is replaced.
    2. Tool approvals are rejected automatically while the thread is out of sync. Both interaction.requests were resolved as rejected about 1 ms after they were sent, presumably because no run was open to show the approval in. The agent then reports "the command was declined" to a user who never saw it. A safer default would be to show the request (or hold it) rather than reject it.

    Switching the thread to another provider recovered it, matching the earlier follow-up.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions