Skip to content

feat: add Dispatch agents with streamed chat and background processes - #1067

Draft
niiyeboah wants to merge 281 commits into
mainfrom
agt_683b115bc1e9/dispatch-harness-research
Draft

niiyeboah wants to merge 281 commits into
mainfrom
agt_683b115bc1e9/dispatch-harness-research

Conversation

@niiyeboah

@niiyeboah niiyeboah commented Sep 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Adds an opt-in Dispatch agent that runs Claude Code, Codex, or OpenCode and streams its work into the existing Chat feed. Enable Dispatch agent (beta) in Settings, then choose the provider and model when creating a session. Each provider uses its CLI installation and existing login on the server.

User experience

  • Each turn shows the prompt, collapsible tool activity, output and diffs, and the result. Reviews, pins, peer messages, and presence appear in the same feed. Review cards remain fully visible in compact peer rows.
  • A task list above the composer shows current work and completion progress. Every Dispatch agent can maintain it through dispatch_update_tasks, including Claude Code sessions without a native todo tool. Native ACP plans also work; no goal workflow is required.
  • Queued messages support Send now and Remove. Stop and Ctrl+C interrupt the active turn. The slash menu shows provider commands, and the model picker supports the options the provider exposes.
  • New Dispatch sessions show an animated startup card in Chat with plain-language stages and estimated progress through workspace setup, dependency installation, provider connection, and session settings. Progress advances only on observed stage changes, never completes on a timer, and respects reduced motion. The composer stays disabled until startup finishes; failures show the existing error/sign-in guidance. The Terminal segment remains available for provider login and diagnostics.
  • /usage separates context usage, subscription limits, and reported API cost. Reports show their observation time. Claude attempts to refresh subscription usage through the CLI's existing credentials and retains the last report on failure. Codex selects reports by event timestamp, not log modification time. Monthly cost budgets are configurable for providers that report cost.
  • A service restart marks the active turn interrupted, resumes the provider session when supported, and redelivers queued messages. When a provider cannot resume, Dispatch preserves the chat history and explains that the provider session restarted.

Implementation and limits

  • Providers communicate over Agent Client Protocol (ACP). Model prefixes select the adapter: claude/, codex/, gemini/, or opencode/.
  • The backend records stream events and serializes prompts and task updates per agent. Schema changes add agent_stream_events and agent_chat_messages.delivery_text; both migrations are additive and guarded.
  • Capabilities vary by provider: Claude supports nested subagent activity; Codex reports tokens without cost; Gemini sets its model at launch and reports no ACP usage. Provider-plan data can be unavailable or stale; the UI says when it was last reported instead of presenting a refresh time as a new observation.
  • Claude's direct subscription refresh requires the CLI's file-backed OAuth credential. Other credential storage falls back to the local Claude /usage report. Dispatch does not save or expose provider credentials.
  • Launch guidance also asks CLI agents to continue into accepted work, and persona reviews default to their parent's agent type and provider.

Background processes

  • New session-scoped dispatch_background_process tool starts non-interactive shell commands without blocking a turn; completion automatically queues an agent message. Supports list, inspect, and stop.
  • Background processes and tasks share compact headers, typography, padding, and row styles. A single right-aligned chevron expands the process list; running work appears first in a four-row preview, with Show more/fewer for history. Open a row for command, working directory, elapsed time, live output, exit status, and Stop. Touch targets remain accessible on mobile.
  • Status and bounded output persist across page reloads. Session stop terminates process groups; server restart marks unfinished records interrupted rather than restarting commands. An additive agent_background_processes table stores the records.
  • Limits: four concurrent processes per session, 16 server-wide, one-hour default timeout (up to 24 hours), and the latest 20 records plus running processes. Agent guidance asks for sequential heavy validation.
  • This is not an interactive terminal and does not retroactively adopt provider-native shell processes. It tracks commands launched through the Dispatch tool.

Long-chat rendering

  • Window chat histories and top-level tool activity rails once they exceed 80 rows, using measured variable heights and a six-row buffer on each side.
  • Preserve older-history anchors, agent-switch reading positions, and activity disclosure state. Keep focused rows mounted.
  • This reduces mounted React/DOM work; it does not discard loaded query data or individually virtualize one large markdown/code result or nested subagent details. Browser Find and selection cover mounted content.
  • Stress tests pass with 5,000 messages (~11 mounted in the checked viewport) and a 1,000-step active turn (~17 mounted). They cover older-page anchors, agent-switch restoration, incoming messages while reading history, jump-to-latest, tool output, and mobile resizing.

Gemini follow-up

Gemini CLI support remains implemented, but Gemini is temporarily hidden from UI provider/model choices and Settings copy. Existing sessions and backend support are retained. Before re-enabling it:

  • Define and validate an in-app/in-chat sign-in flow, including actionable access-denied and account-entitlement errors.
  • Clearly distinguish API-key billing from signed-in account/subscription access and show the active authentication source.
  • Review account eligibility and organizational access with management; management confirmation is still outstanding.

Validation

  • Type checking, production web finalization, formatting, and changed-file web lint checks pass.
  • Prior full unit baseline: 3,586 server tests, 2,143 web tests, 60 browser-extension tests, and four extension-manifest checks. The compact-list update additionally passes all 81 focused web tests. Startup coverage includes stage reporting, slow-stage estimates, opening-prompt delivery, error handling, and composer readiness.
  • The background-process suite covers cancellation during startup, timeout, ownership, bounded output, and shutdown.
  • Full non-live E2E run: 207 passed, 21 expected live-runtime skips, no failures or retries needed. Includes compact task/process styling, keyboard disclosure, Show more/fewer, mobile output/Stop controls and focus restoration, the startup flow, and both long-history stress tests.
  • A separate live-runtime test with the fake Claude ACP adapter starts two background commands, inspects output, stops one, verifies persistence after reload, and checks the completion notification queue.
  • Playwright interactions and shared screenshots cover the background process list/detail on desktop and mobile and Gemini's absence from the provider picker. Browsers were closed.
  • Startup Playwright coverage exercises setup, connection, configuration, sign-in failure, retry, and readiness. Production-built desktop/mobile screenshots verify the startup card, reduced motion, and composer handoff; browsers are closed.
  • Earlier cleanup validation also covered live task updates, full review cards, task-list expansion, dated usage fallback, and real Claude/Codex provider usage reads.

Setup and operational details: docs/10-operations-runbook.md.

niiyeboah and others added 30 commits September 4, 2026 20:50
…ersion

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ches

Turn settles reach the web as agent events, not chat.changed, so the
Harness view sat on a stale trace until a reload. Invalidating alongside
the feed covers agent upserts, reconnects, reviews and messages.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r dsh skills

Steps keep the tool call's raw input (dsh sends it), so an execute step
reads as its command, a read step as its file, and an MCP tool as its
name with a digest of its arguments; steps with nothing underneath are
no longer expandable. Locations fall back to the input or read wrapper.

Dispatch Harness agents lose the redundant Chat segment: the toggle is
Harness | Console and a stored Chat preference lands on Harness.

Typing / in the Harness composer opens a picker over the skills dsh can
load (project .agents/skills, .dsh/skills, and the harness home), served
by GET /api/v1/agents/:id/harness/skills; the persona tells the agent a
leading /<name> means load that skill.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…g, keyed providers only

dsh advertises session config options (a provider-grouped model select and
a reasoning-effort select) over the Agent Client Protocol. The driver keeps
them per session and can set one; the supervisor persists a model switch
on the agent and drops provider groups whose API key the service lacks.
The create dialog's dsh list now comes from a cached probe session, so
every model dsh serves (gpt-5.6-sol, terra, …) is offered; dsh model ids
validate by shape. The Harness pane shows a model · effort chip that opens
the picker; /model in the composer opens it too.

Also stops the turn stream snapping to the bottom on refetch: it follows
only while the reader is at the end.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ugh the picker

Radix Select refuses an item whose value is ""; dsh's reasoning-effort
list opens with one. The picker holds values under a stand-in and decodes
them at the edges.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A persona launch (and an MCP launch) hands the kickoff as initialPrompt,
which CLI agents type into the pane. A dsh agent's first turn is read
from the Chat launch post, which only carried the create dialog's context
prompt, so persona reviews on Dispatch Harness agents started with no
brief and sat idle. The launch post now falls back to initialPrompt for
dsh agents.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…6-only catalog

Step details now format what they show: read output becomes a highlighted
code block with a line-number gutter, path lists render as paths, JSON
tool output and arguments pretty-print with highlighting, and shell
output stays plain. Prompts Dispatch injects (review thread updates,
persona kickoffs, the rename nudge) render as a one-line notice with the
full block on demand instead of as something the user typed.

The harness agent's view segment reads "Chat". The dsh model catalog
carries the provider as a group for section headers, and OpenAI offers
only the gpt-5.6 generation (plus whatever a session is already on).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…nAI route

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Code, JSON, path and plain-output blocks wrap instead of scrolling
sideways; each code line is a grid row so wrapped text stays under its
number. Long blocks clip at a screenful with a Show-all control that
expands them to full height, so the conversation keeps the only
vertical scrollbar. Lines are highlighted one at a time with a definite
language or not at all.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
- The settle-time idle yields to a done/blocked/waiting_user the agent set
  during the turn, so a reviewer's done survives its own turn ending.
- Start on an agent whose shell is alive but whose harness is gone now
  starts the harness; an unexpected child exit (code 0 included) marks the
  agent instead of leaving it running over a dead child.
- A turn cut off by Stop, a crash or a server restart settles: the exit
  handler settles the open turn in place, and start() reconciles rows a
  previous process left open (migration-free, jsonb merge).
- dsh job runs get their launch post, so the job prompt becomes the first
  turn; MCP and template launches deliver the launch header and rendered
  template through a stored delivery text (migration 0050) while Chat
  keeps showing the prompt as written.
- Default OpenAI model is gpt-5.6-sol, matching the catalog; the static
  fallback lists the 5.6 line.
- The overlay is removed when the driver fails to start; the first-turn
  promise is caught; held reports the supervisor's queue for dsh.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The supervisor looks up the agent's active job run and, like the pane
launch, attaches /api/mcp/jobs/<run>/<agent> with the job token so
job_complete/job_failed/job_needs_input/job_log exist, and the persona
carries the job startup guidance. Review #294 item 1426.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…n config fresh

While the agent is still being created the Harness shows the activity
bars with the setup status and the composer is closed, so a prompt cannot
be sent into nothing; the same bars, at glyph size, mark a running turn.
The session config (model, effort, running) now invalidates on the same
events as the feed and polls while the session is not yet up, so the chip
no longer sits on a stale "not running" after the first turn.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Files dropped anywhere on the Harness pane attach to the composer, with
the same overlay the terminal shows. Prompt attachments use the agent
media route, render as thumbnails, and open the lightbox. Questions the
agent posts through dispatch_chat_post are threaded onto their turn by
the server and rendered as option cards in the Harness; a click answers
through the chat answer route, and a typed reply answers the newest open
free-text question, as in Chat.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
dsh runs its shell commands in its own process, so the pane cannot be
that shell. The recorder now appends every settled command (prompt line,
output, status, duration) to <DSH_HOME>/logs/<agent>.log, and the dsh
pane command tails that file in a tmux split above the interactive login
shell, so the Console reads as the agent's terminal and still takes the
user's own commands.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@niiyeboah
niiyeboah marked this pull request as draft September 11, 2026 23:45
niiyeboah and others added 17 commits September 11, 2026 17:57
An install that ran the v0.38.14 release has 0051_agent-chat-reactions in
pgmigrations where this branch ships 0051_agent-stream-events. node-pg-migrate
compares the stored names against the shipped files position by position, so
the server threw before the first migration ran and every start after the
update crash-looped. The record is dead bookkeeping: this branch ships the same
table as 0057_agent-chat-reactions, and that file is guarded with IF NOT
EXISTS, so re-applying it on top of the existing table is a no-op.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…resence

Every dispatch_event landed twice: as a row in the feed and, in the same
words, in the presence line above the composer. The row was the duplicate,
and each one moved the reader's tail and fired the follow logic for content
that said nothing new. Status entries now lay out no row at all; the server
still emits them and the presence line still shows the latest one.

The presence line also moves above the harness chrome, so what the agent is
doing now reads before the model, auth and usage chips, which are controls
rather than news. Non-harness agents see no change: their chrome is null.

Removed with it: collapseFeed, StatusLine and the status cluster in ChatFeed,
which had no other callers.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The feed followed the bottom with an instant scrollTo after each render, so
every change in the live turn's size was a snap: a step row landing, the
thinking row appearing after its 500ms delay and vanishing on the next tool
call, a streamed line, and the largest one, the rail folding from its full
height to a one-line summary on settle. Depending on the size of the
answer the whole feed jumped by that much at once.

The turn's rail and answer now sit in one AutoHeight body, a wrapper that
measures its content with a ResizeObserver and animates its height between
sizes on the harness's standard curve (instant under reduced motion). The
pane pins the bottom through those frames with a ResizeObserver on the feed
content while the reader is following, so older rows glide upward instead
of the reader chasing the bottom. Measured on a live turn at 50ms: the
distance from the bottom stayed at 0px throughout, and the settle collapsed
180 to 74px across four samples rather than one.

Without a ResizeObserver the wrapper stays auto and the content sizes
itself as before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A pass over the whole branch for code and comments that carry no weight.

Server: unused types and a struct field that was never read
(StatusPayload, StreamPayloadByKind, Live.stderrTail), exports nothing
imports (TERMINAL_OUTPUT_MAX_BYTES, AUTONOMOUS_IDLE_MS, modelOptionOf,
isUsageEngineId, HARNESS_MODEL_HALF, the feed limit re-exports,
SOURCE_RANK, locationsFromInput), the three unreachable 503 branches on
the harness usage routes (registerSystemRoutes has one caller and it
always wires them), and the chatSurface flag buildHarnessPersona took
and never read.

Web: the PromptKit-parity fields the server never emits (retry and
skipped step statuses, clarification results, attempts, context chips,
error hints, prompt attachments) and the PromptLine branch that rendered
a typed prompt, which the turn entry never reaches since typed prompts
render as the user's own post. The ungrouped AgentModelSelect path no
longer carries the group rendering the grouped early return already
handles. Two dead exports in chat-composer.

Tests and fixtures: an unused stderr handle on the fake ACP agent, a
duplicated constant, stderr traces in the E2E fake, and comments that
named review rounds or people instead of behavior.

The assisted-update manifest now counts the migrations this release
ships and lists the v0.38.14 reactions record the runner forgets at boot.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… autonomous interrupt

Two follow-ups to the delayed-ACP-tail fix, from the code review.

A text chunk that arrived after a prompt settled opened an assistant row
under that turn and nothing closed it, so the next prompt's reply landed
in the same row and read as part of the previous turn. The started
branch now closes any open text before it opens the new turn row, with
a recorder test for the sequence.

interruptAutonomous ran from the HTTP request outside the per-agent
event chain the recorder's callers are expected to keep, so it could
interleave with a driver event mid-write. Both the driver listener and
the interrupt now go through one chain helper on the supervisor.

The trailing window's comment says why it stays open until the next
prompt: no bridged engine starts work of its own after a response.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ve symlinks in the path picker

From the backend security review.

The chat message behind a turn's prompt is looked up by an id parsed
out of prompt text, which can embed another agent's message or a
review body verbatim; the lookup now filters by the agent as well, so
a stray id reads as a prompt with no chat text rather than as another
agent's message. Same for the queue.

Plan entries, status rows, tool titles and location lists were the
last engine-supplied payloads written without a bound; each has one
now, and the driver's stderr tail is capped by bytes and per line as
well as by line count.

The @ path picker resolves both the typed directory and the working
tree before deciding whether it is inside the tree, so a symlink that
leaves the tree lists directories only, like any other outside path.

The per-agent usage route had no caller and no shared type; removed
with its helper.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…onfig path, probe engines on the harness PATH

From the infra review.

The adapter for Claude Code spawns the claude CLI as its own child and
only reaps it from a process exit hook, so a SIGKILL to the adapter
(start failure, the last rung of stop, killAll at shutdown) orphaned a
CLI holding full-access permissions and a live MCP token. The driver
now spawns each adapter detached, at the head of its own process
group, and every rung signals the group, falling back to the child
alone when the group is already gone.

Claude Code keeps `.claude.json` in the home directory (or in
CLAUDE_CONFIG_DIR), not under ~/.claude/, so the provider card could
never read it; the path is fixed and the test pins the exact one.

The login probes ran bare engine names on the service's own PATH,
which lacks ~/.local/bin, so a runbook-style install said "unavailable"
for an engine whose agents start fine. One harnessSearchPath helper
now serves the engine spawn and the probes.

The prerelease migration fix-up runs in one transaction, so a boot cut
between the DELETE and the type renames cannot leave rows unrenamed.

binsFor honours the CODEX_BIN alias the config accepts, through a
codexBinConfigured flag on AppConfig.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…its; document the v0.38.14 rollback repair

From the release readiness review.

An install that predates the installer's 30 s ExitTimeOut keeps
launchd's 20 s default, and the shutdown sequence could reach 24 s
with a stalled archive and stalled deliveries ahead of the harness
ladder, so launchd would SIGKILL the server mid-ladder and orphan
engine children. The bounded harness teardown now runs first, right
after the job service, so it always completes inside the budget; only
the pre-existing waits can be cut short.

Rolling back to v0.38.14 fails at boot because that release numbers
the reactions table 0051 where this one stores agent_stream_events,
and node-pg-migrate compares the lists position by position. The
assisted-update manifest and the runbook now name both targets and
carry the verified bookkeeping repair for v0.38.14; v0.38.13 boots as
before.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
"Console" promises a log of the agent's work. A dispatch agent never
writes there: its output goes to Chat, and the pane is a bare shell
that only earns its keep for the engine's login command. A reader who
flipped to it found an empty prompt and no reason for it.

The segment now reads "Terminal" for that type alone, aria-label
included, and the two buttons that open it say "Opening Terminal".
Every other type still writes to the pane and still reads "Console".

Recovered from an unpushed worktree. The rest of that worktree's WIP
moved status events out of the feed and led the composer bar with the
presence line, which 7c5b5b1 already does more thoroughly, so only
this part came across.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The unit test proves the prop wiring; this proves the label survives the
real toggle, and that clicking it still raises the pane. Screenshot moved
ahead of the click so dispatch-one-feed.png still captures the feed it is
named for.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Three things the pane got wrong as soon as a conversation got busy.

A queued message rendered as a two-line excerpt, a chip row and two
buttons, about 70px, with no cap on how many. Four of them walked the
composer off a short pane. A queued message is a thing to act on rather
than read, so the row is now one line and past two the rest fold behind
a count; the opened list is bounded and scrolls inside itself.

Agent-to-agent posts rendered in full, and a handful between two turns
buried the conversation the user was actually in. The header already
says who spoke to whom and when, so the body now waits behind a click.
Bounded by height rather than line-clamp: a post body is arbitrary
markdown, and -webkit-line-clamp needs a -webkit-box, which breaks
lists and code blocks.

The pane also opened in the wrong directory. `-il` sources the user's
rc, so an rc ending in `cd ~/some-repo` runs after the setup script's
`cd "$EFFECTIVE_CWD"` and wins. Verified on this host: three harness
agents sat in ~/neo while their cwd was a worktree elsewhere. CLI
agents never showed it because they exec a binary instead of a shell,
which is why this went unnoticed; `terminal` agents had it too. A
one-shot hook on the first prompt is the only place later than the rc.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Drives the fold the way a reader meets it: a peer writing six lines into
the feed, bounded under 90px until Show more, taller after. The unit test
proves the state; this proves the height the reader actually sees.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The fold I shipped earlier kept the post's full anatomy, an avatar column,
a header row, a clipped body and a Show more button, and only clipped the
middle. That saved lines but read as a third kind of thing in the feed.
The turn stream already has a row for "something happened that you did
not type": the Dispatch notice, a gutter icon, a chip, the opening of the
text inline and a chevron. A peer's post is the same class of event, so it
now takes that row.

What a peer post carries stays: sender, relation, recipient, time, the
sender's own icon, indent and tint for a side conversation, and the
Sending / Not delivered word, which moves into the row so it still reads
while the body is folded. The copy action sits outside the toggle, since
a button inside a button is not HTML.

The old avatar and header columns are not rendered for a peer at all,
rather than hidden. A hidden copy would have left two authors in the DOM
for every peer post, one of them read aloud.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The notice row this borrows from opens with its chip and reads the text
after it, so the eye finds the kind of row before the words. The peer row
had the name first and the chip second, which put the two rows out of
step at the one point they are meant to match.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@niiyeboah niiyeboah changed the title feat: Dispatch Harness, an ACP agent type with four engines and a turn-stream view feat: add Dispatch agents with streamed chat, tasks, and usage Sep 15, 2026
@niiyeboah niiyeboah changed the title feat: add Dispatch agents with streamed chat, tasks, and usage feat: add Dispatch agents with streamed chat and background processes Sep 15, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant