|
| 1 | +--- |
| 2 | +title: "Human-in-the-loop" |
| 3 | +sidebarTitle: "Human-in-the-loop" |
| 4 | +description: "Pause the agent mid-response to ask the user a clarifying question, then resume with their answer." |
| 5 | +--- |
| 6 | + |
| 7 | +Some turns need to stop and ask the user something before they can finish — picking between options, confirming a destructive action, or clarifying an ambiguous request. The AI SDK calls this **human-in-the-loop** (HITL), and the building block is a tool with no `execute` function. |
| 8 | + |
| 9 | +When the LLM calls a tool that has no `execute`, `streamText` ends with the tool call still pending. The turn completes cleanly, the frontend renders UI to collect the answer, and when the user responds, a new turn resumes with the answer merged into the same assistant message. |
| 10 | + |
| 11 | +## How it works |
| 12 | + |
| 13 | +``` |
| 14 | +Turn N: |
| 15 | + User message → run() |
| 16 | + LLM streams text → calls askUser tool (no execute) |
| 17 | + streamText ends with tool-call in `input-available` state |
| 18 | + onTurnComplete fires (finishReason = "tool-calls") |
| 19 | + Agent idle |
| 20 | +
|
| 21 | +Frontend: |
| 22 | + Renders question + option buttons from tool input |
| 23 | + User clicks → addToolOutput({ tool, toolCallId, output }) |
| 24 | + sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls |
| 25 | + → sendMessage() fires next turn |
| 26 | +
|
| 27 | +Turn N+1: |
| 28 | + hydrateMessages / accumulator sees the updated assistant message |
| 29 | + run() is called, LLM continues from the tool result |
| 30 | + onTurnComplete fires (finishReason = "stop", responseMessage is the FULL merged message) |
| 31 | +``` |
| 32 | + |
| 33 | +The AI SDK's `toUIMessageStream` automatically reuses the assistant message ID across the pause (we pass `originalMessages` internally), so `responseMessage` in the post-resume `onTurnComplete` is the **full merged message** — the original text, the completed tool call, and any follow-up content — not just the new parts. |
| 34 | + |
| 35 | +## Backend: define the tool |
| 36 | + |
| 37 | +A HITL tool has an `inputSchema` describing what the model can ask, but **no `execute` function**. When the LLM calls it, `streamText` returns control to your agent. |
| 38 | + |
| 39 | +```ts trigger/my-chat.ts |
| 40 | +import { chat } from "@trigger.dev/sdk/ai"; |
| 41 | +import { streamText, tool } from "ai"; |
| 42 | +import { openai } from "@ai-sdk/openai"; |
| 43 | +import { z } from "zod"; |
| 44 | + |
| 45 | +const askUser = tool({ |
| 46 | + description: |
| 47 | + "Ask the user a clarifying question when you need their input. " + |
| 48 | + "Present 2-4 options for them to pick from.", |
| 49 | + inputSchema: z.object({ |
| 50 | + question: z.string(), |
| 51 | + options: z |
| 52 | + .array( |
| 53 | + z.object({ |
| 54 | + id: z.string(), |
| 55 | + label: z.string(), |
| 56 | + description: z.string().optional(), |
| 57 | + }) |
| 58 | + ) |
| 59 | + .min(2) |
| 60 | + .max(4), |
| 61 | + }), |
| 62 | + // No execute function — streamText ends, the frontend supplies the output |
| 63 | + // via addToolOutput, and the next turn continues from the result. |
| 64 | +}); |
| 65 | + |
| 66 | +export const myChat = chat.agent({ |
| 67 | + id: "my-chat", |
| 68 | + run: async ({ messages, signal }) => { |
| 69 | + return streamText({ |
| 70 | + model: openai("gpt-4o"), |
| 71 | + messages, |
| 72 | + tools: { askUser }, |
| 73 | + abortSignal: signal, |
| 74 | + }); |
| 75 | + }, |
| 76 | +}); |
| 77 | +``` |
| 78 | + |
| 79 | +## Frontend: render the question and collect the answer |
| 80 | + |
| 81 | +Two pieces on the client: |
| 82 | + |
| 83 | +1. **UI for the pending tool call** — render when the tool part is in `input-available` state, i.e. the LLM has called the tool but there's no output yet. |
| 84 | +2. **Auto-send on resolution** — use `sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls` so answering kicks off the next turn without the user having to hit "send." |
| 85 | + |
| 86 | +```tsx |
| 87 | +import { useChat, lastAssistantMessageIsCompleteWithToolCalls } from "@ai-sdk/react"; |
| 88 | +import { useTriggerChatTransport } from "@trigger.dev/react-hooks"; |
| 89 | + |
| 90 | +function ChatView({ chatId, accessToken }: { chatId: string; accessToken: string }) { |
| 91 | + const transport = useTriggerChatTransport({ task: "my-chat", accessToken }); |
| 92 | + const { messages, sendMessage, addToolOutput } = useChat({ |
| 93 | + id: chatId, |
| 94 | + transport, |
| 95 | + sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls, |
| 96 | + }); |
| 97 | + |
| 98 | + return ( |
| 99 | + <> |
| 100 | + {messages.map((m) => |
| 101 | + m.parts.map((part, i) => { |
| 102 | + if (part.type === "tool-askUser" && part.state === "input-available") { |
| 103 | + return ( |
| 104 | + <AskUserCard |
| 105 | + key={i} |
| 106 | + question={part.input.question} |
| 107 | + options={part.input.options} |
| 108 | + onAnswer={(opt) => |
| 109 | + addToolOutput({ |
| 110 | + tool: "askUser", |
| 111 | + toolCallId: part.toolCallId, |
| 112 | + output: { optionId: opt.id, label: opt.label }, |
| 113 | + }) |
| 114 | + } |
| 115 | + /> |
| 116 | + ); |
| 117 | + } |
| 118 | + if (part.type === "text") return <Markdown key={i}>{part.text}</Markdown>; |
| 119 | + return null; |
| 120 | + }) |
| 121 | + )} |
| 122 | + </> |
| 123 | + ); |
| 124 | +} |
| 125 | +``` |
| 126 | + |
| 127 | +`addToolOutput` patches the assistant message locally with `state: "output-available"` and fills in `output`. `lastAssistantMessageIsCompleteWithToolCalls` detects that every pending tool call now has a result, and `useChat` fires a new `sendMessage` — the backend picks it up as the next turn. |
| 128 | + |
| 129 | +## Detecting a paused turn in `onTurnComplete` |
| 130 | + |
| 131 | +Two ways to detect "this turn paused for user input" vs "this turn finished normally": |
| 132 | + |
| 133 | +### Via `finishReason` (recommended) |
| 134 | + |
| 135 | +The AI SDK's finish reason is surfaced on every `onTurnComplete` event. If the model stopped on tool calls, it's `"tool-calls"`: |
| 136 | + |
| 137 | +```ts |
| 138 | +onTurnComplete: async ({ finishReason, responseMessage }) => { |
| 139 | + if (finishReason === "tool-calls") { |
| 140 | + // Turn paused — assistant message has pending tool call(s) |
| 141 | + const pending = responseMessage?.parts.filter( |
| 142 | + (p) => p.type.startsWith("tool-") && p.state === "input-available" |
| 143 | + ); |
| 144 | + // Persist as a checkpoint / partial turn |
| 145 | + } else { |
| 146 | + // finishReason === "stop" — normal completion |
| 147 | + // Persist as a completed turn |
| 148 | + } |
| 149 | +}; |
| 150 | +``` |
| 151 | + |
| 152 | +<Note> |
| 153 | + `finishReason` is only undefined for manual `chat.pipe()` flows or aborted streams. For the common `run() → return streamText(...)` pattern it's always populated. |
| 154 | +</Note> |
| 155 | + |
| 156 | +### Via response parts |
| 157 | + |
| 158 | +If you need more nuance (e.g. which specific tool is pending), inspect the parts directly: |
| 159 | + |
| 160 | +```ts |
| 161 | +function pendingToolCalls(message: UIMessage): string[] { |
| 162 | + return message.parts |
| 163 | + .filter((p) => p.type.startsWith("tool-") && p.state === "input-available") |
| 164 | + .map((p) => p.toolCallId); |
| 165 | +} |
| 166 | +``` |
| 167 | + |
| 168 | +Both `finishReason === "tool-calls"` and `pendingToolCalls(responseMessage).length > 0` are equivalent in practice. Use `finishReason` for dispatch, parts for detail. |
| 169 | + |
| 170 | +## Persistence: one message vs one record per pause |
| 171 | + |
| 172 | +Because the AI SDK reuses the assistant message ID across the pause, the "same turn" from the user's perspective maps to **two `onTurnComplete` firings** on the server — but both receive a `responseMessage` with the **same `id`**, and the second firing's `responseMessage` contains the fully merged content. |
| 173 | + |
| 174 | +Two common persistence patterns: |
| 175 | + |
| 176 | +### Overwrite on every turn (simplest) |
| 177 | + |
| 178 | +Just store the latest `uiMessages` array on every `onTurnComplete`. The paused-turn write is overwritten by the resume-turn write; the final DB state has the full merged message. |
| 179 | + |
| 180 | +```ts |
| 181 | +onTurnComplete: async ({ chatId, uiMessages }) => { |
| 182 | + await db.chat.update({ |
| 183 | + where: { id: chatId }, |
| 184 | + data: { messages: uiMessages }, |
| 185 | + }); |
| 186 | +}, |
| 187 | +``` |
| 188 | + |
| 189 | +Use this unless you specifically need an audit trail. |
| 190 | + |
| 191 | +### Checkpoint nodes (immutable history) |
| 192 | + |
| 193 | +For apps that want every pause point recorded as its own immutable snapshot (branching, replay, diff review), save a checkpoint when paused and a sibling when complete: |
| 194 | + |
| 195 | +```ts |
| 196 | +onTurnComplete: async ({ chatId, responseMessage, finishReason, uiMessages }) => { |
| 197 | + if (!responseMessage) return; |
| 198 | + |
| 199 | + if (finishReason === "tool-calls") { |
| 200 | + // Paused — save a checkpoint |
| 201 | + await db.turnCheckpoint.create({ |
| 202 | + data: { |
| 203 | + chatId, |
| 204 | + messageId: responseMessage.id, |
| 205 | + parts: responseMessage.parts, |
| 206 | + kind: "partial", |
| 207 | + }, |
| 208 | + }); |
| 209 | + } else { |
| 210 | + // Completed — save a sibling with the merged full message |
| 211 | + await db.turnCheckpoint.create({ |
| 212 | + data: { |
| 213 | + chatId, |
| 214 | + messageId: responseMessage.id, |
| 215 | + parts: responseMessage.parts, |
| 216 | + kind: "final", |
| 217 | + }, |
| 218 | + }); |
| 219 | + } |
| 220 | + |
| 221 | + // Always update the canonical chat record for `hydrateMessages` to load |
| 222 | + await db.chat.update({ |
| 223 | + where: { id: chatId }, |
| 224 | + data: { messages: uiMessages }, |
| 225 | + }); |
| 226 | +}; |
| 227 | +``` |
| 228 | + |
| 229 | +Both writes see `responseMessage.id` as the same value — they're checkpoints of the same logical message. Grouping by `messageId` + ordering by `createdAt` gives you the progression. |
| 230 | + |
| 231 | +## Multi-pause turns |
| 232 | + |
| 233 | +A single logical turn can pause more than once — the LLM asks question A, gets the answer, thinks, then asks question B before finishing. Each pause fires its own `onTurnComplete` with `finishReason === "tool-calls"`; only the last firing has `finishReason === "stop"`. The checkpoint pattern above handles this naturally — each pause adds a new checkpoint sharing the same `responseMessage.id`. |
| 234 | + |
| 235 | +## Gotchas |
| 236 | + |
| 237 | +- **Don't set an `execute` function on the HITL tool.** If it has one, `streamText` will call it immediately instead of handing control back. |
| 238 | +- **The frontend must use `sendAutomaticallyWhen`.** Without it, the user has to press Enter after answering — `addToolOutput` updates local state but doesn't fire a new turn by itself. |
| 239 | +- **Don't mutate `responseMessage` in `onTurnComplete`.** It's the captured snapshot. To add custom parts, use `chat.response.append()` in `onBeforeTurnComplete` (while the stream is open). |
| 240 | +- **Stop handling.** If the user stops the run while a pause is active (`chat.stop()` on the transport), `onTurnComplete` fires with `stopped: true` and `finishReason` reflecting the last successful step. Treat stopped paused turns the same as stopped normal turns. |
0 commit comments