Skip to content

Commit 98c40da

Browse files
committed
docs(ai-chat): add human-in-the-loop patterns page
Covers askUser-style mid-turn user input end-to-end: defining a no-execute tool, rendering pending tool calls on the frontend with addToolOutput + sendAutomaticallyWhen, detecting paused turns via finishReason, and persistence patterns (overwrite vs checkpoint nodes) for apps that need an immutable audit trail. Resolves TRI-8404.
1 parent c0155d5 commit 98c40da

2 files changed

Lines changed: 242 additions & 1 deletion

File tree

Lines changed: 240 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,240 @@
1+
---
2+
title: "Human-in-the-loop"
3+
sidebarTitle: "Human-in-the-loop"
4+
description: "Pause the agent mid-response to ask the user a clarifying question, then resume with their answer."
5+
---
6+
7+
Some turns need to stop and ask the user something before they can finish — picking between options, confirming a destructive action, or clarifying an ambiguous request. The AI SDK calls this **human-in-the-loop** (HITL), and the building block is a tool with no `execute` function.
8+
9+
When the LLM calls a tool that has no `execute`, `streamText` ends with the tool call still pending. The turn completes cleanly, the frontend renders UI to collect the answer, and when the user responds, a new turn resumes with the answer merged into the same assistant message.
10+
11+
## How it works
12+
13+
```
14+
Turn N:
15+
User message → run()
16+
LLM streams text → calls askUser tool (no execute)
17+
streamText ends with tool-call in `input-available` state
18+
onTurnComplete fires (finishReason = "tool-calls")
19+
Agent idle
20+
21+
Frontend:
22+
Renders question + option buttons from tool input
23+
User clicks → addToolOutput({ tool, toolCallId, output })
24+
sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls
25+
→ sendMessage() fires next turn
26+
27+
Turn N+1:
28+
hydrateMessages / accumulator sees the updated assistant message
29+
run() is called, LLM continues from the tool result
30+
onTurnComplete fires (finishReason = "stop", responseMessage is the FULL merged message)
31+
```
32+
33+
The AI SDK's `toUIMessageStream` automatically reuses the assistant message ID across the pause (we pass `originalMessages` internally), so `responseMessage` in the post-resume `onTurnComplete` is the **full merged message** — the original text, the completed tool call, and any follow-up content — not just the new parts.
34+
35+
## Backend: define the tool
36+
37+
A HITL tool has an `inputSchema` describing what the model can ask, but **no `execute` function**. When the LLM calls it, `streamText` returns control to your agent.
38+
39+
```ts trigger/my-chat.ts
40+
import { chat } from "@trigger.dev/sdk/ai";
41+
import { streamText, tool } from "ai";
42+
import { openai } from "@ai-sdk/openai";
43+
import { z } from "zod";
44+
45+
const askUser = tool({
46+
description:
47+
"Ask the user a clarifying question when you need their input. " +
48+
"Present 2-4 options for them to pick from.",
49+
inputSchema: z.object({
50+
question: z.string(),
51+
options: z
52+
.array(
53+
z.object({
54+
id: z.string(),
55+
label: z.string(),
56+
description: z.string().optional(),
57+
})
58+
)
59+
.min(2)
60+
.max(4),
61+
}),
62+
// No execute function — streamText ends, the frontend supplies the output
63+
// via addToolOutput, and the next turn continues from the result.
64+
});
65+
66+
export const myChat = chat.agent({
67+
id: "my-chat",
68+
run: async ({ messages, signal }) => {
69+
return streamText({
70+
model: openai("gpt-4o"),
71+
messages,
72+
tools: { askUser },
73+
abortSignal: signal,
74+
});
75+
},
76+
});
77+
```
78+
79+
## Frontend: render the question and collect the answer
80+
81+
Two pieces on the client:
82+
83+
1. **UI for the pending tool call** — render when the tool part is in `input-available` state, i.e. the LLM has called the tool but there's no output yet.
84+
2. **Auto-send on resolution** — use `sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls` so answering kicks off the next turn without the user having to hit "send."
85+
86+
```tsx
87+
import { useChat, lastAssistantMessageIsCompleteWithToolCalls } from "@ai-sdk/react";
88+
import { useTriggerChatTransport } from "@trigger.dev/react-hooks";
89+
90+
function ChatView({ chatId, accessToken }: { chatId: string; accessToken: string }) {
91+
const transport = useTriggerChatTransport({ task: "my-chat", accessToken });
92+
const { messages, sendMessage, addToolOutput } = useChat({
93+
id: chatId,
94+
transport,
95+
sendAutomaticallyWhen: lastAssistantMessageIsCompleteWithToolCalls,
96+
});
97+
98+
return (
99+
<>
100+
{messages.map((m) =>
101+
m.parts.map((part, i) => {
102+
if (part.type === "tool-askUser" && part.state === "input-available") {
103+
return (
104+
<AskUserCard
105+
key={i}
106+
question={part.input.question}
107+
options={part.input.options}
108+
onAnswer={(opt) =>
109+
addToolOutput({
110+
tool: "askUser",
111+
toolCallId: part.toolCallId,
112+
output: { optionId: opt.id, label: opt.label },
113+
})
114+
}
115+
/>
116+
);
117+
}
118+
if (part.type === "text") return <Markdown key={i}>{part.text}</Markdown>;
119+
return null;
120+
})
121+
)}
122+
</>
123+
);
124+
}
125+
```
126+
127+
`addToolOutput` patches the assistant message locally with `state: "output-available"` and fills in `output`. `lastAssistantMessageIsCompleteWithToolCalls` detects that every pending tool call now has a result, and `useChat` fires a new `sendMessage` — the backend picks it up as the next turn.
128+
129+
## Detecting a paused turn in `onTurnComplete`
130+
131+
Two ways to detect "this turn paused for user input" vs "this turn finished normally":
132+
133+
### Via `finishReason` (recommended)
134+
135+
The AI SDK's finish reason is surfaced on every `onTurnComplete` event. If the model stopped on tool calls, it's `"tool-calls"`:
136+
137+
```ts
138+
onTurnComplete: async ({ finishReason, responseMessage }) => {
139+
if (finishReason === "tool-calls") {
140+
// Turn paused — assistant message has pending tool call(s)
141+
const pending = responseMessage?.parts.filter(
142+
(p) => p.type.startsWith("tool-") && p.state === "input-available"
143+
);
144+
// Persist as a checkpoint / partial turn
145+
} else {
146+
// finishReason === "stop" — normal completion
147+
// Persist as a completed turn
148+
}
149+
};
150+
```
151+
152+
<Note>
153+
`finishReason` is only undefined for manual `chat.pipe()` flows or aborted streams. For the common `run() → return streamText(...)` pattern it's always populated.
154+
</Note>
155+
156+
### Via response parts
157+
158+
If you need more nuance (e.g. which specific tool is pending), inspect the parts directly:
159+
160+
```ts
161+
function pendingToolCalls(message: UIMessage): string[] {
162+
return message.parts
163+
.filter((p) => p.type.startsWith("tool-") && p.state === "input-available")
164+
.map((p) => p.toolCallId);
165+
}
166+
```
167+
168+
Both `finishReason === "tool-calls"` and `pendingToolCalls(responseMessage).length > 0` are equivalent in practice. Use `finishReason` for dispatch, parts for detail.
169+
170+
## Persistence: one message vs one record per pause
171+
172+
Because the AI SDK reuses the assistant message ID across the pause, the "same turn" from the user's perspective maps to **two `onTurnComplete` firings** on the server — but both receive a `responseMessage` with the **same `id`**, and the second firing's `responseMessage` contains the fully merged content.
173+
174+
Two common persistence patterns:
175+
176+
### Overwrite on every turn (simplest)
177+
178+
Just store the latest `uiMessages` array on every `onTurnComplete`. The paused-turn write is overwritten by the resume-turn write; the final DB state has the full merged message.
179+
180+
```ts
181+
onTurnComplete: async ({ chatId, uiMessages }) => {
182+
await db.chat.update({
183+
where: { id: chatId },
184+
data: { messages: uiMessages },
185+
});
186+
},
187+
```
188+
189+
Use this unless you specifically need an audit trail.
190+
191+
### Checkpoint nodes (immutable history)
192+
193+
For apps that want every pause point recorded as its own immutable snapshot (branching, replay, diff review), save a checkpoint when paused and a sibling when complete:
194+
195+
```ts
196+
onTurnComplete: async ({ chatId, responseMessage, finishReason, uiMessages }) => {
197+
if (!responseMessage) return;
198+
199+
if (finishReason === "tool-calls") {
200+
// Paused — save a checkpoint
201+
await db.turnCheckpoint.create({
202+
data: {
203+
chatId,
204+
messageId: responseMessage.id,
205+
parts: responseMessage.parts,
206+
kind: "partial",
207+
},
208+
});
209+
} else {
210+
// Completed — save a sibling with the merged full message
211+
await db.turnCheckpoint.create({
212+
data: {
213+
chatId,
214+
messageId: responseMessage.id,
215+
parts: responseMessage.parts,
216+
kind: "final",
217+
},
218+
});
219+
}
220+
221+
// Always update the canonical chat record for `hydrateMessages` to load
222+
await db.chat.update({
223+
where: { id: chatId },
224+
data: { messages: uiMessages },
225+
});
226+
};
227+
```
228+
229+
Both writes see `responseMessage.id` as the same value — they're checkpoints of the same logical message. Grouping by `messageId` + ordering by `createdAt` gives you the progression.
230+
231+
## Multi-pause turns
232+
233+
A single logical turn can pause more than once — the LLM asks question A, gets the answer, thinks, then asks question B before finishing. Each pause fires its own `onTurnComplete` with `finishReason === "tool-calls"`; only the last firing has `finishReason === "stop"`. The checkpoint pattern above handles this naturally — each pause adds a new checkpoint sharing the same `responseMessage.id`.
234+
235+
## Gotchas
236+
237+
- **Don't set an `execute` function on the HITL tool.** If it has one, `streamText` will call it immediately instead of handing control back.
238+
- **The frontend must use `sendAutomaticallyWhen`.** Without it, the user has to press Enter after answering — `addToolOutput` updates local state but doesn't fire a new turn by itself.
239+
- **Don't mutate `responseMessage` in `onTurnComplete`.** It's the captured snapshot. To add custom parts, use `chat.response.append()` in `onBeforeTurnComplete` (while the stream is open).
240+
- **Stop handling.** If the user stops the run while a pause is active (`chat.stop()` on the transport), `onTurnComplete` fires with `stopped: true` and `finishReason` reflecting the last successful step. Treat stopped paused turns the same as stopped normal turns.

docs/docs.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -108,7 +108,8 @@
108108
"ai-chat/patterns/version-upgrades",
109109
"ai-chat/patterns/database-persistence",
110110
"ai-chat/patterns/branching-conversations",
111-
"ai-chat/patterns/code-sandbox"
111+
"ai-chat/patterns/code-sandbox",
112+
"ai-chat/patterns/human-in-the-loop"
112113
]
113114
},
114115
"ai-chat/client-protocol",

0 commit comments

Comments
 (0)