Skip to content

Commit 1fe3166

Browse files
committed
docs(ai-chat): warn against chat.defer for onTurnStart message persistence
`chat.defer(db.chat.update(...))` in `onTurnStart` is fire-and-forget — the hook resolves and streaming begins before the write lands. A mid-stream page refresh then reads `[]` from the DB, the resumed SSE stream pushes the assistant into an empty array, and the user's message disappears from the rendered conversation. - patterns/database-persistence.mdx: replace the misleading "optionally use chat.defer" line with an awaited persistence + a Warning showing wrong/right examples and the failure mode. Update the minimal pseudocode to use await. - features.mdx (chat.defer reference): swap the misleading example (db.chat.update inside onTurnStart) for an analytics-tracking example. Add a Warning cross-linking back to the persistence doc. Reserve chat.defer for writes whose timing has no resume implication.
1 parent 1a8b08e commit 1fe3166

2 files changed

Lines changed: 29 additions & 7 deletions

File tree

docs/ai-chat/features.mdx

Lines changed: 8 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -162,14 +162,14 @@ onTurnComplete: async ({ chatId }) => {
162162

163163
Use `chat.defer()` to run background work in parallel with streaming. The deferred promise runs alongside the LLM response and is awaited (with a 5s timeout) before `onTurnComplete` fires.
164164

165-
This moves non-blocking work (DB writes, analytics, etc.) out of the critical path:
165+
This moves non-blocking work (analytics, audit logs, search-index writes, cache warming) out of the critical path:
166166

167167
```ts
168168
export const myChat = chat.agent({
169169
id: "my-chat",
170-
onTurnStart: async ({ chatId, uiMessages }) => {
171-
// Persist messages without blocking the LLM call
172-
chat.defer(db.chat.update({ where: { id: chatId }, data: { messages: uiMessages } }));
170+
onTurnStart: async ({ chatId, runId }) => {
171+
// Analytics — fire-and-forget, irrelevant to resume.
172+
chat.defer(analytics.track("turn_started", { chatId, runId }));
173173
},
174174
run: async ({ messages, signal }) => {
175175
return streamText({ model: openai("gpt-4o"), messages, abortSignal: signal });
@@ -179,6 +179,10 @@ export const myChat = chat.agent({
179179

180180
`chat.defer()` can be called from anywhere during a turn — hooks, `run()`, or nested helpers. All deferred promises are collected and awaited together before `onTurnComplete`.
181181

182+
<Warning>
183+
**Don't use `chat.defer()` for the message-history write in `onTurnStart`.** That write must land *before* the model starts streaming, otherwise a mid-stream page refresh will read `[]` from your DB and lose the user's message from the rendered conversation. See [Database persistence — `onTurnStart`](/ai-chat/patterns/database-persistence#onturnstart). Reserve `chat.defer` for writes whose timing has no resume implication.
184+
</Warning>
185+
182186
---
183187

184188
## Custom data parts

docs/ai-chat/patterns/database-persistence.mdx

Lines changed: 21 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -48,8 +48,25 @@ If you skip preload, do the equivalent in **`onChatStart`** when **`preloaded`**
4848

4949
### `onTurnStart`
5050

51-
- Persist **`uiMessages`** (full accumulated history including the new user turn) **before** streaming starts — so a mid-stream refresh still shows the user’s message.
52-
- Optionally use [`chat.defer()`](/ai-chat/features#chat-defer) so the write does not block the model if your driver is slow.
51+
- **`await`** persist **`uiMessages`** (full accumulated history including the new user turn) **before** the hook returns — `chat.agent` does not begin streaming until `onTurnStart` resolves, so this is what bounds "user message is durable before the stream".
52+
53+
<Warning>
54+
**Don't use [`chat.defer()`](/ai-chat/features#chat-defer) for the message write here.** `chat.defer` is fire-and-forget — the hook resolves before the write lands and the stream starts immediately. If the user refreshes mid-stream, the next page load reads `[]` from your DB, the resumed SSE stream pushes the assistant into an empty array, and the user's message disappears from the rendered conversation forever.
55+
56+
```ts
57+
// ❌ Bad — non-blocking write, mid-stream refresh drops the user message.
58+
onTurnStart: async ({ chatId, uiMessages }) => {
59+
chat.defer(db.chat.update({ where: { id: chatId }, data: { messages: uiMessages } }));
60+
},
61+
62+
// ✅ Good — awaited, durable before the model starts.
63+
onTurnStart: async ({ chatId, uiMessages }) => {
64+
await db.chat.update({ where: { id: chatId }, data: { messages: uiMessages } });
65+
},
66+
```
67+
68+
`chat.defer` is for writes whose timing doesn't matter for resume — analytics, audit logs, search-index updates, etc. Anything the next page load reads needs to land before the stream begins.
69+
</Warning>
5370

5471
### `onTurnComplete`
5572

@@ -128,7 +145,8 @@ chat.agent({
128145
},
129146

130147
onTurnStart: async ({ chatId, uiMessages }) => {
131-
chat.defer(saveConversationMessages(chatId, uiMessages));
148+
// Awaited, not chat.defer — see the warning in `onTurnStart` above.
149+
await saveConversationMessages(chatId, uiMessages);
132150
},
133151

134152
onTurnComplete: async ({ chatId, uiMessages, chatAccessToken, lastEventId }) => {

0 commit comments

Comments
 (0)