Skip to content

feat(embedded-agent): force a final structured-output turn instead of ending empty - #1309

Draft
betegon wants to merge 1 commit into
mainfrom
feat/embedded-agent-final-output-step
Draft

betegon wants to merge 1 commit into
mainfrom
feat/embedded-agent-final-output-step

Conversation

@betegon

@betegon betegon commented Sep 14, 2026

Copy link
Copy Markdown
Member

Problem

The embedded translation agent (search_events, search_issues, search_issue_events) runs the AI SDK's own multi-step tool loop (stepCountIs(5)). Its most common failure is spending that whole budget calling tools (fetch attributes, validate, …) and never committing to the structured output — it isn't "aware" it's about to run out of turns. The run ends empty, which surfaces to users as UserInputError ("could not construct a valid query"). Seen live in prod (e.g. a count_unique per hour query that failed once, then worked on an identical rerun).

What this does

Make the agent's last step answer-only, using the SDK-native prepareStep callback:

  • prepareStep is a first-class generateText option the SDK calls before each step of its loop; it lets you override that step's settings (toolChoice, messages, …).
  • On the final step (stepNumber === AGENT_STEP_LIMIT - 1) we return toolChoice: "none" and append a short "this is your final step, respond now" message.

With tools disabled, answering is the model's only move — so instead of ending empty it responds with what it already gathered, within the same run, keeping all its context. No blind full retry, no doubled latency. If the forced turn still yields nothing, the existing UserInputError stands.

Why not a retry

An earlier cut (#1308, closed) retried the whole generateText call. That throws away the context the agent built and doesn't address the root cause — the model never committing before the budget runs out. Forcing the final turn is the framework-native way to fix it.

Notes on the trade-off

The last of the 5 steps becomes answer-only, so the agent gets up to 4 tool-calling steps + 1 guaranteed answer, instead of 5 tool steps that could end without an answer. In practice a translation uses ~2–3 tool calls, so this only ever converts a would-be-wasted final tool call into an answer. Happy path is unaffected — the forced step is only reached if every prior step was a tool call.

Test plan

  • pnpm --filter @sentry/mcp-core tsc / biome lint / format — clean; full mcp-core suite — 1420 passed.
  • The forced turn only fires on the rare no-output path, which is stochastic and can't be reproduced on demand, so there's no dedicated unit test for it (a mocked generateText never runs the real step loop). The existing callEmbeddedAgent tests still pass, confirming the wiring and the unchanged error paths.

🤖 Generated with Claude Code

… ending empty

The embedded translation agent (search_events, search_issues,
search_issue_events) runs the AI SDK's multi-step tool loop. Its top
no-output failure is the model spending every step calling tools (fetch
attributes, validate, …) and never committing to the structured output —
it isn't "aware" it's about to run out of turns. That surfaces to users as
a UserInputError ("could not construct a valid query").

Use the SDK-native prepareStep to make the agent's last step answer-only:
disable tools (toolChoice: "none") and tell it to emit the output now, so
it answers with what it already gathered within the same run — no full
retry, no lost context. If that final turn still yields nothing, the
existing UserInputError stands.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@betegon
betegon force-pushed the feat/embedded-agent-final-output-step branch from 1b21667 to ad054b4 Compare September 14, 2026 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant