Skip to content

feat(embedded-agent): retry once on a transient no-output result - #1308

Closed
betegon wants to merge 1 commit into
mainfrom
feat/embedded-agent-retry-no-output
Closed

betegon wants to merge 1 commit into
mainfrom
feat/embedded-agent-retry-no-output

Conversation

@betegon

@betegon betegon commented Sep 14, 2026

Copy link
Copy Markdown
Member

Problem

The embedded translation agent (used by search_events, search_issues, search_issue_events) is stochastic. It occasionally exhausts its step budget without emitting structured output, which callEmbeddedAgent turns into a UserInputError — "could not construct a valid query". An identical retry almost always succeeds.

This was observed live in prod right after the time-series feature (#1305) shipped: unique users affected per hour failed on the first attempt with that exact error, then returned a correct series on an identical retry. So the query was fine — the model just missed once.

What this does

Retry the agent once on the transient no-output failure before surfacing it:

  • Retries only on NoOutputGeneratedError and unrescued NoObjectGeneratedError (the "model gave us nothing usable this time" cases).
  • Everything else — provider outages, quota/rate-limit, config errors, agent-signalled bad input, unexpected errors — is mapped and thrown immediately, exactly as before.
  • Only the failing tail pays the extra call (capped at 2 attempts); successful translations are unaffected.

Benefits every embedded-agent tool, not just time series.

How

callEmbeddedAgent is split into three pieces so the retry policy is explicit and the attempt body is untouched:

  • callEmbeddedAgent — the retry loop.
  • runEmbeddedAgentAttempt — one attempt: translate, capture tool calls, parse output, and rescue NoObjectGeneratedError (per attempt). Throws raw on any other failure.
  • mapEmbeddedAgentError — maps a terminal (post-retry) failure to the typed error handlers expect. Behaviour and messages are unchanged from the old catch block.

Test plan

  • pnpm --filter @sentry/mcp-core tsc — clean; biome lint/format — clean; full mcp-core suite — 1421 passed.
  • New unit test: first attempt NoOutputGeneratedError → second attempt succeeds → result returned, generateText called twice, retry logged, no Sentry issue filed.
  • Existing callEmbeddedAgent tests (provider errors, rescue, give-up-with-UserInputError, config errors) still pass unchanged.

Notes

  • This is the "retry" half of the make-it-work tail we discussed. The other idea — routing no-output to the agent-less withProviderFallback fallback — was not done here: that fallback has no time-series path and would not have recovered the query above.

🤖 Generated with Claude Code

The embedded translation agent (search_events, search_issues,
search_issue_events) is stochastic: it occasionally exhausts its step
budget without emitting structured output, which surfaces as a
UserInputError ("could not construct a valid query"). An identical retry
almost always succeeds — observed live in prod, where a
count_unique-per-hour timeseries query failed on the first attempt and
worked on an identical second one.

Retry once on NoOutputGeneratedError / unrescued NoObjectGeneratedError
before mapping to a user-facing error. callEmbeddedAgent is split into a
retry loop, a single-attempt runner (runEmbeddedAgentAttempt), and the
terminal error mapper (mapEmbeddedAgentError) so the retry policy is
explicit and the attempt body is unchanged. Benefits every embedded-agent
tool, and only the failing tail pays the extra call (capped at 2).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@betegon

betegon commented Sep 14, 2026

Copy link
Copy Markdown
Member Author

Superseded by #1309. Switching from a blind full retry to a framework-native final-output turn (prepareStep + toolChoice:'none' on the closing step), which keeps the agent's context and addresses the root cause (the model never committing to output before its step budget runs out) rather than re-running from scratch.

@betegon betegon closed this Sep 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant