fix(eve): classify gateway stream-assembler desync as terminal, not recoverable (#1227) - #1439
Open
iroiro147 wants to merge 1 commit into
Open
fix(eve): classify gateway stream-assembler desync as terminal, not recoverable (#1227)#1439iroiro147 wants to merge 1 commit into
iroiro147 wants to merge 1 commit into
Conversation
…ecoverable (vercel#1227) The AI stream assembler's desync (`text part <id> not found`, raised from consumeStreamContent when an interleaved provider stream references a part the assembler no longer tracks — observed with deepseek/deepseek-v4-flash reasoning parts via the AI Gateway) fell through classifyModelCallError's recoverable catch-all. eve then parked the session for a durable step retry that re-ran the entire tool loop from the start: production measured 15-70 minute silent replay loops and 50-69M replayed cache-read tokens per run. A malformed stream frame is deterministic — replaying the turn re-emits the same broken sequence, and there is no partial model state to recover. Add a dedicated isStreamAssemblerDesyncError matcher (message-prefix + cause-chain walk) and return "terminal" so the step fails fast instead of replaying. Signed-off-by: iroiro147 <sarthak.singh@juspay.in>
Contributor
|
@iroiro147 is attempting to deploy a commit to the Vercel Team on Vercel. A member of the Team first needs to authorize it. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #1227.
Problem
When a task-mode (declared) subagent streams through the AI Gateway, an interleaved provider stream (observed with
deepseek/deepseek-v4-flashreasoning parts) intermittently desyncs the AI SDK stream assembler, throwingError: text part <id> not foundfromconsumeStreamContent. That error carried no gateway signals, no status code, and no SDKisRetryableflag, so it fell throughclassifyModelCallError's final branch and was classified recoverable.In task mode eve cannot park for user-driven recovery, so recoverable model-call failures are rethrown for Workflow's durable step retry — which replays the tool loop from the last committed snapshot. Because the desync is deterministic (the provider emits the same malformed frame sequence on replay), the retry re-ran the entire turn repeatedly: production measured single coder turns as 15–70 minute silent replay loops and parent runs of 50–69M replayed cache-read tokens (~5M for clean runs).
Change
classifyModelCallErrornow detects the stream-assembler desync via a dedicatedisStreamAssemblerDesyncErrormatcher and returns "terminal" so the step fails fast instead of parking for an unbounded durable replay.text part <hex-id> not foundmessage shape (the SDK throws a plainErrorhere; the part id is a per-stream hex token, so we match the message prefix) and walks the cause chain, mirroring the existingisNoOutputGeneratedErrorpredicate.isRetryableduck-typing and the transient/catalog branches, so a desync that the SDK also marks retryable still resolvesterminal.Why this is the right classification
Replaying cannot fix a broken stream frame and there is no partial model state to preserve — a desynced assembler has already lost the parts it needs. The failure is neither transient (nothing to wait out) nor a fixable configuration mistake in the usual sense; it is a deterministic malformed response. Failing the step surfaces the real error immediately and lets the operator switch models (the issue's own workaround: moving subagents off
deepseek/deepseek-v4-flash), rather than silently burning a turn's compute on a loop that cannot converge.Non-goals (deliberately out of scope, tracked separately):
vercel/ai), not eve's tool loop; eve can only classify.Docs
docs/subagents.mdx→ the retry/recovery paragraph now states that a malformed-provider stream desync is terminal, not recoverable, so eve fails the step instead of replaying the whole tool loop.Validation
model-call-error.test.ts: desync classified terminal even when markedisRetryable; a bare desync (no gateway/status/retryable signals — the exact production shape) is terminal, not the recoverable catch-all;isStreamAssemblerDesyncErrormatches direct + cause-chain shapes and rejects lookalikes. 40/40 unit pass.tsc -p tsconfig.json --noEmitclean;oxlintclean on both source files;oxfmt --checkclean.packages/eve/.changeset/gateway-stream-desync-terminal-1227.md(patch).