Skip to content

feat(agent): stream watchdog core and typed stream errors (DEV-723 1/3) - #136

Draft
LukasParke wants to merge 2 commits into
mainfrom
feat/dev-723-stream-watchdog-core
Draft

LukasParke wants to merge 2 commits into
mainfrom
feat/dev-723-stream-watchdog-core

Conversation

@LukasParke

@LukasParke LukasParke commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Stack 1/3: stream watchdog core and typed stream errors

Linear: DEV-723. This ports the stall-watchdog work from typescript-sdk's legacy callModel into the agent engine, which is the source of truth.

Stack: this PR (ports OpenRouterTeam/typescript-sdk#770 and OpenRouterTeam/typescript-sdk#774) → #137 streamTimeout option (ports OpenRouterTeam/typescript-sdk#773) → #138 stall retries (ports OpenRouterTeam/typescript-sdk#775). OpenRouterTeam/typescript-sdk#776 (public raw-stream helpers) is deliberately not ported. It belongs in the SDK.

What this PR changes

stream-errors.ts

  • StreamStalledError carries phase (first_content | between_content), timeoutMs, elapsedMs and receivedAnyContent. Its retryable getter is true only when no content had arrived.
  • StreamFailedError carries kind (response_failed | error_event | truncated), code, errorType, response and a retryable hint. retryable is true for server_error, server, timeout, rate_limit_exceeded, provider_overloaded and provider_unavailable.
  • kind is new compared with the SDK version. It replaces the bare Error('Stream ended without completion event') with a typed error that callers can still tell apart.
  • Both errors are exported from the package root.

stream-watchdog.ts

  • applyStreamWatchdog / applyResponsesStreamWatchdog enforce two deadlines on the parsed event stream:
    • firstContentMs: from stream start to the first content-bearing event.
    • contentIntervalMs: the maximum gap between content events.
  • On expiry the wrapped stream errors and the source is cancelled.
  • Nothing in callModel uses it yet. PR 2 wires it in.
  • Changes from the SDK version, based on its review threads:
    • The error event is neutral, not terminal. Devin and Perry both flagged that a terminal error disarms the watchdog. A stream that sends error and then goes quiet would hang forever, even though both consume paths keep waiting after an error event.
    • response.fusion_call.analysis.completed counts as content. Perry flagged that otherwise a fusion stream with only an analysis result would raise a false stall.
    • It accepts the request's signal. An abort disarms the watchdog, so a user abort or timeoutMs is never reported as a stall.

The same errors on every consume path (from OpenRouterTeam/typescript-sdk#774)

  • These paths now throw StreamFailedError:
    • consumeStreamForCompletion
    • extractCompletionFromBuffer
    • the initial-stream capture (captureInitialStreamEvent)
    • the follow-up pipe (pipeAndConsumeStream)
  • A stream-level error event is recorded rather than thrown. It surfaces only when the stream ends without completing. An error followed by response.completed still resolves.
  • Previously, pipeAndConsumeStream threw on response.failed with a meaningless message, and consumeStreamForCompletion ignored error events entirely.

Stream getters surface failures

  • Without tools, getTextStream, getFullResponsesStream, getItemsStream, getReasoningStream, getToolStream and getToolCallsStream iterate the initial stream directly. They used to end silently on a failed or truncated response.
  • They now reject with the same error getText() throws.
  • cancel() and an early break are not treated as failures.
  • With tools, the getters already rejected through executionPromise. They now reject with the typed error.

Structural gate

  • model-result.ts was already at the gate's fan-out ceiling of 15 edges. The new modules reach it through a stream-runtime.ts re-export façade that also carries ReusableReadableStream, the same pattern as async-tools.ts.
  • The watchdog's state lives in a small class so every function stays under max_cc 15.

Behavior changes (minor)

  • The thrown type changes from Error to StreamFailedError, which is an Error subclass. Message text changed for response.failed, for example Response failed (server_error): ….
  • No-tools stream getters now reject on failed or truncated streams instead of completing. One existing test (hooks-post-model-call) asserted the silent completion. It now asserts the typed rejection.

Tests

  • stream-watchdog.test.ts (22 tests) uses fake timers throughout. It covers:
    • the keep-alive pathology (created + role prelude, then silence)
    • heartbeats that do not reset a deadline
    • per-content re-arm, and contentIntervalMs on its own
    • terminal disarm, error-event stays armed, abort disarms
    • source cancellation and onStall firing exactly once
    • no timers left behind
  • stream-failed-error.test.ts (58 tests) runs every getter against response.failed, error-then-close and truncated streams. It covers no-tools runs, the initial turn with tools, and the follow-up turn with tools. It also covers error-then-completion resolving, active-consumers replay keeping the error event, and cancel/break not counting as failures.

Port notes (Python / Go bots)

Carry:

  • both error types, including the kind field and the transient-code set
  • the event classification tables: the content set, and the terminal set, which deliberately excludes error
  • the watchdog semantics:
    • eager pump
    • deadlines measured on parsed events, not transport bytes
    • abort disarms the watchdog
    • a stall cancels the source
  • the "record error events, surface them only at end of stream" rule, on every consume path
  • the rule that iterators must reject, not end, on a failed or truncated stream

The ReusableReadableStream / onValue plumbing is TypeScript-specific.

Verification

pnpm install --frozen-lockfile, pnpm typecheck, pnpm lint, pnpm test (agent 1316, mcp 12, coverage ratchet holds), pnpm build, pnpm verify:packages, and sentrux gate (no degradation). CI is green.

Once this stack lands, OpenRouterTeam/typescript-sdk#770, OpenRouterTeam/typescript-sdk#773, OpenRouterTeam/typescript-sdk#774 and OpenRouterTeam/typescript-sdk#775 can be closed. OpenRouterTeam/typescript-sdk#776 stays with the SDK.

🤖 Generated with Claude Code

Port of OpenRouterTeam/typescript-sdk#770 and #774.

- stream-errors.ts: StreamStalledError (phase, timeoutMs, elapsedMs,
  receivedAnyContent, retryable) and StreamFailedError (kind, code,
  errorType, response, retryable).
- stream-watchdog.ts: applyStreamWatchdog / applyResponsesStreamWatchdog
  with firstContentMs and contentIntervalMs deadlines, classified by
  parsed event type. Differs from the SDK version per its review threads:
  `error` events are neutral rather than terminal (an error followed by
  silence must still stall), `response.fusion_call.analysis.completed`
  counts as content, and the request's abort signal disarms the watchdog
  so a user abort is never reported as a stall.
- response.failed, stream-level error events and truncated streams throw
  StreamFailedError on every consume path: consumeStreamForCompletion,
  extractCompletionFromBuffer, the initial-stream capture and the
  follow-up pipe. Error events are recorded and surfaced only when the
  stream ends without completing.
- No-tools stream getters re-surface initial-stream failures after their
  consumer ends, instead of ending silently on a failed or truncated
  response.

Not wired into callModel yet: that lands with the `timeout` option.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
- Split applyStreamWatchdog into a small StreamWatchdog state class
  (cc 17 -> under the max_cc 15 ceiling). No behavior change.
- Route ReusableReadableStream and StreamFailedError through a
  stream-runtime.ts re-export façade, following the async-tools.ts
  pattern. model-result.ts stays at 15 outbound edges, the gate's
  no_god_files ceiling.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant