fix(models): split first-event and idle stream timeouts

[devin-ai-integration]

Summary

A provider that accepts a streaming request and then emits nothing is indistinguishable, to _with_idle_timeout, from a stream that died mid-generation — both are "no event for LLM_STREAM_IDLE_TIMEOUT". That conflation is expensive in both directions: the idle budget has to stay generous enough to cover a slow prefill (large prompt, cold/queued upstream, reasoning before the first token), which makes real mid-stream hangs slow to detect, and a route that is silent on arrival gets the identical request replayed as a generic transient error — 5 attempts, each paying a full timeout window plus backoff, before the agent finally crashes.

Two independent knobs now:

# was: one window for every gap
event = await asyncio.wait_for(iterator.__anext__(), timeout)

# now: the wait for the FIRST event has its own budget
event_timeout = timeout if yielded_event else effective_first_event_timeout

LLM_STREAM_FIRST_EVENT_TIMEOUT defaults to 0, meaning "inherit LLM_STREAM_IDLE_TIMEOUT", so an unconfigured deployment behaves exactly as before. Operators wanting the useful split set a tight idle budget and a generous first-event one:

LLM_STREAM_IDLE_TIMEOUT=30           # a mid-stream gap is almost always a real fault
LLM_STREAM_FIRST_EVENT_TIMEOUT=300   # prefill / queueing / pre-first-token reasoning is not

The messages stay distinguishable in logs and in the raised error (produced no first event within Xs vs. the unchanged produced no event for Xs), and both guards are still neutralised under LLM_DISABLE_STREAMING, where the wrapper's single event only arrives once the request is complete.

The retry side changes behaviour by default, deliberately. The guard now raises ModelStreamTimeoutError (a TimeoutError subclass, so any other consumer's classification is unaffected) and _run_cycle gives it its own consecutive-retry cap of 2, counted separately from _MAX_TRANSIENT_MODEL_RETRIES = 5 and reset by any other transient error. Retrying a stream timeout is not like retrying a 429: each attempt costs a full window, and a route that went silent once usually goes silent again — so a chronically silent upstream now surfaces after ~2 windows instead of burning 5.

Prior art for the split: openclaw#65898 (firstTokenTimeoutSeconds alongside the idle window), hermes-agent#32042 (TTFB watchdog while zero events have arrived), livekit/agents#6173 (bounding time-to-first-chunk so failover can happen); hermes-agent#7069 is the retry half of the problem — timeout mid-prefill, immediate replay, prefill restarts, never enough time to finish.

Link to Devin session: https://app.devin.ai/sessions/5411e06b88bc4f52ac4330aa8d2a1ebe

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment, CI, and merge conflict monitoring

[greptile-apps]

Greptile Summary

This PR separates the model stream’s first-event timeout from its subsequent idle timeout and gives stream timeouts an independent retry limit.

• Adds LLM_STREAM_FIRST_EVENT_TIMEOUT, inheriting the idle timeout when set to zero.
• Raises a dedicated ModelStreamTimeoutError with distinct first-event and idle-gap messages.
• Limits consecutive stream-timeout retries independently from other transient provider failures.
• Adds coverage for timeout selection, configuration propagation, disabled streaming, and retry exhaustion.

Confidence Score: 5/5

The PR appears safe to merge, with no concrete blocking or independently actionable non-blocking issues identified.

The timeout selection, configuration propagation, provider routing, cleanup behavior, and independent retry accounting are internally consistent with the documented behavior and covered by focused tests.

Important Files Changed

Filename	Overview
strix/config/models.py	Adds the dedicated timeout exception, splits first-event and idle-event watchdogs, and propagates both settings through provider construction.
strix/config/settings.py	Adds a validated first-event timeout setting whose zero default preserves the previous idle-timeout behavior.
strix/core/execution.py	Introduces separate consecutive retry accounting and a lower retry cap for model stream timeouts.
tests/test_execution_transient_retry.py	Verifies that repeated stream timeouts stop after the dedicated retry limit.
tests/test_stream_idle_timeout.py	Covers first-event timeout behavior, later idle gaps, configuration propagation, and timeout suppression when streaming is disabled.

_{Reviews (1): Last reviewed commit: "fix(models): split first-event and idle ..." | Re-trigger Greptile}