Skip to content

fix: apply SSE timeout to HTTP client and fix mutable default argument in llms.py#25

Open
amathxbt wants to merge 2 commits into
nesaorg:mainfrom
amathxbt:fix/sse-timeout-not-applied-to-http-client
Open

fix: apply SSE timeout to HTTP client and fix mutable default argument in llms.py#25
amathxbt wants to merge 2 commits into
nesaorg:mainfrom
amathxbt:fix/sse-timeout-not-applied-to-http-client

Conversation

@amathxbt
Copy link
Copy Markdown

@amathxbt amathxbt commented May 9, 2026

Bugs

Two bugs in demo/nesa/backend/llms.py:

Bug 1 — SSE timeout silently ignored

sse_message_handler accepts a timeout parameter (default 60s) but passes timeout=None to the httpx client:

# Before (broken) — connection can hang forever
async with client.stream(
    "POST", ..., timeout=None  # ignores the timeout parameter!
) as response:

Impact: SSE connections to the inference backend can hang indefinitely regardless of the timeout argument. This blocks the event loop and can deadlock the server under network failures.

Bug 2 — Mutable default argument

perform_inference uses a mutable list as a default value:

# Before (broken)
history: Optional[List[str]] = [],  # shared across all calls!

Impact: Python creates this list object once at function-definition time. If any caller mutates history in place, the mutation persists across subsequent calls — a classic Python footgun that can cause cross-session state corruption.

Fix

  1. Pass the actual timeout argument to the httpx stream call.
  2. Changed default to None with an or [] guard at the call site.
# After (fixed)
async with client.stream("POST", ..., timeout=timeout) as response:

# and
history: Optional[List[str]] = None,
...
prompt_template = generate_prompt_template(..., history=history or [])

amathxbt added 2 commits May 9, 2026 19:43
Two bugs in llms.py:

1. sse_message_handler accepted a 'timeout' parameter but passed
   timeout=None to the HTTP client, making it hang indefinitely
   regardless of the timeout value. Now passes the actual timeout.

2. perform_inference used a mutable list [] as the default value for
   'history'. Python creates this list once at function definition time,
   so mutations across calls can leak state between sessions.
   Changed default to None and uses 'history or []' at call site.
Two bugs fixed in llms.py:

1. sse_message_handler accepted a timeout parameter but passed
   timeout=None to httpx, so connections could hang forever regardless
   of the timeout value. Now correctly passes the timeout argument.

2. perform_inference used history=[] as a default argument. Python
   creates the list object once at definition time, so state can leak
   between calls if the list is ever mutated. Changed to None with
   an "or []" guard at the call site.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant