Interrupt blocked queue puts during sync WebSocket teardown - #1123
Interrupt blocked queue puts during sync WebSocket teardown#1123Kludex wants to merge 2 commits into
Conversation
|
Docs preview: https://ad0c18c8-httpx2-docs.pydantic.workers.dev |
Merging this PR will not alter performance
Comparing Footnotes
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fd40776cf8
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| except queue.Full: | ||
| if self._should_close.is_set(): | ||
| raise ShouldClose() from None |
There was a problem hiding this comment.
Preserve terminal events when the peer closes
When the bounded queue is full for 100 ms as a peer CloseConnection arrives, _background_receive has already set _should_close, so this branch raises ShouldClose and silently discards the close event. After the caller drains the buffered messages, its next receive() therefore blocks indefinitely or times out instead of raising WebSocketDisconnect; the same loss affects queued network and oversized-message errors because those paths call close() before _put_event(). The interruption condition needs to distinguish local context teardown from closure initiated while delivering a terminal event.
Useful? React with 👍 / 👎.
PR overviewAll previously flagged issues have been addressed. No open security concerns remain on this pull request. Security reviewNo open security issues remain on this pull request. Fixed/addressed: 1 · PR risk: 0/10 |
There was a problem hiding this comment.
2 issues found across 2 files
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="tests/httpx2/websockets/test_api.py">
<violation number="1" location="tests/httpx2/websockets/test_api.py:928">
P3: The test is an anyio async test but uses blocking `time.sleep()` in its busy-wait loop and final sleep, which stalls the event loop instead of yielding. It works because the code under test runs on background threads, but consider using an async-aware sleep (e.g. `anyio.sleep` / `asyncio.sleep`) or driving the loop off-thread to keep the async test idiomatic and avoid blocking the loop for up to ~5s.</violation>
<violation number="2" location="tests/httpx2/websockets/test_api.py:930">
P3: This regression test catches a deadlock by hanging rather than by failing an assertion: on a regression the `with` block exit blocks forever and the test never finishes, so CI stalls instead of reporting a red build. Consider wrapping the exit in a timeout (e.g. run the teardown in a helper with a deadline, or rely on pytest-timeout) so a regression yields a clean failure, and/or drain the queue after the blocking scenario to keep the wall-clock cost tight.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| break | ||
| time.sleep(0.05) | ||
| assert ws._events.full() | ||
| time.sleep(0.3) |
There was a problem hiding this comment.
P3: This regression test catches a deadlock by hanging rather than by failing an assertion: on a regression the with block exit blocks forever and the test never finishes, so CI stalls instead of reporting a red build. Consider wrapping the exit in a timeout (e.g. run the teardown in a helper with a deadline, or rely on pytest-timeout) so a regression yields a clean failure, and/or drain the queue after the blocking scenario to keep the wall-clock cost tight.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/httpx2/websockets/test_api.py, line 930:
<comment>This regression test catches a deadlock by hanging rather than by failing an assertion: on a regression the `with` block exit blocks forever and the test never finishes, so CI stalls instead of reporting a red build. Consider wrapping the exit in a timeout (e.g. run the teardown in a helper with a deadline, or rely on pytest-timeout) so a regression yields a clean failure, and/or drain the queue after the blocking scenario to keep the wall-clock cost tight.</comment>
<file context>
@@ -903,6 +903,33 @@ def wait_for_session_threads(expected: int) -> None:
+ break
+ time.sleep(0.05)
+ assert ws._events.full()
+ time.sleep(0.3)
+
+
</file context>
| for _ in range(100): | ||
| if ws._events.full(): | ||
| break | ||
| time.sleep(0.05) |
There was a problem hiding this comment.
P3: The test is an anyio async test but uses blocking time.sleep() in its busy-wait loop and final sleep, which stalls the event loop instead of yielding. It works because the code under test runs on background threads, but consider using an async-aware sleep (e.g. anyio.sleep / asyncio.sleep) or driving the loop off-thread to keep the async test idiomatic and avoid blocking the loop for up to ~5s.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/httpx2/websockets/test_api.py, line 928:
<comment>The test is an anyio async test but uses blocking `time.sleep()` in its busy-wait loop and final sleep, which stalls the event loop instead of yielding. It works because the code under test runs on background threads, but consider using an async-aware sleep (e.g. `anyio.sleep` / `asyncio.sleep`) or driving the loop off-thread to keep the async test idiomatic and avoid blocking the loop for up to ~5s.</comment>
<file context>
@@ -903,6 +903,33 @@ def wait_for_session_threads(expected: int) -> None:
+ for _ in range(100):
+ if ws._events.full():
+ break
+ time.sleep(0.05)
+ assert ws._events.full()
+ time.sleep(0.3)
</file context>
There was a problem hiding this comment.
1 issue found across 2 files (changes from recent commits).
Prompt for AI agents (unresolved issues)
Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.
<file name="src/httpx2/httpx2/websockets/_api.py">
<violation number="1" location="src/httpx2/httpx2/websockets/_api.py:161">
P2: Teardown can still take minutes when the full queue blocks a receive batch containing many small frames, because each freed slot incurs another 10 ms join wait before the producer can reach its next `put()`. Drain immediately while items are available, and only use the timed join when the queue is empty.</violation>
</file>
Reply with feedback, questions, or to request a fix.
Re-trigger cubic
| while task.is_alive(): | ||
| with contextlib.suppress(queue.Empty): | ||
| self._events.get_nowait() | ||
| task.join(timeout=0.01) |
There was a problem hiding this comment.
P2: Teardown can still take minutes when the full queue blocks a receive batch containing many small frames, because each freed slot incurs another 10 ms join wait before the producer can reach its next put(). Drain immediately while items are available, and only use the timed join when the queue is empty.
Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/httpx2/httpx2/websockets/_api.py, line 161:
<comment>Teardown can still take minutes when the full queue blocks a receive batch containing many small frames, because each freed slot incurs another 10 ms join wait before the producer can reach its next `put()`. Drain immediately while items are available, and only use the timed join when the queue is empty.</comment>
<file context>
@@ -149,9 +149,16 @@ def __exit__(
+ while task.is_alive():
+ with contextlib.suppress(queue.Empty):
+ self._events.get_nowait()
+ task.join(timeout=0.01)
def ping(self, payload: bytes = b"") -> threading.Event:
</file context>
| self._background_keepalive_ping_task.join() | ||
| self._join_discarding_events(self._background_keepalive_ping_task) | ||
|
|
||
| def _join_discarding_events(self, task: threading.Thread) -> None: |
There was a problem hiding this comment.
Can we add a check for sys.version with a call queue.shutdown(immediate=True)? This doesn't need requires-python bump
When the sync session's bounded events queue is full, the background receive thread blocks indefinitely on
queue.Queue.put(). Sinceclose()never drains the queue, the.join()in__exit__then hangs forever, so exiting thewithblock deadlocks whenever more thanqueue_sizeunread messages are pending.The fix is confined to teardown:
__exit__now drains the queue while joining the background threads, which unblocks any pendingput()and lets them exit. Everything else - backpressure, event ordering,receive()semantics - is unchanged, and close/error events are still delivered in order behind buffered messages. The async session is unaffected since its cancel scope already cancels pending sends.test_exit_with_full_queuereproduces the hang onmain, andtest_close_delivered_after_draining_full_queuechecks the close event is still delivered after draining a queue that had filled up.Once
requires-pythonreaches 3.13, the drain helper can be replaced byQueue.shutdown(immediate=True).AI Disclaimer
This PR was developed with the assistance of either Claude or Codex. I've reviewed and verified the changes.