Skip to content

Interrupt blocked queue puts during sync WebSocket teardown - #1123

Open
Kludex wants to merge 2 commits into
mainfrom
ws-sync-teardown-full-queue
Open

Interrupt blocked queue puts during sync WebSocket teardown#1123
Kludex wants to merge 2 commits into
mainfrom
ws-sync-teardown-full-queue

Conversation

@Kludex

@Kludex Kludex commented Aug 7, 2026

Copy link
Copy Markdown
Member

When the sync session's bounded events queue is full, the background receive thread blocks indefinitely on queue.Queue.put(). Since close() never drains the queue, the .join() in __exit__ then hangs forever, so exiting the with block deadlocks whenever more than queue_size unread messages are pending.

The fix is confined to teardown: __exit__ now drains the queue while joining the background threads, which unblocks any pending put() and lets them exit. Everything else - backpressure, event ordering, receive() semantics - is unchanged, and close/error events are still delivered in order behind buffered messages. The async session is unaffected since its cancel scope already cancels pending sends.

test_exit_with_full_queue reproduces the hang on main, and test_close_delivered_after_draining_full_queue checks the close event is still delivered after draining a queue that had filled up.

Once requires-python reaches 3.13, the drain helper can be replaced by Queue.shutdown(immediate=True).

AI Disclaimer

This PR was developed with the assistance of either Claude or Codex. I've reviewed and verified the changes.

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown

Comment thread src/httpx2/httpx2/websockets/_api.py Outdated
@codspeed-hq

codspeed-hq Bot commented Aug 7, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 16 untouched benchmarks
⏩ 7 skipped benchmarks1


Comparing ws-sync-teardown-full-queue (fb68d03) with main (dec24ad)

Open in CodSpeed

Footnotes

  1. 7 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: fd40776cf8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/httpx2/httpx2/websockets/_api.py Outdated
Comment on lines +495 to +497
except queue.Full:
if self._should_close.is_set():
raise ShouldClose() from None

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve terminal events when the peer closes

When the bounded queue is full for 100 ms as a peer CloseConnection arrives, _background_receive has already set _should_close, so this branch raises ShouldClose and silently discards the close event. After the caller drains the buffered messages, its next receive() therefore blocks indefinitely or times out instead of raising WebSocketDisconnect; the same loss affects queued network and oversized-message errors because those paths call close() before _put_event(). The interruption condition needs to distinguish local context teardown from closure initiated while delivering a terminal event.

Useful? React with 👍 / 👎.

Comment thread src/httpx2/httpx2/websockets/_api.py Outdated
@veria-ai

veria-ai Bot commented Aug 7, 2026

Copy link
Copy Markdown

PR overview

All previously flagged issues have been addressed. No open security concerns remain on this pull request.

Security review

No open security issues remain on this pull request.

Fixed/addressed: 1 · PR risk: 0/10

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2 issues found across 2 files

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="tests/httpx2/websockets/test_api.py">

<violation number="1" location="tests/httpx2/websockets/test_api.py:928">
P3: The test is an anyio async test but uses blocking `time.sleep()` in its busy-wait loop and final sleep, which stalls the event loop instead of yielding. It works because the code under test runs on background threads, but consider using an async-aware sleep (e.g. `anyio.sleep` / `asyncio.sleep`) or driving the loop off-thread to keep the async test idiomatic and avoid blocking the loop for up to ~5s.</violation>

<violation number="2" location="tests/httpx2/websockets/test_api.py:930">
P3: This regression test catches a deadlock by hanging rather than by failing an assertion: on a regression the `with` block exit blocks forever and the test never finishes, so CI stalls instead of reporting a red build. Consider wrapping the exit in a timeout (e.g. run the teardown in a helper with a deadline, or rely on pytest-timeout) so a regression yields a clean failure, and/or drain the queue after the blocking scenario to keep the wall-clock cost tight.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

Comment thread src/httpx2/httpx2/websockets/_api.py Outdated
Comment thread tests/httpx2/websockets/test_api.py Outdated
break
time.sleep(0.05)
assert ws._events.full()
time.sleep(0.3)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: This regression test catches a deadlock by hanging rather than by failing an assertion: on a regression the with block exit blocks forever and the test never finishes, so CI stalls instead of reporting a red build. Consider wrapping the exit in a timeout (e.g. run the teardown in a helper with a deadline, or rely on pytest-timeout) so a regression yields a clean failure, and/or drain the queue after the blocking scenario to keep the wall-clock cost tight.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/httpx2/websockets/test_api.py, line 930:

<comment>This regression test catches a deadlock by hanging rather than by failing an assertion: on a regression the `with` block exit blocks forever and the test never finishes, so CI stalls instead of reporting a red build. Consider wrapping the exit in a timeout (e.g. run the teardown in a helper with a deadline, or rely on pytest-timeout) so a regression yields a clean failure, and/or drain the queue after the blocking scenario to keep the wall-clock cost tight.</comment>

<file context>
@@ -903,6 +903,33 @@ def wait_for_session_threads(expected: int) -> None:
+                        break
+                    time.sleep(0.05)
+                assert ws._events.full()
+                time.sleep(0.3)
+
+
</file context>

for _ in range(100):
if ws._events.full():
break
time.sleep(0.05)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P3: The test is an anyio async test but uses blocking time.sleep() in its busy-wait loop and final sleep, which stalls the event loop instead of yielding. It works because the code under test runs on background threads, but consider using an async-aware sleep (e.g. anyio.sleep / asyncio.sleep) or driving the loop off-thread to keep the async test idiomatic and avoid blocking the loop for up to ~5s.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At tests/httpx2/websockets/test_api.py, line 928:

<comment>The test is an anyio async test but uses blocking `time.sleep()` in its busy-wait loop and final sleep, which stalls the event loop instead of yielding. It works because the code under test runs on background threads, but consider using an async-aware sleep (e.g. `anyio.sleep` / `asyncio.sleep`) or driving the loop off-thread to keep the async test idiomatic and avoid blocking the loop for up to ~5s.</comment>

<file context>
@@ -903,6 +903,33 @@ def wait_for_session_threads(expected: int) -> None:
+                for _ in range(100):
+                    if ws._events.full():
+                        break
+                    time.sleep(0.05)
+                assert ws._events.full()
+                time.sleep(0.3)
</file context>

@cubic-dev-ai cubic-dev-ai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

1 issue found across 2 files (changes from recent commits).

Prompt for AI agents (unresolved issues)

Check if these issues are valid — if so, understand the root cause of each and fix them. If appropriate, use sub-agents to investigate and fix each issue separately.


<file name="src/httpx2/httpx2/websockets/_api.py">

<violation number="1" location="src/httpx2/httpx2/websockets/_api.py:161">
P2: Teardown can still take minutes when the full queue blocks a receive batch containing many small frames, because each freed slot incurs another 10 ms join wait before the producer can reach its next `put()`. Drain immediately while items are available, and only use the timed join when the queue is empty.</violation>
</file>

Reply with feedback, questions, or to request a fix.

Re-trigger cubic

while task.is_alive():
with contextlib.suppress(queue.Empty):
self._events.get_nowait()
task.join(timeout=0.01)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2: Teardown can still take minutes when the full queue blocks a receive batch containing many small frames, because each freed slot incurs another 10 ms join wait before the producer can reach its next put(). Drain immediately while items are available, and only use the timed join when the queue is empty.

Prompt for AI agents
Check if this issue is valid — if so, understand the root cause and fix it. At src/httpx2/httpx2/websockets/_api.py, line 161:

<comment>Teardown can still take minutes when the full queue blocks a receive batch containing many small frames, because each freed slot incurs another 10 ms join wait before the producer can reach its next `put()`. Drain immediately while items are available, and only use the timed join when the queue is empty.</comment>

<file context>
@@ -149,9 +149,16 @@ def __exit__(
+        while task.is_alive():
+            with contextlib.suppress(queue.Empty):
+                self._events.get_nowait()
+            task.join(timeout=0.01)
 
     def ping(self, payload: bytes = b"") -> threading.Event:
</file context>

self._background_keepalive_ping_task.join()
self._join_discarding_events(self._background_keepalive_ping_task)

def _join_discarding_events(self, task: threading.Thread) -> None:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add a check for sys.version with a call queue.shutdown(immediate=True)? This doesn't need requires-python bump

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants