Skip to content

fix(realtime): decouple bounded receive buffering from inference - #3686

Draft
LauraGPT wants to merge 1 commit into
mainfrom
codex/realtime-receive-pressure-20260909
Draft

fix(realtime): decouple bounded receive buffering from inference#3686
LauraGPT wants to merge 1 commit into
mainfrom
codex/realtime-receive-pressure-20260909

Conversation

@LauraGPT

@LauraGPT LauraGPT commented Sep 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

Related to #3528; this PR does not close the issue or claim the reporter's L20 workload is fixed.

  • Receive independently of session inference into a bounded, ordered application FIFO (128 messages / 16 MiB by default). Overflow explicitly closes with 1013; it is not successful final delivery.
  • Defer only provisional decodes when the next queued message is audio. Preserve PCM and command order, completed-segment inputs, and COMMIT/STOP paths.
  • Join owned receive/worker tasks on exit and suppress successful results after an in-flight operation outlives receive failure.
  • Document the queue/ping relationship, memory boundaries, source-vs-PyPI availability, and hardware acceptance requirements.

Verification

  • Exact candidate on ind-gpu8, Python 3.12.3: 142 tests passed with websockets 17.1; 142 passed with 10.4.
  • New real-loopback tests use synthetic acoustic sessions: finite blocked-decoder bursts retain Pong and ordered audio/final delivery; sustained overflow gives 1013; connections have independent queues.
  • Earlier candidates fail the added receive/cleanup guards. The payload-lifetime test still rejects persistent references after one event-loop completion turn.
  • Repository docs: 268 passed / 4 dependency-related skips, followed by all 23 tests in the affected browser/Sphinx modules passing without skips. All four missing checks were executed.
  • Two site builds: 267 files / 202 HTML pages each, validation passed and all bytes identical.
  • Independent review of source, workflow and docs: no remaining findings. Test authorship and review are distinguished in the retained evidence.

Remaining Acceptance

Draft intentionally: the new Python 3.10/10.4 and Python 3.12/17.1 CI must pass on this exact head. Actual model low-concurrency behavior, final transcript quality, preview freshness, shared-engine fairness and the reporter's paced 47-second / 16-client L20 workload still require acceptance. H100 availability is not L20 evidence.

Fewer partials may change context/fallback observations. No transcript-equivalence or throughput improvement is claimed. Queue limits bound application payloads, not all protocol/session/process memory. Waiting for thread completion does not force-stop a hung inference backend.

No PyPI release, production deployment, issue closure or repeated request for the reporter's parameter grid is included.

Signed-off-by: zhifu gao <zhifu.gzf@alibaba-inc.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant