Add opt-in DirectPolling microphone capture mode (Quest capture reliability) - #379
Open
faryal-9 wants to merge 1 commit into
Open
Add opt-in DirectPolling microphone capture mode (Quest capture reliability)#379faryal-9 wants to merge 1 commit into
faryal-9 wants to merge 1 commit into
Conversation
…ility) The default capture path plays the microphone clip through an AudioSource and captures it via AudioProbe's OnAudioFilterRead. That couples two clocks: the microphone driver writes the clip ring at the input device's rate while the AudioSource reads it at the output pipeline's rate. On devices where those clocks drift, the filter's read position slides against the microphone's write position until it consumes stale or not-yet-written regions of the ring, which surfaces as intermittent silence and crackle. We measured this in production on Meta Quest 3 (whose capture pipeline runs at 24 kHz). This adds an opt-in MicrophoneCaptureMode.DirectPolling that reads 10 ms frames straight out of the clip with Microphone.GetPosition/GetData on the main thread and raises AudioRead directly - no AudioSource or AudioProbe involved, so the microphone's own write position is the only clock and drift cannot accumulate. This is the same capture strategy Photon Voice and Mumble use on the same hardware. Details: - Default behavior is unchanged: the existing constructor keeps the filter-based path; the new mode is selected via a constructor overload. - Ring wrap-around is handled modularly; a transient GetPosition() == -1 (observed on some Android device states) is skipped rather than fed into GetData. - After a main-thread hitch (GC, scene load) the backlog beyond a 300 ms catch-up budget is dropped oldest-first: the native audio source ingests 10 ms frames into a bounded queue at real time, so emitting hundreds of ms of stale audio in one tick overflows it, and real-time voice cannot use that audio anyway. - Channel shape is adapted to the source's resolved device format (mono -> N duplication for the common Android mono-clip case, N -> mono averaging, first-channel fallback otherwise). - A throwing AudioRead subscriber is contained (throttled warning): the loop is the sole frame producer, so an escaped exception would leave the microphone permanently silent. - Stop/restart (including the iOS pause/resume path) invalidate the loop via a generation counter. Tested on Meta Quest 3 (Unity 2022.3, Android, 24 kHz mono capture, stereo source): multi-hour sessions with repeated network drops and SDK reconnects show steady 10 ms frame production (~250 packets per 5 s via OutboundRtp counters) with no silence or drift, where the filter-based path previously degraded. Editor (Windows, 48 kHz) verified against a remote peer for parity.
faryal-9
requested review from
MaxHeimbrock,
cloudwebrtc,
ladvoc and
xianshijing-lk
as code owners
August 27, 2026 11:36
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
The default microphone capture path plays the mic clip through an
AudioSourceand captures it viaAudioProbe'sOnAudioFilterRead. That couples two clocks: the microphone driver writes the clip ring at the input device's rate, while the AudioSource reads it at the output pipeline's rate. On devices where those clocks drift, the filter's read position slides against the microphone's write position until it consumes stale or not-yet-written regions of the ring — which surfaces as intermittent silence and crackle.We measured this in production on Meta Quest 3, whose capture pipeline runs at 24 kHz (the device-format mismatch family reported in #172; the format half was fixed by #304, but the two-clock transport remains).
Fix
An opt-in
MicrophoneCaptureMode.DirectPollingthat reads 10 ms frames straight out of the microphone clip withMicrophone.GetPosition/AudioClip.GetDataon the main thread and raisesAudioReaddirectly — no AudioSource or AudioProbe involved, so the microphone's own write position is the only clock and drift cannot accumulate. This is the same capture strategy Photon Voice and Mumble use on the same hardware.Default behavior is unchanged: the existing constructor keeps the filter-based path; the new mode is selected via a constructor overload.
Details
GetPosition() == -1(observed on some Android device states) is skipped rather than fed intoGetData.AudioReadsubscriber is contained with a throttled warning: the loop is the sole frame producer, so an escaped exception would leave the microphone permanently silent.Tested
This ports the capture loop we have been running in production in our Quest app (as a vendored SDK modification) onto the current
mainlayout.OutboundRtppacket counters) with no silence and no drift — where the filter-based path previously degraded.Fixes #172 (for apps that opt in).