Configure the audio session before recording starts, not only on connect - #1179
Draft
MaxHeimbrock wants to merge 1 commit into
Draft
Configure the audio session before recording starts, not only on connect#1179MaxHeimbrock wants to merge 1 commit into
MaxHeimbrock wants to merge 1 commit into
Conversation
On iOS the audio engine refuses to open the microphone unless the audio session already permits recording, and livekit_client owns that session: it disables flutter_webrtc's own session management at plugin registration. The policy was only pushed to native from Room.connect, so any recording that starts earlier ran against the app-default soloAmbient category and was rejected with kAudioEngineErrorAudioSessionInvalidCategory (-9001), surfacing as `AudioProcessingException(applyFailed): Audio engine returned error code: -9001`. That is the default path for agent sessions, since SessionOptions enables pre-connect audio, which buffers the microphone before connect and therefore failed on every attempt. A pre-join microphone preview hits the same wall without pre-connect audio involved (#1165). Push the policy from the capture path instead, where the microphone actually opens. AudioManager.prepareRecording() is a no-op once a policy has been pushed, so it never re-applies a configuration to a live session, and in manual mode, where the app owns the session. Fixes #1165 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
On iOS the audio engine refuses to open the microphone unless the audio session already permits recording, and livekit_client owns that session: it disables flutter_webrtc's own session management at plugin registration (
LiveKitPlugin.swift,setAudioSessionManagementEnabled(false)). But the policy was only pushed to native fromRoom.connect—NativeAudioManagement.start()→AudioManager.applyOptionsForConnect()→configureNativeAudio— so any recording that starts earlier ran against the app-defaultsoloAmbientcategory. The engine's pre-enable check rejects that withkAudioEngineErrorAudioSessionInvalidCategory(-9001), surfacing asAudioProcessingException(applyFailed): Audio engine returned error code: -9001.That is the default path for agent sessions:
SessionOptions.preConnectAudiodefaults to true, andwithPreConnectAudiobuffers the microphone before connect. BecausestartRecordingthrows beforeconnectis reached, the native policy cache is never seeded, so it fails on every attempt rather than only the first. A pre-join microphone preview hits the same wall with no pre-connect audio involved, which is how #1165 was reported.The fix pushes the policy from the capture path instead, where the microphone actually opens:
LocalAudioTrack.startCapturecalls the newAudioManager.prepareRecording()just beforeNative.startLocalRecording. That one choke point covers pre-connect buffering, a standalone pre-join track, and normal mic publish.prepareRecording()is deliberately narrow. It is a no-op off iOS, a no-op inAudioSessionManagementMode.manual(the app owns the session and is responsible for a recording-capable category), and a no-op once a policy has been pushed — tracked by a new_hasPushedAppleSessionPolicyflag set at both existing Apple push sites. That last guard is not just an optimization: re-pushing mid-call while the engine has playout only would resolve the playback category throughselectCategoryByEngineStateand apply it moments before recording starts, so the guard keeps this off the hot path entirely.The added unit test only asserts the off-iOS no-op —
lkPlatform()readsdart:io Platformwith no override hook, so no test in this repo can reach an iOS-gated branch.flutter analyze,flutter test,dart format --set-exit-if-changedandimport_sorter --exit-if-changedare all clean.How this went unnoticed
The precondition was never explicit.
PreConnectAudioBuffer(#830) opens the microphone outside the room lifecycle by design, and nothing stated who guarantees the session permits recording — it was satisfied incidentally, by flutter_webrtc'sensureAudioSessionongetUserMediaand by track-counting that fired on publish, which a pre-connect track has not reached.Two commits in 2.9.0, a day apart, removed that cover and then made the failure loud:
55af814) took sole ownership of the iOS session: it is wheresetAudioSessionManagementEnabled(false)first appears, killing flutter_webrtc's incidental configuration, and where track-counting became engine-driven lifecycle whose only Dart entry point is connect. After it, no path configures the session before connect.d1fe342) moved the ADM start intoLocalAudioTrack.startCaptureand made it fail-fast. Its description mentions the preconnect path, but the review lens there was audio processing options, so the session category beside it went unexamined. Combined with the fork's specific error codes, a latent misconfiguration became a named hard failure.The fail-fast was not the mistake — it exposed the real one. #1042 reports what looks like the same path failing at 2.7.0 with the generic
adm api failed with code: -1, five months earlier, which is likely the same root cause under a code too vague to act on.Two things kept it out of test range:
flutter build ios --release --no-codesign— a compile, with no simulator or device run.lkPlatform()readsdart:io Platformwith no seam, so on the test host every iOS-gated branch inAudioManageris unreachable. The existing audio tests cover policy resolution and channel argument encoding — pure Dart either side of the platform gate, never the gate itself.Worth considering separately from this PR: the native pre-enable check is gated
#if !TARGET_OS_OSX, so it runs on the simulator too — driving the example app through connect-with-mic there would have caught this end to end, and is the only layer that would have. A test seam forlkPlatform()would make these branches assertable at all, andwillEnableEnginefindingcachedConfigurationnil is worth logging loudly rather than silently returning proceed and letting the native check fail a layer later.Fixes #1165
Testing
I tested this locally with the Flutter Agent Starter: