Skip to content

Resolve a default audio session from engine state when no policy was pushed - #1182

Open
hiroshihorie wants to merge 2 commits into
mainfrom
hiroshi/native-audio-session-preset
Open

Resolve a default audio session from engine state when no policy was pushed#1182
hiroshihorie wants to merge 2 commits into
mainfrom
hiroshi/native-audio-session-preset

Conversation

@hiroshihorie

Copy link
Copy Markdown
Member

Problem

On iOS the WebRTC audio engine refuses to enable recording unless the audio session category permits input. Its pre-enable check returns kAudioEngineErrorAudioSessionInvalidCategory (-9001), which the SDK reported as AudioProcessingException(applyFailed): Audio engine returned error code: -9001.

livekit_client owns the iOS audio session since #1108 (LiveKitPlugin.swift disables flutter_webrtc's session management at registration). The native engine observer (LKAudioEngineObserver.willEnableEngine) is the right hook and runs before the engine's check, but it only applied a configuration that Dart had pushed. In automatic mode the only push site was Room.connect, so anything that started recording earlier met an empty cache, the observer returned "proceed" with the session still soloAmbient, and the engine rolled back:

  • Room.withPreConnectAudio, which SessionOptions.preConnectAudio enables by default, so the Flutter agent starter failed on every "Start call"
  • a pre-join microphone preview ([bug] Audio engine returned error code #1165)
  • an engine start driven from native before the Flutter side exists, for example the plugin's static setEngineAvailability on a CallKit killed-state wake

Because the preconnect throw happens before connect, the cache was never seeded and the failure repeated on every attempt.

How the Swift SDK handles this

AudioSessionEngineObserver.engineWillEnable derives the session configuration from the requested engine state alone (playAndRecord presets while recording, playback for playout only), synchronously inside the engine's enable call. Nothing is configured "before connect". The engine asks, the observer configures, the engine starts. The Flutter plugin already has the same observer in the same place, it just had no built-in policy.

Fix

LKAudioEngineObserver.effectiveConfigurationLocked now resolves a built-in playAndRecord preset (allowBluetooth | allowBluetoothA2DP | allowAirPlay, voiceChat, or videoChat when the speaker is preferred) whenever nothing has been pushed and automatic management is on. The existing playout-only playback branch applies to it as well. Manual mode still leaves the session alone. The Dart-pushed policy becomes an override rather than a prerequisite, and for the default AudioSessionOptions.communication it pushes the same values, so the connect-time push does not change the live session.

Dart passes a new preferSpeakerOutput flag on configureNativeAudio so the preset picks the same mode the Dart policy would. The preset and ResolvedAudioSessionPolicy.appleConfiguration are marked keep-in-sync. Making native the single source of truth for the automatic-mode policy is a follow-up, since it changes the channel contract.

Error mapping

Audio device module results now get their own error codes on the startLocalRecording and setEngineAvailability channels, mirroring client-sdk-swift's checkAdmResult:

ADM result Native error code Dart exception
-9000 InsufficientDevicePermission deviceAccessDenied TrackCreateException
-9001 AudioSessionInvalidCategory audioSessionInvalidCategory AudioSessionException (new)
-4100 FailedToConfigureAudioSession audioSessionConfigureFailed AudioSessionException (new)
anything else caller fallback (applyFailed, setEngineAvailability) unchanged

AudioSessionException is a new public class, hence the second changeset.

Relation to #1179

@MaxHeimbrock's #1179 diagnosed this first and fixes it by pushing the policy from LocalAudioTrack.startCapture (prepareRecording()). This PR takes the other layer: it makes the native observer self-sufficient like Swift's, so engine enables that never pass through Dart are covered too, and no Dart-side flag has to track native cache state. With this merged, prepareRecording() becomes redundant.

Testing

  • Flutter agent starter on an iPhone 17 Pro (iOS 27.0): unpatched, Start call failed with -9001 on every attempt. Patched, startCapture() succeeds two seconds before the connect-time policy push, the preconnect buffer is sent to the agent, and the call connects.
  • flutter analyze, flutter test, dart format --set-exit-if-changed, import_sorter --exit-if-changed clean.
  • New unit tests cover the error mapping and the new channel argument. The iOS-gated branches are not reachable from unit tests (lkPlatform() has no seam), same limitation as Configure the audio session before recording starts, not only on connect #1179 noted.

Fixes #1165
Refs #1042

…pushed

On iOS the audio engine refuses to enable recording unless the audio
session category permits input, and livekit_client owns that session.
The native engine observer only applied a configuration that Dart had
pushed, and the only push site in automatic mode was Room.connect. Any
recording that started earlier (pre-connect audio, a pre-join microphone
preview, an engine start driven from native before the Flutter side
exists) ran against the app-default soloAmbient category and failed with
kAudioEngineErrorAudioSessionInvalidCategory (-9001), reported as
AudioProcessingException(applyFailed).

The observer now resolves a built-in playAndRecord preset from engine
state when nothing has been pushed and automatic management is on,
matching the Swift SDK's AudioSessionEngineObserver, which derives the
session from engine state alone. The Dart-pushed policy becomes an
override rather than a prerequisite. Dart passes preferSpeakerOutput so
the preset picks the same mode the Dart policy would.

Audio device module results -9000, -9001 and -4100 now get their own
error codes and surface as TrackCreateException or the new
AudioSessionException instead of an audio processing failure.
…ping helpers

AudioManager prefers speaker output by default, so the built-in preset
now defaults to videoChat as well. Otherwise the connect-time push would
switch the live session from voiceChat to videoChat.

Also simplifies effectiveConfigurationLocked and the Dart error mapping,
and shortens the changeset entries.
@hiroshihorie
hiroshihorie marked this pull request as ready for review August 27, 2026 16:47

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✅ Devin Review: No Issues Found

Devin Review analyzed this PR and found no bugs or issues to report.

Devin Review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Audio engine returned error code

1 participant