Skip to content

Configure the audio session before recording starts, not only on connect - #1179

Draft
MaxHeimbrock wants to merge 1 commit into
mainfrom
max/fix-ios-audio-session-before-recording
Draft

Configure the audio session before recording starts, not only on connect#1179
MaxHeimbrock wants to merge 1 commit into
mainfrom
max/fix-ios-audio-session-before-recording

Conversation

@MaxHeimbrock

@MaxHeimbrock MaxHeimbrock commented Aug 26, 2026

Copy link
Copy Markdown
Contributor

On iOS the audio engine refuses to open the microphone unless the audio session already permits recording, and livekit_client owns that session: it disables flutter_webrtc's own session management at plugin registration (LiveKitPlugin.swift, setAudioSessionManagementEnabled(false)). But the policy was only pushed to native from Room.connectNativeAudioManagement.start()AudioManager.applyOptionsForConnect()configureNativeAudio — so any recording that starts earlier ran against the app-default soloAmbient category. The engine's pre-enable check rejects that with kAudioEngineErrorAudioSessionInvalidCategory (-9001), surfacing as AudioProcessingException(applyFailed): Audio engine returned error code: -9001.

That is the default path for agent sessions: SessionOptions.preConnectAudio defaults to true, and withPreConnectAudio buffers the microphone before connect. Because startRecording throws before connect is reached, the native policy cache is never seeded, so it fails on every attempt rather than only the first. A pre-join microphone preview hits the same wall with no pre-connect audio involved, which is how #1165 was reported.

The fix pushes the policy from the capture path instead, where the microphone actually opens: LocalAudioTrack.startCapture calls the new AudioManager.prepareRecording() just before Native.startLocalRecording. That one choke point covers pre-connect buffering, a standalone pre-join track, and normal mic publish.

prepareRecording() is deliberately narrow. It is a no-op off iOS, a no-op in AudioSessionManagementMode.manual (the app owns the session and is responsible for a recording-capable category), and a no-op once a policy has been pushed — tracked by a new _hasPushedAppleSessionPolicy flag set at both existing Apple push sites. That last guard is not just an optimization: re-pushing mid-call while the engine has playout only would resolve the playback category through selectCategoryByEngineState and apply it moments before recording starts, so the guard keeps this off the hot path entirely.

The added unit test only asserts the off-iOS no-op — lkPlatform() reads dart:io Platform with no override hook, so no test in this repo can reach an iOS-gated branch. flutter analyze, flutter test, dart format --set-exit-if-changed and import_sorter --exit-if-changed are all clean.

How this went unnoticed

The precondition was never explicit. PreConnectAudioBuffer (#830) opens the microphone outside the room lifecycle by design, and nothing stated who guarantees the session permits recording — it was satisfied incidentally, by flutter_webrtc's ensureAudioSession on getUserMedia and by track-counting that fired on publish, which a pre-connect track has not reached.

Two commits in 2.9.0, a day apart, removed that cover and then made the failure loud:

  • feat(audio): add AudioManager session and routing APIs #1108 (55af814) took sole ownership of the iOS session: it is where setAudioSessionManagementEnabled(false) first appears, killing flutter_webrtc's incidental configuration, and where track-counting became engine-driven lifecycle whose only Dart entry point is connect. After it, no path configures the session before connect.
  • Apply audio processing options when local capture starts #1115 (d1fe342) moved the ADM start into LocalAudioTrack.startCapture and made it fail-fast. Its description mentions the preconnect path, but the review lens there was audio processing options, so the session category beside it went unexamined. Combined with the fork's specific error codes, a latent misconfiguration became a named hard failure.

The fail-fast was not the mistake — it exposed the real one. #1042 reports what looks like the same path failing at 2.7.0 with the generic adm api failed with code: -1, five months earlier, which is likely the same root cause under a code too vague to act on.

Two things kept it out of test range:

  • CI's iOS job is flutter build ios --release --no-codesign — a compile, with no simulator or device run.
  • Unit tests cannot reach the code at all: lkPlatform() reads dart:io Platform with no seam, so on the test host every iOS-gated branch in AudioManager is unreachable. The existing audio tests cover policy resolution and channel argument encoding — pure Dart either side of the platform gate, never the gate itself.

Worth considering separately from this PR: the native pre-enable check is gated #if !TARGET_OS_OSX, so it runs on the simulator too — driving the example app through connect-with-mic there would have caught this end to end, and is the only layer that would have. A test seam for lkPlatform() would make these branches assertable at all, and willEnableEngine finding cachedConfiguration nil is worth logging loudly rather than silently returning proceed and letting the native check fail a layer later.

Fixes #1165

Testing

I tested this locally with the Flutter Agent Starter:

On iOS the audio engine refuses to open the microphone unless the audio
session already permits recording, and livekit_client owns that session:
it disables flutter_webrtc's own session management at plugin
registration. The policy was only pushed to native from Room.connect, so
any recording that starts earlier ran against the app-default
soloAmbient category and was rejected with
kAudioEngineErrorAudioSessionInvalidCategory (-9001), surfacing as
`AudioProcessingException(applyFailed): Audio engine returned error
code: -9001`.

That is the default path for agent sessions, since SessionOptions
enables pre-connect audio, which buffers the microphone before connect
and therefore failed on every attempt. A pre-join microphone preview
hits the same wall without pre-connect audio involved (#1165).

Push the policy from the capture path instead, where the microphone
actually opens. AudioManager.prepareRecording() is a no-op once a policy
has been pushed, so it never re-applies a configuration to a live
session, and in manual mode, where the app owns the session.

Fixes #1165

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug] Audio engine returned error code

1 participant