Repository navigation
feat: audio width follows the output route; report lost audio - #51
Merged
Merged
Conversation
…utput on tvOS PlayerSession.displayCriteria describes the display mode the selected video wants, from what the engine outputs: the refresh rate after deinterlacing (measured from enqueued frames once there are any, snapped to the film and broadcast grid or no preference) and the dynamic range after Dolby Vision conversion. DisplayCriteriaMatcher (tvOS, opt-in) applies it through the window's AVDisplayManager and holds the session's start, bounded, while the TV switches; PlayerSession.holdStart/releaseStartHold carry that hold. LumePlayer.matchesDisplayCriteria wires it into the facade. TrackInfo.Video gains isInterlaced and the Dolby Vision configuration record.
The engine has no Dolby Vision output path: it presents the base layer of every profile it does not convert. Dolby Vision streams now ask for that base layer's range (compatibility ID before the first frame, the frames' transfer function after), and the synthesized dvh1 description is gone.
DolbyVisionDetection.Record now comes from the public DolbyVisionConfiguration instead of parsing AV_PKT_DATA_DOVI_CONF a second time, and DolbyVisionConfiguration.baseLayerIsIPT asks Record.claimsIPT, so the profile 5/10.0/20 rule lives in one place. Its doc now says what it is since conversion became a per-frame decision: the stream's claim.
The audio decoder read the negotiated output width once, at creation, so a route change mid-playback (HDMI receiver to AirPlay, HomePods) kept feeding the old width. AudioOutputRoute now follows the shared audio session's negotiated width through route-change notifications (stereo on macOS); the decoder refits per AudioLayoutPolicy at the next frame boundary, with an explicit ITU-R BS.775 DownmixMatrix (centre and surrounds -3 dB, LFE omitted) set into swresample, and mono goes out on both speakers. When the route narrows, the renderer keeps audio decoded for the wider route away from the renderer. PlayerSession.audioHealth (also in Diagnostics) reports audio that died while video plays on: decoder failed, renderer failed, or nothing reaching the renderer at a running playhead (AudioLossDetector), the latter also as .error with the new EngineError.Code.audioLost. Additive API only.
bilipp
force-pushed
the
feat/audio-route-channels
branch
from
October 12, 2026 22:33
984532d to
4310850
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #46.
Summary
Audio width that follows the output route. Until now the audio decoder read the session's negotiated output width (
AVAudioSession.outputNumberOfChannels) once, when the decoder was created, and kept it for the life of the session. The downmix was swresample's automatic matrix, picked implicitly by sample format. So:.renderFailed). Going the other way, from AirPlay to HDMI, stayed downmixed to stereo until the next session.Now:
AudioOutputRouteobserves the shared audio session's negotiated width throughrouteChangeNotification(every reason, including.routeConfigurationChange). On macOS it is fixed at stereo. Tests and hosts can drive it withupdate. The engine still only observes the session: category, activation and preferred channel count stay the host's job.AudioLayoutPolicyis a pure value type that maps source layout × route width to the target layout:DownmixMatrix: explicit ITU-R BS.775 levels (centre and surrounds −3 dB, LFE omitted), built byswr_build_matrix2and set withswr_set_matrix, so the matrix no longer depends on the source's sample format. It is deliberately not normalized, for loudness parity with stereo channels on zap. The documented trade-off is clipping on peaks where every channel is near full scale.AudioDecoderreads the route width per decoded frame and rebuilds swr when it changes. It emits the pending buffer first, so the switch lands on a frame boundary. Same-rate swr holds no samples back, so nothing is lost.SystemRenderer: when the route narrows, audio already decoded for the wider route is kept out of the renderer. That covers queued frames, retained refill audio, and what the renderer itself still holds. The narrowing therefore costs the renderer's lead in silence, about what a route change costs anyway, instead of risking a failed renderer.Does the system downmix by itself? On macOS it does: 8-channel LPCM with a channel bitmap, fed to
AVSampleBufferAudioRendereron built-in 2-channel speakers, renders (status.rendering, clock running). I checked this locally; it is not a committed test, because CI runners have no stable audio device. On tvOS the evidence goes the other way: the switch frommaximumOutputNumberOfChannelstooutputNumberOfChannelswas made because a renderer fed more channels than the session outputs failed outright on device, with layout tags attached. So the engine keeps mixing on iOS/tvOS and does not pass multichannel through to narrower routes. The suggestedmaximumOutputNumberOfChannelsis deliberately not used.Audio lost.
PlayerSession.audioHealth(.healthy/.absent/.lost(reason)) is also inDiagnosticsand in its log line. Reasons:decoderFailed: the decoder gave up. Its.decodeFailederror was already emitted, and the lane was silently excluded.rendererFailed: the audio renderer failed. The session still fails with.renderFailed, as before.starved: the clock runs and video is ahead of the playhead, but no audio has been enqueued beyond the playhead for 2 s of media.The detector (
AudioLossDetector) is a pure value type. Nothing counts while paused, buffering or seeking, when video is dry too (that is buffering), for audio-only sources, after the audio track's end, or across a seek. Gaps inside the audio don't count either, because the high-water mark is past them.starvedis reported once per episode as.error(EngineError(code: .audioLost))and clears when audio comes back.API (additive only — Lume compiles unchanged)
AudioOutputRoute(+Snapshot),AudioOutputLayout,AudioHealth,AudioLossReason.PlayerConfiguration.audioOutputRoute: AudioOutputRoute?. Defaults to nil, which follows the system session.PlayerSession.audioHealthandPlayerSession.audioOutput.DiagnosticsgainsaudioHealth,audioOutputandaudioRoute; its description gainsaout=… route=…and, only when lost,audio=lost(…).AudioDecoder.init(parameters:input:output:route:)andcurrentOutput. The existingmaxOutputChannels:initializer is kept and pins a fixed route.SystemRenderer.init(muted:audioOutputRoute:), with the new parameter defaulted.EngineError.Code.audioLost. Lume only logs.errorevents (LumeEngineCoordinator.handle(event:)) and never switches overEngineError.Code..erroralready carries non-fatal per-lane failures (a failed audio decoder is reported the same way without changing state), so this fits the existing semantics.PlayerEventcases.Tests
AudioLayoutPolicy: speaker sets match FFmpeg's named layouts; passthrough when the source fits; fold-down ladder; no invented LFE; mono → stereo; unnameable channels remixed.DownmixMatrix: 5.1→2.0 and 7.1→2.0 coefficients (centre −3 dB into both sides, surrounds into their own side, LFE 0, no cross-feed); 7.1→5.1; mono→stereo; the centre survives every stereo fold-down and no non-LFE channel is dropped.AudioRoute(decoder, forced widths):surround71.mkvat 8/6/2 andtruehd.mkvat 8/2. Bitmaps are checked, and the centre-only tone arrives in both stereo channels at −3 dB.basic.mp4mono → stereo.AudioLossDetector: every branch, including legitimate silences, pause/seek, recovery and re-reporting, and permanent losses.DecoderTests: the AAC decode test now expects its mono sine as 2 channels (mono → stereo on a stereo route — the one intended behaviour change for existing callers).AudioRouteSession(new fixturesurround51.ts: H.264 + AC-3 5.1, centre-only tone):.lost(.starved)and exactly one.error(.audioLost)..absent.Verification
swift test: full suite green (see the coordinator report for counts).xcodebuild buildfor generic tvOS and iOS: no new warnings.Not verified (needs an Apple TV)
aout=6>2ch@2 route=2ch/2 AirPlay.aout=1>2ch) plays on both sides.route=width; HomePods set as the Apple TV's default output may negotiate more than 2.route=6ch/6 HDMIOutput, or 8 at max).aout=6>6ch@6), with each channel on the right speaker (centre on centre, LFE on sub).aout=8>6ch@6).aout=follows the route.audio=lostand no.audioLost.route=…changes count up.outputNumberOfChannelsfollows a route change on its own, or the host has to re-assertsetPreferredOutputNumberOfChannelsafter one. Ifroute=2ch/8shows on HDMI after switching back, Lume should re-apply its preferred count on route change.audio=lost(starved)over long live sessions (ad splices, encoder restarts, reconnects).