Fix video republishing issues - #1008
Conversation
WebRTC semantics checkVerified against the libwebrtc tree that reusing the same
Ordering caveat that this fix relies on: Net effect for screen share: |
|
This problem still exists |
`republishAllTracks()` routes every local track through `unpublish()`, which calls `track.stop()` and tears down the capturer. For iOS broadcast extension and ReplayKit screen capture this kills the IPC and the extension self-terminates — the track gets re-added to the new publisher but no frames flow. Detach screen share publications from the old publisher peer connection without going through `unpublish()` so the capturer keeps running, then let `_publish` reattach it to the new publisher. Resolves #1004 Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
republishAllTracks() aborted the whole loop on the first _publish failure and only logged a generic error at the call site, hiding which track failed. Catch per track, log the source, and keep re-publishing the rest so one track's failure no longer blocks the others. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
5725d0b to
7075330
Compare
817bf1a to
2b0a4ff
Compare
The detach block added in 723e321 hand-rolled a partial unpublish, which skipped four steps unpublish() performs: sender removal from the publisher, didUnpublishTrack, publication.set(track: nil) and onUnpublish(). On the room move path (_republishLocalTracks) transports are never torn down, so leaving the sender in place kept the old sendonly transceiver attached to the running capturer and _publish added a second one for the same media track. Add an internal _unpublish(publication:notify:stopTrack:) and keep the public unpublish() as a forwarder so the ObjC selector is unchanged. _unpublish is the old body with the stop gated on the flag, so tracks that pass stopTrack: true behave exactly as before. Screen shares take the same path with stopTrack: false, which stops the transceiver and frees the m-line while Track.start()'s already-started guard keeps the capturer counter at 1. The detached publication also drops its track reference, so it stops observing the capturer that is deliberately left running, and its per-codec senders are cleared because _publish re-attaches the main sender but never the rtpSenderForCodec entries. Also restore the republish contract weakened in 7075330: per-track failures are still logged individually, but the first error is rethrown so both call sites' error handling is reachable again. Muted screen shares are no longer detached-then-skipped, which silently dropped their publication with no delegate event. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2b0a4ff to
e586b70
Compare
republishAllTracks() cleared rtpSenderForCodec only for the screen share it detaches. _publish re-attaches the main rtpSender but never the per-codec entries, so after a full reconnect any other video track published with a backup codec still pointed at senders of the destroyed peer connection: VideoTrack._set(subscribedCodec:) matched one, reported the codec as still sent, and publish(additionalVideoCodec:) never re-added it. Clear the dict in _unpublish once the simulcast senders have been read and removed, so it applies to every unpublished track. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
VideoCapturer.stopCapture() reset dimensionsCompleter but left _state.dimensions at its last value, and set(dimensions:) only resumes the completer when the value changes. A source that resumes at the same resolution therefore never resolved dimensions again, so _publish's "Waiting for dimensions to resolve" timed out after 10s and the re-publish failed. CameraCapturer already worked around this with its own set(dimensions: nil) on stop; BufferCapturer and BroadcastScreenCapturer did not, so no buffer-backed video track could be re-published after a full reconnect — the second half of the screen share failure. Nil the dimensions in the base class instead, which resets the completer as a side effect. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Parameterized over microphone, camera and screen share: each track is published, the room is put through debug_simulate(scenario: .fullReconnect), and the test waits on a RoomDelegate-backed AsyncCompleter for the replacement publication rather than polling. Screen share must come back with its capturer untouched (startStopCounter still 1, no .stopped transition); every other source is expected to be torn down and started again. Writing it surfaced that the screen share predicate applied the name fallback to any source, while Participant.getTrackPublication(source:) only applies it to .unknown — createBufferTrack keeps the screen share name by default, so a `source: .camera` track was matched as a screen share. Narrow the predicate to match. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Looked at whether the keep-alive should extend to camera tracks: the mechanism generalizes for free — widening the predicate to any unmuted video source is one line, and the parameterized test passes with the camera case flipped to |
Summary
republishAllTracks()routes every local track throughunpublish(), which callstrack.stop()and tears down the capturer. For iOS broadcast extension and ReplayKit screen capture this closes the IPC and the extension self-terminates — the track gets re-added to the new publisher but no frames flow.Fixed by giving
unpublish()an internalstopTrack:variant and taking that path for screen shares, so the capturer keeps running while the track is properly detached from the old publisher and reattached to the new one.unpublish(publication:notify:)stays as-is (public + ObjC selector unchanged) and forwards to a new internal_unpublish(publication:notify:stopTrack:)._unpublishis the previous body with the stop gated on the flag, so anything passingstopTrack: truebehaves exactly as before.stopTrack: false. Everything elseunpublish()does still happens: sender + simulcast-sender removal from the publisher,publisherShouldNegotiate(),onUnpublish(),didUnpublishTrack.rtpSenderForCodec.removeAll()in_unpublish, once the simulcast senders have been removed from the publisher._publishre-attaches the mainrtpSenderbut never the per-codec ones, so a stale entry made_set(subscribedCodec:)report a match against a sender of the destroyed peer connection andpublish(additionalVideoCodec:)never re-added the backup codec.VideoCapturer.stopCapture()now nils the cached dimensions. It resetdimensionsCompleterbut left_state.dimensionsat its last value, andset(dimensions:)only resumes the completer when the value changes — so a source resuming at the same resolution never resolved dimensions again and_publishtimed out in "Waiting for dimensions to resolve".CameraCapturerhad its ownset(dimensions: nil)on stop;BufferCapturerandBroadcastScreenCapturerdid not, so no buffer-backed video track could be re-published after a full reconnect. This is the second half of the screen share failure.stopTrack: trueteardown, same per-track error log as the previousunpublishAll().Resolves #1004
Why keeping the capturer alive works
Track.start()short-circuits ontrackState == .started, so skippingstop()leaves the capturer'sstartStopCounterat 1 and_publish'strack.start()is a no-op. Nothing else in_publishtouches the capture source.On the WebRTC side,
Transport.remove(track:)is_pc.removeTrack(sender)followed bystopInternal()on the video transceiver. So the old sender's track is nulled (sink detached, encoder stops) and the transceiver is stopped, making its m-section recyclable in unified plan; the debounced negotiation lets the removal and the followingaddTransceivercoalesce into a single offer. TheLKRTCVideoSourceis owned by theVideoCapturerand theLKRTCVideoTrackbyLocalVideoTrack— neither is owned by the peer connection — so the same media track is handed to the new PC with capture uninterrupted.The two republish paths differ in whether transports are live, and both are now correct:
Room+EngineDelegate)cleanUpRTC())removeTrackbeforeclose()" constraint inTransport.cleanUp_republishLocalTracks)_publishno longer adds a second transceiver for the same media trackE2EE also round-trips again:
didUnpublishTrackdisables and drops the old frame cryptor,didPublishTrackbuilds a fresh one bound to the new sender.Test plan
preferredBackupCodecsurvives a full reconnect with the backup codec re-publishedTests
Tests/LiveKitCoreTests/Room/RepublishTracksTests.swift, parameterized over microphone / camera / screen share. Each case publishes the track, runsdebug_simulate(scenario: .fullReconnect), and waits on aRoomDelegate-backedAsyncCompleterfor the replacement publication. Screen share must return with its capturer untouched (startStopCounterstill 1, no.stoppedtransition); every other source is expected to be torn down and started again. Frames come from a syntheticCVPixelBufferfeed, so the test needs no network and runs in ~3.5s.Verified it fails without the fix: with
keepCapturingforced tofalse, the screen share case fails onstopSpy.stops == 0while the other two still pass.Known gaps
_publish's catch still callstrack.stop()and kills the capture source. The error is at least surfaced now.🤖 Generated with Claude Code