Skip to content

fix: don't skip receiver frame cryptor when track metadata arrives late (+ sync upstream, drop fork-only changes) - #3

Draft
td-famedly wants to merge 4 commits into
mainfrom
agentic-td/e2ee-receiver-frame-cryptor-late-metadata-8104
Draft

fix: don't skip receiver frame cryptor when track metadata arrives late (+ sync upstream, drop fork-only changes)#3
td-famedly wants to merge 4 commits into
mainfrom
agentic-td/e2ee-receiver-frame-cryptor-late-metadata-8104

Conversation

@td-famedly

@td-famedly td-famedly commented Jul 7, 2026

Copy link
Copy Markdown
Member

Problem

A remote track can be subscribed before its TrackInfo metadata (mimeType / encryption) has been applied to the publication. The TrackSubscribedEvent handler in E2EEManager decided on that incomplete metadata:

  • mimeType.split('/')[1] threw a RangeError on an empty mime type, killing the handler.
  • An unset encryption read as kNone, so the frame cryptor was skipped.

Either way no frame cryptor was ever created for the track, so with insertable streams enabled the frames stayed encrypted forever — black video / silent audio that only a renegotiation (e.g. toggling a screenshare) could unstick.

Changes

E2EE fix (lib/src/e2ee/e2ee_manager.dart):

  • Wait (bounded by connectOptions.timeouts.publish) for the publication's TrackInfo (latestInfo) / mimeType to be populated before deciding whether a frame cryptor is needed, and abort cleanly when the track is unsubscribed or replaced while waiting. Confirmed-unencrypted tracks exit the wait immediately (Bugbot finding).
  • Parse the codec from the mime type safely instead of throwing on malformed values, and still create the cryptor (without updateCodec) if the mime type never arrives for an encrypted track.
  • Attach onFrameCryptorStateChanged right after creating a cryptor (before updateCodec) so E2EE state changes are always surfaced.
  • Wrap both cryptor setup paths in try/catch with a warning log so failures are visible instead of silently dropped.
  • Expose the frame / data-packet cryptor factories as @visibleForTesting fields so tests can inject fakes instead of hitting platform channels.

Regression tests (test/e2ee/e2ee_manager_test.dart):

  • Encrypted track subscribed before its metadata arrives → frame cryptor is created once the metadata is applied, and cryptor state changes surface as TrackE2EEStateEvent.
  • Encrypted track whose mimeType never arrives → frame cryptor is still created after the publish timeout.
  • Confirmed-unencrypted track → cryptor setup is skipped promptly instead of polling until the publish timeout.

All three were verified to fail against the pre-fix handler (RangeError on the first two, delayed-skip timeout on the third). Test infra: fake FrameCryptor / DataPacketCryptor factories, media stream fakes from room_e2e_test moved to a shared mock (plus a fake RTCRtpReceiver), and E2EContainer accepts custom ConnectOptions.

Removed fork-only changes (separate commit):

  • Restored lib/src/core/room.dart and lib/src/track/web/_audio_html.dart to their upstream (livekit/client-sdk-flutter) versions, dropping the [audioElementLogs] debug logging and the 5×1s retry loop for _getRemoteParticipantBySid. That retry loop delayed media track event handling and made the room_e2e_test "tracks arriving before participant metadata" test time out.

The branch is rebased onto the fork's main (which now includes upstream's "Certificate pinning (livekit#1065)"), so after this PR the fork differs from upstream main only by the E2EE fix and its tests.

Verification

  • dart format . --set-exit-if-changed — no changes.
  • dart run import_sorter:main --no-comments --exit-if-changed — no changes.
  • dart run scripts/check_version.dart — all version checks passed.
  • flutter analyze — no issues.
  • flutter test — all 344 tests pass (341 + the 3 new E2EE regression tests).
Open in Web Open in Cursor 

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using high effort and found 1 potential issue.

Fix All in Cursor

Bugbot Autofix prepared a fix for the issue found in the latest run.

  • ✅ Fixed: Unencrypted remote tracks wait publish timeout
    • Changed the wait-loop condition from encryptionType == kNone (which is also true for confirmed-unencrypted tracks) to latestInfo == null, so plaintext remotes with a valid mimeType exit the loop immediately instead of polling until the publish timeout.

Create PR

Or push these changes by commenting:

@cursor push af34777981
Preview (af34777981)
diff --git a/lib/src/e2ee/e2ee_manager.dart b/lib/src/e2ee/e2ee_manager.dart
--- a/lib/src/e2ee/e2ee_manager.dart
+++ b/lib/src/e2ee/e2ee_manager.dart
@@ -132,7 +132,7 @@
     final timeout = participant.room.connectOptions.timeouts.publish;
     final deadline = DateTime.now().add(timeout);
     var codec = _codecFromMimeType(publication.mimeType);
-    while ((codec.isEmpty || publication.encryptionType == EncryptionType.kNone) && DateTime.now().isBefore(deadline)) {
+    while ((codec.isEmpty || publication.latestInfo == null) && DateTime.now().isBefore(deadline)) {
       await Future.delayed(const Duration(milliseconds: 200));
       // Manager was cleaned up, or the track was unsubscribed / replaced while
       // waiting; a newer subscription will run its own setup.

You can send follow-ups to the cloud agent here.

Reviewed by Cursor Bugbot for commit e52bdb8. Configure here.

return;
}
codec = _codecFromMimeType(publication.mimeType);
}

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Unencrypted remote tracks wait publish timeout

Medium Severity

The remote frame-cryptor wait loop keeps running while publication.encryptionType == EncryptionType.kNone, but that value also means a confirmed unencrypted track once TrackInfo is applied. Plaintext remotes with a valid mimeType therefore poll until connectOptions.timeouts.publish (often 10s) before the handler can skip setup, delaying every such subscribe in E2EE-enabled rooms.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit e52bdb8. Configure here.

@CLAassistant

CLAassistant commented Jul 7, 2026

Copy link
Copy Markdown

CLA assistant check
All committers have signed the CLA.

@cursor cursor Bot changed the title fix: don't skip receiver frame cryptor when track metadata arrives late fix: don't skip receiver frame cryptor when track metadata arrives late (+ sync upstream, drop fork-only changes) Jul 7, 2026
cursoragent and others added 2 commits July 7, 2026 08:58
A remote track can be subscribed before its TrackInfo metadata (mimeType /
encryption) has been applied to the publication. The TrackSubscribedEvent
handler decided on that incomplete metadata: 'mimeType.split('/')[1]' threw
a RangeError on an empty mime type (killing the handler) and an unset
encryption read as kNone skipped the cryptor. Either way no frame cryptor
was ever created for the track, so with insertable streams enabled the
frames stayed encrypted forever - black video / silent audio that only a
renegotiation (e.g. toggling a screenshare) could unstick.

- wait (bounded by connectOptions.timeouts.publish) for mimeType/encryption
  to be populated before deciding whether a frame cryptor is needed, and
  abort cleanly when the track is unsubscribed or replaced while waiting
- parse the codec from the mime type safely instead of throwing on
  malformed values, and still create the cryptor (without updateCodec) if
  the mime type never arrives for an encrypted track
- attach onFrameCryptorStateChanged right after creating a cryptor (before
  updateCodec) so E2EE state changes are always surfaced
- wrap both cryptor setup paths in try/catch with a warning log so failures
  are visible instead of silently dropped

Signed-off-by: Cursor Agent <cursoragent@cursor.com>
Restore lib/src/core/room.dart and lib/src/track/web/_audio_html.dart to
their upstream (livekit/client-sdk-flutter) versions, dropping the
fork-only audioElementLogs debug logging and the 5x1s retry loop for
_getRemoteParticipantBySid. The retry loop delayed media track event
handling and made the room_e2e_test 'tracks arriving before participant
metadata' test time out.

Co-authored-by: td <td-famedly@users.noreply.github.com>
@cursor
cursor Bot force-pushed the agentic-td/e2ee-receiver-frame-cryptor-late-metadata-8104 branch from 5e7b04b to 13d6085 Compare July 7, 2026 08:58
cursoragent and others added 2 commits July 7, 2026 13:54
EncryptionType.kNone is both 'metadata not applied yet' and 'confirmed
unencrypted', so plaintext remote tracks in E2EE rooms polled the wait
loop for the full publish timeout before skipping cryptor setup. Loop on
publication.latestInfo == null instead, which only covers the
metadata-not-yet-applied case; incomplete metadata is still awaited via
the empty-codec check.

Addresses Bugbot finding on PR #3.

Co-authored-by: td <td-famedly@users.noreply.github.com>
Cover the three E2EEManager remote-track behaviors:
- an encrypted track subscribed before its mimeType/encryption metadata
  arrives gets a frame cryptor once the metadata is applied, and cryptor
  state changes surface as TrackE2EEStateEvent (previously the handler
  died on a RangeError and no cryptor was ever created)
- an encrypted track whose mimeType never arrives still gets a frame
  cryptor (without updateCodec) after the publish timeout
- a confirmed-unencrypted track skips cryptor setup promptly instead of
  polling until the publish timeout

To make E2EEManager testable without platform channels, expose the
frame/data-packet cryptor factories as @VisibleForTesting fields.
Test-infra changes: fake FrameCryptor/DataPacketCryptor factories, the
media stream fakes from room_e2e_test moved to a shared mock (plus a
fake RTCRtpReceiver), and E2EContainer accepts custom ConnectOptions.

All three tests were verified to fail against the pre-fix handler
(RangeError on the first two, delayed-skip timeout on the third).

Co-authored-by: td <td-famedly@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants