test(e2e): Evaluate replay assertion on Android - #6684
Merged
Conversation
Re-adds the assertReplay.yml check removed in #6072. That assertion was flaky because buffer-mode replay (replaysOnErrorSampleRate) only attaches a replay_id to the error event if the native replay buffer captured a frame before the error fired; on slow CI the exception was captured while the buffer was still empty, so the event was sent permanently without a replay_id and no server-side retry could recover it. Make the test deterministic by priming the buffer before capturing the exception: a new side-effect-free "Replay Ping" counter in the e2e harness is tapped repeatedly to mutate the view hierarchy so the native capture records frames first. iOS only, matching the assertion scope (Android does not reliably capture replays in CI, see #4277). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Runs the captureReplay buffer-priming and assertReplay check on Android in addition to iOS. Android has never asserted replays in CI (see #4277, "android doesn't seem to capture replays in CI"); this re-evaluates that now that deterministic buffer priming lands the assertion reliably on iOS. Relax the codec assertion from an exact `ftypmp42` major brand to a valid MP4 container check (the "ftyp" box), since iOS (AVAssetWriter) and Android (MediaMuxer) can emit different major brands for the same valid MP4. The Android E2E result on this PR is the deliverable: it empirically shows whether Android now captures a replay in CI. If red, keep the assertion iOS-only. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Contributor
Semver Impact of This PR⚪ None (no version bump detected) 📋 Changelog PreviewThis is how your changes will appear in the changelog.
🤖 This preview updates automatically when you update the PR. |
8 tasks
Contributor
iOS (legacy) Performance metrics 🚀
|
Contributor
iOS (new) Performance metrics 🚀
|
antonis
marked this pull request as ready for review
September 7, 2026 15:05
Contributor
Android (new) Performance metrics 🚀
|
Contributor
Android (legacy) Performance metrics 🚀
|
…tion-android # Conflicts: # dev-packages/e2e-tests/maestro/captureReplay.yml # dev-packages/e2e-tests/maestro/utils/assertReplay.yml
Contributor
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Note
Stacked on top of #6683 (iOS replay assertion). Review/merge that first; this PR's base auto-retargets to
mainonce it lands. The diff below is only the Android delta.📢 Type of change
📜 Description
Extends the
captureReplaye2e test to run the buffer-priming step and theassertReplaycheck on Android, not just iOS — the second half of #6082 ("evaluate adding an equivalent Android check").maestro/captureReplay.yml— remove theplatform: iOSgates so priming +assertReplayrun on both platforms. TheReplay Pingprimer (testID='replayPing') is already platform-agnostic (RNtestIDmaps to a resource-id Maestro can tap on Android).maestro/utils/assertReplay.yml— relax the codec assertion from the exactftypmp42major brand to a valid-MP4-container check (theftypbox at byte offset 4). iOS (AVAssetWriter) emitsmp42; Android (MediaMuxer) can emit a different brand such asisomfor the same valid MP4, so the exact-brand check would false-fail on Android. iOS still passes (ftypmp42starts withftyp).💡 Motivation and Context
Second half of #6082
💚 How did you test it?
CI
📝 Checklist
sendDefaultPIIis enabled.🔮 Next steps