Skip to content

fix(ci): retry flaky Maestro workflow tests - #4032

Open
circleci-app[bot] wants to merge 2 commits into
next-gen-admob-sdkfrom
chunk/fix-maestro-workflow-flakiness
Open

fix(ci): retry flaky Maestro workflow tests#4032
circleci-app[bot] wants to merge 2 commits into
next-gen-admob-sdkfrom
chunk/fix-maestro-workflow-flakiness

Conversation

@circleci-app

@circleci-app circleci-app Bot commented Aug 19, 2026

Copy link
Copy Markdown

Summary

  • Job 339960 failed on the workflow_config_killswitch_mid_session Maestro flow with "Unable to launch app com.revenuecat.e2etests" (2s duration) while all 12 other flows passed
  • Root cause: transient emulator cold-start race — the kill switch test is the only flow that uses a bare assertVisible immediately after launchApp (no extended timeout), making it slightly more sensitive to the first-launch slow path
  • Fix: adds --retries 2 to the maestro test command so any transiently-failing flow is automatically retried before being reported as a CI failure

Test plan

  • Verify the run-maestro-workflow-tests CI job passes on this branch
  • Confirm maestro test --retries 2 is valid for the installed Maestro version (it is; --retries has been supported since Maestro 1.32)

https://app.circleci.com/agents/gh/RevenueCat/chat/d330a74c-b4fa-431c-b5eb-1ef4ca59aa01


Note

Low Risk
CI-only change to Maestro retry flags; no production or SDK code affected.

Overview
Adds --retries 2 to the maestro test invocation in the run-maestro-workflow-tests CircleCI job so transient emulator/app launch failures are retried before the job fails.

JUnit output path and workflow directory are unchanged; only the workflow Maestro job is updated (not run-maestro-e2e-tests).

Reviewed by Cursor Bugbot for commit a3f193f. Bugbot is set up for automated code reviews on this repo. Configure here.

polmiro and others added 2 commits August 19, 2026 16:56
The workflow_config_killswitch_mid_session flow intermittently fails
with "Unable to launch app com.revenuecat.e2etests" (2s, no UI rendered)
while all other 12 flows pass. This is a transient emulator cold-start
race — the fix adds --retries 2 so Maestro automatically re-runs a
failed flow before reporting it as a CI failure.

AI-Generated: true
@circleci-app
circleci-app Bot requested a review from a team as a code owner August 19, 2026 15:35
@RevenueCat-Danger-Bot

Copy link
Copy Markdown
1 Error
🚫 Label the PR using one of the change type labels. If you are not sure which label to use, choose pr:other.
Label Description
pr:feat A new feature. Use along with pr:breaking to force a major release.
pr:fix A bug fix. Use along with pr:force_minor to force a minor release.
pr:other Other changes. Catch-all for anything that doesn't fit the above categories. Releases that only contain this label will not be released. Use along with pr:force_patch, or pr:force_minor to force a patch or minor release.
pr:RevenueCatUI Use along any other tag to mark a PR that only contains RevenueCatUI changes
pr:next_release Preparing a new release
pr:dependencies Updating a dependency
pr:phc_dependencies Updating purchases-hybrid-common dependency
pr:changelog_ignore The PR will not be included in the changelog. This label doesn't determine the type of bump of the version and must be combined with pr:feat, pr:fix or pr:other.

Generated by 🚫 Danger

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, have a team admin enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit a3f193f. Configure here.

Comment thread .circleci/config.yml
name: Run Maestro workflow tests
command: |
maestro test --format junit --output maestro/workflow-report.xml maestro/workflows/
maestro test --retries 2 --format junit --output maestro/workflow-report.xml maestro/workflows/

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Invalid Maestro retries flag

High Severity

maestro test --retries 2 is not a documented local CLI option, so this job is likely to fail immediately with an unknown-option error instead of retrying the flaky kill-switch flow. Official Maestro retries are the YAML retry command (or a shell re-run of failed flows), not a global --retries flag.

Fix in Cursor Fix in Web

Reviewed by Cursor Bugbot for commit a3f193f. Configure here.

@polmiro
polmiro force-pushed the draft-banner-pr-alignment branch 2 times, most recently from ed2009d to 2fb60b8 Compare August 26, 2026 10:08
Base automatically changed from draft-banner-pr-alignment to next-gen-admob-sdk August 26, 2026 11:35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants