feat(expo): add semantic locators for React native SDK to be used in E2E Mobile tests#1859
Conversation
- Add Maestro login flow test for iOS and Android (auth/login.yaml) - Add shared utils for dynamic email generation and Mailosaur OTP retrieval - Add testIDs to login screen, wallet display, and shared wallet components - Wire Metro to watch shared/ workspace directory - Add .env.maestro template for local test runs - Add nightly GitHub Actions workflow (e2e-mobile-regression-tests.yml): parallel iOS (macos-latest) and Android (ubuntu-latest + KVM) jobs, JUnit XML output, and combined Slack notification to #team-qa-regression-test-alerts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:195-197
**Android emulator API level doesn't match local verification**
The PR description states that the login flow was verified locally on API 36 (`Medium_Phone_API_36.1`), but the CI emulator is configured with `api-level: 29`. This means CI will run against a meaningfully older Android version than the one that was tested. Differences in network security policy enforcement, WebView behavior, and system permission dialogs between API 29 and API 36 can cause the Maestro test to behave differently — potentially producing false passes or confusing failures that don't reflect the locally-verified behavior.
### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:200-201
**Android Metro startup uses a fixed sleep instead of the polling used for iOS**
The iOS job polls `http://localhost:8081/status` every 2 s for up to 60 s and logs when Metro is ready. The Android job just sleeps for a fixed 15 s, which is both potentially insufficient on a cold Gradle/emulator start and wastes time when Metro is already up. Using the same polling approach ensures the test suite starts exactly when the bundler is ready.
```suggestion
CI=1 npx expo start --port 8081 --localhost &
for i in $(seq 1 30); do
if curl -sf http://localhost:8081/status 2>/dev/null | grep -q "packager-status:running"; then
echo "Metro ready after $((i * 2))s"
break
fi
sleep 2
done
```
Reviews (1): Last reviewed commit: "feat(expo): add Maestro E2E tests and ni..." | Re-trigger Greptile |
- Split long JSX lines in BalanceCard, TransferForm, and 03-wallet-display to satisfy Biome line-length rules - Convert single quotes to double quotes in getOtp.js and generateEmail.js - Bump Android emulator API level from 29 → 35 to match local verification - Replace fixed sleep(15) with Metro readiness polling (mirrors iOS job) Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
apps/wallets/expo/tests/e2e/specs/auth/login.yaml:11-12
In Maestro, `clearKeychain: true` is iOS-only and is silently ignored on Android. Without `clearState: true`, the Android app's AsyncStorage and SecureStore (where auth tokens are persisted) are never wiped between runs. On a reused emulator — any local developer run after the first — the app opens already authenticated, `email-input` is never shown, and the test fails with a misleading "element not visible" error rather than indicating the state issue.
```suggestion
- launchApp:
clearKeychain: true
clearState: true
```
### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:86-92
**Metro startup timeout falls through silently**
Both the iOS job (here) and the Android job (inside the `reactivecircus` script) poll Metro for 60 s then fall through without error. On a CI runner with an empty Turbo/Metro cache, the initial transform of a large monorepo can exceed 60 s, causing Maestro to run against an unready bundler. The resulting failures appear as `extendedWaitUntil` timeouts ("email-input not visible") rather than a Metro startup error, making the root cause very hard to diagnose. Adding a hard exit after the loop makes the failure mode explicit: `curl -sf http://localhost:8081/status 2>/dev/null | grep -q "packager-status:running" || { echo "::error::Metro bundler did not become ready within 60 s"; exit 1; }`. Apply the same guard in the Android `script:` block.
Reviews (2): Last reviewed commit: "fix(expo): fix Biome formatting and addr..." | Re-trigger Greptile |
Collapse http.post() call to single-line opening per Biome style. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
clearKeychain: true is iOS-only; clearState: true is required to wipe AsyncStorage/SecureStore on Android between test runs, preventing false "element not visible" failures when auth tokens persist across runs. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:87-98
**Metro startup failure passes silently**
If Metro never becomes ready within the 60-second window, the polling loop exits without signalling failure and the job continues directly to Maestro. The Maestro test run then fails with bundler connection errors rather than a clear "Metro did not start" message, making the root cause hard to diagnose. Adding a post-loop check (e.g. `echo "::error::Metro did not start in time" && exit 1`) would surface the real failure immediately. The same gap exists in the Android job's equivalent polling block.
### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:261-295
**`overallFailed` doesn't account for cancelled jobs**
When a job is cancelled (e.g. because it hit the 60-minute timeout), its `result` is `'cancelled'`, not `'failure'`. The current check only covers `'failure'`, so a cancelled job would make the Slack notification display ✅ Passed with 0 tests while the CI run actually did not complete. Adding `|| iosOutcome === 'cancelled' || androidOutcome === 'cancelled'` to `overallFailed` prevents this misleading green signal.
Reviews (4): Last reviewed commit: "fix(expo): add clearState to launchApp f..." | Re-trigger Greptile |
- Remove METRO_PID env var written to GITHUB_ENV but never read - Remove unused step ID run-ios-tests (no step/condition references it) - Update login.yaml comment to mention both clearKeychain and clearState Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers Maestro framework overview, prerequisites (CLI install, running device, Metro, Mailosaur credentials), how to run all/specific tests, test file structure, how the auth flow works, and CI setup. Also fixes cp target in Setup: .env.template → .env.local (matches .gitignore). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove "Auth suite only" run command (covered by "All tests") and "How the auth test works" step list (self-evident from the YAML flow). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:284-285
**Cancelled jobs silently show as "Passed (0 tests)" in Slack**
GitHub Actions job results can be `success`, `failure`, `cancelled`, or `skipped`. The `overallFailed` guard only checks for `'failure'`, so if either job is cancelled — the most likely outcome of a 60-minute timeout while building a native app — Slack receives a `✅ Passed` header with "0 of 0 tests". The same gap exists in `platformSummary`, which falls through to the default `0✅ of 0 tests (0.0s)` line for any non-failure, non-skipped outcome.
```suggestion
const overallFailed = ios.failed > 0 || android.failed > 0
|| iosOutcome === 'failure' || androidOutcome === 'failure'
|| iosOutcome === 'cancelled' || androidOutcome === 'cancelled';
```
### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:294
**`platformSummary` also needs to handle `'cancelled'`**
Companion fix to the `overallFailed` issue above: a cancelled job with `stats.total === 0` falls through to the default summary line (`0✅ of 0 tests`), obscuring that the job never completed. Treat `cancelled` the same as `failure` here.
```suggestion
if ((outcome === 'failure' || outcome === 'cancelled') && stats.total === 0) return `*${label}:*\n\u274C Job failed (build or setup error)`;
```
Reviews (5): Last reviewed commit: "docs(expo): remove CI section from READM..." | Re-trigger Greptile |
albertoelias-crossmint
left a comment
There was a problem hiding this comment.
left some questions and comments to clean it up a little
- Flatten test path: tests/e2e/specs/ → tests/specs/ (removes redundant nesting) - Extract inline Slack script to apps/wallets/expo/scripts/notify-slack.js - Update workflow to call the script instead of inlining 120 lines of JS - Update package.json scripts and README to reflect new paths Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prompt To Fix All With AIFix the following 3 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 3
.github/workflows/e2e-mobile-regression-tests.yml:87-120
**Metro timeout exits silently, masking startup failures**
Both the iOS step (lines 87–97) and the Android inline script (lines 203–211) loop for 30 × 2 s = 60 s waiting for Metro, but neither emits an error nor exits non-zero if the loop exhausts without Metro becoming ready. When that happens, the `maestro test` command runs against an app that can't load its JS bundle, producing "element not found" failures that look like test logic regressions rather than a Metro infrastructure issue. The `|| true` on the Maestro command then suppresses the exit code, and the `if [ ! -f … ]` check becomes the only tripwire — with a message that gives no hint that Metro never started.
Adding a readiness check after each loop — e.g. `if ! curl -sf http://localhost:8081/status 2>/dev/null | grep -q "packager-status:running"; then echo "::error::Metro did not start within 60 s"; exit 1; fi` — would abort early with a clear diagnostic instead of proceeding into the test suite.
### Issue 2 of 3
apps/wallets/expo/README.md:318-402
**README references incorrect paths in two places**
The introductory paragraph says test flows live in `` `tests/e2e/specs/` `` but the actual directory (and every other reference in this file and the CI workflow) is `tests/specs/`. A developer following this sentence would `ls tests/e2e/specs/` and find nothing. Additionally, the directory tree at the bottom lists `scripts/notify-slack.js` as a child of `tests/` when the file actually lives at `apps/wallets/expo/scripts/notify-slack.js`.
### Issue 3 of 3
apps/wallets/expo/scripts/notify-slack.js:1-5
**New Node.js script is JavaScript rather than TypeScript**
The team's convention is to write new code in TypeScript. `notify-slack.js` is a straightforward Node.js script with no GraalJS constraints (unlike `generateEmail.js` and `getOtp.js`, which must be plain JS for the Maestro runtime), so it could be converted to TypeScript with minimal effort. Using `.js` here is an inconsistency against the broader repo standard.
Reviews (6): Last reviewed commit: "refactor(expo): address Alberto's review..." | Re-trigger Greptile |
…rmatting After the polling loop, explicitly check if Metro is actually running and exit with a clear error message if it is not, preventing misleading test failures when Metro silently times out. Also fix Biome formatting in notify-slack.js (long regex line and overallFailed). Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prompt To Fix All With AIFix the following 2 code review issues. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:246-269
**Missing `actions/checkout@v4` in `notify-slack` job**
The `notify-slack` job runs on a fresh `ubuntu-latest` runner with no checkout step, so the workspace is empty when `node apps/wallets/expo/scripts/notify-slack.js` executes. Every nightly run will fail this step with `Error: Cannot find module` / `no such file or directory`. The artifact downloads (`actions/download-artifact@v4`) work without a checkout, but the script file itself must be present in the workspace.
### Issue 2 of 2
apps/wallets/expo/scripts/notify-slack.js:1-5
**New CI script written in JavaScript rather than TypeScript**
The company's codebase convention is to use TypeScript for all new files. The `generateEmail.js` and `getOtp.js` scripts have a valid exemption (they run in Maestro's GraalJS runtime, which only accepts JavaScript), but `notify-slack.js` runs in Node.js on the CI runner and has no such constraint — it could be authored in TypeScript like the rest of the workspace.
Reviews (7): Last reviewed commit: "fix(expo): add explicit Metro readiness ..." | Re-trigger Greptile |
Without it the runner has no code, so `node apps/wallets/expo/scripts/notify-slack.js` fails with MODULE_NOT_FOUND and Slack notifications are never sent. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Prompt To Fix All With AIFix the following 1 code review issue. Work through them one at a time, proposing concise fixes.
---
### Issue 1 of 1
apps/wallets/expo/README.md:36
The prose says `tests/e2e/specs/` but the actual directory (confirmed by `package.json` scripts, the CI workflow, and the tree diagram in this same README) is `tests/specs/`. A developer following the prose to locate or run tests manually won't find the directory.
```suggestion
Test flows live in `tests/specs/` and shared scripting utilities in `tests/shared/utils/`.
```
Reviews (8): Last reviewed commit: "fix(ci): add checkout step to notify-sla..." | Re-trigger Greptile |
|
Good catch from Devin — the |
albertoelias-crossmint
left a comment
There was a problem hiding this comment.
leaving approved. i think there are some pending ai comments that look relevant
CI workflow, Maestro flows, test utilities, Slack script, and .env.maestro are all handled by the crossmint-mobile-e2e-tests repo. This PR now only contains the testID additions needed for Maestro's id: selectors to work. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
|
Reviews (9): Last reviewed commit: "refactor(expo): remove e2e test infra, k..." | Re-trigger Greptile |
Thanks! the scope for this PR was changed, so no new changes are needed |
Summary
Adds
testIDprops to interactive elements in the Expo wallet playground app so Maestro'sid:selector can target them in E2E tests.The Maestro flows, CI workflow, and test utilities live in the crossmint-mobile-e2e-tests repo. This PR only ships the semantic locators the tests depend on.
testIDs added
apps/wallets/expo/src/App.tsxlogout-buttonapps/wallets/expo/snippets/02-login-screen.tsxemail-input,send-code-button,otp-input,verify-buttonapps/wallets/expo/snippets/03-wallet-display.tsxcopy-address-button,wallet-addressapps/wallets/shared/src/components/BalanceCard.tsx{token}-balance,fund-button,refresh-balance-buttonapps/wallets/shared/src/components/TransferForm.tsxtoken-selector-{t},recipient-input,amount-input,transfer-button,successful-tx-linkapps/wallets/shared/src/components/ActivityList.tsxload-activity-buttonapps/wallets/shared/src/components/ApprovalTest.tsxapproval-recipient-input,approval-amount-input,prepare-tx-button,transaction-id-input,approve-tx-button,approve-sig-buttonOther changes
metro.config.js— addsapps/wallets/shared/towatchFoldersso Metro hot-reloads shared component changes.gitignore— excludes.env.localREADME.md— updates env file reference from.envto.env.localTest plan