Skip to content

feat(expo): add semantic locators for React native SDK to be used in E2E Mobile tests#1859

Merged
luans-qa merged 12 commits into
mainfrom
feat/expo-maestro-e2e-setup
May 28, 2026
Merged

feat(expo): add semantic locators for React native SDK to be used in E2E Mobile tests#1859
luans-qa merged 12 commits into
mainfrom
feat/expo-maestro-e2e-setup

Conversation

@luans-qa
Copy link
Copy Markdown
Contributor

@luans-qa luans-qa commented May 26, 2026

Summary

Adds testID props to interactive elements in the Expo wallet playground app so Maestro's id: selector can target them in E2E tests.

The Maestro flows, CI workflow, and test utilities live in the crossmint-mobile-e2e-tests repo. This PR only ships the semantic locators the tests depend on.

testIDs added

File IDs
apps/wallets/expo/src/App.tsx logout-button
apps/wallets/expo/snippets/02-login-screen.tsx email-input, send-code-button, otp-input, verify-button
apps/wallets/expo/snippets/03-wallet-display.tsx copy-address-button, wallet-address
apps/wallets/shared/src/components/BalanceCard.tsx {token}-balance, fund-button, refresh-balance-button
apps/wallets/shared/src/components/TransferForm.tsx token-selector-{t}, recipient-input, amount-input, transfer-button, successful-tx-link
apps/wallets/shared/src/components/ActivityList.tsx load-activity-button
apps/wallets/shared/src/components/ApprovalTest.tsx approval-recipient-input, approval-amount-input, prepare-tx-button, transaction-id-input, approve-tx-button, approve-sig-button

Other changes

  • metro.config.js — adds apps/wallets/shared/ to watchFolders so Metro hot-reloads shared component changes
  • .gitignore — excludes .env.local
  • README.md — updates env file reference from .env to .env.local

Test plan

  • Login flow passes on iOS simulator (iPhone 16e, iOS 26.2) — verified locally
  • Login flow passes on Android emulator (Medium_Phone_API_36.1, API 36) — verified locally

- Add Maestro login flow test for iOS and Android (auth/login.yaml)
- Add shared utils for dynamic email generation and Mailosaur OTP retrieval
- Add testIDs to login screen, wallet display, and shared wallet components
- Wire Metro to watch shared/ workspace directory
- Add .env.maestro template for local test runs
- Add nightly GitHub Actions workflow (e2e-mobile-regression-tests.yml):
  parallel iOS (macos-latest) and Android (ubuntu-latest + KVM) jobs,
  JUnit XML output, and combined Slack notification to #team-qa-regression-test-alerts

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@changeset-bot
Copy link
Copy Markdown

changeset-bot Bot commented May 26, 2026

⚠️ No Changeset found

Latest commit: 5e17d59

Merging this PR will not cause a version bump for any packages. If these changes should not result in a new version, you're good to go. If these changes should result in a version bump, you need to add a changeset.

This PR includes no changesets

When changesets are added to this PR, you'll see the packages that this PR includes changesets for and the associated semver types

Click here to learn what changesets are, and how to add one.

Click here if you're a maintainer who wants to add a changeset to this PR

@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 26, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:195-197
**Android emulator API level doesn't match local verification**

The PR description states that the login flow was verified locally on API 36 (`Medium_Phone_API_36.1`), but the CI emulator is configured with `api-level: 29`. This means CI will run against a meaningfully older Android version than the one that was tested. Differences in network security policy enforcement, WebView behavior, and system permission dialogs between API 29 and API 36 can cause the Maestro test to behave differently — potentially producing false passes or confusing failures that don't reflect the locally-verified behavior.

### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:200-201
**Android Metro startup uses a fixed sleep instead of the polling used for iOS**

The iOS job polls `http://localhost:8081/status` every 2 s for up to 60 s and logs when Metro is ready. The Android job just sleeps for a fixed 15 s, which is both potentially insufficient on a cold Gradle/emulator start and wastes time when Metro is already up. Using the same polling approach ensures the test suite starts exactly when the bundler is ready.

```suggestion
                      CI=1 npx expo start --port 8081 --localhost &
                      for i in $(seq 1 30); do
                        if curl -sf http://localhost:8081/status 2>/dev/null | grep -q "packager-status:running"; then
                          echo "Metro ready after $((i * 2))s"
                          break
                        fi
                        sleep 2
                      done
```

Reviews (1): Last reviewed commit: "feat(expo): add Maestro E2E tests and ni..." | Re-trigger Greptile

greptile-apps[bot]

This comment was marked as resolved.

@luans-qa luans-qa self-assigned this May 26, 2026
- Split long JSX lines in BalanceCard, TransferForm, and 03-wallet-display
  to satisfy Biome line-length rules
- Convert single quotes to double quotes in getOtp.js and generateEmail.js
- Bump Android emulator API level from 29 → 35 to match local verification
- Replace fixed sleep(15) with Metro readiness polling (mirrors iOS job)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 26, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
apps/wallets/expo/tests/e2e/specs/auth/login.yaml:11-12
In Maestro, `clearKeychain: true` is iOS-only and is silently ignored on Android. Without `clearState: true`, the Android app's AsyncStorage and SecureStore (where auth tokens are persisted) are never wiped between runs. On a reused emulator — any local developer run after the first — the app opens already authenticated, `email-input` is never shown, and the test fails with a misleading "element not visible" error rather than indicating the state issue.

```suggestion
- launchApp:
    clearKeychain: true
    clearState: true
```

### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:86-92
**Metro startup timeout falls through silently**

Both the iOS job (here) and the Android job (inside the `reactivecircus` script) poll Metro for 60 s then fall through without error. On a CI runner with an empty Turbo/Metro cache, the initial transform of a large monorepo can exceed 60 s, causing Maestro to run against an unready bundler. The resulting failures appear as `extendedWaitUntil` timeouts ("email-input not visible") rather than a Metro startup error, making the root cause very hard to diagnose. Adding a hard exit after the loop makes the failure mode explicit: `curl -sf http://localhost:8081/status 2>/dev/null | grep -q "packager-status:running" || { echo "::error::Metro bundler did not become ready within 60 s"; exit 1; }`. Apply the same guard in the Android `script:` block.

Reviews (2): Last reviewed commit: "fix(expo): fix Biome formatting and addr..." | Re-trigger Greptile

greptile-apps[bot]

This comment was marked as resolved.

Collapse http.post() call to single-line opening per Biome style.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 26, 2026

Security Review

  • Unverified remote code execution in CI (.github/workflows/e2e-mobile-regression-tests.yml, lines 101–103 and 185–187): both the iOS and Android jobs install Maestro via curl -Ls "https://get.maestro.mobile.dev" | bash. This runs unversioned, unchecksumed code from a remote server inside the CI environment where MAILOSAUR_API_KEY, SERVER_ID_MAILOSAUR, and NEXT_PUBLIC_CROSSMINT_AUTH_SMART_WALLET_API_KEY are present as environment variables. A supply-chain compromise or MITM at this endpoint would grant full access to those secrets.
Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:87-98
**Metro startup timeout exits silently**

If Metro doesn't respond within 60 s the loop exits without failing the step, and Maestro runs immediately afterwards against a non-functional bundler. Every test case will report cryptic failures (blank screen, element not found) with no indication that Metro was the culprit. The same pattern exists in the Android script block. Adding a guard after the loop makes CI failures immediately diagnosable.

### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:100-103
**Unversioned `curl | bash` for Maestro installation**

Both the iOS and Android jobs (lines 100–103 and 184–187) install Maestro by piping an unversioned install script directly from `get.maestro.mobile.dev` into `bash`. This gives the remote server — or anyone who compromises it — arbitrary code execution in the CI environment alongside your Mailosaur and Crossmint API secrets. Consider pinning a specific Maestro release and verifying its checksum, or using the official `mobile-dev-inc/action-maestro-cloud` GitHub Action which handles installation safely.

Reviews (3): Last reviewed commit: "fix(expo): fix remaining Biome format er..." | Re-trigger Greptile

greptile-apps[bot]

This comment was marked as resolved.

clearKeychain: true is iOS-only; clearState: true is required to wipe
AsyncStorage/SecureStore on Android between test runs, preventing false
"element not visible" failures when auth tokens persist across runs.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 26, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:87-98
**Metro startup failure passes silently**

If Metro never becomes ready within the 60-second window, the polling loop exits without signalling failure and the job continues directly to Maestro. The Maestro test run then fails with bundler connection errors rather than a clear "Metro did not start" message, making the root cause hard to diagnose. Adding a post-loop check (e.g. `echo "::error::Metro did not start in time" && exit 1`) would surface the real failure immediately. The same gap exists in the Android job's equivalent polling block.

### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:261-295
**`overallFailed` doesn't account for cancelled jobs**

When a job is cancelled (e.g. because it hit the 60-minute timeout), its `result` is `'cancelled'`, not `'failure'`. The current check only covers `'failure'`, so a cancelled job would make the Slack notification display ✅ Passed with 0 tests while the CI run actually did not complete. Adding `|| iosOutcome === 'cancelled' || androidOutcome === 'cancelled'` to `overallFailed` prevents this misleading green signal.

Reviews (4): Last reviewed commit: "fix(expo): add clearState to launchApp f..." | Re-trigger Greptile

luans-qa and others added 4 commits May 26, 2026 17:02
- Remove METRO_PID env var written to GITHUB_ENV but never read
- Remove unused step ID run-ios-tests (no step/condition references it)
- Update login.yaml comment to mention both clearKeychain and clearState

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Covers Maestro framework overview, prerequisites (CLI install, running
device, Metro, Mailosaur credentials), how to run all/specific tests,
test file structure, how the auth flow works, and CI setup.

Also fixes cp target in Setup: .env.template → .env.local (matches .gitignore).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Remove "Auth suite only" run command (covered by "All tests") and
"How the auth test works" step list (self-evident from the YAML flow).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 26, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:284-285
**Cancelled jobs silently show as "Passed (0 tests)" in Slack**

GitHub Actions job results can be `success`, `failure`, `cancelled`, or `skipped`. The `overallFailed` guard only checks for `'failure'`, so if either job is cancelled — the most likely outcome of a 60-minute timeout while building a native app — Slack receives a `✅ Passed` header with "0 of 0 tests". The same gap exists in `platformSummary`, which falls through to the default `0✅ of 0 tests (0.0s)` line for any non-failure, non-skipped outcome.

```suggestion
                  const overallFailed = ios.failed > 0 || android.failed > 0
                      || iosOutcome === 'failure' || androidOutcome === 'failure'
                      || iosOutcome === 'cancelled' || androidOutcome === 'cancelled';
```

### Issue 2 of 2
.github/workflows/e2e-mobile-regression-tests.yml:294
**`platformSummary` also needs to handle `'cancelled'`**

Companion fix to the `overallFailed` issue above: a cancelled job with `stats.total === 0` falls through to the default summary line (`0✅ of 0 tests`), obscuring that the job never completed. Treat `cancelled` the same as `failure` here.

```suggestion
                      if ((outcome === 'failure' || outcome === 'cancelled') && stats.total === 0) return `*${label}:*\n\u274C Job failed (build or setup error)`;
```

Reviews (5): Last reviewed commit: "docs(expo): remove CI section from READM..." | Re-trigger Greptile

Copy link
Copy Markdown
Collaborator

@albertoelias-crossmint albertoelias-crossmint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

left some questions and comments to clean it up a little

Comment thread .github/workflows/e2e-mobile-regression-tests.yml Outdated
Comment thread .github/workflows/e2e-mobile-regression-tests.yml Outdated
Comment thread .github/workflows/e2e-mobile-regression-tests.yml Outdated
Comment thread apps/wallets/expo/tests/specs/auth/login.yaml Outdated
Comment thread apps/wallets/expo/tests/specs/auth/login.yaml Outdated
- Flatten test path: tests/e2e/specs/ → tests/specs/ (removes redundant nesting)
- Extract inline Slack script to apps/wallets/expo/scripts/notify-slack.js
- Update workflow to call the script instead of inlining 120 lines of JS
- Update package.json scripts and README to reflect new paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 27, 2026

Prompt To Fix All With AI
Fix the following 3 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 3
.github/workflows/e2e-mobile-regression-tests.yml:87-120
**Metro timeout exits silently, masking startup failures**

Both the iOS step (lines 87–97) and the Android inline script (lines 203–211) loop for 30 × 2 s = 60 s waiting for Metro, but neither emits an error nor exits non-zero if the loop exhausts without Metro becoming ready. When that happens, the `maestro test` command runs against an app that can't load its JS bundle, producing "element not found" failures that look like test logic regressions rather than a Metro infrastructure issue. The `|| true` on the Maestro command then suppresses the exit code, and the `if [ ! -f … ]` check becomes the only tripwire — with a message that gives no hint that Metro never started.

Adding a readiness check after each loop — e.g. `if ! curl -sf http://localhost:8081/status 2>/dev/null | grep -q "packager-status:running"; then echo "::error::Metro did not start within 60 s"; exit 1; fi` — would abort early with a clear diagnostic instead of proceeding into the test suite.

### Issue 2 of 3
apps/wallets/expo/README.md:318-402
**README references incorrect paths in two places**

The introductory paragraph says test flows live in `` `tests/e2e/specs/` `` but the actual directory (and every other reference in this file and the CI workflow) is `tests/specs/`. A developer following this sentence would `ls tests/e2e/specs/` and find nothing. Additionally, the directory tree at the bottom lists `scripts/notify-slack.js` as a child of `tests/` when the file actually lives at `apps/wallets/expo/scripts/notify-slack.js`.

### Issue 3 of 3
apps/wallets/expo/scripts/notify-slack.js:1-5
**New Node.js script is JavaScript rather than TypeScript**

The team's convention is to write new code in TypeScript. `notify-slack.js` is a straightforward Node.js script with no GraalJS constraints (unlike `generateEmail.js` and `getOtp.js`, which must be plain JS for the Maestro runtime), so it could be converted to TypeScript with minimal effort. Using `.js` here is an inconsistency against the broader repo standard.

Reviews (6): Last reviewed commit: "refactor(expo): address Alberto's review..." | Re-trigger Greptile

greptile-apps[bot]

This comment was marked as resolved.

…rmatting

After the polling loop, explicitly check if Metro is actually running and
exit with a clear error message if it is not, preventing misleading test
failures when Metro silently times out.

Also fix Biome formatting in notify-slack.js (long regex line and overallFailed).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Comment thread .github/workflows/e2e-mobile-regression-tests.yml Outdated
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 27, 2026

Prompt To Fix All With AI
Fix the following 2 code review issues. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 2
.github/workflows/e2e-mobile-regression-tests.yml:246-269
**Missing `actions/checkout@v4` in `notify-slack` job**

The `notify-slack` job runs on a fresh `ubuntu-latest` runner with no checkout step, so the workspace is empty when `node apps/wallets/expo/scripts/notify-slack.js` executes. Every nightly run will fail this step with `Error: Cannot find module` / `no such file or directory`. The artifact downloads (`actions/download-artifact@v4`) work without a checkout, but the script file itself must be present in the workspace.

### Issue 2 of 2
apps/wallets/expo/scripts/notify-slack.js:1-5
**New CI script written in JavaScript rather than TypeScript**

The company's codebase convention is to use TypeScript for all new files. The `generateEmail.js` and `getOtp.js` scripts have a valid exemption (they run in Maestro's GraalJS runtime, which only accepts JavaScript), but `notify-slack.js` runs in Node.js on the CI runner and has no such constraint — it could be authored in TypeScript like the rest of the workspace.

Reviews (7): Last reviewed commit: "fix(expo): add explicit Metro readiness ..." | Re-trigger Greptile

devin-ai-integration[bot]

This comment was marked as resolved.

Without it the runner has no code, so `node apps/wallets/expo/scripts/notify-slack.js`
fails with MODULE_NOT_FOUND and Slack notifications are never sent.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 27, 2026

Prompt To Fix All With AI
Fix the following 1 code review issue. Work through them one at a time, proposing concise fixes.

---

### Issue 1 of 1
apps/wallets/expo/README.md:36
The prose says `tests/e2e/specs/` but the actual directory (confirmed by `package.json` scripts, the CI workflow, and the tree diagram in this same README) is `tests/specs/`. A developer following the prose to locate or run tests manually won't find the directory.

```suggestion
Test flows live in `tests/specs/` and shared scripting utilities in `tests/shared/utils/`.
```

Reviews (8): Last reviewed commit: "fix(ci): add checkout step to notify-sla..." | Re-trigger Greptile

Copy link
Copy Markdown
Contributor

@devin-ai-integration devin-ai-integration Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 1 new potential issue.

View 7 additional findings in Devin Review.

Open in Devin Review

Comment thread apps/wallets/expo/scripts/notify-slack.js Outdated
@luans-qa
Copy link
Copy Markdown
Contributor Author

Good catch from Devin — the notify-slack job was missing actions/checkout@v4, so node apps/wallets/expo/scripts/notify-slack.js would fail with MODULE_NOT_FOUND on a fresh runner. Fixed in 73edd59.

Copy link
Copy Markdown
Collaborator

@albertoelias-crossmint albertoelias-crossmint left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

leaving approved. i think there are some pending ai comments that look relevant

CI workflow, Maestro flows, test utilities, Slack script, and .env.maestro
are all handled by the crossmint-mobile-e2e-tests repo. This PR now only
contains the testID additions needed for Maestro's id: selectors to work.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
@greptile-apps
Copy link
Copy Markdown
Contributor

greptile-apps Bot commented May 28, 2026

Reviews (9): Last reviewed commit: "refactor(expo): remove e2e test infra, k..." | Re-trigger Greptile

@luans-qa luans-qa changed the title feat(expo): add Maestro E2E tests and nightly CI workflow feat(expo): add semantic locators for React native SDK to be used in E2E Mobile tests May 28, 2026
@luans-qa
Copy link
Copy Markdown
Contributor Author

leaving approved. i think there are some pending ai comments that look relevant

Thanks! the scope for this PR was changed, so no new changes are needed

@luans-qa luans-qa merged commit 296aff5 into main May 28, 2026
2 checks passed
@luans-qa luans-qa deleted the feat/expo-maestro-e2e-setup branch May 28, 2026 20:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants