Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 13 additions & 2 deletions dev.yml
Original file line number Diff line number Diff line change
Expand Up @@ -253,6 +253,11 @@ commands:
desc: Build the android sample app and install it to the booted emulator
run: platforms/android/samples/CheckoutKitAndroidDemo/gradlew -p platforms/android/samples/CheckoutKitAndroidDemo installDebug

e2e:
desc: Run the Android sample Maestro tests, filtered by tag
syntax: "[--tags TAG[,TAG...]] [--exclude-tags TAG[,TAG...]]"
run: ./e2e/scripts/run_local_e2e kotlin-android "$@"

test:
desc: Run all library and demo app tests
run: |
Expand Down Expand Up @@ -399,6 +404,12 @@ commands:
fi
cd Samples
../Scripts/xcode_run build CheckoutKitSwiftDemo

e2e:
desc: Run the Swift sample Maestro tests, filtered by tag
syntax: "[--tags TAG[,TAG...]] [--exclude-tags TAG[,TAG...]]"
run: ./e2e/scripts/run_local_e2e swift-ios "$@"

test:
desc: |
`dev swift test` - Run all tests for the ShopifyCheckoutKit-Package and the sample apps.
Expand Down Expand Up @@ -567,11 +578,11 @@ commands:
ios:
desc: Run the React Native iOS Maestro tests
syntax: "[--tags TAG[,TAG...]] [--exclude-tags TAG[,TAG...]]"
run: cd platforms/react-native && ./scripts/e2e_maestro_ios "$@"
run: ./e2e/scripts/run_local_e2e react-native-ios "$@"
android:
desc: Run the React Native Android Maestro tests
syntax: "[--tags TAG[,TAG...]] [--exclude-tags TAG[,TAG...]]"
run: cd platforms/react-native && ./scripts/e2e_maestro_android "$@"
run: ./e2e/scripts/run_local_e2e react-native-android "$@"
lint:
desc: Run all React Native lint checks (Swift, module, sample)
aliases: [style]
Expand Down
128 changes: 74 additions & 54 deletions e2e/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,45 +3,47 @@
This directory contains Maestro end-to-end flows and configuration for Checkout
Kit sample apps. Two complementary setups live here:

- A **local** React Native suite, run with `dev rn e2e`, that exercises guest and
hardcoded buyer identity checkouts from seeded carts through Shopify checkout
and back to the app. Tags select which tests run.
- **Local** runs, one command per target, that build the sample app, install it on
the booted device, and run the tests in `tests/`. Tags select which tests run.
- A **CI matrix** that expands applications and OS version tags into BrowserStack
Maestro run rows. Every row runs the whole `tests/` folder and tags select what
runs inside it.

## Run locally
Local runs call `scripts/run_local_e2e`, which builds and installs the target
before delegating the Maestro invocation to `scripts/run_maestro`. CI applies the
same environment contract through the BrowserStack run plan.

Run `dev up` first to provision the local toolchain. Install Maestro separately
and make sure `maestro --version` succeeds before running these flows.
## Run locally

### React Native checkout smoke (`dev rn e2e`)
Run `dev up` first to provision the local toolchain, including the pinned Maestro
version. The runners resolve that version through `scripts/maestro_bin`, so a
separate Maestro installation is not needed.

Run the matching command from the repo root.
### One command per target

React Native iOS:
Boot a simulator or emulator first, because Maestro drives the device the app runs
on. Then run the matching command from the repo root.

```bash
dev rn e2e ios
```
| Target | Command |
|---|---|
| Swift iOS | `dev swift e2e` |
| Kotlin Android | `dev android e2e` |
| React Native iOS | `dev rn e2e ios` |
| React Native Android | `dev rn e2e android` |

React Native Android:

```bash
dev rn e2e android
```

Both commands run every test in `tests/`. Narrow a run with `--tags`:
Each command runs the tests in `tests/shared/` and its target namespace under
`tests/`. Narrow a run with `--tags`:

```bash
dev swift e2e --tags cart
dev rn e2e ios --tags checkout
dev rn e2e ios --tags smoke
dev rn e2e android --tags cart,checkout
dev android e2e --tags cart,checkout
```

Both options match **any** listed tag, because that is how Maestro filters.
`--tags cart,checkout` runs the cart tests and the checkout tests. `--exclude-tags`
skips tests carrying any listed tag and defaults to `flaky,wip`.
skips tests carrying any listed tag. `config.yaml` quarantines `flaky` and `wip`
for every run, so those need no command line option.

### Tags

Expand All @@ -58,36 +60,32 @@ enforces it.
A platform tag marks a capability only one platform has, such as Apple Pay. It
must never mark a test that is merely not ported yet.

The React Native commands start Metro if needed, build and launch the target
sample app, then run Maestro. They require the standard storefront `.env` setup,
but the E2E flows seed their own carts through the bootstrap deep link. The
React Native bootstrap link accepts `buyerIdentityMode`, so guest and hardcoded
buyer identity scenarios share the same cart setup path. No manual sample cart
Every command calls `scripts/run_local_e2e`, which selects the device, builds
and installs the target, and then calls `scripts/run_maestro`. React Native targets
also start Metro if needed. All four need the standard storefront `.env` setup,
but the flows seed their own carts through the control link, so no manual cart
setup is required.

### Shared launch smoke
React Native E2E runs should use the released native SDK artifacts declared by
the React Native sample configuration, not local in-repo native SDK overrides.

The launch smoke launches a sample app and waits for the shared ready marker
exposed by that app, using the same environment contract used by CI.
### The control link

React Native iOS:
The samples share one command channel: a deep link on the app's own scheme.

```bash
E2E_APP_ID=com.shopify.checkoutkit.reactnativedemo \
E2E_READY_MARKER=checkout-kit-sample-ready \
maestro --platform ios test e2e/tests/shared/launch-smoke.yaml
```

React Native Android:

```bash
E2E_APP_ID=com.shopify.checkoutkit.reactnativedemo \
E2E_READY_MARKER=checkout-kit-sample-ready \
maestro --platform android test e2e/tests/shared/launch-smoke.yaml
<app_id>://e2e/<command>?<parameters>
```

React Native E2E runs should use the released native SDK artifacts declared by
the React Native sample configuration, not local in-repo native SDK overrides.
The scheme equals the app id on all four targets, so `scripts/run_maestro` derives
`E2E_CONTROL_LINK` rather than taking it as an argument. Commands are `reset`,
`cart` and `signIn`. Each sample parses the link in its own E2E folder and runs the
command through one `E2EController`, so sample code that merchants read holds a
single hook.

The link goes to an app that already runs. A stopped app would need a second entry
point on every platform, because iOS delivers a cold-start URL through the scene
connection options and Android through the launch intent.

## Matrix

Expand All @@ -104,7 +102,7 @@ Current applications:
- Kotlin Android sample app
- Swift iOS sample app

Current OS version tags:
Current OS version tag:

- `latest`

Expand Down Expand Up @@ -145,28 +143,50 @@ ruby e2e/scripts/e2e_matrix_to_browserstack_run_plan count
- `config.yaml` configures Maestro for shared platform behavior and quarantines
the `flaky` and `wip` tags.
- `flows/` contains reusable Maestro subflows for app setup and checkout steps.
- `tests/shared/` holds the tests every target runs through the CI matrix.
- `tests/<platform>/` holds platform-local tests. The matrix may ignore their tags.
- `tests/shared/launch-smoke.yaml` is the shared launch smoke test.
- `tests/shared/cart-from-control-link.yaml` seeds a cart through the control link
and waits for the cart marker.
- `tests/react-native/checkout-guest.yaml` composes the React Native guest
checkout smoke test from those subflows.
- `tests/react-native/checkout-hardcoded-buyer-identity.yaml` verifies checkout
from a bootstrapped cart with hardcoded buyer identity.
- `scripts/run_local_e2e` builds and installs any of the four local targets.
- `scripts/run_maestro` is their single Maestro invocation. It holds the
environment contract and target-specific test-file selection in one place.
- `config/matrix.yml`, `lib/e2e_matrix_to_browserstack_run_plan.rb`, and
`scripts/` drive the BrowserStack run plan.
- `tests/shared/launch-smoke.yaml` is the shared launch smoke test.

Maestro resolves the `flows:` glob in `config.yaml` relative to the path on the
command line. BrowserStack passes the workspace root because `scripts/zip_e2e_tests`
puts `config.yaml`, `tests/` and `flows/` side by side there. Local runs instead
pass the shared and target-specific test files selected by `scripts/run_maestro`.

## Shared app contract

Shared flows rely on stable cross-app identifiers. The launch smoke requires each
target app to expose this ready marker:
Shared flows rely on stable cross-app identifiers. Every target app must expose
these markers:

| Marker | Appears when |
|---|---|
| `checkout-kit-sample-ready` | the app finished launching |
| `cart-checkout-ready` | the cart holds at least one line |

`cart-checkout-ready` is the assertion for the whole control link path. It appears
only after the app parsed the link, resolved a variant, created a cart, added the
line, and navigated to the cart.

- `checkout-kit-sample-ready`
`flows/app/bootstrap-cart-from-link.yaml` takes `E2E_CART_PARAMS`, the query string
for the `cart` command. A test that does not care about buyer identity omits
`buyerIdentityMode` and keeps the app's configured mode.

Future shared flows should add identifiers here before they are used across
React Native, Swift, and Android sample apps.

## Scope

These flows catch regressions in the React Native sample app integration
surface: cart bootstrap, buyer identity configuration, checkout presentation,
checkout completion, and return to the sample app. They are not a replacement
for checkout-web browser-based coverage or for future native Swift and Android
sample-app E2E coverage.
These flows catch regressions in the sample app integration surface on all four
targets: cart bootstrap, buyer identity configuration, checkout presentation,
checkout completion, and return to the sample app. They are not a replacement for
checkout-web browser-based coverage.
1 change: 1 addition & 0 deletions e2e/config/matrix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ tests_path: tests
tags:
include:
- launch
- cart
exclude:
- flaky
- wip
Expand Down
50 changes: 31 additions & 19 deletions e2e/flows/app/bootstrap-cart-from-link.yaml
Original file line number Diff line number Diff line change
@@ -1,21 +1,33 @@
appId: ${E2E_APP_ID}
---
- launchApp:
clearState: true
arguments:
# iOS-only launch arguments; Android ignores them.
AppleLocale: en_US
AppleLanguages: "(en)"
- extendedWaitUntil:
visible:
id: ${E2E_READY_MARKER}
timeout: 10000
# Reset app data before testing the deep-link cold-start path. The app is
# stopped so openLink is the launch that seeds the cart.
- stopApp
- openLink: "${E2E_CONTROL_LINK}/cart?productIndex=0&quantity=1&buyerIdentityMode=${BUYER_IDENTITY_MODE}"
- waitForAnimationToEnd
- extendedWaitUntil:
visible:
id: cart-checkout-ready
timeout: 10000
- runFlow: launch.yaml
# The wait above can match a view hierarchy the dying process left behind, which sends the
# link seconds before the new activity exists. Android then drops it, because a deep link
# has no delivery guarantee. One repeat covers that, and the command is safe to repeat:
# every cart command resets the cart before it seeds one.
- retry:
Comment thread
kieran-osgood-shopify marked this conversation as resolved.
maxRetries: 1
commands:
# The link goes to the running app. launchApp cleared the state above, so the cart is
# already empty, and a running app is what the account journey needs later: iOS delivers
# a cold-start URL through the scene connect options and Android through the launch
# intent, so a stopped app would need a second entry point on every platform.
# Each test supplies E2E_CART_PARAMS, so tests that do not care about buyer
# identity leave the app's configured mode alone.
- openLink: "${E2E_CONTROL_LINK}/cart?${E2E_CART_PARAMS}"
- waitForAnimationToEnd
# iOS asks the user to confirm before it hands a custom scheme to an app. Android
# does not, so the tap is conditional rather than platform-gated.
- runFlow:
when:
visible: "Open"
commands:
- tapOn: "Open"
- waitForAnimationToEnd
# The cart marker only appears after the controller resolved a variant, created a
# cart, added the line, and navigated. Reaching it proves the whole command path.
# That is two network round trips, so the timeout covers a slow device on a slow shop.
- extendedWaitUntil:
visible:
id: cart-checkout-ready
timeout: 60000
45 changes: 45 additions & 0 deletions e2e/flows/app/launch.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
appId: ${E2E_APP_ID}
---
# Clearing state is the default. Set CLEAR_STATE to "false" only when a test needs to
# preserve state from an earlier launch.
- runFlow:
when:
true: ${typeof CLEAR_STATE === 'undefined' || CLEAR_STATE != 'false'}
commands:
- launchApp:
clearState: true
arguments:
# iOS-only launch arguments; Android ignores them.
AppleLocale: en_US
AppleLanguages: "(en)"
- runFlow:
when:
true: ${typeof CLEAR_STATE !== 'undefined' && CLEAR_STATE == 'false'}
commands:
- launchApp:
arguments:
AppleLocale: en_US
AppleLanguages: "(en)"
# Avoid relaunching healthy apps. BrowserStack can report the state-clearing reinstall and
# launch as complete while iOS remains on SpringBoard, so probe briefly before recovering.
- extendedWaitUntil:
visible:
id: ${E2E_READY_MARKER}
timeout: 10000
optional: true
- runFlow:
when:
notVisible:
id: ${E2E_READY_MARKER}
commands:
- retry:
maxRetries: 1
commands:
- launchApp:
arguments:
AppleLocale: en_US
AppleLanguages: "(en)"
- extendedWaitUntil:
visible:
id: ${E2E_READY_MARKER}
timeout: 60000
2 changes: 1 addition & 1 deletion e2e/lib/e2e_matrix_to_browserstack_run_plan.rb
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ def tests_path
# BrowserStack runs Maestro against this path inside the uploaded test suite, and Maestro
# resolves the `flows:` glob in config.yaml relative to it. scripts/zip_e2e_tests puts
# config.yaml, tests/, and flows/ side by side at the suite root, so the root is the
# only path where that glob resolves.
# only path where that glob resolves. The local runners pass the same value.
def workspace_path
"."
end
Expand Down
Loading
Loading