Skip to content

test(smoke): setup playwright test suite#684

Draft
vinzenzLIFI wants to merge 15 commits intomainfrom
test/emb-331-create-playwright-test-suite
Draft

test(smoke): setup playwright test suite#684
vinzenzLIFI wants to merge 15 commits intomainfrom
test/emb-331-create-playwright-test-suite

Conversation

@vinzenzLIFI
Copy link
Copy Markdown

@vinzenzLIFI vinzenzLIFI commented Apr 2, 2026

Which Linear task is linked to this PR?

EMB-330 — CI: Smoke test examples on dependency and code changes

Why was it implemented this way?

What this does

Adds a Playwright E2E smoke test suite and a GitHub Actions workflow that verifies widget examples still render correctly when dependencies or example code changes.

The smoke tests verify 3 things against each example app:

  1. The widget container renders with the Exchange heading, From/To buttons, and Send input
  2. The Settings view opens via the cog icon, all setting rows are present, and back-navigation works
  3. Token selection works end-to-end — opens the From selector, picks a token, opens the To selector, picks a second token, both reflected in the Exchange view

The tests use a Component Object Model (not traditional POM — the widget is a single-page component with internal navigation, no URL-based page transitions). Selectors use ARIA roles and accessible names scoped to the widget root ([id^="widget-app-expanded-container"]), making them framework-agnostic. No data-testid attributes or source code changes to examples are required.

The CI workflow (examples-smoke-test.yml) runs on PRs with the check-examples label:

  1. Detects what changed — shared dependency change (any packages/* file, lockfile, root config, or e2e/ change) triggers all 10 examples; an isolated examples/<name>/ change triggers only that example
  2. Builds and tests in parallel — each affected example gets its own matrix shard (build → serve → run Playwright tests)
  3. Uploads Playwright reports on failure as artifacts for debugging

Why only 10 examples?

All 10 that are compatible have been verified:

  • vite, connectkit, nextjs, nextjs15, privy, privy-ethers, rainbowkit, reown, svelte, zustand-widget-config

The remaining examples are excluded for documented reasons:

Category Examples Reason
Build/serve failures nextjs14, nextjs14-page-router, nuxt, react-router-7, remix Pre-existing dependency issues (missing @metamask/connect-evm, Rollup polyfill error, ESM shim rejection)
Different widget mode deposit-flow, nft-checkout subvariant: 'custom' renders "Deposit"/"NFT Checkout" UI instead of "Exchange" — needs its own test assertions
Iframe isolation vite-iframe, vite-iframe-wagmi Widget inside <iframe> — requires page.frameLocator() variant
Non-root route tanstack-router Widget at /widget, not /
Auth-gated dynamic Requires wallet auth init before widget renders
Vue wrapper vue React-in-Vue (veaury) bridge doesn't expose widget root ID
Empty nextjs-page-router No package.json

Each can be addressed incrementally. The 10 covered examples already catch the most important signal: "did a package change break the widget in a real framework integration?"

Why no auto-approve?

The workflow is gated behind the check-examples label intentionally. Auto-approve was considered (EMB-332) but deferred because of an unresolved concern:

If the workflow triggers automatically on every PR (no label), it would only test examples — but the PR might contain broader changes (backend logic, build config, other packages). Auto-approving based solely on example smoke tests passing could give a false sense of security for PRs that touch more than just examples.

The label-scoped approach is safer: a human explicitly says "this PR needs example verification", the workflow runs, and the results are visible. But even with the label, someone could add it to any PR and get a green check that only reflects example health — not the full PR quality.

Whether to auto-approve, and under what conditions, is an open discussion point. Options include: auto-approve only when the PR exclusively touches examples/ paths, require a separate approval for non-example changes, or keep it manual.

Note on selector robustness

The selectors use ARIA roles and visible text (e.g. getByRole('button', { name: /^From/ })) rather than data-testid attributes. This is intentional — we want to add data-testid to the widget source in the future, but they would need to be stripped from production builds since the widget is a library consumed by third parties. Since the smoke tests run against built example apps (not dev mode), data-testid attributes would not be available at test time unless we also solve the build-stripping problem first.

The current ARIA-based selectors are stable enough for smoke-level verification but may need updates if i18n strings or accessible names change. This is a known tradeoff documented in the README.

Visual showcase (Screenshots or Videos)

N/A — CI workflow, no UI changes.

Checklist before requesting a review

  • I have performed a self-review and testing of my code.
  • This pull request is focused and addresses a single problem.
  • If this PR modifies the Widget API or adds new features that require documentation, I have updated the documentation in the public-docs repository.

@vinzenzLIFI vinzenzLIFI force-pushed the test/emb-331-create-playwright-test-suite branch from afedcc4 to 294e07c Compare April 14, 2026 08:39
@vinzenzLIFI vinzenzLIFI force-pushed the test/emb-331-create-playwright-test-suite branch from 294e07c to e6456d3 Compare May 5, 2026 15:28
vinzenzLIFI and others added 7 commits May 6, 2026 14:27
The smoke spec mixed playground-only tests with @example-tagged tests
intended for portability. Split it cleanly:

- Move playground tests to tests/playground/smoke.spec.ts and drop the
  @example tags
- Add testIgnore for tests/profiles/** so the playground config does
  not pick up the upcoming examples profile specs
- Start widget-playground-vite automatically via webServer; reuse an
  existing server locally
- Repoint the e2e smoketest script to tests/playground and remove the
  now-unused smoke:example passthrough at root and in e2e/

Co-authored-by: Cursor <cursoragent@cursor.com>
Introduce a second Playwright config dedicated to the example apps in
examples/. Each active example becomes its own Playwright project with
a baseURL bound to its serve port and a profile spec that reflects how
the widget renders.

- e2e/examples.config.ts is the single source of truth: each entry
  declares package, port, build/serve commands, mountPath, and profile
- e2e/playwright.examples.config.ts generates one project per active
  example. No webServer block — the server lifecycle is owned by the
  caller (local scripts or CI) so the same suite runs uniformly in
  both contexts
- Three profile specs cover today's shapes:
    * widget-smoke — standard + routed (mountPath from project metadata)
    * iframe       — LiFiWidgetLight wrapping
    * nft          — NFT checkout subvariant
- Add e2e:examples / e2e:example scripts (passthrough at root, real
  invocation in e2e/) that take the new config explicitly so the
  default playground config stays untouched
- Ignore playwright-report-examples/ alongside playwright-report/

Co-authored-by: Cursor <cursoragent@cursor.com>
Add bash orchestrators that build a single example, start its preview
server on a known port, run its Playwright project, and tear the
server down — both for one-shot use and a sequential loop over every
active example.

- scripts/test-example.sh: build → serve → wait-for-ready → run
  Playwright with --project <name> against the new examples config →
  kill server. Mirrors the per-example metadata in
  e2e/examples.config.ts so local runs match CI exactly
- scripts/test-all-examples.sh: invokes the per-example script for
  every active entry sequentially; exit code surfaces the first
  failure
- Wire pnpm test:example <name> and pnpm test:examples at the root

Co-authored-by: Cursor <cursoragent@cursor.com>
Swap the old single-job examples-smoke-test workflow for one that
mirrors how local runs work and only tests what changed.

- detect-changes job computes which examples need testing: a change
  inside examples/<name>/ runs that example, while a change to
  packages/widget/**, packages/wallet-management/**,
  packages/widget-provider*/**, or e2e/** runs all 17 active
  examples
- Each selected example runs as its own matrix job (parallel,
  fail-fast: false) so one broken example does not mask the others
- Each job builds workspace packages, installs example deps, builds
  the example, starts its preview server with PORT honoured for
  remix/react-router-7, waits for readiness, then runs the matching
  Playwright project against the examples config

Drop the old examples-smoke-test.yml — the new workflow supersedes it
and the playground smoke path is now covered by the playground suite.

Co-authored-by: Cursor <cursoragent@cursor.com>
Rewrite e2e/README.md to describe the two coexisting suites and how
to run, extend, and debug them.

- Up-front summary table mapping each suite to its config and what it
  tests
- Prerequisite section calling out the workspace package build that
  must run before example tests, including the exact pnpm filter
  command CI uses (avoids stale dist artifacts on first runs)
- Per-profile assertion summary for standard / routed / iframe / nft
- Per-example build/serve notes covering the vite-build vs build
  distinction (MUI v7 props + @lifi/types mismatch), the PORT env var
  for remix and react-router-7, and Next.js / Nuxt defaults
- Known broken examples table linking to the EMB-349 / EMB-350 /
  EMB-351 tickets, with the unblocking criterion
- Stale-directory note pointing at the leftover examples/nextjs14*
  scaffolds that are intentionally not wired up
- Architecture, selector strategy, and CI sections kept

Drop the now-unused e2e/.env.test.example placeholder; the playground
config reads BASE_URL straight from the environment and the README
no longer references .env.test.

Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerate pnpm-lock.yaml so it matches the e2e workspace as it has
been since e6456d3 (fix: duplicate node and ts dependencies for e2e
tests), which removed @types/node and typescript from
e2e/package.json without re-running pnpm install. The remaining diff
is peer-graph re-hoisting: @playwright/test now appears as a peer
qualifier on the resolved next 15.5.15 / 16.2.4 entries (next-app
examples consume both), and eventemitter3 floats from 5.0.1 to 5.0.4
within its declared range.

No package.json changes — purely the lockfile catching up so future
installs are reproducible.

Co-authored-by: Cursor <cursoragent@cursor.com>
… tests

Raise expect.timeout to 15s in playwright.examples.config.ts so widget
root assertions don't time out on slow CI runners.

Add webServer to playwright.config.ts so the playground dev server starts
automatically when running playground tests locally.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant