test(smoke): setup playwright test suite#684
Draft
vinzenzLIFI wants to merge 15 commits intomainfrom
Draft
Conversation
afedcc4 to
294e07c
Compare
… is used for comparison
294e07c to
e6456d3
Compare
The smoke spec mixed playground-only tests with @example-tagged tests intended for portability. Split it cleanly: - Move playground tests to tests/playground/smoke.spec.ts and drop the @example tags - Add testIgnore for tests/profiles/** so the playground config does not pick up the upcoming examples profile specs - Start widget-playground-vite automatically via webServer; reuse an existing server locally - Repoint the e2e smoketest script to tests/playground and remove the now-unused smoke:example passthrough at root and in e2e/ Co-authored-by: Cursor <cursoragent@cursor.com>
Introduce a second Playwright config dedicated to the example apps in
examples/. Each active example becomes its own Playwright project with
a baseURL bound to its serve port and a profile spec that reflects how
the widget renders.
- e2e/examples.config.ts is the single source of truth: each entry
declares package, port, build/serve commands, mountPath, and profile
- e2e/playwright.examples.config.ts generates one project per active
example. No webServer block — the server lifecycle is owned by the
caller (local scripts or CI) so the same suite runs uniformly in
both contexts
- Three profile specs cover today's shapes:
* widget-smoke — standard + routed (mountPath from project metadata)
* iframe — LiFiWidgetLight wrapping
* nft — NFT checkout subvariant
- Add e2e:examples / e2e:example scripts (passthrough at root, real
invocation in e2e/) that take the new config explicitly so the
default playground config stays untouched
- Ignore playwright-report-examples/ alongside playwright-report/
Co-authored-by: Cursor <cursoragent@cursor.com>
Add bash orchestrators that build a single example, start its preview server on a known port, run its Playwright project, and tear the server down — both for one-shot use and a sequential loop over every active example. - scripts/test-example.sh: build → serve → wait-for-ready → run Playwright with --project <name> against the new examples config → kill server. Mirrors the per-example metadata in e2e/examples.config.ts so local runs match CI exactly - scripts/test-all-examples.sh: invokes the per-example script for every active entry sequentially; exit code surfaces the first failure - Wire pnpm test:example <name> and pnpm test:examples at the root Co-authored-by: Cursor <cursoragent@cursor.com>
Swap the old single-job examples-smoke-test workflow for one that mirrors how local runs work and only tests what changed. - detect-changes job computes which examples need testing: a change inside examples/<name>/ runs that example, while a change to packages/widget/**, packages/wallet-management/**, packages/widget-provider*/**, or e2e/** runs all 17 active examples - Each selected example runs as its own matrix job (parallel, fail-fast: false) so one broken example does not mask the others - Each job builds workspace packages, installs example deps, builds the example, starts its preview server with PORT honoured for remix/react-router-7, waits for readiness, then runs the matching Playwright project against the examples config Drop the old examples-smoke-test.yml — the new workflow supersedes it and the playground smoke path is now covered by the playground suite. Co-authored-by: Cursor <cursoragent@cursor.com>
Rewrite e2e/README.md to describe the two coexisting suites and how to run, extend, and debug them. - Up-front summary table mapping each suite to its config and what it tests - Prerequisite section calling out the workspace package build that must run before example tests, including the exact pnpm filter command CI uses (avoids stale dist artifacts on first runs) - Per-profile assertion summary for standard / routed / iframe / nft - Per-example build/serve notes covering the vite-build vs build distinction (MUI v7 props + @lifi/types mismatch), the PORT env var for remix and react-router-7, and Next.js / Nuxt defaults - Known broken examples table linking to the EMB-349 / EMB-350 / EMB-351 tickets, with the unblocking criterion - Stale-directory note pointing at the leftover examples/nextjs14* scaffolds that are intentionally not wired up - Architecture, selector strategy, and CI sections kept Drop the now-unused e2e/.env.test.example placeholder; the playground config reads BASE_URL straight from the environment and the README no longer references .env.test. Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerate pnpm-lock.yaml so it matches the e2e workspace as it has been since e6456d3 (fix: duplicate node and ts dependencies for e2e tests), which removed @types/node and typescript from e2e/package.json without re-running pnpm install. The remaining diff is peer-graph re-hoisting: @playwright/test now appears as a peer qualifier on the resolved next 15.5.15 / 16.2.4 entries (next-app examples consume both), and eventemitter3 floats from 5.0.1 to 5.0.4 within its declared range. No package.json changes — purely the lockfile catching up so future installs are reproducible. Co-authored-by: Cursor <cursoragent@cursor.com>
… tests Raise expect.timeout to 15s in playwright.examples.config.ts so widget root assertions don't time out on slow CI runners. Add webServer to playwright.config.ts so the playground dev server starts automatically when running playground tests locally. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which Linear task is linked to this PR?
EMB-330 — CI: Smoke test examples on dependency and code changes
Why was it implemented this way?
What this does
Adds a Playwright E2E smoke test suite and a GitHub Actions workflow that verifies widget examples still render correctly when dependencies or example code changes.
The smoke tests verify 3 things against each example app:
The tests use a Component Object Model (not traditional POM — the widget is a single-page component with internal navigation, no URL-based page transitions). Selectors use ARIA roles and accessible names scoped to the widget root (
[id^="widget-app-expanded-container"]), making them framework-agnostic. Nodata-testidattributes or source code changes to examples are required.The CI workflow (
examples-smoke-test.yml) runs on PRs with thecheck-exampleslabel:packages/*file, lockfile, root config, ore2e/change) triggers all 10 examples; an isolatedexamples/<name>/change triggers only that exampleWhy only 10 examples?
All 10 that are compatible have been verified:
The remaining examples are excluded for documented reasons:
@metamask/connect-evm, Rollup polyfill error, ESM shim rejection)subvariant: 'custom'renders "Deposit"/"NFT Checkout" UI instead of "Exchange" — needs its own test assertions<iframe>— requirespage.frameLocator()variant/widget, not/package.jsonEach can be addressed incrementally. The 10 covered examples already catch the most important signal: "did a package change break the widget in a real framework integration?"
Why no auto-approve?
The workflow is gated behind the
check-exampleslabel intentionally. Auto-approve was considered (EMB-332) but deferred because of an unresolved concern:If the workflow triggers automatically on every PR (no label), it would only test examples — but the PR might contain broader changes (backend logic, build config, other packages). Auto-approving based solely on example smoke tests passing could give a false sense of security for PRs that touch more than just examples.
The label-scoped approach is safer: a human explicitly says "this PR needs example verification", the workflow runs, and the results are visible. But even with the label, someone could add it to any PR and get a green check that only reflects example health — not the full PR quality.
Whether to auto-approve, and under what conditions, is an open discussion point. Options include: auto-approve only when the PR exclusively touches
examples/paths, require a separate approval for non-example changes, or keep it manual.Note on selector robustness
The selectors use ARIA roles and visible text (e.g.
getByRole('button', { name: /^From/ })) rather thandata-testidattributes. This is intentional — we want to adddata-testidto the widget source in the future, but they would need to be stripped from production builds since the widget is a library consumed by third parties. Since the smoke tests run against built example apps (not dev mode),data-testidattributes would not be available at test time unless we also solve the build-stripping problem first.The current ARIA-based selectors are stable enough for smoke-level verification but may need updates if i18n strings or accessible names change. This is a known tradeoff documented in the README.
Visual showcase (Screenshots or Videos)
N/A — CI workflow, no UI changes.
Checklist before requesting a review