Add E2E UI smoke test to run in BrowserStack

[rwood-moz]

What changed?

Add separate E2E tests that can run against stormbox stage and prod in BrowserStack. Starting with a UI smoke test that validates and exercises all of the main Webmail UI elements. More UI-based E2E tests will be added in the future that will test Webmail functionality more in-depth.

Why?

The existing E2E tests in /tests/e2e (which run in CI against PRs) require a local stormbox dev stack for DB/cache/JMAP calls, and are not built to run in BrowserStack against a deployed stormbox instance. Adding a separate set of E2E tests specifically to be able to run in BrowserStack against stage and prod.

Details

Added a standalone BrowserStack E2E test package under tests/browserstack for UI-only testing against deployed Stormbox stage/prod. This includes its own package.json, package-lock.json, tsconfig.json, and pinned dependencies for Playwright 1.59.1, BrowserStack SDK 1.57.2, and dotenv 16.6.1.

Implemented desktop and mobile ui-smoke coverage that signs in through the Stormbox UI using TB Pro credentials from .env.browserstack, then verifies the basic authenticated Stormbox UI appears and exercises the elements. Desktop uses a dedicated tests/desktop/auth.desktop.ts setup file to save auth state, while mobile signs in per test.

Added shared constants, auth helpers, and a Stormbox page object for the new suite. Updated the BrowserStack Playwright config to load .env.browserstack and match auth.desktop.ts.

Added tests/browserstack/README.md documenting that these tests target deployed stage/prod only, not the local stack, with setup and run instructions for local Playwright and BrowserStack runs. Added tests/browserstack/.env.browserstack to .gitignore so credentials stay local.

Also added corresponding GHA workflows to run the new tests nightly in BrowserStack on desktop Firefox, Chromium, and Safari, as well as Chrome on Android mobile.

Limitations and Notes

Stormbox requires Android Chrome 148+ (sharedworkers) which BrowserStack doesn't support yet, so when the test runs on Android it will skip if sharedworkers/148+ is not supported.

Worked with Codex AI to get the initial framework and test in place, then further refined the code and the test myself.

Applicable Issues

Fixes #17.

QA Log

Ran this new set of E2E tests:

• Firefox desktop on my local machine
• Playwright's simulated google pixel 10 viewport on my local machine
• Firefox desktop on OSX on BrowserStack
• Chromium desktop on Win 11 on BrowserStack
• Safari desktop on OSX on BrowserStack
• Android chrome on a Google Pixel 10 device on BrowserStack * note: test skips as noted above *

[Sancus]

How come this isn't a matrixed workflow? you can do stuff like:

strategy:
      fail-fast: false
      max-parallel: 4
      matrix:
        include:
          - label: Desktop Chromium
            script: e2e:browserstack:desktop:chrome
          - label: Desktop Firefox
            script: e2e:browserstack:desktop:firefox
          - label: Desktop Safari
            script: e2e:browserstack:desktop:safari
          - label: Android Chrome
            script: e2e:browserstack:mobile:android:chrome

I think that would save a lot of repetition?

[rwood-moz]

Interesting, thanks.

So the reason why the workflows are split up into their own files is so that we can manually trigger the tests (in the GHA UI) on just one single platform instead of having to run all of them. This is useful if the tests are failing on one platform only and I want to rerun or debug. Also with separate workflows we get the workflow history for each of the platforms too. However I'll look into moving a bunch of the repetitive workflow code into a reusable workflow that can be called with params/inputs by each of the platform workflows, and make them small.

[Sancus]

Fable thinks this message doesn't align with Playwright's output and should be if (!/browserstack_error: Timed out waiting for event|page\.goto: Timeout \d+ms exceeded/i.test(message)) {? Not sure if correct.