Skip to content

e2e: native mode for sessions that cannot build the image - #8

Merged
meltforce merged 2 commits into
mainfrom
claude/e2e-native
Nov 13, 2026
Merged

meltforce merged 2 commits into
mainfrom
claude/e2e-native

Conversation

@meltforce

@meltforce meltforce commented Nov 13, 2026 •

Copy link
Copy Markdown
Owner

Lets a cloud session run the end-to-end checks, including the Playwright tests from #7, before it pushes. Until now it could not: docker build fails there on the egress proxy's certificate, so the image path of scripts/e2e is unavailable. The app is unchanged.

Acceptance criteria

  1. E2E_NATIVE=1 scripts/e2e builds the frontend and the binary (with the build SHA), starts them against Postgres from compose.test.yaml with the fixtures, and runs the HTTP checks and the Playwright tests, without docker build.
  2. The native instance listens on a free loopback port with the health listener off, so it does not collide with anything on 8080/8081.
  3. On exit (success or failure) the app process and the database are removed; on failure the app log is printed.
  4. Without E2E_NATIVE the image path and the upgrade test are unchanged.
  5. CLAUDE.md and the verify skill ask for scripts/e2e on a change under web/, and name the native mode for a session where docker build fails.

Test per criterion

1, 2. E2E_NATIVE=1 scripts/e2e locally (output below; port 64951 was chosen by the script).
3. Success: same run, docker ps -a lists no vimmary-e2e-native-* container afterwards. Failure: scripts/e2e-native_test.sh (run by scripts/lint) starts the native mode with a config the app cannot parse (E2E_CONFIG) and checks the exit code, the early-exit message, the app's startup failed log line and that no container remains. Counter-check: with the tail of the log removed from scripts/e2e, the log check fails (below).
4. The CI e2e job of this PR runs the image path.
5. Documentation only.

Test output

$ time E2E_NATIVE=1 scripts/e2e
==> building the frontend and the binary
==> starting vimmary on http://127.0.0.1:64951
ok   /version reports d12fa191fc87
ok   /version reports a reachable database
ok   /api/v1/videos lists YouTube rows only
ok   /api/v1/videos?source=podcast lists podcast rows only
ok   /api/v1/search finds a fixture by keyword
ok   /api/v1/stats counts the rows
ok   / serves the frontend
==> browser tests (e2e/)
Running 8 tests using 1 worker
  ✓  1 [chromium] › tests/library.spec.ts:11:1 › the library lists the fixture videos and no podcast episodes (443ms)
  ✓  2 [chromium] › tests/library.spec.ts:19:1 › opening a video shows its summary (251ms)
  ✓  3 [chromium] › tests/library.spec.ts:29:1 › search narrows the library to the matching video (203ms)
  ✓  4 [chromium] › tests/library.spec.ts:46:3 › /inbox renders (134ms)
  ✓  5 [chromium] › tests/library.spec.ts:46:3 › /channels renders (133ms)
  ✓  6 [chromium] › tests/library.spec.ts:46:3 › /stats renders (132ms)
  ✓  7 [chromium] › tests/library.spec.ts:46:3 › /settings renders (135ms)
  ✓  8 [chromium] › tests/library.spec.ts:52:1 › without application login the nav offers no sign-out (135ms)
  8 passed (2.1s)
e2e ok (native): d12fa191fc873a5dc23fd316dba36fc6d28115a3
... 17,223 total

$ docker ps -a --format '{{.Names}}' | grep -c e2e-native
0

$ scripts/lint
...
==> script tests
ok   e2e-native: a broken config fails the run (exit 1)
ok   e2e-native: the run names the early exit
ok   e2e-native: the app log is printed
ok   e2e-native: no database container remains
...
lint ok

# counter-check, `tail -n 80 "$tmp/vimmary.log"` replaced by `:` (not committed)
$ scripts/e2e-native_test.sh
ok   e2e-native: a broken config fails the run (exit 1)
ok   e2e-native: the run names the early exit
FAIL e2e-native: the app log is printed
ok   e2e-native: no database container remains

scripts/e2e-native_test.sh needs Docker; without it the test is skipped, except with CI=true.

Not yet run in a cloud session.

Production changes

None

🤖 Generated with Claude Code

A cloud session cannot run scripts/e2e: docker build fails on the egress
proxy's certificate, and E2E_BASE_URL needs an instance the session would
have to assemble itself. E2E_NATIVE=1 builds the frontend and the binary,
starts them against Postgres from compose.test.yaml with the fixtures, and
runs the HTTP checks and the Playwright tests. The upgrade test needs
images and stays with CI.

The instance listens on a free loopback port with the health listener off,
so it does not collide with a running vimmary or other apps on 8080/8081.

CLAUDE.md and the verify skill now ask for scripts/e2e on a change under
web/ as well, since the browser tests cover the frontend.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Review: request changes

Commit d82ee67

This PR adds an E2E_NATIVE=1 mode to scripts/e2e, so a cloud session can run the end-to-end and browser checks without docker build. It also updates CLAUDE.md and the verify skill to say when to use it. The app itself is unchanged and nothing is needed in the homelab repo. The code looks sound, and the image path is untouched when E2E_NATIVE is unset. I request changes on one point. Acceptance criterion 3 says the app log is printed on failure, and the PR text admits no test exercises that branch. A failing run, such as a wrong SHA or an app that exits at startup, should show the log tail and still remove the database. The success case is covered by the shown run.

Blockers

  • scripts/e2e:89 (rule 1): Acceptance criterion 3 (on failure the app log is printed, and the app and database are still removed) has no test. The PR text says the failure branch is not exercised. — Add a test (for example under the scripts tests that lint runs) that forces a failure in native mode, such as a wrong BUILD_SHA or a bad config. It should assert that the log tail is printed and that no vimmary-e2e-native container remains. Alternatively, drop the failure-log claim from the criteria.

@github-actions

Copy link
Copy Markdown

preview: d82ee67 is live at https://knox.coydog-fence.ts.net:8091/

The review of #8 asked for a test of acceptance criterion 3: on a failed
run the app's log is printed and the database container removed.
scripts/e2e-native_test.sh, run by scripts/lint, starts the native mode
with a config the app cannot parse (E2E_CONFIG, new) and checks the exit
code, the early-exit message, the app's startup-failure log line, and that
no vimmary-e2e-native container remains. Without Docker it is skipped,
except in CI.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@github-actions

Copy link
Copy Markdown

Review: approve

Commit 17ce558

This PR adds an E2E_NATIVE=1 mode to scripts/e2e. A cloud session that cannot run docker build can then run the end-to-end checks and the Playwright tests. The app is unchanged. The mode builds the frontend and the binary with the build SHA, then starts the app on a free loopback port with the health listener off. It uses Postgres from compose.test.yaml. On exit, success or failure, it removes the app process and the database, and on failure it prints the app log. I read the code and did not run it. Each acceptance criterion has a matching check: the run output covers 1–2, the new scripts/e2e-native_test.sh covers the failure path (added to scripts/lint), the unchanged image path covers 4, and the documentation edits cover 5. The test output in the PR text comes from the commands it names. No migrations, config keys, variables or volumes are added, so nothing is needed in the homelab repo, and nothing weakens a check or touches /version or /healthz behaviour. The new lint test needs Docker; it is skipped without it, and fails with CI=true, which is the right behaviour.

@github-actions

Copy link
Copy Markdown

preview: 17ce558 is live at https://knox.coydog-fence.ts.net:8091/

@meltforce
meltforce merged commit 0310986 into main Nov 13, 2026
6 checks passed
@meltforce
meltforce deleted the claude/e2e-native branch November 13, 2026 17:10
@github-actions

Copy link
Copy Markdown

live: 0310986 is deployed to production (run).

This branch was previously deployed

1 inactive deployment
preview — 17ce5589 Deployed Nov 13, 2026 by meltforce via deploy #10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant