test: version-pinned end-to-end suite for generated apps

[dawsontoth]

What & why

A way to test create-harper's generated apps against real Harper, with the Harper version under test as a knob — so we catch changed API surfaces, instability, and defaults create-harper should follow before a Harper release breaks a scaffolded app.

It's the deeper tier above the existing per-PR runtimeSmoke.js; both now share one isolated-Harper boot module.

How a run works

node template.tests/e2e/run.js --template <name> [--harper <spec> | --harper-ref <ref>]

1. Generate the app with the real CLI (throwaway temp dir)
2. Overlay e2e fixtures (schema + custom resource + a framework component that fetches the API)
3. Install app deps
4. Resolve Harper (no global install) — see below
5. Build the app if it has a build step
6. Playwright boots one isolated Harper, seeds a record, runs the specs

Picking the Harper version (precedence)

Mechanism	Purpose
HARPER_BIN=…	Reuse an existing install/build (a .js path is launched via node)
--harper-ref <sha|branch|tag>	Unpublished Harper built from a commit — clones harperfast/harper (public; override via --harper-repo), npm install + npm run build, runs dist/bin/harper.js
--harper <spec> (default latest)	Any npm spec: latest, next, 5.2.0-beta.1, a tarball …

Scenarios

Every template runs the API specs (framework-agnostic Harper surface); SPA templates also run the frontend + browser specs.

Spec	Proves	Templates
crud	Schema-driven auto-REST lifecycle (create/read/update/filter/delete), exact status codes asserted as a version canary	all
resource	A custom resources/ class loads and answers GET with JSON	all
frontend	The frontend is served at /	SPA
browser	A real React/Vue/vanilla component fetches the API and renders it in a browser	SPA

CI

• Per PR — unchanged: the fast runtimeSmoke.js check (integration.yaml) against latest Harper.
• Nightly + on demand — e2e.yaml: scheduled latest and next across a representative template subset (vanilla-ts, react-ts, vue-ts, react-ts-ssr), files a deduped e2e-canary issue on failure. workflow_dispatch inputs: harper-version, harper-ref, template.

No cruft in users' apps

Fixtures live under template.tests/ — outside template-*/ (so scaffolding can't copy them) and outside the npm files allowlist (so they're not published). Verified: npm pack --dry-run ships zero e2e/fixture files.

Validated locally (real Harper)

• react-ts @ 5.1.22 — 4/4 (incl. browser rendering live API data)
• vanilla-ts @ 5.1.22 — 4/4 (plain-DOM overlay, no build step)
• react-ts @ next (5.2.0-beta.1) — 4/4
• react-ts @ --harper-ref (5.2.0-beta.1 built from source) — 4/4
• react-ts-ssr @ 5.1.22 — API specs pass, frontend/browser correctly skipped → green
• format, lint, and all 213 unit tests still pass

Findings this already surfaced

• No GraphQL query-over-HTTP endpoint in Harper (/graphql → 404). "GraphQL" = schema DSL + auto REST/WS; dropped that scenario rather than assert a nonexistent surface.
• SSR / returns 404 under harper run (SSR templates set static.index:false; / is served by the entry-server). Treated as an unverified SSR concern, not a create-harper regression — the frontend/browser specs are gated off for SSR (E2E_SSR), which run the API specs only. Confirming intended SSR serving is a good first use of the harness.
• Harper main doesn't pass its own tsc typecheck right now (TS2345 in resources/DatabaseTransaction.ts). Harper ships anyway — build-tools/build.sh runs npm run build || true since tsc still emits dist/ — so run.js mirrors that: tolerate the build's exit code, require the entrypoint was emitted.

Notes / decisions to weigh

• Adds @playwright/test as a devDependency (+ npx playwright install chromium in CI, cached).
• The --harper-ref source build is the heavy path (full install + tsc emit, ~1–2 min); it's on-demand only, not in the nightly schedule.
• Source builds use a shallow fetch-by-SHA (works on GitHub); a mirror that disallows arbitrary-SHA fetches would need a full-clone fallback.
• Commit typed test: (patch bump per .releaserc.json's type: * rule), since this is internal tooling, not a user-facing feature.

🤖 Generated with Claude Code

[gemini-code-assist]

Code Review

This pull request introduces a comprehensive end-to-end testing suite using Playwright, which scaffolds a real application, overlays fixtures, boots an isolated Harper instance, and runs browser and API specs. It also refactors the existing runtime smoke test to share the new Harper booting logic. The review feedback highlights several critical improvements: registering signal handlers (SIGINT/SIGTERM) in both the Harper boot utility and the runner script to ensure proper cleanup of detached processes and temporary directories on interruption; ensuring heavy temporary directories (harperPrefix and harperSrc) are always cleaned up even when --keep is specified; and noting that git fetch will fail with short SHAs, requiring documentation or handling for full 40-character SHAs.