diff --git a/.github/workflows/publish.yml b/.github/workflows/publish.yml index 8ab38b1..8bf0156 100644 --- a/.github/workflows/publish.yml +++ b/.github/workflows/publish.yml @@ -28,6 +28,14 @@ jobs: node-version: "20.x" cache: "npm" + # The environment-capture tests drive real tmux servers. They skip when the + # binary is absent locally, so CI installs it to keep that coverage running. + - name: Install tmux + run: | + sudo apt-get update + sudo apt-get install -y tmux + tmux -V + - name: Install dependencies run: npm ci diff --git a/CLAUDE.md b/CLAUDE.md index dc670c4..b564fba 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -10,6 +10,8 @@ npm test # Run vitest once npm run dev # Watch mode build ``` +**Optional test prerequisite: `tmux`.** The environment-capture integration tests in `src/environment/runtime.test.ts` drive real tmux servers. They skip themselves when `tmux` is not on `PATH`, except when `CI` is set — CI installs tmux explicitly, so a missing binary there is a failure, not a skip. + ## Architecture ``` @@ -18,8 +20,10 @@ src/ ├── commands/ # One file per CLI command (install, start, stop, exec, diff, pr, clean) ├── browser/ # agent-browser CLI wrappers (session, capture, interact, navigate) ├── server/ # Dev server detection, startup, port waiting +├── environment/ # Owned tmux panes, direct processes, file tails, evidence capture ├── session/state.ts # .session.json lifecycle (save/load/clear) ├── session/metadata.ts # Persistent per-session metadata (branch, commit) for PR matching +├── session/teardown.ts # Release the active session's owned environment ├── artifacts/ # Output generation (viewer.html, SUMMARY.md, PR format) └── utils/ # Config, exec helpers, port utils, error patterns, GitHub API ``` @@ -32,15 +36,17 @@ src/ - **Build before test** — CLI runs from `dist/`, always `npm run build` after code changes - **agent-browser** — external peer dependency (Rust CLI + Node daemon). All browser commands go through `ab()` in `utils/exec.ts` which calls `agent-browser ` via `execSync` - **Session state** — `start` writes `.session.json`, `exec` and `stop` read it. `stop` clears it. Don't assume session exists without checking +- **Owned environment** — `.session.json` is the only record of the processes and tmux sockets a session owns. Any path that discards it must first call `releaseActiveSessionEnvironment` (`src/session/teardown.ts`) and must keep the file when release fails. Cleanup targets recorded immutable identities, never a reusable pid or socket name - **Session metadata** — `start` writes `metadata.json` inside each session folder with git branch/commit. This persists after `stop` and is used by `pr` to match sessions to branches - **Per-session subfolders** — artifacts go in `proofshot-artifacts/YYYY-MM-DD_HH-mm-ss_slug/` ## Command lifecycle -1. `proofshot start` — spawns dev server, opens browser, starts recording, saves session state + writes `metadata.json` with git branch/commit +1. `proofshot start` — starts configured environment/log capture, spawns dev server, opens browser, starts recording, saves session state + writes `metadata.json` with git branch/commit 2. `proofshot exec ` — logs action to `session-log.json`, forwards to `agent-browser` -3. `proofshot stop` — collects errors, stops recording, trims video, generates SUMMARY.md + viewer.html, clears session +3. `proofshot stop` — collects errors, stops recording, releases the owned environment, trims video, generates SUMMARY.md + viewer.html, clears session 4. `proofshot pr [number]` — finds sessions for current branch, uploads artifacts to GitHub, posts PR comment +5. `proofshot clean` — releases the active session's environment, then removes the output directory ## Adding a new command @@ -70,6 +76,8 @@ Edit `src/utils/error-patterns.ts` — add a new entry to the `PATTERNS` array: | `session.webm` | `start` | Video recording (Playwright screencast) | | `session-log.json` | `exec` (appended each call) | Action timeline with relative timestamps | | `server.log` | `start` (piped stdout+stderr) | All dev server output | +| `environment.ndjson` | `start` (capture workers) | Canonical timestamped evidence per configured environment source | +| `logs/*.log` | `start` (capture workers) | One bounded plain-text log per environment source | | `console-output.log` | `stop` | Browser console output | | `step-*.png` | `exec screenshot` | Screenshots at key moments | | `SUMMARY.md` | `stop` | Markdown report with errors and screenshots | @@ -92,4 +100,9 @@ Edit `src/utils/error-patterns.ts` — add a new entry to the `PATTERNS` array: - `proofshot exec` has special shell quoting logic (`buildShellCommand` in exec.ts) — `eval` commands get single-quoted, args with special chars get auto-quoted - Video trimming adjusts session-log.json timestamps to match the trimmed video (see `trimOffsetSec` in stop.ts) - Server log capture only works when proofshot starts the server itself — if the port is already occupied, we skip spawning and get no server logs +- `--run` and `config.environment` are mutually exclusive (both start the app); `start` rejects the combination up front +- Config validation fails `start` closed, but `stop`/`clean` load config through `loadConfigForTeardown` and only warn — an invalid config must never strand owned resources +- Tmux panes are a single PTY stream, so pane evidence is always `stream: "pty"`; only direct processes keep `stdout`/`stderr` apart +- `connection.ownership: "attach"` outranks "own what you created" — an attach-only tmux server/session/pane is never terminated, so `launch.stopCommand` is rejected in that mode +- `stop` detects a mid-session capture gap two ways, because the workers fail in two shapes: a surviving pid file with a dead process means the helper was killed, while a tmux pane that exited closes its pipe as a clean EOF and is only visible through `#{pane_pipe}` — so the pane check must run before teardown detaches the pipes — and teardown clears `captureAttached` as it detaches, so a retry after a partial failure cannot mistake its own work for a gap. Either way `stop` records the gap and exits non-zero after finishing teardown - The `consoleErrors`/`consoleOutput` from agent-browser are point-in-time snapshots collected at stop time diff --git a/README.md b/README.md index d157a65..fbd705b 100644 --- a/README.md +++ b/README.md @@ -103,6 +103,8 @@ Each session produces a timestamped folder in `./proofshot-artifacts/`: | `session-log.json` | Action timeline with timestamps and element data | | `server.log` | Dev server stdout/stderr (when using `--run`) | | `console-output.log` | Browser console output | +| `environment.ndjson` | Timestamped canonical evidence from configured tmux, process, and file sources | +| `logs/*.log` | One bounded plain-text log per configured environment source |

ProofShot artifacts folder @@ -158,6 +160,30 @@ You can also configure browser launch behavior in `proofshot.config.json`: Set `browser.configPath` when you need ProofShot to run `agent-browser` against a project-specific config instead of inheriting `~/.agent-browser/config.json`. Relative paths are resolved from the directory that contains `proofshot.config.json`. +For applications with multiple runners, configure ProofShot to own and capture tmux panes, direct processes, and file tails: + +```json +{ + "environment": { + "kind": "processes", + "commands": [ + { "id": "web", "group": "frontend", "command": "npm run dev" }, + { "id": "api", "group": "backend", "command": "npm run api" } + ], + "readiness": [ + { "kind": "http", "url": "http://127.0.0.1:3000/health" } + ] + }, + "logs": { + "sources": [ + { "id": "worker", "kind": "file", "path": "./logs/worker.log" } + ] + } +} +``` + +Direct process commands are captured automatically; declare `logs.sources` only for custom source identities or additional files. Use either `--run` or `environment`, not both. See the [configuration reference](content/docs/reference/configuration.mdx) for owned tmux sockets, external launcher contracts, source naming, and readiness checks. + ### `proofshot stop` Stop recording, collect errors, generate proof artifacts. @@ -211,6 +237,8 @@ Remove the `./proofshot-artifacts/` directory. proofshot clean ``` +If a session is still active there, `clean` first releases the processes and tmux sockets it owns — and refuses to delete the directory if that fails, since `.session.json` is the only record of them. + ### `proofshot doctor` Print the current ProofShot environment, including config path, browser mode, viewport, installed binaries, and any active session. @@ -226,13 +254,14 @@ proofshot doctor | Agent | Install location | |-------|-----------------| | **Claude Code** | `~/.claude/skills/proofshot/SKILL.md` | -| **Cursor** | `~/.cursor/rules/proofshot.mdc` | +| **Cursor** | `~/.cursor/skills/proofshot/SKILL.md` | | **Codex (OpenAI)** | `~/.codex/skills/proofshot/SKILL.md` | | **OpenCode** | `~/.config/opencode/skills/proofshot/SKILL.md` | | **Gemini CLI** | Appends to `~/.gemini/GEMINI.md` | | **Windsurf** | Appends to `~/.codeium/windsurf/memories/global_rules.md` | All skills install at **user level** — no per-project configuration needed. +When upgrading Cursor installations, ProofShot preserves the previous `proofshot.mdc` rule as a non-loading `.migrated` backup after the skill is written successfully. ## Try It @@ -280,6 +309,8 @@ npm test # Run tests npm run dev # Watch mode ``` +Install `tmux` to run the environment-capture integration tests locally — they skip themselves when it is missing, and CI installs it so the coverage always runs there. + Three sample apps in `test/fixtures/` cover different UI patterns for end-to-end testing: a SaaS dashboard (`sample-app`), a kanban board (`todo-app`), and a chat interface (`chat-app`). Built on [agent-browser](https://github.com/vercel-labs/agent-browser) by Vercel. diff --git a/bin/proofshot.ts b/bin/proofshot.ts index 7bd7cd1..ded606a 100644 --- a/bin/proofshot.ts +++ b/bin/proofshot.ts @@ -1,4 +1,9 @@ +import chalk from 'chalk'; import { createCLI } from '../src/cli.js'; +import { formatErrorDetail } from '../src/utils/errors.js'; const program = createCLI(); -program.parse(); +program.parseAsync().catch((error) => { + console.error(chalk.red('✗') + ` ${formatErrorDetail(error)}`); + process.exit(1); +}); diff --git a/content/docs/concepts/how-it-works.mdx b/content/docs/concepts/how-it-works.mdx index 6bdac70..b0671ed 100644 --- a/content/docs/concepts/how-it-works.mdx +++ b/content/docs/concepts/how-it-works.mdx @@ -45,14 +45,17 @@ ProofShot uses a three-phase model. `proofshot start` initializes the session: 1. Check if the port is available (fail fast on conflicts) -2. Spawn the dev server if `--run` is provided, pipe output to `server.log` -3. Wait for the port to respond (polls every 500ms, 30s timeout) -4. Open headless Chromium -5. Start video recording -6. Write `.session.json` (active session state) and `metadata.json` (git branch/commit, persists after stop) +2. Start the configured `environment` and `logs` sources — tmux panes, direct processes, and file tails — and wait for their readiness checks +3. Spawn the dev server if `--run` is provided, pipe output to `server.log` +4. Wait for the port to respond (polls every 500ms, 30s timeout) +5. Open headless Chromium +6. Start video recording +7. Write `.session.json` (active session state) and `metadata.json` (git branch/commit, persists after stop) Recording is mandatory. If it fails after 3 retries, the session aborts. +`--run` and `environment` are two ways to start the same app, so ProofShot rejects using both. `.session.json` is written before the environment starts and updated as each resource is claimed, so a crashed start still leaves the owned processes and sockets recoverable. + ### Phase 2: Exec (repeated) Each `proofshot exec` call: @@ -70,10 +73,13 @@ Each `proofshot exec` call: 1. Collects browser console errors and output (point-in-time snapshot) 2. Stops video recording 3. Closes the browser -4. Trims video dead time using ffmpeg (5s buffer before first action, 3s after last). Adjusts all `session-log.json` timestamps by the trim offset. -5. Scans `server.log` with multi-language regex patterns for errors -6. Generates `SUMMARY.md` and `viewer.html` -7. Clears `.session.json` +4. Stops environment capture and releases every process and tmux socket the session owns, by recorded identity +5. Trims video dead time using ffmpeg (5s buffer before first action, 3s after last). Adjusts all `session-log.json` timestamps by the trim offset. +6. Scans `server.log` with multi-language regex patterns for errors +7. Generates `SUMMARY.md` and `viewer.html` +8. Clears `.session.json` + +Environment teardown is the one step that is not best-effort: if a resource cannot be released, `stop` fails and keeps `.session.json` so nothing is silently orphaned. ## Design principles diff --git a/content/docs/faq.mdx b/content/docs/faq.mdx index cd0799d..70d435f 100644 --- a/content/docs/faq.mdx +++ b/content/docs/faq.mdx @@ -42,7 +42,7 @@ When ffmpeg is available, `proofshot stop` cuts dead time from the video — kee You need to run `proofshot start` first. Each session writes `.session.json` — if it's missing, there's no active session to operate on. **Server errors aren't being detected** -Server log capture only works when ProofShot starts the server itself via `--run`. If your server was already running on the port, ProofShot skips spawning and gets no logs. +`server.log` capture only works when ProofShot starts the server itself via `--run`. If your server was already running on the port, ProofShot skips spawning and gets no `server.log`. To capture an app you start yourself — or more than one process — declare `environment` and `logs.sources` in [the config](/docs/reference/configuration): ProofShot can attach to an existing tmux session or tail a log file. Those sources are recorded as evidence in `environment.ndjson` and `logs/`; the multi-language error scan still runs only on `server.log`. **The browser window doesn't appear** ProofShot runs headless by default. Use `--headed` to see the browser window: `proofshot start --headed`. diff --git a/content/docs/guides/configure.mdx b/content/docs/guides/configure.mdx index 1ef8961..f5a91b5 100644 --- a/content/docs/guides/configure.mdx +++ b/content/docs/guides/configure.mdx @@ -41,15 +41,7 @@ proofshot start --port 8080 --output ./my-artifacts --headed ## All configuration options -| Option | Type | Default | Description | -|--------|------|---------|-------------| -| `devServer.port` | `number` | `3000` | Port your dev server runs on | -| `devServer.startupTimeout` | `number` | `30000` | Max wait time (ms) for the dev server to start | -| `output` | `string` | `./proofshot-artifacts` | Directory for session artifacts | -| `defaultPages` | `string[]` | `["/"]` | Pages to open by default | -| `viewport.width` | `number` | `1280` | Browser viewport width in pixels | -| `viewport.height` | `number` | `720` | Browser viewport height in pixels | -| `headless` | `boolean` | `true` | Run the browser without a visible window | +Every option, type, and default lives in the [configuration reference](/docs/reference/configuration) — including the `browser`, `environment`, and `logs` blocks for multi-process apps, tmux panes, and file tails. You don't need a config file for most projects. The defaults work for standard setups, and CLI flags cover one-off overrides. diff --git a/content/docs/guides/install-skills.mdx b/content/docs/guides/install-skills.mdx index b5e7f64..440cb2d 100644 --- a/content/docs/guides/install-skills.mdx +++ b/content/docs/guides/install-skills.mdx @@ -51,13 +51,15 @@ All skills install at **user level** — one install works across every project | Agent | Location | Strategy | |-------|----------|----------| | **Claude Code** | `~/.claude/skills/proofshot/SKILL.md` | Standalone file | -| **Cursor** | `~/.cursor/rules/proofshot.mdc` | Standalone file | +| **Cursor** | `~/.cursor/skills/proofshot/SKILL.md` | Standalone skill | | **Codex** | `~/.codex/skills/proofshot/SKILL.md` | Standalone file | | **Gemini CLI** | `~/.gemini/GEMINI.md` | Appended with markers | | **Windsurf** | `~/.codeium/windsurf/memories/global_rules.md` | Appended with markers | Tools using the **append** strategy wrap ProofShot's content in `` / `` markers. Running `proofshot install` again replaces only the marked section — your other content is preserved. +When upgrading an existing Cursor installation, ProofShot writes the new skill first and then renames `~/.cursor/rules/proofshot.mdc` to a non-loading `.migrated` backup. Custom rule edits are preserved, and a failed skill write leaves the legacy rule active. + ## What the skill teaches the agent The skill file gives your agent: diff --git a/content/docs/guides/verify-feature.mdx b/content/docs/guides/verify-feature.mdx index 0aabe94..7498e87 100644 --- a/content/docs/guides/verify-feature.mdx +++ b/content/docs/guides/verify-feature.mdx @@ -131,7 +131,7 @@ You need to run `proofshot start` first. The session state is stored in `.sessio Install [ffmpeg](https://ffmpeg.org/) and ProofShot will automatically trim dead time from the video (5s buffer before the first action, 3s buffer after the last). **No server errors detected but I know there are errors** -Server error detection only works when ProofShot starts the server itself (`--run`). If your server was already running, ProofShot can't capture its logs. +The error scan reads `server.log`, which only exists when ProofShot starts the server itself (`--run`). To capture an app you start yourself, or more than one process, see [the FAQ](/docs/faq) and the [configuration reference](/docs/reference/configuration). ## What's next? diff --git a/content/docs/quick-start.mdx b/content/docs/quick-start.mdx index c75b9b8..314bd6d 100644 --- a/content/docs/quick-start.mdx +++ b/content/docs/quick-start.mdx @@ -53,7 +53,7 @@ Detected tools: Installing skills... ✓ Claude Code → ~/.claude/skills/proofshot/SKILL.md - ✓ Cursor → ~/.cursor/rules/proofshot.mdc + ✓ Cursor → ~/.cursor/skills/proofshot/SKILL.md Done! Your AI agent now knows how to use ProofShot. ``` diff --git a/content/docs/reference/artifacts.mdx b/content/docs/reference/artifacts.mdx index be5d03b..2b980c5 100644 --- a/content/docs/reference/artifacts.mdx +++ b/content/docs/reference/artifacts.mdx @@ -15,9 +15,12 @@ Each ProofShot session creates a timestamped folder in your output directory (de | `session.webm` | `start` / `stop` | Video recording of the entire session (Playwright screencast). Trimmed by `stop` if ffmpeg is available. | | `session-log.json` | `exec` | Action timeline. Each entry records the command, relative timestamp, and element data (bounding box, label) for `@eN` targets. Appended with each `exec` call. | | `server.log` | `start` | Dev server stdout and stderr. Only created when ProofShot starts the server via `--run`. | +| `environment.ndjson` | `start` | Canonical timestamped evidence from configured tmux panes, direct processes, and file tails. | +| `logs/*.log` | `start` | One bounded plain-text log per configured environment source, named `.log`. | +| `.capture/` | `start` | Internal bookkeeping for the capture workers: `.pid` while a capture is live, plus `.pid.stderr` helper diagnostics that a reported capture gap points you to. Not proof. | | `console-output.log` | `stop` | Browser console output collected at stop time. | | `step-*.png` | `exec screenshot` | Screenshots captured at key moments. Filenames come from the `exec screenshot` argument. | -| `SUMMARY.md` | `stop` | Markdown report containing: session date, description, video link, screenshots, console error count and details, server error count and details, environment info. | +| `SUMMARY.md` | `stop` | Text-based markdown report — see [`SUMMARY.md`](#summarymd) below for its contents. | | `viewer.html` | `stop` | Self-contained interactive HTML viewer. No external dependencies — open it in any browser. | ## `metadata.json` format @@ -66,6 +69,21 @@ Timestamps are relative to session start (in seconds). After video trimming, tim Element data (`bbox`, `label`) is captured before execution for `click`, `fill`, and `type` actions targeting `@eN` references. This data powers the viewer's click ripple overlays and action labels. +## `environment.ndjson` format + +One JSON object per line — the canonical record of everything the configured environment sources emitted. `logs/*.log` is a plain-text rendering of the same content. + +```json +{"version":1,"origin":"environment","group":"frontend","sourceId":"vite","sourceTitle":"Vite","stream":"pty","segment":"live","timestamp":"2025-01-15T10:30:02.412Z","relativeTimeSec":2.412,"text":"ready in 431 ms"} +``` + +- `segment` is `history` for retained scrollback or pre-existing file content (`timestamp` and `relativeTimeSec` are `null` — the time it was written is unknown) and `live` for output captured during the session. +- `stream` is `pty` for tmux panes, `stdout`/`stderr` for direct processes, and `file` for file sources. +- `truncated` marks where a source hit `logs.maxBytesPerSource`. `captureGap` marks a point where output was missed — a rotated or truncated source file, unreadable tmux scrollback, an unparseable evidence row, or a capture that stopped before the session ended. +- `stop` verifies every capture survived the session — both the capture helper and, for tmux, the pane still being piped. A source that stopped early gets a `captureGap` row, a **Capture Gaps** section in `SUMMARY.md`, and a non-zero exit code, because otherwise the remaining evidence reads as complete. + +See the [configuration reference](/docs/reference/configuration) for how sources are declared and named. + ## `viewer.html` A standalone HTML file that serves as the primary proof artifact. See [The Interactive Viewer](/docs/concepts/interactive-viewer) for details on its features. @@ -80,4 +98,6 @@ A text-based report for environments where the HTML viewer isn't practical (PR c - Embedded screenshots - Console error count and details - Server error count and log excerpt +- Links to captured environment source logs +- A **Capture Gaps** section naming any source that stopped feeding ProofShot before the session ended - Environment info (browser, viewport, duration) diff --git a/content/docs/reference/cli.mdx b/content/docs/reference/cli.mdx index a2355f0..889c15f 100644 --- a/content/docs/reference/cli.mdx +++ b/content/docs/reference/cli.mdx @@ -17,7 +17,7 @@ proofshot install [options] | Flag | Description | |------|-------------| -| `--only ` | Only install for specific tools (comma-separated). Values: `claude`, `cursor`, `codex`, `gemini`, `windsurf` | +| `--only ` | Only install for specific tools (comma-separated). Values: `claude`, `cursor`, `codex`, `gemini`, `windsurf`, `opencode` | | `--skip ` | Skip specific tools (comma-separated) | | `--force` | Overwrite existing skill files | @@ -48,6 +48,7 @@ proofshot start [options] | `--description ` | Description of what you're verifying (appears in reports) | — | | `--headed` | Show the browser window (visible Chromium) | `false` | | `--output

` | Custom output directory for artifacts | `./proofshot-artifacts` | +| `--force` | Override a stale session without running `stop` first | `false` | **Examples:** @@ -60,11 +61,16 @@ proofshot start --headed # Show the browser w ``` **What happens:** -1. If `--run` is provided: starts the dev server, pipes output to `server.log`, waits for the port -2. Opens headless Chromium via agent-browser -3. Navigates to `--url` (or `http://localhost:`) -4. Starts video recording (retries up to 3 times) -5. Writes `.session.json` and `metadata.json` (git branch and commit SHA) +1. If `environment` or `logs.sources` are configured: starts the owned tmux panes, direct processes, and file tails, waits for the configured readiness checks, and records evidence to `environment.ndjson` and `logs/` +2. If `--run` is provided: starts the dev server, pipes output to `server.log`, waits for the port +3. Opens headless Chromium via agent-browser +4. Navigates to `--url` (or `http://localhost:`) +5. Starts video recording (retries up to 3 times) +6. Writes `.session.json` and `metadata.json` (git branch and commit SHA) + +`--run` and a configured `environment` are mutually exclusive — pick one way to start your app. See the [configuration reference](/docs/reference/configuration) for the `environment` and `logs` contracts. + +If any startup step fails, ProofShot releases the environment it started before exiting. `--force` releases the previous session's environment before clearing `.session.json`. When that release fails, `start` exits non-zero, keeps `.session.json` as recovery state, and names the resource to resolve — rerun `proofshot stop` once it is resolved. --- @@ -84,10 +90,15 @@ proofshot stop [options] 1. Collects console errors and output from the browser 2. Stops video recording 3. Closes the browser (unless `--no-close`) -4. Trims video dead time (requires ffmpeg): 5s buffer before first action, 3s after last -5. Scans `server.log` for errors across 10+ languages -6. Generates `SUMMARY.md` and `viewer.html` -7. Clears `.session.json` +4. Stops environment capture and releases the processes and tmux sockets the session owns +5. Trims video dead time (requires ffmpeg): 5s buffer before first action, 3s after last +6. Scans `server.log` for errors across 10+ languages +7. Generates `SUMMARY.md` and `viewer.html` +8. Clears `.session.json` + +If releasing an owned resource fails, `stop` exits non-zero and keeps `.session.json` as recovery state so the remaining processes and sockets are still recorded. Resolve the reported resource and run `proofshot stop` again. + +If a capture stopped before the session ended, `stop` still completes teardown and writes every artifact, then records the gap in `environment.ndjson` and `SUMMARY.md` and exits non-zero — incomplete evidence is never reported as a clean run. --- @@ -188,3 +199,5 @@ proofshot clean ``` Deletes `./proofshot-artifacts/` (or the configured output directory). No flags. + +If an active session is recorded there, `clean` first releases the environment it owns, because `.session.json` is the only record of those processes and sockets. When that release fails, `clean` refuses to delete the directory, keeps the recovery state, and exits non-zero. diff --git a/content/docs/reference/configuration.mdx b/content/docs/reference/configuration.mdx index b34e516..6ad2e58 100644 --- a/content/docs/reference/configuration.mdx +++ b/content/docs/reference/configuration.mdx @@ -11,6 +11,8 @@ ProofShot is configured via `proofshot.config.json`. This file is optional — d ProofShot searches for `proofshot.config.json` starting from your current directory and walking up to the filesystem root. This supports monorepo layouts where the config lives at the repo root. +Relative paths inside the file — `output`, `browser.configPath`, `environment` working directories, and `file` log source paths — resolve from the directory that contains `proofshot.config.json`, not from the directory you run ProofShot in. + ## All options ```json filename="proofshot.config.json" @@ -25,7 +27,42 @@ ProofShot searches for `proofshot.config.json` starting from your current direct "width": 1280, "height": 720 }, - "headless": true + "headless": true, + "environment": { + "kind": "tmux", + "launch": { + "kind": "panes", + "panes": [ + { + "id": "vite", + "title": "Vite", + "group": "frontend", + "command": "npm run dev" + }, + { + "id": "api", + "title": "API", + "group": "backend", + "command": "npm run api" + } + ] + }, + "readiness": [ + { "kind": "http", "url": "http://127.0.0.1:3000/health" } + ] + }, + "logs": { + "stripAnsi": true, + "maxBytesPerSource": 5242880, + "sources": [ + { + "id": "vite", + "group": "frontend", + "kind": "tmux-pane", + "match": { "connectionKey": "vite" } + } + ] + } } ``` @@ -71,6 +108,55 @@ The viewport size affects video recording resolution and screenshot dimensions. When `true`, the browser runs without a visible window. Set to `false` (or use `--headed` CLI flag) to see the browser during testing — useful for debugging. +### `environment` + +`environment.kind` is either: + +- `tmux`: ProofShot owns a dedicated tmux socket/session or connects to an external launcher on the launcher-reported socket. +- `processes`: ProofShot starts multiple direct commands and preserves separate `stdout` and `stderr` streams. + +For `tmux`, `launch.kind: "panes"` accepts `{ id, title, group, cwd, command, env }` entries under `launch.panes`, plus an optional `launch.sessionName`. `launch.kind: "external-command"` runs one launcher and reads the tmux target from its stdout; the required `connection.format` selects the wire format — `"json"` for structured JSON or `"tmux-attach-command"` for a `tmux -L attach -t ` line. Set `connection.ownership` to `attach` when the launcher only reports an existing session. Launchers that create resources must provide a stable `connection.socket` hint or `launch.stopCommand`; they also support `launch.timeoutMs`. Structured JSON is preferred: + +```json +{ + "tmux": { + "socket": "/tmp/dev.sock", + "session": "dev", + "panes": [ + { "key": "vite", "paneId": "%12", "title": "Vite", "group": "frontend" } + ] + } +} +``` + +`connection.ownership: "attach"` always wins: ProofShot never terminates the launcher, tmux server, session, or panes of an attach-only environment, even when that start's launcher created them — it only detaches its own `pipe-pane` capture. Because nothing is ever stopped in that mode, `launch.stopCommand` is rejected alongside `ownership: "attach"` rather than silently ignored. + +Otherwise ProofShot snapshots a hinted tmux socket before launch and owns only the server/session identities created by that start. Shared launchers that create a session must provide `stopCommand`. ProofShot never runs `tmux kill-server` against the default or an unowned socket. + +For `processes`, list the commands under `environment.commands` — the same `{ id, title, group, cwd, command, env }` entry shape used by tmux panes. Each `id` must be unique, and `logs.sources` entries of kind `process` reference it via `processId`. + +`readiness` accepts HTTP checks (`url`) and TCP checks (`host`, `port`), each with an optional `timeoutMs`. + +### `logs` + +Every source requires a stable `id`; optional `group` values preserve the source's logical role in canonical evidence. Supported source kinds: + +- `tmux-pane`: match by launcher `connectionKey`, stable `@proofshot-source` `tag`, or exact `session:window.pane` `target`. +- `process`: select a direct environment command by `processId`. +- `file`: capture an existing file, including `--url` attach workflows. + +`tmux-pane` sources require `environment.kind: "tmux"` and `process` sources require `environment.kind: "processes"`. A mismatch — or a declared source with no `environment` block — is rejected as a config error instead of being silently dropped from the evidence. `file` sources need no `environment`. + +Pane titles resolve in this order: launcher mapping title, non-empty tmux pane title, then `Pane `. Duplicate display titles gain a pane-number suffix without changing source identity. + +Tmux panes are one PTY byte stream and are always recorded as `stream: "pty"`; stdout and stderr cannot be recovered after tmux multiplexes them. ProofShot installs `pipe-pane` before backfilling retained scrollback. Canonical evidence keeps untimed `history` and timestamped `live` segments plus their overlap/capture-gap boundary. + +`maxBytesPerSource` bounds the combined canonical NDJSON and plain-text log bytes written for each source, and records truncation instead of silently discarding integrity state. It must be at least `512` bytes. + +## Validation + +`proofshot start` fails fast on an unreadable or invalid config and names the offending field, so a typo never silently downgrades capture to defaults. `proofshot stop` and `proofshot clean` instead warn and continue with the resolvable output directory, so an invalid config can never strand a session's owned processes and sockets. + ## CLI flag precedence Command-line flags override config file values. Config file values override defaults. diff --git a/docs/architecture.md b/docs/architecture.md index 5841538..1af47d7 100644 --- a/docs/architecture.md +++ b/docs/architecture.md @@ -58,14 +58,23 @@ proofshot start --run "npm run dev" --port 3000 --description "Login flow" ``` 1. Check if port is already occupied (fail fast if `--run` conflicts) -2. Spawn dev server as a detached process, pipe stdout/stderr to `server.log` -3. Wait for port to become available (polls every 500ms, 30s timeout) -4. Open headless Chromium via agent-browser -5. Start video recording (Playwright screencast → `.webm`) -6. Write `.session.json` to the artifacts directory +2. Start the configured environment (`environment` and `logs.sources`): own tmux panes or direct processes, attach file tails, and wait for readiness checks +3. Spawn dev server as a detached process, pipe stdout/stderr to `server.log` +4. Wait for port to become available (polls every 500ms, 30s timeout) +5. Open headless Chromium via agent-browser +6. Start video recording (Playwright screencast → `.webm`) +7. Write `.session.json` to the artifacts directory Recording is mandatory. If it fails after 3 retries, the session aborts. Video proof is the whole point. +Step 2 owns real resources, so it is strict rather than best-effort: + +- `--run` and a configured `environment` are mutually exclusive — both start the app, and running both would double-start it. +- `.session.json` is written *before* the environment starts and re-saved as each process and socket identity is claimed. A failed start can then release exactly what it created, and a failed release keeps the session file as recovery state instead of orphaning resources. +- `src/session/teardown.ts` is the shared entry point for every path that would otherwise discard that record (`start --force`, `clean`). + +The config contract for sources, readiness, and ownership lives in `content/docs/reference/configuration.mdx`. + ### Exec ``` @@ -95,11 +104,14 @@ proofshot stop 1. Collect console errors and console output from the browser (point-in-time snapshots) 2. Stop video recording 3. Close browser (unless `--no-close`) -4. **Trim video** — remove dead time before first action (5s buffer) and after last action (3s buffer) using ffmpeg. Adjust all `session-log.json` timestamps by the trim offset to stay in sync with the trimmed video. -5. **Extract server errors** — scan `server.log` with multi-language regex patterns -6. Generate `SUMMARY.md` — markdown report with description, video link, screenshots, and errors -7. Generate `viewer.html` — standalone interactive viewer -8. Clear `.session.json` +4. **Release the owned environment** — terminate the recorded process and socket identities, never a name that could have been reused +5. **Trim video** — remove dead time before first action (5s buffer) and after last action (3s buffer) using ffmpeg. Adjust all `session-log.json` timestamps by the trim offset to stay in sync with the trimmed video. +6. **Extract server errors** — scan `server.log` with multi-language regex patterns +7. Generate `SUMMARY.md` — markdown report with description, video link, screenshots, errors, and links to the captured environment logs +8. Generate `viewer.html` — standalone interactive viewer +9. Clear `.session.json` + +Step 4 fails the command rather than degrading: an unreleased resource leaves `.session.json` in place so a later `proofshot stop` can finish the job. ## Interactive viewer @@ -174,7 +186,8 @@ src/ │ ├── exec.ts # Action logging + agent-browser passthrough │ ├── diff.ts # Visual regression (screenshot comparison) │ ├── pr.ts # GitHub PR description formatting -│ └── clean.ts # Artifact directory removal +│ ├── clean.ts # Environment release + artifact directory removal +│ └── doctor.ts # Local environment and active session inspection ├── browser/ │ ├── session.ts # Browser open/close, console collection │ ├── capture.ts # Recording start/stop, screenshots, diffs @@ -182,18 +195,33 @@ src/ │ └── interact.ts # Click, fill, type, scroll, press ├── server/ │ └── start.ts # Dev server spawn + port waiting +├── environment/ +│ ├── runtime.ts # Start/stop the owned environment and log capture +│ ├── types.ts # Environment config, ownership state, evidence events +│ ├── evidence.ts # environment.ndjson append/read, ANSI normalization +│ ├── workers.ts # Detached capture workers for processes and file tails +│ ├── tmux.ts # tmux environment orchestration +│ ├── tmux-launch.ts # Owned socket/session creation, external launchers +│ ├── tmux-panes.ts # Pane selection, titles, source identity +│ ├── tmux-identity.ts # Socket identity capture and verification +│ ├── tmux-cleanup.ts # Ownership-checked tmux teardown +│ └── tmux-command.ts # tmux/shell invocation helpers ├── session/ -│ └── state.ts # .session.json read/write/clear +│ ├── state.ts # .session.json read/write/clear +│ ├── metadata.ts # Per-session git branch/commit for PR matching +│ └── teardown.ts # Release the active session's environment ├── artifacts/ │ ├── viewer.ts # Interactive HTML viewer generation │ ├── summary.ts # Markdown summary generation │ ├── pr-format.ts # PR description formatting │ └── bundle.ts # Artifact bundling └── utils/ - ├── config.ts # Config file search + merge + ├── config.ts # Config file search + validation + merge ├── exec.ts # ab() and exec() shell wrappers ├── error-patterns.ts # Multi-language regex patterns + ├── errors.ts # CLI error rendering, including nested causes ├── port.ts # isPortOpen, waitForPort + ├── process.ts # Owned-process identity, signalling, termination └── skills.ts # Skill file bundling ``` @@ -205,8 +233,14 @@ src/ **Minimal dependencies.** Only three production dependencies: `commander` (CLI framework), `chalk` (terminal colors), `detect-port` (port checking). agent-browser is an optional peer dependency. This keeps the install fast and the supply chain small. -**Session state in the output directory.** `.session.json` lives alongside artifacts, not in a global location. This allows parallel sessions in different projects and ensures `proofshot clean` removes everything. +**Session state in the output directory.** `.session.json` lives alongside artifacts, not in a global location. This allows parallel sessions in different projects and ensures `proofshot clean` removes everything. It is also the only record of the process and socket identities a session owns, so `clean` releases them before deleting the directory and refuses to delete it while a release is failing. **Config file walk-up.** `proofshot.config.json` is searched from cwd upward to filesystem root, supporting monorepo layouts where config lives at the repo root. **Graceful degradation everywhere.** Missing ffmpeg skips video trimming. Failed element data capture skips overlays. Browser already closed gets a silent catch. Console errors unavailable shows "0 errors". Non-critical failures never abort a session. + +**Except for owned resources.** Starting and releasing an owned environment fails loudly. A silently dropped log source produces evidence that looks complete but is not, and a silently leaked process or tmux socket outlives the session — so both are errors, and an invalid `proofshot.config.json` now fails `start` instead of falling back to defaults. + +**A capture that died is a failed verification, not a warning.** Every capture worker removes its pid file on a clean exit, so `stop` treats a surviving pid file whose process is gone as a mid-session gap. That signal alone is not enough for tmux: a pane whose command exits is destroyed by tmux, which closes the capture pipe as a clean EOF, so the helper shuts down exactly as it would at stop time. `stop` therefore also verifies each recorded pane still reports `#{pane_pipe}` — before teardown detaches those pipes, which is the only window where the two are distinguishable. Teardown still runs to completion and every artifact is still written — then the gap is recorded in `environment.ndjson` (`captureGap`) and `SUMMARY.md`, and `stop` exits non-zero. The alternative is a proof bundle that reads as complete while missing the output that mattered. + +**Attach-only environments are never terminated.** `connection.ownership: "attach"` outranks the "own what you created" rule, so a tmux server, session, or pane reached in attach mode survives `stop` even when this start's launcher created it; ProofShot only detaches its own `pipe-pane`. Since nothing is ever stopped in that mode, pairing it with `launch.stopCommand` is a config error rather than a silently ignored setting. diff --git a/proofshot-spec.md b/proofshot-spec.md index 93a55f3..2753895 100644 --- a/proofshot-spec.md +++ b/proofshot-spec.md @@ -2,6 +2,8 @@ ## Product Spec v2.0 +> **Historical design record — not current behavior.** This is the original v2.0 design document, kept for the reasoning behind ProofShot's architecture and product decisions. The shipped CLI has since diverged: commands, flags, config keys, session state, and the artifact set described below are no longer accurate (for example, there is no `proofshot init` command and no `--no-server` flag). For what ProofShot actually does today, see [`README.md`](README.md) for usage, `content/docs/reference/cli.mdx` for commands and flags, `content/docs/reference/configuration.mdx` for config, and [`docs/architecture.md`](docs/architecture.md) for the implemented architecture. Do not treat anything here as a contract. + **One-liner:** Give any AI coding agent eyes. It builds a feature → ProofShot records video proof it works. **Tagline:** "Cursor charges $200/mo for agents that can see what they build. Here's the same thing, free, for every agent." @@ -117,7 +119,7 @@ This does: 3. Installs the skill file for the detected agent: - Claude Code: `.claude/skills/proofshot/SKILL.md` - Codex: `codex.md` / `AGENTS.md` append - - Cursor: `.cursor/rules/proofshot.mdc` + - Cursor: `.cursor/skills/proofshot/SKILL.md` - General: `PROOFSHOT.md` in project root ### Usage — The Developer Does Nothing Different @@ -249,6 +251,8 @@ No server errors detected. ## 4. CLI Commands +> Historical — the shipped command surface differs (no `init`, no `--no-server`, plus commands and flags not designed here). `content/docs/reference/cli.mdx` is the authoritative CLI reference. + ### `proofshot init` Detects framework, creates config, installs skill file. @@ -450,12 +454,12 @@ a SUMMARY.md with video, screenshots, and error report. - The proof artifacts in ./proofshot-artifacts/ can be referenced in commit messages or PRs ``` -### Cursor Rule: `.cursor/rules/proofshot.mdc` +### Cursor Skill: `.cursor/skills/proofshot/SKILL.md` ```markdown --- -description: Visual verification of UI changes using ProofShot -globs: ["**/*.tsx", "**/*.jsx", "**/*.vue", "**/*.svelte", "**/*.html"] +name: proofshot +description: Visually verifies UI changes with browser recordings, screenshots, console output, and named environment logs. Use after building or modifying user-facing features. --- After modifying UI files, visually verify changes with this workflow: @@ -534,7 +538,7 @@ proofshot/ │ └── config.ts # Config file reading/writing ├── skills/ │ ├── claude/SKILL.md -│ ├── cursor/proofshot.mdc +│ ├── cursor/SKILL.md │ ├── codex/AGENTS.md │ └── generic/PROOFSHOT.md └── test/ diff --git a/skills/cursor/SKILL.md b/skills/cursor/SKILL.md new file mode 100644 index 0000000..0a210bc --- /dev/null +++ b/skills/cursor/SKILL.md @@ -0,0 +1,21 @@ +--- +name: proofshot +description: Visually verifies UI changes with browser recordings, screenshots, console output, and named environment logs. Use after building or modifying user-facing features. +--- + +# ProofShot visual verification + +Use ProofShot after changing UI behavior: + +1. Start a session: + `proofshot start --run "your-dev-command" --port PORT --description "what you are verifying"` + Use `proofshot.config.json` environment and log sources instead of `--run` when verification needs multiple processes, tmux panes, or file tails. +2. Drive the browser with `proofshot exec`: + - `proofshot exec snapshot -i` + - `proofshot exec click @e3` + - `proofshot exec fill @e2 "text"` + - `proofshot exec screenshot step.png` +3. Stop and bundle evidence: + `proofshot stop` + +Take screenshots before and after important actions. Read the browser snapshot and captured logs to verify the expected behavior, then fix and repeat if evidence contains errors. diff --git a/skills/cursor/proofshot.mdc b/skills/cursor/proofshot.mdc deleted file mode 100644 index 49ca396..0000000 --- a/skills/cursor/proofshot.mdc +++ /dev/null @@ -1,20 +0,0 @@ ---- -description: Visual verification of UI changes using ProofShot -globs: ["**/*.tsx", "**/*.jsx", "**/*.vue", "**/*.svelte", "**/*.html"] ---- - -ProofShot is an open-source, agent-agnostic CLI for visual verification — works with any agent, no vendor lock-in. - -After modifying UI files, visually verify changes with this workflow: - -1. Start session: `proofshot start --run "your-dev-command" --port PORT --description "what you are verifying"` - Always use --run so proofshot captures server logs. Only omit --run if the server was started by the user. - If the port is in use, proofshot kills the existing process. Add --force if a stale session blocks startup. -2. Drive browser: Use `proofshot exec` commands to navigate, click, fill forms, and take screenshots -3. Stop session: `proofshot stop` to bundle video + screenshots + error report - -Key proofshot exec commands: -- `proofshot exec snapshot -i` — see interactive elements -- `proofshot exec click @e3` — click an element -- `proofshot exec fill @e2 "text"` — fill a form field -- `proofshot exec screenshot step.png` — capture a moment diff --git a/src/commands/clean.ts b/src/commands/clean.ts index 8eeb088..a590908 100644 --- a/src/commands/clean.ts +++ b/src/commands/clean.ts @@ -1,10 +1,13 @@ import * as fs from 'fs'; import * as path from 'path'; import chalk from 'chalk'; -import { loadConfig } from '../utils/config.js'; +import { loadConfigForTeardown } from '../utils/config.js'; +import { formatErrorDetail } from '../utils/errors.js'; +import { releaseActiveSessionEnvironment } from '../session/teardown.js'; export async function cleanCommand(): Promise { - const config = loadConfig(); + const { config, error: configError } = loadConfigForTeardown(); + reportConfigError(configError); const outputDir = path.resolve(config.output); if (!fs.existsSync(outputDir)) { @@ -12,6 +15,30 @@ export async function cleanCommand(): Promise { return; } + const cleanupError = await releaseActiveSessionEnvironment(outputDir); + if (cleanupError) { + console.error( + chalk.red('✗') + + ` Refusing to remove ${outputDir}: ${formatErrorDetail(cleanupError)}`, + ); + console.error( + chalk.dim( + ' Recovery state was retained. Resolve the resource issue, then run "proofshot stop" again.', + ), + ); + process.exit(1); + } + fs.rmSync(outputDir, { recursive: true, force: true }); console.log(chalk.green('✓') + ` Removed ${chalk.dim(outputDir)}`); } + +function reportConfigError(error: Error | null): void { + if (!error) { + return; + } + console.error(chalk.yellow('⚠') + ` ${error.message}`); + console.error( + chalk.dim(' Continuing teardown with the resolvable output directory.'), + ); +} diff --git a/src/commands/install.test.ts b/src/commands/install.test.ts new file mode 100644 index 0000000..4699e2d --- /dev/null +++ b/src/commands/install.test.ts @@ -0,0 +1,79 @@ +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { installCommand } from './install.js'; + +const mocks = vi.hoisted(() => ({ + home: '', +})); + +vi.mock('os', async (importOriginal) => { + const actual = await importOriginal(); + return { + ...actual, + homedir: () => mocks.home, + }; +}); + +let root: string; + +beforeEach(() => { + root = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-install-test-')); + mocks.home = root; + fs.mkdirSync(path.join(root, '.cursor'), { recursive: true }); + vi.spyOn(console, 'log').mockImplementation(() => {}); + vi.spyOn(console, 'error').mockImplementation(() => {}); +}); + +afterEach(() => { + vi.restoreAllMocks(); + fs.rmSync(root, { recursive: true, force: true }); +}); + +describe('installCommand Cursor skill', () => { + it('installs ProofShot as a personal Cursor skill', async () => { + await installCommand({ only: 'cursor' }); + + const skillPath = path.join( + root, + '.cursor', + 'skills', + 'proofshot', + 'SKILL.md', + ); + expect(fs.readFileSync(skillPath, 'utf-8')).toContain( + 'name: proofshot', + ); + }); + + it('preserves and disables the legacy Cursor rule after installing the skill', async () => { + const legacyPath = path.join(root, '.cursor', 'rules', 'proofshot.mdc'); + fs.mkdirSync(path.dirname(legacyPath), { recursive: true }); + fs.writeFileSync(legacyPath, 'custom legacy guidance\n'); + + await installCommand({ only: 'cursor' }); + + expect(fs.existsSync(legacyPath)).toBe(false); + expect(fs.readFileSync(`${legacyPath}.migrated`, 'utf-8')).toBe( + 'custom legacy guidance\n', + ); + expect( + fs.existsSync( + path.join(root, '.cursor', 'skills', 'proofshot', 'SKILL.md'), + ), + ).toBe(true); + }); + + it('keeps the legacy rule active when the skill cannot be written', async () => { + const legacyPath = path.join(root, '.cursor', 'rules', 'proofshot.mdc'); + fs.mkdirSync(path.dirname(legacyPath), { recursive: true }); + fs.writeFileSync(legacyPath, 'legacy guidance\n'); + fs.writeFileSync(path.join(root, '.cursor', 'skills'), 'not a directory'); + + await installCommand({ only: 'cursor' }); + + expect(fs.readFileSync(legacyPath, 'utf-8')).toBe('legacy guidance\n'); + expect(fs.existsSync(`${legacyPath}.migrated`)).toBe(false); + }); +}); diff --git a/src/commands/install.ts b/src/commands/install.ts index 79a8b25..245e1ea 100644 --- a/src/commands/install.ts +++ b/src/commands/install.ts @@ -32,6 +32,8 @@ interface ToolDefinition { bundledSkill: string; /** Fallback agent key for inline content generation */ inlineAgent: string; + /** Previously installed file to retire after a successful migration */ + legacyRelativePath?: string; } interface InstallResult { @@ -76,9 +78,10 @@ function getToolDefinitions(): ToolDefinition[] { displayName: 'Cursor', binaryName: 'cursor', configDir: path.join(home, '.cursor'), - skillTarget: { strategy: 'file', relativePath: 'rules/proofshot.mdc' }, - bundledSkill: 'cursor/proofshot.mdc', + skillTarget: { strategy: 'file', relativePath: 'skills/proofshot/SKILL.md' }, + bundledSkill: 'cursor/SKILL.md', inlineAgent: 'cursor', + legacyRelativePath: 'rules/proofshot.mdc', }, { name: 'codex', @@ -268,7 +271,12 @@ function installForTool(tool: ToolDefinition, force: boolean): InstallResult { fs.mkdirSync(targetDir, { recursive: true }); if (tool.skillTarget.strategy === 'file') { - return installFile(tool, targetPath, content, force); + const result = installFile(tool, targetPath, content, force); + const migratedPath = retireLegacyFile(tool); + if (migratedPath) { + result.message = `Legacy rule preserved at ${migratedPath}`; + } + return result; } else { return installAppend(tool, targetPath, content, force); } @@ -283,6 +291,23 @@ function installForTool(tool: ToolDefinition, force: boolean): InstallResult { } } +function retireLegacyFile(tool: ToolDefinition): string | null { + if (!tool.legacyRelativePath) { + return null; + } + const legacyPath = path.join(tool.configDir, tool.legacyRelativePath); + if (!fs.existsSync(legacyPath)) { + return null; + } + + let migratedPath = `${legacyPath}.migrated`; + for (let suffix = 2; fs.existsSync(migratedPath); suffix += 1) { + migratedPath = `${legacyPath}.migrated-${suffix}`; + } + fs.renameSync(legacyPath, migratedPath); + return migratedPath; +} + // --------------------------------------------------------------------------- // Interactive prompt // --------------------------------------------------------------------------- diff --git a/src/commands/start.test.ts b/src/commands/start.test.ts index 96632ae..b0abd6e 100644 --- a/src/commands/start.test.ts +++ b/src/commands/start.test.ts @@ -11,11 +11,14 @@ const mocks = vi.hoisted(() => ({ generateTimestamp: vi.fn(), generateSessionDirName: vi.fn(), saveSession: vi.fn(), + loadSession: vi.fn(), hasActiveSession: vi.fn(), clearSession: vi.fn(), generateAgentBrowserSessionName: vi.fn(), writeMetadata: vi.fn(), execSync: vi.fn(), + startOwnedEnvironment: vi.fn(), + stopOwnedEnvironment: vi.fn(), })); vi.mock('../utils/config.js', () => ({ @@ -43,6 +46,7 @@ vi.mock('../artifacts/bundle.js', () => ({ vi.mock('../session/state.js', () => ({ saveSession: mocks.saveSession, + loadSession: mocks.loadSession, hasActiveSession: mocks.hasActiveSession, clearSession: mocks.clearSession, generateAgentBrowserSessionName: mocks.generateAgentBrowserSessionName, @@ -52,6 +56,11 @@ vi.mock('../session/metadata.js', () => ({ writeMetadata: mocks.writeMetadata, })); +vi.mock('../environment/runtime.js', () => ({ + startOwnedEnvironment: mocks.startOwnedEnvironment, + stopOwnedEnvironment: mocks.stopOwnedEnvironment, +})); + vi.mock('child_process', () => ({ execSync: mocks.execSync, })); @@ -70,6 +79,7 @@ describe('startCommand', () => { headless: true, viewport: { width: 1280, height: 720 }, browser: {}, + logs: { sources: [] }, devServer: { port: 3000, startupTimeout: 1000, @@ -84,6 +94,7 @@ describe('startCommand', () => { if (command === 'git rev-parse HEAD') return 'deadbeef'; throw new Error(`unexpected command: ${command}`); }); + mocks.stopOwnedEnvironment.mockResolvedValue(undefined); }); afterEach(() => { @@ -131,4 +142,142 @@ describe('startCommand', () => { expect(mocks.startRecording).not.toHaveBeenCalled(); expect(mocks.saveSession).not.toHaveBeenCalled(); }); + + it('persists environment ownership before continuing browser startup', async () => { + const environment = { + kind: 'processes', + evidencePath: '/tmp/environment.ndjson', + sources: [], + processes: [], + }; + mocks.loadConfig.mockReturnValue({ + output: './proofshot-artifacts', + headless: true, + viewport: { width: 1280, height: 720 }, + browser: {}, + environment: { + kind: 'processes', + commands: [{ id: 'api', command: 'npm run api' }], + }, + logs: { sources: [] }, + devServer: { + port: 3000, + startupTimeout: 1000, + }, + }); + mocks.startOwnedEnvironment.mockImplementation( + async (_config, _logs, _sessionDir, _sessionName, _startTime, onState) => { + onState(environment); + return environment; + }, + ); + + await startCommand({}); + + expect(mocks.startOwnedEnvironment).toHaveBeenCalledTimes(1); + expect(mocks.openBrowser).toHaveBeenCalledTimes(1); + expect(mocks.saveSession).toHaveBeenLastCalledWith( + expect.objectContaining({ + environment, + recordingActive: true, + }), + ); + }); + + it('refuses to override an active session whose environment cannot be stopped', async () => { + const session = { + outputDir: '/tmp/proofshot-force-override', + environment: { + kind: 'processes', + evidencePath: '/tmp/proofshot-force-override/environment.ndjson', + sources: [], + processes: [], + }, + }; + mocks.hasActiveSession.mockReturnValue(true); + mocks.loadSession.mockReturnValue(session); + mocks.stopOwnedEnvironment.mockRejectedValue( + new AggregateError( + [ + new Error('Log helper for api did not stop.'), + new Error('Owned tmux server did not stop.'), + ], + 'One or more tmux cleanup steps failed.', + ), + ); + + await expect(startCommand({ force: true })).rejects.toThrow('process.exit:1'); + + expect(mocks.stopOwnedEnvironment).toHaveBeenCalledWith(session.environment); + expect(mocks.clearSession).not.toHaveBeenCalled(); + expect(mocks.saveSession).toHaveBeenCalledWith(session); + expect(mocks.openBrowser).not.toHaveBeenCalled(); + + const output = vi + .mocked(console.error) + .mock.calls.map((call) => call.join(' ')) + .join('\n'); + expect(output).toContain('Log helper for api did not stop.'); + expect(output).toContain('Owned tmux server did not stop.'); + expect(output.match(/One or more tmux cleanup steps failed\./g)).toHaveLength(1); + }); + + it('stops the recorded environment before clearing a forced session', async () => { + const session = { + outputDir: '/tmp/proofshot-force-override', + environment: { + kind: 'processes', + evidencePath: '/tmp/proofshot-force-override/environment.ndjson', + sources: [], + processes: [], + }, + }; + mocks.hasActiveSession.mockReturnValue(true); + mocks.loadSession.mockReturnValue(session); + + await startCommand({ force: true }); + + expect(mocks.stopOwnedEnvironment).toHaveBeenCalledTimes(1); + expect(mocks.clearSession).toHaveBeenCalledTimes(1); + expect(mocks.openBrowser).toHaveBeenCalledTimes(1); + }); + + it('retains environment recovery state when failed startup cannot clean it', async () => { + const environment = { + kind: 'processes', + evidencePath: '/tmp/environment.ndjson', + sources: [], + processes: [], + }; + mocks.loadConfig.mockReturnValue({ + output: './proofshot-artifacts', + headless: true, + viewport: { width: 1280, height: 720 }, + browser: {}, + environment: { + kind: 'processes', + commands: [{ id: 'api', command: 'npm run api' }], + }, + logs: { sources: [] }, + devServer: { + port: 3000, + startupTimeout: 1000, + }, + }); + mocks.startOwnedEnvironment.mockImplementation( + async (_config, _logs, _sessionDir, _sessionName, _startTime, onState) => { + onState(environment); + throw new Error('readiness failed'); + }, + ); + mocks.stopOwnedEnvironment.mockRejectedValue(new Error('process still alive')); + + await expect(startCommand({})).rejects.toThrow('process.exit:1'); + + expect(mocks.clearSession).not.toHaveBeenCalled(); + expect(mocks.saveSession).toHaveBeenLastCalledWith( + expect.objectContaining({ environment }), + ); + expect(mocks.openBrowser).not.toHaveBeenCalled(); + }); }); diff --git a/src/commands/start.ts b/src/commands/start.ts index f9632ac..5695351 100644 --- a/src/commands/start.ts +++ b/src/commands/start.ts @@ -12,8 +12,15 @@ import { hasActiveSession, clearSession, generateAgentBrowserSessionName, + type SessionState, } from '../session/state.js'; import { writeMetadata } from '../session/metadata.js'; +import { releaseActiveSessionEnvironment } from '../session/teardown.js'; +import { formatErrorDetail } from '../utils/errors.js'; +import { + startOwnedEnvironment, + stopOwnedEnvironment, +} from '../environment/runtime.js'; interface StartOptions { description?: string; @@ -37,6 +44,19 @@ export async function startCommand(options: StartOptions): Promise { if (hasActiveSession(outputDir)) { if (options.force) { + const cleanupError = await releaseActiveSessionEnvironment(outputDir); + if (cleanupError) { + console.error( + chalk.red('✗') + + ` Refusing to override the active session: ${formatErrorDetail(cleanupError)}`, + ); + console.error( + chalk.dim( + ' Recovery state was retained. Resolve the reported resource issue, then run "proofshot stop" again.', + ), + ); + process.exit(1); + } clearSession(outputDir); console.log(chalk.yellow('⚠') + chalk.dim(' Cleared stale session')); } else { @@ -48,6 +68,14 @@ export async function startCommand(options: StartOptions): Promise { } } + if (options.run && config.environment) { + console.error( + chalk.red('✗') + + ' Use either --run or config.environment, not both.', + ); + process.exit(1); + } + ensureOutputDir(outputDir); const sessionDirName = generateSessionDirName(timestamp, options.description || null); @@ -85,6 +113,50 @@ export async function startCommand(options: StartOptions): Promise { }); let serverAlreadyRunning = true; + const session: SessionState = { + startedAt: new Date().toISOString(), + description: options.description || null, + outputDir, + sessionDir, + sessionName, + videoPath, + serverErrorLog, + port: config.devServer.port, + serverCommand: options.run || null, + serverAlreadyRunning, + recordingActive: false, + environment: null, + viewport: { width: config.viewport.width, height: config.viewport.height }, + }; + const capturesEnvironment = + config.environment !== undefined || (config.logs?.sources || []).length > 0; + + if (capturesEnvironment) { + saveSession(session); + try { + session.environment = await startOwnedEnvironment( + config.environment, + config.logs || {}, + sessionDir, + sessionName, + new Date(session.startedAt).getTime(), + (environment) => { + session.environment = environment; + saveSession(session); + }, + ); + saveSession(session); + console.log(chalk.green('✓') + ' Environment and log capture started'); + } catch (error) { + const cleanupError = await cleanupEnvironmentAfterFailedStart(session); + console.error( + chalk.red('✗') + + ` Failed to start environment capture: ${formatErrorDetail(error)}`, + ); + reportCleanupError(cleanupError); + process.exit(1); + } + } if (options.run) { console.log(chalk.dim(`Starting: ${options.run}`)); @@ -99,7 +171,9 @@ export async function startCommand(options: StartOptions): Promise { console.log(chalk.green('✓') + ` Dev server started on :${config.devServer.port}`); console.log(chalk.dim(` Server logs → ${serverErrorLog}`)); } catch (error: any) { + const cleanupError = await cleanupEnvironmentAfterFailedStart(session); console.error(chalk.red('✗') + ` Failed to start dev server: ${error.message}`); + reportCleanupError(cleanupError); process.exit(1); } } else { @@ -114,12 +188,14 @@ export async function startCommand(options: StartOptions): Promise { openBrowser(openUrl, config.viewport, config.headless, sessionName, config.browser); console.log(chalk.green('✓') + ' Browser ready'); } catch (error: any) { - closeBrowser(); + closeBrowserAfterFailedStart(sessionName); + const cleanupError = await cleanupEnvironmentAfterFailedStart(session); console.error( chalk.red('✗') + ` Failed to open browser: ${error.message}\n` + chalk.dim('Make sure agent-browser is installed: npm install -g agent-browser'), ); + reportCleanupError(cleanupError); process.exit(1); } @@ -147,7 +223,8 @@ export async function startCommand(options: StartOptions): Promise { } if (!recordingStarted) { - closeBrowser(); + closeBrowserAfterFailedStart(sessionName); + const cleanupError = await cleanupEnvironmentAfterFailedStart(session); console.error( chalk.red('✗') + ` Failed to initialize recording after ${RECORDING_RETRIES} attempts: ${lastError?.message}\n` + @@ -157,23 +234,13 @@ export async function startCommand(options: StartOptions): Promise { chalk.dim(' 2. Try "proofshot clean" then re-run "proofshot start"\n') + chalk.dim(' 3. If the port was already in use, stop the old server first'), ); + reportCleanupError(cleanupError); process.exit(1); } - saveSession({ - startedAt: new Date().toISOString(), - description: options.description || null, - outputDir, - sessionDir, - sessionName, - videoPath, - serverErrorLog, - port: config.devServer.port, - serverCommand: options.run || null, - serverAlreadyRunning, - recordingActive: true, - viewport: { width: config.viewport.width, height: config.viewport.height }, - }); + session.serverAlreadyRunning = serverAlreadyRunning; + session.recordingActive = true; + saveSession(session); console.log(''); console.log(chalk.green.bold('✅ ProofShot session started')); @@ -197,3 +264,39 @@ export async function startCommand(options: StartOptions): Promise { console.log(''); console.log(`When done, run: ${chalk.white('proofshot stop')}`); } + +async function cleanupEnvironmentAfterFailedStart( + session: SessionState, +): Promise { + try { + await stopOwnedEnvironment(session.environment); + clearSession(session.outputDir); + return null; + } catch (error) { + saveSession(session); + return error instanceof Error ? error : new Error(String(error)); + } +} + +function reportCleanupError(error: Error | null): void { + if (!error) { + return; + } + console.error( + chalk.yellow('⚠') + + ` Recovery state retained because cleanup failed: ${formatErrorDetail(error)}`, + ); + console.error( + chalk.dim( + ' Resolve the reported resource issue, then run "proofshot stop" again.', + ), + ); +} + +function closeBrowserAfterFailedStart(sessionName: string): void { + try { + closeBrowser(sessionName); + } catch { + // The browser may not have finished creating a session. + } +} diff --git a/src/commands/stop.test.ts b/src/commands/stop.test.ts new file mode 100644 index 0000000..a80a666 --- /dev/null +++ b/src/commands/stop.test.ts @@ -0,0 +1,159 @@ +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { afterEach, beforeEach, describe, expect, it, vi } from 'vitest'; +import { stopCommand } from './stop.js'; +import type { SessionState } from '../session/state.js'; + +const mocks = vi.hoisted(() => ({ + loadConfigForTeardown: vi.fn(), + loadSession: vi.fn(), + clearSession: vi.fn(), + saveSession: vi.fn(), + getConsoleErrors: vi.fn(), + getConsoleOutput: vi.fn(), + getConsoleOutputJson: vi.fn(), + stopOwnedEnvironment: vi.fn(), + recordCaptureHealthFailures: vi.fn(), +})); + +vi.mock('../utils/config.js', () => ({ + loadConfigForTeardown: mocks.loadConfigForTeardown, +})); + +vi.mock('../session/state.js', () => ({ + loadSession: mocks.loadSession, + clearSession: mocks.clearSession, + saveSession: mocks.saveSession, +})); + +vi.mock('../browser/session.js', () => ({ + closeBrowser: vi.fn(), + getConsoleErrors: mocks.getConsoleErrors, + getConsoleOutput: mocks.getConsoleOutput, + getConsoleOutputJson: mocks.getConsoleOutputJson, +})); + +vi.mock('../browser/capture.js', () => ({ + stopRecording: vi.fn(), +})); + +vi.mock('../environment/runtime.js', () => ({ + stopOwnedEnvironment: mocks.stopOwnedEnvironment, + recordCaptureHealthFailures: mocks.recordCaptureHealthFailures, +})); + +describe('stopCommand environment cleanup', () => { + beforeEach(() => { + vi.spyOn(console, 'log').mockImplementation(() => {}); + vi.spyOn(console, 'error').mockImplementation(() => {}); + vi.spyOn(process, 'exit').mockImplementation(((code?: number) => { + throw new Error(`process.exit:${code ?? 0}`); + }) as never); + mocks.loadConfigForTeardown.mockReturnValue({ + config: { + output: '/tmp/proofshot-environment-stop', + browser: {}, + }, + error: null, + }); + mocks.getConsoleErrors.mockReturnValue(''); + mocks.getConsoleOutput.mockReturnValue(''); + mocks.getConsoleOutputJson.mockReturnValue([]); + mocks.recordCaptureHealthFailures.mockReturnValue([]); + }); + + afterEach(() => { + vi.restoreAllMocks(); + Object.values(mocks).forEach((mock) => mock.mockReset()); + }); + + it('retains session state when owned environment cleanup fails', async () => { + const session: SessionState = { + startedAt: new Date().toISOString(), + description: null, + outputDir: '/tmp/proofshot-environment-stop', + sessionDir: '/tmp/proofshot-environment-stop/session', + sessionName: 'proofshot-test', + videoPath: '/tmp/proofshot-environment-stop/session/session.webm', + serverErrorLog: '/tmp/proofshot-environment-stop/session/server.log', + port: 3000, + serverCommand: null, + serverAlreadyRunning: true, + recordingActive: false, + environment: { + kind: 'processes', + evidencePath: '/tmp/proofshot-environment-stop/session/environment.ndjson', + sources: [], + processes: [], + }, + }; + mocks.loadSession.mockReturnValue(session); + mocks.stopOwnedEnvironment.mockRejectedValue( + new AggregateError( + [new Error('Environment process api did not stop.')], + 'One or more environment processes did not stop.', + ), + ); + + await expect(stopCommand({})).rejects.toThrow('process.exit:1'); + + expect(mocks.clearSession).not.toHaveBeenCalled(); + expect(mocks.saveSession).toHaveBeenCalledWith(session); + expect(session.environment).not.toBeNull(); + expect( + vi + .mocked(console.error) + .mock.calls.map((call) => call.join(' ')) + .join('\n'), + ).toContain('Environment process api did not stop.'); + }); + + it('completes cleanup but exits non-zero when a capture stopped early', async () => { + const sessionDir = fs.mkdtempSync( + path.join(os.tmpdir(), 'proofshot-stop-capture-health-'), + ); + const session: SessionState = { + startedAt: new Date().toISOString(), + description: null, + outputDir: sessionDir, + sessionDir, + sessionName: 'proofshot-test', + videoPath: path.join(sessionDir, 'session.webm'), + serverErrorLog: path.join(sessionDir, 'server.log'), + port: 3000, + serverCommand: null, + serverAlreadyRunning: true, + recordingActive: false, + environment: { + kind: 'processes', + evidencePath: path.join(sessionDir, 'environment.ndjson'), + sources: [], + processes: [], + }, + }; + mocks.loadSession.mockReturnValue(session); + mocks.stopOwnedEnvironment.mockResolvedValue(undefined); + mocks.recordCaptureHealthFailures.mockReturnValue([ + 'Capture for "api" stopped before "proofshot stop" — logs/api.log is incomplete.', + ]); + + try { + await expect(stopCommand({})).rejects.toThrow('process.exit:1'); + + expect(mocks.stopOwnedEnvironment).toHaveBeenCalledOnce(); + expect(mocks.clearSession).toHaveBeenCalled(); + expect(fs.readFileSync(path.join(sessionDir, 'SUMMARY.md'), 'utf-8')).toContain( + '## Capture Gaps', + ); + expect( + vi + .mocked(console.log) + .mock.calls.map((call) => call.join(' ')) + .join('\n'), + ).toContain('logs/api.log is incomplete'); + } finally { + fs.rmSync(sessionDir, { recursive: true, force: true }); + } + }); +}); diff --git a/src/commands/stop.ts b/src/commands/stop.ts index 2600ed3..7a7f7fd 100644 --- a/src/commands/stop.ts +++ b/src/commands/stop.ts @@ -2,15 +2,21 @@ import * as fs from 'fs'; import * as path from 'path'; import { execSync } from 'child_process'; import chalk from 'chalk'; -import { loadConfig } from '../utils/config.js'; +import { loadConfigForTeardown } from '../utils/config.js'; +import { formatErrorDetail } from '../utils/errors.js'; import { setAgentBrowserDefaults } from '../utils/exec.js'; import { closeBrowser, getConsoleErrors, getConsoleOutput, getConsoleOutputJson } from '../browser/session.js'; import { stopRecording } from '../browser/capture.js'; -import { loadSession, clearSession } from '../session/state.js'; +import { loadSession, clearSession, saveSession } from '../session/state.js'; import { writeViewer, type TimestampedLogEntry } from '../artifacts/viewer.js'; import { extractServerErrors } from '../utils/error-patterns.js'; import { loadSessionLog } from './exec.js'; import { estimateTokenUsage, formatTokenUsage, type TokenUsage } from '../utils/token-usage.js'; +import { + recordCaptureHealthFailures, + stopOwnedEnvironment, +} from '../environment/runtime.js'; +import type { ResolvedLogSourceState } from '../environment/types.js'; /** * Parse server.log lines with "epochMs\ttext" format. @@ -54,7 +60,13 @@ interface StopOptions { } export async function stopCommand(options: StopOptions): Promise { - const config = loadConfig(); + const { config, error: configError } = loadConfigForTeardown(); + if (configError) { + console.error(chalk.yellow('⚠') + ` ${configError.message}`); + console.error( + chalk.dim(' Continuing teardown with the resolvable output directory.'), + ); + } setAgentBrowserDefaults({ configPath: config.browser.configPath }); const outputDir = path.resolve(config.output); @@ -72,6 +84,8 @@ export async function stopCommand(options: StopOptions): Promise { const startTime = new Date(session.startedAt).getTime(); const durationMs = Date.now() - startTime; const durationSec = Math.round(durationMs / 1000); + const environmentSources = session.environment?.sources || []; + const browserWasActive = session.recordingActive; // Step 1: Collect console errors and output console.log(chalk.dim('Collecting errors...')); @@ -97,16 +111,51 @@ export async function stopCommand(options: StopOptions): Promise { } // Step 2: Stop recording - console.log(chalk.dim('Stopping recording...')); - stopRecording(session.sessionName); + if (session.recordingActive) { + console.log(chalk.dim('Stopping recording...')); + stopRecording(session.sessionName); + session.recordingActive = false; + saveSession(session); + } // Step 3: Close browser (unless --no-close) - if (!options.noClose) { + if (!options.noClose && browserWasActive) { console.log(chalk.dim('Closing browser...')); closeBrowser(session.sessionName); } - // Step 4: Read server log (with timestamp parsing) + // Step 4: Verify capture health, then stop the owned environment and helpers. + // Cleanup still runs to completion when a capture died — the incomplete + // evidence is reported at the end instead of being presented as a clean run. + const captureHealthFailures = recordCaptureHealthFailures( + session.environment, + startTime, + ); + if (captureHealthFailures.length > 0) { + saveSession(session); + } + if (session.environment) { + console.log(chalk.dim('Stopping environment capture...')); + try { + await stopOwnedEnvironment(session.environment); + } catch (error) { + saveSession(session); + console.error( + chalk.red('✗') + + ` Failed to stop environment capture: ${formatErrorDetail(error)}`, + ); + console.error( + chalk.dim( + ' Recovery state was retained. Resolve the resource issue, then run "proofshot stop" again.', + ), + ); + process.exit(1); + } + session.environment = null; + saveSession(session); + } + + // Step 5: Read server log (with timestamp parsing) let serverLog = ''; let serverEntries: TimestampedLogEntry[] = []; if (fs.existsSync(session.serverErrorLog)) { @@ -129,7 +178,7 @@ export async function stopCommand(options: StopOptions): Promise { let trimOffsetSec = 0; if (fs.existsSync(session.videoPath)) { trimOffsetSec = trimVideo(session.videoPath, screenshots, sessionDir, startTime, sessionLog); - } else if (session.recordingActive) { + } else if (browserWasActive) { console.log( chalk.yellow('⚠') + ' Recording was active but no video file was produced.\n' + @@ -165,6 +214,8 @@ export async function stopCommand(options: StopOptions): Promise { tokenUsage, durationSec, outputDir: sessionDir, + environmentSources, + captureHealthFailures, }); fs.writeFileSync(summaryPath, summary); @@ -213,13 +264,23 @@ export async function stopCommand(options: StopOptions): Promise { // Step 9: Print results console.log(''); - console.log(chalk.green.bold('✅ ProofShot verification complete')); + console.log( + captureHealthFailures.length > 0 + ? chalk.red.bold('❌ ProofShot verification incomplete — environment capture stopped early') + : chalk.green.bold('✅ ProofShot verification complete'), + ); console.log(''); if (fs.existsSync(session.videoPath)) { console.log(`📹 Video: ${chalk.dim(session.videoPath)} (${durationSec}s)`); } console.log(`📸 Screenshots: ${screenshots.length} captured`); + if (environmentSources.length > 0) { + console.log( + `📚 Environment: ${environmentSources.length} source(s) → ${chalk.dim(path.join(sessionDir, 'logs'))}` + + (captureHealthFailures.length > 0 ? chalk.red(' (incomplete)') : ''), + ); + } console.log(`📝 Summary: ${chalk.dim(summaryPath)}`); if (viewerPath) { console.log(`🎬 Viewer: ${chalk.dim(viewerPath)}`); @@ -259,6 +320,22 @@ export async function stopCommand(options: StopOptions): Promise { console.log(chalk.dim(` ... and ${serverErrorLines.length - 10} more (see SUMMARY.md)`)); } } + + // Incomplete evidence must never exit 0 — every owned resource has been + // released by this point, so only the reporting outcome is left to decide. + if (captureHealthFailures.length > 0) { + console.log(''); + console.log(chalk.red.bold('Capture gaps:')); + for (const failure of captureHealthFailures) { + console.log(chalk.red(` ${failure}`)); + } + console.log( + chalk.dim( + ` Recorded with captureGap in ${path.join(sessionDir, 'environment.ndjson')}`, + ), + ); + process.exit(1); + } } interface SummaryData { @@ -274,6 +351,8 @@ interface SummaryData { tokenUsage?: TokenUsage | null; durationSec: number; outputDir: string; + environmentSources: ResolvedLogSourceState[]; + captureHealthFailures: string[]; } function generateProofSummary(data: SummaryData): string { @@ -343,6 +422,23 @@ Full session recording: [${relativeVideo}](./${relativeVideo}) (${data.durationS md += '\n'; } + if (data.environmentSources.length > 0) { + md += `## Environment Logs\n\n`; + for (const source of data.environmentSources) { + md += `- **${source.title}** (${source.group}/${source.kind}): [${path.basename(source.logPath)}](./logs/${path.basename(source.logPath)})\n`; + } + md += '\n'; + } + + if (data.captureHealthFailures.length > 0) { + md += `## Capture Gaps\n\n`; + md += `Environment capture stopped early, so the logs above are incomplete:\n\n`; + for (const failure of data.captureHealthFailures) { + md += `- ${failure}\n`; + } + md += `\nEach gap is recorded in \`environment.ndjson\` with \`captureGap: true\`.\n\n`; + } + // Environment md += `## Environment - Browser: Chromium (headless) diff --git a/src/environment/evidence.test.ts b/src/environment/evidence.test.ts new file mode 100644 index 0000000..ac0a788 --- /dev/null +++ b/src/environment/evidence.test.ts @@ -0,0 +1,36 @@ +import { describe, expect, it } from 'vitest'; +import { normalizeLogText } from './evidence.js'; + +const ESC = '\u001B'; +const BEL = '\u0007'; + +describe('normalizeLogText', () => { + it('strips BEL-terminated OSC sequences without leaving residue', () => { + expect(normalizeLogText(`${ESC}]0;my-title${BEL}ready in 431 ms`)).toBe( + 'ready in 431 ms', + ); + expect( + normalizeLogText(`${ESC}]2;dev server${BEL}${ESC}[32mVITE ready${ESC}[0m`), + ).toBe('VITE ready'); + }); + + it('strips ST-terminated titles and leaves unterminated escapes line-local', () => { + expect(normalizeLogText(`${ESC}]0;npm run dev${ESC}\\listening`)).toBe( + 'listening', + ); + // An unterminated sequence must never consume the line that follows it. + expect( + normalizeLogText(`${ESC}]0;never closed\nreal line`).endsWith('\nreal line'), + ).toBe(true); + }); + + it('strips remaining control characters and normalizes newlines', () => { + expect(normalizeLogText(`a${BEL}bc\r\nd\rE`)).toBe('abc\nd\nE'); + }); + + it('keeps ANSI colors when stripping is disabled', () => { + expect(normalizeLogText(`${ESC}[31mred${ESC}[0m`, false)).toBe( + `${ESC}[31mred${ESC}[0m`, + ); + }); +}); diff --git a/src/environment/evidence.ts b/src/environment/evidence.ts new file mode 100644 index 0000000..4e15cd6 --- /dev/null +++ b/src/environment/evidence.ts @@ -0,0 +1,87 @@ +import * as fs from 'fs'; +import type { EvidenceEvent } from './types.js'; + +const ANSI_PATTERN = + // eslint-disable-next-line no-control-regex + /[\u001B\u009B][[\]()#;?]*(?:(?:(?:[a-zA-Z\d]*(?:;[-a-zA-Z\d/#&.:=?%@~_]+)*)?\u0007)|(?:(?:\d{1,4}(?:[;:]\d{0,4})*)?[\dA-PR-TZcf-nq-uy=><~]))/g; +const CONTROL_PATTERN = /[\u0000-\u0008\u000B\u000C\u000E-\u001A\u001C-\u001F\u007F]/g; + +// OSC bodies carry arbitrary text — shells and npm scripts set titles such as +// "npm run dev" — which the body class inside ANSI_PATTERN cannot express. The +// body stops at a newline or a further escape, so an unterminated sequence can +// never swallow real log lines. +const OSC_PATTERN = + // eslint-disable-next-line no-control-regex + /(?:\u001B\]|\u009D)[^\u0007\u001B\u009C\n]*(?:\u0007|\u001B\\|\u009C)/g; + +/** + * Escape sequences are stripped before the remaining control characters, + * because OSC sequences terminate on BEL or ST — both of which + * `CONTROL_PATTERN` would otherwise remove first, leaving the sequence body + * spliced into the surrounding log text. + */ +export function normalizeLogText(text: string, stripAnsi = true): string { + const normalized = text.replace(/\r\n?/g, '\n'); + return ( + stripAnsi + ? normalized.replace(OSC_PATTERN, '').replace(ANSI_PATTERN, '') + : normalized + ).replace(CONTROL_PATTERN, ''); +} + +export function appendEvidenceEvent(filePath: string, event: EvidenceEvent): void { + fs.appendFileSync(filePath, JSON.stringify(event) + '\n'); +} + +export function loadEvidenceEvents(filePath: string): EvidenceEvent[] { + if (!fs.existsSync(filePath)) { + return []; + } + + return fs + .readFileSync(filePath, 'utf-8') + .split('\n') + .filter(Boolean) + .map((line, index) => { + try { + const parsed: unknown = JSON.parse(line); + return isEvidenceEvent(parsed) + ? parsed + : malformedEvidenceEvent(index + 1); + } catch { + return malformedEvidenceEvent(index + 1); + } + }); +} + +function isEvidenceEvent(value: unknown): value is EvidenceEvent { + if (typeof value !== 'object' || value === null) return false; + const event = value as Partial; + return ( + event.version === 1 && + (event.origin === 'environment' || event.origin === 'browser') && + typeof event.group === 'string' && + typeof event.sourceId === 'string' && + typeof event.sourceTitle === 'string' && + typeof event.text === 'string' && + (event.relativeTimeSec === null || + (typeof event.relativeTimeSec === 'number' && + Number.isFinite(event.relativeTimeSec))) + ); +} + +function malformedEvidenceEvent(line: number): EvidenceEvent { + return { + version: 1, + origin: 'environment', + group: 'environment', + sourceId: 'capture-health', + sourceTitle: 'Capture health', + stream: 'stderr', + segment: 'live', + timestamp: null, + relativeTimeSec: null, + text: `[malformed canonical evidence row at line ${line}]`, + captureGap: true, + }; +} diff --git a/src/environment/runtime.test.ts b/src/environment/runtime.test.ts new file mode 100644 index 0000000..42974d8 --- /dev/null +++ b/src/environment/runtime.test.ts @@ -0,0 +1,876 @@ +import * as fs from 'fs'; +import * as os from 'os'; +import * as path from 'path'; +import { execFileSync } from 'child_process'; +import { afterEach, beforeEach, describe, expect, it } from 'vitest'; +import { loadEvidenceEvents } from './evidence.js'; +import { + recordCaptureHealthFailures, + startOwnedEnvironment, + stopOwnedEnvironment, +} from './runtime.js'; +import type { + EnvironmentState, + LauncherEnvironmentState, + ProcessEnvironmentState, + TmuxEnvironmentState, +} from './types.js'; +import { processIdentityMatches } from '../utils/process.js'; + +let root: string; +const states: EnvironmentState[] = []; +const extraTmuxSockets: string[] = []; + +/** + * tmux coverage is mandatory in CI, which installs the binary explicitly, and + * skipped on contributor machines that do not have it. + */ +const tmuxAvailable = (() => { + try { + execFileSync('tmux', ['-V'], { stdio: 'ignore' }); + return true; + } catch { + return false; + } +})(); +const skipWithoutTmux = !tmuxAvailable && !process.env.CI; + +beforeEach(() => { + root = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-environment-test-')); +}); + +afterEach(async () => { + for (const state of states.splice(0)) { + await stopOwnedEnvironment(state).catch(() => {}); + } + for (const socket of extraTmuxSockets.splice(0)) { + try { + execFileSync('tmux', ['-S', socket, 'kill-server'], { stdio: 'ignore' }); + } catch { + // The fixture may already have exited. + } + } + fs.rmSync(root, { recursive: true, force: true }); +}); + +describe('owned environment capture', () => { + it.skipIf(skipWithoutTmux)('captures named tmux panes as history and live PTY evidence', async () => { + const sessionDir = path.join(root, 'session'); + fs.mkdirSync(sessionDir, { recursive: true }); + let latestState: EnvironmentState | null = null; + const started = await startOwnedEnvironment( + { + kind: 'tmux', + launch: { + kind: 'panes', + panes: [ + // The live markers repeat so the assertions below hold however long + // the capture pipe takes to attach; a single line printed once would + // land in the history snapshot instead on a loaded machine. + { + id: 'vite', + title: 'Vite', + group: 'frontend', + command: + "printf 'vite-history\\n'; while :; do printf '\\033[31mvite-live\\033[0m\\n'; sleep 0.2; done", + }, + { + id: 'api', + title: 'API', + group: 'backend', + command: + "printf 'api-history\\n'; while :; do printf 'api-live\\n'; sleep 0.2; done", + }, + ], + }, + }, + { + stripAnsi: true, + sources: [ + { + id: 'frontend-vite', + group: 'frontend', + kind: 'tmux-pane', + match: { connectionKey: 'vite' }, + }, + { + id: 'backend-api', + group: 'backend', + kind: 'tmux-pane', + match: { connectionKey: 'api' }, + }, + ], + }, + sessionDir, + 'ps-tmux-test', + Date.now(), + (updated) => { + latestState = updated; + }, + ); + if (started) states.push(started); + const state = requireTmuxState(started); + + await waitForLiveEvidence(state.evidencePath, { + 'frontend-vite': 'vite-live', + 'backend-api': 'api-live', + }); + const events = loadEvidenceEvents(state.evidencePath); + expect(events).toEqual( + expect.arrayContaining([ + expect.objectContaining({ + sourceId: 'frontend-vite', + sourceTitle: 'Vite', + group: 'frontend', + stream: 'pty', + text: 'vite-history', + }), + expect.objectContaining({ + sourceId: 'frontend-vite', + segment: 'history', + text: '[tmux history/live capture boundary]', + }), + expect.objectContaining({ + sourceId: 'frontend-vite', + segment: 'live', + text: 'vite-live', + }), + expect.objectContaining({ + sourceId: 'backend-api', + sourceTitle: 'API', + group: 'backend', + text: 'api-live', + }), + ]), + ); + expect(fs.readFileSync(path.join(sessionDir, 'logs', 'frontend-vite.log'), 'utf-8')) + .not.toContain('\u001b'); + expect(latestState).toMatchObject({ + kind: 'tmux', + ownsServer: true, + ownsSession: true, + }); + + await stopOwnedEnvironment(state); + expect(processIdentityMatches(state.serverProcess)).toBe(false); + expect(fs.existsSync(state.socket.path)).toBe(false); + await expect(stopOwnedEnvironment(state)).resolves.toBeUndefined(); + states.pop(); + }, 15000); + + it.skipIf(skipWithoutTmux)('does not kill a shared external tmux session', async () => { + const socket = path.join(root, 'shared.sock'); + extraTmuxSockets.push(socket); + execFileSync( + 'tmux', + [ + '-S', + socket, + 'new-session', + '-d', + '-s', + 'shared', + "sleep 0.5; printf 'shared-live\\n'; sleep 30", + ], + { stdio: 'ignore' }, + ); + const sessionDir = path.join(root, 'shared-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + const started = await startOwnedEnvironment( + { + kind: 'tmux', + launch: { + kind: 'external-command', + command: `printf 'tmux -S ${socket} attach -t shared\\n'`, + }, + connection: { + source: 'stdout', + format: 'tmux-attach-command', + ownership: 'attach', + }, + }, + { + sources: [ + { + id: 'shared-pane', + kind: 'tmux-pane', + match: { target: 'shared:0.0' }, + }, + ], + }, + sessionDir, + 'ps-shared-test', + Date.now(), + () => {}, + ); + if (started) states.push(started); + const state = requireTmuxState(started); + expect(state.ownsServer).toBe(false); + expect(state.ownsSession).toBe(false); + await waitForEvidence(state.evidencePath, ['shared-live']); + + await stopOwnedEnvironment(state); + expect(() => + execFileSync('tmux', ['-S', socket, 'has-session', '-t', 'shared']), + ).not.toThrow(); + states.pop(); + }, 15000); + + it.skipIf(skipWithoutTmux)('never owns a tmux server its attach-only launcher created', async () => { + const socket = path.join(root, 'attach-created.sock'); + extraTmuxSockets.push(socket); + expect(fs.existsSync(socket)).toBe(false); + const sessionDir = path.join(root, 'attach-created-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + + const started = await startOwnedEnvironment( + { + kind: 'tmux', + launch: { + kind: 'external-command', + command: + `tmux -S ${socket} new-session -d -s created "printf 'created-live\\n'; sleep 30" && ` + + `printf 'tmux -S ${socket} attach -t created\\n'`, + }, + connection: { + source: 'stdout', + format: 'tmux-attach-command', + ownership: 'attach', + socket, + }, + }, + { + sources: [ + { + id: 'created-pane', + kind: 'tmux-pane', + match: { target: 'created:0.0' }, + }, + ], + }, + sessionDir, + 'ps-attach-created', + Date.now(), + () => {}, + ); + if (started) states.push(started); + const state = requireTmuxState(started); + + expect(state.ownsServer).toBe(false); + expect(state.ownsSession).toBe(false); + await waitForEvidence(state.evidencePath, ['created-live']); + + await stopOwnedEnvironment(state); + expect( + execFileSync( + 'tmux', + ['-S', socket, 'display-message', '-p', '-t', 'created:0.0', '#{pane_pipe}'], + { encoding: 'utf-8' }, + ).trim(), + ).toBe('0'); + expect(() => + execFileSync('tmux', ['-S', socket, 'has-session', '-t', 'created']), + ).not.toThrow(); + states.pop(); + }, 15000); + + it.skipIf(skipWithoutTmux)('reports a capture gap when a tmux pane dies mid-session', async () => { + const sessionDir = path.join(root, 'pane-death-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + const started = await startOwnedEnvironment( + { + kind: 'tmux', + launch: { + kind: 'panes', + panes: [ + { id: 'api', command: "printf 'api-live\\n'; sleep 30" }, + { id: 'web', command: "printf 'web-live\\n'; sleep 30" }, + ], + }, + }, + {}, + sessionDir, + 'ps-pane-death', + Date.now(), + () => {}, + ); + if (started) states.push(started); + const state = requireTmuxState(started); + await waitForEvidence(state.evidencePath, ['api-live', 'web-live']); + expect(recordCaptureHealthFailures(state)).toEqual([]); + + const apiPane = state.panes.find((pane) => pane.sourceId === 'api'); + const apiCapture = state.captures.find((capture) => capture.sourceId === 'api'); + if (!apiPane || !apiCapture) throw new Error('expected a recorded api pane'); + execFileSync( + 'tmux', + ['-S', state.socket.path, 'kill-pane', '-t', apiPane.paneId], + { stdio: 'ignore' }, + ); + // The pipe helper exits cleanly on the pane's EOF, so its pid file is gone + // and only the pane's own pipe state can still reveal the gap. + await waitFor(() => !fs.existsSync(apiCapture.pidFile)); + + const failures = recordCaptureHealthFailures(state, Date.now()); + expect(failures).toHaveLength(1); + expect(failures[0]).toContain('"api"'); + expect(state.healthFailures).toEqual(failures); + expect( + loadEvidenceEvents(state.evidencePath).filter( + (event) => event.captureGap && event.sourceId === 'api', + ), + ).toHaveLength(1); + + await stopOwnedEnvironment(state); + states.pop(); + }, 15000); + + it.skipIf(skipWithoutTmux)('does not fabricate pane gaps when a failed stop is retried', async () => { + const socket = path.join(root, 'retry.sock'); + extraTmuxSockets.push(socket); + execFileSync( + 'tmux', + [ + '-S', + socket, + 'new-session', + '-d', + '-s', + 'shared', + "sleep 0.5; printf 'shared-live\\n'; sleep 30", + ], + { stdio: 'ignore' }, + ); + const sessionDir = path.join(root, 'retry-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + const started = await startOwnedEnvironment( + { + kind: 'tmux', + launch: { + kind: 'external-command', + command: `printf 'tmux -S ${socket} attach -t shared\\n'`, + }, + connection: { + source: 'stdout', + format: 'tmux-attach-command', + ownership: 'attach', + }, + }, + { + sources: [ + { + id: 'shared-pane', + kind: 'tmux-pane', + match: { target: 'shared:0.0' }, + }, + ], + }, + sessionDir, + 'ps-stop-retry', + Date.now(), + () => {}, + ); + if (started) states.push(started); + const state = requireTmuxState(started); + await waitForEvidence(state.evidencePath, ['shared-live']); + expect(recordCaptureHealthFailures(state)).toEqual([]); + + // Fail a teardown step that runs after the pipes are already detached, + // which is the state the documented "run stop again" recovery starts from. + state.stopCommand = 'exit 1'; + await expect(stopOwnedEnvironment(state)).rejects.toThrow(); + expect(state.panes.every((pane) => !pane.captureAttached)).toBe(true); + + state.stopCommand = undefined; + expect(recordCaptureHealthFailures(state, Date.now())).toEqual([]); + expect(state.healthFailures).toBeUndefined(); + expect( + loadEvidenceEvents(state.evidencePath).some((event) => event.captureGap), + ).toBe(false); + await expect(stopOwnedEnvironment(state)).resolves.toBeUndefined(); + + expect(() => + execFileSync('tmux', ['-S', socket, 'has-session', '-t', 'shared']), + ).not.toThrow(); + states.pop(); + }, 15000); + + it.skipIf(skipWithoutTmux)('never runs an attach-only stop command when start fails', async () => { + const marker = path.join(root, 'attach-stop-marker'); + const sessionDir = path.join(root, 'attach-stop-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + + await expect( + startOwnedEnvironment( + { + kind: 'tmux', + launch: { + kind: 'external-command', + command: "printf 'not-json\\n'", + stopCommand: `printf 'ran\\n' > ${marker}`, + }, + connection: { format: 'json', ownership: 'attach' }, + }, + { sources: [] }, + sessionDir, + 'ps-attach-stop', + Date.now(), + () => {}, + ), + ).rejects.toThrow(); + + expect(fs.existsSync(marker)).toBe(false); + }, 15000); + + it('reports a capture gap when a capture worker dies mid-session', async () => { + const sessionDir = path.join(root, 'capture-health-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + const started = await startOwnedEnvironment( + { + kind: 'processes', + commands: [{ id: 'api', command: "printf 'api-ready\\n'; sleep 30" }], + }, + {}, + sessionDir, + 'ps-capture-health', + Date.now(), + () => {}, + ); + if (started) states.push(started); + const state = requireProcessState(started); + await waitForEvidence(state.evidencePath, ['api-ready']); + expect(recordCaptureHealthFailures(state)).toEqual([]); + + const capture = state.processes[0]; + process.kill(capture.process.pid, 'SIGKILL'); + await waitFor(() => !processIdentityMatches(capture.process)); + + const failures = recordCaptureHealthFailures(state, Date.now()); + expect(failures).toHaveLength(1); + expect(failures[0]).toContain('"api"'); + expect(state.healthFailures).toEqual(failures); + expect( + loadEvidenceEvents(state.evidencePath).filter( + (event) => event.captureGap && event.sourceId === 'api', + ), + ).toHaveLength(1); + + await stopOwnedEnvironment(state); + states.pop(); + }, 15000); + + it.skipIf(skipWithoutTmux)('persists and cleans a timed-out external launcher identity', async () => { + const sessionDir = path.join(root, 'timed-out-launcher'); + fs.mkdirSync(sessionDir, { recursive: true }); + let pendingState: EnvironmentState | null = null; + + await expect( + startOwnedEnvironment( + { + kind: 'tmux', + launch: { + kind: 'external-command', + command: 'sleep 30', + timeoutMs: 100, + }, + connection: { + format: 'json', + ownership: 'attach', + }, + }, + { sources: [] }, + sessionDir, + 'ps-timed-out-launcher', + Date.now(), + (state) => { + pendingState = state; + }, + ), + ).rejects.toThrow(/timed out/); + + expect(pendingState).toMatchObject({ kind: 'launcher' }); + const launcherState = requireLauncherState(pendingState); + expect(processIdentityMatches(launcherState.launcher.process)).toBe(false); + }, 15000); + + it('preserves direct stdout/stderr and file history/live evidence', async () => { + const sessionDir = path.join(root, 'process-session'); + const filePath = path.join(root, 'worker.log'); + fs.mkdirSync(sessionDir, { recursive: true }); + fs.writeFileSync(filePath, 'file-history\n'); + const started = await startOwnedEnvironment( + { + kind: 'processes', + commands: [ + { + id: 'api', + title: 'API', + group: 'backend', + command: "printf 'process-out\\n'; printf 'process-err\\n' >&2; sleep 30", + }, + ], + }, + { + sources: [ + { + id: 'api', + kind: 'process', + processId: 'api', + group: 'backend', + }, + { + id: 'worker', + kind: 'file', + path: filePath, + group: 'backend', + }, + ], + }, + sessionDir, + 'ps-process-test', + Date.now(), + () => {}, + ); + if (started) states.push(started); + fs.appendFileSync(filePath, 'file-live\n'); + const state = requireProcessState(started); + const fileCapture = state.processes.find( + (capture) => capture.sourceId === 'worker', + ); + expect(fileCapture && processIdentityMatches(fileCapture.process)).toBe(true); + await waitForEvidence( + state.evidencePath, + ['process-out', 'process-err', 'file-history', 'file-live'], + ); + + const events = loadEvidenceEvents(state.evidencePath); + expect(events).toEqual( + expect.arrayContaining([ + expect.objectContaining({ sourceId: 'api', stream: 'stdout', text: 'process-out' }), + expect.objectContaining({ sourceId: 'api', stream: 'stderr', text: 'process-err' }), + expect.objectContaining({ + sourceId: 'worker', + stream: 'file', + segment: 'history', + text: 'file-history', + }), + expect.objectContaining({ + sourceId: 'worker', + stream: 'file', + segment: 'live', + text: 'file-live', + }), + ]), + ); + + expect( + fs.readdirSync(path.join(sessionDir, 'logs')).sort(), + ).toEqual(['api.log', 'worker.log']); + + await stopOwnedEnvironment(state); + expect( + state.processes.every((capture) => !processIdentityMatches(capture.process)), + ).toBe(true); + states.pop(); + }, 15000); + + it('refuses a log source the environment kind cannot capture', async () => { + const sessionDir = path.join(root, 'mismatched-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + + await expect( + startOwnedEnvironment( + { + kind: 'processes', + commands: [{ id: 'api', command: 'sleep 30' }], + }, + { + sources: [ + { id: 'vite', kind: 'tmux-pane', match: { tag: 'vite' } }, + ], + }, + sessionDir, + 'ps-source-kind-mismatch', + Date.now(), + () => {}, + ), + ).rejects.toThrow(/cannot be captured by environment kind "processes"/); + expect(fs.existsSync(path.join(sessionDir, 'logs'))).toBe(false); + }); + + it('launches every process definition when only some sources are customized', async () => { + const sessionDir = path.join(root, 'all-processes-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + const started = await startOwnedEnvironment( + { + kind: 'processes', + commands: [ + { id: 'api', command: "printf 'api-ready\\n'; sleep 30" }, + { id: 'worker', command: "printf 'worker-ready\\n'; sleep 30" }, + ], + }, + { + sources: [ + { + id: 'custom-api', + kind: 'process', + processId: 'api', + }, + ], + }, + sessionDir, + 'ps-all-processes', + Date.now(), + () => {}, + ); + if (started) states.push(started); + const state = requireProcessState(started); + + expect(state.processes.map((capture) => capture.sourceId).sort()).toEqual([ + 'custom-api', + 'worker', + ]); + await waitForEvidence(state.evidencePath, ['api-ready', 'worker-ready']); + + await stopOwnedEnvironment(state); + states.pop(); + }, 15000); + + it.skipIf(skipWithoutTmux)('refuses to replace a pre-existing pipe-pane consumer', async () => { + const socket = path.join(root, 'p.sock'); + extraTmuxSockets.push(socket); + execFileSync( + 'tmux', + ['-S', socket, 'new-session', '-d', '-s', 'shared', 'sleep 30'], + { stdio: 'ignore' }, + ); + execFileSync( + 'tmux', + [ + '-S', + socket, + 'pipe-pane', + '-t', + 'shared:0.0', + `cat > ${path.join(root, 'existing.log')}`, + ], + { stdio: 'ignore' }, + ); + const sessionDir = path.join(root, 'preexisting-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + + await expect( + startOwnedEnvironment( + { + kind: 'tmux', + launch: { + kind: 'external-command', + command: `printf 'tmux -S ${socket} attach -t shared\\n'`, + }, + connection: { + source: 'stdout', + format: 'tmux-attach-command', + ownership: 'attach', + }, + }, + { + sources: [ + { + id: 'shared-pane', + kind: 'tmux-pane', + match: { target: 'shared:0.0' }, + }, + ], + }, + sessionDir, + 'ps-existing-pipe', + Date.now(), + () => {}, + ), + ).rejects.toThrow(/already has a pipe-pane consumer/); + + expect( + execFileSync( + 'tmux', + ['-S', socket, 'display-message', '-p', '-t', 'shared:0.0', '#{pane_pipe}'], + { encoding: 'utf-8' }, + ).trim(), + ).toBe('1'); + }, 15000); + + it('cleans process ownership when readiness fails', async () => { + const sessionDir = path.join(root, 'readiness-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + let latestState: EnvironmentState | null = null; + + await expect( + startOwnedEnvironment( + { + kind: 'processes', + commands: [ + { + id: 'api', + command: 'sleep 30', + }, + ], + readiness: [ + { + kind: 'http', + url: 'http://127.0.0.1:1/health', + timeoutMs: 200, + }, + ], + }, + {}, + sessionDir, + 'ps-readiness-failure', + Date.now(), + (state) => { + latestState = state; + }, + ), + ).rejects.toThrow(/Environment readiness failed/); + + const recoveryState = requireProcessState(latestState); + expect( + recoveryState.processes.every( + (capture) => !processIdentityMatches(capture.process), + ), + ).toBe(true); + }, 15000); + + it('records truncation and cleans descendants after a coordinator dies', async () => { + const sessionDir = path.join(root, 'truncation-session'); + fs.mkdirSync(sessionDir, { recursive: true }); + const started = await startOwnedEnvironment( + { + kind: 'processes', + commands: [ + { + id: 'noisy', + command: `${process.execPath} -e "console.log('x'.repeat(1024)); setInterval(() => {}, 1000)"`, + }, + ], + }, + { + maxBytesPerSource: 512, + }, + sessionDir, + 'ps-truncation', + Date.now(), + () => {}, + ); + if (started) states.push(started); + const state = requireProcessState(started); + await waitForEvidence(state.evidencePath, ['capture truncated']); + expect( + loadEvidenceEvents(state.evidencePath).some((event) => event.truncated), + ).toBe(true); + expect( + fs.statSync(state.evidencePath).size + + fs.statSync(path.join(sessionDir, 'logs', 'noisy.log')).size, + ).toBeLessThanOrEqual(512); + const coordinator = state.processes[0].process; + process.kill(coordinator.pid, 'SIGKILL'); + await new Promise((resolve) => setTimeout(resolve, 100)); + + await stopOwnedEnvironment(state); + expect(processIdentityMatches(coordinator)).toBe(false); + states.pop(); + }, 15000); +}); + +function requireProcessState( + state: EnvironmentState | null, +): ProcessEnvironmentState { + if (!state || state.kind !== 'processes') { + throw new Error('expected process state'); + } + return state; +} + +function requireTmuxState( + state: EnvironmentState | null, +): TmuxEnvironmentState { + if (!state || state.kind !== 'tmux') { + throw new Error('expected tmux state'); + } + return state; +} + +function requireLauncherState( + state: EnvironmentState | null, +): LauncherEnvironmentState { + if (!state || state.kind !== 'launcher') { + throw new Error('expected persisted launcher state'); + } + return state; +} + +async function waitFor( + condition: () => boolean, + timeoutMs = 2000, +): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + if (condition()) return; + await new Promise((resolve) => setTimeout(resolve, 25)); + } + throw new Error('Condition was not met before the timeout.'); +} + +/** Wait until each source has recorded the given text in its live segment. */ +async function waitForLiveEvidence( + evidencePath: string, + expected: Record, + timeoutMs = 5000, +): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + const events = loadEvidenceEvents(evidencePath); + const pending = Object.entries(expected).filter( + ([sourceId, text]) => + !events.some( + (event) => + event.sourceId === sourceId && + event.segment === 'live' && + event.text.includes(text), + ), + ); + if (pending.length === 0) return; + await new Promise((resolve) => setTimeout(resolve, 50)); + } + const raw = fs.existsSync(evidencePath) + ? fs.readFileSync(evidencePath, 'utf-8') + : ''; + throw new Error( + `Missing live evidence: ${JSON.stringify(expected)}\n${raw}`, + ); +} + +async function waitForEvidence( + evidencePath: string, + texts: string[], + timeoutMs = 5000, +): Promise { + const deadline = Date.now() + timeoutMs; + while (Date.now() < deadline) { + const events = loadEvidenceEvents(evidencePath); + if (texts.every((text) => events.some((event) => event.text.includes(text)))) { + return; + } + await new Promise((resolve) => setTimeout(resolve, 50)); + } + const raw = fs.existsSync(evidencePath) + ? fs.readFileSync(evidencePath, 'utf-8') + : ''; + const captureDir = path.join(path.dirname(evidencePath), '.capture'); + const helperErrors = fs.existsSync(captureDir) + ? fs + .readdirSync(captureDir) + .filter((file) => file.endsWith('.stderr')) + .map((file) => `${file}:\n${fs.readFileSync(path.join(captureDir, file), 'utf-8')}`) + .join('\n') + : ''; + throw new Error(`Missing evidence: ${texts.join(', ')}\n${raw}\n${helperErrors}`); +} diff --git a/src/environment/runtime.ts b/src/environment/runtime.ts new file mode 100644 index 0000000..7f3f2ae --- /dev/null +++ b/src/environment/runtime.ts @@ -0,0 +1,460 @@ +import * as fs from 'fs'; +import * as net from 'net'; +import * as path from 'path'; +import { appendEvidenceEvent } from './evidence.js'; +import { startFileCapture, startProcessCapture } from './workers.js'; +import { + findUnpipedPanes, + startTmuxEnvironment, + stopTmuxEnvironment, +} from './tmux.js'; +import type { + EnvironmentConfig, + EnvironmentState, + LogSourceConfig, + LogsConfig, + ProcessDefinition, + ProcessEnvironmentState, + ReadinessCheck, + ResolvedLogSourceState, + TmuxEnvironmentConfig, + TmuxEnvironmentState, + ProcessesEnvironmentConfig, +} from './types.js'; +import { + ownedProcessTreeIsAlive, + processIdentityMatches, + terminateOwnedProcessTree, +} from '../utils/process.js'; + +export async function startOwnedEnvironment( + environment: EnvironmentConfig | undefined, + logs: LogsConfig, + sessionDir: string, + sessionName: string, + startTimeMs: number, + onState: (state: EnvironmentState) => void, +): Promise { + assertSourcesMatchEnvironment(environment, logs); + const fileSources = (logs.sources || []).filter( + (source): source is Extract => + source.kind === 'file', + ); + if (!environment && fileSources.length === 0) { + return null; + } + + let state: EnvironmentState; + if (environment?.kind === 'tmux') { + state = await startTmuxEnvironment( + environment, + logs, + sessionDir, + sessionName, + startTimeMs, + onState, + ); + } else { + state = await startProcessEnvironment( + environment?.kind === 'processes' ? environment.commands : [], + logs, + sessionDir, + startTimeMs, + onState, + ); + } + + try { + state = await attachFileSources( + state, + fileSources, + logs, + sessionDir, + startTimeMs, + onState, + ); + if (environment) { + await waitForReadiness(environment.readiness || []); + } + return state; + } catch (error) { + await stopOwnedEnvironment(state).catch(() => {}); + throw error; + } +} + +/** + * Only the environment kind that owns a source can capture it, so a mismatch + * must fail closed instead of dropping declared evidence. + */ +function assertSourcesMatchEnvironment( + environment: EnvironmentConfig | undefined, + logs: LogsConfig, +): void { + const requiredKind = + environment?.kind === 'tmux' + ? 'tmux-pane' + : environment?.kind === 'processes' + ? 'process' + : undefined; + for (const source of logs.sources || []) { + if (source.kind === 'file' || source.kind === requiredKind) continue; + throw new Error( + `Log source ${source.id} of kind "${source.kind}" cannot be captured by environment kind "${ + environment?.kind || 'none' + }".`, + ); + } +} + +/** + * Verify every configured source recorded output for the whole session. + * + * Two signals are needed because the workers fail in two different shapes. A + * worker removes its pid file on each clean exit, so a pid file that outlives + * its recorded process identity means the helper itself was killed. A tmux + * pane that exits instead closes its capture pipe as a clean EOF, so the + * helper shuts down normally and only the pane's own `#{pane_pipe}` state + * still distinguishes a dead source from a healthy one — which is why this + * runs before teardown detaches those pipes. + * + * Either way the evidence produced looks complete but is not, so each gap is + * written into canonical evidence, recorded on the state that teardown may + * have to persist, and returned for the caller to surface. + */ +export function recordCaptureHealthFailures( + state: EnvironmentState | null | undefined, + startTimeMs?: number, +): string[] { + // A launcher state records only the external command, which is expected to + // have exited; it owns no capture worker to verify. + if (!state || state.kind === 'launcher') { + return []; + } + const target = state; + const failures: string[] = []; + const reported = new Set(); + const recordGap = (sourceId: string, summary: string, evidenceText: string): void => { + reported.add(sourceId); + failures.push(summary); + const source = target.sources.find((candidate) => candidate.id === sourceId); + const now = Date.now(); + try { + appendEvidenceEvent(target.evidencePath, { + version: 1, + origin: 'environment', + group: source?.group || 'environment', + sourceId, + sourceTitle: source?.title || sourceId, + stream: source?.stream || 'stderr', + segment: 'live', + timestamp: new Date(now).toISOString(), + relativeTimeSec: + startTimeMs === undefined ? null : Math.max(0, (now - startTimeMs) / 1000), + text: evidenceText, + captureGap: true, + }); + } catch (error) { + failures.push( + `Could not record the capture gap for "${sourceId}" in ${ + target.evidencePath + }: ${error instanceof Error ? error.message : String(error)}`, + ); + } + }; + const incompleteLog = (sourceId: string): string => { + const source = target.sources.find((candidate) => candidate.id === sourceId); + return source ? ` — logs/${path.basename(source.logPath)} is incomplete` : ''; + }; + + const captures = target.kind === 'tmux' ? target.captures : target.processes; + for (const capture of captures) { + if (!capture.pidFile) continue; + if (processIdentityMatches(capture.process)) continue; + if (!fs.existsSync(capture.pidFile)) continue; + + recordGap( + capture.sourceId, + `Capture for "${capture.sourceId}" stopped before "proofshot stop"${incompleteLog( + capture.sourceId, + )}. Helper diagnostics: ${capture.pidFile}.stderr`, + '[capture stopped before the session ended; later output was not recorded]', + ); + } + + if (target.kind === 'tmux') { + for (const pane of findUnpipedPanes(target)) { + if (reported.has(pane.sourceId)) continue; + recordGap( + pane.sourceId, + `tmux pane ${pane.paneId} for "${pane.sourceId}" exited or lost its capture pipe before "proofshot stop"${incompleteLog( + pane.sourceId, + )}.`, + '[tmux pane stopped feeding ProofShot before the session ended; later output was not recorded]', + ); + } + } + + if (failures.length > 0) { + target.healthFailures = failures; + } + return failures; +} + +export async function stopOwnedEnvironment( + state: EnvironmentState | null | undefined, +): Promise { + if (!state) { + return; + } + switch (state.kind) { + case 'tmux': + await stopTmuxEnvironment(state); + return; + case 'launcher': + await terminateOwnedProcessTree(state.launcher.process, { graceMs: 1000 }); + if (ownedProcessTreeIsAlive(state.launcher.process)) { + throw new Error('External environment launcher did not stop.'); + } + return; + case 'processes': { + const errors: Error[] = []; + for (const capture of state.processes) { + try { + await terminateOwnedProcessTree(capture.process, { graceMs: 1000 }); + if (ownedProcessTreeIsAlive(capture.process)) { + throw new Error(`Environment process ${capture.sourceId} did not stop.`); + } + } catch (error) { + errors.push(error instanceof Error ? error : new Error(String(error))); + } + } + if (errors.length > 0) { + throw new AggregateError( + errors, + 'One or more environment processes did not stop.', + ); + } + return; + } + default: { + const exhaustiveState: never = state; + return exhaustiveState; + } + } +} + +async function startProcessEnvironment( + definitions: ProcessDefinition[], + logs: LogsConfig, + sessionDir: string, + startTimeMs: number, + onState: (state: EnvironmentState) => void, +): Promise { + const evidencePath = path.join(sessionDir, 'environment.ndjson'); + const logsDir = path.join(sessionDir, 'logs'); + const captureDir = path.join(sessionDir, '.capture'); + fs.mkdirSync(logsDir, { recursive: true }); + fs.mkdirSync(captureDir, { recursive: true, mode: 0o700 }); + fs.writeFileSync(evidencePath, '', { flag: 'a', mode: 0o600 }); + + const configuredSources = (logs.sources || []).filter( + (source): source is Extract => + source.kind === 'process', + ); + const sourceByProcessId = new Map(); + for (const source of configuredSources) { + if (sourceByProcessId.has(source.processId)) { + throw new Error( + `Multiple log sources reference process ${source.processId}; each process can be launched only once.`, + ); + } + sourceByProcessId.set(source.processId, source); + } + for (const source of configuredSources) { + if (!definitions.some((definition) => definition.id === source.processId)) { + throw new Error( + `Log source ${source.id} references unknown process ${source.processId}.`, + ); + } + } + const sources = definitions.map( + (definition) => + sourceByProcessId.get(definition.id) || { + id: definition.id, + title: definition.title, + group: definition.group, + kind: 'process' as const, + processId: definition.id, + }, + ); + validateUniqueIds(sources.map((source) => source.id)); + + let state: ProcessEnvironmentState = { + kind: 'processes', + evidencePath, + sources: [], + processes: [], + }; + onState(state); + try { + for (const sourceConfig of sources) { + const definition = definitions.find( + (candidate) => candidate.id === sourceConfig.processId, + ); + if (!definition) throw new Error(`Missing process ${sourceConfig.processId}.`); + const source: ResolvedLogSourceState = { + id: sourceConfig.id, + title: sourceConfig.title || definition.title || definition.id, + group: sourceConfig.group || definition.group || 'environment', + kind: 'process', + stream: 'stdout', + logPath: path.join(logsDir, `${sourceConfig.id}.log`), + }; + const process = await startProcessCapture( + definition, + source, + evidencePath, + captureDir, + startTimeMs, + logs.maxBytesPerSource || 5 * 1024 * 1024, + logs.stripAnsi !== false, + ); + state = { + ...state, + sources: [...state.sources, source], + processes: [...state.processes, process], + }; + onState(state); + } + return state; + } catch (error) { + await stopOwnedEnvironment(state).catch(() => {}); + throw error; + } +} + +async function attachFileSources( + state: EnvironmentState, + fileSources: Array>, + logs: LogsConfig, + sessionDir: string, + startTimeMs: number, + onState: (state: EnvironmentState) => void, +): Promise { + if (fileSources.length === 0) { + return state; + } + if (state.kind === 'launcher') { + throw new Error('Cannot attach file sources before the environment launcher exits.'); + } + const knownIds = new Set(state.sources.map((source) => source.id)); + const logsDir = path.join(sessionDir, 'logs'); + const captureDir = path.join(sessionDir, '.capture'); + fs.mkdirSync(captureDir, { recursive: true, mode: 0o700 }); + for (const fileSource of fileSources) { + if (knownIds.has(fileSource.id)) { + throw new Error(`Duplicate log source id: ${fileSource.id}`); + } + knownIds.add(fileSource.id); + const source: ResolvedLogSourceState = { + id: fileSource.id, + title: fileSource.title || path.basename(fileSource.path), + group: fileSource.group || 'environment', + kind: 'file', + stream: 'file', + logPath: path.join(logsDir, `${fileSource.id}.log`), + }; + const capture = await startFileCapture( + fileSource.path, + source, + state.evidencePath, + captureDir, + startTimeMs, + logs.maxBytesPerSource || 5 * 1024 * 1024, + logs.stripAnsi !== false, + ); + state = + state.kind === 'tmux' + ? { + ...state, + sources: [...state.sources, source], + captures: [...state.captures, capture], + } + : { + ...state, + sources: [...state.sources, source], + processes: [...state.processes, capture], + }; + onState(state); + } + return state; +} + +async function waitForReadiness(checks: ReadinessCheck[]): Promise { + for (const check of checks) { + const timeoutMs = check.timeoutMs || 30 * 1000; + const deadline = Date.now() + timeoutMs; + let lastError = 'not ready'; + while (Date.now() < deadline) { + try { + if (check.kind === 'http') { + const response = await fetch(check.url, { + signal: AbortSignal.timeout(Math.min(2000, timeoutMs)), + }); + if (response.ok) { + lastError = ''; + break; + } + lastError = `HTTP ${response.status}`; + } else { + await connectTcp(check.host || '127.0.0.1', check.port); + lastError = ''; + break; + } + } catch (error) { + lastError = error instanceof Error ? error.message : String(error); + } + await new Promise((resolve) => setTimeout(resolve, 100)); + } + if (lastError) { + const target = + check.kind === 'http' + ? check.url + : `${check.host || '127.0.0.1'}:${check.port}`; + throw new Error(`Environment readiness failed for ${target}: ${lastError}`); + } + } +} + +function connectTcp(host: string, port: number): Promise { + return new Promise((resolve, reject) => { + const socket = net.createConnection({ host, port }); + const timer = setTimeout(() => { + socket.destroy(); + reject(new Error('TCP readiness timed out')); + }, 2000); + socket.once('connect', () => { + clearTimeout(timer); + socket.end(); + resolve(); + }); + socket.once('error', (error) => { + clearTimeout(timer); + reject(error); + }); + }); +} + +function validateUniqueIds(ids: string[]): void { + const seen = new Set(); + for (const id of ids) { + if (!/^[A-Za-z0-9_-]+$/.test(id)) { + throw new Error(`Invalid log source id: ${id}`); + } + if (seen.has(id)) { + throw new Error(`Duplicate log source id: ${id}`); + } + seen.add(id); + } +} diff --git a/src/environment/tmux-cleanup.ts b/src/environment/tmux-cleanup.ts new file mode 100644 index 0000000..05e9c19 --- /dev/null +++ b/src/environment/tmux-cleanup.ts @@ -0,0 +1,140 @@ +import * as fs from 'fs'; +import { runCommand, tmuxExec } from './tmux-command.js'; +import { + assertSocketIdentity, + captureSocketIdentity, + tmuxHasSession, +} from './tmux-identity.js'; +import type { TmuxEnvironmentState } from './types.js'; +import { + captureProcessIdentity, + processIdentitiesMatch, + processIdentityMatches, + terminateOwnedProcess, + terminateOwnedProcessTree, +} from '../utils/process.js'; + +export async function stopTmuxEnvironment( + state: TmuxEnvironmentState, +): Promise { + const errors: Error[] = []; + let socketMatches = false; + let socketIdentityError: Error | null = null; + if (fs.existsSync(state.socket.path)) { + try { + assertSocketIdentity(state); + socketMatches = true; + } catch (error) { + socketIdentityError = toError(error); + } + } + + const currentServer = captureProcessIdentity(state.serverProcess.pid); + const serverIdentityReused = Boolean( + currentServer && !processIdentitiesMatch(currentServer, state.serverProcess), + ); + if (serverIdentityReused) { + errors.push(new Error('tmux server identity changed; refusing widened cleanup.')); + } + const serverMatches = processIdentityMatches(state.serverProcess); + if ( + socketIdentityError && + (serverMatches || + state.captures.some((capture) => processIdentityMatches(capture.process))) + ) { + errors.push(socketIdentityError); + } + + if (serverMatches && socketMatches) { + // Detaching first releases panes ProofShot does not own before any shutdown + // runs, and stops output reaching a helper that is about to be terminated. + for (const pane of state.panes.filter( + (candidate) => candidate.captureAttached, + )) { + try { + tmuxExec(state.socket.path, ['pipe-pane', '-t', pane.paneId]); + // A later step can still fail and leave this state to be retried, and + // by then an unpiped pane is this teardown's own work rather than a + // mid-session gap. + pane.captureAttached = false; + } catch { + // A launcher-provided shutdown may already have removed the pane. + } + } + try { + if (state.stopCommand) { + await runCommand(state.stopCommand, state.stopCwd || process.cwd()); + } else if (state.ownsSession && tmuxHasSession(state)) { + tmuxExec(state.socket.path, ['kill-session', '-t', state.sessionName]); + } + } catch (error) { + errors.push(toError(error)); + } + } + + for (const capture of state.captures) { + try { + await terminateOwnedProcess(capture.process, { graceMs: 500 }); + if (processIdentityMatches(capture.process)) { + throw new Error(`Log helper for ${capture.sourceId} did not stop.`); + } + } catch (error) { + errors.push(toError(error)); + } + } + + if (state.ownsServer && !serverIdentityReused) { + if (processIdentityMatches(state.serverProcess) && socketMatches) { + try { + tmuxExec(state.socket.path, ['kill-server']); + } catch { + // Exact process-session termination below is the verified fallback. + } + } + if (processIdentityMatches(state.serverProcess)) { + try { + await terminateOwnedProcessTree(state.serverProcess, { graceMs: 500 }); + } catch (error) { + errors.push(toError(error)); + } + } + if (processIdentityMatches(state.serverProcess)) { + errors.push(new Error('Owned tmux server did not stop.')); + } + } + + if ( + state.ownsServer && + socketMatches && + !processIdentityMatches(state.serverProcess) && + fs.existsSync(state.socket.path) + ) { + try { + const currentSocket = captureSocketIdentity(state.socket.path); + if ( + currentSocket.inode !== state.socket.inode || + currentSocket.uid !== state.socket.uid + ) { + throw new Error('tmux socket changed before final cleanup.'); + } + fs.unlinkSync(state.socket.path); + } catch (error) { + errors.push(toError(error)); + } + } + if ( + state.ownsSession && + processIdentityMatches(state.serverProcess) && + socketMatches && + tmuxHasSession(state) + ) { + errors.push(new Error(`Owned tmux session ${state.sessionName} did not stop.`)); + } + if (errors.length > 0) { + throw new AggregateError(errors, 'One or more tmux cleanup steps failed.'); + } +} + +function toError(error: unknown): Error { + return error instanceof Error ? error : new Error(String(error)); +} diff --git a/src/environment/tmux-command.ts b/src/environment/tmux-command.ts new file mode 100644 index 0000000..6f0fd90 --- /dev/null +++ b/src/environment/tmux-command.ts @@ -0,0 +1,97 @@ +import { execFileSync } from 'child_process'; +import { + captureProcessIdentity, + spawnShellCommand, + terminateOwnedProcessTree, +} from '../utils/process.js'; +import type { ProcessIdentity } from '../utils/process.js'; + +type CommandOutcome = + | { kind: 'exit'; code: number | null } + | { kind: 'timeout' }; + +export function assertTmuxAvailable(): void { + try { + execFileSync('tmux', ['-V'], { stdio: 'pipe' }); + } catch { + throw new Error('tmux is required for environment.kind "tmux".'); + } +} + +/** + * `maxBuffer` defaults to Node's 1 MB, which is smaller than the scrollback + * ProofShot budgets for pane history, so callers that read bulk output must + * raise it or the whole start fails with ENOBUFS. + */ +export function tmuxExec( + socketPath: string, + args: string[], + options: { maxBuffer?: number } = {}, +): string { + return execFileSync('tmux', ['-S', socketPath, ...args], { + encoding: 'utf-8', + stdio: ['ignore', 'pipe', 'pipe'], + ...(options.maxBuffer === undefined ? {} : { maxBuffer: options.maxBuffer }), + }).trimEnd(); +} + +export async function runCommand( + command: string, + cwd: string, + onStarted?: (identity: ProcessIdentity) => void, + timeoutMs = 30_000, +): Promise { + const child = spawnShellCommand(command, { + cwd, + detached: true, + stdio: ['ignore', 'pipe', 'pipe'], + }); + const identity = child.pid ? captureProcessIdentity(child.pid) : null; + if (!identity) { + // Without an identity there is nothing to record in `.session.json`, so the + // just-spawned pid is the only handle left for avoiding an orphan launcher. + try { + if (child.pid) { + process.kill(-child.pid, 'SIGKILL'); + } + } catch { + // The launcher may already have exited. + } + throw new Error('ProofShot could not capture the external launcher identity.'); + } + try { + onStarted?.(identity); + } catch (error) { + await terminateOwnedProcessTree(identity); + throw error; + } + + let stdout = ''; + let stderr = ''; + child.stdout?.on('data', (chunk) => { + stdout += chunk.toString(); + }); + child.stderr?.on('data', (chunk) => { + stderr += chunk.toString(); + }); + const outcome = await new Promise((resolve, reject) => { + const timer = setTimeout(() => resolve({ kind: 'timeout' }), timeoutMs); + child.once('error', reject); + child.once('close', (code) => { + clearTimeout(timer); + resolve({ kind: 'exit', code }); + }); + }); + if (outcome.kind === 'timeout') { + await terminateOwnedProcessTree(identity); + throw new Error(`External environment command timed out after ${timeoutMs}ms.`); + } + const exitCode = outcome.code; + if (exitCode !== 0) { + await terminateOwnedProcessTree(identity); + throw new Error( + `External environment command failed with code ${String(exitCode)}: ${stderr.trim()}`, + ); + } + return stdout.trim(); +} diff --git a/src/environment/tmux-identity.ts b/src/environment/tmux-identity.ts new file mode 100644 index 0000000..07f2ead --- /dev/null +++ b/src/environment/tmux-identity.ts @@ -0,0 +1,41 @@ +import * as fs from 'fs'; +import { tmuxExec } from './tmux-command.js'; +import type { SocketIdentity, TmuxEnvironmentState } from './types.js'; +import { processIdentityMatches } from '../utils/process.js'; + +export function captureSocketIdentity(socketPath: string): SocketIdentity { + const stat = fs.lstatSync(socketPath); + if (!stat.isSocket() || stat.isSymbolicLink()) { + throw new Error(`tmux socket is not an owned Unix socket: ${socketPath}`); + } + const uid = process.getuid?.(); + if (uid !== undefined && stat.uid !== uid) { + throw new Error(`tmux socket is owned by uid ${stat.uid}, expected ${uid}.`); + } + return { path: socketPath, inode: stat.ino, uid: stat.uid }; +} + +export function assertSocketIdentity(state: TmuxEnvironmentState): void { + if (!fs.existsSync(state.socket.path)) { + if (!processIdentityMatches(state.serverProcess)) { + return; + } + throw new Error('Owned tmux socket disappeared while its server is still alive.'); + } + const current = captureSocketIdentity(state.socket.path); + if (current.inode !== state.socket.inode || current.uid !== state.socket.uid) { + throw new Error('tmux socket identity changed; refusing widened cleanup.'); + } +} + +export function tmuxHasSession(state: TmuxEnvironmentState): boolean { + if (!processIdentityMatches(state.serverProcess)) { + return false; + } + try { + tmuxExec(state.socket.path, ['has-session', '-t', state.sessionName]); + return true; + } catch { + return false; + } +} diff --git a/src/environment/tmux-launch.ts b/src/environment/tmux-launch.ts new file mode 100644 index 0000000..4bba877 --- /dev/null +++ b/src/environment/tmux-launch.ts @@ -0,0 +1,426 @@ +import * as fs from 'fs'; +import * as path from 'path'; +import { execFileSync } from 'child_process'; +import { runCommand, tmuxExec } from './tmux-command.js'; +import { captureSocketIdentity } from './tmux-identity.js'; +import type { + ExternalTmuxConnection, + TmuxEnvironmentConfig, + TmuxEnvironmentState, + TmuxPaneDefinition, +} from './types.js'; +import { captureProcessIdentity } from '../utils/process.js'; +import type { ProcessIdentity } from '../utils/process.js'; + +export type PaneMapping = { + key: string; + paneId: string; + title?: string; + group?: string; +}; + +export type TmuxConnection = { + socketPath: string; + sessionName: string; + paneMappings: PaneMapping[]; + ownsServer: boolean; + ownsSession: boolean; +}; + +type PaneOutput = { + paneId: string; + paneIndex: number; + panePid: number; +}; + +export function startOwnedTmux( + config: TmuxEnvironmentConfig, + proofShotSessionName: string, + onStarted: (connection: TmuxConnection) => void, +): TmuxConnection { + if (config.launch.kind !== 'panes' || config.launch.panes.length === 0) { + throw new Error('tmux pane launch requires at least one pane.'); + } + const paneIds = new Set(); + for (const pane of config.launch.panes) { + validateId(pane.id); + if (paneIds.has(pane.id)) { + throw new Error(`Duplicate tmux pane id: ${pane.id}`); + } + paneIds.add(pane.id); + buildPaneCommand(pane); + } + + const socketDir = createOwnedSocketDir(); + const socketPath = path.join(socketDir, `${proofShotSessionName}.sock`); + if (fs.existsSync(socketPath)) { + throw new Error(`Refusing to reuse an existing tmux socket: ${socketPath}`); + } + + const sessionName = config.launch.sessionName || proofShotSessionName; + const [firstPane, ...remainingPanes] = config.launch.panes; + const first = parsePaneOutput( + tmuxExec(socketPath, [ + 'new-session', + '-d', + '-P', + '-F', + '#{pane_id}\t#{pane_index}\t#{pane_pid}', + '-s', + sessionName, + '-n', + 'environment', + '-c', + firstPane.cwd || config.cwd || process.cwd(), + buildPaneCommand(firstPane), + ]), + ); + const mappings: PaneMapping[] = [ + { + key: firstPane.id, + paneId: first.paneId, + title: firstPane.title, + group: firstPane.group, + }, + ]; + onStarted({ + socketPath, + sessionName, + paneMappings: [...mappings], + ownsServer: true, + ownsSession: true, + }); + configurePane(socketPath, first.paneId, firstPane.id, firstPane.title); + + for (const pane of remainingPanes) { + const created = parsePaneOutput( + tmuxExec(socketPath, [ + 'split-window', + '-d', + '-P', + '-F', + '#{pane_id}\t#{pane_index}\t#{pane_pid}', + '-t', + `${sessionName}:environment`, + '-c', + pane.cwd || config.cwd || process.cwd(), + buildPaneCommand(pane), + ]), + ); + configurePane(socketPath, created.paneId, pane.id, pane.title); + mappings.push({ + key: pane.id, + paneId: created.paneId, + title: pane.title, + group: pane.group, + }); + } + tmuxExec(socketPath, ['select-layout', '-t', `${sessionName}:environment`, 'tiled']); + return { + socketPath, + sessionName, + paneMappings: mappings, + ownsServer: true, + ownsSession: true, + }; +} + +export async function startExternalTmux( + config: TmuxEnvironmentConfig, + onLauncherStarted: (identity: ProcessIdentity) => void, +): Promise { + if (config.launch.kind !== 'external-command' || !config.connection) { + throw new Error('External tmux launch requires a connection contract.'); + } + const hintedSocket = config.connection.socket; + // An undisclosed socket cannot be snapshotted, so it is treated as pre-existing. + const socketExistedBefore = hintedSocket ? fs.existsSync(hintedSocket) : true; + const attachOnly = config.connection.ownership === 'attach'; + if (!attachOnly && socketExistedBefore && !config.launch.stopCommand) { + throw new Error( + 'External tmux launch against an existing or undisclosed socket requires stopCommand.', + ); + } + + const output = await runCommand( + config.launch.command, + config.cwd || process.cwd(), + onLauncherStarted, + config.launch.timeoutMs, + ); + const parsed = + config.connection.format === 'json' + ? parseJsonConnection(output) + : parseAttachCommand(output, config.cwd || process.cwd()); + // `connection.ownership: "attach"` always wins: the tmux server, session and + // panes belong to the user even when this start's launcher happened to create + // them, so ProofShot never records an ownership claim over them. + const ownsCreatedSocket = + !attachOnly && + hintedSocket !== undefined && + path.resolve(hintedSocket) === path.resolve(parsed.socketPath) && + !socketExistedBefore; + return { + ...parsed, + ownsServer: ownsCreatedSocket, + ownsSession: ownsCreatedSocket, + }; +} + +export function createTmuxState( + config: TmuxEnvironmentConfig, + connection: TmuxConnection, + evidencePath: string, +): TmuxEnvironmentState { + const serverPid = Number( + tmuxExec(connection.socketPath, ['display-message', '-p', '#{pid}']), + ); + const serverProcess = captureProcessIdentity(serverPid); + if (!serverProcess) { + throw new Error('ProofShot could not capture the exact tmux server identity.'); + } + return { + kind: 'tmux', + evidencePath, + sources: [], + socket: captureSocketIdentity(connection.socketPath), + serverProcess, + sessionName: connection.sessionName, + ownsServer: connection.ownsServer, + ownsSession: connection.ownsSession, + panes: [], + captures: [], + stopCommand: + config.launch.kind === 'external-command' && + config.connection?.ownership !== 'attach' + ? config.launch.stopCommand + : undefined, + stopCwd: config.cwd, + }; +} + +/** + * Create the control-socket directory under a world-writable `/tmp`. + * + * `mkdir -p` follows a pre-existing symlink and never re-checks the ownership or + * permissions of components it did not create, which would let another local + * user host — and later substitute — the tmux socket. tmux skips its own socket + * directory check for `-S`, so every component is verified here instead. + */ +function createOwnedSocketDir(): string { + const uid = process.getuid?.() ?? process.pid; + const root = path.join('/tmp', `proofshot-${uid}`); + const socketDir = path.join(root, 'tmux'); + createVerifiedDirectory(root, uid); + createVerifiedDirectory(socketDir, uid); + return socketDir; +} + +function createVerifiedDirectory(directory: string, uid: number): void { + try { + fs.mkdirSync(directory, { mode: 0o700 }); + } catch (error) { + if ((error as NodeJS.ErrnoException).code !== 'EEXIST') { + throw error; + } + } + const stats = fs.lstatSync(directory); + if (!stats.isDirectory()) { + throw new Error( + `Refusing to use tmux socket directory ${directory}: not a directory.`, + ); + } + if (process.getuid && stats.uid !== uid) { + throw new Error( + `Refusing to use tmux socket directory ${directory}: owned by uid ${stats.uid}.`, + ); + } + if ((stats.mode & 0o077) !== 0) { + throw new Error( + `Refusing to use tmux socket directory ${directory}: permissions must be 0700.`, + ); + } +} + +function configurePane( + socketPath: string, + paneId: string, + sourceId: string, + title?: string, +): void { + validateId(sourceId); + tmuxExec(socketPath, [ + 'set-option', + '-p', + '-t', + paneId, + '@proofshot-source', + sourceId, + ]); + if (title) { + tmuxExec(socketPath, ['select-pane', '-t', paneId, '-T', title]); + } +} + +function buildPaneCommand(pane: TmuxPaneDefinition): string { + const assignments = Object.entries(pane.env || {}).map(([key, value]) => { + if (!/^[A-Za-z_][A-Za-z0-9_]*$/.test(key)) { + throw new Error(`Invalid environment variable name: ${key}`); + } + return `${key}=${shellQuote(value)}`; + }); + return assignments.length > 0 + ? `env ${assignments.join(' ')} ${pane.command}` + : pane.command; +} + +function parsePaneOutput(output: string): PaneOutput { + const [paneId, paneIndex, panePid] = output.split('\t'); + if ( + !paneId || + !Number.isInteger(Number(paneIndex)) || + !Number.isInteger(Number(panePid)) + ) { + throw new Error(`Unexpected tmux pane output: ${output}`); + } + return { + paneId, + paneIndex: Number(paneIndex), + panePid: Number(panePid), + }; +} + +function parseJsonConnection(output: string): TmuxConnection { + const parsed = JSON.parse(output) as Partial; + if ( + !parsed.tmux || + !path.isAbsolute(parsed.tmux.socket) || + typeof parsed.tmux.session !== 'string' || + parsed.tmux.session.length === 0 || + (parsed.tmux.panes !== undefined && !Array.isArray(parsed.tmux.panes)) + ) { + throw new Error('External launcher returned invalid tmux JSON.'); + } + + const paneMappings: PaneMapping[] = []; + const keys = new Set(); + const paneIds = new Set(); + for (const [index, pane] of (parsed.tmux.panes || []).entries()) { + if ( + typeof pane !== 'object' || + pane === null || + typeof pane.key !== 'string' || + !/^[A-Za-z0-9_-]+$/.test(pane.key) || + typeof pane.paneId !== 'string' || + !/^%\d+$/.test(pane.paneId) || + (pane.title !== undefined && typeof pane.title !== 'string') || + (pane.group !== undefined && typeof pane.group !== 'string') + ) { + throw new Error(`External launcher returned invalid pane mapping at index ${index}.`); + } + if (keys.has(pane.key) || paneIds.has(pane.paneId)) { + throw new Error('External launcher returned duplicate pane mappings.'); + } + keys.add(pane.key); + paneIds.add(pane.paneId); + paneMappings.push(pane); + } + return { + socketPath: parsed.tmux.socket, + sessionName: parsed.tmux.session, + paneMappings, + ownsServer: false, + ownsSession: false, + }; +} + +function parseAttachCommand(output: string, cwd: string): TmuxConnection { + const tokens = tokenizeShellCommand(output); + const tmuxIndex = tokens.findIndex((token) => path.basename(token) === 'tmux'); + const attachIndex = tokens.findIndex( + (token, index) => + index > tmuxIndex && (token === 'attach' || token === 'attach-session'), + ); + const targetIndex = tokens.indexOf('-t', attachIndex + 1); + const socketIndex = tokens.indexOf('-S', tmuxIndex + 1); + const labelIndex = tokens.indexOf('-L', tmuxIndex + 1); + if ( + tmuxIndex < 0 || + attachIndex < 0 || + targetIndex < 0 || + !tokens[targetIndex + 1] || + (socketIndex < 0 && labelIndex < 0) + ) { + throw new Error('External launcher did not emit a supported tmux attach command.'); + } + + const flag = socketIndex >= 0 ? '-S' : '-L'; + const valueIndex = socketIndex >= 0 ? socketIndex + 1 : labelIndex + 1; + const value = tokens[valueIndex]; + const sessionName = tokens[targetIndex + 1]; + if (!value) { + throw new Error('External launcher emitted a tmux socket flag without a value.'); + } + const socketPath = + flag === '-S' + ? path.resolve(cwd, value) + : execFileSync( + 'tmux', + ['-L', value, 'display-message', '-p', '#{socket_path}'], + { encoding: 'utf-8' }, + ).trim(); + return { + socketPath, + sessionName, + paneMappings: [], + ownsServer: false, + ownsSession: false, + }; +} + +function tokenizeShellCommand(command: string): string[] { + const tokens: string[] = []; + let current = ''; + let quote: "'" | '"' | null = null; + let escaping = false; + for (const character of command.trim()) { + if (escaping) { + current += character; + escaping = false; + } else if (character === '\\' && quote !== "'") { + escaping = true; + } else if (quote) { + if (character === quote) { + quote = null; + } else { + current += character; + } + } else if (character === "'" || character === '"') { + quote = character; + } else if (/\s/.test(character)) { + if (current) { + tokens.push(current); + current = ''; + } + } else { + current += character; + } + } + if (escaping || quote) { + throw new Error('External launcher emitted an unterminated tmux attach command.'); + } + if (current) { + tokens.push(current); + } + return tokens; +} + +function validateId(id: string): void { + if (!/^[A-Za-z0-9_-]+$/.test(id)) { + throw new Error(`Invalid log source id: ${id}`); + } +} + +function shellQuote(value: string): string { + return `'${value.replace(/'/g, `'\\''`)}'`; +} diff --git a/src/environment/tmux-panes.ts b/src/environment/tmux-panes.ts new file mode 100644 index 0000000..fdf5ce4 --- /dev/null +++ b/src/environment/tmux-panes.ts @@ -0,0 +1,183 @@ +import * as path from 'path'; +import { tmuxExec } from './tmux-command.js'; +import type { PaneMapping, TmuxConnection } from './tmux-launch.js'; +import type { + LogSourceConfig, + LogsConfig, + ResolvedLogSourceState, + TmuxEnvironmentConfig, + TmuxPaneState, +} from './types.js'; + +export type ResolvedTmuxPane = { + pane: TmuxPaneState; + source: ResolvedLogSourceState; +}; + +type TmuxSource = { + config: Extract; + mapping?: PaneMapping; +}; + +export function resolveTmuxPanes( + config: TmuxEnvironmentConfig, + logs: LogsConfig, + connection: TmuxConnection, + logsDir: string, +): ResolvedTmuxPane[] { + const panes = resolveTmuxSources(config, logs, connection).map( + ({ config: sourceConfig, mapping }) => + resolvePane( + connection.socketPath, + connection.sessionName, + sourceConfig, + mapping, + logsDir, + ), + ); + const resolvedPaneIds = new Set(); + for (const { pane } of panes) { + if (resolvedPaneIds.has(pane.paneId)) { + throw new Error(`Multiple log sources resolved to tmux pane ${pane.paneId}.`); + } + resolvedPaneIds.add(pane.paneId); + } + disambiguateTitles(panes); + return panes; +} + +function resolveTmuxSources( + config: TmuxEnvironmentConfig, + logs: LogsConfig, + connection: TmuxConnection, +): TmuxSource[] { + const configured = (logs.sources || []).filter( + (source): source is Extract => + source.kind === 'tmux-pane', + ); + if (configured.length > 0) { + return configured.map((source) => { + const connectionKey = + 'connectionKey' in source.match + ? source.match.connectionKey + : undefined; + return { + config: source, + mapping: connectionKey + ? connection.paneMappings.find( + (mapping) => mapping.key === connectionKey, + ) + : undefined, + }; + }); + } + if (config.launch.kind !== 'panes') { + return []; + } + return connection.paneMappings.map((mapping) => ({ + config: { + id: mapping.key, + title: mapping.title, + group: mapping.group, + kind: 'tmux-pane', + match: { connectionKey: mapping.key }, + }, + mapping, + })); +} + +function resolvePane( + socketPath: string, + sessionName: string, + sourceConfig: Extract, + mapping: PaneMapping | undefined, + logsDir: string, +): ResolvedTmuxPane { + let target: string; + if ('connectionKey' in sourceConfig.match) { + if (!mapping) { + throw new Error( + `No tmux pane mapping matched connection key "${sourceConfig.match.connectionKey}".`, + ); + } + target = mapping.paneId; + } else if ('tag' in sourceConfig.match) { + const tag = sourceConfig.match.tag; + const matches = tmuxExec(socketPath, [ + 'list-panes', + '-s', + '-t', + sessionName, + '-F', + '#{pane_id}\t#{@proofshot-source}', + ]) + .split('\n') + .filter((line) => line.split('\t')[1] === tag); + if (matches.length !== 1) { + throw new Error( + `Expected one tmux pane tagged "${tag}", found ${matches.length}.`, + ); + } + target = matches[0].split('\t')[0]; + } else { + target = sourceConfig.match.target; + } + + const fields = tmuxExec(socketPath, [ + 'display-message', + '-p', + '-t', + target, + '#{pane_id}\t#{pane_index}\t#{pane_pid}\t#{pane_title}\t#{session_name}\t#{session_name}:#{window_name}.#{pane_index}', + ]).split('\t'); + if (fields.length !== 6) { + throw new Error(`Could not resolve tmux pane metadata for ${target}.`); + } + if (fields[4] !== sessionName) { + throw new Error( + `tmux pane ${fields[0]} belongs to session "${fields[4]}", expected "${sessionName}".`, + ); + } + + const paneIndex = Number(fields[1]); + const tmuxTitle = fields[3].trim(); + const title = + mapping?.title || + (tmuxTitle.length > 0 ? tmuxTitle : `Pane ${paneIndex}`); + const group = sourceConfig.group || mapping?.group || 'environment'; + const source: ResolvedLogSourceState = { + id: sourceConfig.id, + title, + group, + kind: 'tmux-pane', + stream: 'pty', + logPath: path.join(logsDir, `${sourceConfig.id}.log`), + }; + return { + source, + pane: { + paneId: fields[0], + paneIndex, + panePid: Number(fields[2]), + sourceId: source.id, + title, + group, + target: fields[5], + captureAttached: false, + }, + }; +} + +function disambiguateTitles(panes: ResolvedTmuxPane[]): void { + const counts = new Map(); + for (const pane of panes) { + counts.set(pane.source.title, (counts.get(pane.source.title) || 0) + 1); + } + for (const pane of panes) { + if ((counts.get(pane.source.title) || 0) > 1) { + const title = `${pane.source.title} (Pane ${pane.pane.paneIndex})`; + pane.source.title = title; + pane.pane.title = title; + } + } +} diff --git a/src/environment/tmux.ts b/src/environment/tmux.ts new file mode 100644 index 0000000..c324dc4 --- /dev/null +++ b/src/environment/tmux.ts @@ -0,0 +1,326 @@ +import * as fs from 'fs'; +import * as path from 'path'; +import { + appendHistory, + buildTmuxPipeCommand, + waitForCaptureProcess, + type WorkerConfig, +} from './workers.js'; +import { appendEvidenceEvent } from './evidence.js'; +import { stopTmuxEnvironment } from './tmux-cleanup.js'; +import { assertTmuxAvailable, runCommand, tmuxExec } from './tmux-command.js'; +import { assertSocketIdentity } from './tmux-identity.js'; +import { + createTmuxState, + startExternalTmux, + startOwnedTmux, +} from './tmux-launch.js'; +import type { TmuxConnection } from './tmux-launch.js'; +import { resolveTmuxPanes } from './tmux-panes.js'; +import type { + EnvironmentState, + LauncherEnvironmentState, + LogsConfig, + TmuxEnvironmentConfig, + TmuxEnvironmentState, + TmuxPaneState, +} from './types.js'; +import { + processIdentityMatches, + terminateOwnedProcessTree, +} from '../utils/process.js'; + +export { stopTmuxEnvironment }; + +const FALLBACK_HISTORY_LINES = 5000; + +/** + * Report the panes that stopped feeding ProofShot before teardown began. + * + * A pane whose command exits is destroyed by tmux, which closes the capture + * pipe as a clean EOF, so the helper shuts down exactly as it would at stop + * time and leaves no pid file behind. `#{pane_pipe}` is the only signal that + * separates the two, and it is only trustworthy before teardown detaches the + * pipes. When the server or socket identity no longer matches, every attached + * pane is treated as a gap, since nothing can be verified against a server + * ProofShot no longer recognizes. + */ +export function findUnpipedPanes(state: TmuxEnvironmentState): TmuxPaneState[] { + const attached = state.panes.filter((pane) => pane.captureAttached); + if (attached.length === 0) { + return []; + } + if (!processIdentityMatches(state.serverProcess)) { + return attached; + } + try { + assertSocketIdentity(state); + } catch { + return attached; + } + return attached.filter((pane) => { + try { + return ( + tmuxExec(state.socket.path, [ + 'display-message', + '-p', + '-t', + pane.paneId, + '#{pane_pipe}', + ]) !== '1' + ); + } catch { + return true; + } + }); +} + +/** + * Read a pane's retained scrollback. + * + * The capture crosses a pipe as a single buffer, and an attach-only pane can + * hold far more scrollback than the evidence budget once `history-limit` is + * raised. `appendHistory` keeps only the newest `historyBudget` bytes, so the + * buffer is sized from that budget and a pane that still overflows it falls + * back to a bounded line window — deep scrollback costs history, not the whole + * `start`. Returns null when no scrollback could be read at all. + */ +function capturePaneHistory( + socketPath: string, + paneId: string, + historyBudget: number, +): string | null { + const maxBuffer = Math.max(historyBudget * 4, 16 * 1024 * 1024); + const requests = [ + ['capture-pane', '-p', '-S', '-', '-t', paneId], + ['capture-pane', '-p', '-S', `-${FALLBACK_HISTORY_LINES}`, '-t', paneId], + ]; + for (const request of requests) { + try { + return tmuxExec(socketPath, request, { maxBuffer }); + } catch { + // Retry with a bounded window before giving up on the pane's history. + } + } + return null; +} + +/** + * Release a tmux server ProofShot took ownership of before it could record an + * immutable identity. Without a captured identity the recorded-state teardown + * path cannot run, so the launcher's own stop command and the just-created + * socket are the only ownership-safe handles left. + */ +async function releaseUnverifiedTmux( + config: TmuxEnvironmentConfig, + connection: TmuxConnection | null, +): Promise { + const stopCommand = + config.launch.kind === 'external-command' && + config.connection?.ownership !== 'attach' + ? config.launch.stopCommand + : undefined; + if (stopCommand) { + await runCommand(stopCommand, config.cwd || process.cwd()).catch(() => {}); + } + if (!connection || (!connection.ownsServer && !connection.ownsSession)) { + return; + } + try { + tmuxExec( + connection.socketPath, + connection.ownsServer + ? ['kill-server'] + : ['kill-session', '-t', connection.sessionName], + ); + } catch { + // The launcher's stop command may already have removed the server. + } +} + +export async function startTmuxEnvironment( + config: TmuxEnvironmentConfig, + logs: LogsConfig, + sessionDir: string, + proofShotSessionName: string, + startTimeMs: number, + onState: (state: EnvironmentState) => void, +): Promise { + assertTmuxAvailable(); + const evidencePath = path.join(sessionDir, 'environment.ndjson'); + const logsDir = path.join(sessionDir, 'logs'); + const captureDir = path.join(sessionDir, '.capture'); + fs.mkdirSync(logsDir, { recursive: true }); + fs.mkdirSync(captureDir, { recursive: true, mode: 0o700 }); + fs.writeFileSync(evidencePath, '', { flag: 'a', mode: 0o600 }); + + let state: TmuxEnvironmentState | null = null; + let pendingLauncher: LauncherEnvironmentState | null = null; + let unverifiedConnection: TmuxConnection | null = null; + let connection: TmuxConnection; + try { + connection = + config.launch.kind === 'panes' + ? startOwnedTmux(config, proofShotSessionName, (startedConnection) => { + unverifiedConnection = startedConnection; + const startedState = createTmuxState( + config, + startedConnection, + evidencePath, + ); + state = startedState; + onState(startedState); + }) + : await startExternalTmux(config, (launcher) => { + pendingLauncher = { + kind: 'launcher', + evidencePath, + sources: [], + launcher: { + sourceId: 'external-launcher', + process: launcher, + pidFile: '', + }, + }; + onState(pendingLauncher); + }); + if (!state) { + unverifiedConnection = connection; + const connectedState = createTmuxState( + config, + connection, + evidencePath, + ); + state = connectedState; + onState(connectedState); + } + } catch (error) { + if (state) { + await stopTmuxEnvironment(state).catch(() => {}); + } else { + // Assigned from the launcher callback, which control-flow analysis cannot + // see, so the declared type has to be restored before the null check. + const launcherState = pendingLauncher as LauncherEnvironmentState | null; + if (launcherState) { + await terminateOwnedProcessTree(launcherState.launcher.process).catch(() => {}); + } + await releaseUnverifiedTmux(config, unverifiedConnection); + } + throw error; + } + + try { + if (!state) { + throw new Error('tmux environment ownership state was not initialized.'); + } + let activeState: TmuxEnvironmentState = state; + const panes = resolveTmuxPanes(config, logs, connection, logsDir); + activeState = { + ...activeState, + panes: panes.map(({ pane }) => pane), + sources: panes.map(({ source }) => source), + }; + state = activeState; + onState(activeState); + + for (const pane of panes) { + const pipeStatus = tmuxExec(connection.socketPath, [ + 'display-message', + '-p', + '-t', + pane.pane.paneId, + '#{pane_pipe}', + ]); + if (pipeStatus === '1') { + throw new Error( + `tmux pane ${pane.pane.paneId} already has a pipe-pane consumer.`, + ); + } + + const pidFile = path.join(captureDir, `${pane.source.id}.pid`); + const sourceBudget = logs.maxBytesPerSource || 5 * 1024 * 1024; + const historyBudget = Math.max(1, Math.floor(sourceBudget / 2)); + const workerConfig: WorkerConfig = { + evidencePath, + logPath: pane.source.logPath, + pidFile, + startTimeMs, + maxBytes: Math.max(1, sourceBudget - historyBudget), + stripAnsi: logs.stripAnsi !== false, + source: pane.source, + }; + tmuxExec(connection.socketPath, [ + 'pipe-pane', + '-t', + pane.pane.paneId, + buildTmuxPipeCommand(workerConfig), + ]); + pane.pane.captureAttached = true; + activeState = { + ...activeState, + panes: activeState.panes.map((ownedPane) => + ownedPane.paneId === pane.pane.paneId + ? { ...ownedPane, captureAttached: true } + : ownedPane, + ), + }; + state = activeState; + onState(activeState); + + const history = capturePaneHistory( + connection.socketPath, + pane.pane.paneId, + historyBudget, + ); + if (history === null) { + appendEvidenceEvent(evidencePath, { + version: 1, + origin: 'environment', + group: pane.source.group, + sourceId: pane.source.id, + sourceTitle: pane.source.title, + stream: 'pty', + segment: 'history', + timestamp: null, + relativeTimeSec: null, + text: '[tmux scrollback could not be read; history is missing]', + captureGap: true, + }); + } else { + appendHistory( + history, + pane.source, + evidencePath, + historyBudget, + logs.stripAnsi !== false, + 'pty', + ); + } + appendEvidenceEvent(evidencePath, { + version: 1, + origin: 'environment', + group: pane.source.group, + sourceId: pane.source.id, + sourceTitle: pane.source.title, + stream: 'pty', + segment: 'history', + timestamp: null, + relativeTimeSec: null, + text: '[tmux history/live capture boundary]', + }); + const capture = await waitForCaptureProcess(pane.source.id, pidFile); + activeState = { + ...activeState, + captures: [...activeState.captures, capture], + }; + state = activeState; + onState(activeState); + } + return activeState; + } catch (error) { + if (state) { + await stopTmuxEnvironment(state).catch(() => {}); + } + throw error; + } +} diff --git a/src/environment/types.ts b/src/environment/types.ts new file mode 100644 index 0000000..efe13d1 --- /dev/null +++ b/src/environment/types.ts @@ -0,0 +1,190 @@ +import type { ProcessIdentity } from '../utils/process.js'; + +export type EnvironmentGroup = 'frontend' | 'backend' | string; + +export type ReadinessCheck = + | { kind: 'http'; url: string; timeoutMs?: number } + | { kind: 'tcp'; host?: string; port: number; timeoutMs?: number }; + +export type TmuxPaneDefinition = { + id: string; + title?: string; + group?: EnvironmentGroup; + cwd?: string; + command: string; + env?: Record; +}; + +export type TmuxLaunchConfig = + | { kind: 'panes'; panes: TmuxPaneDefinition[]; sessionName?: string } + | { + kind: 'external-command'; + command: string; + stopCommand?: string; + timeoutMs?: number; + }; + +export type TmuxConnectionConfig = { + source?: 'stdout'; + format: 'json' | 'tmux-attach-command'; + socket?: string; + ownership?: 'attach' | 'create'; +}; + +export type TmuxEnvironmentConfig = { + kind: 'tmux'; + launch: TmuxLaunchConfig; + cwd?: string; + connection?: TmuxConnectionConfig; + readiness?: ReadinessCheck[]; +}; + +export type ProcessDefinition = { + id: string; + title?: string; + group?: EnvironmentGroup; + cwd?: string; + command: string; + env?: Record; +}; + +export type ProcessesEnvironmentConfig = { + kind: 'processes'; + commands: ProcessDefinition[]; + readiness?: ReadinessCheck[]; +}; + +export type EnvironmentConfig = TmuxEnvironmentConfig | ProcessesEnvironmentConfig; + +export type TmuxPaneMatch = + | { connectionKey: string } + | { tag: string } + | { target: string }; + +export type LogSourceConfig = + | { + id: string; + title?: string; + group?: EnvironmentGroup; + kind: 'tmux-pane'; + match: TmuxPaneMatch; + } + | { + id: string; + title?: string; + group?: EnvironmentGroup; + kind: 'process'; + processId: string; + } + | { + id: string; + title?: string; + group?: EnvironmentGroup; + kind: 'file'; + path: string; + }; + +export type LogsConfig = { + stripAnsi?: boolean; + maxBytesPerSource?: number; + sources?: LogSourceConfig[]; +}; + +export type SocketIdentity = { + path: string; + inode: number; + uid: number; +}; + +export type CaptureProcessState = { + sourceId: string; + process: ProcessIdentity; + pidFile: string; +}; + +export type TmuxPaneState = { + paneId: string; + paneIndex: number; + panePid: number; + sourceId: string; + title: string; + group: EnvironmentGroup; + target: string; + captureAttached: boolean; +}; + +export type TmuxEnvironmentState = { + kind: 'tmux'; + evidencePath: string; + sources: ResolvedLogSourceState[]; + socket: SocketIdentity; + serverProcess: ProcessIdentity; + sessionName: string; + ownsServer: boolean; + ownsSession: boolean; + panes: TmuxPaneState[]; + captures: CaptureProcessState[]; + healthFailures?: string[]; + stopCommand?: string; + stopCwd?: string; +}; + +export type ProcessEnvironmentState = { + kind: 'processes'; + evidencePath: string; + sources: ResolvedLogSourceState[]; + processes: CaptureProcessState[]; + healthFailures?: string[]; +}; + +export type LauncherEnvironmentState = { + kind: 'launcher'; + evidencePath: string; + sources: []; + launcher: CaptureProcessState; +}; + +export type EnvironmentState = + | TmuxEnvironmentState + | ProcessEnvironmentState + | LauncherEnvironmentState; + +export type ResolvedLogSourceState = { + id: string; + title: string; + group: EnvironmentGroup; + kind: LogSourceConfig['kind']; + stream: EvidenceEvent['stream']; + logPath: string; +}; + +export type EvidenceEvent = { + version: 1; + origin: 'environment' | 'browser'; + group: string; + sourceId: string; + sourceTitle: string; + navigationId?: string; + pageUrl?: string; + stream: 'pty' | 'stdout' | 'stderr' | 'file' | 'console'; + segment: 'history' | 'live'; + timestamp: string | null; + relativeTimeSec: number | null; + text: string; + presentationHidden?: boolean; + truncated?: boolean; + captureGap?: boolean; +}; + +export type ExternalTmuxConnection = { + tmux: { + socket: string; + session: string; + panes?: Array<{ + key: string; + paneId: string; + title?: string; + group?: EnvironmentGroup; + }>; + }; +}; diff --git a/src/environment/workers.ts b/src/environment/workers.ts new file mode 100644 index 0000000..5a5c654 --- /dev/null +++ b/src/environment/workers.ts @@ -0,0 +1,444 @@ +import * as fs from 'fs'; +import * as path from 'path'; +import { spawn } from 'child_process'; +import { normalizeLogText } from './evidence.js'; +import type { + CaptureProcessState, + EvidenceEvent, + ProcessDefinition, + ResolvedLogSourceState, +} from './types.js'; +import { + captureProcessIdentity, + getShellExecutable, + type ProcessIdentity, +} from '../utils/process.js'; + +export type WorkerConfig = { + evidencePath: string; + logPath: string; + pidFile?: string; + startTimeMs: number; + maxBytes: number; + stripAnsi: boolean; + source: ResolvedLogSourceState; + command?: string; + cwd?: string; + env?: Record; + shellPath?: string; + offset?: number; + fileDevice?: number; + fileInode?: number; +}; + +const COMMON_WORKER_SOURCE = String.raw` +const fs = require('fs'); +const config = JSON.parse(Buffer.from(process.argv[1], 'base64').toString('utf8')); +const ansiPattern = /[\u001B\u009B][[\]()#;?]*(?:(?:(?:[a-zA-Z\d]*(?:;[-a-zA-Z\d/#&.:=?%@~_]+)*)?\u0007)|(?:(?:\d{1,4}(?:[;:]\d{0,4})*)?[\dA-PR-TZcf-nq-uy=><~]))/g; +const controlPattern = /[\u0000-\u0008\u000B\u000C\u000E-\u001A\u001C-\u001F\u007F]/g; +const oscPattern = /(?:\u001B\]|\u009D)[^\u0007\u001B\u009C\n]*(?:\u0007|\u001B\\|\u009C)/g; +let bytesWritten = 0; +let truncated = false; +function normalize(text) { + const normalized = text.replace(/\r\n?/g, '\n'); + return (config.stripAnsi + ? normalized.replace(oscPattern, '').replace(ansiPattern, '') + : normalized + ).replace(controlPattern, ''); +} +function writeEvent(text, stream, segment = 'live', extra = {}) { + const normalized = normalize(text); + if (normalized.length === 0) return; + const now = Date.now(); + const event = { + version: 1, + origin: 'environment', + group: config.source.group, + sourceId: config.source.id, + sourceTitle: config.source.title, + stream, + segment, + timestamp: new Date(now).toISOString(), + relativeTimeSec: Math.max(0, (now - config.startTimeMs) / 1000), + text: normalized, + ...extra, + }; + const serialized = JSON.stringify(event) + '\n'; + const logLine = normalized + '\n'; + const bytes = Buffer.byteLength(serialized) + Buffer.byteLength(logLine); + const truncationEvent = { + ...event, + text: '[ProofShot capture truncated at configured byte limit]', + truncated: true, + }; + const truncationSerialized = JSON.stringify(truncationEvent) + '\n'; + const truncationLogLine = truncationEvent.text + '\n'; + const truncationBytes = + Buffer.byteLength(truncationSerialized) + + Buffer.byteLength(truncationLogLine); + if (bytesWritten + bytes + truncationBytes > config.maxBytes) { + if (!truncated) { + truncated = true; + if (bytesWritten + truncationBytes <= config.maxBytes) { + bytesWritten += truncationBytes; + fs.appendFileSync(config.evidencePath, truncationSerialized); + fs.appendFileSync(config.logPath, truncationLogLine); + } + } + return; + } + bytesWritten += bytes; + fs.appendFileSync(config.evidencePath, serialized); + fs.appendFileSync(config.logPath, logLine); +} +function attachLines(stream, streamName) { + let buffer = ''; + stream.on('data', (chunk) => { + buffer += chunk.toString().replace(/\r\n?/g, '\n'); + const lines = buffer.split('\n'); + buffer = lines.pop() || ''; + for (const line of lines) writeEvent(line, streamName); + }); + stream.on('end', () => { + if (buffer.length > 0) writeEvent(buffer, streamName); + buffer = ''; + }); +} +if (config.pidFile) { + fs.writeFileSync(config.pidFile, String(process.pid), { mode: 0o600 }); +} +function removePidFile() { + if (config.pidFile) { + try { fs.unlinkSync(config.pidFile); } catch {} + } +} +`; + +const TMUX_PIPE_RUNNER_SOURCE = `${COMMON_WORKER_SOURCE} +attachLines(process.stdin, 'pty'); +process.stdin.on('end', () => { + removePidFile(); + process.exit(0); +}); +process.on('SIGTERM', () => { + removePidFile(); + process.exit(0); +}); +`; + +const PROCESS_RUNNER_SOURCE = `${COMMON_WORKER_SOURCE} +const { spawn } = require('child_process'); +let stopping = false; +const child = spawn(config.command, { + cwd: config.cwd, + env: { ...process.env, ...config.env }, + shell: config.shellPath, + stdio: ['ignore', 'pipe', 'pipe'], +}); +attachLines(child.stdout, 'stdout'); +attachLines(child.stderr, 'stderr'); +child.on('error', (error) => writeEvent(error.stack || error.message || String(error), 'stderr')); +child.on('close', (code) => { + writeEvent( + stopping + ? '[process stopped by ProofShot]' + : '[process exited with code ' + (code == null ? 'unknown' : code) + ']', + 'stderr', + ); + removePidFile(); + process.exit(stopping ? 0 : (code == null ? 1 : code)); +}); +for (const signal of ['SIGINT', 'SIGTERM']) { + process.on(signal, () => { + stopping = true; + try { child.kill(signal); } catch {} + }); +} +`; + +const FILE_RUNNER_SOURCE = `${COMMON_WORKER_SOURCE} +let offset = config.offset || 0; +let fileDevice = config.fileDevice; +let fileInode = config.fileInode; +let buffered = ''; +function readAvailable() { + let fd; + try { + fd = fs.openSync(config.filePath, 'r'); + } catch { + return; + } + const stat = fs.fstatSync(fd); + if ( + (fileDevice !== undefined && stat.dev !== fileDevice) || + (fileInode !== undefined && stat.ino !== fileInode) || + stat.size < offset + ) { + offset = 0; + writeEvent('[file rotated or truncated]', 'file', 'live', { captureGap: true }); + } + fileDevice = stat.dev; + fileInode = stat.ino; + if (stat.size === offset) { + fs.closeSync(fd); + return; + } + const length = Math.min(stat.size - offset, 64 * 1024); + const buffer = Buffer.alloc(length); + const bytesRead = fs.readSync(fd, buffer, 0, length, offset); + fs.closeSync(fd); + offset += bytesRead; + buffered += buffer.subarray(0, bytesRead).toString().replace(/\\r\\n?/g, '\\n'); + const lines = buffered.split('\\n'); + buffered = lines.pop() || ''; + for (const line of lines) writeEvent(line, 'file'); +} +const timer = setInterval(readAvailable, 100); +function stop() { + clearInterval(timer); + if (buffered.length > 0) writeEvent(buffered, 'file'); + removePidFile(); + process.exit(0); +} +process.on('SIGINT', stop); +process.on('SIGTERM', stop); +`; + +export function buildTmuxPipeCommand(config: WorkerConfig): string { + const encodedConfig = encodeConfig(config); + return [ + shellQuote(process.execPath), + '-e', + shellQuote(TMUX_PIPE_RUNNER_SOURCE), + shellQuote(encodedConfig), + ].join(' '); +} + +export async function waitForCaptureProcess( + sourceId: string, + pidFile: string, + timeoutMs = 2000, +): Promise { + const deadline = Date.now() + timeoutMs; + do { + const identity = readPidIdentity(pidFile); + if (identity) { + return { sourceId, process: identity, pidFile }; + } + await new Promise((resolve) => setTimeout(resolve, 25)); + } while (Date.now() < deadline); + + throw new Error(`ProofShot could not capture the log helper identity for ${sourceId}.`); +} + +export async function startProcessCapture( + definition: ProcessDefinition, + source: ResolvedLogSourceState, + evidencePath: string, + captureDir: string, + startTimeMs: number, + maxBytes: number, + stripAnsi: boolean, +): Promise { + const pidFile = path.join(captureDir, `${source.id}.pid`); + const config: WorkerConfig = { + evidencePath, + logPath: source.logPath, + pidFile, + startTimeMs, + maxBytes, + stripAnsi, + source, + command: definition.command, + cwd: definition.cwd, + env: definition.env, + shellPath: getShellExecutable(), + }; + return startDetachedWorker(source.id, pidFile, PROCESS_RUNNER_SOURCE, config); +} + +export async function startFileCapture( + filePath: string, + source: ResolvedLogSourceState, + evidencePath: string, + captureDir: string, + startTimeMs: number, + maxBytes: number, + stripAnsi: boolean, +): Promise { + const pidFile = path.join(captureDir, `${source.id}.pid`); + let offset = 0; + let fileDevice: number | undefined; + let fileInode: number | undefined; + let liveMaxBytes = maxBytes; + if (fs.existsSync(filePath)) { + const fd = fs.openSync(filePath, 'r'); + try { + const stat = fs.fstatSync(fd); + offset = stat.size; + fileDevice = stat.dev; + fileInode = stat.ino; + const historyBudget = Math.max(1, Math.floor(maxBytes / 2)); + liveMaxBytes = Math.max(1, maxBytes - historyBudget); + const historyLength = Math.min(stat.size, historyBudget); + const history = Buffer.alloc(historyLength); + fs.readSync(fd, history, 0, historyLength, stat.size - historyLength); + appendHistory( + history.toString('utf-8'), + source, + evidencePath, + historyBudget, + stripAnsi, + 'file', + ); + } finally { + fs.closeSync(fd); + } + } + + const config = { + evidencePath, + logPath: source.logPath, + pidFile, + startTimeMs, + maxBytes: liveMaxBytes, + stripAnsi, + source, + offset, + fileDevice, + fileInode, + filePath, + }; + return startDetachedWorker(source.id, pidFile, FILE_RUNNER_SOURCE, config); +} + +export function appendHistory( + raw: string, + source: ResolvedLogSourceState, + evidencePath: string, + maxBytes: number, + stripAnsi: boolean, + stream: EvidenceEvent['stream'], +): void { + const normalized = normalizeLogText(raw, stripAnsi); + const lines = normalized.split('\n').filter((line) => line.length > 0); + const retained: Array<{ event: EvidenceEvent; serialized: string; logLine: string }> = []; + let retainedBytes = 0; + let truncated = false; + for (let index = lines.length - 1; index >= 0; index -= 1) { + const event: EvidenceEvent = { + version: 1, + origin: 'environment', + group: source.group, + sourceId: source.id, + sourceTitle: source.title, + stream, + segment: 'history', + timestamp: null, + relativeTimeSec: null, + text: lines[index], + }; + const serialized = JSON.stringify(event) + '\n'; + const logLine = `${lines[index]}\n`; + const eventBytes = + Buffer.byteLength(serialized) + Buffer.byteLength(logLine); + if (retainedBytes + eventBytes > maxBytes) { + truncated = true; + break; + } + retained.unshift({ event, serialized, logLine }); + retainedBytes += eventBytes; + } + if (truncated && retained.length > 0) { + while (retained.length > 0) { + retained[0].event.truncated = true; + retained[0].serialized = JSON.stringify(retained[0].event) + '\n'; + retainedBytes = retained.reduce( + (total, entry) => + total + + Buffer.byteLength(entry.serialized) + + Buffer.byteLength(entry.logLine), + 0, + ); + if (retainedBytes <= maxBytes) break; + retained.shift(); + } + } + if (truncated && retained.length === 0) { + const event: EvidenceEvent = { + version: 1, + origin: 'environment', + group: source.group, + sourceId: source.id, + sourceTitle: source.title, + stream, + segment: 'history', + timestamp: null, + relativeTimeSec: null, + text: '[ProofShot capture truncated at configured byte limit]', + truncated: true, + }; + const serialized = JSON.stringify(event) + '\n'; + const logLine = `${event.text}\n`; + if ( + Buffer.byteLength(serialized) + Buffer.byteLength(logLine) <= + maxBytes + ) { + retained.push({ event, serialized, logLine }); + } + } + for (const entry of retained) { + fs.appendFileSync(evidencePath, entry.serialized); + fs.appendFileSync(source.logPath, entry.logLine); + } +} + +async function startDetachedWorker( + sourceId: string, + pidFile: string, + workerSource: string, + config: object, +): Promise { + fs.mkdirSync(path.dirname(pidFile), { recursive: true, mode: 0o700 }); + const errorFd = fs.openSync(`${pidFile}.stderr`, 'a', 0o600); + const worker = spawn(process.execPath, ['-e', workerSource, encodeConfig(config)], { + detached: true, + stdio: ['ignore', 'ignore', errorFd], + }); + fs.closeSync(errorFd); + worker.unref(); + + let identity = worker.pid ? captureProcessIdentity(worker.pid) : null; + for (let attempt = 0; !identity && attempt < 20; attempt += 1) { + await new Promise((resolve) => setTimeout(resolve, 10)); + identity = worker.pid ? captureProcessIdentity(worker.pid) : null; + } + if (!identity) { + try { + if (worker.pid) { + process.kill(-worker.pid, 'SIGKILL'); + } + } catch { + // The worker may already have exited. + } + throw new Error(`ProofShot could not capture the runner identity for ${sourceId}.`); + } + return { sourceId, process: identity, pidFile }; +} + +function readPidIdentity(pidFile: string): ProcessIdentity | null { + try { + const pid = Number(fs.readFileSync(pidFile, 'utf-8').trim()); + return captureProcessIdentity(pid); + } catch { + return null; + } +} + +function encodeConfig(config: object): string { + return Buffer.from(JSON.stringify(config)).toString('base64'); +} + +function shellQuote(value: string): string { + return `'${value.replace(/'/g, `'\\''`)}'`; +} diff --git a/src/index.ts b/src/index.ts index 8c73d55..6959baa 100644 --- a/src/index.ts +++ b/src/index.ts @@ -10,3 +10,17 @@ export { writeViewer, generateViewer } from './artifacts/viewer.js'; export type { SessionLogEntry } from './commands/exec.js'; export { writeMetadata, loadMetadata, findSessionsForBranch, type SessionMetadata } from './session/metadata.js'; export { formatPRComment, type PRCommentData } from './artifacts/pr-format.js'; +export { + startOwnedEnvironment, + stopOwnedEnvironment, +} from './environment/runtime.js'; +export { + loadEvidenceEvents, + normalizeLogText, +} from './environment/evidence.js'; +export type { + EnvironmentConfig, + EnvironmentState, + EvidenceEvent, + LogsConfig, +} from './environment/types.js'; diff --git a/src/session/state.ts b/src/session/state.ts index 7cbccf9..5a434f6 100644 --- a/src/session/state.ts +++ b/src/session/state.ts @@ -1,5 +1,6 @@ import * as fs from 'fs'; import * as path from 'path'; +import type { EnvironmentState } from '../environment/types.js'; const SESSION_FILENAME = '.session.json'; @@ -15,6 +16,7 @@ export interface SessionState { serverCommand: string | null; serverAlreadyRunning: boolean; recordingActive: boolean; + environment?: EnvironmentState | null; viewport?: { width: number; height: number }; } diff --git a/src/session/teardown.ts b/src/session/teardown.ts new file mode 100644 index 0000000..08bfbd5 --- /dev/null +++ b/src/session/teardown.ts @@ -0,0 +1,29 @@ +import { loadSession, saveSession } from './state.js'; +import { stopOwnedEnvironment } from '../environment/runtime.js'; + +/** + * Stop the environment recorded in the active session before its state is + * discarded. + * + * `.session.json` is the only record of the owned process and socket identities, + * so anything that erases it (`start --force`, `clean`) has to release those + * resources first and keep the recovery state when it cannot. Returns the + * cleanup failure, or null when there is nothing left to own. + */ +export async function releaseActiveSessionEnvironment( + outputDir: string, +): Promise { + const session = loadSession(outputDir); + if (!session?.environment) { + return null; + } + try { + await stopOwnedEnvironment(session.environment); + } catch (error) { + saveSession(session); + return error instanceof Error ? error : new Error(String(error)); + } + session.environment = null; + saveSession(session); + return null; +} diff --git a/src/utils/config.test.ts b/src/utils/config.test.ts index 5bd1bfd..57ba938 100644 --- a/src/utils/config.test.ts +++ b/src/utils/config.test.ts @@ -2,7 +2,7 @@ import * as fs from 'fs'; import * as os from 'os'; import * as path from 'path'; import { describe, expect, it } from 'vitest'; -import { loadConfig } from './config.js'; +import { loadConfig, loadConfigForTeardown } from './config.js'; describe('loadConfig', () => { it('merges nested browser config with defaults', () => { @@ -40,4 +40,179 @@ describe('loadConfig', () => { ignoreHttpsErrors: false, }); }); + + it('resolves control output against the ancestor config from a subdirectory', () => { + const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-output-path-')); + const nested = path.join(tempDir, 'nested', 'consumer'); + fs.mkdirSync(nested, { recursive: true }); + fs.writeFileSync( + path.join(tempDir, 'proofshot.config.json'), + JSON.stringify({ output: './project-proof' }), + ); + + expect(loadConfig(nested).output).toBe(path.join(tempDir, 'project-proof')); + }); + + it('resolves environment runners and file sources relative to the config', () => { + const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-environment-config-')); + fs.writeFileSync( + path.join(tempDir, 'proofshot.config.json'), + JSON.stringify({ + environment: { + kind: 'tmux', + launch: { + kind: 'panes', + panes: [ + { id: 'vite', command: 'npm run dev', cwd: './frontend' }, + { id: 'api', command: 'npm run api' }, + ], + }, + cwd: './workspace', + }, + logs: { + sources: [ + { id: 'vite', kind: 'tmux-pane', match: { connectionKey: 'vite' } }, + { id: 'worker', kind: 'file', path: './logs/worker.jsonl' }, + ], + }, + }), + ); + + const config = loadConfig(tempDir); + expect(config.environment).toMatchObject({ + kind: 'tmux', + cwd: path.join(tempDir, 'workspace'), + launch: { + panes: [ + { id: 'vite', cwd: path.join(tempDir, 'frontend') }, + { id: 'api', cwd: path.join(tempDir, 'workspace') }, + ], + }, + }); + expect(config.logs.sources?.[1]).toMatchObject({ + kind: 'file', + path: path.join(tempDir, 'logs', 'worker.jsonl'), + }); + }); + + it('fails closed for malformed or unsafe capture configuration', () => { + const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-invalid-config-')); + const configPath = path.join(tempDir, 'proofshot.config.json'); + + fs.writeFileSync( + configPath, + JSON.stringify({ environment: { kind: 'tmuxx' } }), + ); + expect(() => loadConfig(tempDir)).toThrow(/environment\.kind/); + + fs.writeFileSync( + configPath, + JSON.stringify({ + logs: { + sources: [{ id: '../escape', kind: 'file', path: './server.log' }], + }, + }), + ); + expect(() => loadConfig(tempDir)).toThrow(/logs\.sources\[0\]\.id/); + + fs.writeFileSync(configPath, '{not-json'); + expect(() => loadConfig(tempDir)).toThrow(/Invalid ProofShot config/); + }); + + it('rejects an attach-only launcher that also declares a stop command', () => { + const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-attach-stop-')); + const configPath = path.join(tempDir, 'proofshot.config.json'); + const environment = { + kind: 'tmux', + launch: { + kind: 'external-command', + command: 'npm run dev:tmux', + stopCommand: 'npm run dev:stop', + }, + connection: { format: 'json', ownership: 'attach', socket: '/tmp/dev.sock' }, + }; + + fs.writeFileSync(configPath, JSON.stringify({ environment })); + expect(() => loadConfig(tempDir)).toThrow(/attach.*cannot be combined with launch\.stopCommand/); + + fs.writeFileSync( + configPath, + JSON.stringify({ + environment: { ...environment, connection: { ...environment.connection, ownership: 'create' } }, + }), + ); + expect(() => loadConfig(tempDir)).not.toThrow(); + }); + + it('rejects log sources the configured environment cannot capture', () => { + const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-source-kind-')); + const configPath = path.join(tempDir, 'proofshot.config.json'); + + fs.writeFileSync( + configPath, + JSON.stringify({ + environment: { kind: 'processes', commands: [{ id: 'api', command: 'npm run api' }] }, + logs: { sources: [{ id: 'vite', kind: 'tmux-pane', match: { tag: 'vite' } }] }, + }), + ); + expect(() => loadConfig(tempDir)).toThrow(/requires environment\.kind "tmux"/); + + fs.writeFileSync( + configPath, + JSON.stringify({ + environment: { + kind: 'tmux', + launch: { kind: 'panes', panes: [{ id: 'vite', command: 'npm run dev' }] }, + }, + logs: { sources: [{ id: 'api', kind: 'process', processId: 'api' }] }, + }), + ); + expect(() => loadConfig(tempDir)).toThrow(/requires environment\.kind "processes"/); + + fs.writeFileSync( + configPath, + JSON.stringify({ + logs: { sources: [{ id: 'api', kind: 'process', processId: 'api' }] }, + }), + ); + expect(() => loadConfig(tempDir)).toThrow(/no environment/); + }); +}); + +describe('loadConfigForTeardown', () => { + it('resolves the recorded output directory when the config is invalid', () => { + const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-teardown-config-')); + fs.writeFileSync( + path.join(tempDir, 'proofshot.config.json'), + JSON.stringify({ + output: './project-proof', + environment: { + kind: 'tmux', + launch: { + kind: 'panes', + panes: [ + { id: 'vite', command: 'npm run dev' }, + { id: 'vite', command: 'npm run api' }, + ], + }, + }, + }), + ); + + const { config, error } = loadConfigForTeardown(tempDir); + expect(error?.message).toMatch(/Duplicate environment\.launch\.panes id/); + expect(config.output).toBe(path.join(tempDir, 'project-proof')); + }); + + it('reports no error for a valid config', () => { + const tempDir = fs.mkdtempSync(path.join(os.tmpdir(), 'proofshot-teardown-valid-')); + fs.writeFileSync( + path.join(tempDir, 'proofshot.config.json'), + JSON.stringify({ output: './proof' }), + ); + + const { config, error } = loadConfigForTeardown(tempDir); + expect(error).toBeNull(); + expect(config.output).toBe(path.join(tempDir, 'proof')); + }); }); diff --git a/src/utils/config.ts b/src/utils/config.ts index 4209ceb..36a8e30 100644 --- a/src/utils/config.ts +++ b/src/utils/config.ts @@ -1,5 +1,10 @@ import * as fs from 'fs'; import * as path from 'path'; +import type { + EnvironmentConfig, + LogsConfig, + LogSourceConfig, +} from '../environment/types.js'; export interface DevServerConfig { port: number; @@ -24,11 +29,17 @@ export interface ProofShotConfig { viewport: ViewportConfig; headless: boolean; browser: BrowserConfig; + environment?: EnvironmentConfig; + logs?: LogsConfig; +} + +export interface ResolvedProofShotConfig extends ProofShotConfig { + logs: LogsConfig; } const CONFIG_FILENAME = 'proofshot.config.json'; -const DEFAULT_CONFIG: ProofShotConfig = { +const DEFAULT_CONFIG: ResolvedProofShotConfig = { devServer: { port: 3000, startupTimeout: 30000, @@ -40,6 +51,11 @@ const DEFAULT_CONFIG: ProofShotConfig = { browser: { ignoreHttpsErrors: false, }, + logs: { + stripAnsi: true, + maxBytesPerSource: 5 * 1024 * 1024, + sources: [], + }, }; /** @@ -59,13 +75,14 @@ export function findConfigPath(startDir?: string): string | null { /** * Load config from disk, merging with defaults. */ -export function loadConfig(startDir?: string): ProofShotConfig { +export function loadConfig(startDir?: string): ResolvedProofShotConfig { const configPath = findConfigPath(startDir); if (!configPath) return { ...DEFAULT_CONFIG }; try { const raw = fs.readFileSync(configPath, 'utf-8'); const parsed = JSON.parse(raw); + validateConfig(parsed); const configDir = path.dirname(configPath); const resolvedBrowser = { ...DEFAULT_CONFIG.browser, @@ -74,16 +91,442 @@ export function loadConfig(startDir?: string): ProofShotConfig { if (resolvedBrowser.configPath) { resolvedBrowser.configPath = path.resolve(configDir, resolvedBrowser.configPath); } + const environment = resolveEnvironmentConfig(parsed.environment, configDir); + const logs = resolveLogsConfig(parsed.logs, configDir); return { ...DEFAULT_CONFIG, ...parsed, + output: path.resolve( + configDir, + typeof parsed.output === 'string' ? parsed.output : DEFAULT_CONFIG.output, + ), devServer: { ...DEFAULT_CONFIG.devServer, ...parsed.devServer }, viewport: { ...DEFAULT_CONFIG.viewport, ...parsed.viewport }, browser: resolvedBrowser, + environment, + logs, }; + } catch (error) { + const message = error instanceof Error ? error.message : String(error); + throw new Error(`Invalid ProofShot config at ${configPath}: ${message}`); + } +} + +export interface TeardownConfig { + config: ResolvedProofShotConfig; + error: Error | null; +} + +/** + * Resolve the configuration needed to tear an existing session down. + * + * Startup fails closed on an invalid config, but teardown must still be able to + * reach the recorded session and its owned resources, so validation failures are + * reported to the caller instead of thrown. + */ +export function loadConfigForTeardown(startDir?: string): TeardownConfig { + try { + return { config: loadConfig(startDir), error: null }; + } catch (error) { + return { + config: resolveTeardownFallback(startDir), + error: error instanceof Error ? error : new Error(String(error)), + }; + } +} + +function resolveTeardownFallback(startDir?: string): ResolvedProofShotConfig { + const fallback: ResolvedProofShotConfig = { + ...DEFAULT_CONFIG, + browser: { ...DEFAULT_CONFIG.browser }, + logs: { ...DEFAULT_CONFIG.logs, sources: [] }, + }; + const configPath = findConfigPath(startDir); + if (!configPath) return fallback; + + const configDir = path.dirname(configPath); + let output = DEFAULT_CONFIG.output; + try { + const parsed = JSON.parse(fs.readFileSync(configPath, 'utf-8')); + if (typeof parsed?.output === 'string') { + output = parsed.output; + } + if (typeof parsed?.browser?.configPath === 'string') { + fallback.browser.configPath = path.resolve(configDir, parsed.browser.configPath); + } } catch { - return { ...DEFAULT_CONFIG }; + // An unreadable config still resolves the default output directory. } + fallback.output = path.resolve(configDir, output); + return fallback; +} + +function validateConfig(value: unknown): void { + assertRecord(value, 'config'); + assertOptionalString(value.output, 'output'); + assertOptionalBoolean(value.headless, 'headless'); + assertOptionalStringArray(value.defaultPages, 'defaultPages'); + + if (value.devServer !== undefined) { + assertRecord(value.devServer, 'devServer'); + assertOptionalPositiveInteger(value.devServer.port, 'devServer.port', 65535); + assertOptionalPositiveInteger( + value.devServer.startupTimeout, + 'devServer.startupTimeout', + ); + } + if (value.viewport !== undefined) { + assertRecord(value.viewport, 'viewport'); + assertOptionalPositiveInteger(value.viewport.width, 'viewport.width'); + assertOptionalPositiveInteger(value.viewport.height, 'viewport.height'); + } + if (value.browser !== undefined) { + assertRecord(value.browser, 'browser'); + assertOptionalString(value.browser.configPath, 'browser.configPath'); + assertOptionalString(value.browser.executablePath, 'browser.executablePath'); + assertOptionalBoolean(value.browser.ignoreHttpsErrors, 'browser.ignoreHttpsErrors'); + } + validateEnvironment(value.environment); + validateLogs(value.logs); + validateSourceEnvironmentAlignment(value.environment, value.logs); +} + +/** + * A declared log source that the configured environment cannot capture would be + * silently dropped at runtime, so reject the combination while it is still a + * config error. + */ +function validateSourceEnvironmentAlignment( + environment: unknown, + logs: unknown, +): void { + if (typeof logs !== 'object' || logs === null) return; + const sources = (logs as Record).sources; + if (!Array.isArray(sources)) return; + + const environmentKind = + typeof environment === 'object' && environment !== null + ? (environment as Record).kind + : undefined; + const requiredKind = + environmentKind === 'tmux' + ? 'tmux-pane' + : environmentKind === 'processes' + ? 'process' + : undefined; + + sources.forEach((source, index) => { + const kind = (source as Record).kind; + if (kind === 'file' || kind === requiredKind) return; + throw new Error( + `logs.sources[${index}].kind "${String(kind)}" requires environment.kind ` + + `"${kind === 'tmux-pane' ? 'tmux' : 'processes'}", found ` + + `${environmentKind === undefined ? 'no environment' : `"${String(environmentKind)}"`}`, + ); + }); +} + +function validateEnvironment(value: unknown): void { + if (value === undefined) return; + assertRecord(value, 'environment'); + validateReadiness(value.readiness); + + if (value.kind === 'tmux') { + assertRecord(value.launch, 'environment.launch'); + assertOptionalString(value.cwd, 'environment.cwd'); + if (value.launch.kind === 'panes') { + if (!Array.isArray(value.launch.panes) || value.launch.panes.length === 0) { + throw new Error('environment.launch.panes must be a non-empty array'); + } + validateDefinitions(value.launch.panes, 'environment.launch.panes'); + assertOptionalString( + value.launch.sessionName, + 'environment.launch.sessionName', + ); + if (value.connection !== undefined) { + throw new Error('environment.connection is only valid for external-command'); + } + return; + } + if (value.launch.kind === 'external-command') { + assertNonEmptyString(value.launch.command, 'environment.launch.command'); + assertOptionalString( + value.launch.stopCommand, + 'environment.launch.stopCommand', + ); + assertOptionalPositiveInteger( + value.launch.timeoutMs, + 'environment.launch.timeoutMs', + ); + assertRecord(value.connection, 'environment.connection'); + if ( + value.connection.format !== 'json' && + value.connection.format !== 'tmux-attach-command' + ) { + throw new Error( + 'environment.connection.format must be "json" or "tmux-attach-command"', + ); + } + if ( + value.connection.source !== undefined && + value.connection.source !== 'stdout' + ) { + throw new Error('environment.connection.source must be "stdout"'); + } + assertOptionalString(value.connection.socket, 'environment.connection.socket'); + if ( + value.connection.ownership !== undefined && + value.connection.ownership !== 'attach' && + value.connection.ownership !== 'create' + ) { + throw new Error( + 'environment.connection.ownership must be "attach" or "create"', + ); + } + if ( + value.connection.ownership === 'attach' && + value.launch.stopCommand !== undefined + ) { + throw new Error( + 'environment.connection.ownership "attach" cannot be combined with launch.stopCommand; attach-only environments are never stopped by ProofShot', + ); + } + if ( + value.connection.ownership !== 'attach' && + value.connection.socket === undefined && + value.launch.stopCommand === undefined + ) { + throw new Error( + 'external-command requires connection.socket or launch.stopCommand for cleanup', + ); + } + return; + } + throw new Error( + 'environment.launch.kind must be "panes" or "external-command"', + ); + } + + if (value.kind === 'processes') { + if (!Array.isArray(value.commands)) { + throw new Error('environment.commands must be an array'); + } + validateDefinitions(value.commands, 'environment.commands'); + return; + } + throw new Error('environment.kind must be "tmux" or "processes"'); +} + +function validateDefinitions(value: unknown[], field: string): void { + const ids = new Set(); + value.forEach((candidate, index) => { + const item = `${field}[${index}]`; + assertRecord(candidate, item); + assertSafeId(candidate.id, `${item}.id`); + if (ids.has(candidate.id)) throw new Error(`Duplicate ${field} id: ${candidate.id}`); + ids.add(candidate.id); + assertNonEmptyString(candidate.command, `${item}.command`); + assertOptionalString(candidate.title, `${item}.title`); + assertOptionalString(candidate.group, `${item}.group`); + assertOptionalString(candidate.cwd, `${item}.cwd`); + if (candidate.env !== undefined) { + assertRecord(candidate.env, `${item}.env`); + for (const [key, envValue] of Object.entries(candidate.env)) { + if (typeof envValue !== 'string') { + throw new Error(`${item}.env.${key} must be a string`); + } + } + } + }); +} + +function validateReadiness(value: unknown): void { + if (value === undefined) return; + if (!Array.isArray(value)) throw new Error('environment.readiness must be an array'); + value.forEach((candidate, index) => { + const item = `environment.readiness[${index}]`; + assertRecord(candidate, item); + assertOptionalPositiveInteger(candidate.timeoutMs, `${item}.timeoutMs`); + if (candidate.kind === 'http') { + assertNonEmptyString(candidate.url, `${item}.url`); + return; + } + if (candidate.kind === 'tcp') { + assertOptionalString(candidate.host, `${item}.host`); + assertOptionalPositiveInteger(candidate.port, `${item}.port`, 65535, true); + return; + } + throw new Error(`${item}.kind must be "http" or "tcp"`); + }); +} + +function validateLogs(value: unknown): void { + if (value === undefined) return; + assertRecord(value, 'logs'); + assertOptionalBoolean(value.stripAnsi, 'logs.stripAnsi'); + assertOptionalPositiveInteger(value.maxBytesPerSource, 'logs.maxBytesPerSource'); + if ( + value.maxBytesPerSource !== undefined && + value.maxBytesPerSource < 512 + ) { + throw new Error('logs.maxBytesPerSource must be at least 512 bytes'); + } + if (value.sources === undefined) return; + if (!Array.isArray(value.sources)) throw new Error('logs.sources must be an array'); + + const ids = new Set(); + value.sources.forEach((candidate, index) => { + const item = `logs.sources[${index}]`; + assertRecord(candidate, item); + assertSafeId(candidate.id, `${item}.id`); + if (ids.has(candidate.id)) throw new Error(`Duplicate log source id: ${candidate.id}`); + ids.add(candidate.id); + assertOptionalString(candidate.title, `${item}.title`); + assertOptionalString(candidate.group, `${item}.group`); + + if (candidate.kind === 'tmux-pane') { + assertRecord(candidate.match, `${item}.match`); + const keys = ['connectionKey', 'tag', 'target'].filter( + (key) => candidate.match[key] !== undefined, + ); + if (keys.length !== 1) { + throw new Error(`${item}.match must set exactly one pane selector`); + } + assertNonEmptyString(candidate.match[keys[0]], `${item}.match.${keys[0]}`); + return; + } + if (candidate.kind === 'process') { + assertSafeId(candidate.processId, `${item}.processId`); + return; + } + if (candidate.kind === 'file') { + assertNonEmptyString(candidate.path, `${item}.path`); + return; + } + throw new Error(`${item}.kind is unsupported`); + }); +} + +function assertRecord( + value: unknown, + field: string, +): asserts value is Record { + if (typeof value !== 'object' || value === null || Array.isArray(value)) { + throw new Error(`${field} must be an object`); + } +} + +function assertSafeId(value: unknown, field: string): asserts value is string { + assertNonEmptyString(value, field); + if (!/^[a-zA-Z0-9][a-zA-Z0-9_-]*$/.test(value)) { + throw new Error(`${field} must contain only letters, numbers, "_" or "-"`); + } +} + +function assertNonEmptyString( + value: unknown, + field: string, +): asserts value is string { + if (typeof value !== 'string' || value.length === 0) { + throw new Error(`${field} must be a non-empty string`); + } +} + +function assertOptionalString(value: unknown, field: string): void { + if (value !== undefined && typeof value !== 'string') { + throw new Error(`${field} must be a string`); + } +} + +function assertOptionalBoolean(value: unknown, field: string): void { + if (value !== undefined && typeof value !== 'boolean') { + throw new Error(`${field} must be a boolean`); + } +} + +function assertOptionalStringArray(value: unknown, field: string): void { + if ( + value !== undefined && + (!Array.isArray(value) || value.some((entry) => typeof entry !== 'string')) + ) { + throw new Error(`${field} must be an array of strings`); + } +} + +function assertOptionalPositiveInteger( + value: unknown, + field: string, + maximum = Number.MAX_SAFE_INTEGER, + required = false, +): void { + if (value === undefined && !required) return; + if ( + !Number.isInteger(value) || + (value as number) <= 0 || + (value as number) > maximum + ) { + throw new Error(`${field} must be a positive integer no greater than ${maximum}`); + } +} + +function resolveEnvironmentConfig( + value: unknown, + configDir: string, +): EnvironmentConfig | undefined { + if (typeof value !== 'object' || value === null) { + return undefined; + } + const environment = value as EnvironmentConfig; + if (environment.kind === 'tmux') { + const launch = + environment.launch.kind === 'panes' + ? { + ...environment.launch, + panes: environment.launch.panes.map((pane) => ({ + ...pane, + cwd: path.resolve(configDir, pane.cwd || environment.cwd || '.'), + })), + } + : environment.launch; + return { + ...environment, + cwd: path.resolve(configDir, environment.cwd || '.'), + connection: environment.connection?.socket + ? { + ...environment.connection, + socket: path.resolve(configDir, environment.connection.socket), + } + : environment.connection, + launch, + }; + } + if (environment.kind === 'processes') { + return { + ...environment, + commands: environment.commands.map((command) => ({ + ...command, + cwd: path.resolve(configDir, command.cwd || '.'), + })), + }; + } + return undefined; +} + +function resolveLogsConfig(value: unknown, configDir: string): LogsConfig { + const logs = + typeof value === 'object' && value !== null + ? (value as LogsConfig) + : DEFAULT_CONFIG.logs; + const sources: LogSourceConfig[] = (logs.sources || []).map((source) => + source.kind === 'file' + ? { ...source, path: path.resolve(configDir, source.path) } + : source, + ); + return { + ...DEFAULT_CONFIG.logs, + ...logs, + sources, + }; } /** diff --git a/src/utils/errors.test.ts b/src/utils/errors.test.ts new file mode 100644 index 0000000..9457794 --- /dev/null +++ b/src/utils/errors.test.ts @@ -0,0 +1,42 @@ +import { describe, expect, it } from 'vitest'; +import { formatError, formatErrorDetail } from './errors.js'; + +describe('formatErrorDetail', () => { + it('names every cause collected by cleanup', () => { + const error = new AggregateError( + [ + new Error('Log helper for vite did not stop.'), + new Error('Owned tmux server did not stop.'), + ], + 'One or more tmux cleanup steps failed.', + ); + + expect(formatErrorDetail(error)).toBe( + [ + 'One or more tmux cleanup steps failed.', + ' - Log helper for vite did not stop.', + ' - Owned tmux server did not stop.', + ].join('\n'), + ); + }); + + it('indents nested aggregate causes', () => { + const error = new AggregateError( + [new AggregateError([new Error('inner')], 'outer cause')], + 'summary', + ); + + expect(formatErrorDetail(error)).toBe( + ['summary', ' - outer cause', ' - inner'].join('\n'), + ); + }); + + it('falls back to the plain message for other errors', () => { + expect(formatErrorDetail(new Error('boom'))).toBe('boom'); + expect(formatErrorDetail(new AggregateError([], 'nothing collected'))).toBe( + 'nothing collected', + ); + expect(formatErrorDetail('raw failure')).toBe('raw failure'); + expect(formatError(new Error('boom'))).toBe('boom'); + }); +}); diff --git a/src/utils/errors.ts b/src/utils/errors.ts new file mode 100644 index 0000000..718991e --- /dev/null +++ b/src/utils/errors.ts @@ -0,0 +1,27 @@ +/** + * Render an error's own message, without its causes. + */ +export function formatError(error: unknown): string { + return error instanceof Error ? error.message : String(error); +} + +/** + * Render an error for the CLI, naming every nested cause. + * + * Cleanup collects independent failures into an `AggregateError` whose message is + * only a summary, so the actionable causes live in `errors` and have to be spelled + * out or the user is told to resolve a problem that was never named. + */ +export function formatErrorDetail(error: unknown, indent = ' '): string { + const summary = formatError(error); + const causes = error instanceof AggregateError ? error.errors : []; + if (!Array.isArray(causes) || causes.length === 0) { + return summary; + } + return ( + summary + + causes + .map((cause) => `\n${indent}- ${formatErrorDetail(cause, `${indent} `)}`) + .join('') + ); +} diff --git a/src/utils/process.test.ts b/src/utils/process.test.ts index 8d73db5..c6ee6c1 100644 --- a/src/utils/process.test.ts +++ b/src/utils/process.test.ts @@ -1,11 +1,35 @@ +import * as fs from 'fs'; +import { spawn } from 'child_process'; import { afterEach, describe, expect, it, vi } from 'vitest'; import { + captureProcessIdentity, findExecutablePath, getShellExecutable, + isDetachedProcessIdentity, + parseLinuxProcStat, + parseUnixProcessIdentity, parseWindowsNetstatOutput, readCommandVersion, + terminateOwnedProcess, + terminateOwnedProcessTree, } from './process.js'; +function waitForExit(pid: number, timeoutMs = 3000): Promise { + const deadline = Date.now() + timeoutMs; + return new Promise((resolve, reject) => { + const poll = () => { + if (!captureProcessIdentity(pid)) { + resolve(); + } else if (Date.now() >= deadline) { + reject(new Error(`process ${pid} did not exit`)); + } else { + setTimeout(poll, 25); + } + }; + poll(); + }); +} + describe('getShellExecutable', () => { it('uses cmd.exe on Windows when ComSpec is missing', () => { expect(getShellExecutable('win32', {})).toBe('cmd.exe'); @@ -62,3 +86,95 @@ describe('readCommandVersion', () => { expect(execSpy).toHaveBeenCalledWith('ffmpeg --version', expect.any(Object)); }); }); + +describe('process ownership', () => { + it('parses immutable Linux ownership fields', () => { + if (process.platform !== 'linux') return; + const stat = fs.readFileSync(`/proc/${process.pid}/stat`, 'utf-8'); + const identity = parseLinuxProcStat(stat); + expect(identity).toMatchObject({ pid: process.pid }); + expect(identity?.processGroupId).toBeGreaterThan(0); + expect(identity?.sessionId).toBeGreaterThan(0); + expect(identity?.startTime).toMatch(/^\d+$/); + }); + + it('parses macOS process ownership fields', () => { + expect( + parseUnixProcessIdentity( + 321, + ' 321 0 Sun Aug 9 19:35:36 2026 \n', + ), + ).toEqual({ + pid: 321, + processGroupId: 321, + sessionId: 0, + startTime: 'Sun Aug 9 19:35:36 2026', + }); + }); + + it('terminates only the exact detached process session it owns', async () => { + if (process.platform === 'win32') return; + const owned = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { + detached: true, + stdio: 'ignore', + }); + const unrelated = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { + detached: true, + stdio: 'ignore', + }); + owned.unref(); + unrelated.unref(); + + const ownedIdentity = captureProcessIdentity(owned.pid!); + const unrelatedIdentity = captureProcessIdentity(unrelated.pid!); + expect(ownedIdentity && isDetachedProcessIdentity(ownedIdentity)).toBe(true); + expect(unrelatedIdentity && isDetachedProcessIdentity(unrelatedIdentity)).toBe(true); + + try { + await expect( + terminateOwnedProcessTree(ownedIdentity, { graceMs: 200 }), + ).resolves.toBe(true); + await waitForExit(owned.pid!); + expect(captureProcessIdentity(unrelated.pid!)).not.toBeNull(); + + await expect( + terminateOwnedProcessTree( + ownedIdentity && { ...ownedIdentity, startTime: `${ownedIdentity.startTime}-reused` }, + { graceMs: 20 }, + ), + ).resolves.toBe(false); + expect(captureProcessIdentity(unrelated.pid!)).not.toBeNull(); + } finally { + await terminateOwnedProcessTree(unrelatedIdentity, { graceMs: 200 }); + await waitForExit(unrelated.pid!); + } + // Ownership checks shell out to `ps` on every poll, which outruns the 5s + // default timeout on a loaded machine. + }, 15000); + + it('terminates an exact helper without widening to its process group', async () => { + const helper = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { + stdio: 'ignore', + }); + const unrelated = spawn(process.execPath, ['-e', 'setInterval(() => {}, 1000)'], { + stdio: 'ignore', + }); + const helperIdentity = captureProcessIdentity(helper.pid!); + const unrelatedIdentity = captureProcessIdentity(unrelated.pid!); + expect(helperIdentity).not.toBeNull(); + expect(unrelatedIdentity).not.toBeNull(); + + try { + await expect( + terminateOwnedProcess(helperIdentity, { graceMs: 200 }), + ).resolves.toBe(true); + await waitForExit(helper.pid!); + expect(captureProcessIdentity(unrelated.pid!)).not.toBeNull(); + } finally { + if (unrelatedIdentity) { + process.kill(unrelatedIdentity.pid, 'SIGKILL'); + await waitForExit(unrelatedIdentity.pid); + } + } + }, 15000); +}); diff --git a/src/utils/process.ts b/src/utils/process.ts index 7d13cb5..351c634 100644 --- a/src/utils/process.ts +++ b/src/utils/process.ts @@ -1,7 +1,36 @@ -import { execSync, spawn, type ChildProcess, type SpawnOptions } from 'child_process'; +import * as fs from 'fs'; +import { + execFileSync, + execSync, + spawn, + type ChildProcess, + type SpawnOptions, +} from 'child_process'; type ExecSyncLike = typeof execSync; +/** + * Immutable identity for a process which started an isolated process session. + * + * A PID alone is not sufficient ownership proof because the operating system + * can reuse it. `startTime` lets cleanup reject a recycled PID, while the + * process/session group ids let ProofShot terminate only descendants created + * by the detached process it started. + */ +export interface ProcessIdentity { + pid: number; + processGroupId: number; + sessionId: number; + startTime: string; + /** Stable boot token preventing cross-boot PID/start-time collisions. */ + bootId?: string; +} + +export interface TerminateProcessTreeOptions { + graceMs?: number; + pollIntervalMs?: number; +} + export function getShellExecutable( platform = process.platform, env: NodeJS.ProcessEnv = process.env, @@ -23,6 +52,359 @@ export function spawnShellCommand( }); } +/** Parse the ownership fields from Linux `/proc//stat`. */ +export function parseLinuxProcStat(stat: string): ProcessIdentity | null { + const closeParen = stat.lastIndexOf(')'); + if (closeParen < 0) return null; + + const pid = Number(stat.slice(0, stat.indexOf(' '))); + const fields = stat.slice(closeParen + 2).trim().split(/\s+/); + const processGroupId = Number(fields[2]); + const sessionId = Number(fields[3]); + const startTime = fields[19]; + + if ( + !Number.isInteger(pid) || + !Number.isInteger(processGroupId) || + !Number.isInteger(sessionId) || + !startTime + ) { + return null; + } + + return { pid, processGroupId, sessionId, startTime }; +} + +/** Parse the ownership fields emitted by BSD/POSIX ps implementations. */ +export function parseUnixProcessIdentity( + pid: number, + output: string, +): ProcessIdentity | null { + const match = output.trim().match(/^(\d+)\s+(\d+)\s+(.+)$/); + if (!match) return null; + + const processGroupId = Number(match[1]); + const sessionId = Number(match[2]); + const startTime = match[3]; + if ( + !Number.isInteger(processGroupId) || + processGroupId <= 0 || + !Number.isInteger(sessionId) || + sessionId < 0 || + !startTime + ) { + return null; + } + + return { pid, processGroupId, sessionId, startTime }; +} + +/** + * Detached children are session leaders on Linux and process-group leaders on + * macOS, whose ps implementation reports a zero session id. + */ +export function isDetachedProcessIdentity( + identity: ProcessIdentity, + platform = process.platform, +): boolean { + if (platform === 'darwin') { + return identity.processGroupId === identity.pid; + } + return identity.sessionId === identity.pid; +} + +let cachedBootToken: string | null | undefined; + +/** + * The boot token is constant for the lifetime of this process, and identity + * capture runs inside termination poll loops, so read it only once. + */ +function readBootToken(): string | null { + if (cachedBootToken === undefined) { + cachedBootToken = captureBootToken(); + } + return cachedBootToken; +} + +function captureBootToken(): string | null { + try { + if (process.platform === 'linux') { + return ( + fs.readFileSync('/proc/sys/kernel/random/boot_id', 'utf-8').trim() || null + ); + } + if (process.platform === 'darwin') { + // `kern.boottime` ends in a locale/timezone-rendered date, so the epoch + // seconds are the only stable part of the reading. + const output = execFileSync('sysctl', ['-n', 'kern.boottime'], { + encoding: 'utf-8', + stdio: ['ignore', 'pipe', 'pipe'], + env: { ...process.env, TZ: 'UTC' }, + }).trim(); + const seconds = output.match(/\bsec\s*=\s*(\d+)/); + return seconds ? `boot-${seconds[1]}` : output || null; + } + } catch { + return null; + } + return null; +} + +/** + * Capture the current immutable identity for a process. + * Returns null when the process is already gone or cannot be inspected. + */ +export function captureProcessIdentity(pid: number): ProcessIdentity | null { + if (!Number.isInteger(pid) || pid <= 0) return null; + + if (process.platform === 'linux') { + try { + const identity = parseLinuxProcStat(fs.readFileSync(`/proc/${pid}/stat`, 'utf-8')); + const bootId = readBootToken(); + if (!identity || !bootId) return null; + return { ...identity, bootId }; + } catch { + return null; + } + } + + if (process.platform !== 'win32') { + try { + const sessionField = process.platform === 'darwin' ? 'sess=' : 'sid='; + const output = execFileSync( + 'ps', + ['-o', 'pgid=', '-o', sessionField, '-o', 'lstart=', '-p', String(pid)], + { + encoding: 'utf-8', + stdio: ['ignore', 'pipe', 'pipe'], + env: { ...process.env, TZ: 'UTC' }, + }, + ); + const identity = parseUnixProcessIdentity(pid, output); + if (!identity) return null; + if (process.platform !== 'darwin') return identity; + const bootId = readBootToken(); + return bootId ? { ...identity, bootId } : null; + } catch { + return null; + } + } + + // PowerShell exposes the process creation timestamp. If that immutable token + // cannot be read, refuse ownership instead of treating a reusable PID as + // sufficient proof for taskkill /T. + try { + const script = + `(Get-Process -Id ${pid} -ErrorAction Stop).StartTime.ToUniversalTime().Ticks`; + const startTime = execFileSync( + 'powershell.exe', + ['-NoLogo', '-NoProfile', '-NonInteractive', '-Command', script], + { encoding: 'utf-8', stdio: ['ignore', 'pipe', 'pipe'] }, + ).trim(); + if (!/^\d+$/.test(startTime)) return null; + return { pid, processGroupId: pid, sessionId: pid, startTime }; + } catch { + return null; + } +} + +export function processIdentityMatches(identity: ProcessIdentity): boolean { + const current = captureProcessIdentity(identity.pid); + return Boolean(current && processIdentitiesMatch(current, identity)); +} + +export function processIdentitiesMatch( + left: ProcessIdentity, + right: ProcessIdentity, +): boolean { + return ( + left.pid === right.pid && + left.processGroupId === right.processGroupId && + left.sessionId === right.sessionId && + left.startTime === right.startTime && + left.bootId === right.bootId + ); +} + +function listProcessGroupsInSession(sessionId: number): number[] { + const groups = new Set(); + + if (process.platform === 'linux') { + let entries: string[] = []; + try { + entries = fs.readdirSync('/proc'); + } catch { + return []; + } + + for (const entry of entries) { + if (!/^\d+$/.test(entry)) continue; + try { + const identity = parseLinuxProcStat( + fs.readFileSync(`/proc/${entry}/stat`, 'utf-8'), + ); + if (identity?.sessionId === sessionId) { + groups.add(identity.processGroupId); + } + } catch { + // The process may exit while /proc is being scanned. + } + } + return [...groups]; + } + + if (process.platform !== 'win32') { + try { + const sessionField = process.platform === 'darwin' ? 'sess=' : 'sid='; + const output = execFileSync('ps', ['-axo', `pgid=,${sessionField}`], { + encoding: 'utf-8', + stdio: ['ignore', 'pipe', 'pipe'], + }); + for (const line of output.split(/\r?\n/)) { + const match = line.trim().match(/^(\d+)\s+(\d+)$/); + if (match && Number(match[2]) === sessionId) { + groups.add(Number(match[1])); + } + } + } catch { + return []; + } + } + + return [...groups]; +} + +function processGroupIsAlive(processGroupId: number): boolean { + try { + process.kill(-processGroupId, 0); + return true; + } catch (error) { + return (error as NodeJS.ErrnoException).code === 'EPERM'; + } +} + +export function ownedProcessTreeIsAlive(identity: ProcessIdentity): boolean { + if (process.platform === 'win32') return processIdentityMatches(identity); + + const current = captureProcessIdentity(identity.pid); + if (current && !processIdentitiesMatch(current, identity)) return false; + + if (process.platform === 'darwin') { + return processGroupIsAlive(identity.processGroupId); + } + return listProcessGroupsInSession(identity.sessionId).length > 0; +} + +function signalOwnedTree(identity: ProcessIdentity, signal: NodeJS.Signals): boolean { + if (process.platform === 'win32') return false; + + const current = captureProcessIdentity(identity.pid); + if (current && !processIdentitiesMatch(current, identity)) return false; + + if (!isDetachedProcessIdentity(identity)) return false; + if (process.platform === 'darwin') { + if (!processGroupIsAlive(identity.processGroupId)) return false; + try { + process.kill(-identity.processGroupId, signal); + return true; + } catch { + return false; + } + } + + // Detached children created by ProofShot are session leaders. If that leader + // has already exited, its session id cannot be reused while descendants from + // that session remain, so scanning the recorded session stays ownership-safe. + const groups = listProcessGroupsInSession(identity.sessionId); + if (groups.length === 0) return false; + + let signalled = false; + for (const groupId of groups) { + if (!Number.isInteger(groupId) || groupId <= 0) continue; + try { + process.kill(-groupId, signal); + signalled = true; + } catch { + // A group can exit between discovery and signalling. + } + } + return signalled; +} + +/** + * Terminate only the detached process session represented by `identity`. + * Missing/already-dead processes are an idempotent no-op. A recycled PID is + * rejected rather than widening cleanup to a name or port match. + */ +export async function terminateOwnedProcessTree( + identity: ProcessIdentity | null | undefined, + options: TerminateProcessTreeOptions = {}, +): Promise { + if (!identity) return false; + + if (process.platform === 'win32') { + if (!processIdentityMatches(identity)) return false; + try { + execFileSync('taskkill', ['/F', '/T', '/PID', String(identity.pid)], { + stdio: 'pipe', + }); + return true; + } catch { + return false; + } + } + + if (!ownedProcessTreeIsAlive(identity)) return false; + const signalled = signalOwnedTree(identity, 'SIGTERM'); + if (!signalled) return false; + + const graceMs = options.graceMs ?? 1500; + const pollIntervalMs = options.pollIntervalMs ?? 50; + const deadline = Date.now() + graceMs; + while (Date.now() < deadline && ownedProcessTreeIsAlive(identity)) { + await new Promise((resolve) => setTimeout(resolve, pollIntervalMs)); + } + + if (ownedProcessTreeIsAlive(identity)) { + signalOwnedTree(identity, 'SIGKILL'); + const killDeadline = Date.now() + 500; + while (Date.now() < killDeadline && ownedProcessTreeIsAlive(identity)) { + await new Promise((resolve) => setTimeout(resolve, pollIntervalMs)); + } + } + return true; +} + +export async function terminateOwnedProcess( + identity: ProcessIdentity | null | undefined, + options: TerminateProcessTreeOptions = {}, +): Promise { + if (!identity || !processIdentityMatches(identity)) { + return false; + } + + const graceMs = options.graceMs ?? 1500; + const pollIntervalMs = options.pollIntervalMs ?? 50; + try { + process.kill(identity.pid, 'SIGTERM'); + } catch { + return false; + } + + const deadline = Date.now() + graceMs; + while (Date.now() < deadline && processIdentityMatches(identity)) { + await new Promise((resolve) => setTimeout(resolve, pollIntervalMs)); + } + if (processIdentityMatches(identity)) { + try { + process.kill(identity.pid, 'SIGKILL'); + } catch { + return false; + } + } + return true; +} + export function parseWindowsNetstatOutput(output: string, port: number): number[] { const pids = new Set(); diff --git a/src/utils/skills.ts b/src/utils/skills.ts index 10db083..4fb1b10 100644 --- a/src/utils/skills.ts +++ b/src/utils/skills.ts @@ -103,24 +103,26 @@ Default upload mode uses the official GitHub contents API on a \`proofshot-artif if (agent === 'cursor') { return `--- -description: Visual verification of UI changes using ProofShot -globs: ["**/*.tsx", "**/*.jsx", "**/*.vue", "**/*.svelte", "**/*.html"] +name: proofshot +description: Visually verifies UI changes with browser recordings, screenshots, console output, and named environment logs. Use after building or modifying user-facing features. --- -After modifying UI files, visually verify changes with this workflow: +# ProofShot visual verification -1. Start session: \`proofshot start --run "your-dev-command" --port PORT --description "what you are verifying"\` - If the server is already running, omit --run. -2. Drive browser: Use \`proofshot exec\` commands to navigate, click, fill forms, and take screenshots -3. Stop session: \`proofshot stop\` to bundle video + screenshots + error report -4. (Optional) Post to PR: \`proofshot pr\` to upload proof to the GitHub PR - Default provider uses the official contents API. Use \`--upload-provider github-web-attachments\` only if you specifically want GitHub attachment URLs. +Use ProofShot after changing UI behavior: -Key proofshot exec commands: -- \`proofshot exec snapshot -i\` — see interactive elements -- \`proofshot exec click @e3\` — click an element -- \`proofshot exec fill @e2 "text"\` — fill a form field -- \`proofshot exec screenshot step.png\` — capture a moment +1. Start a session: + \`proofshot start --run "your-dev-command" --port PORT --description "what you are verifying"\` + Use \`proofshot.config.json\` environment and log sources instead of \`--run\` when verification needs multiple processes, tmux panes, or file tails. +2. Drive the browser with \`proofshot exec\`: + - \`proofshot exec snapshot -i\` + - \`proofshot exec click @e3\` + - \`proofshot exec fill @e2 "text"\` + - \`proofshot exec screenshot step.png\` +3. Stop and bundle evidence: + \`proofshot stop\` + +Take screenshots before and after important actions. Read the browser snapshot and captured logs to verify the expected behavior, then fix and repeat if evidence contains errors. `; }