Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 8 additions & 0 deletions .github/workflows/publish.yml
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,14 @@ jobs:
node-version: "20.x"
cache: "npm"

# The environment-capture tests drive real tmux servers. They skip when the
# binary is absent locally, so CI installs it to keep that coverage running.
- name: Install tmux
run: |
sudo apt-get update
sudo apt-get install -y tmux
tmux -V

- name: Install dependencies
run: npm ci

Expand Down
17 changes: 15 additions & 2 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ npm test # Run vitest once
npm run dev # Watch mode build
```

**Optional test prerequisite: `tmux`.** The environment-capture integration tests in `src/environment/runtime.test.ts` drive real tmux servers. They skip themselves when `tmux` is not on `PATH`, except when `CI` is set — CI installs tmux explicitly, so a missing binary there is a failure, not a skip.

## Architecture

```
Expand All @@ -18,8 +20,10 @@ src/
├── commands/ # One file per CLI command (install, start, stop, exec, diff, pr, clean)
├── browser/ # agent-browser CLI wrappers (session, capture, interact, navigate)
├── server/ # Dev server detection, startup, port waiting
├── environment/ # Owned tmux panes, direct processes, file tails, evidence capture
├── session/state.ts # .session.json lifecycle (save/load/clear)
├── session/metadata.ts # Persistent per-session metadata (branch, commit) for PR matching
├── session/teardown.ts # Release the active session's owned environment
├── artifacts/ # Output generation (viewer.html, SUMMARY.md, PR format)
└── utils/ # Config, exec helpers, port utils, error patterns, GitHub API
```
Expand All @@ -32,15 +36,17 @@ src/
- **Build before test** — CLI runs from `dist/`, always `npm run build` after code changes
- **agent-browser** — external peer dependency (Rust CLI + Node daemon). All browser commands go through `ab()` in `utils/exec.ts` which calls `agent-browser <command>` via `execSync`
- **Session state** — `start` writes `.session.json`, `exec` and `stop` read it. `stop` clears it. Don't assume session exists without checking
- **Owned environment** — `.session.json` is the only record of the processes and tmux sockets a session owns. Any path that discards it must first call `releaseActiveSessionEnvironment` (`src/session/teardown.ts`) and must keep the file when release fails. Cleanup targets recorded immutable identities, never a reusable pid or socket name
- **Session metadata** — `start` writes `metadata.json` inside each session folder with git branch/commit. This persists after `stop` and is used by `pr` to match sessions to branches
- **Per-session subfolders** — artifacts go in `proofshot-artifacts/YYYY-MM-DD_HH-mm-ss_slug/`

## Command lifecycle

1. `proofshot start` — spawns dev server, opens browser, starts recording, saves session state + writes `metadata.json` with git branch/commit
1. `proofshot start` — starts configured environment/log capture, spawns dev server, opens browser, starts recording, saves session state + writes `metadata.json` with git branch/commit
2. `proofshot exec <args>` — logs action to `session-log.json`, forwards to `agent-browser`
3. `proofshot stop` — collects errors, stops recording, trims video, generates SUMMARY.md + viewer.html, clears session
3. `proofshot stop` — collects errors, stops recording, releases the owned environment, trims video, generates SUMMARY.md + viewer.html, clears session
4. `proofshot pr [number]` — finds sessions for current branch, uploads artifacts to GitHub, posts PR comment
5. `proofshot clean` — releases the active session's environment, then removes the output directory

## Adding a new command

Expand Down Expand Up @@ -70,6 +76,8 @@ Edit `src/utils/error-patterns.ts` — add a new entry to the `PATTERNS` array:
| `session.webm` | `start` | Video recording (Playwright screencast) |
| `session-log.json` | `exec` (appended each call) | Action timeline with relative timestamps |
| `server.log` | `start` (piped stdout+stderr) | All dev server output |
| `environment.ndjson` | `start` (capture workers) | Canonical timestamped evidence per configured environment source |
| `logs/*.log` | `start` (capture workers) | One bounded plain-text log per environment source |
| `console-output.log` | `stop` | Browser console output |
| `step-*.png` | `exec screenshot` | Screenshots at key moments |
| `SUMMARY.md` | `stop` | Markdown report with errors and screenshots |
Expand All @@ -92,4 +100,9 @@ Edit `src/utils/error-patterns.ts` — add a new entry to the `PATTERNS` array:
- `proofshot exec` has special shell quoting logic (`buildShellCommand` in exec.ts) — `eval` commands get single-quoted, args with special chars get auto-quoted
- Video trimming adjusts session-log.json timestamps to match the trimmed video (see `trimOffsetSec` in stop.ts)
- Server log capture only works when proofshot starts the server itself — if the port is already occupied, we skip spawning and get no server logs
- `--run` and `config.environment` are mutually exclusive (both start the app); `start` rejects the combination up front
- Config validation fails `start` closed, but `stop`/`clean` load config through `loadConfigForTeardown` and only warn — an invalid config must never strand owned resources
- Tmux panes are a single PTY stream, so pane evidence is always `stream: "pty"`; only direct processes keep `stdout`/`stderr` apart
- `connection.ownership: "attach"` outranks "own what you created" — an attach-only tmux server/session/pane is never terminated, so `launch.stopCommand` is rejected in that mode
- `stop` detects a mid-session capture gap two ways, because the workers fail in two shapes: a surviving pid file with a dead process means the helper was killed, while a tmux pane that exited closes its pipe as a clean EOF and is only visible through `#{pane_pipe}` — so the pane check must run before teardown detaches the pipes — and teardown clears `captureAttached` as it detaches, so a retry after a partial failure cannot mistake its own work for a gap. Either way `stop` records the gap and exits non-zero after finishing teardown
- The `consoleErrors`/`consoleOutput` from agent-browser are point-in-time snapshots collected at stop time
33 changes: 32 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,8 @@ Each session produces a timestamped folder in `./proofshot-artifacts/`:
| `session-log.json` | Action timeline with timestamps and element data |
| `server.log` | Dev server stdout/stderr (when using `--run`) |
| `console-output.log` | Browser console output |
| `environment.ndjson` | Timestamped canonical evidence from configured tmux, process, and file sources |
| `logs/*.log` | One bounded plain-text log per configured environment source |

<p align="center">
<img src="brand-assets/screenshots/artifacts-folder.png" alt="ProofShot artifacts folder" width="480" />
Expand Down Expand Up @@ -158,6 +160,30 @@ You can also configure browser launch behavior in `proofshot.config.json`:

Set `browser.configPath` when you need ProofShot to run `agent-browser` against a project-specific config instead of inheriting `~/.agent-browser/config.json`. Relative paths are resolved from the directory that contains `proofshot.config.json`.

For applications with multiple runners, configure ProofShot to own and capture tmux panes, direct processes, and file tails:

```json
{
"environment": {
"kind": "processes",
"commands": [
{ "id": "web", "group": "frontend", "command": "npm run dev" },
{ "id": "api", "group": "backend", "command": "npm run api" }
],
"readiness": [
{ "kind": "http", "url": "http://127.0.0.1:3000/health" }
]
},
"logs": {
"sources": [
{ "id": "worker", "kind": "file", "path": "./logs/worker.log" }
]
}
}
```

Direct process commands are captured automatically; declare `logs.sources` only for custom source identities or additional files. Use either `--run` or `environment`, not both. See the [configuration reference](content/docs/reference/configuration.mdx) for owned tmux sockets, external launcher contracts, source naming, and readiness checks.

### `proofshot stop`

Stop recording, collect errors, generate proof artifacts.
Expand Down Expand Up @@ -211,6 +237,8 @@ Remove the `./proofshot-artifacts/` directory.
proofshot clean
```

If a session is still active there, `clean` first releases the processes and tmux sockets it owns — and refuses to delete the directory if that fails, since `.session.json` is the only record of them.

### `proofshot doctor`

Print the current ProofShot environment, including config path, browser mode, viewport, installed binaries, and any active session.
Expand All @@ -226,13 +254,14 @@ proofshot doctor
| Agent | Install location |
|-------|-----------------|
| **Claude Code** | `~/.claude/skills/proofshot/SKILL.md` |
| **Cursor** | `~/.cursor/rules/proofshot.mdc` |
| **Cursor** | `~/.cursor/skills/proofshot/SKILL.md` |
| **Codex (OpenAI)** | `~/.codex/skills/proofshot/SKILL.md` |
| **OpenCode** | `~/.config/opencode/skills/proofshot/SKILL.md` |
| **Gemini CLI** | Appends to `~/.gemini/GEMINI.md` |
| **Windsurf** | Appends to `~/.codeium/windsurf/memories/global_rules.md` |

All skills install at **user level** — no per-project configuration needed.
When upgrading Cursor installations, ProofShot preserves the previous `proofshot.mdc` rule as a non-loading `.migrated` backup after the skill is written successfully.

## Try It

Expand Down Expand Up @@ -280,6 +309,8 @@ npm test # Run tests
npm run dev # Watch mode
```

Install `tmux` to run the environment-capture integration tests locally — they skip themselves when it is missing, and CI installs it so the coverage always runs there.

Three sample apps in `test/fixtures/` cover different UI patterns for end-to-end testing: a SaaS dashboard (`sample-app`), a kanban board (`todo-app`), and a chat interface (`chat-app`).

Built on [agent-browser](https://github.com/vercel-labs/agent-browser) by Vercel.
Expand Down
7 changes: 6 additions & 1 deletion bin/proofshot.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,9 @@
import chalk from 'chalk';
import { createCLI } from '../src/cli.js';
import { formatErrorDetail } from '../src/utils/errors.js';

const program = createCLI();
program.parse();
program.parseAsync().catch((error) => {
console.error(chalk.red('✗') + ` ${formatErrorDetail(error)}`);
process.exit(1);
});
24 changes: 15 additions & 9 deletions content/docs/concepts/how-it-works.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,14 +45,17 @@ ProofShot uses a three-phase model.
`proofshot start` initializes the session:

1. Check if the port is available (fail fast on conflicts)
2. Spawn the dev server if `--run` is provided, pipe output to `server.log`
3. Wait for the port to respond (polls every 500ms, 30s timeout)
4. Open headless Chromium
5. Start video recording
6. Write `.session.json` (active session state) and `metadata.json` (git branch/commit, persists after stop)
2. Start the configured `environment` and `logs` sources — tmux panes, direct processes, and file tails — and wait for their readiness checks
3. Spawn the dev server if `--run` is provided, pipe output to `server.log`
4. Wait for the port to respond (polls every 500ms, 30s timeout)
5. Open headless Chromium
6. Start video recording
7. Write `.session.json` (active session state) and `metadata.json` (git branch/commit, persists after stop)

Recording is mandatory. If it fails after 3 retries, the session aborts.

`--run` and `environment` are two ways to start the same app, so ProofShot rejects using both. `.session.json` is written before the environment starts and updated as each resource is claimed, so a crashed start still leaves the owned processes and sockets recoverable.

### Phase 2: Exec (repeated)

Each `proofshot exec` call:
Expand All @@ -70,10 +73,13 @@ Each `proofshot exec` call:
1. Collects browser console errors and output (point-in-time snapshot)
2. Stops video recording
3. Closes the browser
4. Trims video dead time using ffmpeg (5s buffer before first action, 3s after last). Adjusts all `session-log.json` timestamps by the trim offset.
5. Scans `server.log` with multi-language regex patterns for errors
6. Generates `SUMMARY.md` and `viewer.html`
7. Clears `.session.json`
4. Stops environment capture and releases every process and tmux socket the session owns, by recorded identity
5. Trims video dead time using ffmpeg (5s buffer before first action, 3s after last). Adjusts all `session-log.json` timestamps by the trim offset.
6. Scans `server.log` with multi-language regex patterns for errors
7. Generates `SUMMARY.md` and `viewer.html`
8. Clears `.session.json`

Environment teardown is the one step that is not best-effort: if a resource cannot be released, `stop` fails and keeps `.session.json` so nothing is silently orphaned.

## Design principles

Expand Down
2 changes: 1 addition & 1 deletion content/docs/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ When ffmpeg is available, `proofshot stop` cuts dead time from the video — kee
You need to run `proofshot start` first. Each session writes `.session.json` — if it's missing, there's no active session to operate on.

**Server errors aren't being detected**
Server log capture only works when ProofShot starts the server itself via `--run`. If your server was already running on the port, ProofShot skips spawning and gets no logs.
`server.log` capture only works when ProofShot starts the server itself via `--run`. If your server was already running on the port, ProofShot skips spawning and gets no `server.log`. To capture an app you start yourself — or more than one process — declare `environment` and `logs.sources` in [the config](/docs/reference/configuration): ProofShot can attach to an existing tmux session or tail a log file. Those sources are recorded as evidence in `environment.ndjson` and `logs/`; the multi-language error scan still runs only on `server.log`.

**The browser window doesn't appear**
ProofShot runs headless by default. Use `--headed` to see the browser window: `proofshot start --headed`.
Expand Down
10 changes: 1 addition & 9 deletions content/docs/guides/configure.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -41,15 +41,7 @@ proofshot start --port 8080 --output ./my-artifacts --headed

## All configuration options

| Option | Type | Default | Description |
|--------|------|---------|-------------|
| `devServer.port` | `number` | `3000` | Port your dev server runs on |
| `devServer.startupTimeout` | `number` | `30000` | Max wait time (ms) for the dev server to start |
| `output` | `string` | `./proofshot-artifacts` | Directory for session artifacts |
| `defaultPages` | `string[]` | `["/"]` | Pages to open by default |
| `viewport.width` | `number` | `1280` | Browser viewport width in pixels |
| `viewport.height` | `number` | `720` | Browser viewport height in pixels |
| `headless` | `boolean` | `true` | Run the browser without a visible window |
Every option, type, and default lives in the [configuration reference](/docs/reference/configuration) — including the `browser`, `environment`, and `logs` blocks for multi-process apps, tmux panes, and file tails.

<Callout type="info">
You don't need a config file for most projects. The defaults work for standard setups, and CLI flags cover one-off overrides.
Expand Down
4 changes: 3 additions & 1 deletion content/docs/guides/install-skills.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -51,13 +51,15 @@ All skills install at **user level** — one install works across every project
| Agent | Location | Strategy |
|-------|----------|----------|
| **Claude Code** | `~/.claude/skills/proofshot/SKILL.md` | Standalone file |
| **Cursor** | `~/.cursor/rules/proofshot.mdc` | Standalone file |
| **Cursor** | `~/.cursor/skills/proofshot/SKILL.md` | Standalone skill |
| **Codex** | `~/.codex/skills/proofshot/SKILL.md` | Standalone file |
| **Gemini CLI** | `~/.gemini/GEMINI.md` | Appended with markers |
| **Windsurf** | `~/.codeium/windsurf/memories/global_rules.md` | Appended with markers |

Tools using the **append** strategy wrap ProofShot's content in `<!-- proofshot:start -->` / `<!-- proofshot:end -->` markers. Running `proofshot install` again replaces only the marked section — your other content is preserved.

When upgrading an existing Cursor installation, ProofShot writes the new skill first and then renames `~/.cursor/rules/proofshot.mdc` to a non-loading `.migrated` backup. Custom rule edits are preserved, and a failed skill write leaves the legacy rule active.

## What the skill teaches the agent

The skill file gives your agent:
Expand Down
2 changes: 1 addition & 1 deletion content/docs/guides/verify-feature.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ You need to run `proofshot start` first. The session state is stored in `.sessio
Install [ffmpeg](https://ffmpeg.org/) and ProofShot will automatically trim dead time from the video (5s buffer before the first action, 3s buffer after the last).

**No server errors detected but I know there are errors**
Server error detection only works when ProofShot starts the server itself (`--run`). If your server was already running, ProofShot can't capture its logs.
The error scan reads `server.log`, which only exists when ProofShot starts the server itself (`--run`). To capture an app you start yourself, or more than one process, see [the FAQ](/docs/faq) and the [configuration reference](/docs/reference/configuration).

## What's next?

Expand Down
2 changes: 1 addition & 1 deletion content/docs/quick-start.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -53,7 +53,7 @@ Detected tools:

Installing skills...
✓ Claude Code → ~/.claude/skills/proofshot/SKILL.md
✓ Cursor → ~/.cursor/rules/proofshot.mdc
✓ Cursor → ~/.cursor/skills/proofshot/SKILL.md

Done! Your AI agent now knows how to use ProofShot.
```
Expand Down
Loading