Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
21 commits
Select commit Hold shift + click to select a range
19573c9
fix: isolate ProofShot session lifecycle
justinTM Jul 28, 2026
f71514b
fix: close isolated session review gaps
justinTM Jul 28, 2026
f5ceeba
fix(lifecycle): make isolated process ownership portable
alvarohulse Aug 10, 2026
edf9dec
fix(artifacts): preserve valid recorded evidence
alvarohulse Aug 10, 2026
02d12e8
fix(packaging): support Git and tarball installs
alvarohulse Aug 10, 2026
b2a0703
fix(lifecycle): retain exact recovery ownership
alvarohulse Aug 10, 2026
a4af87d
feat(environment): capture owned multi-source logs
alvarohulse Aug 10, 2026
fd95da3
feat(evidence): unify viewer integrity and verdicts
alvarohulse Aug 10, 2026
3adc134
feat(publication): enforce artifact provenance
alvarohulse Aug 10, 2026
b424156
docs(reliability): explain owned evidence workflows
alvarohulse Aug 10, 2026
b21d593
fix(environment): preserve external pane consumers
alvarohulse Aug 10, 2026
689f868
fix(portability): stabilize Linux browser fixtures
alvarohulse Aug 10, 2026
99bf50d
fix(portability): prefer agent-browser Chromium on Linux
alvarohulse Aug 10, 2026
64df1f2
fix(browser): retain slow-loading target pages
alvarohulse Aug 10, 2026
a3f4191
fix(evidence): align canonical timeline with recording
alvarohulse Aug 10, 2026
f28529e
fix(browser): quote navigation URLs safely
alvarohulse Aug 10, 2026
4b70245
fix(lifecycle): preserve exact recovery ownership
alvarohulse Aug 10, 2026
802e9e2
fix(reliability): harden evidence lifecycle and publication
alvarohulse Aug 10, 2026
de9f5ba
feat(publication): complete explicit evidence shipping
alvarohulse Aug 10, 2026
c04ab8b
fix(evidence): preserve actions in trimmed media
alvarohulse Aug 10, 2026
1eda32f
fix(publication): upload proof to the PR head repository
alvarohulse Aug 10, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
49 changes: 49 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,49 @@
name: CI

on:
pull_request:

permissions:
contents: read

concurrency:
group: ci-${{ github.event.pull_request.number }}
cancel-in-progress: true

jobs:
verify:
name: Build and Test
runs-on: ubuntu-latest
timeout-minutes: 15

steps:
- name: Checkout code
uses: actions/checkout@v4

- name: Setup Node.js
uses: actions/setup-node@v4
with:
node-version: "20.x"
cache: npm

- name: Install system test dependencies
run: sudo apt-get update && sudo apt-get install --yes ffmpeg tmux

- name: Install dependencies
run: npm ci

- name: Build
run: npm run build

- name: Verify build artifacts
run: |
test -f dist/bin/proofshot.js
test -f dist/src/index.js
node dist/bin/proofshot.js --help
npm pack --dry-run

- name: Run tests
run: npm test

- name: Run FFmpeg integration test
run: npm run test:ffmpeg
3 changes: 3 additions & 0 deletions PROOFSHOT.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,13 +11,16 @@ After building or modifying UI features, verify with this workflow:

ProofShot keeps all `proofshot exec` commands inside the same isolated `agent-browser` session that was created by `proofshot start`, so recording, screenshots, and browser actions stay aligned.

Use `--url` on `start` when verification must begin on a specific target. In an isolated HOME, ProofShot discovers executable-only Chrome/Chromium installs from system/account locations; use `--browser-executable /absolute/path/to/chrome` to select one explicitly.

Key proofshot exec commands:
- `proofshot exec snapshot -i` — see interactive elements
- `proofshot exec click @e3` — click an element
- `proofshot exec fill @e2 "text"` — fill a form field
- `proofshot exec screenshot step.png` — capture a moment

Artifacts saved to ./proofshot-artifacts/ including video, screenshots, errors, and summary.
Custom `--output` paths do not move active control state, so a separate `proofshot stop` still finds the session. `stop` is idempotent; after `stop --no-close`, run a later plain `stop` to close that exact retained browser without rebundling.
You can customize browser launch behavior in `proofshot.config.json`, including HTTPS error ignoring, a custom browser executable path, and a project-specific `agent-browser` config path.

Use `proofshot doctor` when the local setup looks wrong. It prints the current config path, browser mode, viewport, installed binaries, and any active ProofShot session.
48 changes: 39 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -78,11 +78,12 @@ Three-step workflow: **start**, **test**, **stop**.
proofshot start --run "npm run dev" --port 3000 --description "Login form verification"

# 2. Test — the AI agent drives the browser
agent-browser snapshot -i # See interactive elements
agent-browser open http://localhost:3000/login # Navigate
agent-browser fill @e2 "test@example.com" # Fill form
agent-browser click @e5 # Click submit
agent-browser screenshot ./proofshot-artifacts/step-login.png # Capture proof
proofshot exec snapshot -i # See interactive elements
proofshot exec open http://localhost:3000/login # Navigate
proofshot exec fill @e2 "test@example.com" # Fill form
proofshot exec click @e5 # Click submit
proofshot exec assert-visible "#account-home" # Record an expected selector
proofshot exec screenshot step-login.png # Capture proof

# 3. Stop — bundle video + screenshots + errors into proof artifacts
proofshot stop
Expand All @@ -97,20 +98,23 @@ Each session produces a timestamped folder in `./proofshot-artifacts/`:
| File | Description |
|------|-------------|
| `session.webm` | Video recording of the entire session |
| `viewer.html` | Standalone interactive viewer with scrub bar, timeline, and Console/Server log tabs |
| `viewer.html` | Standalone interactive viewer with scrub bar, canonical timeline, and grouped Environment/Browser source tabs |
| `SUMMARY.md` | Markdown report with errors, screenshots, and video |
| `step-*.png` | Screenshots captured at key moments |
| `session-log.json` | Action timeline with timestamps and element data |
| `server.log` | Dev server stdout/stderr (when using `--run`) |
| `console-output.log` | Browser console output |
| `evidence.json` | Canonical browser/environment events, incidents, source integrity, and media timing |
| `verdict.json` | Structured `PASS`, `FAIL`, `INCOMPLETE`, or `BLOCKED` verdict |
| `artifact-manifest.json` | Finalized repository/commit provenance and ordered artifact hashes |

<p align="center">
<img src="brand-assets/screenshots/artifacts-folder.png" alt="ProofShot artifacts folder" width="480" />
<br>
<em>Generated artifacts for a single verification session</em>
</p>

The viewer also includes tabs for browsing console and server logs, with error highlighting and timestamps synced to the video:
The viewer includes grouped Environment and Browser evidence tabs, with incident highlighting and timed live rows synchronized to the video:

<p align="center">
<img src="brand-assets/screenshots/viewer-console.png" alt="ProofShot Viewer — console logs tab" width="100%" />
Expand Down Expand Up @@ -140,6 +144,7 @@ proofshot start # Server already running
proofshot start --run "npm run dev" --port 3000 # Start and capture server
proofshot start --description "Verify checkout flow" # Add description to report
proofshot start --url http://localhost:3000/login # Open specific URL
proofshot start --browser-executable /path/to/chrome # Reuse an exact browser binary
proofshot start --headed # Show browser (debugging)
proofshot start --force # Override a stale session from a previous crash
```
Expand All @@ -158,6 +163,10 @@ You can also configure browser launch behavior in `proofshot.config.json`:

Set `browser.configPath` when you need ProofShot to run `agent-browser` against a project-specific config instead of inheriting `~/.agent-browser/config.json`. Relative paths are resolved from the directory that contains `proofshot.config.json`.

ProofShot discovers system and account-level Chrome/Chromium installs even when the command runs with an isolated `HOME`. If no runnable browser is found, `start` prints the exact `agent-browser install` action. An explicit `--browser-executable` takes precedence for one run.

`--output` changes only where evidence is written. Active control state stays in the configured/default output directory, so later `proofshot exec` and `proofshot stop` processes can find the same session.

### `proofshot stop`

Stop recording, collect errors, generate proof artifacts.
Expand All @@ -167,6 +176,8 @@ proofshot stop # Stop session and close browser
proofshot stop --no-close # Stop but keep browser open
```

`stop` is idempotent. With `--no-close`, ProofShot retains exact ownership metadata after bundling; run a later plain `proofshot stop` to close that browser without rebuilding the artifacts.

### `proofshot exec`

Pass-through to agent-browser with automatic session logging. Captures timestamps, element data, and resolves screenshot paths.
Expand All @@ -175,9 +186,12 @@ When a ProofShot session is active, `proofshot exec` reuses the same isolated `a

```bash
proofshot exec click @e3
proofshot exec assert-visible "#checkout-complete"
proofshot exec screenshot step-checkout.png
```

Failed `assert-visible` checks are recorded in `session-log.json` and contribute to the structured verdict.

### `proofshot diff`

Compare current screenshots against a baseline for visual regression.
Expand All @@ -188,20 +202,34 @@ proofshot diff --baseline ./previous-artifacts

### `proofshot pr`

Upload session artifacts to GitHub and post a verification comment on the PR. Finds all sessions recorded on the current branch, uploads screenshots and video, and posts a formatted comment with embedded screenshots.
Upload one finalized, provenance-compatible session to GitHub and post a verification comment. ProofShot validates the target PR head, source state, artifact paths, and hashes before upload; it never combines historical sessions.

```bash
proofshot pr # Auto-detect PR from current branch
proofshot pr 42 # Target a specific PR
proofshot pr --session proofshot-2026-08-09_19-00-00
proofshot pr --session checkout-session --session receipt-session
proofshot pr --session proofshot-2026-08-09_19-00-00 --screenshot checkout.png --screenshot receipt.png
proofshot pr --dry-run # Preview the markdown without posting
proofshot pr --upload-provider github-web-attachments # Use GitHub's internal attachment flow
```

By default, ProofShot uses the official GitHub repository contents API and uploads artifacts to a dedicated `proofshot-artifacts` branch. This works with normal `gh` authentication and `GH_TOKEN`.

Auto-selection succeeds only when exactly one complete `PASS` or `FAIL` session matches the PR head. Repeat `--session` to publish several explicit compatible sessions, and repeat `--screenshot` to preserve exact artifact selection order. Pre-manifest sessions require both `--session` and `--legacy-session`; that opt-in cannot bypass a present or invalid finalized manifest. A partial upload never posts a PR comment.

The `github-web-attachments` provider is still available for inline GitHub-hosted media, but it relies on GitHub's internal web upload endpoint and may reject browser-based `gh auth login` OAuth sessions.

Converts `.webm` video to `.mp4` if `ffmpeg` is available.
### `proofshot session`

Inspect and clean durable recovery records after an interrupted or incomplete cleanup:

```bash
proofshot session list
proofshot session clean --session <session-id>
```

Cleanup validates persisted process identities and never widens to a name-, port-, or default-socket kill.

### `proofshot clean`

Expand All @@ -211,6 +239,8 @@ Remove the `./proofshot-artifacts/` directory.
proofshot clean
```

`clean` refuses while active or retained session control state exists. Run `proofshot stop` first so ProofShot does not discard exact process ownership metadata.

### `proofshot doctor`

Print the current ProofShot environment, including config path, browser mode, viewport, installed binaries, and any active session.
Expand Down
19 changes: 10 additions & 9 deletions content/docs/concepts/how-it-works.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -45,11 +45,11 @@ ProofShot uses a three-phase model.
`proofshot start` initializes the session:

1. Check if the port is available (fail fast on conflicts)
2. Spawn the dev server if `--run` is provided, pipe output to `server.log`
2. Spawn an isolated dev-server process session if `--run` is provided, pipe timestamped output to `server.log`, and persist its immutable PID/process-group identity
3. Wait for the port to respond (polls every 500ms, 30s timeout)
4. Open headless Chromium
4. Open the requested URL in a short, collision-safe agent-browser session and persist its daemon identity
5. Start video recording
6. Write `.session.json` (active session state) and `metadata.json` (git branch/commit, persists after stop)
6. Write `.session.json` to the configured/default control directory and `metadata.json` beside the evidence (git branch/commit, persists after stop)

Recording is mandatory. If it fails after 3 retries, the session aborts.

Expand All @@ -69,18 +69,19 @@ Each `proofshot exec` call:

1. Collects browser console errors and output (point-in-time snapshot)
2. Stops video recording
3. Closes the browser
4. Trims video dead time using ffmpeg (5s buffer before first action, 3s after last). Adjusts all `session-log.json` timestamps by the trim offset.
5. Scans `server.log` with multi-language regex patterns for errors
6. Generates `SUMMARY.md` and `viewer.html`
7. Clears `.session.json`
3. Closes the exact owned browser session
4. Stops only the dev-server process session created by this run
5. Trims video dead time using ffmpeg (5s buffer before first action, 3s after last). Adjusts all `session-log.json` timestamps by the trim offset.
6. Scans `server.log` with multi-language regex patterns for errors
7. Generates `SUMMARY.md` and `viewer.html`
8. Clears `.session.json` (or retains exact browser ownership after `--no-close`)

## Design principles

**Graceful degradation.** Missing ffmpeg? Video trimming is skipped. Element data capture fails? Overlays are skipped. Browser already closed? Silent catch. Non-critical failures never abort a session.

**Minimal dependencies.** Three production dependencies: `commander`, `chalk`, `detect-port`. agent-browser is an optional peer dependency. Small install, small supply chain.

**Session isolation.** `.session.json` lives in the output directory, not globally. This supports parallel sessions in different projects.
**Session isolation.** Each project keeps control state in its configured/default output directory. A CLI-only custom evidence path cannot hide the session from a later process, while different projects still run independently.

**ESM-only.** All imports use explicit `.js` extensions for correct resolution after TypeScript compilation.
10 changes: 7 additions & 3 deletions content/docs/concepts/interactive-viewer.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The `viewer.html` file generated by `proofshot stop` is a self-contained HTML vi

```
┌─────────────────────────────────────────────────────┐
│ Header: description, console/server error badges │
│ Header: verdict and canonical evidence badges │
├──────────────────────────┬──────────────────────────┤
│ │ │
│ Video panel (62%) │ Timeline panel (38%) │
Expand Down Expand Up @@ -42,9 +42,13 @@ Overlays are scaled from the original viewport size to the current video display

**Keyboard navigation.** Left and right arrow keys jump between action markers on the scrub bar.

**Error badges.** Top-right corner displays console and server error counts. Green means clean, red means errors were found.
**Evidence groups.** Environment events have a merged tab followed by grouped source tabs such as **Frontend · Vite** and **Backend · API**. Browser console/errors stay in a separate Browser group segmented by detected navigation URL.

**Sync.** Playing the video highlights the current action in the timeline and auto-scrolls it into view. The two panels stay in lockstep.
**Integrity badges.** Each source reports presentation-hidden lines, truncation, capture gaps, and grouped incidents. These values and the top-level `PASS`/`FAIL`/`INCOMPLETE`/`BLOCKED` badge come from the same canonical evidence rows.

**History/live boundaries.** Tmux scrollback appears first as untimed history; live PTY rows then synchronize to the timeline. The boundary identifies deduplicated overlap or a possible capture gap.

**Sync.** Playing the video highlights current actions and timed evidence rows. Untimed or non-finite rows cannot seek. If the action timeline extends beyond the recording, the viewer warns instead of shortening the timeline and clamps seeks to available media.

**Responsive.** On smaller screens, the video and timeline stack vertically.

Expand Down
9 changes: 6 additions & 3 deletions content/docs/faq.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -25,10 +25,10 @@ No. ffmpeg is optional. Without it, ProofShot skips video trimming (you get the
The skill file installed by `proofshot install` teaches your agent the three-step workflow, all browser commands, and when to take screenshots. You just prompt naturally — "verify this feature with proofshot" — and the agent handles the rest.

**What are `@eN` references?**
Stable handles to interactive elements on a page. When your agent runs `agent-browser snapshot -i`, it gets a list like `@e1: button "Submit"`, `@e2: input "Email"`. These references persist across commands within a session, so the agent can target elements reliably without CSS selectors.
Stable handles to interactive elements on a page. When your agent runs `proofshot exec snapshot -i`, it gets a list like `@e1: button "Submit"`, `@e2: input "Email"`. These references persist across commands within a session, so the agent can target elements reliably without CSS selectors.

**Can I run multiple sessions at the same time?**
Yes. Session state (`.session.json`) lives in the output directory, not globally. Different projects with different output directories can run sessions concurrently.
Yes. Session state (`.session.json`) lives in each project's configured/default output directory, not globally. Different projects can run concurrently. A one-run `--output` override moves evidence without changing where that project finds active control state.

**What languages does error detection support?**
JavaScript/Node.js, Python, Ruby/Rails, Go, Java/Kotlin, Rust, PHP, C#/.NET, Elixir/Phoenix, plus generic patterns for `FATAL`, `CRITICAL`, and segfaults. See [How to add error patterns](/docs/guides/add-error-pattern) to extend support.
Expand All @@ -39,7 +39,10 @@ When ffmpeg is available, `proofshot stop` cuts dead time from the video — kee
## Troubleshooting

**"No active session" when running exec or stop**
You need to run `proofshot start` first. Each session writes `.session.json` — if it's missing, there's no active session to operate on.
You need to run `proofshot start` before `exec`. `stop` is idempotent, so it succeeds without changing artifacts when the session is already stopped.

**Chrome is installed, but an isolated HOME cannot find it**
ProofShot checks system paths and executable-only browser caches under the real account home without reusing a browser profile or storage. You can also pass one exact path with `proofshot start --browser-executable /absolute/path/to/chrome`. If nothing is runnable, run the `agent-browser install` command printed by `proofshot start`.

**Server errors aren't being detected**
Server log capture only works when ProofShot starts the server itself via `--run`. If your server was already running on the port, ProofShot skips spawning and gets no logs.
Expand Down
Loading