Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
5053538
Check every webview token the agent-browser host puts on the command …
nedtwigg Sep 24, 2026
72b5e04
Debounce sync-to-pane through the pane observer, not the window resize
nedtwigg Sep 24, 2026
47f8617
Paint the stream provisionally after keyboard input, not only pointer…
nedtwigg Sep 24, 2026
c26f119
Paint the loading page while a crisp capture waits behind a blocking …
nedtwigg Sep 24, 2026
bcb4e4e
Keep the proxied HTML body's own charset and encoding when instrumenting
nedtwigg Sep 24, 2026
396c47e
Make iframe failures visible and one click from agent-browser
nedtwigg Sep 24, 2026
afd4d8e
Point dor-cli's addressing note at the host's argv shapes
nedtwigg Sep 24, 2026
14bd4b5
Tell agents which browser command to use, and name the fix in every r…
nedtwigg Sep 24, 2026
332c262
Create no agent-browser new-tab pane on a host that cannot launch one
nedtwigg Sep 24, 2026
4c9a8b6
Keep the webview argv check a plain boolean
nedtwigg Sep 24, 2026
063ba84
Word the iframe panel-error rule plainly
nedtwigg Sep 24, 2026
e00d689
Parse webview browser commands once, for both provider hosts
nedtwigg Sep 24, 2026
829c2be
Consolidate the iframe refusal, launch binding, and capture paths
nedtwigg Sep 24, 2026
24eb100
Address round-1 review: URL-bar refusal, iframe entry table, capture …
nedtwigg Sep 24, 2026
d1d3fcb
Bound the agent-browser capture spawn with the new spawn timeout
nedtwigg Sep 24, 2026
40ad49a
Address round-2 review: one URL for the iframe swap, load-tagged repo…
nedtwigg Sep 24, 2026
e417650
State that the Display modal and the iframe swap judge the same URL
nedtwigg Sep 24, 2026
dd92ba8
Keep the Display modal story's chrome snapshot stable
nedtwigg Sep 24, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
110 changes: 82 additions & 28 deletions docs/specs/dor-browser.md

Large diffs are not rendered by default.

16 changes: 14 additions & 2 deletions docs/specs/dor-browser.rationale.md
Original file line number Diff line number Diff line change
Expand Up @@ -42,10 +42,14 @@ The persisted `wsPort` mirror can lag the controller's already-live port after a

**What parking is worth.** Lath leaves stay mounted, so a background window would otherwise retain every pane's ~20Hz decode and screenshot round trips. The ~1s debounce rides through transient visibility flips and StrictMode remounts without rebuilding the connection.

**What the two-stage split buys.** Three things at once: pointer feedback that does not wait on a screenshot child-process round trip, a resting image sharp on HiDPI, and an idle animated page that does not pay to decode the stream continuously. Either path alone gives up one of the three.
**What the two-stage split buys.** Three things at once: input feedback that does not wait on a screenshot child-process round trip, a resting image sharp on HiDPI, and an idle animated page that does not pay to decode the stream continuously. Either path alone gives up one of the three.

**Why keys open the window too.** A keystroke's echo is the most latency-sensitive paint there is; outside the window it waited a whole crisp capture — ~120ms, plus up to ~180ms of loop pacing in a burst — while a hover repainted from the stream in ~50ms. On HiDPI the typed text is CSS-resolution until 250ms after the last key, then sharpens, as hover already did.

**Why no crisp capture starts inside the provisional window.** A host screenshot round trip is ~120ms against a ~20Hz stream, so *every* capture started while provisional frames are still landing is superseded before it resolves; the shots skipped would never have drawn anything.

**Why an overdue capture paints the stream and is never re-issued.** `open` holds the daemon's queue until the page loads, up to 25s (see [Pop-Out](#pop-out)), and the URL bar, `dor ab open` and every relaunch run it. A capture issued meanwhile waits behind it, so the canvas stayed on the previous page for the whole load while the stream was already showing the new one. An 8s watchdog then freed the slot and spawned another `screenshot` into the same queue — about three blocked CLI processes by 25s — and on VS Code the first reply's unlink could delete the second capture's file. VS Code's adapter gave up on a reply after 10s and the webview re-asked the same way, posting the full JPEG to both requests once it came; every adapter now waits 30s, and the host still joins concurrent captures, since surfaces can share a session. When each overdue paint counted against the held capture, it was discarded on arrival and re-taken — an extra full capture on every slow navigation, the crisp frame a round trip late. Its wait's pulses still owe one follow-up: a capture queued behind `open` is taken at the end of its round trip, but one slow in itself may have been taken before the page's last change. The overdue round trip timed the page load, not a capture, so the loop clamps it before it enters the pacing average; otherwise the next slow load would wait ~16s to count as overdue.

**Why a stale-dropped capture must leave the loop dirty.** A provisional frame can supersede the host capture or its pending bitmap decode. Nothing is guaranteed to pulse the loop again: a single pointer move over a static page pulses exactly once, and that one pulse is consumed by the very capture the provisional paint supersedes — leaving the pane on the blurry provisional frame until the page happens to change on its own.

**Why every non-crisp painter must bump the draw generation.** The byte-dedup compares an incoming capture against the last crisp draw. A resting page whose crisp bytes match that draw dedups to a no-op and strands the pane on the blurry provisional frame; a freshly re-attached canvas mounts blank and has the same problem.
Expand Down Expand Up @@ -74,7 +78,9 @@ A post-open blank-tab sweep can become such a query when a later relaunch, expli

**Why standalone passes a screenshot path, not bytes.** The sidecar stdio is a JSON-lines pipe shared with PTY traffic; a base64 frame on it would bloat every capture and interleave with terminal output.

**Why `binaryPath` needs a gate of its own.** The subcommand allowlist covers arguments, not the executable: `streamStatus`, `open` and `popOut` supply their own args and each take a `binaryPath`, so an allowlist on subcommands never sees one. And the value is persisted into the pane's params, so an unchecked one is not a one-shot — it is arbitrary local execution in the extension host or the Tauri sidecar on every subsequent launch. Dropping rather than failing degrades a stale or hostile value to "resolve it yourself".
**Why the verb alone is no boundary.** agent-browser honors launch options after the verb: `agent-browser --session x open about:blank --executable-path /nonexistent` fails with `Failed to launch Chrome at "/nonexistent"` (checked against 0.31.1, 2026-09-23). A verb-only allowlist therefore let an allowed `open`, `back` or `tab` carry `--executable-path`, `--args`, `--extension`, `--init-script`, `--profile`, `--state` or `--proxy` past the `binaryPath` gate; it also passed `close --all` (every session), `tab new <url>`, and `screenshot <path>`, which writes an image over any file the user can write. A session name becomes `<socket dir>/<session>.pid`, whose pid a relaunch SIGTERMs, so a `/` in it reaches outside that directory. The two hosts first parsed the same argv separately and drifted within a day: agent-browser took any URL scheme and any DPR, Playwright http(s) and DPR ≤ 10 — so one parser serves both.

**Why `binaryPath` needs a gate of its own.** The argv check covers arguments, not the executable: `streamStatus`, `open` and `popOut` supply their own args and each take a `binaryPath`, so a check on `command`'s argv never sees one. And the value is persisted into the pane's params, so an unchecked one is not a one-shot — it is arbitrary local execution in the extension host or the Tauri sidecar on every subsequent launch. Dropping rather than failing degrades a stale or hostile value to "resolve it yourself".

**Why the screenshot path is private.** The frame is a picture of the user's authenticated browser, written by an external process under the ambient umask, so a derivable name in the shared temp directory is readable by anything else on the machine for as long as it exists. Precedent: `standalone/sidecar/clipboard-ops.js` applies the same discipline, cleanup included, to clipboard images.

Expand All @@ -92,10 +98,16 @@ A post-open blank-tab sweep can become such a query when a later relaunch, expli

The built-in local-file viewer supplies its own content boundary and permits the inline shim, so removing its CSP would expand active documents' resource access. Its response opts into preservation without new renderer or host-bridge state. The proxy adds an independent ancestor policy: CSP policies intersect, so no directive parser or partial reconstruction can accidentally weaken the upstream. An opt-in upstream with stricter framing or script restrictions keeps those restrictions even if the shim cannot run.

**Why a UTF-16 body is recognized by its BOM too.** The browser's BOM sniff wins over any header, so a `text/html` body with no charset but `FF FE` is UTF-16; its latin1 scan finds no markers (every character is followed by a NUL), and the fallback spliced the shim in ahead of the BOM, turning the page to mojibake. A UTF-8 BOM ahead of the fallback position had the same fate.

**Why the HTML path keeps the body's own encoding.** Each was reproduced against the proxy (2026-09-23): relabelling every HTML response `charset=utf-8` overrode both the upstream header and any `<meta charset>`, so a Shift_JIS or windows-1252 page mis-decoded; an upstream that compresses without being asked had the shim prepended to its gzip bytes with `content-encoding: gzip` kept, and the frame failed with `ERR_CONTENT_DECODING_FAILED`; and a valid document with neither `</head>` nor `<body>` got the shim before `<!doctype html>`, switching it to quirks mode. Deleting `Accept-Encoding` on every request sent a remote upstream's scripts and styles uncompressed, typically 3-5x the bytes.

**Why a grant gets its own origin instead of a path token.** A dedicated origin keeps root-relative resources and client-side routers working with no body URL rewriting; a path token would have to survive every link, redirect and `fetch` the page makes.

## Iframe Shim

**Why the uninstrumented check waits for a first report.** The proxy instruments `text/html` only, and the parent cannot read a cross-origin frame's content type. A frame judged from its first load flagged every working non-HTML page — `dor iframe …/health.json`, and every image or PDF the file-viewer Tool frames directly. Waiting for one report means the frame has shown it carries the shim, so a later silent load is a real change; a link from an instrumented page to a PDF still flags, which the banner's wording ("not HTML, …") admits.

**Why the CLI's own check is not enough.** `open-window` carries a string the framed page chose, and the new-tab prompt in front of it is user consent, not a boundary — the user is agreeing to open a pane, not vetting a scheme. The same check gates `surface.iframe`, a wire protocol on the control socket rather than the CLI, so nothing upstream of it has already filtered.

**Why the panel checks again.** Every writer of `params.url` ends at the panel, and on a host with no proxy the raw fallback hands that string straight to `<iframe src>` under a sandbox that keeps `allow-same-origin`. Enumerating the writers is the fragile half: the header's URL editor was one the guarded callers did not cover, because `normalizeNavUrl` deliberately keeps a typed `javascript:` or `data:` scheme so the address bar can carry one. React blanks a `javascript:` `src` prop and nothing else, so `data:text/html,…` framed verbatim (reproduced in `IframePanel.test.tsx`, 2026-09) — a framework mitigation the code never claimed, for one scheme out of the set.
Expand Down
21 changes: 13 additions & 8 deletions docs/specs/dor-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -582,8 +582,9 @@ terminal verbs use (`dor read surface:3`).
`--surface` is the third of the mutually exclusive identity flags
(`docs/specs/dor-browser.md` → Managed identity); any two of the three fail
(`--key and --surface are mutually exclusive`). It changes *addressing* only:
every other argument is still forwarded verbatim, and the host-side subcommand
allowlist is untouched.
every other argument is still forwarded verbatim, and the host's webview argv
shapes are untouched (`docs/specs/dor-browser.md` → Agent-Browser Host
Capabilities).

**Resolution is host-side**, mirroring `surface.resolveOpen`: the CLI sends the
handle to `surface.resolveAgentBrowser` and forwards the session it gets back.
Expand All @@ -597,8 +598,11 @@ and the host applies two gates in order:
with no browser fails with the shared capability wording under [`dor
list`](#current-implemented-commands).
- **Render-mode-gated.** Past that gate, a browser Surface on the `iframe`
renderer is a browser with nothing to drive: `surface 'surface:2' is not
agent-browser rendered (render_mode: iframe)`.
renderer is a browser with nothing to drive, and one the other provider
renders is driven by the other CLI. **The refusal must name the command that
works**: `surface 'surface:2' is not agent-browser rendered (render_mode:
pw-screencast) — drive it with dor pw --surface surface:2`, or for an iframe
`… open its page with dor ab open <its url>`.

Neither gate covers an agent-browser Surface the context menu created eagerly,
whose daemon boot has not yet named it ([dor-browser.md](dor-browser.md) → Pane
Expand All @@ -610,8 +614,8 @@ Like every handle target, `--surface` requires a live control endpoint.
Source of truth: `extractSessionFlags` in `dor/src/commands/browser-cli.ts`,
`resolveSession` in `dor/src/commands/agent-browser.ts`, `resolveAgentBrowser` in
`dor/src/protocol.ts`, `ResolveAgentBrowserSessionRequest` / `Response` in
`dor/src/commands/types.ts`, `requireBrowserSurface` and the
`surface.resolveAgentBrowser` handler in
`dor/src/commands/types.ts`, `requireBrowserSurface`,
`requireAutomationSession` and the `surface.resolveAgentBrowser` handler in
`lib/src/components/wall/use-dor-control.ts`, and
`agentBrowserSessionFromParams` in `lib/src/components/wall/browser-surface.ts`.

Expand All @@ -627,8 +631,9 @@ stays unsupported.
named by its Workspace-stable `surface:N` ref, or rediscovered after layout
churn by `--command` / `--cwd` / `--port`, and `dor ensure`'s command+cwd match
is an implicit key that also lets an agent adopt a command the user started by
hand. Only browser Surfaces carry an explicit join key (`dor ab --key <name>`),
because their session is held externally by `agent-browser`.
hand. Only browser Surfaces carry an explicit join key (`dor ab --key <name>`,
`dor pw --key <name>`), because their session is held externally by the browser
CLI.

The worked examples — dev-server sharing, sub-agent launch and await, paired
browser keys, multi-worktree, minimized watchers, port-owner handoff, safe
Expand Down
40 changes: 28 additions & 12 deletions dor/skill.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,19 @@ These override your usual defaults. They matter more than anything else here:

The rest of this guide is how to do everything well.

## Which browser command

- **`dor ab`** — the default: a browser you read and drive, in a pane the user
watches.
- **`dor pw`** — when the user or the project uses Playwright, or the browser
you are handed already runs it (`render_mode` `pw-*`).
- **`dor iframe`** — only to show the human a local `http://` page. You cannot
read or drive it, it keeps no logins, and it refuses `https://`.

`dor list` shows each browser's `render_mode`: drive an `ab-*` browser with
`dor ab --surface <ref>`, a `pw-*` one with `dor pw --surface <ref>`, and an
`iframe` one with neither — open its page with `dor ab open <url>`.

## Targeting: three ways to name a surface

Action commands (`read`, `send`, `await`, `kill`) take a surface handle — there is
Expand All @@ -41,8 +54,8 @@ three ways:
(`--command`, `--cwd`, `--port`) into a handle.

Text output is designed for you to read: it is terse and carries the same
refs. Reach for `--json` (every command except `dor ab` supports it) only
when a shell script or pipeline using `jq` consumes the output.
refs. Reach for `--json` (every command except `dor ab` and `dor pw` supports
it) only when a shell script or pipeline using `jq` consumes the output.

## Surface handles

Expand Down Expand Up @@ -181,9 +194,9 @@ A Window holds several Workspaces, each with its own surfaces and its own
you were started in, and creating one is a change the user sees. When you do,
name one as `workspace:<n>` (positional) or `workspace:<name>`, and pass
`--workspace <ref>` to any command — `split`, `ensure`, `read`, `send`,
`await`, `kill`, `iframe`, `ab` — to act in another one. A surface's stable id
finds it in any Workspace without that flag; `surface:N` does not, since every
Workspace has one. `close` refuses a Workspace holding your running work
`await`, `kill`, `iframe`, `ab`, `pw` — to act in another one. A surface's
stable id finds it in any Workspace without that flag; `surface:N` does not,
since every Workspace has one. `close` refuses a Workspace holding your running work
unless you pass `--force`.

### `dor ab` / `dor agent-browser` — agent-drivable browser pane
Expand Down Expand Up @@ -216,12 +229,16 @@ open it with `dor ab` instead). The three identity flags are mutually exclusive.

### `dor pw` / `dor playwright` — Playwright browser pane

Use your installed `@playwright/cli` (`npm i -g @playwright/cli`). Override its
path with `DORMOUSE_PLAYWRIGHT_BIN`. Chromium panes share the Display, viewport,
input and popout controls of `dor ab`.
Forwards to your installed `@playwright/cli` (`npm i -g @playwright/cli`;
override its path with `DORMOUSE_PLAYWRIGHT_BIN`).

**Launch once with `open`, then navigate with `goto`.** Playwright's `open`
restarts the browser, dropping every tab and cookie, where `dor ab open` only
navigates.

```sh
dor pw --key app open :5173
dor pw --key app goto http://localhost:5173/settings
dor pw --key app snapshot
dor pw --key app click e15
dor pw --surface surface:4 goto :8080
Expand All @@ -230,8 +247,7 @@ dor pw --surface surface:4 goto :8080
`--key` defaults to `default` and is separate from agent-browser keys. The first
command fixes the native project cwd; later commands and relative paths use it.
`--session` (or `-s`) uses a raw native session in the caller's project instead.
These identities and `--surface` are mutually exclusive. Native Playwright
`open` restarts the browser; `goto` navigates its current tab. Other arguments
These identities and `--surface` are mutually exclusive. Other arguments
belong to `playwright-cli`; `dor pw --help` describes the wrapper.

## Recipes
Expand Down Expand Up @@ -313,5 +329,5 @@ dor kill surface:N --confirm-if-read "npm run dev"
just read the surface yourself.
- **Scope:** `dor` sees the current workspace only. Terminals ring bells and
carry todo flags (`[ringing]`/`[todo]` in `dor list`); browser surfaces are
the only ones with explicit keys, because their sessions live in
`agent-browser`.
the only ones with explicit keys, because their sessions live in the browser
CLI (`agent-browser` or `playwright-cli`).
5 changes: 3 additions & 2 deletions dor/src/commands/agent-browser.ts
Original file line number Diff line number Diff line change
Expand Up @@ -92,8 +92,9 @@ command surface. The binary is resolved from PATH (override with
DORMOUSE_AGENT_BROWSER_BIN) and is never bundled; install it with:
${INSTALL_HINT}

After a successful command, dor opens (or reuses) the browser surface bound to
the session: one session is always exactly one surface.
After a successful command, dor opens the browser surface bound to the session,
or reuses the one it already has. A Playwright browser (render_mode pw-*) is
driven with dor pw --surface instead.

In an "open" command, dor also resolves a Dormouse target in place of a URL:
a schemeless host:port (and the ":<port>" localhost shorthand) defaults to
Expand Down
7 changes: 4 additions & 3 deletions dor/src/commands/iframe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -33,13 +33,14 @@ export const iframeCommand: Command = {
command: buildCommand<IframeFlags, [string], DorCommandContext>({
docs: {
brief: 'Open a target in an iframe surface.',
fullDescription: `Opens a target in a high-fidelity iframe surface for human inspection.
fullDescription: `Opens an http:// page in a high-fidelity iframe surface for a human to look at.

Agents cannot read or drive an iframe surface, and it drops the page's cookies, so logins do not work in it. For those, and for any https:// page, which the iframe refuses, use \`dor ab open <url>\`.

If the caller surface is an untouched terminal, Dormouse replaces that terminal with the iframe. Otherwise Dormouse creates a split next to the caller/focused surface.

The target is one of:
<url> An absolute http:// or https:// URL (an explicit scheme is
always honored).
<url> An absolute http:// URL.
host:port A schemeless host:port, defaulted to http:// (e.g.
localhost:5173, box.ts.net:3000). The explicit port marks a
dev/infra server, which is http far more often than not.
Expand Down
Loading
Loading