Skip to content

One host-owned viewer socket for both browser providers; no CDP in the webview - #778

Merged
nedtwigg merged 8 commits into
browser-providersfrom
browser-viewer-socket
Sep 24, 2026
Merged

nedtwigg merged 8 commits into
browser-providersfrom
browser-viewer-socket

Conversation

@nedtwigg

Copy link
Copy Markdown
Member

Both providers now stream through one host-owned viewer socket, so the webview never talks to a daemon directly and never holds CDP. Stacked on #777. Items C8 and D2 from the browser-stack deep dive.

One host-owned viewer socket (C8)

  • One listener for everything. lib/src/host/browser-viewer.ts is a single loopback listener. Each upgrade needs the listener's own Host plus a single-use 60 s grant from the new view op (BrowserStreamGrants). There is one BrowserView per socket.
  • agent-browser. The host dials the daemon stream (loopback only; the dial times out after 5 s), drops the daemon's ~20 Hz re-broadcast by raw Buffer.equals, and parses and decodes only changed frames.
  • Playwright. It feeds the same view. Its per-viewer server is gone.
  • One topology on all three hosts. Deleted: the VS Code TCP relay and its grant table (plus its loopback-lint exemption), and standalone's direct-to-daemon dial.
  • Binary frames. Each frame is a 12-byte header (provisional/crisp, CSS viewport) plus the JPEG; state is sent as JSON text. The webview uses binaryType='arraybuffer' and decodes only the latest frame (one in flight plus one replaceable pending). The canvas stays at crisp size, so switching between provisional and crisp frames no longer reallocates it. Playwright acks are paced to 50 ms.
  • Crisp capture moved into the host. The screenshot loop now runs host-side with the same pacing, input window, overdue, stale-drop and dedup rules. Each capture uses a fresh private file, read into memory and deleted however the capture ends (browser-capture.ts). Deleted:
    • Tauri browser_screenshot;
    • the sidecar file transport;
    • the dev harness readCapture;
    • the webview screenshot loop;
    • the screenshot / streamUrl / cdpUrl ops.
  • Paste is input_text for both providers. The host expands it for agent-browser.
  • Relaunch gating is host-local. While a launch or close of a browser runs, the host refuses every other op on it except close. agent-browser operations and captures run only on a daemon liveDaemon proves live: a pid file from this boot, a live process, and an accepting stream port. No host operation ever starts a daemon. A launch or close ends every viewer socket on that browser.

No CDP in the webview (D2)

  • Deleted: startCdpObserver, reconcileCdp and the cdpUrl op.
  • A headed agent-browser view reads get cdp-url host-side, bounded to 10 s and loopback only, and pushes page {url,title}.
  • A browser that goes away on its own sends status {connected:false} and closes the socket, so auto-revert no longer waits out three reconnects.

Measurements

Measured 2026-09-24 on Node 24.18 with a synthetic 20 Hz daemon sending ~133 KB frames; the numbers are also recorded in the rationale.

Page Host CPU, 1 / 4 / 8 panes Sent to the webview
Static 1.3% / 3.4% / 4.7% nothing
Animated 2.9% / 7.9% / 11.1% ~1.9 MB/s per pane
  • Event-loop p99 delay stayed ≤ 12.7 ms, so no worker thread is needed.
  • The old webview djb2 dedup cost 208 µs per frame; a raw compare costs ~2.5 µs.

Specs: new "Viewer Socket" section in docs/specs/dor-browser.md, with Browser Connection, Browser Host, the provider sections and Pop-Out rewritten. security-local.md swaps the relay FAIL IF line for a viewer-listener one. vscode.md, standalone.md and dor-cli.md are updated, plus the rationale files.

Testing

  • Root pnpm test passes: lib 3929, vscode-ext 181, standalone, website, dor 190, and all lints, including loopback-lint, which detects the new listener with its guard.
  • Argos passes in Chromium and WebKit (294/294 each).
  • cargo check passes.
  • Each of the 52 targeted reverts turns its test red.
  • Runtime, in the innerdogfood harness: the view grant, the guarded upgrade from the real webview, tabs/status relay, navigation and the debug stats line are verified. Frames were not seen at runtime: no local agent-browser Chrome emitted screencast frames during the session, so frame relay is covered by tests and the synthetic benchmark only. The real-Playwright opt-in test was not run.

🤖 Generated with Claude Code

https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

Deploying mouseterm with  Cloudflare Pages  Cloudflare Pages

Latest commit: 7754d61
Status: ✅  Deploy successful!
Preview URL: https://a8ed32c0.mouseterm.pages.dev
Branch Preview URL: https://browser-viewer-socket.mouseterm.pages.dev

View logs

@nedtwigg
nedtwigg marked this pull request as ready for review September 24, 2026 11:42

@dormouse-bot dormouse-bot left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BrowserView.close limits the close reason to 120 characters, but ws limits it to 123 bytes. If the reason is longer than 123 bytes in UTF-8, WebSocket#close throws RangeError: The message must not be greater than 123 bytes synchronously. One caller passes arbitrary text: the rejection handler in openView ((error) => view.close(1011, messageOf(error))). That text can be playwright-cli stderr, a spawn error, or a message that names a path under a non-ASCII home directory. When it throws there:

  • dispose() has already run, so the view is gone from views while the webview's socket stays open. The pane waits forever and never reconnects.
  • The throw rejects the promise that .then(ok, err) returns, and nothing handles it. standalone/sidecar/main.js installs no unhandledRejection handler, so on Node's default --unhandled-rejections=throw the sidecar exits and takes every terminal with it.

The inline suggestion cuts the reason by byte length instead.

Comment thread lib/src/host/browser-viewer.ts
@argos-ci

argos-ci Bot commented Sep 24, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Argos notifications ↗︎

Build Status Details Updated (UTC)
storybook-chromium (Inspect) ✅ No changes detected - Sep 24, 2026, 4:08 PM
storybook-webkit (Inspect) 👍 Approved by Ned Twigg 1 changed Sep 24, 2026, 4:08 PM
Deployment Status Branch Updated (UTC)
preview (Open) Ready browser-viewer-socket Sep 24, 2026, 4:08 PM

@nedtwigg
nedtwigg added this pull request to stack #782 September 24, 2026 15:10
The rule that nothing may reach a browser between a relaunch's close and
its reopen was enforced by the controller's drive gate, which saw only its
own Surface's requests. The host serializes every launch and close, so it
now refuses every operation but launch, attach and close on a browser a
launch is replacing or a close is ending, whichever Surface asks.

agent-browser starts a daemon at about:blank for any verb run with none up,
so its provider now runs an operation only while the session's pid file
names a live process. A daemon gone while its pane was hidden, or one in a
socket directory the host does not share, is refused instead of restarted.

With both in the host, the controller's gate no longer holds back an
unparked pane's operations until its stream opens; only its catch-up waits,
and a navigation driven directly supersedes one still pending.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe
nedtwigg and others added 7 commits September 24, 2026 09:00
The webview reached a browser three ways — the agent-browser daemon's
stream directly in standalone, through a VS Code-only TCP relay with its own
grant table, and through Playwright's own guarded server — ran a screenshot
loop whose every capture crossed each host's IPC twice, decoded base64 for
every frame it painted, and held a browser-level CDP socket (`get cdp-url`)
to follow a popped-out window. It also issued the daemon commands that
raced relaunches.

Now one loopback listener in the host serves a viewer socket per Surface,
guarded by its own Host and a single-use grant the `view` operation issues:

- The host subscribes to the provider: agent-browser's daemon stream,
  dialed on loopback only, its ~20 Hz re-broadcast dropped by raw compare
  and each changed frame decoded once; Playwright's CDP screencast, decoded
  once, its acks paced to ~20 fps.
- Frames reach the webview as binary messages tagged provisional or crisp;
  status, tabs, url and a headed window's page as JSON; input comes back
  rebuilt field by field. A paste is text for both providers.
- The two-stage paint runs host-side: which stream frames to send, and the
  crisp-capture loop with its pacing, input window and overdue rules. The
  webview decodes latest-only and keeps the canvas at crisp size.
- A headed window's page is followed over CDP held in the host, and a
  browser that goes on its own is reported gone, so auto-revert no longer
  waits out three reconnects.
- A launch or close ends every socket on its browser at once.

Deletes the webview screenshot loop and CDP observer, the `screenshot`,
`streamUrl` and `cdpUrl` operations, Tauri's `browser_screenshot`, the
sidecar's file transport, the dev harness's capture re-encoding, and the
VS Code relay with its loopback-lint exemption. The stream a launch or
attach answers is renamed from `wsPort`, since a Playwright one is the
host's number for its connection rather than a port.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe
A headed view must still receive a tab list or URL large enough to pass
for a frame, a Playwright viewer whose input backs up behind CDP closes,
a socket backed up past 2 MB skips provisional frames only, and a repaint
resends the last frame. A view's input window now starts closed rather
than at time zero. The agent-browser stream's explicit port-range check is
gone: a port the URL cannot hold already fails the dial.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe
Declare the host's viewer server, its views and its captures up front, drop
a daemon stream frame's second dedup path, keep the paste expansion private
to the agent-browser provider, and give the controller one setter for the
viewport size a status or a frame reports.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe
- One proof of an agent-browser daemon, `liveDaemon` — a pid file from this
  boot naming a live process beside a stream port that accepts — now gates
  a stop's signal, every operation and capture, and the headed window's
  `get cdp-url`, where operations had checked only the pid.
- `dor ab` hands over the port it read after every command, so the viewer
  dials a handed-over loopback port for any bind; it captures that browser
  only when the same proof names that port, and otherwise sends every
  changed frame, never reading the caller's socket directory.
- Every capture writes a fresh, randomly named file, read into memory and
  deleted however the capture ends — a failed or killed one included — so
  no frame waits on disk; with the file transport gone, the per-reader
  copies and the pruning of unread frames are moot.
- The viewer's upstream dials end by 5 s and `get cdp-url` by 10 s, and the
  viewer listener closes every connection at shutdown.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe
A stream frame small enough to pass for a control message is compared
against the last like any other. The viewer listener's
closeAllConnections at shutdown goes: Node's own server.close already ends
a connection holding a half-sent request, so nothing observed it.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe
A viewer socket whose provider could not subscribe was closed with that
error as its reason, cut to 120 characters. ws caps a close reason at 123
UTF-8 bytes and throws a RangeError, synchronously, past that — a Playwright
CLI's stderr, or a path under a non-ASCII home, is well past it. The throw
came after the view had disposed itself, so the webview's socket stayed
open with nothing behind it, and it rejected the promise the rejection
handler returned, which nothing handled: on Node's default, the sidecar
would exit with every terminal in it.

Every close the host makes with a reason now goes through `closeSocket`,
which cuts the reason to the longest whole-character prefix of 123 bytes
and terminates the socket rather than throw. `openView`'s subscription
chain ends in a catch, so a throw anywhere in attaching or dropping a
subscription is logged, never an unhandled rejection.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NCV5Uq6FeZbBfzfpk3vrpe
@nedtwigg
nedtwigg force-pushed the browser-viewer-socket branch from 24b5fad to 7754d61 Compare September 24, 2026 16:05
@nedtwigg

Copy link
Copy Markdown
Member Author

Restacked onto #777 after #776 landed on main. The close-reason fix from the resolved thread (was 24b5fad) is now 7754d61. No other changes.

@nedtwigg
nedtwigg merged commit 53a1fc2 into main Sep 24, 2026
15 checks passed
@nedtwigg
nedtwigg deleted the browser-viewer-socket branch September 24, 2026 16:45

This branch is waiting to be deployed

1 waiting deployment
hosted-preview — 7754d611 Waiting Sep 24, 2026 by nedtwigg via cleanup #367
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants