Skip to content

feat: let agents drive a browser in the sandbox, and people use it with them - #303

Draft
tjholm wants to merge 1 commit into
sandbox-network-accessfrom
sandbox-desktop
Draft

tjholm wants to merge 1 commit into
sandbox-network-accessfrom
sandbox-desktop

Conversation

@tjholm

@tjholm tjholm commented Oct 6, 2026

Copy link
Copy Markdown
Member

Agents get Playwright MCP's browser_ tools, run by a Chromium of their own in each thread on their own desktop in the pod's sandbox. People open that desktop from the Sandbox button at the top of the agent's chat, which says when the agent is using it, or from a browser call in the thread, and can watch, click and type, through a WebSocket the API relays from the desktop's VNC server, behind the thread's permissions. Opening it uses the sandbox as a turn would, resuming or making it, holds it awake, and starts the desktop without a browser; the agent's browser joins the same desktop. read_file shows images to models that take them. The sandbox interface gains endpoint(port), through OpenSandbox's port proxy or E2B's host, with header routing on E2B Embed.


Stack created with GitHub Stacks CLI • Give Feedback 💬

@tjholm
tjholm added this pull request to stack #305 October 6, 2026 03:35
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Oct 6, 2026 •

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
✅ Deployment successful!
View logs
sugabots-website a2c3912 Oct 06 2026, 05:09 AM

…th them

Agents get Playwright MCP's browser_ tools, run by a Chromium of their own in each thread on their own desktop in the pod's sandbox. People open that desktop from the Sandbox button at the top of the agent's chat, which says when the agent is using it, or from a browser call in the thread, and can watch, click and type, through a WebSocket the API relays from the desktop's VNC server, behind the thread's permissions. Opening it uses the sandbox as a turn would, resuming or making it, holds it awake, and starts the desktop without a browser; the agent's browser joins the same desktop. read_file shows images to models that take them. The sandbox interface gains endpoint(port), through OpenSandbox's port proxy or E2B's host, with header routing on E2B Embed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant