-
Notifications
You must be signed in to change notification settings - Fork 1.7k
feat(integrations): fx harness example (MCP config) #2776
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
84dba82
feat(integrations): fx harness example (MCP config)
miguelg719 0133f8c
docs(integrations): harden fx headless guidance
miguelg719 1483e5e
fix(integrations): accurate fx env-inheritance docs; bound sdk-parity…
miguelg719 123317f
Apply suggestion from @miguelg719
miguelg719 0c546d4
docs(integrations): drop project id from fx allowlist example
miguelg719 71d1045
test(extension): use runtime version in identity marker
miguelg719 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,4 @@ | ||
| { | ||
| "max_agent_steps": 60, | ||
| "max_tool_result_bytes": 262144 | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,118 @@ | ||
| # fx + Stagehand facade over MCP/stdio | ||
|
|
||
| [fx](https://fx.sh) consumes the Stagehand facade as a standard MCP server — no integration | ||
| code, just an entry in fx's user-global MCP config. This directory ships the config template, | ||
| a project config that sizes fx's limits for browser work, and a skill carrying the facade | ||
| usage guidance. | ||
|
|
||
| <!-- Verified against fx v0.0.3; fx is experimental and its config surface may change. --> | ||
|
|
||
| ## Setup | ||
|
|
||
| Use Node.js 24 or newer. From the repository root, build the integrations package first: | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| pnpm exec turbo run build --filter @browserbasehq/stagehand-integrations | ||
| ``` | ||
|
|
||
| Install fx (pin the version — fx is experimental) and authenticate with Vercel AI Gateway: | ||
|
|
||
| ```bash | ||
| curl -fsSL https://fx.sh/setup.sh | bash -s -- v0.0.3 | ||
| fx login # or: export AI_GATEWAY_API_KEY=... | ||
| ``` | ||
|
|
||
| Export the browser credentials (Browserbase is the default and recommended backend): | ||
|
|
||
| ```bash | ||
| export BROWSERBASE_API_KEY=bb_live_... | ||
| ``` | ||
|
|
||
| ## Configure | ||
|
|
||
| fx loads MCP servers only from the user-global `~/.fx/mcp.json`; repository-local MCP config | ||
| is deliberately never loaded. Merge `mcp.json` from this directory into `~/.fx/mcp.json`, | ||
| filling in the absolute path to your checkout, then run `/mcp reload` in an open session (or | ||
| just start a new one). | ||
|
|
||
| Leave the `environment` block out, as the template does: fx passes the full shell environment | ||
| to the server when no block is set, so the exports above are the only configuration. Setting | ||
| `environment` replaces the child environment wholesale — even `PATH` and `HOME` disappear — | ||
| so if you must pin variables there, restate `PATH` explicitly. | ||
|
|
||
| ## Run | ||
|
|
||
| Run fx from this directory — this is required, not optional: the `skills/stagehand-facade` | ||
| skill teaches the model the exact `mcp_stagehand_*` tool names (fx's tool search cannot find | ||
| them), and without it runs stall in discovery and fall back to shell exploration. Running here | ||
| also picks up `.fx.json` (which raises `max_tool_result_bytes` — | ||
| page snapshots exceed fx's 64 KB default — and `max_agent_steps`, since fx's tool discovery | ||
| adds a `mcp_search_tools`/`mcp_select_tool` round trip before the browser tools are callable) | ||
| and the `skills/stagehand-facade` skill: | ||
|
|
||
| ```bash | ||
| cd packages/integrations/fx | ||
| fx ask --json "Use the stagehand browser tools: open https://example.com, snapshot it, and report the heading citing the snapshot ID." | ||
| ``` | ||
|
|
||
| The three tools surface as `mcp_stagehand_run`, `mcp_stagehand_snapshot`, and | ||
| `mcp_stagehand_screenshot`. fx v0.0.3's `mcp_search_tools` returns no results for this server, | ||
| so the shipped skill instructs the model to select the tools by those exact names instead — | ||
| without the skill, runs stall in discovery. Headless runs cannot answer permission prompts; either pre-allow | ||
| the tools in `~/.fx/settings.json`: | ||
|
|
||
| ```json | ||
| { | ||
| "permission": { | ||
| "mcp_stagehand_run": "allow", | ||
| "mcp_stagehand_snapshot": "allow", | ||
| "mcp_stagehand_screenshot": "allow" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| or pass `--auto`, accepting that fx adjudicates each gated call with an extra model request. | ||
| For browser-only workflows, also deny fx's shell tool — if the model cannot find the browser | ||
| tools (for example when the skill is not loaded), it falls back to exploring the machine with | ||
| `run_command`, which can dump your environment (including credentials) into the model | ||
| transcript: | ||
|
|
||
| ```json | ||
| { | ||
| "permission": { | ||
| "mcp_stagehand_run": "allow", | ||
| "mcp_stagehand_snapshot": "allow", | ||
| "mcp_stagehand_screenshot": "allow", | ||
| "run_command": "deny" | ||
| } | ||
| } | ||
| ``` | ||
|
|
||
| <Note> | ||
| fx starts MCP servers with a fixed 10-second timeout and discards their stderr. The facade | ||
| connects immediately and launches the browser lazily on the first tool call, so startup fits | ||
| the budget — but if the server misbehaves, debug it standalone (spawn the bin directly and | ||
| speak JSON-RPC over stdio) rather than through fx. | ||
| </Note> | ||
|
|
||
| ## Security model | ||
|
|
||
| The `run` tool executes model-authored JavaScript inside the Stagehand browser extension's | ||
| service worker — browser-side, never on your machine. Browserbase is the recommended isolation | ||
| boundary: the privileged execution environment is a disposable cloud browser. With no | ||
| `environment` block, the server inherits your shell environment. The facade reads | ||
| `STAGEHAND_*`/`BROWSERBASE_*` variables and can also infer a model-provider key | ||
| (`OPENAI_API_KEY`, `ANTHROPIC_API_KEY`, or a Google key) from its environment for optional | ||
| Stagehand model configuration. To keep provider keys away from the facade entirely, use an | ||
| `environment` allowlist instead of inheritance — restating `PATH` and `HOME`, which fx drops | ||
| when any block is set: | ||
|
|
||
| ```json | ||
| "environment": { | ||
| "PATH": "/usr/local/bin:/usr/bin:/bin", | ||
| "HOME": "/Users/you", | ||
| "STAGEHAND_BROWSER": "browserbase", | ||
| "BROWSERBASE_API_KEY": "bb_live_..." | ||
| } | ||
| ``` | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,12 @@ | ||
| { | ||
|
miguelg719 marked this conversation as resolved.
|
||
| "mcp": { | ||
| "stagehand": { | ||
| "type": "stdio", | ||
| "command": [ | ||
| "node", | ||
| "/absolute/path/to/stagehand/packages/integrations/core/dist/facade/stdio-server.mjs" | ||
| ], | ||
| "required": true | ||
| } | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,16 @@ | ||
| --- | ||
| name: stagehand-facade | ||
| description: How to drive the Stagehand browser tools (run, snapshot, screenshot) exposed over MCP. | ||
| --- | ||
|
|
||
| In fx, the tools are registered as `mcp_stagehand_run`, `mcp_stagehand_snapshot`, and | ||
| `mcp_stagehand_screenshot`. Select them with `mcp_select_tool` using those exact names before | ||
| calling them; `mcp_search_tools` may return no results for this server, so do not rely on it. | ||
|
|
||
| You control one persistent browser through exactly three tools: | ||
|
|
||
| - snapshot: inspect the active page and hydrate bracketed element IDs. | ||
| - run: provide either snapshot actions or JavaScript using the Playwright-shaped page API. | ||
| - screenshot: inspect the rendered page visually. | ||
|
|
||
| Use snapshot actions for simple interactions and run code for multi-step workflows. Pass run exactly one of code or actions; every action uses "op" and "id", never "kind" or "ref". Snapshot IDs are valid only for the latest snapshot of the active page; snapshot again after navigation or stale IDs. Do not launch another browser. |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.