-
Notifications
You must be signed in to change notification settings - Fork 1.7k
feat(mastra): add Stagehand code-mode MCP example #2627
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
shrey150
wants to merge
17
commits into
shrey/stg-2765-codemode-vercel
Choose a base branch
from
shrey/stg-2765-codemode-mastra
base: shrey/stg-2765-codemode-vercel
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
17 commits
Select commit
Hold shift + click to select a range
0d14ea2
feat(mastra): add Stagehand code-mode MCP example
shrey150 3901388
Merge updated Vercel integration parent
shrey150 27a2252
Merge updated Vercel integration parent
shrey150 938a0c2
Merge updated Vercel integration parent
shrey150 e4367d1
fix(mastra): satisfy repository lint
shrey150 e4d7acf
Merge refreshed Vercel integration parent
shrey150 027c7dd
Merge remote-tracking branch 'origin/shrey/stg-2765-codemode-vercel' …
shrey150 e987646
refactor(mastra): use sandboxed code-mode MCP
shrey150 e8a31bb
Merge branch 'shrey/stg-2765-codemode-vercel' of https://github.com/b…
shrey150 56e5a77
Merge latest Vercel sandbox hardening into Mastra
shrey150 4da6eed
fix(mastra): gate live proof on credentials
shrey150 b43e827
Merge sandbox CI graph fix into Mastra
shrey150 7d72238
fix(mastra): sanitize sandbox adapter failures
shrey150 f86dbb4
Merge latest Vercel review fixes into Mastra
shrey150 6db5cdf
test(mastra): cover resolved discovery errors
shrey150 e8d4ad8
Merge latest Vercel contract tests into Mastra
shrey150 288f4da
Merge deterministic Vercel stderr test into Mastra
shrey150 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Mastra with Stagehand code mode | ||
|
|
||
| This example gives a Mastra agent one browser tool: `code_execute`. The Stagehand code-mode MCP | ||
| server runs inside the package-installed [Vercel Sandbox example](../vercel-sandbox); Mastra connects | ||
| to its authenticated Streamable HTTP endpoint and keeps one MCP client and browser session alive for | ||
| the agent handle's lifetime. | ||
|
|
||
| The agent instructions come directly from the MCP tool description. The adapter does not copy the | ||
| Stagehand executor, schema, or code-mode skill, so the agent and the tool cannot drift onto different | ||
| browser APIs. | ||
|
|
||
| ## Run the end-to-end proof | ||
|
|
||
| From the repository root, install dependencies, build and pack the exact Stagehand artifacts, then | ||
| run the Mastra proof: | ||
|
|
||
| ```bash | ||
| pnpm install | ||
| pnpm exec turbo run build --filter @browserbasehq/stagehand-codemode | ||
| pnpm --filter @browserbasehq/stagehand-integrations-example-vercel-sandbox pack:artifacts | ||
| STAGEHAND_SANDBOX_ARTIFACTS="$PWD/packages/integrations/examples/vercel-sandbox/.artifacts" \ | ||
| BROWSERBASE_API_KEY=<api-key> \ | ||
| BROWSERBASE_PROJECT_ID=<project-id> \ | ||
| VERCEL_OIDC_TOKEN=<oidc-token> \ | ||
| OPENAI_API_KEY=<openai-key> \ | ||
| pnpm --dir packages/integrations/examples/mastra e2e | ||
| ``` | ||
|
|
||
| For local Vercel credentials, replace `VERCEL_OIDC_TOKEN` with `VERCEL_TEAM_ID`, | ||
| `VERCEL_PROJECT_ID`, and `VERCEL_TOKEN`. | ||
|
|
||
| The proof exercises one live package-installed sandbox and one persistent Mastra MCP client. It: | ||
|
|
||
| 1. invokes `code_execute` twice directly and verifies the same page ID and DOM marker; | ||
| 2. makes a real Mastra model select `code_execute` and modify that existing page; | ||
| 3. invokes the tool again to independently verify the model's browser-side change; | ||
| 4. proves `OPENAI_API_KEY` and a host-only marker never enter model-generated code; and | ||
| 5. disconnects Mastra before stopping and deleting the sandbox, emitting `PASS` only after cleanup. | ||
|
|
||
| Set `MASTRA_MODEL` to use a model other than `openai/gpt-5-mini`. | ||
|
|
||
| ## Use the agent | ||
|
|
||
| ```ts | ||
| import { createStagehandSandbox } from "@browserbasehq/stagehand-integrations-example-vercel-sandbox"; | ||
|
|
||
| import { createStagehandAgent } from "./src/agent.js"; | ||
|
|
||
| const connection = await createStagehandSandbox({ | ||
| packageArtifactsPath: process.env.STAGEHAND_SANDBOX_ARTIFACTS!, | ||
| browserbaseApiKey: process.env.BROWSERBASE_API_KEY!, | ||
| browserbaseProjectId: process.env.BROWSERBASE_PROJECT_ID!, | ||
| }); | ||
| const stagehand = await createStagehandAgent(connection); | ||
|
|
||
| try { | ||
| const response = await stagehand.agent.generate("Open example.com and return the page heading.", { | ||
| maxSteps: 8, | ||
| }); | ||
| console.log(response.text); | ||
| } finally { | ||
| await stagehand.close(); | ||
| await connection.close(); | ||
| } | ||
| ``` | ||
|
|
||
| The outer application owns the sandbox connection. Disconnect the Mastra MCP client before closing | ||
| that connection so the HTTP transport can finish cleanly. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,24 @@ | ||
| { | ||
| "name": "@browserbasehq/stagehand-integrations-example-mastra", | ||
| "version": "4.0.0", | ||
| "private": true, | ||
| "type": "module", | ||
| "scripts": { | ||
| "e2e": "tsx src/e2e.ts", | ||
| "test:contract": "tsx --test src/*.test.ts", | ||
| "typecheck": "tsc --noEmit" | ||
| }, | ||
| "dependencies": { | ||
| "@browserbasehq/stagehand-integrations-example-vercel-sandbox": "workspace:*", | ||
| "@mastra/core": "catalog:", | ||
| "@mastra/mcp": "catalog:" | ||
| }, | ||
| "devDependencies": { | ||
| "@types/node": "catalog:", | ||
| "tsx": "catalog:", | ||
| "typescript": "catalog:" | ||
| }, | ||
| "engines": { | ||
| "node": ">=22.18.0" | ||
| } | ||
| } |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,76 @@ | ||
| import assert from "node:assert/strict"; | ||
| import test from "node:test"; | ||
|
|
||
| import type { MCPClient } from "@mastra/mcp"; | ||
|
|
||
| import { | ||
| loadStagehandCodeTools, | ||
| StagehandMastraSetupError, | ||
| StagehandMastraToolContractError, | ||
| } from "./agent.js"; | ||
|
|
||
| const canonicalDescription = "# Stagehand V4 code-mode syntax\nUse the persistent page."; | ||
| const codeExecute = { description: canonicalDescription, execute: async () => undefined }; | ||
|
|
||
| void test("accepts exactly one canonical code_execute tool", async () => { | ||
| const tools = await loadStagehandCodeTools( | ||
| fakeMcp({ toolsets: { stagehand: { code_execute: codeExecute } }, errors: {} }), | ||
| ); | ||
| assert.equal(tools.code_execute, codeExecute); | ||
| }); | ||
|
|
||
| void test("sanitizes transport discovery errors", async () => { | ||
| const secret = "https://sandbox.example.test/mcp?token=do-not-reflect"; | ||
| const mcp = { | ||
| listToolsetsWithErrors: async () => { | ||
| throw new Error(secret); | ||
| }, | ||
| } as unknown as MCPClient; | ||
|
|
||
| await assert.rejects(loadStagehandCodeTools(mcp), (error: unknown) => { | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
|
||
| assert.ok(error instanceof StagehandMastraSetupError); | ||
| assert.equal(error.message, "Could not configure the Mastra Stagehand agent."); | ||
| assert.equal(error.message.includes(secret), false); | ||
| return true; | ||
| }); | ||
| }); | ||
|
|
||
| void test("sanitizes resolved MCP discovery errors", async () => { | ||
| const secret = "https://sandbox.example.test/mcp?token=do-not-reflect"; | ||
|
|
||
| await assert.rejects( | ||
| loadStagehandCodeTools( | ||
| fakeMcp({ | ||
| toolsets: {}, | ||
| errors: { stagehand: new Error(secret) }, | ||
| }), | ||
| ), | ||
| (error: unknown) => { | ||
| assert.ok(error instanceof StagehandMastraSetupError); | ||
| assert.equal(error.message, "Could not configure the Mastra Stagehand agent."); | ||
| assert.equal(error.message.includes(secret), false); | ||
| return true; | ||
| }, | ||
| ); | ||
| }); | ||
|
|
||
| void test("rejects non-canonical toolsets with a fixed typed error", async () => { | ||
| await assert.rejects( | ||
| loadStagehandCodeTools( | ||
| fakeMcp({ | ||
| toolsets: { stagehand: { unexpected_secret_tool: codeExecute } }, | ||
| errors: {}, | ||
| }), | ||
| ), | ||
| { | ||
| name: StagehandMastraToolContractError.name, | ||
| message: "The Stagehand MCP server returned an invalid tool contract.", | ||
| }, | ||
| ); | ||
| }); | ||
|
|
||
| function fakeMcp(result: unknown): MCPClient { | ||
| return { | ||
| listToolsetsWithErrors: async () => result, | ||
| } as unknown as MCPClient; | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| import { Agent } from "@mastra/core/agent"; | ||
| import { MCPClient } from "@mastra/mcp"; | ||
| import type { StagehandSandboxConnection } from "@browserbasehq/stagehand-integrations-example-vercel-sandbox"; | ||
|
|
||
| type RemoteStagehandConnection = Pick<StagehandSandboxConnection, "url" | "token">; | ||
|
|
||
| export class StagehandMastraSetupError extends Error { | ||
| override readonly name = "StagehandMastraSetupError"; | ||
|
|
||
| constructor() { | ||
| super("Could not configure the Mastra Stagehand agent."); | ||
| } | ||
| } | ||
|
|
||
| export class StagehandMastraToolContractError extends Error { | ||
| override readonly name = "StagehandMastraToolContractError"; | ||
|
|
||
| constructor() { | ||
| super("The Stagehand MCP server returned an invalid tool contract."); | ||
| } | ||
| } | ||
|
|
||
| export function createStagehandMcpClient(connection: RemoteStagehandConnection): MCPClient { | ||
| return new MCPClient({ | ||
| id: "stagehand-codemode", | ||
| servers: { | ||
| stagehand: { | ||
| url: connection.url, | ||
| fetch: async (input, init) => { | ||
| const headers = new Headers(init?.headers); | ||
| headers.set("Authorization", `Bearer ${connection.token}`); | ||
| return fetch(input, { ...init, headers }); | ||
| }, | ||
| }, | ||
| }, | ||
| }); | ||
| } | ||
|
|
||
| export async function loadStagehandCodeTools(mcp: MCPClient) { | ||
| let discovery: Awaited<ReturnType<MCPClient["listToolsetsWithErrors"]>>; | ||
| try { | ||
| discovery = await mcp.listToolsetsWithErrors(); | ||
| } catch { | ||
| throw new StagehandMastraSetupError(); | ||
| } | ||
| const { toolsets, errors } = discovery; | ||
| if (Object.keys(errors).length > 0) { | ||
| throw new StagehandMastraSetupError(); | ||
| } | ||
|
|
||
| const stagehandTools = toolsets.stagehand ?? {}; | ||
| const toolNames = Object.keys(stagehandTools); | ||
| if (toolNames.length !== 1 || toolNames[0] !== "code_execute") { | ||
| throw new StagehandMastraToolContractError(); | ||
| } | ||
|
|
||
| const codeExecute = stagehandTools.code_execute; | ||
| if (!codeExecute) { | ||
| throw new StagehandMastraToolContractError(); | ||
| } | ||
|
|
||
| const guidance = codeExecute.description?.trim(); | ||
| if (!guidance?.includes("# Stagehand V4 code-mode syntax")) { | ||
| throw new StagehandMastraToolContractError(); | ||
| } | ||
|
|
||
| return { code_execute: codeExecute }; | ||
| } | ||
|
|
||
| export type StagehandCodeTools = Awaited<ReturnType<typeof loadStagehandCodeTools>>; | ||
|
|
||
| export type StagehandAgentHandle = { | ||
| agent: Agent; | ||
| tools: StagehandCodeTools; | ||
| close: () => Promise<void>; | ||
| }; | ||
|
|
||
| export async function createStagehandAgent( | ||
| connection: RemoteStagehandConnection, | ||
| model = process.env.MASTRA_MODEL ?? "openai/gpt-5-mini", | ||
| ): Promise<StagehandAgentHandle> { | ||
| const mcp = createStagehandMcpClient(connection); | ||
|
cubic-dev-ai[bot] marked this conversation as resolved.
|
||
|
|
||
| try { | ||
| const tools = await loadStagehandCodeTools(mcp); | ||
| const instructions = tools.code_execute.description; | ||
| if (!instructions) { | ||
| throw new StagehandMastraToolContractError(); | ||
| } | ||
|
|
||
| const agent = new Agent({ | ||
| id: "stagehand-browser-agent", | ||
| name: "Stagehand browser agent", | ||
| instructions, | ||
| model, | ||
| tools, | ||
| }); | ||
|
|
||
| return { | ||
| agent, | ||
| tools, | ||
| close: () => mcp.disconnect(), | ||
| }; | ||
| } catch (error) { | ||
| await mcp.disconnect().catch(() => undefined); | ||
| if ( | ||
| error instanceof StagehandMastraSetupError || | ||
| error instanceof StagehandMastraToolContractError | ||
| ) { | ||
| throw error; | ||
| } | ||
| throw new StagehandMastraSetupError(); | ||
| } | ||
| } | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.