Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/codemode-host-apis.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@cloudflare/codemode": patch
---

Add framework-neutral `execute`, `search`, and `describe` methods to the durable Code Mode runtime handle so MCP servers and other non-AI-SDK hosts can invoke execution and discovery directly. Search and describe results now identify methods that require approval.
16 changes: 16 additions & 0 deletions docs/codemode/runtime.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,8 @@ const runtime = createCodemodeRuntime({
| Handle method | Purpose |
| ---------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `runtime.tool(options?)` | The single model-facing AI SDK tool, `codemode({ code })` |
| `runtime.execute({ code })` | Run code directly from a non-AI-SDK host, such as an MCP tool handler |
| `runtime.search(query)` / `runtime.describe(target)` | Search connector methods and snippets, then fetch one target's on-demand TypeScript documentation |
| `runtime.pending(executionId?)` | Actions awaiting approval — drives approval UIs; no id aggregates all paused runs |
| `runtime.approve({ executionId })` | Approve the pending action and continue via replay |
| `runtime.reject({ seq, executionId })` | Reject a pending action; ends the execution. Returns `false` if it was a no-op (action no longer pending — approved or rejected elsewhere) |
Expand All @@ -34,6 +36,20 @@ const runtime = createCodemodeRuntime({
| `runtime.saveSnippet(name, opts?)` | Promote an execution's script to a reusable [snippet](./snippets.md) |
| `runtime.snippets()` / `runtime.deleteSnippet(name)` | List / remove saved snippets |

`execute()`, `search()`, and `describe()` expose the same runtime to hosts that are not using the AI SDK. For example, an MCP server can register separate `search` and `execute` tools without reaching through `runtime.tool().execute` or duplicating connector discovery:

```ts
server.registerTool("search", searchOptions, async ({ query }) =>
toMcpResult(await runtime.search(query))
);

server.registerTool("execute", executeOptions, async ({ code }) =>
toMcpResult(await runtime.execute({ code }))
);
```

Search and describe results mark methods with `requiresApproval: true` when their connector annotation requires it.

## The sandbox API (`codemode.*`)

The runtime also provides the model's API. Inside the sandbox, `codemode` is a global with four methods — discover, learn, do-once, reuse:
Expand Down
12 changes: 12 additions & 0 deletions packages/codemode/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -220,6 +220,18 @@ const result = streamText({
});
```

For non-AI-SDK hosts such as MCP servers, use the same runtime directly:

```ts
const matches = await runtime.search("create issue");
const docs = await runtime.describe(matches.results[0].path);
const outcome = await runtime.execute({
code: `async () => github.create_issue({ title: "Bug" })`
});
```

Search and describe results include `requiresApproval: true` for protected connector methods. A paused `execute()` outcome can be resolved with the same `approve()` and `reject()` methods used by an approval UI.

Inside the sandbox, each connector is available as a global named after the connector. The `codemode` platform SDK provides discovery:

```ts
Expand Down
3 changes: 3 additions & 0 deletions packages/codemode/src/connectors/describe.ts
Original file line number Diff line number Diff line change
Expand Up @@ -81,6 +81,9 @@ export function describeTarget(
return {
path: `${candidate.name}.${methodName}`,
description: candidate.descriptors[methodName]?.description,
...(candidate.annotations?.[methodName]?.requiresApproval
? { requiresApproval: true }
: {}),
types: renderMethodTypes(methodName, candidate.descriptors),
kind: "method"
};
Expand Down
5 changes: 5 additions & 0 deletions packages/codemode/src/connectors/search.ts
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ type SearchableItem = {
connector: string;
method: string;
description?: string;
requiresApproval?: boolean;
kind: "method" | "snippet";
};

Expand Down Expand Up @@ -153,6 +154,7 @@ function scoreMatch(item: SearchableItem, query: string): SearchResult | null {
connector: item.connector,
method: item.method,
description: item.description,
...(item.requiresApproval ? { requiresApproval: true } : {}),
kind: item.kind,
score
};
Expand All @@ -172,6 +174,9 @@ export function searchConnectors(
connector: desc.name,
method: methodName,
description: descriptor?.description,
...(desc.annotations?.[methodName]?.requiresApproval
? { requiresApproval: true }
: {}),
kind: "method"
});
}
Expand Down
4 changes: 4 additions & 0 deletions packages/codemode/src/connectors/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,6 +79,8 @@ export type SearchResult = {
connector: string;
method: string;
description?: string;
/** Whether invoking this method pauses for approval. */
requiresApproval?: boolean;
kind: "method" | "snippet";
score: number;
};
Expand All @@ -96,6 +98,8 @@ export type SearchOutput = {
export type DescribeOutput = {
path: string;
description?: string;
/** Whether invoking this method pauses for approval. */
requiresApproval?: boolean;
types: string;
kind: "connector" | "method" | "snippet";
};
46 changes: 45 additions & 1 deletion packages/codemode/src/runtime-handle.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,11 @@
import type { CodemodeConnector } from "./connectors";
import {
describeTarget,
searchConnectors,
type CodemodeConnector,
type ConnectorDescription,
type DescribeOutput,
type SearchOutput
} from "./connectors";
import type { Executor } from "./executor";
import {
createProxyTool,
Expand All @@ -11,6 +18,7 @@ import {
rollbackCodemode,
validateConnectorNames,
type CodemodeTool,
type ProxyToolInput,
type ProxyToolOutput,
type TransformResult
} from "./proxy-tool";
Expand Down Expand Up @@ -80,7 +88,14 @@ export type CodemodeExpireOptions = {
};

export interface CodemodeRuntimeHandle {
/** Create the AI SDK-compatible model-facing Code Mode tool. */
tool(options?: CodemodeRuntimeToolOptions): CodemodeTool;
/** Execute code directly, without adapting the runtime to an AI SDK tool. */
execute(input: ProxyToolInput): Promise<ProxyToolOutput>;
/** Search connector methods and saved snippets. */
search(query: string): Promise<SearchOutput>;
/** Get on-demand TypeScript documentation for a connector method or snippet. */
describe(target: string): Promise<DescribeOutput>;
approve(options: CodemodeApproveOptions): Promise<ProxyToolOutput>;
/**
* Reject a pending action, ending the run. Returns whether the reject
Expand Down Expand Up @@ -121,6 +136,8 @@ export function createCodemodeRuntime(

class DefaultCodemodeRuntimeHandle implements CodemodeRuntimeHandle {
#options: CreateCodemodeRuntimeOptions;
#executionTool?: CodemodeTool;
#descriptionsPromise?: Promise<ConnectorDescription[]>;

constructor(options: CreateCodemodeRuntimeOptions) {
validateConnectorNames(options.connectors);
Expand All @@ -140,6 +157,27 @@ class DefaultCodemodeRuntimeHandle implements CodemodeRuntimeHandle {
});
}

execute(input: ProxyToolInput): Promise<ProxyToolOutput> {
this.#executionTool ??= this.tool();
return this.#executionTool.execute(input, undefined);
}

async search(query: string): Promise<SearchOutput> {
const [descriptions, snippets] = await Promise.all([
this.#descriptions(),
this.snippets()
]);
return searchConnectors(query, descriptions, snippets);
}

async describe(target: string): Promise<DescribeOutput> {
const [descriptions, snippets] = await Promise.all([
this.#descriptions(),
this.snippets()
]);
return describeTarget(target, descriptions, snippets);
}

approve(options: CodemodeApproveOptions): Promise<ProxyToolOutput> {
return resumeCodemode({
ctx: this.#options.ctx,
Expand Down Expand Up @@ -224,6 +262,12 @@ class DefaultCodemodeRuntimeHandle implements CodemodeRuntimeHandle {
return this.#runtime().deleteSnippet(name);
}

#descriptions(): Promise<ConnectorDescription[]> {
return (this.#descriptionsPromise ??= Promise.all(
this.#options.connectors.map((connector) => connector.describe())
));
}

#runtime() {
return getCodemodeRuntime(this.#options.ctx, this.#options.name);
}
Expand Down
83 changes: 83 additions & 0 deletions packages/codemode/src/tests/runtime-handle.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -103,6 +103,89 @@ describe("createCodemodeRuntime", () => {
});
});

it("executes directly without an AI SDK adapter", async () => {
const runtimeStub = {
begin: vi.fn(async () => "exec_direct"),
getExecution: vi.fn(async () => null),
complete: vi.fn(async () => undefined)
};
const executor = createMockExecutor("direct result");
const runtime = createCodemodeRuntime({
ctx: createMockCtx(runtimeStub),
executor,
connectors: []
});

await expect(runtime.execute({ code: "async () => 42" })).resolves.toEqual({
status: "completed",
executionId: "exec_direct",
result: "direct result",
logs: undefined
});
expect(executor.execute).toHaveBeenCalled();
});

it("searches and describes connectors without running sandbox code", async () => {
const runtimeStub = {
listSnippets: vi.fn(async () => [
{
name: "triage",
description: "Triage open issues",
code: "async () => []",
savedAt: 1
}
])
};
const describe = vi.fn(async () => ({
name: "github",
instructions: "GitHub issues",
descriptors: {
list_issues: {
description: "List repository issues",
inputSchema: { type: "object" }
},
create_issue: {
description: "Create a repository issue",
inputSchema: {
type: "object",
properties: { title: { type: "string" } },
required: ["title"]
}
}
},
annotations: { create_issue: { requiresApproval: true } }
}));
const connector = {
name: () => "github",
describe
} as unknown as import("../connectors").CodemodeConnector;
const runtime = createCodemodeRuntime({
ctx: createMockCtx(runtimeStub),
executor: createMockExecutor(),
connectors: [connector]
});

await expect(runtime.search("create issue")).resolves.toMatchObject({
results: [
{
path: "github.create_issue",
kind: "method",
requiresApproval: true
}
]
});
await expect(
runtime.describe("github.create_issue")
).resolves.toMatchObject({
path: "github.create_issue",
description: "Create a repository issue",
requiresApproval: true,
kind: "method"
});
expect(describe).toHaveBeenCalledTimes(1);
expect(runtimeStub.listSnippets).toHaveBeenCalledTimes(2);
});

it("executes as a plain tool through AI SDK streamText", async () => {
const runtimeStub = {
begin: vi.fn(async () => "exec_1"),
Expand Down
Loading