feat: OpenAI function calling for client tools – with hard execution lockdown

[dobexx]

What

Implements the full client-side function-calling roundtrip (OpenAI tools array → Claude → tool_calls → client executes → tool result → final answer). This makes the proxy work with OpenWebUI tools/functions, MCP toolchains in IDEs, and any OpenAI-compatible agent client.

Since Claude Code has no client-executed tool protocol in --print mode, tool definitions are rendered into the prompt with a strict <tool_call> JSON envelope convention, and responses containing the envelope are converted into proper OpenAI tool_calls (finish_reason: "tool_calls") – streaming and non-streaming. While tools are active, streamed text is buffered so the raw envelope never leaks into the client UI. role="tool" messages are accepted and rendered as <tool_result> context for the follow-up turn.

⚠️ The security angle (why this PR matters beyond the feature)

Today, any client that can reach the proxy can make Claude execute arbitrary commands on the host – the CLI runs with --dangerously-skip-permissions, and its full toolbox (Bash, Read, Write, Edit, WebSearch, …) is active by default. Whoever holds the (previously non-existent) API key effectively gets a remote shell in the container.

This PR changes the trust model:

Request	Behavior
With tools array	--tools "" disables all built-in CLI tools. Claude can only request tool calls; execution happens exclusively in the client (OpenWebUI, IDE, …). Zero local execution.
Without tools	Unchanged (CLI tools active – needed e.g. for reading staged images)

The lockdown is automatic and per-request – no configuration needed, no way for a client to escalate into host execution through the function-calling path.

Tested

End-to-end in production: OpenWebUI (tool: terminal command function) → proxy → Claude Opus 5 → tool_calls → OpenWebUI executes → result fed back → final natural-language answer. Envelope never visible in the UI.

Known limitation

The envelope convention is prompt-based, not native function calling – models can occasionally break format under heavy multi-tool scenarios (falls back to plain text, never to local execution). Works reliably for typical client toolchains.

Note

Based on current main; if you also take #15 (auth/admin) and #16 (vision/effort/cache), I'll rebase to resolve the small overlaps in routes.ts/manager.ts.

[sourcery-ai]

Hey - I've found 2 issues, and left some high level feedback:

• The AUTH_EXPIRED_MESSAGE and default model "claude-sonnet-4" are hardcoded in multiple places; consider centralizing these into a shared config/defaults so they can be localized and updated without touching the routing logic.
• parseToolCall assumes a single <tool_call> JSON block per response and ignores malformed/multiple envelopes; if clients may send more complex tool_call payloads, you might want more robust parsing and clearer failure behavior instead of silently falling back to plain text.
• stageImages and cleanupImages currently swallow all errors silently; adding minimal structured logging (e.g., when a download exceeds MAX_IMAGE_BYTES or write fails) would make diagnosing image-related issues much easier without impacting normal flow.

Prompt for AI Agents

Please address the comments from this code review:

## Overall Comments
- The AUTH_EXPIRED_MESSAGE and default model "claude-sonnet-4" are hardcoded in multiple places; consider centralizing these into a shared config/defaults so they can be localized and updated without touching the routing logic.
- parseToolCall assumes a single <tool_call> JSON block per response and ignores malformed/multiple envelopes; if clients may send more complex tool_call payloads, you might want more robust parsing and clearer failure behavior instead of silently falling back to plain text.
- stageImages and cleanupImages currently swallow all errors silently; adding minimal structured logging (e.g., when a download exceeds MAX_IMAGE_BYTES or write fails) would make diagnosing image-related issues much easier without impacting normal flow.

## Individual Comments

### Comment 1
<location path="src/server/routes.ts" line_range="489-490" />
<code_context>
+        }
+      }
+      // No tool call detected - flush the buffered text as regular content
+      if (expectsClientTools && accumulatedText && !res.writableEnded) {
+        const textChunk = {
+          id: `chatcmpl-${requestId}`,
+          object: "chat.completion.chunk",
</code_context>
<issue_to_address>
**issue (bug_risk):** Mark that content was emitted when flushing buffered tool text so the final finish_reason is correct.

In the `expectsClientTools` branch, flushing `accumulatedText` as a regular content chunk does not update `hasEmittedText`. This means the final done chunk can still have `finish_reason: null` even though content was sent. Set `hasEmittedText = true` when emitting `textChunk` so `createDoneChunk` returns `finish_reason: "stop"` correctly.
</issue_to_address>

### Comment 2
<location path="src/subprocess/manager.ts" line_range="106-115" />
<code_context>
+      let mime = img.mimeType;
+
+      if (img.sourceUrl) {
+        const res = await fetch(img.sourceUrl, {
+          signal: AbortSignal.timeout(5000),
+        });
+        if (!res.ok) continue;
+        const contentLength = Number(res.headers.get("content-length") || 0);
+        if (contentLength > MAX_IMAGE_BYTES) continue;
+        mime = mime || res.headers.get("content-type") || "";
+        buffer = Buffer.from(await res.arrayBuffer());
+      } else {
+        buffer = Buffer.from(img.data, "base64");
</code_context>
<issue_to_address>
**🚨 suggestion (security):** Guard remote image downloads against unbounded memory usage while streaming.

For responses lacking a reliable `content-length`, this code still reads the entire body into memory via `arrayBuffer()` before applying `MAX_IMAGE_BYTES`, which allows very large files to be downloaded and can lead to memory exhaustion. Please enforce the size limit while streaming (abort once the cumulative bytes exceed `MAX_IMAGE_BYTES`) or skip downloads entirely when `content-length` is unavailable if a `disableRemoteImages`-style behavior is acceptable.

```suggestion
      if (img.sourceUrl) {
        const res = await fetch(img.sourceUrl, {
          signal: AbortSignal.timeout(5000),
        });
        if (!res.ok) continue;

        const contentLengthHeader = res.headers.get("content-length");
        if (!contentLengthHeader) continue;

        const contentLength = Number(contentLengthHeader);
        if (!Number.isFinite(contentLength) || contentLength <= 0 || contentLength > MAX_IMAGE_BYTES) continue;

        mime = mime || res.headers.get("content-type") || "";
        buffer = Buffer.from(await res.arrayBuffer());
      } else {
```
</issue_to_address>

---

Sourcery is free for open source - if you like our reviews please consider sharing them ✨

• X
• Mastodon
• LinkedIn
• Facebook

_{Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.}

[dobexx]

Thanks for the review – addressed in the latest commits:

1. hasEmittedText on buffered flush: now set (together with isFirst = false) when the buffered client-tool text flushes, so the final chunk carries the correct finish_reason.
2. Unbounded image downloads: the size limit is now enforced while streaming the body – the download aborts as soon as it exceeds MAX_IMAGE_BYTES, instead of buffering first and checking later.

On the high-level notes (centralizing AUTH_EXPIRED_MESSAGE / default model): agreed as a follow-up cleanup; kept as-is here to keep the diff focused.

@sourcery-ai review

[Saimonokuma]

Is fab and Op , with V supported?

[dobexx]

@Saimonokuma Yes – both. The proxy normalizes model names but keeps the major version, so claude-opus-5 (and Fable 5) work as requested models; verified live with Opus 5.

Vision is supported via image_url content blocks (base64 data URLs or remote http(s) URLs): images are staged as temp files and Claude views them via its Read tool. Tested end-to-end with OpenWebUI + claude-opus-5.

Note: vision itself ships in #16 (this PR stacks the client-tool function calling on top).