Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
22 changes: 22 additions & 0 deletions src/crates/services/services-integrations/src/speech/realtime.rs
Original file line number Diff line number Diff line change
Expand Up @@ -597,6 +597,28 @@ fn session_create_payload(
"loudness": config.loudness,
},
},
// Architecture boundary for future agents:
//
// This is the provider-hosted Voice model's client control-plane
// tool list. It is intentionally independent from the tool registry
// assembled for a normal BitFun Agent session. In particular,
// `run_bitfun_task` delegates one complete user intent to a newly
// created Agent session; Voice never receives, mirrors, or proxies
// that session's filesystem, terminal, MCP, browser, or other tools.
// The delegated session resolves its own tools and permissions in
// the normal workspace execution path.
//
// Add a Voice tool only for a direct client-level operation that
// cannot be expressed as an Agent task. A new Voice tool requires
// coordinated changes in all of these places:
// 1. its provider schema below;
// 2. `VoiceFunctionCommand`, `parseFunctionCall`, and
// `handleFunctionCall` in
// `src/web-ui/src/flow_chat/components/voice/useRealtimeVoiceCall.ts`;
// 3. focused Rust payload-contract and Web UI dispatch tests.
// Workspace execution capabilities belong in the normal Agent tool
// registry and become available to delegated tasks without being
// copied into this list.
"tools": [
{
"type": "function",
Expand Down
22 changes: 22 additions & 0 deletions src/web-ui/src/flow_chat/components/voice/useRealtimeVoiceCall.ts
Original file line number Diff line number Diff line change
Expand Up @@ -82,6 +82,22 @@ function silentPcm16Base64(sampleRate: number): string {
return window.btoa(binary);
}

/**
* Client control-plane commands exposed to the provider-hosted Voice model.
*
* This union mirrors the client tool schemas in
* `src/crates/services/services-integrations/src/speech/realtime.rs`; it is not
* the workspace Agent tool contract. `run_task` crosses that boundary by
* delegating a complete request to a normal Agent session. The delegated Agent
* independently resolves its filesystem, terminal, MCP, browser, and other
* tools together with their usual permission policy.
*
* Extension rule for future agents:
* - Add direct BitFun client operations here and in the Rust Voice schema.
* - Add workspace execution abilities to the normal Agent tool registry; do
* not mirror individual Agent tools into Voice.
* - Keep the parser, dispatcher, provider schema, and focused tests in sync.
*/
type VoiceFunctionCommand =
| { kind: 'get_client_context' }
| { kind: 'switch_workspace'; workspaceReference: string }
Expand Down Expand Up @@ -362,6 +378,12 @@ export function useRealtimeVoiceCallController(disabled = false): RealtimeVoiceC
setAudioLevel(0);
}, [clearAssistantSpeechFallbackTimer]);

/**
* Dispatches Voice client commands only. It may read or mutate client state,
* create one normal Agent session, or cancel the task owned by this call. It
* must not become a second executor for workspace Agent tools; task-level
* delegation and lifecycle events are the boundary between the two systems.
*/
const handleFunctionCall = useCallback(async (
callSessionId: string,
call: SpeechRealtimeFunctionCall,
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,8 @@ function workspaceForSession(
* Build a compact, public snapshot for the realtime model. This contains only
* navigation/session facts already visible in the controller UI; it excludes
* message contents, tool payloads, credentials, and private Agent reasoning.
* This is context data for the Voice control plane, not a workspace Agent tool
* registry. Do not add Agent tool schemas or execution capabilities here.
*/
export function buildVoiceClientContext(voiceTask: VoiceOwnedTaskContext | null = null) {
const workspaceState = workspaceManager.getState();
Expand Down
15 changes: 15 additions & 0 deletions src/web-ui/src/flow_chat/components/voice/voiceTaskBridge.ts
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,21 @@ async function waitForSettledSession(sessionId: string): Promise<void> {
});
}

/**
* Delegation boundary between client-level Voice and workspace execution.
*
* This creates a regular `agentic` FlowChat session. That session follows the
* same product assembly, workspace adapters, tool registry, plugin/MCP setup,
* and permission flow as a task started from the normal UI. Voice supplies only
* the complete task intent and target workspace, then observes progress,
* cancellation, and the final public result.
*
* Do not copy the workspace Agent's tools into the Voice provider session and
* do not proxy individual Agent tool calls through this bridge. If a new
* workspace capability should be usable from Voice, implement it in the normal
* Agent execution path; delegated sessions will inherit it automatically. Only
* direct client-control operations need a new Voice function command.
*/
export async function runBitFunVoiceTask(
task: string,
options: RunVoiceTaskOptions,
Expand Down
Loading