diff --git a/api-reference/server/frames/data-frames.mdx b/api-reference/server/frames/data-frames.mdx index 48ca7a4e..bfb83cb0 100644 --- a/api-reference/server/frames/data-frames.mdx +++ b/api-reference/server/frames/data-frames.mdx @@ -327,10 +327,15 @@ Frames for configuring LLM function calling behavior and output settings at runt ### LLMSetToolsFrame -Sets the available tools for LLM function calling. The format of tool definitions typically follows JSON Schema conventions, though the exact structure depends on the LLM provider. - - - List of tool/function definitions for the LLM. +Changes the set of tools advertised to the LLM mid-conversation. + + + The tools to advertise. May be a `ToolsSchema`, a plain list of direct + functions and/or `FunctionSchema` objects, a list of provider-specific tool + dicts, or `NOT_GIVEN` to clear all tools. Direct functions and + `FunctionSchema`s with a bundled `handler` are auto-registered with the LLM + service, meaning no manual handler registration is needed. Any such tool + dropped from the advertised set is automatically unregistered. ### LLMSetToolChoiceFrame diff --git a/api-reference/server/services/llm/inception.mdx b/api-reference/server/services/llm/inception.mdx index a7effe33..49441cbe 100644 --- a/api-reference/server/services/llm/inception.mdx +++ b/api-reference/server/services/llm/inception.mdx @@ -120,10 +120,17 @@ llm = InceptionLLMService( ### With Function Calling ```python +from pipecat.processors.aggregators.llm_context import LLMContext from pipecat.services.inception import InceptionLLMService from pipecat.services.llm_service import FunctionCallParams -async def get_weather(params: FunctionCallParams): +# A direct function: schema is auto-derived from the signature and docstring +async def get_weather(params: FunctionCallParams, location: str): + """Get the current weather. + + Args: + location: The city and state, e.g. "San Francisco, CA". + """ await params.result_callback({"temperature": "75", "conditions": "sunny"}) llm = InceptionLLMService( @@ -133,7 +140,7 @@ llm = InceptionLLMService( ), ) -llm.register_function("get_weather", get_weather) +context = LLMContext(tools=[get_weather]) ``` ## Notes diff --git a/api-reference/server/services/llm/novita.mdx b/api-reference/server/services/llm/novita.mdx index be5bee5a..409dd21a 100644 --- a/api-reference/server/services/llm/novita.mdx +++ b/api-reference/server/services/llm/novita.mdx @@ -110,12 +110,16 @@ llm = NovitaLLMService( ### With Function Calling ```python -from pipecat.adapters.schemas.function_schema import FunctionSchema -from pipecat.adapters.schemas.tools_schema import ToolsSchema from pipecat.processors.aggregators.llm_context import LLMContext from pipecat.services.llm_service import FunctionCallParams -async def get_weather(params: FunctionCallParams): +# A direct function: schema is auto-derived from the signature and docstring +async def get_weather(params: FunctionCallParams, location: str): + """Get the current weather. + + Args: + location: The city and state, e.g. "San Francisco, CA". + """ await params.result_callback({"temperature": "75", "conditions": "sunny"}) llm = NovitaLLMService( @@ -125,22 +129,7 @@ llm = NovitaLLMService( ), ) -llm.register_function("get_weather", get_weather) - -weather_function = FunctionSchema( - name="get_weather", - description="Get the current weather", - properties={ - "location": { - "type": "string", - "description": "City and state, e.g. San Francisco, CA", - }, - }, - required=["location"], -) - -tools = ToolsSchema(standard_tools=[weather_function]) -context = LLMContext(tools=tools) +context = LLMContext(tools=[get_weather]) ``` ## Notes diff --git a/api-reference/server/utilities/service-switchers/llm-switcher.mdx b/api-reference/server/utilities/service-switchers/llm-switcher.mdx index d4607a58..52d6f4bc 100644 --- a/api-reference/server/utilities/service-switchers/llm-switcher.mdx +++ b/api-reference/server/utilities/service-switchers/llm-switcher.mdx @@ -69,6 +69,12 @@ result = await llm_switcher.run_inference( ### register_function() + + You don't need `register_function` when your tool's handler is bundled on its + `FunctionSchema`: listing it in `LLMContext(tools=[...])` or adding it via + `LLMSetToolsFrame` is enough. + + Register a function handler with all LLMs in the switcher, regardless of which is currently active. ```python @@ -103,7 +109,15 @@ llm_switcher.register_function( ### register_direct_function() -Register a direct function handler with all LLMs in the switcher, regardless of which is currently active. Direct functions provide more control over function call execution. + + **Deprecated since 1.4.0.** Direct functions now register automatically on + every member LLM (active or not) when you list them in + `LLMContext(tools=[...])`, so explicit registration is no longer needed — the + tools keep working across service switches. Push an `LLMSetToolsFrame` to + change tools mid-session. This method will be removed in a future version. + + +Register a direct function handler with all LLMs in the switcher, regardless of which is currently active. ```python llm_switcher.register_direct_function( diff --git a/pipecat/fundamentals/context-summarization.mdx b/pipecat/fundamentals/context-summarization.mdx index 5dffa193..ded8e455 100644 --- a/pipecat/fundamentals/context-summarization.mdx +++ b/pipecat/fundamentals/context-summarization.mdx @@ -137,31 +137,15 @@ from pipecat.frames.frames import LLMSummarizeContextFrame from pipecat.services.llm_service import FunctionCallParams async def summarize_conversation(params: FunctionCallParams): - """Trigger manual context summarization via a pipeline frame.""" + """Summarize and compress the conversation history. Call this when the user asks you to summarize the conversation or when you want to free up context space.""" await params.result_callback({"status": "summarization_requested"}) await params.llm.queue_frame(LLMSummarizeContextFrame()) ``` -Register this as a function call tool so the LLM can invoke it when the user asks to summarize: +Above, `summarize_conversation` is a [direct function](/pipecat/learn/function-calling#1-define-a-tool). List it in the context's `tools` so the LLM can invoke it when the user asks to summarize: ```python -from pipecat.adapters.schemas.function_schema import FunctionSchema -from pipecat.adapters.schemas.tools_schema import ToolsSchema - -llm.register_function("summarize_conversation", summarize_conversation) - -summarize_function = FunctionSchema( - name="summarize_conversation", - description=( - "Summarize and compress the conversation history. " - "Call this when the user asks you to summarize the conversation " - "or when you want to free up context space." - ), - properties={}, - required=[], -) -tools = ToolsSchema(standard_tools=[summarize_function]) -context = LLMContext(messages, tools=tools) +context = LLMContext(messages, tools=[summarize_conversation]) ``` On-demand summarization works even when `enable_auto_context_summarization` is `False` — the summarizer is always created internally to handle manually pushed frames. You can also pass a per-request `LLMContextSummaryConfig` to override the default settings: diff --git a/pipecat/learn/context-management.mdx b/pipecat/learn/context-management.mdx index b5049705..fa297245 100644 --- a/pipecat/learn/context-management.mdx +++ b/pipecat/learn/context-management.mdx @@ -85,40 +85,28 @@ user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context) Key properties: - **`messages`**: List of conversation messages (user, assistant, developer, tool) -- **`tools`**: Optional `ToolsSchema` defining available functions +- **`tools`**: Optional available functions — a list of direct functions and/or `FunctionSchema` objects (or a `ToolsSchema`) - **`tool_choice`**: Optional strategy for tool selection ### 2. Context with Function Calling -Context can also include tools (function definitions) that the LLM can call during conversations: +Context can also include [tools](/pipecat/learn/function-calling#1-define-a-tool) (function definitions) that the LLM can call during conversations: ```python -from pipecat.adapters.schemas.function_schema import FunctionSchema -from pipecat.adapters.schemas.tools_schema import ToolsSchema - -# Define available functions -weather_function = FunctionSchema( - name="get_current_weather", - description="Get the current weather", - properties={ - "location": { - "type": "string", - "description": "The city and state, e.g. San Francisco, CA", - }, - "format": { - "type": "string", - "enum": ["celsius", "fahrenheit"], - "description": "The temperature unit to use.", - }, - }, - required=["location", "format"], -) +from pipecat.services.llm_service import FunctionCallParams + +# A direct function: schema is auto-derived from the signature and docstring +async def get_current_weather(params: FunctionCallParams, location: str, format: str): + """Get the current weather. -# Create tools schema -tools = ToolsSchema(standard_tools=[weather_function]) + Args: + location: The city and state, e.g. "San Francisco, CA". + format: The temperature unit to use. Must be either "celsius" or "fahrenheit". + """ + await params.result_callback({"conditions": "sunny", "temperature": "75"}) # Create context with both messages and tools -context = LLMContext(messages, tools) +context = LLMContext(messages, tools=[get_current_weather]) user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context) ``` diff --git a/pipecat/learn/function-calling.mdx b/pipecat/learn/function-calling.mdx index 0dd85302..0aea2196 100644 --- a/pipecat/learn/function-calling.mdx +++ b/pipecat/learn/function-calling.mdx @@ -45,7 +45,7 @@ For example, you could give your bot the ability to: Here's how it works: -1. You define functions the LLM can use and register them to the LLM service used in your pipeline +1. You define functions the LLM can use and make them available to the LLM service used in your pipeline 2. When needed, the LLM requests a function call 3. Your application executes any corresponding functions 4. The result is sent back to the LLM @@ -53,19 +53,169 @@ Here's how it works: ## Implementation -### 1. Define Functions +### 1. Define a tool -Pipecat provides a standardized `FunctionSchema` that works across all supported LLM providers. This makes it easy to define functions once and use them with any provider. +A tool needs two things: a handler — the code to run when the LLM calls the tool — and a schema that describes the tool to the LLM (its name, what it does, and its parameters) so the model knows it exists and how to call it. -As a shorthand, you could also bypass specifying a function configuration at all and instead use "direct" functions. Under the hood, these are converted to `FunctionSchema`s. +The preferred way to define a tool is with a **direct function**: a single async function that is _both_ the handler and the schema. Pipecat auto-derives the tool's metadata — name, description, parameter properties (with their descriptions), and which parameters are required — from the function's signature and docstring. -#### Using the Standard Schema (Recommended) +The first parameter is always `params` (a `FunctionCallParams`); the tool's own arguments follow. Document each argument in a Google-style docstring. + +```python +from pipecat.services.llm_service import FunctionCallParams + +async def get_current_weather(params: FunctionCallParams, location: str, format: str): + """Get the current weather. + + Args: + location: The city and state, e.g. "San Francisco, CA". + format: The temperature unit to use. Must be either "celsius" or "fahrenheit". Infer this from the user's location. + """ + weather_data = {"conditions": "sunny", "temperature": "75"} + await params.result_callback(weather_data) +``` + + + The direct-function schema generator doesn't yet map `Literal` types to a + JSON-schema `enum`. Express enum-like constraints in the docstring prose + instead (e.g. _'Must be either "celsius" or "fahrenheit"'_), as shown above. + If you need a strict `enum` in the schema, use the [verbose + `FunctionSchema`](#advanced-defining-tools-with-functionschema) pattern. + + +### 2. Add the tool to the context + +List your direct functions in `LLMContext(tools=[...])`: + +```python +from pipecat.processors.aggregators.llm_context import LLMContext +from pipecat.processors.aggregators.llm_response_universal import LLMContextAggregatorPair + +context = LLMContext(tools=[get_current_weather, get_restaurant_recommendation]) +user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context) +``` + +The bot's personality (e.g. "You are a helpful assistant") is set via [`system_instruction`](/pipecat/learn/context-management#using-system_instruction-recommended) in the LLM service's Settings, not as a context message. Tools are automatically converted to the correct format for your LLM provider through adapters. + +### 3. Create the pipeline + +Include your LLM service in the pipeline: + +```python +# Create the pipeline +pipeline = Pipeline([ + transport.input(), # Input from the transport + stt, # STT processing + user_aggregator, # User context aggregation + llm, # LLM processing + tts, # TTS processing + transport.output(), # Output to the transport + assistant_aggregator, # Assistant context aggregation +]) +``` + +## Per-Tool Options with `@tool_options` + +By default, a direct function is cancelled if the user interrupts, and it uses the LLM service's global timeout. To override either, decorate the function with `@tool_options`. The decorator only attaches call options — the schema is still auto-derived — so decorated functions can stay at module level. + +```python +import asyncio + +from pipecat.adapters.schemas.direct_function import tool_options +from pipecat.services.llm_service import FunctionCallParams + +@tool_options(cancel_on_interruption=False, timeout_secs=30) +async def get_current_weather(params: FunctionCallParams, location: str, format: str): + """Get the current weather. + + Args: + location: The city and state, e.g. "San Francisco, CA". + format: The temperature unit to use. Must be either "celsius" or "fahrenheit". Infer this from the user's location. + """ + # Simulate a long-running API call. + await asyncio.sleep(20) + await params.result_callback({"conditions": "nice", "temperature": "75"}) +``` + +**Options:** + +- **`cancel_on_interruption`** (default `True`): When `True`, the call is cancelled if the user interrupts. When `False`, the call is treated as **asynchronous** — see below. +- **`timeout_secs`** (default `None`): Per-tool timeout in seconds. Overrides the global `function_call_timeout_secs` for this function. Use a longer timeout for slow operations (e.g. database queries) or a shorter one for quick lookups. + + + `@tool_options` also sets call options on the handler of a + [`FunctionSchema`](#advanced-defining-tools-with-functionschema) tool, not + just a direct function. + + + + On an [`LLMWorker`](/api-reference/server/workers/llm-context-worker), mark + tool methods with `@tool` instead. It applies the same options _and_ marks the + method for automatic collection as one of the worker's own tools. + + +### Synchronous vs. asynchronous calls + +With `cancel_on_interruption=True` (the default), the call is **synchronous**: the LLM waits for the result before generating its next response. This ensures the LLM has complete information before responding. + +With `cancel_on_interruption=False`, the call is **asynchronous**: the LLM continues the conversation immediately without waiting. Once the result returns, it's injected back into the context as a developer message, triggering a new LLM inference at that point. This enables truly non-blocking calls where the conversation proceeds while the function runs in the background. Async calls can also send [intermediate updates](#intermediate-results-for-async-functions) before their final result. + +#### Async function call cancellation + +For async functions (`cancel_on_interruption=False`), you can also enable model-directed cancellation: + +```python +llm = OpenAILLMService( + api_key="your-api-key", + enable_async_tool_cancellation=True, +) +``` + +When `enable_async_tool_cancellation=True` and at least one async function is available, Pipecat automatically adds the built-in `cancel_async_tool_call` tool and supporting system instructions. The LLM can call that tool to cancel a stale in-progress async function call — for example, when the user changes their request before a long-running lookup completes. + +## Changing Tools Mid-Conversation + +To change the set of tools the LLM can use during a session, push an `LLMSetToolsFrame`. Its `tools` field takes the same things as `LLMContext(tools=[...])` — a list of direct functions and/or `FunctionSchema` objects. Whatever you pass becomes the LLM's new tool set. + +```python +from pipecat.frames.frames import LLMSetToolsFrame +from pipecat.processors.aggregators.llm_context import NOT_GIVEN + +# Make get_current_weather the only tool the LLM can call +await worker.queue_frame(LLMSetToolsFrame(tools=[get_current_weather])) + +# Clear all tools +await worker.queue_frame(LLMSetToolsFrame(tools=NOT_GIVEN)) +``` + +## Tools Across Service Switches + +When you use an [`LLMSwitcher`](/api-reference/server/utilities/service-switchers/llm-switcher) to swap LLM providers mid-session, the tools you list in `LLMContext(tools=[...])` are available on whichever provider is active. You define them once for the whole switcher. + +```python +from pipecat.pipeline.llm_switcher import LLMSwitcher + +llm_switcher = LLMSwitcher(llms=[llm_openai, llm_google]) + +# Tools in the context are available on whichever provider is active +context = LLMContext(tools=[get_current_weather, get_restaurant_recommendation]) +user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context) +``` + +## Advanced: Defining Tools with `FunctionSchema` + +Direct functions cover most cases. Reach for the verbose `FunctionSchema` pattern when you need explicit control over the schema — for example, a strict `enum` constraint (which the direct-function generator doesn't yet emit) — or when the tool's handler isn't shaped like a direct function. + +A `FunctionSchema` spells out the tool's name, description, and parameters by hand. Pass the `handler` that runs when the LLM calls the tool as the schema's `handler`, then list the schema in `LLMContext(tools=[...])` — exactly as you would a direct function. ```python from pipecat.adapters.schemas.function_schema import FunctionSchema -from pipecat.adapters.schemas.tools_schema import ToolsSchema +from pipecat.services.llm_service import FunctionCallParams + +async def fetch_weather_from_api(params: FunctionCallParams): + weather_data = {"conditions": "sunny", "temperature": "75"} + await params.result_callback(weather_data) -# Define a function using the standard schema weather_function = FunctionSchema( name="get_current_weather", description="Get the current weather in a location", @@ -76,67 +226,48 @@ weather_function = FunctionSchema( }, "format": { "type": "string", + # A strict enum — the kind of explicit control a direct function can't yet express. "enum": ["celsius", "fahrenheit"], "description": "The temperature unit to use.", }, }, - required=["location", "format"] + required=["location", "format"], + handler=fetch_weather_from_api, # bundle the handler on the schema ) -# Create a tools schema with your functions -tools = ToolsSchema(standard_tools=[weather_function]) - -# Pass this to your LLM context -context = LLMContext(tools=tools) -user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context) +# List the schema in the context, just like a direct function. +context = LLMContext(tools=[weather_function]) ``` -The bot's personality (e.g. "You are a helpful assistant") is set via [`system_instruction`](/pipecat/learn/context-management#using-system_instruction-recommended) in the LLM service's Settings, not as a context message. The `ToolsSchema` will be automatically converted to the correct format for your LLM provider through adapters. - -#### Using Direct Functions (Shorthand) - -You can bypass specifying a function configuration as a `FunctionSchema` and instead pass the function directly to your `ToolsSchema`. Pipecat will auto-configure the function, gathering relevant metadata from its signature and docstring. Metadata includes: - -- name -- description -- properties (including individual property descriptions) -- list of required properties - -Note that the function signature is a bit different when using direct functions. The first parameter is `FunctionCallParams`, followed by any others necessary for the function. +To override the handler's default call options, decorate it with [`@tool_options`](#per-tool-options-with-@tool_options) — the same decorator direct functions use, with the same [synchronous vs. asynchronous](#synchronous-vs-asynchronous-calls) semantics: ```python -from pipecat.adapters.schemas.tools_schema import ToolsSchema -from pipecat.services.llm_service import FunctionCallParams +from pipecat.adapters.schemas.direct_function import tool_options -# Define a direct function -async def get_current_weather(params: FunctionCallParams, location: str, format: str): - """Get the current weather. +@tool_options(cancel_on_interruption=False, timeout_secs=30) +async def fetch_weather_from_api(params: FunctionCallParams): + ... +``` - Args: - location: The city and state, e.g. "San Francisco, CA". - format: The temperature unit to use. Must be either "celsius" or "fahrenheit". - """ - weather_data = {"conditions": "sunny", "temperature": "75"} - await params.result_callback(weather_data) +These schemas behave just like direct functions everywhere else in this guide — swap them mid-conversation with an `LLMSetToolsFrame`, and they keep working across an `LLMSwitcher`'s providers. -# Create a tools schema, passing your function directly to it -tools = ToolsSchema(standard_tools=[get_current_weather]) +### Registering a handler manually -# Pass this to your LLM context -context = LLMContext(tools=tools) -user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context) +Bundling the handler on the schema (above) is the recommended approach. If you'd rather keep the handler separate, list a handler-free `FunctionSchema` in the context as usual and register its handler by name: + +```python +# weather_function here is the same schema, just defined without handler=. +context = LLMContext(tools=[weather_function]) +llm.register_function("get_current_weather", fetch_weather_from_api) ``` -#### Provider-Specific Custom Tools +To remove the tool, un-advertise it with an [`LLMSetToolsFrame`](#changing-tools-mid-conversation); call `llm.unregister_function(...)` only afterward, since unregistering a still-advertised tool leaves the LLM able to call a handler that's no longer there. + +This is uncommon — bundling keeps a tool and its handler together — but the option is there when you need to manage registration directly. -`LLMContext` expects tools to be provided as a `ToolsSchema`. For normal -function calling, prefer `standard_tools` with `FunctionSchema` or direct -functions so Pipecat can convert them to each provider's native format. +### Provider-Specific Custom Tools -When a provider has tools that don't fit Pipecat's standard function schema, -add those provider-native definitions through `ToolsSchema.custom_tools`. These -custom tools are passed only to the matching adapter and are appended to the -converted standard tools. +For normal function calling, prefer `standard_tools` with `FunctionSchema` or direct functions so Pipecat can convert them to each provider's native format. When a provider has tools that don't fit Pipecat's standard function schema, add those provider-native definitions through `ToolsSchema.custom_tools`. These custom tools are passed only to the matching adapter and are appended to the converted standard tools. @@ -195,117 +326,12 @@ tools = ToolsSchema( - For normal callable functions, use `FunctionSchema` or direct functions + For normal callable functions, use direct functions or `FunctionSchema` instead of provider-native function definitions. Today, `custom_tools` is supported for OpenAI-family adapters and Gemini. Anthropic standard functions should be represented with `FunctionSchema`. -### 2. Register Function Handlers - -Register handlers for your functions using one of these [LLM service methods](https://reference-server.pipecat.ai/en/latest/api/pipecat.services.llm_service.html#llm-service): - -- `register_function` -- `register_direct_function` - -Which one you use depends on whether your function is a ["direct" function](#using-direct-functions-shorthand). - - - -```python Non-Direct Function -from pipecat.services.llm_service import FunctionCallParams - -llm = OpenAILLMService(api_key="your-api-key") - -# Main function handler - called to execute the function -async def fetch_weather_from_api(params: FunctionCallParams): - # Fetch weather data from your API - weather_data = {"conditions": "sunny", "temperature": "75"} - await params.result_callback(weather_data) - -# Register the function -llm.register_function( - "get_current_weather", - fetch_weather_from_api, - cancel_on_interruption=True, # Cancel if user interrupts (default: True) - timeout_secs=30.0, # Optional: Override global timeout for this function -) -``` - -```python Direct Function -from pipecat.services.llm_service import FunctionCallParams - -llm = OpenAILLMService(api_key="your-api-key") - -# Direct function -async def get_current_weather(params: FunctionCallParams, location: str, format: str): - """Get the current weather. - - Args: - location: The city and state, e.g. "San Francisco, CA". - format: The temperature unit to use. Must be either "celsius" or "fahrenheit". - """ - weather_data = {"conditions": "sunny", "temperature": "75"} - await params.result_callback(weather_data) - -# Register direct function -llm.register_direct_function( - get_current_weather, - cancel_on_interruption=False, # Don't cancel on interruption - timeout_secs=60.0, # Optional: Override global timeout for this function -) -``` - - - -**Key registration options:** - -- **`cancel_on_interruption=True`** (default): Function call is cancelled if user interrupts -- **`cancel_on_interruption=False`**: Function call continues as async; LLM doesn't wait for result before continuing -- **`timeout_secs=None`** (default): Optional per-tool timeout in seconds. Overrides the global `function_call_timeout_secs` for this specific function - -Use `cancel_on_interruption=False` for long-running operations or when you want the LLM to continue the conversation without waiting. When set to `False`, the function call is treated as **asynchronous**: the LLM continues the conversation immediately without waiting for the result. Once the result returns, it's injected back into the context as a developer message, triggering a new LLM inference at that point. This allows for truly non-blocking function calls where the conversation can proceed while the function executes in the background. Async function calls can also send intermediate updates before the final result. - -Use `cancel_on_interruption=True` (the default) when the LLM should wait for the function result before responding. This ensures the LLM has the complete information before generating its next response. - -Use `timeout_secs` to set a specific timeout for a function that differs from the global default. For example, you might want a longer timeout for database queries or shorter timeouts for quick lookups. - -#### Async Function Call Cancellation - -If you register async function calls with `cancel_on_interruption=False`, you can also enable model-directed cancellation: - -```python -llm = OpenAILLMService( - api_key="your-api-key", - enable_async_tool_cancellation=True, -) -``` - -When `enable_async_tool_cancellation=True` and at least one async function is registered, Pipecat automatically adds the built-in `cancel_async_tool_call` tool and supporting system instructions. The LLM can call that tool to cancel a stale in-progress async function call, for example when the user changes their request before a long-running lookup completes. - -### 3. Create the Pipeline - -Include your LLM service in your pipeline with the registered functions: - -```python -# Initialize the LLM context with your function schemas -context = LLMContext(tools=tools) - -# Create the context aggregator to collect the user and assistant context -user_aggregator, assistant_aggregator = LLMContextAggregatorPair(context) - -# Create the pipeline -pipeline = Pipeline([ - transport.input(), # Input from the transport - stt, # STT processing - user_aggregator, # User context aggregation - llm, # LLM processing - tts, # TTS processing - transport.output(), # Output to the transport - assistant_aggregator, # Assistant context aggregation -]) -``` - ## Function Handler Details ### FunctionCallParams @@ -363,7 +389,7 @@ Your function handler should: 1. Receive necessary arguments, either: - From `params.arguments` - - Directly from function arguments, if using [direct functions](#using-direct-functions-shorthand) + - Directly from function arguments, if using [direct functions](#1-define-a-tool) 2. Process data or call external services 3. Return results via `params.result_callback(result)` @@ -444,8 +470,8 @@ resources = AppResources( user_id="user-123" ) -# Pass resources to the pipeline task -task = PipelineWorker( +# Pass resources to the pipeline worker +worker = PipelineWorker( pipeline, app_resources=resources ) @@ -470,12 +496,12 @@ async def query_user_preferences(params: FunctionCallParams): - Useful for database connections, API clients, caches, or any shared state - `PipelineWorker(tool_resources=...)` and `FunctionCallParams.tool_resources` are - deprecated aliases retained for compatibility. Prefer + `PipelineWorker(tool_resources=...)` and `FunctionCallParams.tool_resources` + are deprecated aliases retained for compatibility. Prefer `PipelineWorker(app_resources=...)` and `params.app_resources`. -## Controlling Function Call Behavior (Advanced) +## Advanced: Controlling Function Call Behavior When returning results from a function handler, you can control how the LLM processes those results using a `FunctionCallResultProperties` object passed to the result callback. @@ -496,7 +522,7 @@ class FunctionCallResultProperties: - **`run_llm=True`**: Run LLM after function call (default behavior) - **`run_llm=False`**: Don't run LLM after function call (useful for chained calls) - **`on_context_updated`**: Async callback executed after the function result is added to context -- **`is_final=False`**: Treat this as an intermediate result for an async function call. Only use this for functions registered with `cancel_on_interruption=False` +- **`is_final=False`**: Treat this as an intermediate result for an async function call. Only use this for async functions (`cancel_on_interruption=False`) Skip LLM execution (`run_llm=False`) when you have back-to-back function @@ -539,13 +565,16 @@ async def query_database(params: FunctionCallParams): ### Intermediate Results for Async Functions -Async function calls can send progress updates before their final result. Register the function with `cancel_on_interruption=False`, then call `params.result_callback(..., properties=FunctionCallResultProperties(is_final=False))` for each intermediate update. Finish with a normal `params.result_callback(...)`. +Async function calls can send progress updates before their final result. Make the function async with `@tool_options(cancel_on_interruption=False)`, then call `params.result_callback(..., properties=FunctionCallResultProperties(is_final=False))` for each intermediate update. Finish with a normal `params.result_callback(...)`. ```python +from pipecat.adapters.schemas.direct_function import tool_options from pipecat.frames.frames import FunctionCallResultProperties from pipecat.services.llm_service import FunctionCallParams +@tool_options(cancel_on_interruption=False) async def track_delivery(params: FunctionCallParams): + """Track a delivery, reporting each status update until it arrives.""" await params.result_callback( {"status": "picked_up"}, properties=FunctionCallResultProperties(is_final=False), @@ -557,12 +586,6 @@ async def track_delivery(params: FunctionCallParams): ) await params.result_callback({"status": "delivered"}) - -llm.register_function( - "track_delivery", - track_delivery, - cancel_on_interruption=False, -) ``` Intermediate results are injected into the LLM context as async-tool developer messages. They do not close the function call; the call remains in progress until the final result is sent. @@ -571,7 +594,7 @@ Intermediate results are injected into the LLM context as async-tool developer m - **Function calling extends LLM capabilities** beyond training data to real-time information - **Context integration is automatic** - function calls and results are stored in conversation history -- **Multiple definition approaches** - use standard schema for portability, direct functions for simplicity +- **Direct functions are the preferred approach** - one async function is both schema and handler; list it in `LLMContext(tools=[...])` or add it via `LLMSetToolsFrame` to make it available. When you need explicit schema control, use a `FunctionSchema` with its `handler` bundled in - **Async function calls are opt-in** - set `cancel_on_interruption=False` for deferred results, intermediate updates, and optional async-tool cancellation - **Pipeline integration is seamless** - functions work within your existing voice AI architecture - **Advanced control available** - fine-tune LLM execution and monitor function call lifecycle