docs(chatkit): document Telnyx relay and OpenAI voice setup

chatkit.mdx

@@ -617,6 +617,66 @@ export const POST = chatKitEndpoint({
 ChatKit preserves the messages inside a session. [Memory](/memory) stores selected knowledge that should be available across sessions, channels, or agents. Use both when an agent needs conversational continuity and longer-lived organizational context.
 
 
+## Configure realtime voice
+
+Voice settings belong to your Tilde agent. In agent registration, select a voice
+profile and its models, voice, and maximum conversation duration. You can change
+these settings in the agent editor later.
+
+- **Text agent with speech**: Tilde transcribes incoming audio, invokes your normal
+  `chatKitEndpoint` callback, and speaks its streamed text response.
+- **Telnyx Conversation Relay**: Telnyx handles recognition and speech synthesis
+  for incoming phone calls. Tilde receives caller text, invokes your normal
+  callback, and streams its response text to Telnyx. This profile is phone-only.
+- **OpenAI Realtime**: the realtime model generates spoken responses directly.
+  Tilde records the final transcripts without invoking your text endpoint again.
+
+Your text callback receives `context.audio` for transcribed speech turns and
+`context.telnyx` for Telnyx calls. Provider facts are supplied in the signed
+request. A subsequent typed message does not become a speech turn merely because
+its session previously contained a call.
+
+Each call creates a normal ChatKit session. Browser media admission is one-time,
+expires after five minutes, and remains subject to current session membership.
+Call duration is limited by the configured maximum. Call recordings are not
+retained by this initial realtime implementation.
+
+For a manual test, use the SDK repository's `examples/realtime-voice` example. It
+registers three agents. Its microphone page supports the two OpenAI modes; a
+dedicated relay endpoint handles phone calls. To receive calls, configure a
+**Telnyx Voice** chat provider with an existing encrypted Telnyx Voice credential,
+Voice API application, phone number, and default agent. This creates a real
+ChatKit channel that owns the caller's participant route.
+
+Choose **self-managed webhook setup** to copy Tilde's returned webhook URL into
+Telnyx yourself. Choose **managed webhook setup** to let Tilde update the existing
+application's webhook URL using your credential. Managed setup does not provision
+or fund a Telnyx account, purchase a number, or assign numbers to applications.
+Use a dedicated test application for the manual example. The Tilde API must be publicly
+reachable through HTTPS and WSS. Hookdeck webhook replay alone cannot carry live
+bidirectional calls.
+
+For relay, select `telnyx_relay`, transcription model `deepgram/nova-3`, voice
+`Telnyx.Ultra.Callie`, language `en-US`, and interruption enabled. The Telnyx
+route uses its own credential; relay does not require an OpenAI speech key.
+Your callback continues to use its own text model and tools. The SDK example
+accepts `TELNYX_AGENT_MODE=telnyx_relay` and keeps the browser demos available.
+
+Partial relay transcripts do not start agent turns. When speech interrupts a
+response, Tilde retains generated text and separately records the spoken prefix
+reported by the carrier. The SDK annotates both text and UI history so the next
+turn can distinguish generated words from that reported prefix. Caller ID does
+not authorize access to a Tilde human's personal tools.
+
+The [Telnyx Conversation Relay guide](https://developers.telnyx.com/docs/voice/programmable-voice/conversation-relay)
+describes the text WebSocket protocol used between Telnyx and Tilde.
+
+The initial browser implementation streams audio through the Rust API. Direct
+browser-to-provider WebRTC and direct SIP routing are separate transport options
+and are not implied by the OpenAI Realtime profile. Native Realtime uses its
+configured instructions and does not inherit endpoint tools. Browser voice
+identifies its caller but does not yet establish personal-tool federation.
+
 ## Change resources through native tools
 
 Agents use native Tilde API/MCP operations under their existing permissions.

llms/chatkit.md

@@ -191,6 +191,63 @@ Use `tilde_list_signal_provider_instances` and `tilde_list_signal_rules` before
 In application code, handle typed GitHub, Slack, Sentry, and Firecrawl metadata as shown in the [human ChatKit guide](https://trytilde.ai/docs/chatkit). `onUnprocessed` runs once per unprocessed message; later conversions reuse its cached result.
 
 
+## Agent-owned realtime audio
+
+Use the selected tenant host and explicit `team_id` for these REST operations:
+
+1. `GET /api/v1/chatkit/audio/profiles` returns supported profile defaults and
+   server-authored fields. Render these descriptors rather than generating
+   provider-specific setup instructions in frontend code.
+2. Register an HTTP agent with optional `audio` configuration, or use
+   `PUT /api/v1/team/{team_id}/chatkit/agents/{agent_id}/audio` with `{ "audio":
+   <configuration> }`. Set `audio` to null on the PUT route to disable voice.
+3. Configuration fields are `mode` (`pipeline`, `realtime`, or `telnyx_relay`), `credential_id`
+   (optional), `stt_model`, `tts_model`, `realtime_model`, `voice`, `instructions`,
+   `language` (default `en-US`), `interruptible` (default true), and
+   `max_duration_seconds` (10–1800). The OpenAI Audio credential source is
+   `chatkit_openai_audio`; omitting it uses the server OpenAI key for OpenAI modes.
+   Relay uses `stt_model: "deepgram/nova-3"`, `voice: "Telnyx.Ultra.Callie"`,
+   and null `credential_id`; its phone route owns the Telnyx credential.
+4. `POST /api/v1/team/{team_id}/chatkit/agents/{agent_id}/audio/sessions` creates a
+   normal browser session for OpenAI modes and returns `audio_session`, `websocket_path`, and a one-time
+   token. Connect with WebSocket subprotocols `chatkit-audio` and `token.<token>`.
+   This endpoint rejects `telnyx_relay`; relay starts from an incoming call.
+   Send mono signed PCM16 little-endian audio at 24 kHz as base64 `audio` frames.
+5. `PUT /api/v1/team/{team_id}/chatkit/agents/{agent_id}/audio/telnyx` accepts
+   `credential_id` (source `chatkit_telnyx_voice`), `public_key`, `phone_number`,
+   `connection_id`, and public HTTPS `media_base_url`. It returns `route` and
+   `webhook_url`; successful setup also returns the assigned
+   `route.channel_inbox_id`. Use the webhook URL in the dedicated Telnyx application.
+6. The generic channel catalog entry is `chatkit.chat_channel.telnyx_voice`,
+   provider `chatkit.channel.telnyx_voice`. Use auth method
+   `chatkit.channel.telnyx_voice.auth.self_managed` for a returned URL or
+   `chatkit.channel.telnyx_voice.auth.managed` for Tilde to update the existing
+   Voice API application's webhook. Both use your existing encrypted
+   `chatkit_telnyx_voice` credential and existing number/application. Pass the
+   same five setup fields and the normal default agent selection. Managed setup
+   updates webhook configuration; it does not buy, assign, or fund numbers.
+   Calls use the resulting channel as their participant origin.
+
+Pipeline mode invokes the existing callback only when a user speech turn is
+ready. Rust synthesizes the response. Relay invokes the same signed callback
+with `context.audio.mode = "telnyx_relay"`; Telnyx transcribes and synthesizes,
+while Tilde exchanges text frames with the carrier. Realtime mode owns spoken generation;
+transcript observations must not trigger another model turn or external send.
+`context.audio` and `context.telnyx` come from typed, server-authored speech
+provenance rather than client message metadata. Both persisted text and UI
+messages can carry `speech`. Interrupted generated speech retains its original
+text with `interrupted: true`, optional `played_audio_ms`, and optional
+`reported_spoken_text` supplied by the carrier. Preserve the reported prefix
+separately; do not rewrite it as the complete generated response. SDK history
+conversion adds the corresponding annotation before the original content.
+
+The manual browser/carrier example is `examples/realtime-voice` in `trytilde/dispatch`
+(the `@trytilde/sdk` packages). It never buys phone numbers or changes existing carrier routing. Agent
+settings and credential setup references are portable; live connections and
+media tokens are not exported. Configure Telnyx number/application bindings
+again in the destination installation. Native mode does not inherit endpoint
+tools, and browser voice does not establish personal-tool federation.
+
 ## Change resources through native tools
 
 Agents use native Tilde API/MCP operations under their existing permissions.