docs: add a voice agent guide to Learn - #484
Open
joshua-mo-143 wants to merge 4 commits into
Open
Conversation
Voice agents look like they need a realtime speech-to-speech socket, which Venice does not expose. Walk through building one from the three endpoints that already exist instead, so the STT to LLM to TTS shape is visible without a framework in the way. Covers the two venice_parameters that change how the agent sounds, why the chat stream is split into sentences before TTS rather than forwarded as tokens, and why a raw PCM sink needs its response checked before playback. Co-authored-by: Cursor <cursoragent@cursor.com>
Contributor
|
Preview deployment for your docs. Learn more about Mintlify Previews.
|
The language belongs in the description and the card metadata, not the heading. Keeps the title in line with the other Learn pages and leaves room for the guide to be read as the pattern rather than one implementation. Co-authored-by: Cursor <cursoragent@cursor.com>
The reference repo swapped pw-record and pw-play for sounddevice, so the guide's audio section described code that no longer exists. Rewrite it against the PortAudio streams and drop the Linux-only framing. Two changes are worth more than a find and replace. Recordings never touch the disk now, so the privacy section describes an in-memory WAV rather than a temp file that gets cleaned up. And a raw output stream needs writes aligned to sample boundaries, which HTTP chunk boundaries do not respect, so the odd-byte carry buffer is now explained rather than elided. Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a Learn guide that builds a terminal voice agent on Venice, walking through the reference implementation at joshua-mo-143/venice-voice-agent-demo.
Venice has no realtime speech-to-speech socket, which reads like a gap until you notice a voice agent is three ordinary HTTP calls in a loop. This is the same STT → LLM → TTS shape as the existing LiveKit Agents guide, with the framework removed so the three requests are visible.
New page:
learn/voice-agent.mdx→/learn/voice-agent. Bylined Joshua Mo, 27 August 2026.The sections that got the most space are the ones the demo is actually opinionated about:
include_venice_system_prompt: Falseanddisable_thinking, and what each costs if left onDr.andU.S.response_format="pcm"withstreaming: True, and the status / content-type / first-chunk checks — a JSON error body written into a raw sink plays as a burst of noisePcmPlayer, andabort()versusclose()Also includes a privacy section, since this is the first Learn guide that turns on a microphone. Recordings are assembled in memory and given a WAV header with
waverather than being written to a temp file, so there's nothing on disk to clean up.Supporting changes
learn.mdx— card in the Projects row. Projects rather than Walkthroughs because every existing Projects entry is repo-backed and every Walkthrough is notebook- or inline-based.docs.json— English nav path only.llms.txt— one entry under Learn.Per
agents.md, locale copies are untouched for Mintlify to regenerate after merge.Note on the third commit
The demo repo switched local audio from PipeWire (
pw-record/pw-play) tosounddevice/ PortAudio after the guide was first written, so it's no longer Linux-only. The third commit rewrites the audio section against the new code and re-checks the surrounding claims — most visibly the privacy section, which described a temp file that no longer exists.Test plan
Verified:
.mdxfilesdocs.jsonstill parses as valid JSONnvidia/parakeet-tdt-0.6b-v3,zai-org-glm-5-2,tts-kokoro,af_sky) match those already documented inguides/integrations/livekit-agents.mdxandguides/media/*audio.py; setup commands and thesounddevice>=0.5.6pin match itspyproject.tomland.env.example67dd1e2Worth a reviewer's eye:
joshua-mo-143/venice-*-demolinks, but flagging itagents.mdsays to resolve them at runtime. There's a callout saying so, but the code uses env-var defaults like the other guides.