Skip to content

docs: add a voice agent guide to Learn - #484

Open
joshua-mo-143 wants to merge 4 commits into
mainfrom
docs-venice-voice-agent-guide
Open

docs: add a voice agent guide to Learn#484
joshua-mo-143 wants to merge 4 commits into
mainfrom
docs-venice-voice-agent-guide

Conversation

@joshua-mo-143

@joshua-mo-143 joshua-mo-143 commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Summary

Adds a Learn guide that builds a terminal voice agent on Venice, walking through the reference implementation at joshua-mo-143/venice-voice-agent-demo.

Venice has no realtime speech-to-speech socket, which reads like a gap until you notice a voice agent is three ordinary HTTP calls in a loop. This is the same STT → LLM → TTS shape as the existing LiveKit Agents guide, with the framework removed so the three requests are visible.

New page: learn/voice-agent.mdx/learn/voice-agent. Bylined Joshua Mo, 27 August 2026.

The sections that got the most space are the ones the demo is actually opinionated about:

  • include_venice_system_prompt: False and disable_thinking, and what each costs if left on
  • Yielding sentences rather than tokens, which is what lets TTS start before the model finishes, plus why the splitter waits for trailing whitespace and how it avoids breaking on Dr. and U.S.
  • response_format="pcm" with streaming: True, and the status / content-type / first-chunk checks — a JSON error body written into a raw sink plays as a burst of noise
  • Writing to a raw output stream on sample boundaries, since HTTP chunk boundaries don't respect them and a split 16-bit sample byte-shifts everything after it
  • The queue-and-thread handoff, the lazily created PcmPlayer, and abort() versus close()

Also includes a privacy section, since this is the first Learn guide that turns on a microphone. Recordings are assembled in memory and given a WAV header with wave rather than being written to a temp file, so there's nothing on disk to clean up.

Supporting changes

  • learn.mdx — card in the Projects row. Projects rather than Walkthroughs because every existing Projects entry is repo-backed and every Walkthrough is notebook- or inline-based.
  • docs.json — English nav path only.
  • llms.txt — one entry under Learn.

Per agents.md, locale copies are untouched for Mintlify to regenerate after merge.

Note on the third commit

The demo repo switched local audio from PipeWire (pw-record / pw-play) to sounddevice / PortAudio after the guide was first written, so it's no longer Linux-only. The third commit rewrites the audio section against the new code and re-checks the surrounding claims — most visibly the privacy section, which described a temp file that no longer exists.

Test plan

Verified:

  • All 14 internal doc links resolve to real .mdx files
  • docs.json still parses as valid JSON
  • Every icon name used already appears elsewhere in the repo (Tabler set)
  • Model IDs (nvidia/parakeet-tdt-0.6b-v3, zai-org-glm-5-2, tts-kokoro, af_sky) match those already documented in guides/integrations/livekit-agents.mdx and guides/media/*
  • Every symbol the guide names exists in the demo's current audio.py; setup commands and the sounddevice>=0.5.6 pin match its pyproject.toml and .env.example
  • No stale PipeWire references left anywhere in the repo
  • Demo repo's 18 tests pass at 67dd1e2
  • Mintlify Deployment and link-rot checks pass (vale-spellcheck skips)

Worth a reviewer's eye:

  • The 14 min read-time estimate on the card, scaled by byte count against neighbouring guides (lands between the RAG bot at 9 min and the Rust gateway at 18 min)
  • Whether linking the reference repo under a personal account is still the convention — it matches the four existing joshua-mo-143/venice-*-demo links, but flagging it
  • The guide documents specific model IDs while agents.md says to resolve them at runtime. There's a callout saying so, but the code uses env-var defaults like the other guides.

Voice agents look like they need a realtime speech-to-speech socket, which
Venice does not expose. Walk through building one from the three endpoints
that already exist instead, so the STT to LLM to TTS shape is visible without
a framework in the way.

Covers the two venice_parameters that change how the agent sounds, why the
chat stream is split into sentences before TTS rather than forwarded as
tokens, and why a raw PCM sink needs its response checked before playback.

Co-authored-by: Cursor <cursoragent@cursor.com>
@mintlify

mintlify Bot commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

Preview deployment for your docs. Learn more about Mintlify Previews.

Project Status Preview Updated (UTC)
veniceai 🟢 Ready View Preview Aug 27, 2026, 8:17 PM

@joshua-mo-143 joshua-mo-143 changed the title docs: add a Python voice agent guide to Learn docs: add a voice agent guide to Learn Aug 27, 2026
The language belongs in the description and the card metadata, not the
heading. Keeps the title in line with the other Learn pages and leaves room
for the guide to be read as the pattern rather than one implementation.

Co-authored-by: Cursor <cursoragent@cursor.com>
The reference repo swapped pw-record and pw-play for sounddevice, so the
guide's audio section described code that no longer exists. Rewrite it
against the PortAudio streams and drop the Linux-only framing.

Two changes are worth more than a find and replace. Recordings never touch
the disk now, so the privacy section describes an in-memory WAV rather than
a temp file that gets cleaned up. And a raw output stream needs writes
aligned to sample boundaries, which HTTP chunk boundaries do not respect, so
the odd-byte carry buffer is now explained rather than elided.

Co-authored-by: Cursor <cursoragent@cursor.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant