Skip to content

Live voice (experimental), voice orchestration, P0 fixes, and composer QoL#57

Merged
ojowwalker77 merged 2 commits into
mainfrom
teacode/live-voice
Jul 24, 2026
Merged

Live voice (experimental), voice orchestration, P0 fixes, and composer QoL#57
ojowwalker77 merged 2 commits into
mainfrom
teacode/live-voice

Conversation

@ojowwalker77

Copy link
Copy Markdown
Owner

Rebased onto main (v0.12.0). Everything here is a single commit — the branch was all working-tree work before this PR.

Live voice (experimental, Codex)

  • WebRTC microphone input + streamed assistant audio playback (remote-track <audio> sink).
  • Realtime transcripts and Thinking Orbs voice states (connecting / listening / speaking / muted).
  • Full-screen voice-focus surface that owns the transcript pane (sidebar/header/composer chrome preserved).
  • Mic/audio frames kept out of the durable orchestration event log; only control state + final transcripts persist. SDP preserved as opaque protocol data (trailing CRLF intact).
  • Capability-gated: the voice affordance only appears on Codex threads.

Voice as orchestrator

  • TeaCode MCP thread controls — threads_list / threads_create / threads_send / threads_read — so voice can spawn separate-context threads and drive other providers.

P0 fixes

  • No implicit browser auth on startup. Cursor and Grok are gated behind explicit connect; Codex / OpenCode / Kilo / Pi / Claude audited clean. XAI_API_KEY still works with no browser.
  • New Thread exposes the real model picker (the centered tray previously omitted it).
  • Settings: default thread workspace behavior — new worktree / current branch / pinned base branch.

Composer QoL

  • Per-provider quota ring in the composer underbar: remaining-% + a ring that fills for what's left and recolors green→red as it drains; click for the full usage popover. Reuses the existing tone/usage logic; hidden when a provider has no usage data.
  • Do-things ⌘K palette: switch model, switch provider, and spawn a new thread pre-set to a chosen provider — all from the keyboard.

Deferred by design (not bugs)

  • ⌘K branch-switch (needs cwd derivation that only lives in ChatView today — TODO seam left).
  • ⌘K voice toggle (its hook is bound to ChatView's lifecycle).
  • Error surfacing: traced end-to-end and already correct — server preserves detail/cause, the thread banner shows the raw message, toasts use err.message. The one real offender (voice masking a generic stderr line) was already fixed.

Verification

  • bun fmt / bun lint (0 errors) / bun typecheck (8/8 packages) green on the integrated tree.
  • Affected web tests: 119 passing (voice, realtime audio, quota ring, palette commands, appSettings, threadBootstrap, wsNativeApi). Server auth-gating tests: 106 passing.

⚠️ Live voice depends on OpenAI's experimental realtime backend and a separate voice allowance — availability is upstream-gated.

Live voice (experimental, Codex): WebRTC mic + streamed audio playback, realtime transcripts, Thinking Orbs voice states, full-screen voice focus surface. Voice/audio kept out of the durable event log.

TeaCode MCP thread controls (threads_list/create/send/read) so voice can orchestrate and spawn separate-context threads across providers.

P0 fixes: providers never launch interactive browser auth on startup (Cursor + Grok gated; Codex/OpenCode/Kilo/Pi/Claude audited clean); New Thread exposes the real model picker; Settings controls default thread workspace behavior (new worktree / current branch / pinned base).

Composer QoL: per-provider remaining-quota ring in the composer underbar (green->red as it drains, opens the shared usage popover); do-things Cmd-K palette (switch model, switch provider, new thread in a chosen provider).
@github-actions github-actions Bot added size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list. labels Jul 24, 2026
@ojowwalker77
ojowwalker77 merged commit 276eaef into main Jul 24, 2026
14 checks passed
@ojowwalker77
ojowwalker77 deleted the teacode/live-voice branch July 24, 2026 21:52
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size:XXL vouch:trusted PR author is trusted by repo permissions or the VOUCHED list.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant