Repository navigation
feat(voice): Groq whisper as the primary dictation backend, streamed in pieces - #120
Merged
Merged
Conversation
…in pieces Groq whisper-large-v3-turbo joins the cloud speech providers and becomes the recommended backend. Its key comes from settings or a host-injected GROQ_API_KEY, like AssemblyAI. On web and desktop, a Groq recording streams over its own socket; the server cuts the audio at pauses once a piece reaches eight seconds and uploads each piece while the person is still talking, so release waits only for the last piece. A CI test feeds 30 dictations in real time against a modelled Groq latency and fails above 400 ms release-to-text. Model: claude-opus-5-5 Task: CLO-919 Agent: Claude Code Lane: magpie Harness: t3/claude-code Model-family: Claude Model-generation: 5.5 Skills-SHA: 7a14a4240d1e Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review fixes on the Groq dictation stream: - uploads fork into the stream's scope, so feed needs no Scope, and the route path is typed as a router path; the server typecheck passes again - a rate-limited or unavailable piece is retried twice; a permanent failure fails the stream on the next audio frame instead of at release - the floor drops from 8 s to 4 s and forced cuts from 20 s to 12 s, so short dictations upload during speech and the tail stays bounded - a quiet microphone's audio is still sent; stray clicks and long silences inside a piece are dropped - with live transcription off, Groq uses the batch endpoint, and the disclosure no longer claims streaming on mobile - the context budget edit is reverted; that drift predates this branch Model: claude-opus-5-5 Task: CLO-919 Agent: Claude Code Lane: magpie Harness: t3/claude-code Model-family: Claude Model-generation: 5.5 Skills-SHA: 7a14a4240d1e Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Owner
Author
{"reviewed_head":"14e158c2ba0a4f6c165a0f264febb2889d39ec8f","reviewer":"grok-4.7-build (claudeAgent, read-only)","findings":[{"id":"cached-config-keeps-host-key","severity":"nit","summary":"Effect's default ConfigProvider copies process.env on its first read, which startup config does before the speech layer deletes GROQ_API_KEY, so a later Config read could still see it. Nothing reads that variable again after the delete, and a child process inherits process.env, which the delete does clear.","outcome":"skipped"}]} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Dictation needs under 400 ms from key release to text (the shared dictation work, Linear CLO-919). The CPU reference was a p50 of 1431 ms, and every cloud provider here uploaded only after release.
What changed
GROQ_API_KEY, which is removed from the server's environment so agent subprocesses never inherit it, the same wayASSEMBLYAI_API_KEYworks. Custom words go in as Whisper'sprompt. AssemblyAI is unchanged./ws/voice/cloud/groq, using the existing speech frame protocol. The server cuts audio at pauses once a piece reaches 4 s (forced at 12 s, at the quietest frame) and uploads each piece while the person is still talking. Release then waits only for the tail. The 4 s floor keeps one continuous dictation at or under 15 requests a minute, inside the free tier's 20. Silence and stray clicks are dropped, but a quiet microphone's audio is still sent. A rate-limited or unavailable piece is retried twice. A permanent failure fails the stream on the next audio frame, so it doesn't wait for release. With live transcription off, Groq uses the batch endpoint. Groq errors reach the user as their own message.GroqSpeechStream.test.tsfeeds 30 dictations of 3 to 70 s in real time onTestClockthrough the real segmenter and stream. It asserts structurally that any tail left at release is at most 10 s of audio. It then fails if release-to-text exceeds 400 ms against a modelled Groq latency of 230 ms + 12 ms per audio second. Modelled result: p50 0 ms, p95 261 ms, max 264 ms. The model is an assumption. It leaves out real Groq variance and the socket round trip, so it does not prove 400 ms live. Live numbers wait on a Groq key. Other tests cover piece bounds, no lost speech, silence, clicks, quiet microphones, forced cuts, retries and failing early.Verified locally (ASUS, Node 26): typecheck passes for contracts, shared, client-runtime, server (
t3), web and mobile. The Groq, stream, CloudSpeech, settings and secrets tests pass. The webvoiceBackendstest passes. Two existing failures happen identically on the unchanged base, and CI runs neither:CloudVoiceConfiguration.browser.test.tsxfails 7/7 with aspeechCustomWordsflatMaperror, andAgentContextBudget.test.tsmeasures 55,240 MCP bytes against a budget of 53,783. Neither is changed here.Not verified: live Groq latency, rate limits and WER all wait on a free Groq key in Infisical.
Deploy: fleet
Model: claude-opus-5-5
Task: CLO-919
🤖 Generated with Claude Code