Repository navigation
fix(web): voice input starts recording on press instead of waiting on the server - #115
Merged
nullStack65 merged 7 commits intoNov 20, 2026
Merged
Conversation
… the server Pressing the composer transcribe button waited for two or three server round trips (a status refresh, the status again inside prepare, and the stream ticket) before it opened the microphone. It then created a fresh AudioContext and loaded the worklet module one step at a time. Over a 300 ms link that is 0.8 to 1.9 s of "starting". The web controller now opens capture right away and buffers PCM while the server prepares. It flushes the buffer into the live stream when the stream is ready, or falls back to buffered transcription if the recording ended first. The status refresh runs alongside capture. One AudioContext with the loaded worklet stays warm and suspended between recordings. The microphone stream is never kept open. Released worklet processors now end instead of lingering. Mobile keeps its sequential start. Model: claude-opus-5-5 Task: coding Agent: T3 Claude Code (CLO-919 button lane) Lane: magpie Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…the hook Review fixes for the press-to-recording change: - If the server preparation fails after recording has started, the controller now stops the capture and keeps the audio. The next press prepares again and transcribes that audio, without recording again. - The composer starts capture at once even before the first status answer, unless a known status says the model is missing or the destination is unsupported. - The warmed AudioContext is suspended once its worklet loads, and it is closed if loading fails. Cleanup suspends it even when a resume is still in flight. The worklet release message is no longer cut off by closing the port. - A hook-level browser test fails if a press waits for the status refresh, and it runs in ARC CI with the latency test. Model: claude-opus-5-5 Task: coding Agent: T3 Claude Code (CLO-919 button lane) Lane: magpie Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The ARC pod image has no Chromium system libraries and cannot use sudo, so headless Chromium exited with "libnspr4.so: cannot open shared object file". Download the Ubuntu packages Chromium needs with apt as the runner user, using private state directories so apt still resolves dependencies and verifies the archive signatures, extract them, and add them to LD_LIBRARY_PATH, as the libatomic step already does for Node. Model: claude-opus-5-5 Task: coding Agent: T3 Claude Code (CLO-919 button lane) Lane: magpie Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Starting capture before the first status answer opened the microphone for environments still missing a model, and on a slow runner it raced the mount probe, so setup-wizard tests saw capture instead of setup. Only a known ready status takes the fast path again; an unknown status waits for its check, as before this change. Model: claude-opus-5-5 Task: coding Agent: T3 Claude Code (CLO-919 button lane) Lane: magpie Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Running the whole composer voice hook test file on ARC also ran three setup-wizard tests that ARC had never run before. On the runner, their dialog content renders but is reported as not visible, while the same tests pass locally. Select the press-to-recording and release-to-text tests by name so the latency gate stays and those wizard tests stay out of this step, as they were before. Model: claude-opus-5-5 Task: coding Agent: T3 Claude Code (CLO-919 button lane) Lane: magpie Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars.
|
The retry read the recorder uri after releaseAudioSession, whose cancel clears it, so the next press retried with no audio. Read the uri first. Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.2 <noreply@anthropic.com>
Owner
Author
{"reviewed_head":"0a9091d312e89b29164f227bf5d8ab853c0ad215","reviewer":"grok-4.7-build (claudeAgent, read-only)","findings":[{"id":"late-prep-failure-drops-retry-audio","severity":"important","summary":"finishRecording released the audio session before reading the recorder uri, and that release clears the uri, so a preparation failure after capture started left the retry with no audio. The uri is now read and stored before the release, and the regression test nulls the uri inside releaseRecording.","outcome":"confirmed","rechecked_head":"7e22c515a0c58a0a23b3aed881b2b977f95bb7fa"}]} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Kameron reported that the composer transcribe button shows "starting" for 1 to 3 seconds. The goal is under 400 ms from press to recording. This is item 4 of the shared dictation issue (CLO-919).
Cause
Before it opened the microphone, a press waited for two or three server round trips in a row: the hook's status refresh, the status again inside
prepare, and the media stream ticket. After that it created a newAudioContextand loaded the worklet module, one step at a time.Fix
VoiceInputControllertakes a newcaptureDuringPreparationoption, and web turns it on. Capture starts at once and the phase becomesrecordingwhile the server preparation runs alongside it. Releasing before preparation finishes waits for it. If preparation fails, the recording stops with the server's error. Mobile keeps its sequential start.browserPcmCapturebuffers PCM until the stream is ready, then flushes it into the stream. If the recording ended first, it cancels the stream and uses buffered transcription. One sharedAudioContextwith the loaded worklet is warmed when the composer mounts and is suspended between recordings. The microphone stream is never kept open, so the mic indicator only shows while recording.false, so old processors don't pile up on the shared context.Latency gate (runs in ARC CI)
packages/client-runtime/src/voice-input/controller.test.ts: recording must begin within 400 ms whilepreparenever answers. It fails on the old sequential path.apps/web/src/speech/voiceInputLatency.browser.test.tsuses the reusableapps/web/test/voiceLatencyHarness.ts. It drives real Chromium capture with a fake mic device, and the server is held open until after the press. It fails if press-to-recording or release-to-text goes over 400 ms, for both buffered and streaming. The browser project now launches Chromium with fake media and--disable-audio-output, so the audio clock runs headless without a sound server.Measured on the ASUS (headless Chromium, simulated server round trip)
The number on Kameron's laptop is unverified. The settings test
CloudVoiceConfiguration.browser.test.tsxalready fails on the base branch, so those failures are not from this change.Deploy: none
Model: claude-opus-5-5
Task: coding
🤖 Generated with Claude Code