Skip to content

fix(voice): give the voice-processing I/O unit's output bus a silent source — the phone threw ~330 render errors a second - #205

Merged
kpmmmurphy merged 4 commits into
masterfrom
fix/vpio-output-bus
Sep 4, 2026
Merged

fix(voice): give the voice-processing I/O unit's output bus a silent source — the phone threw ~330 render errors a second#205
kpmmmurphy merged 4 commits into
masterfrom
fix/vpio-output-bus

Conversation

@kpmmmurphy

Copy link
Copy Markdown
Contributor

What the phone's log showed

During the 2026-09-03 phone-vs-Mac voice session (iPhone 17 Pro, iOS 27), with voice processing on the STT input, the audio unit failed every render cycle:

M1K3(libEmbeddedSystemAUs.dylib) <Error>: throwing -1
M1K3(libEmbeddedSystemAUs.dylib) <Error>:   from <private>, render err: -1

21,549 of them in 75 s — a steady ~330/s through every listen, think and speak phase, dipping only while the engine was stopped between endpoint and re-arm. The Mac's unified log shows none. Capture still worked (three clean Mac-driven turns), so this is a CPU + log tax, not a functional break — but the phone's decode fell from 17 to 7 tok/s over the same session, and this storm is one of the two suspects (heat is the other).

Why

With setVoiceProcessingEnabled(true), the engine's I/O unit is a VPIO whose output element renders every cycle regardless. AppleSpeechTranscriber only ever installed an input tap; nothing was attached to outputNode, so that render had no source and failed.

Fix

Touch mainMixerNode (which implicitly connects mixer → output) and set its volume to 0 in installTapAsOwner, right after voice processing is enabled. An input-less mixer renders silence; the bus is satisfied; nothing audible changes. One line plus the rationale, in the shared transcriber (both shells).

Verification

Package builds. Verify-owed by count on device: the fixed build is installed on the phone; the next voice-mode session gets a 20 s render-error count (expected → 0) and two Mac-driven turns to confirm capture is unchanged. I'll append the numbers here.

🤖 Generated with Claude Code

https://claude.ai/code/session_01K2kJAPEHTFU34uKiGArFSP

…source — the phone was throwing ~330 render errors a second

Read off the iPhone 17 Pro's log during the 2026-09-03 phone-vs-Mac voice
session: with voice processing on and nothing attached to outputNode, the
VPIO's output element failed every render cycle (`AURemoteIO … render err:
-1`, 21,549 in 75 s) through every listen, think and speak phase. The Mac
logs none. Touching mainMixerNode implicitly connects mixer → output; an
input-less mixer renders silence, so the bus is satisfied and nothing
audible changes. Verify-owed by count on device (→ 0).

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K2kJAPEHTFU34uKiGArFSP

# ── MurphySig ──────────────────────────────────────────────────────
# Significant decision in this commit? Sign it. https://murphysig.dev/spec
# ───────────────────────────────────────────────────────────────────
@netlify

netlify Bot commented Sep 3, 2026

Copy link
Copy Markdown

Deploy Preview for m1k3 canceled.

Name Link
🔨 Latest commit 32d7f08
🔍 Latest deploy log https://app.netlify.com/projects/m1k3/deploys/6a9b063af8d7050008b6638c

@claude

claude Bot commented Sep 3, 2026

Copy link
Copy Markdown

Reviewed the diff (single file, macos/Sources/M1K3Voice/AppleSpeechTranscriber.swift, +14/-0). Small, well-scoped fix — no concerns blocking merge.

Correctness

  • The root-cause explanation checks out: with setVoiceProcessingEnabled(true), the I/O unit becomes a VPIO, and its output element renders every cycle regardless of whether anything's attached downstream. Touching mainMixerNode does implicitly wire mixer → outputNode at the engine's HAL format, so this gives the output bus a render source without an explicit connect(_:to:format:) call. outputVolume = 0 is the right belt-and-braces (no audible change even if something later scheduled a buffer here, though nothing in this file does).
  • Ordering is correct: the new line sits in installInputTap() (AppleSpeechTranscriber.swift:595), which is always called before audioEngine.prepare()/start() — both on first install (installTapAsOwnerstartEngineIfOwner, lines 543–571) and on route-change reinstall (handleConfigurationChange, line 712), which explicitly stops the engine before calling installInputTap() again. So the mixer is never touched on a running engine.
  • This AVAudioEngine instance is dedicated to capture — grepped for AVAudioPlayerNode/scheduleBuffer/connect( in this file and found nothing else attached to it, so zeroing outputVolume can't silence unrelated playback.
  • Idempotent: re-running installInputTap() on every route change just reassigns the same volume value — harmless.

Concurrency

  • No new Sendable/actor-isolation surface. The single new statement executes inside the existing engineLock.withLock closure (via installTapAsOwner/handleConfigurationChange), consistent with the file's established two-lock discipline (lock for request/session state, engineLock for engine mutations) documented at AppleSpeechTranscriber.swift:80-105. No bare .lock()/.unlock() introduced.

Architecture

  • Business logic stays in M1K3Voice (a Sources/* package), not app-glue — consistent with the module map. No new dependencies, no protocol seam changes.

Provenance

  • Commit carries a signed MurphySig block; the added doc comments (lines 65-68, 586-595) are clearly attributed and dated. No existing Signed: blocks touched.

Tests

  • No swift test coverage added, but this is reasonable here — the fix is a one-line AVAudioEngine graph wiring change gated entirely behind Metal/CoreAudio (VPIO render callback), which swift test cannot exercise per macos/CLAUDE.md's "metallib wall." The PR description correctly frames verification as verify-owed by device count (render-err → 0) rather than claiming false coverage — good discipline, matches the repo's stated convention for MLX/Metal/audio changes.

Minor observation (non-blocking)

  • The three-way "Review: Kev + claude-fable-5.1" comment at lines 65-68 duplicates most of the inline rationale at lines 586-595 almost verbatim. Not wrong, just a little redundant — could eventually be trimmed to a one-line pointer if the file's header comment grows further, but not worth churning now.

Nothing here needs to change before merge; the fix is narrowly targeted and the reasoning is sound.

…m a devicectl launch

A phone on the desk can be driven from the Mac (its M1K3 voice is a mic path),
but devicectl cannot tap the face. With this launch environment set, ChatScreen
enters voice mode as soon as the brain is ready; the Mac's SelfTest env keys are
the precedent. Inert for every ordinary launch. Needed to take the render-error
count for the VPIO fix on device without a human tap.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K2kJAPEHTFU34uKiGArFSP

# ── MurphySig ──────────────────────────────────────────────────────
# Significant decision in this commit? Sign it. https://murphysig.dev/spec
# ───────────────────────────────────────────────────────────────────
@claude

claude Bot commented Sep 3, 2026

Copy link
Copy Markdown

Review — fix/vpio-output-bus (PR #205)

Small, well-scoped pair of changes: a VPIO output-bus fix in the shared transcriber, plus an opt-in iOS harness switch to reach it on-device via devicectl. Both read cleanly against the module's existing conventions.

macos/Sources/M1K3Voice/AppleSpeechTranscriber.swift:583 — the fix itself

  • Diagnosis matches the AVAudioEngine model: once setVoiceProcessingEnabled(true) turns the input node's I/O unit into a VPIO, its output element renders on every cycle regardless of whether anything is attached — and AppleSpeechTranscriber never touched outputNode. Touching mainMixerNode is the documented way to get an implicit mixer→output connection without playing anything (Apple's docs: accessing the property auto-connects it to outputNode on first touch using the output node's format). outputVolume = 0 on top is a reasonable belt-and-braces given this is an STT-only engine.
  • Confirmed audioEngine here (line 98) is a private, dedicated AVAudioEngine instance for this transcriber — not shared with TTS output — so silencing its mixer has no blast radius on the speak path.
  • The line sits in installInputTap(), which is also the reinstall path (handleConfigurationChange, line ~719) for route changes — so a Bluetooth-mic swap mid-session re-applies the fix too, not just first install. Good.
  • Locking: the new line runs inside the existing engineLock.withLock (via installTapAsOwner/handleConfigurationChange), consistent with the file's documented two-lock discipline (engineLock outer, lock inner) — no new concurrency surface introduced.
  • Minor nit, not a bug: audioEngine.mainMixerNode.outputVolume = 0 sits after enableVoiceProcessing(on:) returns, unconditionally — including the catch path where VP setup fails (VP unavailable on some aggregate/virtual devices, per the comment above enableVoiceProcessing). On those devices the render-error problem this fixes doesn't exist (no VPIO), so the line is a no-op cost rather than a bug, but the placement reads as "always needed" when it's really VP-conditional. Not worth restructuring for one property set, but if you want the code to mirror the comment precisely, gating it on inputNode.isVoiceProcessingEnabled (checked after the call) would do it.
  • No new tests, which is correct per this file's own convention — it's verify-by-launch (mic + TCC), and the PR body already documents the verify-owed device count check.

macos/M1K3iOSApp/ChatScreen.swiftM1K3_VOICE_AT_LAUNCH

  • Default-off, env-gated (ProcessInfo.processInfo.environment["M1K3_VOICE_AT_LAUNCH"] == "1"), matches the precedent cited (Mac's SelfTest/SelfTestEnv env-key pattern) — though note it reads ProcessInfo directly rather than through a shared helper, since SelfTestEnv (macos/M1K3App/SelfTest.swift) is scoped to the M1K3App target only and isn't available to M1K3iOSApp. Fine as-is; just flagging why it isn't reusing that type rather than that it should.
  • .task(id: brainReady) + the voiceLaunched @State flag correctly ensures a single trigger even though brainReady flips false/true repeatedly during normal chat (!core.chat.isResponding && core.isReady), which restarts the .task each time.
  • core.enterVoiceMode() (AppCore is @MainActor, AppCore+Voice.swift:105) is itself idempotent (guard voiceLoop == nil, …), so even a hypothetical double-fire (e.g. if the view identity resets and voiceLaunched re-initializes to false) is a safe no-op rather than a bad state — good defense in depth on top of the voiceLaunched guard.
  • Correctly placed as pure SwiftUI glue in M1K3iOSApp/ — no business logic added to the app target beyond wiring an env flag to an existing core.enterVoiceMode() call, consistent with the "dumb glue" discipline the packages/app split expects.
  • Shared file (M1K3iOS / M1K3visionOS per the header), and nothing platform-specific was added (ProcessInfo, @State, .task are all cross-platform), so this should compile fine for both targets.

Provenance

  • No new files; both touched files got Review: Kev + claude-fable-5.1, 2026-09-03 — … header blocks rather than fabricated Signed: lines, consistent with MurphySig convention for modifications to already-signed files.

Nits (non-blocking)

  • Double blank comment line before the new header block at ChatScreen.swift:27-28 (// / // / // Review: …) — cosmetic only.

Nothing here blocks merge. The fix is narrowly targeted, the harness switch is safely inert by default, and both changes stay within their established architectural lanes (pure glue in the app shell, the actual AV fix in the shared, lock-guarded transcriber).

@kpmmmurphy

Copy link
Copy Markdown
Contributor Author

Device verification — done (iPhone 17 Pro, iOS 27 beta, dev build of 0a186ae + c8dd730, launched with M1K3_VOICE_AT_LAUNCH=1, capture idevicesyslog unfiltered for render err):

measure before (same device, earlier tonight) after
libEmbeddedSystemAUs … render err: -1 21,549 in 75 s (~287/s; ~6,600 per 20 s) 0 across the whole capture (23:49:31 → 23:54:05, 3748 app-log lines)
20 s after the mic armed ~6,600 0

Turns unchanged (both spoken at the phone from the Mac):

  • Turn 1 — voice endpoint: complete thought · idle 2.7srun done "A leap year has 366 days…" → voice turn: first sentence 6796ms · synth 65ms · answer 8344ms · 1 sentence → mic re-armed (stt mic input format 48000Hz ch=1).
  • Turn 2 — idle 2.6s → "The largest planet in the solar system is Jupiter." → first sentence 1981ms · synth 5ms · answer 4004ms → re-armed.

Recognition, endpointing, TTS, and re-arm all behave as before the fix; the output bus is simply quiet now.

One observation, not chased: two stt mic input format lines 0.5 s apart at launch (23:49:35.65 / 23:49:36.13), then a single arm per turn. Looks like the route change on session activation reinstalling the tap — pre-existing, unrelated to this PR.

…e — the phone armed twice at every session start

The engine's `.AVAudioEngineConfigurationChange` handler tore the tap down and
reinstalled it unconditionally. On the iPhone that notification fires ~0.5 s
after the first arm of every voice session at an IDENTICAL format (48 kHz mono
→ 48 kHz mono): two `stt mic input format` lines and an engine stop/start for
nothing (2026-09-03 capture, iPhone 17 Pro).

`MicTapReinstallPolicy` (pure, M1K3Voice, 5 red-first tests) decides from the
format the tap was installed at, the format read back, and whether the engine
is still running: keep / restart / reinstall. Real route changes (a new sample
rate or channel count — the Bluetooth HFP switch) reinstall exactly as before;
a same-format notice keeps the tap and only restarts the engine if the system
stopped it.

Verify-by-launch on the phone: one `stt mic input format` line at arm, turns
still endpoint and re-arm.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01K2kJAPEHTFU34uKiGArFSP

# ── MurphySig template (uncomment lines below to sign this commit) ──────
# Significant change detected. If this commit encodes a decision worth
# remembering — architecture, tradeoff, non-obvious fix — sign it.
# Spec: https://murphysig.dev/spec
#
# MurphySig:
#   Signed: Kev + claude-fable-5-1, 2026-09-04
#   Format: MurphySig v0.1
#   Context: <why this exists — the thought, not the diff>
#   Confidence: <0.0-1.0> — <what's uncertain>
#   Open: <unresolved questions>
# ────────────────────────────────────────────────────────────────────────
@claude

claude Bot commented Sep 4, 2026

Copy link
Copy Markdown

Review

Solid, well-scoped fix — good root-cause diagnosis on both bugs (VPIO output-bus starvation, and the same-format config-change bounce), and the second fix is exactly the kind of thing that should get a pure, TDD'd policy type rather than more inline logic in the AVFoundation-facing code. A few notes, nothing blocking.

Nice

  • MicTapReinstallPolicy (macos/Sources/M1K3Voice/MicTapReinstallPolicy.swift) is a clean seam: Sendable value types, no AVFoundation import, three cases pinned by 5 tests including the installed: nil and channel-count edges. This is the right shape for the "which of these AVAudioEngine notifications actually matter" class of decision.
  • installedTapFormat is kept in lockstep with engineOwner — every place engineOwner gets nil'd (teardownEngineIfOwner, the startEngineIfOwner catch, the degenerate-format branch in installInputTap) also nils installedTapFormat. I traced all the reset sites (AppleSpeechTranscriber.swift:260, :579, :616) and didn't find a stale-state gap.
  • restartEngineAfterConfigurationChange() (AppleSpeechTranscriber.swift:762) is correctly documented as "caller holds engineLock" and is only ever invoked from inside handleConfigurationChange's engineLock.withLock — no double-lock/deadlock risk.
  • The mainMixerNode.outputVolume = 0 fix is scoped safely: audioEngine (AppleSpeechTranscriber.swift:104) is a private, dedicated AVAudioEngine() instance used only for STT capture — nothing else in the package attaches a player node to it, so forcing the mixer's output to silence can't clobber TTS/playback routing elsewhere.
  • New file MicTapReinstallPolicy.swift is correctly signed with Prior: Unknown, and the Signed: block states its own confidence (0.85) and what's still open — good provenance discipline, matches the MurphySig convention already in this file.
  • ChatScreen.swift's M1K3_VOICE_AT_LAUNCH switch stays true to "dumb glue": it's a one-line ProcessInfo read, guarded by a voiceLaunched state flag so .task(id: brainReady) re-firing (if brainReady ever flips false→true again) can't re-enter voice mode a second time. Correctly inert unless the env var is exactly "1".

Minor / worth a look

  1. MicTapReinstallPolicy.action's .restart case is the one leg the test suite can't actually validate against reality — it exercises the decision (format same + engine stopped → .restart) but nothing pins whether, after restartEngineAfterConfigurationChange() runs, the existing tap (installed before the stop) still delivers buffers post-restart without a reinstall. That's consistent with the PR's own "verify-owed by launch on the phone" framing, so this isn't a request to add a test — just flagging that this is the one branch where "pure logic + unit test" doesn't fully retire the risk, and it'd be worth explicitly confirming capture continues (not just that the log line is right) when you take the on-device numbers.

  2. AppleSpeechTranscriber.swift:27-28 (ChatScreen.swift around the same area) — two blank // comment lines in a row before the new Review block. Cosmetic only.

  3. Nit: handleConfigurationChange's .reinstall branch calls installInputTap() directly rather than installTapAsOwner(generation) — that's correct here (already under engineLock with ownership pre-checked, and installTapAsOwner would just re-do the same ownership check redundantly), but worth a one-line comment noting why it's the bare installInputTap() and not the "as owner" wrapper, since the two look interchangeable at a glance and a future edit could "fix" this into a subtle double-lock.

Architecture / process check

  • No MLX/Metal/GRDB touched — N/A for those sections.
  • No network egress introduced.
  • Swift 6 concurrency: MicTapFormat/MicTapReinstallPolicy are Sendable value types with no shared mutable state; the existing @unchecked Sendable on AppleSpeechTranscriber isn't touched, and the new mutable installedTapFormat field is documented as engineLock-guarded like its sibling engineOwner — consistent with the class's existing locking discipline.
  • Tests: MicTapReinstallPolicyTests.swift covers same-format/running, same-format/stopped, rate-change (both engine states), channel-change, and no-baseline — good edge-case coverage for a pure decision table.

Nothing here blocks merging.

@claude

claude Bot commented Sep 4, 2026

Copy link
Copy Markdown

Review

Focused review of the three functional pieces: the VPIO output-bus fix, the MicTapReinstallPolicy extraction, and the M1K3_VOICE_AT_LAUNCH harness switch.

1. VPIO output bus fix (AppleSpeechTranscriber.swift:607)

audioEngine.mainMixerNode.outputVolume = 0 inside installInputTap() is the right minimal touch — this engine is capture-only (no AVAudioPlayerNode/scheduled buffers ever attached to it, confirmed by grep), so silencing its mixer has no playback side effect. Good, well-reasoned inline comment tying the fix to the observed render err: -1 storm. Appropriately flagged as verify-owed by count rather than claimed proven — right call given swift test can't touch AVAudioEngine/VPIO at all.

2. MicTapReinstallPolicy (new file + AppleSpeechTranscriber.swift:732-760)

This is the strongest part of the PR — a clean extraction of a real decision (keep / restart / reinstall) into a pure, Sendable, fully-tested type, exactly the seam discipline CLAUDE.md asks for. Lock discipline checks out: every read/write of the new installedTapFormat field (:149, :260, :579, :616, :659, :739) happens under engineLock, matching the existing doc comment's convention ("engineLock-guarded like engineOwner") and the file's documented lock-ordering rule. No bare .lock()/.unlock() introduced.

Test coverage (MicTapReinstallPolicyTests.swift) hits all three actions plus the "no baseline" edge case (installed: nil) and both engine-running states for a real route change — good breadth for a 4-line pure function.

One assumption worth calling out explicitly rather than a bug: .restart relies on (a) audioEngine.isRunning == false reliably indicating the engine self-stopped per Apple's documented pre-notification contract, and (b) a kept tap (never reinstalled) still delivering audio correctly after a bare prepare()+start(). Both are exactly what "Verify-by-launch on the phone" in the commit message already earmarks, so no action needed beyond making sure that on-device pass specifically watches for a silently dead tap (no error logged, just no partials) in the .restart path, not just the render-error count — that failure mode wouldn't show up in the count this PR is instrumenting.

Minor/non-blocking: MicTapFormat (new, sampleRate/channelCount value type) sits right next to the pre-existing MicTapFormatGate.isUsable(sampleRate:channelCount:) (MicTapFormatGate.swift). Not a bug, but isUsable could now take a MicTapFormat instead of two loose primitives, which would read a little more coherently given both live in the same file's vocabulary now. Take-it-or-leave-it.

3. M1K3_VOICE_AT_LAUNCH harness switch (ChatScreen.swift:38, 116-125)

Appropriately thin — AppCore is @MainActor (AppCore.swift:51) and .task(id:) on a SwiftUI view body inherits that isolation, so the direct core.enterVoiceMode() call is safe without extra hopping. voiceLaunched + Self.voiceAtLaunch (read once, env-var gated) keeps this inert for every ordinary launch, matching the stated intent and the Mac SelfTest precedent. No business logic leaked into the app target — this is pure UI wiring, correctly left untested per the TDD_SKIP convention.

Nit: brainReady also flips on core.chat.isResponding (ChatScreen.swift:370), so .task(id: brainReady) re-evaluates on every send/response cycle, not just the readiness transition — harmless given the voiceLaunched latch, but if this switch ever needs to do more than one thing, that's worth narrowing to a dedicated id.

Cosmetic: doubled blank comment line before the new Review: block (ChatScreen.swift:27-28).

Provenance

New file MicTapReinstallPolicy.swift carries a proper Signed:/Prior: Unknown block. AppleSpeechTranscriber.swift's existing Signed: header is left untouched with new Review: entries appended, consistent with the file's own convention. No fabricated authorship.

Summary

No correctness or concurrency defects found. The lock discipline and pure-policy extraction are done well; the one thing I'd want closed out is making sure the on-device verification pass explicitly checks the .restart path keeps producing transcription partials, not just that the render-error count drops to 0.

@kpmmmurphy

Copy link
Copy Markdown
Contributor Author

Device verification of f133652 (the same-format reinstall policy) — done. iPhone 17 Pro, iOS 27 beta, dev build of this branch's tree launched with M1K3_VOICE_AT_LAUNCH=1, USB idevicesyslog capture.

The restart leg fired, exactly as the review asked to see:

19:02:28.982  stt voice processing on (echo cancellation + speech-triggered ducking)
19:02:28.983  stt mic input format 48000.000000Hz ch=1            ← ONE arm (was two)
19:02:29.423  audio configuration notice — mic format unchanged, engine stopped; restarting
19:02:44.341  voice endpoint: complete thought · idle 2.6s · required 2.5s
19:02:55.264  run done: "A leap year has 366 days — 365 days plus an extra day on February 29…"
19:02:55.312  voice turn: first sentence 7118ms · synth 1355ms · answer 10970ms · 2 sentences
19:03:05.017  stt mic input format 48000.000000Hz ch=1            ← the normal per-turn re-arm
19:05:02.480  voice endpoint: complete thought · idle 2.8s          ← spoken after ~2 min of idle
19:05:12.348  run done: "Jupiter. Confirmed by NASA…"
19:05:24.390  stt mic input format 48000.000000Hz ch=1

So on the phone Apple does stop the engine before posting the same-format notice (.restart, never .keep), and the tap kept across that restart still delivers: both turns endpointed, answered and re-armed, including one after a two-minute idle listen. render err: 0 for the whole capture. Two earlier launches on this build showed the identical single-arm + restart sequence.

One false alarm during the run, recorded so nobody chases it: a question spoken ~2 min after an earlier arm produced no log lines at all — the USB capture had died, not the app; the phone's chat-history.sqlite holds that question and its answer.

@kpmmmurphy
kpmmmurphy merged commit e59e7a0 into master Sep 4, 2026
20 checks passed
@kpmmmurphy
kpmmmurphy deleted the fix/vpio-output-bus branch September 4, 2026 18:14
@claude claude Bot mentioned this pull request Sep 5, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant