Improve live mic transcription accuracy at all three stages

[MyNamesEMurray]

Summary

Voice awareness loses accuracy in three places (capture, decode, text repair) and until now the app only worked on the last one. This works all three: a local speech-to-text engine mode, a generated Whisper initial prompt, and phonetic recovery of mangled names, plus a diagnostic that names the LocalVocal setting behind a bad transcript.

Changes

Decode: we now bias the transcriber. We knew the ghost names, the channel, the game and speech.vocabulary, told all of it to the brain, and never told the transcriber that was mishearing it. buildInitialPrompt() writes Whisper's initial prompt from those names. Settings → Voice → LocalVocal tuning shows it with a copy button, next to the preset of plugin settings that decide whether a transcript is usable at all (src/localvocal.js, quoted in LocalVocal's own labels so they can actually be found).

Text repair: phonetic recovery of tracked names. A mishear was only ever fixed if the 20-second mic-check script happened to provoke it. Transcript lines now also get a phonetic pass against the tracked names, so a mangling nobody read aloud is still recovered. Deliberately hard to fire, because a wrong auto-correction silently rewrites real speech for a whole stream: names of 4+ characters, phonetic equality plus a character-similarity floor of 0.6, a stricter 0.8 floor before two spoken words are fused into a one-word name, and the streamer's own word fixes run first and win. Off with speech.matchVocabulary.

Capture: a new engine transcript mode. src/sttengine.js connects to a speech-to-text server on the streamer's own machine (whisper.cpp server, faster-whisper-server, WhisperLive), which gives us what a caption plugin cannot: voice activity gating, so silence is never decoded into filler; a confidence per line, so a half-heard line is dropped rather than answered out loud; and the initial prompt wired in automatically and re-sent when the detected game changes. LocalVocal stays the default and the other modes are unchanged.

Voice health. src/voicehealth.js reads the transcript and names the misconfiguration behind what it sees: duplicate redraws mean partial transcription is on, fragment lines mean the buffer is too short, filler in the silences means VAD is off, identical lines in a row mean the decoder is stuck. Reported and never acted on, since every diagnosis is inferential.

Deliberately not an OBS plugin: a plugin runs inside the OBS process, and the crash already documented in the README is LocalVocal faulting there and taking OBS with it. A separate process buys the same thing at a fraction of the risk.

Verification

• Full CI reproduced locally: syntax check over every JS file, config.example.json parse, npm test (162 tests, 0 failures; the 1 skip is the pre-existing Windows-only SAPI test), npm run smoke → SMOKE OK.
• New tests: phonetic matching and the initial prompt in test/speech.test.js, the engine client in test/sttengine.test.js (fake socket, no network), the diagnostics in test/voicehealth.test.js, and the feed wiring in test/transcript.test.js.
• Booted the server headless and exercised the endpoints for real: /api/voice/tuning returned On stream tonight we talk about Wisp, Beacon, emurray, and Hollow Knight. plus the 8-step preset; /api/voice/health returned all five checks.
• Through the live feed: "bacon" → "Beacon", "hollow night" → "Hollow Knight", a streamer correction winning over the phonetic pass, "pecan pie" left alone, "Thank you." dropped as filler but still counted as evidence, and a non-loopback engine URL refused before dialing.
• Testing surfaced a real false positive, "back on" being rewritten to "Beacon". That is what the stricter fused-pair floor exists for, and it is now a test.

Risk & rollout

• Config: three new keys, all defaulted in config.example.json and documented in the README table: transcript.engineUrl, transcript.engineMinConfidence, speech.matchVocabulary. transcript.mode gains engine. No migration; existing configs deep-merge and behave exactly as before.
• The auto-correction is the thing to review. It rewrites transcript text before the cast sees it, so a bad rule is silent and lasts a stream. The guards and their thresholds are the load-bearing part; speech.matchVocabulary: false turns it off entirely and the streamer's own fixes always win.
• Security posture unchanged. The engine client refuses anything but 127.0.0.1 / localhost / [::1], and does not resolve hostnames (the same rebinding trap the Host check closes). Audio still never leaves the machine, and nothing here touches the API-key or Origin handling.
• Ethics unchanged: no bot output, overlay, AI badge or "simulated viewers" watermark is touched, and all Twitch access stays read-only.
• Cost unchanged: no extra generations. Better transcript accuracy slightly reduces wasted voice replies.
• ws is imported lazily in src/transcript.js, so the non-engine modes (and their tests) still pull nothing.
• Carries Release-Bump: minor.

Checklist

• npm run smoke passes locally (or CI is green)
• Follows the design language in DESIGN.md (if UI changed)
• AI-labeling / "simulated viewers" guarantees intact (if bot output or overlay changed)
• Docs updated (README / config reference) if behavior or config changed
• Targets main

🤖 Generated with Claude Code

https://claude.ai/code/session_01LWa54BF5RjHWC5GFaENZMM

---

Generated by Claude Code