Add Parakeet TDT v2 for English-only transcription

[kedev]

Summary

This PR adds support for Parakeet TDT v2 as an additional transcription model option alongside Parakeet v3 and the existing Whisper models.

Thank you so much for the time & effort you put into this project! I use this daily and it’s been a big help with keeping on top of my workload. I’ve been trying to find a project like this for months and was stoked to come across one of your posts linking to it. The quality and attention to detail you’ve added with the UX and features is fantastic.

Let me know if you have any feedback, happy to fold it in.

Why

My main motivation is to fix an issue I’ve run into with v3, which is when someone speaks English with a strong accent, the model will occasionally interpret the speech as another language and start transcribing in that language. Since v2 only supports English, it avoids that issue and keeps the transcription in English even when accents are heavy.

Another benefit of adding v2 is that it performs better in benchmarks (lower error rate) than v3 on English transcriptions. Since all of my meetings are in English, that is very much appreciated, even if the difference is small.

So in summary:

• English-only transcription avoids unintended language switching with heavily accented English. Gives users a choice between:
  • Parakeet v2 for English-focused transcription.
  • Parakeet v3 for multilingual transcription.
• Slightly better English transcription accuracy than v3.
• FluidAudio already supports both v2 and v3, so this builds on the existing Parakeet integration rather than introducing a new transcription backend.

Product Impact

• Affects: dictation, meetings, agent artifacts (plus model-selection UI/docs).
• Lane: meeting reliability / dictation reliability.
• Why this matters: provides an explicit English-only choice through the existing shared local STT path. Parakeet v3 remains the default; both Whisper choices are preserved.

What changed

• Added Parakeet TDT v2 using the existing pinned FluidAudio 0.15.4 backend.
• Made model readiness, downloads, cache validation and bundled-model lookup version-aware.
• Preserved active concrete model ownership across overlapping work, and drained canceled native initialization/cleanup before loading a successor.
• Protected supported v2 caches and retained guarded legacy migration without overwrite.
• Added parakeet_v2_local to saved artifacts while preserving v3's parakeet_local and existing footer text.
• Added identity, cache migration, ownership, cancellation and artifact-reader regressions, including QA validation of all supported local engine identifiers.

How I checked it

Verified feature commit: 2c6e2bc1, Apple Silicon, Xcode 26.6 with Metal Toolchain 17F109.

• python3 scripts/dev/agent-context.py --base 792d82e1 and bash scripts/dev/agent-preflight.sh: checked owners and the complete changed-path matrix union.
• Matrix union: bash build-deps.sh --force, bash build.sh --no-open, bash run-tests.sh, bash run-integration-smoke.sh, python3 scripts/dev/check-build-source-lists.py, bash run-e2e-smoke.sh, shell syntax checks for both integration entrypoints, and swift test --package-path Tools/TranscriptedQA.
• TRANSCRIPTED_RUNTIME_BUDGET=1 bash build.sh --no-open: exit 0; 88.5 MiB app, 937 ms interactive launch. No live dictation samples, so runtime latency/RTF remains unmeasured. Ambient on-demand model-ready events are not a controlled load-time benchmark.
• Pinned dependency build, including Metal shaders.
• Final full strict-artifact QA bench: 15/15 checks passed, exit 0.
• Full app build and isolated launch smoke; 984 ms to interactive against a 3000 ms budget.
• Fast suite: 12,746 assertions passed, zero failed.
• Integration and deterministic artifact E2E smoke.
• Core package: 1,079 tests executed, 13 skipped, zero failures.
• QA package: 67 tests passed, including real-artifact engine compatibility regressions.
• Real v2 and v3 inference on locally generated English speech, including a network-denied cached run.
• Production lifecycle executor harness: 45 assertions on actual executor against delayed fake dependencies; integrated into integration smoke. Feature 774d8fe9; fast/integration reruns pass.
• Computer Use: all four choices visible; actual v2/v3 synthetic native imports saved expected text and correct distinct engine identifiers; v2 selection persisted across restart.
• Computer Use: both Whisper choices transcribed the same synthetic input and saved the correct distinct engine identifiers. Turbo's cold-start import needed retry once the model was Ready.
• Computer Use: changed selection from v2 to v3 during a 48-minute synthetic import. Saved job stayed v2; v3 loaded only after that job's cleanup.
• Independent full-diff review: no must-fix findings; reviewer independently reran the 45-assertion executor. Harness limitations remain explicit.
• Independent incremental review through 2c6e2bc1: no must-fix; no outstanding code-review blockers.

Final app-created artifact validation: 94 checks pass, zero fail, with seven expected warnings for absent dictations and optional capture-quality metadata on six synthetic imports. Live microphone-dependent dictation/meeting/overlap and hardware paste-back tests were intentionally skipped: the test Mac has no microphone, and permission recognition was unresolved. Full-app network denial was not separately tested; network-denied inference was tested in the exact-dependency harness. Synthetic inference is execution proof, not accent-quality or Bluetooth/Zoom hardware proof. Existing upstream compiler warnings are not represented as warning-free builds.

Risk Review

• Privacy/local-first behavior reviewed: no new off-device payloads or services, permission changes, or real-time audio callback changes.
• Storage/migration impact reviewed: v3 identifiers/fallback retained; compatible caches preserved; no destination merge/overwrite. Format docs, reader smoke and strict QA engine validation updated.
• Public copy is concrete: opt-in English-only v2, unchanged default, no unsupported accuracy claim.
• Release/update impact: N/A — no version, Sparkle, Homebrew, signing, or release changes. Uses the existing pinned FluidAudio model-distribution path.
• Sanitized UI screenshot retained under .agent-review/visuals/; no committed workpad.
• No private transcripts/audio, credentials, personal paths or customer content included.

Notes

Sanitized Settings screenshot

The shared-engine lifecycle changes are needed to prevent native load overlap and preserve active-job model identity when switching variants; this is not a general Speech refactor. Meetings still use the existing adapter/router and Core remains a library boundary.

Additional Notes

Written and prepared with GPT-6 Astra in Codex. Reviewed with Fable 5.1

Diff size

Measured with git diff --numstat 792d82e1 HEAD after the requested cleanup; counts describe the complete contribution, not just the last commit. These are changed text lines, including comments and whitespace, not executable-LOC estimates.

Category	Text files	Added	Removed	Net
Documentation	5	62	2	+60
Main code	10	417	131	+286
Test code and runners	15	773	1	+772
Total	30	1,252	134	+1,118

Classification: all changed Markdown files are documentation, including test READMEs. Main code includes nine app source files plus the QA validator implementation. Test code includes fixtures, test suites and both test-runner scripts. One additional sanitized Settings JPEG (62,712 bytes) has no text-line count: 31 changed files overall.

Main-code files changed

• Sources/Support/TranscriptionModelPreferences.swift — adds v2 as a persisted choice, English-only labels, variant-specific cache requirements and a distinct artifact identifier; keeps v3 as default and existing identifiers compatible.
• Sources/Support/ExistingInstallModelPrefetchPolicy.swift — extends existing-install prefetch eligibility to the selected Parakeet variant, so v2 receives the existing background-download behavior.
• Sources/Support/ModelCacheInventory.swift — recognizes complete v2/v3 caches, protects supported v2 directories from stale-cache deletion, and migrates legacy folders without merging, overwriting or following linked endpoints.
• Sources/Speech/ParakeetEngine.swift — tracks requested versus loaded variant, exposes variant-specific readiness/local-file checks, and finishes deferred teardown after active transcription drains.
• Sources/Speech/ParakeetModelInitDiagnostics.swift — adds selection/prefetch admission rules, a cancellation-safe teardown gate, native-task draining and variant/generation tokens; validates bundled files for the requested variant.
• Sources/Speech/ParakeetModelLifecycle.swift — downloads/loads the selected FluidAudio version, rejects stale asynchronous results, preserves active inference and waits for canceled loads/cleanup before allocating a successor.
• Sources/Speech/TranscriptionModelWarmupOwnership.swift — maps both Parakeet variants to the same runtime, reusing foreground ownership so overlapping work cannot unload each other's model.
• Sources/Speech/STTRouter.swift — routes readiness, warmup and inference by concrete model; establishes recording identity before async warmup, preserves active leases and forwards the matching published download state.
• Sources/UI/Settings/TranscriptedSettingsView.swift — updates the existing Model help text to explain v2 versus v3 and shared-runtime overlap behavior; no new settings screen.
• Tools/TranscriptedQA/Sources/TranscriptedQA/Validators/TranscriptValidator.swift — accepts the new parakeet_v2_local artifact identifier while retaining strict rejection of unknown engines.

Key changes and user impact

• Opt-in English-only transcription
  • User-facing: choose Parakeet V2 in the existing Model picker for dictation and meetings/imports; v3 remains the multilingual default and both Whisper choices remain.
  • Technical: distinct persisted identity routes through the existing shared engine and pinned FluidAudio v2 API.
  • When/benefit: English-only users can deliberately select an English-only model. This adds choice; it does not establish better accented-English accuracy.
• Stable model ownership during switching and overlap
  • User-facing: changing the preference does not replace a model underneath an active job. Overlapping work on the same runtime shares the active model until it finishes.
  • Technical: concrete foreground leases, early recording identity, generation checks and cleanup draining prevent stale publication and overlapping native loads.
  • When/benefit: switching during recording/import, canceling warmup or retrying initialization is designed to preserve job identity and avoid disruption. Active-import switching was tested; microphone overlap remains skipped.
• Reusable downloads and accurate readiness
  • User-facing: the selected variant's download/cache state is shown, and compatible existing downloads are retained.
  • Technical: variant-specific layout validation, safe legacy-folder migration and selected-variant prefetch avoid treating v3 files as ready v2 files or deleting supported caches.
  • When/benefit: first-time v2 selection, return to a cached model and older installations can reuse compatible files and avoid needless re-downloads; incomplete caches remain eligible for repair.
• Correct saved provenance and compatible validation
  • User-facing: v2-generated transcripts identify the actual model; existing v3 artifacts retain their established identity.
  • Technical: the new engine identifier flows through shared metadata, format documentation, reader coverage and the QA allowlist.
  • When/benefit: reviewing/exporting captures or validating an existing library accurately distinguishes v2 output without falsely rejecting it.
• Regression coverage without additional app features
  • User-facing: no new controls, dependencies or runtime services are introduced by the tests.
  • Technical: model/cache/ownership tests, delayed fake-dependency tests of the production lifecycle executor and artifact regressions exercise switching and compatibility.
  • When/benefit: helps catch future regressions in the new variant and unchanged defaults; deterministic tests are not a substitute for the explicitly skipped hardware checks.

Agent handoff

COORD_DONE: GREEN | no PR yet | optional v2 with safe shared model ownership | no GitHub cleanup | human draft review; live checks intentionally skipped | full QA 15/15, four native model imports, active-import switch, independent diff review | lanes used: Codex=full independent reviewer plus cache/artifact reviewer; Claude=skipped, covered by Codex; Local=skipped, no triage needed; Windows=skipped, macOS-only scope | human reviews and opens draft PR

[kedev]

@r3dbars - Fixed merge conflicts. Let me know if you have any other feedback. Thank you! 🙏