loop-tts — self-verifying TTS for agents and pipelines

Your TTS engine does not tell you what it actually said. loop-tts closes the loop: every utterance is generated, transcribed by an independent ASR (Whisper), measured, and re-generated with escalating instructions until it passes quality gates. The result is a WAV file and a structured QC report.

              ┌──────────────────────────────────────────────────────┐
 text ──────▶ │  TTS backend (any HTTP endpoint: POST text → WAV)    │
              └──────────────────────────┬───────────────────────────┘
                                         ▼
              ┌──────────────────────────────────────────────────────┐
              │  independent ASR (faster-whisper) transcribes it     │
              └──────────────────────────┬───────────────────────────┘
                                         ▼
              ┌──────────────────────────────────────────────────────┐
              │  quality gates on what was ACTUALLY said:            │
              │   • real WPM   ≥ target (words transcribed / seconds)│
              │   • integrity  ≥ min   (no swallowed words)          │
              │   • max gap    ≤ limit (no mid-sentence dead air)    │
              └──────────────────────────┬───────────────────────────┘
                                       pass?  no ──▶ escalate (see below)
                                         │ yes      and retry (keep best)
                                         ▼
                              audio + QCReport

Why

TTS models fail in ways you cannot see by reading the text: they talk too slowly, swallow a word, or freeze for two seconds mid-sentence. The classic "generate → listen → maybe retry" workflow is manual and unreliable. Verifying with an independent ASR makes the quality check objective and automatic — and the QC report is machine-readable, which is exactly what agents need.

Verifying TTS output with ASR has been described recently for audiobook pipelines and in closed-loop TTS research. What loop-tts adds:

1. A complete, dependency-light implementation — one package, any backend.
2. Pluggable escalation (below) instead of a fixed retry strategy.
3. A structured QC report designed for agents — exposed as MCP tools and a REST API, so any agent can read the failure and fix the text itself.
4. verify mode — the quality gates run on audio from any TTS engine, turning the loop into a generic pre-publication QA gate.

Escalation strategies

When a chunk fails, the loop retries with a stronger "boost". Which knob it turns depends on your engine's control surface:

strategy	what changes per retry	for engines that…
instruct	the instruct string climbs a ladder (default: live on air → live urgent → breaking-news pace)	accept style instructions (e.g. Qwen3-TTS and other instruct-aware models)
parameter	a payload parameter climbs a ladder, e.g. [{"rate": "+8%"}, {"rate": "+18%"}]	expose a speed/rate parameter
none	nothing — plain retry (pure quality gate)	everything else

The verification loop is engine-agnostic: WPM, integrity, gap detection and keep-best work identically with any backend. Only the escalation ladder is swapped.

Languages

The loop is language-agnostic: it works for any language covered by both your TTS engine and the Whisper verification model (the reference Qwen3-TTS server covers 10 languages; Whisper small covers ~99). Numbers are spelled out with num2words (20+ languages) before comparing the expected text with the transcription. For unspaced scripts (Chinese, Japanese), integrity is computed per character and the rate gate counts characters per minute — set wpm_min accordingly (Mandarin broadcast is roughly 200–260 chars/min).

The default escalation ladder is written in English (it is a style instruction read by the model); translate config.boosts for best results in other languages.

Quality gates

• Real WPM — words in the transcription divided by the real audio duration (not the requested text). Catches slow, "mushy" speech.
• Word integrity — multiset recall of the expected words in the transcription, with numbers spelled out in the target language (1500 ↔ mille cinq cents) so ASR phrasing differences do not false-alarm. Catches swallowed words.
• Max internal gap — longest silence strictly inside the audio (chunk edges are excluded), detected by RMS. Catches mid-sentence dead air.
• Keep best — all attempts are scored (2·ΔWPM + 100·(1−integrity) + 50·Δgap); the best one is kept even if none fully passed.

Install

pip install loop-tts

You need any TTS backend that answers POST /tts with a WAV body (see docs/tts-server.md for a minimal example and notes on open instruction-aware models). The Whisper model (~460 MB for small) is downloaded on first use.

Quickstart (Python)

from loop_tts import TTSLoop, LoopTTSConfig

cfg = LoopTTSConfig(
    tts_url="http://127.0.0.1:8092/tts",   # your TTS backend
    language="fr",                          # fr / en / de / ...
    wpm_min=145,                            # minimum real speaking rate
    escalation="instruct",                  # or "parameter" / "none"
)
loop = TTSLoop(cfg)

path, report = loop.speak(
    "Welcome to the show. Today we talk about sleep, training and one "
    "very opinionated kettlebell.",
    instruct="Warm, conversational tone.",
)
print(report.ok, report.wpm_real, report.integrity, report.attempts)
# True 168.3 1.0 2

CLI

# single utterance (QC report on stdout, exit code 0 = passed)
loop-tts speak "Bonjour, tout le monde." --instruct "Dynamique" --out hello.wav

# multi-voice piece from a JSON file
loop-tts batch segments.json --out episode.wav

# quality-gate ANY existing audio (any TTS engine)
loop-tts verify hello.wav --text "Bonjour, tout le monde."

# HTTP API (localhost by default) — docs at /docs
loop-tts serve --port 8300

# MCP server (stdio)
loop-tts mcp

HTTP API

route	body	returns
POST /v1/tts	{text, speaker?, instruct?, min_wpm?}	{ok, audio_url, report}
POST /v1/tts/batch	{segments: [{text, speaker?, instruct?}], ...}	{ok, audio_url, reports[]}
POST /v1/verify	{audio_path, text}	{ok, report}
GET /v1/health	—	backend + STT status, active gates
GET /v1/audio/{name}	—	the WAV file

The server binds to 127.0.0.1 by default. If you expose it, set LOOP_TTS_API_KEY: every route then requires the X-API-Key header.

For agents (MCP)

loop-tts mcp exposes four tools:

• tts_speak(text, speaker?, instruct?, min_wpm?) → audio path + QC report
• tts_batch(segments) → assembled audio + one report per segment
• tts_verify(audio_path, text) → QC report on any existing audio
• tts_status() → health

The interesting part is the report. When report.ok is false, the report tells the agent what failed (failed_checks), what was actually said (transcribed), and per-attempt detail. The agent can then do what a human would do but faster: rewrite the offending text (paraphrase a swallowed proper noun, split a long sentence, remove an ambiguous opening word) and call tts_speak again. The loop escalates the delivery; the agent can escalate the content.

Configuration

All knobs are plain values in LoopTTSConfig (or env vars):

setting	default	meaning
tts_url	http://127.0.0.1:8092/tts	TTS backend endpoint (LOOP_TTS_URL)
language	en	any code your TTS + Whisper share: fr, de, ru, ja… (LOOP_TTS_LANGUAGE)
speaker	None	opaque pass-through to the backend (LOOP_TTS_SPEAKER)
wpm_min	145	real WPM gate (LOOP_TTS_MIN_WPM)
integrity_min	0.80	word-integrity gate
max_internal_gap	1.6	silence gate (seconds)
max_retries	4	total attempts = 1 + max_retries
escalation	instruct	instruct / parameter / none
boosts	3-level pace ladder	instruct ladder (level 0 = baseline)
param_boosts	[]	payload ladder for parameter mode
chunk_max_words	45	split long texts on sentences (0 = off)
guard_enabled / guard_words / guard_prefix	off	opt-in guard for FR/EN mixes: chunks starting with an ambiguous word (e.g. Salut) get a native filler prepended so the engine doesn't switch accent mid-show; customize for your own language pair
stt_model / stt_device / stt_compute	small / cpu / int8	Whisper verification model
out_dir	out/	where WAV files are written (LOOP_TTS_OUT)
segment_gap_s	0.2	silence between segments in batch()

Security

• No secrets in code or config defaults; the TTS URL is a plain setting.
• HTTP API binds to localhost by default; optional LOOP_TTS_API_KEY shared key; audio is only served from the configured out dir (no path traversal).
• Generated audio and QC reports are local files — nothing is uploaded.

Project layout

loop_tts/
├── config.py    # every knob, one place
├── metrics.py   # WPM, word integrity, internal gap detection
├── stt.py       # lazy faster-whisper wrapper
├── core.py      # the loop: chunk → generate → transcribe → measure → escalate
├── api.py       # FastAPI app
├── mcp.py       # MCP server
└── cli.py       # loop-tts speak|batch|verify|serve|mcp

Disclaimer

loop-tts is an independent project. Qwen3-TTS is referenced only as an example of an open instruction-aware TTS model; loop-tts works with any backend exposing a simple HTTP endpoint.

License

MIT — see LICENSE.