Skip to content

tts: add audio8-TTS-0.1B ONNX INT8 CPU adapter #1

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
Guria wants to merge 1 commit into
base: main
Choose a base branch
from
Open

tts: add audio8-TTS-0.1B ONNX INT8 CPU adapter #1

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 35 additions & 0 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -166,6 +166,41 @@ rather discover samples from a directory than build `VoiceSample` by hand, use
Keep heavy imports (`torch`, model libraries) inside the adapter's methods so the
DSP, parsing, chunking, and quality modules stay importable with numpy alone.

### audio8-TTS (ONNX INT8, CPU) — second engine

Besides the reference Qwen3 adapter, [`ses/tts/audio8.py`](ses/tts/audio8.py)
ships an `Audio8TTSEngine` running
[`Audio8/audio8-TTS-0.1B-ONNX-INT8`](https://huggingface.co/Audio8/audio8-TTS-0.1B-ONNX-INT8)
entirely on CPU with plain `onnxruntime` — no torch, 44.1 kHz mono FP32
output, Apache-2.0 weights. It reproduces the vendor RAS top-p sampling and
always speaks with the model's packaged reference voice (it does **not**
clone the `VoiceSample`), needs ~437 MB of weights on first use (HF-cached,
`SES_AUDIO8_MODEL_DIR` to point at a local checkout instead), and splits long
input into pieces that fit the model's 2048-token window.

```python
from pathlib import Path

from ses import PipelineConfig, VoiceSample, run_pipeline
from ses.tts.audio8 import Audio8TTSEngine # onnxruntime/tokenizers load inside

engine = Audio8TTSEngine(
VoiceSample(name="ignored", audio=Path("in/my_voice.wav"), transcript=None),
PipelineConfig(),
)
run_pipeline(
script_text=Path("script.md").read_text(),
sample=engine.sample,
out_dir=Path("audio/manual-run"),
config=PipelineConfig(),
engine=engine,
)
```

Adapter deps: `onnxruntime`, `tokenizers`, `huggingface_hub` (fetch only —
all optional; the pure test suite and `ses` import skip them). One end-to-end
synthesis test runs when `SES_AUDIO8_E2E=1` is set.

## Script format

Sections are introduced by a heading of the form `## <timestamp> — <Title>`. The
Expand Down
6 changes: 5 additions & 1 deletion docs/DESIGN.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -53,6 +53,7 @@ ses/
tts/
base.py # TTSEngine Protocol (synthesize(text, sample) -> audio)
qwen3.py # Qwen3-TTS voice-clone adapter (reference impl)
audio8.py # audio8-TTS-0.1B ONNX INT8 CPU adapter (onnxruntime; fixed voice)
enhance/
base.py # Enhancer Protocol (enhance(audio, sr) -> audio, sr)
lavasr.py # LavaSR adapter with added-noise fallback (reference impl)
Expand All @@ -78,7 +79,10 @@ class Enhancer(Protocol): # enhance/base.py

Heavy imports (`torch`, `qwen_tts`, `LavaSR`) live **only** inside the adapter
modules and only at instantiation time — the DSP, parsing, chunking, and
quality modules are importable and testable with numpy alone.
quality modules are importable and testable with numpy alone. The audio8
adapter defers `onnxruntime`, `tokenizers`, and `huggingface_hub` the same
way; unlike the others these are light (no torch spin-up), so it works as a
second, CPU-only engine.

### Config

Expand Down
Loading