NeurIPS 2026
Sang-Hoon Lee1,* and Ha-Yeong Choi2,*
1Department of Artificial Intelligence, Ajou University, Suwon, Korea
2Frontier AI Lab, KT Corp., Seoul, Korea
WaveGen is the first end-to-end waveform text-to-speech model achieving state-of-the-art performance on the Emilia training dataset
- The base model, training, inference, and evaluation code is in src/wavegen.
- WaveGen V-Spec is the recommended model for inference. Its 1400k EMA Safetensors bundle is in
ckpts/WaveGen_VSpec/; training, inference, and evaluation are packaged in src/wavegen_v_spec. - WaveGen-Flash generates speech in four steps. Its inference code is in src/wavegen_flash, with a self-contained EMA Safetensors bundle in
ckpts/WaveGen_Flash/.
We welcome GPU resources to support the continued development of WaveGen. If you are interested in supporting this work, please contact Sang-Hoon Lee (sanghoonlee@ajou.ac.kr).
- Released WaveGen Studio, a native Windows app built with C++ and ONNX Runtime CUDA, using BF16 inference, with CUDA Graph support in the native command-line runner. No Python installation is required.
-
We released the WaveGen source code and checkpoint (WaveGen, WaveGen-VSpec) trained on Emilia (24 kHz audio, 50 Hz waveform patches)
-
WaveGen-Flash (4 Steps) is a four-step model distilled from the WaveGen (Vspec). We will share the distillation details soon
- WaveGen was accepted to NeurIPS 2026.
- WaveGen was submitted to NeurIPS 2026.
Use Python 3.10 or newer. Install PyTorch and torchaudio for your system, then install this repository:
cd WaveGen
pip install -e .FFmpeg is required for audio preprocessing. Evaluation dependencies are optional:
pip install -e ".[eval]"| Model | Checkpoint | Use |
|---|---|---|
| WaveGen | 1300k EMA | Pre-trained model |
| WaveGen (Fine-tuned with v-spec losses) | 1400k EMA | Recommended; default for CLI, Gradio, and V-Spec batch evaluation |
| WaveGen-Flash (4 Steps) | 48k EMA | Fixed four-step inference |
The public checkpoints use Safetensors and contain FP32 EMA weights only.
Download the selected model's checkpoint, YAML, and vocabulary from
Hugging Face, then place its model folder
under ckpts/ as shown below.
Results on the complete Seed-en (1,088 utterances) and Seed-zh (2,020 utterances) test sets. WaveGen and V-Spec use 50 Euler steps; WaveGen-Flash uses four Euler steps. All three models use Mel temperature 0.
| Model | EN | ZH | ||||
|---|---|---|---|---|---|---|
| WER (%) ↓ | SIM ↑ | UTMOS ↑ | CER (%) ↓ | SIM ↑ | UTMOS ↑ | |
| WaveGen Pre-trained · 1300k EMA |
1.47 | 0.755 | 3.68 | 1.75 | 0.805 | 2.87 |
| WaveGen V-Spec 1400k EMA · Recommended |
1.39 | 0.743 | 3.59 | 1.65 | 0.783 | 2.79 |
| WaveGen-Flash 48k EMA · 4 steps |
1.49 | 0.727 | 3.53 | 1.59 | 0.768 | 2.74 |
WaveGen achieves state-of-the-art performance among TTS models trained on the Emilia dataset.
See the Hugging Face model card for the evaluation settings.
WaveGen/
├── ckpts/
│ ├── WaveGen/ # base 1300k checkpoint and vocabulary
│ ├── WaveGen_VSpec/ # 1400k EMA checkpoint, configuration, and vocabulary
│ └── WaveGen_Flash/ # four-step EMA checkpoint, configuration, and vocabulary
├── data/ # dataset metadata and evaluation lists
├── docs/static/images/ # model overview (Figure 1)
└── src/
├── wavegen/ # base model
│ ├── configs/
│ ├── eval/
│ ├── infer/
│ ├── model/
│ ├── scripts/
│ └── train/
├── wavegen_v_spec/ # V-Spec training, inference, and evaluation
└── wavegen_flash/ # four-step inference only
The recommended default is WaveGen V-Spec, using the 1,400,000-update EMA checkpoint:
ckpts/WaveGen_VSpec/model_1400000.safetensors
ckpts/WaveGen_VSpec/vocab.txt
ckpts/WaveGen_VSpec/WaveGen_VSpec.yaml
Weights stay in the local ckpts/ directory and are excluded from Git. The
matching vocabulary and model configuration are included with the source.
Run from the repository root:
wavegen_infer-cli \
--ckpt_file ckpts/WaveGen_VSpec/model_1400000.safetensors \
--model_cfg ckpts/WaveGen_VSpec/WaveGen_VSpec.yaml \
--vocab_file ckpts/WaveGen_VSpec/vocab.txt \
--ref_audio reference.wav \
--ref_text "The transcript of the reference audio." \
--gen_text "The text to synthesize."The example configuration uses the same checkpoint:
wavegen_infer-cli -c src/wavegen_v_spec/infer/examples/basic.tomlThe defaults are EMA 1400k, Prompt Anchor (Prompt_xt_Anchor), Scheduled K3
(base CFG 1.25), 50 steps, Mel temperature 0, and seed 0. Inference dropout is
read from the model YAML (0.2). Prompt Anchor interpolates the prompt waveform
and Mel states from noise to the reference at each ODE step and masks prompt
states in the joint-null CFG branch. To use baseline prompt conditioning:
wavegen_infer-cli -c src/wavegen_v_spec/infer/examples/basic.toml --inference_method baselineThe inference method and CFG schedule are separate options. Select constant
guidance with --cfg_schedule constant (CFG 3 by default).
wavegen_infer-cli and wavegen_v_spec_infer-cli both use the V-Spec package.
Without reinstalling an existing environment, use
python -m wavegen_v_spec.infer.infer_cli with src on PYTHONPATH.
For the pre-trained 1300k Safetensors release, select its assets explicitly:
python -m wavegen.infer.infer_cli \
--ckpt_file ckpts/WaveGen/model_1300000.safetensors \
--model_cfg ckpts/WaveGen/WaveGen_SF_Emilia_24k_50Hz_IBDrop02.yaml \
--vocab_file ckpts/WaveGen/vocab.txt \
--cfg_schedule scheduled_k3 \
--cfg_strength 1.25 \
--ref_audio reference.wav \
--ref_text "The transcript of the reference audio." \
--gen_text "The text to synthesize."The base module retains its original local defaults; the paths above select
the public Safetensors bundle. Existing .pt checkpoints remain supported
through an explicit --ckpt_file override with matching model assets.
See inference for the available options,
including Hugging Face loading from the shared repository's WaveGen_VSpec/ folder.
Place model.safetensors, WaveGen_Flash.yaml, and vocab.txt in
ckpts/WaveGen_Flash/, then run:
python -m wavegen_flash.infer.infer_cli \
--ref-audio reference.wav \
--ref-text "The transcript of the reference audio." \
--gen-text "The text to synthesize." \
--output generated_flash.wavFlash uses four Euler steps with the time grid [0, 0.1, 0.25, 0.5, 1.0],
CFG 0, and inference bottleneck dropout 0. The matching YAML stores the complete
sampling settings, with waveform temperature 1 and initial Mel temperature 0.
Mel temperature scales only the initial Mel noise; reference conditioning and
later Mel predictions remain active. Use --mel-temperature 1 to reproduce the
previous initialization. The final Mel output head is skipped by default.
The Safetensors file contains the complete
merged EMA model; no separate base checkpoint is needed. This release includes
inference code and weights; the Flash training recipe is not included.
See the Flash guide for Python, Hugging Face, and batch inference examples.
Run the local demo with the Python environment that already runs WaveGen:
python app.pyOpen http://127.0.0.1:7860, upload reference speech, enter its transcript and
the text to generate. Advanced settings expose Prompt Anchor/baseline inference and
constant/Scheduled K3 guidance. The demo defaults to Prompt Anchor with Scheduled K3
(base CFG 1.25), the recommended V-Spec 1400k EMA Safetensors checkpoint, and 50 steps.
Inference dropout uses model.cfm.inference_bottleneck_dropout from the loaded
model YAML (0.2 in the supplied config).
Select WaveGen-Flash in the model selector for the fixed four-step model. Its sampling schedule, CFG, and dropout follow the Flash configuration.
The app was tested with the existing Gradio 6.17.3 environment. No package
installation or version change is needed in that environment. It loads the
local checkpoint on the first generation request and does not download models.
Use python app.py --help for checkpoint, device, and server options.
Prepare dataset metadata with the scripts in src/wavegen/train/datasets, and set the dataset and output paths in src/wavegen/configs/WaveGen_SF_Emilia_24k_50Hz_IBDrop02.yaml.
accelerate config
accelerate launch src/wavegen/train/train.py --config-name WaveGen_SF_Emilia_24k_50Hz_IBDrop02.yamlSee training for data preparation and training options.
The default V-Spec 1400k Safetensors batch inference and evaluation settings are in
src/wavegen_v_spec/eval/eval_infer_batch.toml. See the
evaluation guide for dataset setup and metrics.
python -m wavegen_v_spec.eval.eval_infer_batch \
--eval-config src/wavegen_v_spec/eval/eval_infer_batch.toml --full-evalSet the dataset and local metric-model paths in the TOML before running evaluation.
For Flash, generate the complete metadata list with
python -m wavegen_flash.eval.eval_infer_batch --meta path/to/meta.lst --output-dir results/flash,
then use the existing metric tools described in the
Flash evaluation notes.
WaveGen can reproduce voice characteristics from a reference recording. To support responsible use of the source code and checkpoints, please follow these guidelines:
- Consent: Use reference speech only with the speaker's informed consent and permission to use the recording. Obtain permission before sharing generated speech that reproduces their voice.
- Misuse: Do not use WaveGen for deceptive impersonation, fraud, harassment, or fabricated statements presented as authentic speech.
- Transparency: Clearly identify shared or published outputs as AI-generated audio.
- Privacy: Protect reference recordings and personal information; do not redistribute them without authorization.
These are usage guidelines, not automatic safety filters. Public-facing services should add appropriate access controls, abuse reporting, and review of suspected misuse.
- PeriodWave [ICLR 2025]: The first CFM-based waveform generation model
- FlowTokenizer [ACL Findings 2026]: The first CFM-based neural audio codec
- ReGen [ICML 2026]: The first CFM-based audio VAE
- F5-TTS [ACL 2025]: A zero-shot TTS model
- WavTTS: Concurrent work from ByteDance. We used the official WavTTS implementation as a baseline during the rebuttal period.
WaveGen source-code contributions are Copyright (c) 2026 Sang-Hoon Lee and released under the MIT License. Reused components retain their original copyright notices and licenses; see third-party notices.
The published model checkpoints are licensed separately under CC BY-NC 4.0.
This work was supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government (MSIT) (RS-2025-02283048, Developing the Next-Generation General AI with Reliability, Ethics, and Adaptability, IITP-2026-RS-2023-00255968, the Artificial Intelligence Convergence Innovation Human Resources Development, RS-2026-25617480, AI Star Fellowship Support Program(Gyeongsang National University)), National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (RS-2026-25518808), the “Advanced GPU Utilization Support Program” funded by the Government of the Republic of Korea (Ministry of Science and ICT), and Artificial intelligence industrial convergence cluster development project funded by the Ministry of Science and ICT (MSIT, Korea)&Gwangju Metropolitan City.
- Advanced GPU Utilization Support Program (WaveGen-S, WaveGen-B)
- Artificial intelligence industrial convergence cluster development project (WaveGen, WaveGen (V-Spec), WaveGen-Flash)