ChainBench-ADD is a delivery-aware audio deepfake benchmark built around matched clean parents, structured post-generation delivery chains, and protocol-aware evaluation. The repository contains the full five-stage construction pipeline, the metadata annotation/export step, and a unified baseline evaluation pipeline for the five benchmark tasks introduced in the ChainBench-ADD paper.
git clone https://github.com/wtalioy/ChainBench-ADD.git
cd ChainBench-ADD
git submodule update --init --recursiveThe main environment drives stages 1, 2, 4, 5, and the eval orchestrator.
conda create -n chainbench-add python=3.11
conda activate chainbench-add
pip install -e .The pipeline expects these tools to be available on PATH:
ffmpegffprobe
If you don't have these dependencies, run:
sudo apt install ffmpegUnder current implementation, each generator/baseline runs inside its own configured conda environment (specified in config/stage3.json and config/eval.json). You can refer to the README of each generator/baseline for installation instructions.
By default, paths resolve relative to the repo root. To run from elsewhere:
export CHAINBENCH_ROOT=/abs/path/to/ChainBench-ADDchainbench stage1 --config config/stage1.json
chainbench stage2 --config config/stage2.json
chainbench stage3 --config config/stage3.json
chainbench stage4 --config config/stage4.json
chainbench stage5 --config config/stage5.jsonchainbench fetchchainbench eval --config config/eval.jsonHelpful modes:
chainbench eval --config config/eval.json --dry-run
chainbench eval --config config/eval.json --eval-only
chainbench eval --config config/eval.json --sample-ratio 0.1
chainbench eval --config config/eval.json --tasks in_chain_detection- Construction Guide — detailed guide to the five-stage benchmark construction pipeline.
- Evaluation Guide — detailed guide to the metadata-driven evaluation pipeline and metrics.