Add licensed LibriSpeech eval baseline

CHANGELOG.md

@@ -19,6 +19,8 @@ This project follows semantic versioning once published to PyPI.
 
 - Added a versioned release regression threshold policy and a Colab CLI workflow for same-backend
   baseline/candidate matrix evidence.
+- Added a pinned, license-recorded LibriSpeech `dev-clean` tiny baseline specification and
+  Colab-only real-speech matrix workflow.
 
 ## 0.6.0 - 2026-07-12
 

ROADMAP.md

@@ -55,13 +55,18 @@ The harness is implemented; the missing value is a repeatable evidence set.
 
 Next deliverables:
 
-1. Define a small licensed speech BWE evaluation set outside the repository.
-2. Commit a threshold-policy example for release regression use.
-3. Record baseline matrices for `sinc-resample` and gated `lavasr-compat` runs.
-4. Add precomputed ASR transcript evidence before integrating any real ASR runtime.
-5. Export one blind listening bundle and document how results map back to objective and stability
+- [x] Define a small licensed speech BWE evaluation set outside the repository.
+- [x] Commit a threshold-policy example for release regression use.
+- [x] Record baseline matrices for `sinc-resample` and gated `lavasr-compat` runs.
+- [ ] Add precomputed ASR transcript evidence before integrating any real ASR runtime.
+- [ ] Export one blind listening bundle and document how results map back to objective and stability
    tables.
 
+First real-speech finding: the eight-speaker LibriSpeech `dev-clean` T4 run had no stability
+failures, but `sinc-resample` substantially outperformed `lavasr-compat` on full-reference
+wideband-16k fidelity metrics. This prevents stable promotion from objective metrics alone and makes
+downstream ASR plus blind listening evidence the next required decision inputs.
+
 Do not collapse quality, downstream usefulness, speed, stability, and governance into one ranking.
 The evaluation policy lives in [docs/EVALUATION.md](docs/EVALUATION.md).
 

docs/ACCELERATORS.md

@@ -225,6 +225,22 @@ also contains a `sinc-resample` matrix, the threshold policy, reports, resolved
 matrix comparison. Override `ASR_BASELINE_REF` and `ASR_CANDIDATE_REF` only with reviewable refs;
 recorded evidence always includes the resolved commits.
 
+For the licensed real-speech baseline, run the pinned LibriSpeech workflow in a fresh T4 session:
+
+```sh
+colab new -s asr-librispeech --gpu T4
+colab exec -s asr-librispeech -f examples/colab_librispeech_eval.py --timeout 3600
+colab download -s asr-librispeech \
+  /content/asr-librispeech-evidence.tar.gz \
+  runs/asr-librispeech-evidence.tar.gz
+colab stop -s asr-librispeech
+```
+
+The workflow downloads the official `dev-clean` archive remotely, verifies its published MD5,
+retains the source license/README/speaker metadata, selects four female and four male speakers, and
+converts one utterance per speaker to 48 kHz. It then runs matching sinc/LavaSR matrices. The source
+archive, converted audio, and model computations never run or persist in the local repository.
+
 For upstream LavaSR parity, install upstream dependencies deliberately and use the environment gates
 in [../tests/README.md](../tests/README.md). Validate AudioSR separately because its external package
 owns checkpoint and accelerator behavior.

docs/EVALUATION.md

@@ -419,6 +419,28 @@ evalsets/
 Keep this dataset outside the repository unless licensing allows redistribution. Default tests use
 generated audio fixtures only.
 
+The first maintained real-speech baseline is specified by
+[`examples/artifacts/librispeech-dev-clean-tiny-v1.json`](../examples/artifacts/librispeech-dev-clean-tiny-v1.json).
+It pins the official LibriSpeech `dev-clean` archive URL and MD5, records the CC BY 4.0 license, and
+selects one utterance from four female and four male speakers deterministically. The Colab workflow
+downloads and converts the audio remotely; neither the source archive nor converted WAV files are
+committed.
+
+The first T4 run covered eight utterances (about 61 seconds total) with `wideband_16k` and
+`lowpass_4k`. Both backends completed all 16 items without stability failures. Mean results included:
+
+| Degrader | Backend | SI-SDR dB | LSD dB | High-band LSD 8–16 kHz | MCD | RTF |
+| --- | --- | ---: | ---: | ---: | ---: | ---: |
+| `wideband_16k` | `sinc-resample` | 43.88 | 5.57 | 6.45 | 2.68 | 0.098 |
+| `wideband_16k` | `lavasr-compat` | 28.62 | 26.94 | 31.78 | 46.71 | 0.175 |
+| `lowpass_4k` | `sinc-resample` | 20.81 | 19.68 | 14.19 | 52.45 | 0.003 |
+| `lowpass_4k` | `lavasr-compat` | 20.81 | 19.82 | 14.17 | 52.45 | 0.074 |
+
+These figures are evidence, not an aggregate ranking. The wideband result shows that LavaSR's
+generated high-frequency content is penalized heavily by reference-fidelity metrics. Stable
+promotion therefore requires downstream ASR and blind listening evidence rather than interpreting
+matrix completion or one spectral score as proof of improvement.
+
 ## Golden Compatibility Validation
 
 Golden validation compares a package-owned compatible backend with an upstream output or a

examples/artifacts/README.md

@@ -8,5 +8,7 @@ These files are small, static examples of the JSON artifacts produced by the CLI
 - `eval-threshold-policy.json`: conservative same-backend release regression limits for matrix
   comparison. It intentionally excludes performance thresholds until repeated device-specific
   variance is available.
+- `librispeech-dev-clean-tiny-v1.json`: source, checksum, license, deterministic selection, and
+  storage policy for the remote-only real-speech evaluation baseline. No dataset audio is committed.
 
 They are intended for release notes, documentation, and downstream CI examples.

examples/artifacts/librispeech-dev-clean-tiny-v1.json

@@ -0,0 +1,28 @@
+{
+  "schema_version": 1,
+  "dataset_id": "librispeech-dev-clean-tiny-v1",
+  "purpose": "Small licensed real-speech full-reference baseline for remote audio super-resolution evaluation.",
+  "source": {
+    "name": "LibriSpeech ASR corpus",
+    "subset": "dev-clean",
+    "url": "https://www.openslr.org/resources/12/dev-clean.tar.gz",
+    "archive_md5": "42e2234ba48799c1f50f24a7926300a1",
+    "project_url": "https://www.openslr.org/12",
+    "license": "CC BY 4.0",
+    "license_url": "https://creativecommons.org/licenses/by/4.0/"
+  },
+  "selection": {
+    "speaker_count": 8,
+    "female_speakers": 4,
+    "male_speakers": 4,
+    "strategy": "Sort dev-clean speaker IDs by sex and choose the first four female and first four male speakers; choose the first FLAC utterance for each speaker.",
+    "language": "en",
+    "source_sample_rate": 16000,
+    "target_sample_rate": 48000
+  },
+  "storage_policy": {
+    "commit_audio": false,
+    "run_location": "Colab CLI remote runtime",
+    "evidence_location": "Ignored runs/ directory or GitHub issue/release attachment"
+  }
+}