Add eval regression baseline workflow

[Tinnci]

Summary

• add a versioned same-backend release regression threshold policy
• add a Colab CLI workflow that compares immutable baseline/candidate refs
• collect sinc-resample and LavaSR matrices, MCD, reports, resolved commits, and one evidence archive
• correct load_time_seconds and total_elapsed_seconds to lower_is_better

Colab T4 evidence

Baseline v0.6.0 at 5e4241c was compared with candidate 7662458 on a Tesla T4. The workflow generated an eight-item deterministic speech evalset and ran wideband_16k and lowpass_4k matrices.

• sinc-resample: 2 runs, 16 items, 0 failures
• LavaSR baseline: 2 runs, 16 items, 0 failures
• LavaSR candidate: 2 runs, 16 items, 0 failures
• matrix comparison: 2 runs passed, 0 regressions
• same-backend quality metrics were identical
• candidate mean T4 RTF was approximately 1.796 versus 1.800 baseline

A remote follow-up verified load_time_seconds and total_elapsed_seconds are now reported as lower_is_better and regenerated the comparison archive. No model or matrix inference ran locally.

Local lightweight checks

• pixi run lint
• pixi run format-check
• pixi run typecheck
• pixi run test: 161 passed, 6 skipped
• artifact policy validation
• script syntax validation
• git diff --check

[Copilot]

Pull request overview

Adds a versioned evaluation regression threshold policy and supporting evidence workflow to enable deterministic baseline/candidate comparisons (same backend) and to document/validate the resulting artifacts. It also corrects metric direction handling so elapsed-time fields are consistently treated as “lower is better” during comparisons.

Changes:

• Introduces a release regression threshold policy artifact (eval-threshold-policy.json) plus tests validating it via load_threshold_policy.
• Adds a Colab-focused regression evidence collection script that clones immutable refs, runs eval matrices, compares results, and archives evidence.
• Updates evaluation metric direction handling to treat load_time_seconds and total_elapsed_seconds as lower_is_better, with a targeted test.

Reviewed changes

Copilot reviewed 9 out of 9 changed files in this pull request and generated 1 comment.

Show a summary per file

File	Description
tests/test_release_artifacts.py	Adds a test ensuring the new threshold policy artifact loads/validates correctly.
tests/test_eval.py	Adds coverage for elapsed-time metrics being treated as lower_is_better in comparisons.
src/audio_super_resolution/evaluation.py	Extends LOWER_IS_BETTER_METRICS to include load_time_seconds and total_elapsed_seconds.
examples/colab_eval_regression.py	New Colab regression evidence workflow script for baseline/candidate eval collection and archiving.
examples/artifacts/README.md	Documents the new threshold policy artifact.
examples/artifacts/eval-threshold-policy.json	Adds the versioned same-backend regression threshold policy.
docs/EVALUATION.md	Documents the maintained threshold policy and cautions against cross-hardware performance comparisons.
docs/ACCELERATORS.md	Adds usage instructions for running the new Colab regression evidence flow.
CHANGELOG.md	Notes the new policy/workflow and the metric direction fix.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.