test(analysis): add Tier 1 real-audio accuracy acceptance

AGENTS.md

@@ -63,6 +63,7 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working
 ## Architecture references
 - `ARCHITECTURE.md`
 - `docs/engineering/acceptance-criteria.md`
+- `docs/doctoring/real-audio-accuracy-acceptance.md`
 - `docs/engineering/harness-engineering.md`
 - `docs/workflow/one-day-delivery-plan.md`
 - `docs/workflow/pr-continuity.md`
@@ -85,6 +86,7 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working
 - Prefer practical, friendly, rehearsal-first wording over academic or authority-heavy language.
 - Do not reduce the product to a chord analyzer when form, timing, player coordination, simplification, and setup cues are the real rehearsal blockers.
 - Do not frame usability as a reason to accept weak analysis quality; BandScope should aim for both easy use and high accuracy.
+- Do not claim harmony or tempo accuracy from mocked feature matrices. Tier 1 decoded-PCM fixtures and metrics live in `docs/doctoring/real-audio-accuracy-acceptance.md`.
 
 ## Safety
 - Do not add network-dependent runtime paths for local analysis.

ARCHITECTURE.md

@@ -1,6 +1,6 @@
 # ARCHITECTURE.md
 
-Last updated: 2026-03-11
+Last updated: 2026-08-16
 
 ## Brand source
 
@@ -127,3 +127,4 @@ Last updated: 2026-03-11
 - Supply-chain docs, workflow pinning, and lockfile verification are part of the default quickcheck path so dependency drift is caught early.
 - Quickcheck and CI are expected to verify dependency review, audit, supplemental inventory, and SBOM baseline presence as part of bootstrap.
 - Cross-platform build workflow presence and trigger coverage are part of the default supply-chain verification path.
+- Real-audio accuracy acceptance is a separate buyer-facing gate from unit coverage. Tier 1 decoded-PCM fixtures, duration-weighted chord recall, and tempo Acc1 live in `docs/doctoring/real-audio-accuracy-acceptance.md` and `bandscope_analysis.accuracy`.

CHANGELOG.md

@@ -4,6 +4,7 @@
 
 ### Added
 
+- Add Tier 1 real-audio accuracy acceptance: a decoded C major WAV must recover `C`, and a 120 BPM click WAV must pass tempo Acc1, with checksum-fail-closed reports.
 - Display the analyzed song tempo (BPM) as a badge in the rehearsal workspace.
 - 각 합주 역할(Role)별 개인 연습 진행도를 0~100% 범위로 기록 및 시각화할 수 있는 연습 진척도(`practiceProgress`) 트래커 기능 추가. UI 컨트롤(슬라이더 및 +/- 버튼)과 한/영 다국어 지원 포함.
 

CLAUDE.md

@@ -73,3 +73,4 @@ Supporting packages:
 - Lockfiles (`package-lock.json`, `uv.lock`, `Cargo.lock`) are committed and must stay in sync; GitHub Actions are SHA-pinned. Adding a direct dependency requires the admission rationale defined in `AGENTS.md` and `docs/security/dependency-policy.md`.
 - CI beyond quickcheck: `gate / ci / rust-check` (Tauri cargo check on macOS) and `build-baseline` Windows/macOS amd64+arm64 native builds are merge gates, alongside CodeQL, dependency-review, sbom, bandit, trivy, secret-scan, and security-audit workflows. Do not weaken or skip them.
 - Version metadata lives in `VERSION`, the root `package.json`, and `CHANGELOG.md`; release flow is tag-driven (see `docs/operations/deploy-runbook.md`).
+- Real-audio accuracy acceptance (decoded PCM, not mocked chroma) lives in `docs/doctoring/real-audio-accuracy-acceptance.md` and `bandscope_analysis.accuracy`.

docs/doctoring/real-audio-accuracy-acceptance.md

@@ -0,0 +1,67 @@
+# Real-audio accuracy acceptance
+
+Next action: before claiming a rehearsal result is accurate, run the Tier 1
+decoded-PCM cases. A green unit suite that never opens a WAV is not buyer
+evidence.
+
+```bash
+uv run --project services/analysis-engine pytest \
+  services/analysis-engine/tests/test_accuracy_acceptance.py
+```
+
+## Why this lock exists
+
+BandScope sells rehearsal guidance. A buyer cannot distinguish “the job
+succeeded” from “the engine heard the chord and tempo that are in the file”
+until decoded PCM is scored against a known label (Raffel et al., 2014).
+
+This layer is Tier 1 of issue #770: tiny, license-clean, deterministic WAV
+fixtures generated in process. It does not claim genre coverage, perceptual
+stem quality, or private-corpus readiness.
+
+## Held cases
+
+- `c-major-triad`: three seconds of C4+E4+G4 written to WAV, decoded, and
+  scored with duration-weighted chord recall. Pass when recall of `C` is at
+  least `0.70` (Odekerken et al., 2021).
+- `click-120-bpm`: eight seconds of 120 BPM clicks decoded by
+  `TemporalAnalyzer`. Pass when estimated tempo satisfies Acc1 at 4%
+  (Schreiber & Müller, 2020). Acc1 does not credit half-time or double-time.
+- Checksum mismatch fails closed. Do not score a tampered file as a pass.
+
+## Claim boundary
+
+A passing case supports only the registered fixture, metric, engine version,
+and tolerance. It does not establish universal musical correctness.
+
+Keys-left, keys-right, and acoustic-guitar roles still carry arrangement
+defaults (`C#`, `Emaj7`, `Eb`). Lead vocal harmony is the role that currently
+reflects the `other` stem recognizer. Do not treat those defaults as
+measured accuracy.
+
+## References
+
+Odekerken, D., Koops, H. V., & Volk, A. (2021). Improving audio chord
+estimation by alignment and integration of crowd-sourced symbolic music.
+*Transactions of the International Society for Music Information Retrieval,
+4*(1), 141–155. https://doi.org/10.5334/tismir.81
+
+Raffel, C., McFee, B., Humphrey, E. J., Salamon, J., Nieto, O., Liang, D., &
+Ellis, D. P. W. (2014). MIR_EVAL: A transparent implementation of common MIR
+metrics. In *Proceedings of the 15th International Society for Music
+Information Retrieval Conference* (pp. 367–372).
+
+Schreiber, H., & Müller, M. (2020). Music tempo estimation: Are we done yet?
+*Transactions of the International Society for Music Information Retrieval,
+3*(1), 111–125. https://doi.org/10.5334/tismir.43
+
+## Security Notes
+
+- Attack surface: generated WAV bytes, SHA-256 digests, and decoded PCM
+  passed into `ChordRecognizer` and `TemporalAnalyzer`.
+- Trust boundary: untrusted audio and manifests; trusted repo-controlled
+  fixture generators and metric floors.
+- Mitigations: no network, no shell, checksum fail-closed, bounded fixture
+  durations, no copyrighted commercial recordings.
+- Test points: deterministic digest, C major recall, 120 BPM Acc1, checksum
+  mismatch, malformed manifest, silence must not pass as C major.

docs/plans/2026-08-16-real-audio-accuracy-acceptance.md

@@ -0,0 +1,50 @@
+# Real-audio accuracy acceptance (Tier 1)
+
+**Goal:** Give BandScope a buyer-readable accuracy gate: decoded PCM from a
+known WAV must recover the expected chord or tempo, with a versioned report.
+
+**Architecture:** A new `bandscope_analysis.accuracy` package generates
+license-clean fixtures, scores them with documented MIR metrics, and emits a
+strict case-report schema. Tests write real WAV files and call production
+`ChordRecognizer` and `TemporalAnalyzer` helpers. Stem separation stays out
+of this slice.
+
+**Tech Stack:** NumPy, soundfile, librosa beat tracking, pytest.
+
+## Security Notes
+
+### Attack surface
+
+Generated WAV files, SHA-256 digests, decoded PCM arrays, and JSON-shaped
+accuracy reports.
+
+### Trust boundary
+
+Untrusted: any on-disk fixture bytes and any parsed report mapping.
+Trusted: in-repo generators, metric definitions, and registered floors.
+
+### Mitigations
+
+- No network and no shell interpolation.
+- Checksum mismatch raises before tempo scoring.
+- Manifest parsing fails closed on missing or mistyped fields.
+- Fixtures are short, synthetic, and license-clean.
+
+### Test points
+
+- Deterministic C major digest
+- Duration-weighted C recall after file write
+- 120 BPM Acc1 after file decode
+- Checksum mismatch
+- Malformed report rejection
+- Silence must not pass as C major
+
+### Realistic threats
+
+A tampered fixture or a missing digest could be scored as a pass and then
+cited as release evidence.
+
+### Remaining risk
+
+Tier 1 does not cover Demucs stems, private commercial recordings, or
+CPU/GPU numeric parity. Those remain later #770 tiers.

services/analysis-engine/src/bandscope_analysis/accuracy/__init__.py

@@ -0,0 +1,42 @@
+"""Real-audio accuracy acceptance helpers for BandScope rehearsal analysis.
+
+Next action: run the Tier 1 fixture tests before claiming a harmony or tempo
+result is accurate. These helpers score decoded PCM against known labels. They
+do not replace Demucs stem separation or private-corpus benchmarks.
+"""
+
+from .evaluate import evaluate_c_major_pcm, evaluate_click_tempo_file
+from .fixtures import (
+    C_MAJOR_LABEL,
+    DEFAULT_CLICK_BPM,
+    DEFAULT_SAMPLE_RATE,
+    assert_fixture_checksum,
+    render_c_major_triad,
+    render_click_track,
+    write_pcm_wav,
+)
+from .manifest import (
+    AccuracyCaseReport,
+    build_case_report,
+    parse_case_report,
+    read_product_version,
+)
+from .metrics import duration_weighted_chord_recall, tempo_acc1
+
+__all__ = [
+    "AccuracyCaseReport",
+    "C_MAJOR_LABEL",
+    "DEFAULT_CLICK_BPM",
+    "DEFAULT_SAMPLE_RATE",
+    "assert_fixture_checksum",
+    "build_case_report",
+    "duration_weighted_chord_recall",
+    "evaluate_c_major_pcm",
+    "evaluate_click_tempo_file",
+    "parse_case_report",
+    "read_product_version",
+    "render_c_major_triad",
+    "render_click_track",
+    "tempo_acc1",
+    "write_pcm_wav",
+]

services/analysis-engine/src/bandscope_analysis/accuracy/evaluate.py

@@ -0,0 +1,79 @@
+"""Score decoded PCM fixtures through production analysis helpers."""
+
+from __future__ import annotations
+
+from pathlib import Path
+
+import numpy as np
+from numpy.typing import NDArray
+
+from bandscope_analysis.accuracy.fixtures import (
+    C_MAJOR_LABEL,
+    DEFAULT_CLICK_BPM,
+    DEFAULT_SAMPLE_RATE,
+    assert_fixture_checksum,
+)
+from bandscope_analysis.accuracy.manifest import AccuracyCaseReport, build_case_report
+from bandscope_analysis.accuracy.metrics import duration_weighted_chord_recall, tempo_acc1
+from bandscope_analysis.chords.chord_recognizer import ChordRecognizer
+from bandscope_analysis.temporal.analyzer import TemporalAnalyzer
+
+C_MAJOR_RECALL_FLOOR = 0.70
+
+
+def evaluate_c_major_pcm(
+    audio: NDArray[np.floating],
+    sample_rate: int = DEFAULT_SAMPLE_RATE,
+    audio_sha256: str = "0" * 64,
+) -> AccuracyCaseReport:
+    """Score a C major triad through the production chord recognizer.
+
+    Args:
+        audio: Decoded mono PCM. Do not pass a chroma matrix.
+        sample_rate: Sample rate of ``audio``.
+        audio_sha256: Digest of the on-disk fixture that produced ``audio``.
+
+    Returns:
+        A case report whose metric is duration-weighted recall of ``C``.
+    """
+    recognizer = ChordRecognizer()
+    tracked = recognizer.recognize(np.asarray(audio, dtype=np.float32), sr=sample_rate)
+    segments = [(item["start_time"], item["end_time"], item["chord"]) for item in tracked]
+    duration = float(len(audio) / sample_rate)
+    recall = duration_weighted_chord_recall(segments, C_MAJOR_LABEL, 0.0, duration)
+    return build_case_report(
+        case_id="c-major-triad",
+        audio_sha256=audio_sha256,
+        metric_name="duration_weighted_chord_recall",
+        metric_value=recall,
+        passed=recall >= C_MAJOR_RECALL_FLOOR,
+        true_label=C_MAJOR_LABEL,
+    )
+
+
+def evaluate_click_tempo_file(
+    audio_path: Path,
+    expected_sha256: str,
+    true_bpm: float = DEFAULT_CLICK_BPM,
+) -> AccuracyCaseReport:
+    """Decode a click-track WAV with TemporalAnalyzer and score Acc1.
+
+    Args:
+        audio_path: On-disk WAV written by ``write_pcm_wav``.
+        expected_sha256: Registered digest. Mismatch fails closed.
+        true_bpm: Known click tempo.
+
+    Returns:
+        A case report whose metric is 1.0 on Acc1 pass and 0.0 on Acc1 fail.
+    """
+    assert_fixture_checksum(audio_path, expected_sha256)
+    features = TemporalAnalyzer().analyze(audio_path)
+    passed = tempo_acc1(features["bpm"], true_bpm)
+    return build_case_report(
+        case_id="click-120-bpm",
+        audio_sha256=expected_sha256,
+        metric_name="tempo_acc1",
+        metric_value=1.0 if passed else 0.0,
+        passed=passed,
+        true_label=f"{true_bpm:g} bpm",
+    )