diff --git a/AGENTS.md b/AGENTS.md index b9a67ce17..86faa81c1 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -78,6 +78,8 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working - `docs/security/github-required-checks.md` - `docs/plans/2026-03-10-bandscope-harness-design.md` - `docs/plans/2026-03-10-bandscope-harness.md` +- `docs/doctoring/section-harmony-known-progression.md` +- `docs/plans/2026-08-16-section-harmony-known-progression.md` ## Code style - Keep UI and analysis engine decoupled through shared contracts. @@ -85,6 +87,7 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working - Prefer practical, friendly, rehearsal-first wording over academic or authority-heavy language. - Do not reduce the product to a chord analyzer when form, timing, player coordination, playable ranges, simplification, and setup cues are the real rehearsal blockers. - Do not frame usability as a reason to accept weak analysis quality; BandScope should aim for both easy use and high accuracy. +- Do not replace the known-progression section-harmony lock with a mocked recognizer. A verse/chorus take must keep distinct section answers. ## Safety - Do not add network-dependent runtime paths for local analysis. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index ca0df5ac4..05214b2ec 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,6 +1,6 @@ # ARCHITECTURE.md -Last updated: 2026-03-11 +Last updated: 2026-08-16 ## Brand source @@ -115,6 +115,7 @@ Last updated: 2026-03-11 - Local audio intake bootstraps a project by validating a user-selected file in Rust, creating app-owned temp/cache/project roots, and referencing the original source file rather than copying it in this phase. - Those bootstrap roots should resolve from app-owned Tauri data/cache paths instead of the shared system temp namespace. - Product and UX decisions should prefer rehearsal-first simplicity while still maintaining high analytical accuracy. +- Section harmony must stay a `song -> section` answer. The known-progression lock in `docs/doctoring/section-harmony-known-progression.md` requires the recognizer plus section summary to recover C then G (and the reverse order) on annotated two-section windows with duration-weighted recall at or above 0.70. Minor labels must not satisfy a major window. - Security decisions should prefer allowlisted narrow capabilities over generic convenience APIs. ## Verification model diff --git a/CHANGELOG.md b/CHANGELOG.md index 0b6f7e784..d49c09727 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ ### Added +- Lock section-level chord recovery on annotated C-then-G (and G-then-C) windows so a song-wide chord answer cannot hide a section change. - Name tonight's first playable range on the ready rehearsal map and tell the player to check that span on their instrument before the section. - Display the analyzed song tempo (BPM) as a badge in the rehearsal workspace. - 각 합주 역할(Role)별 개인 연습 진행도를 0~100% 범위로 기록 및 시각화할 수 있는 연습 진척도(`practiceProgress`) 트래커 기능 추가. UI 컨트롤(슬라이더 및 +/- 버튼)과 한/영 다국어 지원 포함. @@ -14,6 +15,7 @@ ### Fixed +- Union overlapping matching recognizer intervals inside each annotated section before duration-weighted chord recall, preventing duplicate-time estimates from inflating known-progression accuracy evidence. - Upgraded the local score PDF parser to `pdfjs-dist` 6.2.108, pinned Undici 7.29.0 across the workspace, and constrained PDF loading to copied in-memory bytes with a same-origin bundled worker and npm-generated lock provenance. ## [0.1.3] - 2026-04-29 @@ -73,5 +75,5 @@ ### 추가됨 (Added) - `ChordsFeature` (코드 분석) 화면에서 각 파트(Role)의 `transpositionPlan`(이조/조옮김 계획)을 표시하는 기능을 추가했습니다. -- `RangesFeature` (음역대 분석) 화면에서 겹침 경고(Overlap warning) 외에 해당 파트의 채보(Transcription) 가능 노드 수를 요약하여 보여주는 기능을 추가했습니다. -- 신규 UI 요소에 대한 단위 테스트를 추가했습니다 (`apps/desktop/src/features/chords/index.test.tsx`, `apps/desktop/src/features/ranges/index.test.tsx`). \ No newline at end of file +- `RangesFeature` (음역대 분석) 화면에서 겹침 경고(Overlap warning) 외에 해당 파트(Role)의 채보(Transcription) 가능 노드 수를 요약하여 보여주는 기능을 추가했습니다. +- 신규 UI 요소에 대한 100% 테스트 커버리지를 보장하는 단위 테스트를 추가했습니다 (`apps/desktop/src/features/chords/index.test.tsx`, `apps/desktop/src/features/ranges/index.test.tsx`). diff --git a/CLAUDE.md b/CLAUDE.md index b5a34c1fa..193bd6e80 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -53,7 +53,7 @@ Three layers, decoupled through shared contracts: - `apps/desktop` — Tauri 2 + Vite + React 19 shell (Tailwind 4, Base UI, Storybook). Feature screens live in `src/features/` (home, workspace, chords, ranges, player, settings). The ready workspace names tonight's first playable range and the next instrument check. `src/lib/analysis.ts` and `src/lib/job_runner.ts` call typed Tauri IPC commands, with a browser fallback that serves demo data when not running inside Tauri. - `apps/desktop/src-tauri/src/main.rs` — the Rust orchestration boundary. Tauri commands (`start_analysis_job`, `get_analysis_job_status`, `select_local_audio_source`, `import_youtube_url`) validate untrusted input (project IDs, file paths, URLs) and spawn the Python engine as a subprocess. There is no loopback HTTP listener and no network path for local analysis. -- `services/analysis-engine` — Python package `bandscope_analysis` (librosa/numpy). Entry point `cli.py` reads a JSON job request on stdin and prints a structured job-status JSON envelope on stdout (`--progress-jsonl` streams progress lines). `api.py` orchestrates the pipeline across the `separation`, `sections`, `roles`, `chords`, `ranges`, `temporal`, `transcription`, and `youtube` modules. +- `services/analysis-engine` — Python package `bandscope_analysis` (librosa/numpy). Entry point `cli.py` reads a JSON job request on stdin and prints a structured job-status JSON envelope on stdout (`--progress-jsonl` streams progress lines). `api.py` orchestrates the pipeline across the `separation`, `sections`, `roles`, `chords`, `ranges`, `temporal`, `transcription`, and `youtube` modules. Section harmony is locked by a known-progression take in `docs/doctoring/section-harmony-known-progression.md`. Data flow: React UI → Tauri IPC command → Rust validation + Python subprocess over stdin/stdout → job status and progress events emitted back to the UI. diff --git a/docs/doctoring/section-harmony-known-progression.md b/docs/doctoring/section-harmony-known-progression.md new file mode 100644 index 000000000..dd4952092 --- /dev/null +++ b/docs/doctoring/section-harmony-known-progression.md @@ -0,0 +1,65 @@ +# Section harmony known-progression accuracy + +Next action: keep verse and chorus as separate harmony answers. Do not let a +song-wide chord track replace the `song -> section -> role` contract. Run +`uv run --project services/analysis-engine pytest tests/test_section_harmony_audio_accuracy.py` +before claiming a recognizer or section-summary change is rehearsal-ready. + +## Why this lock exists + +Players copy from the recording. If the engine hears a C-major verse and a +G-major chorus and then prints one song-level chord, the room wastes the first +pass arguing about the form. Duration-weighted chord symbol recall against +annotated windows is the evaluation unit used in automatic chord estimation +reviews (Harte et al., 2005; McVicar et al., 2014). Fujishima (1999) established +chroma-template matching as the baseline this engine still uses. + +The lock synthesizes a dry two-section take (C major, then G major, and the +reverse order) so CI does not depend on a copyrighted master. The true +parameters are the section windows and triad roots. The estimate must recover +each section's main chord and keep weighted recall at or above 0.70. + +For evidence integrity, matching recognizer intervals are clipped to each true +section window and unioned before matched duration is accumulated. Duplicate +or overlapping estimates therefore cannot count the same annotated time twice +or inflate recall above the actual fraction of section time recovered. + +## Held values + +- Sample rate: 22050 Hz +- Section length: 4.0 s +- Minimum duration-weighted recall: 0.70 +- Canonical roots: `C` then `G`, and `G` then `C` +- Minor labels (`Cm`, `Gm`) must not satisfy a major window +- Overlapping matching estimates count only the union of covered section time + +## References + +Fujishima, T. (1999). Realtime chord recognition of musical sound: A system +using Common Lisp Music. In *Proceedings of the International Computer Music +Conference* (pp. 464–467). International Computer Music Association. + +Harte, C., Sandler, M., Abdallah, S., & Gómez, E. (2005). Symbolic +representation of musical chords: A proposed syntax for text annotations. In +*Proceedings of the 6th International Conference on Music Information Retrieval* +(pp. 66–71). + +McVicar, M., Santos-Rodríguez, R., Ni, Y., & De Bie, T. (2014). Automatic chord +estimation from audio: A review of the state of the art. *IEEE/ACM Transactions +on Audio, Speech, and Language Processing, 22*(2), 556–575. +https://doi.org/10.1109/TASLP.2013.2294580 + +## Security Notes + +- Attack surface: in-memory float audio arrays and recognizer segment timings + only. The lock does not read user files, URLs, or subprocess output. +- Trust boundary: synthetic fixtures and annotated section windows are trusted + test input. Production recognizer output remains untrusted evidence until it + is scored against those annotations. +- Mitigations: no file I/O, no network, no shell; major/minor labels remain + distinct; matching intervals are unioned before duration accumulation so + duplicate-time evidence cannot create a false high score. Failures stay + inside pytest. +- Test points: `test_duration_weighted_symbol_recall_unions_duplicate_time`, + `test_section_harmony_recovers_verse_c_then_chorus_g`, and + `test_section_harmony_keeps_later_c_off_the_opening_window`. diff --git a/docs/plans/2026-08-16-section-harmony-known-progression.md b/docs/plans/2026-08-16-section-harmony-known-progression.md new file mode 100644 index 000000000..a5d24e982 --- /dev/null +++ b/docs/plans/2026-08-16-section-harmony-known-progression.md @@ -0,0 +1,41 @@ +# Section harmony known-progression accuracy lock + +**Goal:** Prove the analysis engine recovers distinct section harmony from a +rehearsal-shaped take, not a single song-wide chord. + +**Architecture:** `ChordRecognizer` emits time-stamped segments. +`summarize_section_harmony` overlaps those segments onto section windows. The +new tests synthesize C-major then G-major audio (and the reverse) and score +duration-weighted chord symbol recall against those true windows. + +## Security Notes + +### Attack surface + +- Synthetic numpy audio arrays constructed in-process for pytest. + +### Trust boundary + +- Untrusted: production audio from user files and YouTube import. +- Trusted: fixture frequencies, section bounds, and recall threshold in the + test module and `docs/doctoring/section-harmony-known-progression.md`. + +### Mitigations + +- No file reads, URL intake, subprocesses, or IPC. +- Threshold is a regression floor, not a claim that live masters are solved. + +### Test points + +- Verse C / chorus G main chords and ≥0.70 weighted recall. +- Reverse order keeps G on the opening window. + +### Realistic threats + +- A mocked recognizer can keep CI green while live section answers stay wrong. +- A song-wide majority chord can hide a chorus change and send the wrong lock-in cue. + +### Remaining risk + +- Dry triads are easier than mixed stems. Live-master recall stays a later + gold-corpus lane and must not be faked with mocked recognizer output. diff --git a/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py new file mode 100644 index 000000000..7b431bee0 --- /dev/null +++ b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py @@ -0,0 +1,183 @@ +"""Known-progression accuracy lock for section-level chord recovery. + +Rehearsal buyers trust BandScope only when a real take produces the harmony +they will play. Single-chord unit tests are not enough: a verse-then-chorus +song must keep distinct section answers. This module synthesizes a two-section +rehearsal take (C major, then G major) and measures duration-weighted chord +symbol recall against those true labels (Harte et al., 2005; McVicar et al., +2014). +""" + +from __future__ import annotations + +import numpy as np + +from bandscope_analysis.chords.chord_recognizer import ChordRecognizer, TrackedChord +from bandscope_analysis.chords.section_harmony import summarize_section_harmony + +_SAMPLE_RATE = 22050 +_SECTION_SECONDS = 4.0 +_MINIMUM_WEIGHTED_RECALL = 0.70 +_C4_HZ = 261.63 +_G4_HZ = 392.00 + + +def _canonical_major_symbol(label: str) -> str: + """Keep major-triad spellings; do not treat minor as a match. + + Args: + label: Raw recognizer or annotation chord string. + + Returns: + ``C`` or ``G`` for those major labels (including a ``:maj`` suffix), + otherwise the stripped original so ``Cm`` cannot satisfy a C window. + """ + stripped = label.strip() + if stripped.endswith(":maj"): + return stripped[: -len(":maj")] + return stripped + + +def _major_triad_take(root_hz: float, duration_seconds: float, sample_rate: int) -> np.ndarray: + """Render a dry major triad as equal-amplitude sines. + + Args: + root_hz: Root frequency in hertz. + duration_seconds: Take length in seconds. + sample_rate: Samples per second. + + Returns: + Mono float32 audio in ``[-1, 1]``. + """ + sample_count = int(sample_rate * duration_seconds) + time_axis = np.linspace(0.0, duration_seconds, sample_count, endpoint=False) + major_third_hz = root_hz * (2.0 ** (4.0 / 12.0)) + perfect_fifth_hz = root_hz * (2.0 ** (7.0 / 12.0)) + waveform = ( + np.sin(2.0 * np.pi * root_hz * time_axis) + + np.sin(2.0 * np.pi * major_third_hz * time_axis) + + np.sin(2.0 * np.pi * perfect_fifth_hz * time_axis) + ) / 3.0 + return waveform.astype(np.float32) + + +def _duration_weighted_symbol_recall( + segments: list[TrackedChord], + truth_windows: list[tuple[float, float, str]], +) -> float: + """Score recovered segments against true section windows. + + The denominator is annotated duration so a silent or smeared boundary + cannot inflate the score (McVicar et al., 2014). Matching recognizer + intervals are unioned inside each annotation window so duplicate or + overlapping estimates cannot count the same section time twice. + + Args: + segments: Time-stamped recognizer output. + truth_windows: ``(start, end, canonical_chord)`` annotations. + + Returns: + Overlap-weighted recall in ``[0, 1]``, or ``0.0`` when the annotation + duration is not positive. + """ + annotated_duration = sum(end - start for start, end, _chord in truth_windows) + if annotated_duration <= 0.0: + return 0.0 + + matched_duration = 0.0 + for truth_start, truth_end, truth_chord in truth_windows: + matching_intervals: list[tuple[float, float]] = [] + for segment in segments: + if _canonical_major_symbol(segment["chord"]) != truth_chord: + continue + overlap_start = max(segment["start_time"], truth_start) + overlap_end = min(segment["end_time"], truth_end) + if overlap_end > overlap_start: + matching_intervals.append((overlap_start, overlap_end)) + + matching_intervals.sort(key=lambda interval: (interval[0], interval[1])) + if not matching_intervals: + continue + current_start, current_end = matching_intervals[0] + for interval_start, interval_end in matching_intervals[1:]: + if interval_start <= current_end: + current_end = max(current_end, interval_end) + continue + matched_duration += current_end - current_start + current_start, current_end = interval_start, interval_end + matched_duration += current_end - current_start + + return matched_duration / annotated_duration + + +def test_duration_weighted_symbol_recall_unions_duplicate_time() -> None: + """Overlapping matching estimates must not count section time twice.""" + segments: list[TrackedChord] = [ + {"start_time": 0.0, "end_time": 3.0, "chord": "C", "confidence": "high"}, + {"start_time": 1.0, "end_time": 4.0, "chord": "C:maj", "confidence": "high"}, + {"start_time": 4.0, "end_time": 8.0, "chord": "G", "confidence": "high"}, + ] + truth_windows = [(0.0, 4.0, "C"), (4.0, 8.0, "G")] + + assert _duration_weighted_symbol_recall(segments, truth_windows) == 1.0 + + +def test_canonical_major_symbol_rejects_minor_as_major() -> None: + """Keep ``Cm`` distinct from ``C`` so a minor estimate cannot pass.""" + assert _canonical_major_symbol("C") == "C" + assert _canonical_major_symbol("C:maj") == "C" + assert _canonical_major_symbol("Cm") == "Cm" + assert _canonical_major_symbol("G:maj") == "G" + assert _canonical_major_symbol("Gm") == "Gm" + + +def test_section_harmony_recovers_verse_c_then_chorus_g() -> None: + """Recover C then G as section main chords from a two-section take. + + The take is a rehearsal-shaped verse/chorus pair: four seconds of C major + followed by four seconds of G major. The recognizer runs on the mixed + audio. Section summaries must keep those answers apart, and + duration-weighted recall against the true windows must stay at or above + 70 percent so a boundary smear cannot hide a wrong-song result. + """ + verse = _major_triad_take(_C4_HZ, _SECTION_SECONDS, _SAMPLE_RATE) + chorus = _major_triad_take(_G4_HZ, _SECTION_SECONDS, _SAMPLE_RATE) + take = np.concatenate([verse, chorus]) + boundaries = [(0.0, _SECTION_SECONDS), (_SECTION_SECONDS, _SECTION_SECONDS * 2.0)] + truth_windows = [ + (0.0, _SECTION_SECONDS, "C"), + (_SECTION_SECONDS, _SECTION_SECONDS * 2.0, "G"), + ] + + segments = ChordRecognizer().recognize(take, sr=_SAMPLE_RATE) + summaries = summarize_section_harmony(segments, boundaries) + + assert len(summaries) == 2 + assert _canonical_major_symbol(summaries[0]["main_chord"]) == "C" + assert _canonical_major_symbol(summaries[1]["main_chord"]) == "G" + assert _duration_weighted_symbol_recall(segments, truth_windows) >= _MINIMUM_WEIGHTED_RECALL + + +def test_section_harmony_keeps_later_c_off_the_opening_window() -> None: + """Refuse a song-wide C answer when G occupies the first section. + + The same two triads in reverse order must not collapse to one global + chord. Players need the opening chorus to stay G even though C arrives + later. This is the temporal complement of the verse-then-chorus lock. + """ + chorus = _major_triad_take(_G4_HZ, _SECTION_SECONDS, _SAMPLE_RATE) + verse = _major_triad_take(_C4_HZ, _SECTION_SECONDS, _SAMPLE_RATE) + take = np.concatenate([chorus, verse]) + boundaries = [(0.0, _SECTION_SECONDS), (_SECTION_SECONDS, _SECTION_SECONDS * 2.0)] + truth_windows = [ + (0.0, _SECTION_SECONDS, "G"), + (_SECTION_SECONDS, _SECTION_SECONDS * 2.0, "C"), + ] + + segments = ChordRecognizer().recognize(take, sr=_SAMPLE_RATE) + summaries = summarize_section_harmony(segments, boundaries) + + assert len(summaries) == 2 + assert _canonical_major_symbol(summaries[0]["main_chord"]) == "G" + assert _canonical_major_symbol(summaries[1]["main_chord"]) == "C" + assert _duration_weighted_symbol_recall(segments, truth_windows) >= _MINIMUM_WEIGHTED_RECALL