From 4e5417cd1356162f19231540bf834c5f3573865b Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Sun, 16 Aug 2026 16:48:06 +0000 Subject: [PATCH 1/6] test(analysis): lock verse/chorus chord recovery on a known take Keep section harmony honest: a C-then-G rehearsal take, and the reverse order, must recover each section's main chord with duration-weighted recall instead of collapsing to one song-wide answer. Co-authored-by: Seongho Bae --- AGENTS.md | 3 + ARCHITECTURE.md | 1 + CHANGELOG.md | 1 + CLAUDE.md | 2 +- .../section-harmony-known-progression.md | 53 +++++++ ...08-16-section-harmony-known-progression.md | 41 +++++ .../test_section_harmony_audio_accuracy.py | 150 ++++++++++++++++++ 7 files changed, 250 insertions(+), 1 deletion(-) create mode 100644 docs/doctoring/section-harmony-known-progression.md create mode 100644 docs/plans/2026-08-16-section-harmony-known-progression.md create mode 100644 services/analysis-engine/tests/test_section_harmony_audio_accuracy.py diff --git a/AGENTS.md b/AGENTS.md index fca448ce9..a91a59710 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -78,6 +78,8 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working - `docs/security/github-required-checks.md` - `docs/plans/2026-03-10-bandscope-harness-design.md` - `docs/plans/2026-03-10-bandscope-harness.md` +- `docs/doctoring/section-harmony-known-progression.md` +- `docs/plans/2026-08-16-section-harmony-known-progression.md` ## Code style - Keep UI and analysis engine decoupled through shared contracts. @@ -85,6 +87,7 @@ This section applies to any agent (Claude, Codex, Cursor, opencode, ...) working - Prefer practical, friendly, rehearsal-first wording over academic or authority-heavy language. - Do not reduce the product to a chord analyzer when form, timing, player coordination, simplification, and setup cues are the real rehearsal blockers. - Do not frame usability as a reason to accept weak analysis quality; BandScope should aim for both easy use and high accuracy. +- Do not replace the known-progression section-harmony lock with a mocked recognizer. A verse/chorus take must keep distinct section answers. ## Safety - Do not add network-dependent runtime paths for local analysis. diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 3302a6fc3..f05d8f312 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -115,6 +115,7 @@ Last updated: 2026-03-11 - Local audio intake bootstraps a project by validating a user-selected file in Rust, creating app-owned temp/cache/project roots, and referencing the original source file rather than copying it in this phase. - Those bootstrap roots should resolve from app-owned Tauri data/cache paths instead of the shared system temp namespace. - Product and UX decisions should prefer rehearsal-first simplicity while still maintaining high analytical accuracy. +- Section harmony must stay a `song -> section` answer. The known-progression lock in `docs/doctoring/section-harmony-known-progression.md` requires the recognizer plus section summary to recover C then G (and the reverse order) from a two-section take with duration-weighted recall at or above 0.70. - Security decisions should prefer allowlisted narrow capabilities over generic convenience APIs. ## Verification model diff --git a/CHANGELOG.md b/CHANGELOG.md index eea696893..7cd551ea2 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,6 +4,7 @@ ### Added +- Lock section-level chord recovery on a two-section C-then-G (and G-then-C) rehearsal take so a song-wide chord answer cannot hide a wrong form. - Display the analyzed song tempo (BPM) as a badge in the rehearsal workspace. - 각 합주 역할(Role)별 개인 연습 진행도를 0~100% 범위로 기록 및 시각화할 수 있는 연습 진척도(`practiceProgress`) 트래커 기능 추가. UI 컨트롤(슬라이더 및 +/- 버튼)과 한/영 다국어 지원 포함. diff --git a/CLAUDE.md b/CLAUDE.md index 82c2c704a..b1c75948c 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -53,7 +53,7 @@ Three layers, decoupled through shared contracts: - `apps/desktop` — Tauri 2 + Vite + React 19 shell (Tailwind 4, Base UI, Storybook). Feature screens live in `src/features/` (home, workspace, chords, ranges, player, settings). `src/lib/analysis.ts` and `src/lib/job_runner.ts` call typed Tauri IPC commands, with a browser fallback that serves demo data when not running inside Tauri. - `apps/desktop/src-tauri/src/main.rs` — the Rust orchestration boundary. Tauri commands (`start_analysis_job`, `get_analysis_job_status`, `select_local_audio_source`, `import_youtube_url`) validate untrusted input (project IDs, file paths, URLs) and spawn the Python engine as a subprocess. There is no loopback HTTP listener and no network path for local analysis. -- `services/analysis-engine` — Python package `bandscope_analysis` (librosa/numpy). Entry point `cli.py` reads a JSON job request on stdin and prints a structured job-status JSON envelope on stdout (`--progress-jsonl` streams progress lines). `api.py` orchestrates the pipeline across the `separation`, `sections`, `roles`, `chords`, `ranges`, `temporal`, `transcription`, and `youtube` modules. +- `services/analysis-engine` — Python package `bandscope_analysis` (librosa/numpy). Entry point `cli.py` reads a JSON job request on stdin and prints a structured job-status JSON envelope on stdout (`--progress-jsonl` streams progress lines). `api.py` orchestrates the pipeline across the `separation`, `sections`, `roles`, `chords`, `ranges`, `temporal`, `transcription`, and `youtube` modules. Section harmony is locked by a known-progression take in `docs/doctoring/section-harmony-known-progression.md`. Data flow: React UI → Tauri IPC command → Rust validation + Python subprocess over stdin/stdout → job status and progress events emitted back to the UI. diff --git a/docs/doctoring/section-harmony-known-progression.md b/docs/doctoring/section-harmony-known-progression.md new file mode 100644 index 000000000..007eda335 --- /dev/null +++ b/docs/doctoring/section-harmony-known-progression.md @@ -0,0 +1,53 @@ +# Section harmony known-progression accuracy + +Next action: keep verse and chorus as separate harmony answers. Do not let a +song-wide chord track replace the `song -> section -> role` contract. Run +`uv run --project services/analysis-engine pytest tests/test_section_harmony_audio_accuracy.py` +before claiming a recognizer or section-summary change is rehearsal-ready. + +## Why this lock exists + +Players copy from the recording. If the engine hears a C-major verse and a +G-major chorus and then prints one song-level chord, the room wastes the first +pass arguing about the form. Duration-weighted chord symbol recall against +annotated windows is the evaluation unit used in automatic chord estimation +reviews (Harte et al., 2005; McVicar et al., 2014). Fujishima (1999) established +chroma-template matching as the baseline this engine still uses. + +The lock synthesizes a dry two-section take (C major, then G major, and the +reverse order) so CI does not depend on a copyrighted master. The true +parameters are the section windows and triad roots. The estimate must recover +each section's main chord and keep weighted recall at or above 0.70. + +## Held values + +- Sample rate: 22050 Hz +- Section length: 4.0 s +- Minimum duration-weighted recall: 0.70 +- Canonical roots: `C` then `G`, and `G` then `C` + +## References + +Fujishima, T. (1999). Realtime chord recognition of musical sound: A system +using Common Lisp Music. In *Proceedings of the International Computer Music +Conference* (pp. 464–467). International Computer Music Association. + +Harte, C., Sandler, M., Abdallah, S., & Gómez, E. (2005). Symbolic +representation of musical chords: A proposed syntax for text annotations. In +*Proceedings of the 6th International Conference on Music Information Retrieval* +(pp. 66–71). + +McVicar, M., Santos-Rodríguez, R., Ni, Y., & De Bie, T. (2014). Automatic chord +estimation from audio: A review of the state of the art. *IEEE/ACM Transactions +on Audio, Speech, and Language Processing, 22*(2), 556–575. +https://doi.org/10.1109/TASLP.2013.2294580 + +## Security Notes + +- Attack surface: in-memory float audio arrays only. The lock does not read + user files, URLs, or subprocess output. +- Trust boundary: synthetic fixtures are trusted test input. Production + recognizer input remains untrusted decoded audio from stem separation. +- Mitigations: no file I/O, no network, no shell. Failures stay inside pytest. +- Test points: `test_section_harmony_recovers_verse_c_then_chorus_g` and + `test_section_harmony_keeps_later_c_off_the_opening_window`. diff --git a/docs/plans/2026-08-16-section-harmony-known-progression.md b/docs/plans/2026-08-16-section-harmony-known-progression.md new file mode 100644 index 000000000..a5d24e982 --- /dev/null +++ b/docs/plans/2026-08-16-section-harmony-known-progression.md @@ -0,0 +1,41 @@ +# Section harmony known-progression accuracy lock + +**Goal:** Prove the analysis engine recovers distinct section harmony from a +rehearsal-shaped take, not a single song-wide chord. + +**Architecture:** `ChordRecognizer` emits time-stamped segments. +`summarize_section_harmony` overlaps those segments onto section windows. The +new tests synthesize C-major then G-major audio (and the reverse) and score +duration-weighted chord symbol recall against those true windows. + +## Security Notes + +### Attack surface + +- Synthetic numpy audio arrays constructed in-process for pytest. + +### Trust boundary + +- Untrusted: production audio from user files and YouTube import. +- Trusted: fixture frequencies, section bounds, and recall threshold in the + test module and `docs/doctoring/section-harmony-known-progression.md`. + +### Mitigations + +- No file reads, URL intake, subprocesses, or IPC. +- Threshold is a regression floor, not a claim that live masters are solved. + +### Test points + +- Verse C / chorus G main chords and ≥0.70 weighted recall. +- Reverse order keeps G on the opening window. + +### Realistic threats + +- A mocked recognizer can keep CI green while live section answers stay wrong. +- A song-wide majority chord can hide a chorus change and send the wrong lock-in cue. + +### Remaining risk + +- Dry triads are easier than mixed stems. Live-master recall stays a later + gold-corpus lane and must not be faked with mocked recognizer output. diff --git a/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py new file mode 100644 index 000000000..144dd520a --- /dev/null +++ b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py @@ -0,0 +1,150 @@ +"""Known-progression accuracy lock for section-level chord recovery. + +Rehearsal buyers trust BandScope only when a real take produces the harmony +they will play. Single-chord unit tests are not enough: a verse-then-chorus +song must keep distinct section answers. This module synthesizes a two-section +rehearsal take (C major, then G major) and measures duration-weighted chord +symbol recall against those true labels (Harte et al., 2005; McVicar et al., +2014). +""" + +from __future__ import annotations + +import numpy as np + +from bandscope_analysis.chords.chord_recognizer import ChordRecognizer, TrackedChord +from bandscope_analysis.chords.section_harmony import summarize_section_harmony + +_SAMPLE_RATE = 22050 +_SECTION_SECONDS = 4.0 +_MINIMUM_WEIGHTED_RECALL = 0.70 +_C4_HZ = 261.63 +_G4_HZ = 392.00 + + +def _canonical_chord_symbol(label: str) -> str: + """Collapse recognizer spellings onto a major-triad root. + + Args: + label: Raw recognizer or annotation chord string. + + Returns: + A root letter such as ``C`` or ``G``, or the original label when the + token is empty or a no-chord sentinel. + """ + stripped = label.strip() + if not stripped or stripped in {"N", "Unknown"}: + return stripped + head = stripped.split(":", maxsplit=1)[0] + if head.endswith("maj"): + return head[: -len("maj")] + if head.endswith("m") and not head.endswith("dim"): + return head[:-1] + return head + + +def _major_triad_take(root_hz: float, duration_seconds: float, sample_rate: int) -> np.ndarray: + """Render a dry major triad as equal-amplitude sines. + + Args: + root_hz: Root frequency in hertz. + duration_seconds: Take length in seconds. + sample_rate: Samples per second. + + Returns: + Mono float32 audio in ``[-1, 1]``. + """ + sample_count = int(sample_rate * duration_seconds) + time_axis = np.linspace(0.0, duration_seconds, sample_count, endpoint=False) + major_third_hz = root_hz * (2.0 ** (4.0 / 12.0)) + perfect_fifth_hz = root_hz * (2.0 ** (7.0 / 12.0)) + waveform = ( + np.sin(2.0 * np.pi * root_hz * time_axis) + + np.sin(2.0 * np.pi * major_third_hz * time_axis) + + np.sin(2.0 * np.pi * perfect_fifth_hz * time_axis) + ) / 3.0 + return waveform.astype(np.float32) + + +def _duration_weighted_symbol_recall( + segments: list[TrackedChord], + truth_windows: list[tuple[float, float, str]], +) -> float: + """Score recovered segments against true section windows. + + The denominator is annotated duration so a silent or smeared boundary + cannot inflate the score (McVicar et al., 2014). + + Args: + segments: Time-stamped recognizer output. + truth_windows: ``(start, end, canonical_chord)`` annotations. + + Returns: + Overlap-weighted recall in ``[0, 1]``, or ``0.0`` when the annotation + duration is not positive. + """ + annotated_duration = sum(end - start for start, end, _chord in truth_windows) + if annotated_duration <= 0.0: + return 0.0 + + matched_duration = 0.0 + for truth_start, truth_end, truth_chord in truth_windows: + for segment in segments: + overlap = min(segment["end_time"], truth_end) - max(segment["start_time"], truth_start) + if overlap <= 0.0: + continue + if _canonical_chord_symbol(segment["chord"]) == truth_chord: + matched_duration += overlap + return matched_duration / annotated_duration + + +def test_section_harmony_recovers_verse_c_then_chorus_g() -> None: + """Recover C then G as section main chords from a two-section take. + + The take is a rehearsal-shaped verse/chorus pair: four seconds of C major + followed by four seconds of G major. The recognizer runs on the mixed + audio. Section summaries must keep those answers apart, and + duration-weighted recall against the true windows must stay at or above + 70 percent so a boundary smear cannot hide a wrong-song result. + """ + verse = _major_triad_take(_C4_HZ, _SECTION_SECONDS, _SAMPLE_RATE) + chorus = _major_triad_take(_G4_HZ, _SECTION_SECONDS, _SAMPLE_RATE) + take = np.concatenate([verse, chorus]) + boundaries = [(0.0, _SECTION_SECONDS), (_SECTION_SECONDS, _SECTION_SECONDS * 2.0)] + truth_windows = [ + (0.0, _SECTION_SECONDS, "C"), + (_SECTION_SECONDS, _SECTION_SECONDS * 2.0, "G"), + ] + + segments = ChordRecognizer().recognize(take, sr=_SAMPLE_RATE) + summaries = summarize_section_harmony(segments, boundaries) + + assert len(summaries) == 2 + assert _canonical_chord_symbol(summaries[0]["main_chord"]) == "C" + assert _canonical_chord_symbol(summaries[1]["main_chord"]) == "G" + assert _duration_weighted_symbol_recall(segments, truth_windows) >= _MINIMUM_WEIGHTED_RECALL + + +def test_section_harmony_keeps_later_c_off_the_opening_window() -> None: + """Refuse a song-wide C answer when G occupies the first section. + + The same two triads in reverse order must not collapse to one global + chord. Players need the opening chorus to stay G even though C arrives + later. This is the temporal complement of the verse-then-chorus lock. + """ + chorus = _major_triad_take(_G4_HZ, _SECTION_SECONDS, _SAMPLE_RATE) + verse = _major_triad_take(_C4_HZ, _SECTION_SECONDS, _SAMPLE_RATE) + take = np.concatenate([chorus, verse]) + boundaries = [(0.0, _SECTION_SECONDS), (_SECTION_SECONDS, _SECTION_SECONDS * 2.0)] + truth_windows = [ + (0.0, _SECTION_SECONDS, "G"), + (_SECTION_SECONDS, _SECTION_SECONDS * 2.0, "C"), + ] + + segments = ChordRecognizer().recognize(take, sr=_SAMPLE_RATE) + summaries = summarize_section_harmony(segments, boundaries) + + assert len(summaries) == 2 + assert _canonical_chord_symbol(summaries[0]["main_chord"]) == "G" + assert _canonical_chord_symbol(summaries[1]["main_chord"]) == "C" + assert _duration_weighted_symbol_recall(segments, truth_windows) >= _MINIMUM_WEIGHTED_RECALL From 10f39657146d450cf09c8e5fd9b009cea83e4777 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Sun, 16 Aug 2026 16:55:58 +0000 Subject: [PATCH 2/6] test(analysis): reject minor labels in the known-progression lock Keep C/G major windows from passing on Cm/Gm. Document that the section bounds are annotated so the lock is not read as form detection. Co-authored-by: Seongho Bae --- ARCHITECTURE.md | 4 +- CHANGELOG.md | 2 +- .../section-harmony-known-progression.md | 1 + .../test_section_harmony_audio_accuracy.py | 38 ++++++++++--------- 4 files changed, 25 insertions(+), 20 deletions(-) diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index f05d8f312..8e7dd56c5 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -1,6 +1,6 @@ # ARCHITECTURE.md -Last updated: 2026-03-11 +Last updated: 2026-08-16 ## Brand source @@ -115,7 +115,7 @@ Last updated: 2026-03-11 - Local audio intake bootstraps a project by validating a user-selected file in Rust, creating app-owned temp/cache/project roots, and referencing the original source file rather than copying it in this phase. - Those bootstrap roots should resolve from app-owned Tauri data/cache paths instead of the shared system temp namespace. - Product and UX decisions should prefer rehearsal-first simplicity while still maintaining high analytical accuracy. -- Section harmony must stay a `song -> section` answer. The known-progression lock in `docs/doctoring/section-harmony-known-progression.md` requires the recognizer plus section summary to recover C then G (and the reverse order) from a two-section take with duration-weighted recall at or above 0.70. +- Section harmony must stay a `song -> section` answer. The known-progression lock in `docs/doctoring/section-harmony-known-progression.md` requires the recognizer plus section summary to recover C then G (and the reverse order) on annotated two-section windows with duration-weighted recall at or above 0.70. Minor labels must not satisfy a major window. - Security decisions should prefer allowlisted narrow capabilities over generic convenience APIs. ## Verification model diff --git a/CHANGELOG.md b/CHANGELOG.md index 7cd551ea2..d1766fc62 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -4,7 +4,7 @@ ### Added -- Lock section-level chord recovery on a two-section C-then-G (and G-then-C) rehearsal take so a song-wide chord answer cannot hide a wrong form. +- Lock section-level chord recovery on annotated C-then-G (and G-then-C) windows so a song-wide chord answer cannot hide a section change. - Display the analyzed song tempo (BPM) as a badge in the rehearsal workspace. - 각 합주 역할(Role)별 개인 연습 진행도를 0~100% 범위로 기록 및 시각화할 수 있는 연습 진척도(`practiceProgress`) 트래커 기능 추가. UI 컨트롤(슬라이더 및 +/- 버튼)과 한/영 다국어 지원 포함. diff --git a/docs/doctoring/section-harmony-known-progression.md b/docs/doctoring/section-harmony-known-progression.md index 007eda335..fe706896c 100644 --- a/docs/doctoring/section-harmony-known-progression.md +++ b/docs/doctoring/section-harmony-known-progression.md @@ -25,6 +25,7 @@ each section's main chord and keep weighted recall at or above 0.70. - Section length: 4.0 s - Minimum duration-weighted recall: 0.70 - Canonical roots: `C` then `G`, and `G` then `C` +- Minor labels (`Cm`, `Gm`) must not satisfy a major window ## References diff --git a/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py index 144dd520a..6334babe1 100644 --- a/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py +++ b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py @@ -22,25 +22,20 @@ _G4_HZ = 392.00 -def _canonical_chord_symbol(label: str) -> str: - """Collapse recognizer spellings onto a major-triad root. +def _canonical_major_symbol(label: str) -> str: + """Keep major-triad spellings; do not treat minor as a match. Args: label: Raw recognizer or annotation chord string. Returns: - A root letter such as ``C`` or ``G``, or the original label when the - token is empty or a no-chord sentinel. + ``C`` or ``G`` for those major labels (including a ``:maj`` suffix), + otherwise the stripped original so ``Cm`` cannot satisfy a C window. """ stripped = label.strip() - if not stripped or stripped in {"N", "Unknown"}: - return stripped - head = stripped.split(":", maxsplit=1)[0] - if head.endswith("maj"): - return head[: -len("maj")] - if head.endswith("m") and not head.endswith("dim"): - return head[:-1] - return head + if stripped.endswith(":maj"): + return stripped[: -len(":maj")] + return stripped def _major_triad_take(root_hz: float, duration_seconds: float, sample_rate: int) -> np.ndarray: @@ -93,11 +88,20 @@ def _duration_weighted_symbol_recall( overlap = min(segment["end_time"], truth_end) - max(segment["start_time"], truth_start) if overlap <= 0.0: continue - if _canonical_chord_symbol(segment["chord"]) == truth_chord: + if _canonical_major_symbol(segment["chord"]) == truth_chord: matched_duration += overlap return matched_duration / annotated_duration +def test_canonical_major_symbol_rejects_minor_as_major() -> None: + """Keep ``Cm`` distinct from ``C`` so a minor estimate cannot pass.""" + assert _canonical_major_symbol("C") == "C" + assert _canonical_major_symbol("C:maj") == "C" + assert _canonical_major_symbol("Cm") == "Cm" + assert _canonical_major_symbol("G:maj") == "G" + assert _canonical_major_symbol("Gm") == "Gm" + + def test_section_harmony_recovers_verse_c_then_chorus_g() -> None: """Recover C then G as section main chords from a two-section take. @@ -120,8 +124,8 @@ def test_section_harmony_recovers_verse_c_then_chorus_g() -> None: summaries = summarize_section_harmony(segments, boundaries) assert len(summaries) == 2 - assert _canonical_chord_symbol(summaries[0]["main_chord"]) == "C" - assert _canonical_chord_symbol(summaries[1]["main_chord"]) == "G" + assert _canonical_major_symbol(summaries[0]["main_chord"]) == "C" + assert _canonical_major_symbol(summaries[1]["main_chord"]) == "G" assert _duration_weighted_symbol_recall(segments, truth_windows) >= _MINIMUM_WEIGHTED_RECALL @@ -145,6 +149,6 @@ def test_section_harmony_keeps_later_c_off_the_opening_window() -> None: summaries = summarize_section_harmony(segments, boundaries) assert len(summaries) == 2 - assert _canonical_chord_symbol(summaries[0]["main_chord"]) == "G" - assert _canonical_chord_symbol(summaries[1]["main_chord"]) == "C" + assert _canonical_major_symbol(summaries[0]["main_chord"]) == "G" + assert _canonical_major_symbol(summaries[1]["main_chord"]) == "C" assert _duration_weighted_symbol_recall(segments, truth_windows) >= _MINIMUM_WEIGHTED_RECALL From e8461e4bbece72193bf6b4f2a548d616e79deb07 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 03:35:57 +0900 Subject: [PATCH 3/6] test(accuracy): prevent section recall double counting --- .../tests/test_section_harmony_audio_accuracy.py | 12 ++++++++++++ 1 file changed, 12 insertions(+) diff --git a/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py index 6334babe1..d079062fb 100644 --- a/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py +++ b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py @@ -93,6 +93,18 @@ def _duration_weighted_symbol_recall( return matched_duration / annotated_duration +def test_duration_weighted_symbol_recall_unions_duplicate_time() -> None: + """Overlapping matching estimates must not count section time twice.""" + segments: list[TrackedChord] = [ + {"start_time": 0.0, "end_time": 3.0, "chord": "C", "confidence": "high"}, + {"start_time": 1.0, "end_time": 4.0, "chord": "C:maj", "confidence": "high"}, + {"start_time": 4.0, "end_time": 8.0, "chord": "G", "confidence": "high"}, + ] + truth_windows = [(0.0, 4.0, "C"), (4.0, 8.0, "G")] + + assert _duration_weighted_symbol_recall(segments, truth_windows) == 1.0 + + def test_canonical_major_symbol_rejects_minor_as_major() -> None: """Keep ``Cm`` distinct from ``C`` so a minor estimate cannot pass.""" assert _canonical_major_symbol("C") == "C" From 5ffb4203b8d92282a7d2dde6bb2178ad4a0cc3cb Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 03:36:36 +0900 Subject: [PATCH 4/6] fix(accuracy): union section recall intervals --- .../test_section_harmony_audio_accuracy.py | 27 +++++++++++++++---- 1 file changed, 22 insertions(+), 5 deletions(-) diff --git a/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py index d079062fb..7b431bee0 100644 --- a/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py +++ b/services/analysis-engine/tests/test_section_harmony_audio_accuracy.py @@ -68,7 +68,9 @@ def _duration_weighted_symbol_recall( """Score recovered segments against true section windows. The denominator is annotated duration so a silent or smeared boundary - cannot inflate the score (McVicar et al., 2014). + cannot inflate the score (McVicar et al., 2014). Matching recognizer + intervals are unioned inside each annotation window so duplicate or + overlapping estimates cannot count the same section time twice. Args: segments: Time-stamped recognizer output. @@ -84,12 +86,27 @@ def _duration_weighted_symbol_recall( matched_duration = 0.0 for truth_start, truth_end, truth_chord in truth_windows: + matching_intervals: list[tuple[float, float]] = [] for segment in segments: - overlap = min(segment["end_time"], truth_end) - max(segment["start_time"], truth_start) - if overlap <= 0.0: + if _canonical_major_symbol(segment["chord"]) != truth_chord: continue - if _canonical_major_symbol(segment["chord"]) == truth_chord: - matched_duration += overlap + overlap_start = max(segment["start_time"], truth_start) + overlap_end = min(segment["end_time"], truth_end) + if overlap_end > overlap_start: + matching_intervals.append((overlap_start, overlap_end)) + + matching_intervals.sort(key=lambda interval: (interval[0], interval[1])) + if not matching_intervals: + continue + current_start, current_end = matching_intervals[0] + for interval_start, interval_end in matching_intervals[1:]: + if interval_start <= current_end: + current_end = max(current_end, interval_end) + continue + matched_duration += current_end - current_start + current_start, current_end = interval_start, interval_end + matched_duration += current_end - current_start + return matched_duration / annotated_duration From 041aa617b573c446f96075c27404dd9ee4a84a9a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 03:37:08 +0900 Subject: [PATCH 5/6] docs(accuracy): bound section recall evidence --- .../section-harmony-known-progression.md | 23 ++++++++++++++----- 1 file changed, 17 insertions(+), 6 deletions(-) diff --git a/docs/doctoring/section-harmony-known-progression.md b/docs/doctoring/section-harmony-known-progression.md index fe706896c..dd4952092 100644 --- a/docs/doctoring/section-harmony-known-progression.md +++ b/docs/doctoring/section-harmony-known-progression.md @@ -19,6 +19,11 @@ reverse order) so CI does not depend on a copyrighted master. The true parameters are the section windows and triad roots. The estimate must recover each section's main chord and keep weighted recall at or above 0.70. +For evidence integrity, matching recognizer intervals are clipped to each true +section window and unioned before matched duration is accumulated. Duplicate +or overlapping estimates therefore cannot count the same annotated time twice +or inflate recall above the actual fraction of section time recovered. + ## Held values - Sample rate: 22050 Hz @@ -26,6 +31,7 @@ each section's main chord and keep weighted recall at or above 0.70. - Minimum duration-weighted recall: 0.70 - Canonical roots: `C` then `G`, and `G` then `C` - Minor labels (`Cm`, `Gm`) must not satisfy a major window +- Overlapping matching estimates count only the union of covered section time ## References @@ -45,10 +51,15 @@ https://doi.org/10.1109/TASLP.2013.2294580 ## Security Notes -- Attack surface: in-memory float audio arrays only. The lock does not read - user files, URLs, or subprocess output. -- Trust boundary: synthetic fixtures are trusted test input. Production - recognizer input remains untrusted decoded audio from stem separation. -- Mitigations: no file I/O, no network, no shell. Failures stay inside pytest. -- Test points: `test_section_harmony_recovers_verse_c_then_chorus_g` and +- Attack surface: in-memory float audio arrays and recognizer segment timings + only. The lock does not read user files, URLs, or subprocess output. +- Trust boundary: synthetic fixtures and annotated section windows are trusted + test input. Production recognizer output remains untrusted evidence until it + is scored against those annotations. +- Mitigations: no file I/O, no network, no shell; major/minor labels remain + distinct; matching intervals are unioned before duration accumulation so + duplicate-time evidence cannot create a false high score. Failures stay + inside pytest. +- Test points: `test_duration_weighted_symbol_recall_unions_duplicate_time`, + `test_section_harmony_recovers_verse_c_then_chorus_g`, and `test_section_harmony_keeps_later_c_off_the_opening_window`. From 92fe9155a1896e13ba8e06e1762f6848fc2efdfa Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Mon, 17 Aug 2026 03:37:30 +0900 Subject: [PATCH 6/6] docs(changelog): record overlap-safe section recall --- CHANGELOG.md | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index d1766fc62..13cc63525 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -8,6 +8,10 @@ - Display the analyzed song tempo (BPM) as a badge in the rehearsal workspace. - 각 합주 역할(Role)별 개인 연습 진행도를 0~100% 범위로 기록 및 시각화할 수 있는 연습 진척도(`practiceProgress`) 트래커 기능 추가. UI 컨트롤(슬라이더 및 +/- 버튼)과 한/영 다국어 지원 포함. +### Fixed + +- Union overlapping matching recognizer intervals inside each annotated section before duration-weighted chord recall, preventing duplicate-time estimates from inflating known-progression accuracy evidence. + ## [0.1.3] - 2026-04-29 ### Fixed @@ -65,5 +69,5 @@ ### 추가됨 (Added) - `ChordsFeature` (코드 분석) 화면에서 각 파트(Role)의 `transpositionPlan`(이조/조옮김 계획)을 표시하는 기능을 추가했습니다. -- `RangesFeature` (음역대 분석) 화면에서 겹침 경고(Overlap warning) 외에 해당 파트의 채보(Transcription) 가능 노드 수를 요약하여 보여주는 기능을 추가했습니다. +- `RangesFeature` (음역대 분석) 화면에서 겹침 경고(Overlap warning) 외에 해당 파트(Role)의 채보(Transcription) 가능 노드 수를 요약하여 보여주는 기능을 추가했습니다. - 신규 UI 요소에 대한 100% 테스트 커버리지를 보장하는 단위 테스트를 추가했습니다 (`apps/desktop/src/features/chords/index.test.tsx`, `apps/desktop/src/features/ranges/index.test.tsx`).