diff --git a/CHANGELOG.md b/CHANGELOG.md index eea696893..306e07f0a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,11 @@ ## [Unreleased] +### Changed + +- Optimized `_checkerboard_novelty_reference` by replacing Python inner loops with a vectorized NumPy `sliding_window_view` and `einsum` implementation for fallback parity, preserving the Rust kernel as the production arithmetic path. Zero-sized kernels retain the prior all-zero curve instead of entering an invalid `n + 1` window range. + + ### Added - Display the analyzed song tempo (BPM) as a badge in the rehearsal workspace. diff --git a/docs/doctoring/checkerboard-novelty-parity.md b/docs/doctoring/checkerboard-novelty-parity.md new file mode 100644 index 000000000..f5d17b26e --- /dev/null +++ b/docs/doctoring/checkerboard-novelty-parity.md @@ -0,0 +1,48 @@ +# Checkerboard novelty parity and vectorization evidence + +## Scope + +This note documents the mathematical and numerical contract for BandScope's checkerboard-novelty implementation in `services/analysis-engine/src/bandscope_analysis/sections/segmenter.py` and the production Rust kernel in `services/analysis-engine/rust/src/lib.rs`. It is intentionally narrower than an end-to-end claim about music-structure accuracy: the change vectorizes an established reference computation and preserves Rust/NumPy numerical parity; it does not introduce a new segmentation model or claim improved boundary-detection accuracy. + +Primary and current review sources were rechecked on 2026-08-15. The 2024 TISMIR tutorial remains a current peer-reviewed tutorial treatment of novelty functions for music signal processing, while Foote (2000) is the foundational checkerboard/self-similarity formulation and Nieto et al. (2020) remains an authoritative review of audio-based music structure analysis. + +## Algorithmic contract + +BandScope follows the classical novelty-boundary pattern: a local checkerboard-like kernel is correlated with a patch of a self-similarity representation around the main diagonal, producing a one-dimensional novelty curve whose peaks indicate locally contrasting regions. This is consistent with Foote's original method and with the modern tutorial derivation in Müller and Chiu (2024). + +The implementation contract for this PR is: + +- each valid `kernel_size × kernel_size` diagonal patch contributes exactly one novelty value; +- the final valid patch is included for both odd and even kernel sizes; +- inputs shorter than a nonzero kernel preserve the established all-zero output; +- `kernel_size == 0` preserves the established all-zero output instead of constructing an invalid `n + 1` sliding-window range; +- the NumPy implementation uses array views/vectorized contraction rather than Python inner loops and does not intentionally materialize one copy per patch; and +- the Rust production kernel and NumPy reference are required to agree numerically on zero, unit, odd, even, boundary-size, and shorter-than-kernel cases. + +The zero-kernel behavior is a backward-compatibility boundary, not a statement that a zero-sized checkerboard has a musically meaningful interpretation. New callers should use a positive analysis kernel; the zero case remains defined so legacy or defensive call paths fail safely and deterministically. + +## Numerical evidence required by the repository + +The PR's tests separate algorithmic parity from musical-validity claims. They must cover: + +1. an independent scalar/oracle calculation for representative nonzero kernels so the vectorized NumPy path is not tested only against itself; +2. Rust-to-NumPy parity after building and installing the native extension; +3. zero, unit, odd, even, exact-boundary, and shorter-than-kernel inputs; +4. finite output and stable output shape; and +5. repository-wide Python statement and branch coverage at 100% for owned production code, plus Rust tests and the normal BandScope quickcheck. + +A parity test can prove that the optimized implementation preserves the repository's specified arithmetic. It cannot by itself establish that detected boundaries match human musical-form annotations. End-to-end music-structure quality should therefore be evaluated separately on annotated recordings/datasets with boundary-tolerant MIR metrics rather than inferred from micro-kernel parity. + +## Interpretation for product use + +Music structure is subjective, ambiguous, and hierarchical; a novelty curve captures one useful segmentation principle rather than a unique ground truth. Nieto et al. (2020) specifically identify novelty/homogeneity, repetition, regularity, subjectivity, ambiguity, and hierarchy as material concerns for production MSA systems. BandScope should therefore treat this checkerboard kernel as one deterministic computational layer within a broader rehearsal-oriented analysis, expose confidence/uncertainty where downstream decisions depend on inferred structure, and avoid presenting a single novelty segmentation as the only valid interpretation of a song. + +Müller and Chiu (2024) likewise emphasize that useful novelty functions should be stable, precise, computationally efficient, robust to irrelevant variation, and evaluated with tolerance-aware event metrics. The vectorization in this PR addresses computational efficiency and parity only; it deliberately leaves feature design, peak picking, tolerance windows, and corpus-level boundary accuracy to their respective validated layers. + +## References (APA 7th) + +Foote, J. (2000). Automatic audio segmentation using a measure of audio novelty. *Proceedings of the 2000 IEEE International Conference on Multimedia and Expo (ICME 2000)*, *1*, 452–455. https://doi.org/10.1109/ICME.2000.869637 + +Müller, M., & Chiu, C.-Y. (2024). A basic tutorial on novelty and activation functions for music signal processing. *Transactions of the International Society for Music Information Retrieval, 7*(1), 179–194. https://doi.org/10.5334/tismir.202 + +Nieto, O., Mysore, G. J., Wang, C.-i., Smith, J. B. L., Schlüter, J., Grill, T., & McFee, B. (2020). Audio-based music structure analysis: Current trends, open challenges, and applications. *Transactions of the International Society for Music Information Retrieval, 3*(1), 246–263. https://doi.org/10.5334/tismir.54 diff --git a/services/analysis-engine/rust/src/lib.rs b/services/analysis-engine/rust/src/lib.rs index cffa84e2c..8c2c01439 100644 --- a/services/analysis-engine/rust/src/lib.rs +++ b/services/analysis-engine/rust/src/lib.rs @@ -37,13 +37,15 @@ fn checkerboard_novelty<'py>( let half = kernel_size / 2; let mut novelty = Array1::::zeros(n); - // Mirror the Python guard: matrices smaller than the kernel yield zeros. - if n < kernel_size { + // Preserve the legacy all-zero curve for a zero-sized kernel; matrices + // smaller than a nonzero kernel likewise have no valid diagonal patch. + if kernel_size == 0 || n < kernel_size { return Ok(novelty.into_pyarray(py)); } - // valid_range = range(half, n - half) - for i in half..(n - half) { + // Emit one value for every valid K×K diagonal patch. For even kernels, + // this includes the final bottom-right patch that `half..(n - half)` omits. + for i in half..(half + n - kernel_size + 1) { let mut acc = 0.0_f64; // patch = ssm[i-half : i+half, i-half : i+half]; sum(patch * kernel) for r in 0..kernel_size { diff --git a/services/analysis-engine/src/bandscope_analysis/sections/segmenter.py b/services/analysis-engine/src/bandscope_analysis/sections/segmenter.py index 7841e20d2..46a04e38f 100644 --- a/services/analysis-engine/src/bandscope_analysis/sections/segmenter.py +++ b/services/analysis-engine/src/bandscope_analysis/sections/segmenter.py @@ -18,6 +18,7 @@ import librosa import numpy as np +from numpy.lib.stride_tricks import sliding_window_view from numpy.typing import NDArray from .._native import HAVE_RUST, _checkerboard_novelty_rust @@ -121,7 +122,7 @@ def _checkerboard_novelty_reference( half = kernel_size // 2 novelty = np.zeros(n, dtype=np.float64) - if n < kernel_size: + if kernel_size == 0 or n < kernel_size: return novelty # Build the Foote checkerboard kernel. On-diagonal quadrants (within a @@ -134,16 +135,20 @@ def _checkerboard_novelty_reference( kernel[:half, :half] = 1.0 kernel[half:, half:] = 1.0 - # Sum each checkerboard offset across all valid diagonal windows at once. - valid = novelty[half : n - half] - for di in range(-half, half): - for dj in range(-half, half): - value = kernel[di + half, dj + half] - diagonal = np.diagonal(ssm[half + di : n - half + di, half + dj : n - half + dj]) - if value > 0: - valid += diagonal - else: - valid -= diagonal + # ``sliding_window_view`` and ``diagonal`` are zero-copy views. Restrict + # the final window axis to the exact public output slice (even kernels have + # one additional bottom-right window), then contract directly into novelty + # so no K²×N temporary tensor or second O(N) result vector is materialized. + windows = sliding_window_view(ssm, (kernel_size, kernel_size)) + valid_length = n - kernel_size + 1 + diagonal_windows = np.diagonal(windows, axis1=0, axis2=1)[..., :valid_length] + np.einsum( + "ij,ijk->k", + kernel, + diagonal_windows, + out=novelty[half : half + valid_length], + optimize=False, + ) # Normalize by peak absolute magnitude, preserving sign. max_val = np.max(np.abs(novelty)) diff --git a/services/analysis-engine/tests/test_segmenter.py b/services/analysis-engine/tests/test_segmenter.py index 3586c00de..e60857f21 100644 --- a/services/analysis-engine/tests/test_segmenter.py +++ b/services/analysis-engine/tests/test_segmenter.py @@ -3,6 +3,7 @@ from unittest.mock import patch import numpy as np +import pytest from bandscope_analysis.sections.segmenter import ( MAX_SSM_FRAMES, @@ -104,29 +105,42 @@ def test_checkerboard_novelty_short_matrix_returns_zeros() -> None: assert np.array_equal(novelty, np.zeros(2, dtype=np.float64)) -def test_checkerboard_novelty_matches_loop_reference() -> None: - """Ensure diagonal vectorization preserves checkerboard novelty values.""" - rng = np.random.default_rng(42) - ssm = rng.random((48, 48), dtype=np.float64) - ssm = (ssm + ssm.T) / 2.0 - kernel_size = 8 +def _checkerboard_loop_oracle(ssm: np.ndarray, kernel_size: int) -> np.ndarray: + """Compute the Foote novelty curve with explicit centered patch loops.""" + n = ssm.shape[0] half = kernel_size // 2 - expected = np.zeros(ssm.shape[0], dtype=np.float64) + expected = np.zeros(n, dtype=np.float64) + if kernel_size == 0 or n < kernel_size: + return expected - # Foote kernel: +1 on-diagonal quadrants, -1 cross quadrants. kernel = np.full((kernel_size, kernel_size), -1.0, dtype=np.float64) kernel[:half, :half] = 1.0 kernel[half:, half:] = 1.0 - for i in range(half, ssm.shape[0] - half): - patch = ssm[i - half : i + half, i - half : i + half] - expected[i] = np.sum(patch * kernel) + for center in range(half, half + (n - kernel_size + 1)): + start = center - half + patch = ssm[start : start + kernel_size, start : start + kernel_size] + expected[center] = float(np.sum(patch * kernel)) + + max_value = float(np.max(np.abs(expected))) + return expected / max_value if max_value > 0.0 else expected + - max_value = np.max(np.abs(expected)) - expected = expected / max_value +@pytest.mark.parametrize( + ("matrix_size", "kernel_size"), + [(1, 1), (4, 3), (17, 4), (48, 8), (65, 64), (96, 15)], +) +def test_checkerboard_novelty_reference_matches_independent_loop( + matrix_size: int, + kernel_size: int, +) -> None: + """Vectorization preserves even, odd, unit, and boundary-size kernels.""" + rng = np.random.default_rng(matrix_size * 101 + kernel_size) + ssm = rng.random((matrix_size, matrix_size), dtype=np.float64) + ssm = (ssm + ssm.T) / 2.0 np.testing.assert_allclose( - _checkerboard_novelty(ssm, kernel_size=kernel_size), - expected, + _checkerboard_novelty_reference(ssm, kernel_size=kernel_size), + _checkerboard_loop_oracle(ssm, kernel_size), rtol=1e-12, atol=1e-12, ) diff --git a/services/analysis-engine/tests/test_segmenter_short_reference.py b/services/analysis-engine/tests/test_segmenter_short_reference.py new file mode 100644 index 000000000..ecd6ac431 --- /dev/null +++ b/services/analysis-engine/tests/test_segmenter_short_reference.py @@ -0,0 +1,14 @@ +"""Regression coverage for short-input checkerboard novelty reference behavior.""" + +import numpy as np + +from bandscope_analysis.sections.segmenter import _checkerboard_novelty_reference + + +def test_checkerboard_novelty_reference_returns_zeros_when_kernel_is_larger() -> None: + """Return one zero per frame when no centered checkerboard patch can fit.""" + ssm = np.array([[1.0, 0.25], [0.25, 1.0]], dtype=np.float64) + + novelty = _checkerboard_novelty_reference(ssm, kernel_size=4) + + np.testing.assert_array_equal(novelty, np.zeros(2, dtype=np.float64)) diff --git a/services/analysis-engine/tests/test_segmenter_zero_kernel.py b/services/analysis-engine/tests/test_segmenter_zero_kernel.py new file mode 100644 index 000000000..c3f202818 --- /dev/null +++ b/services/analysis-engine/tests/test_segmenter_zero_kernel.py @@ -0,0 +1,34 @@ +"""Regression coverage for zero-sized checkerboard novelty kernels.""" + +from __future__ import annotations + +import numpy as np +import pytest + +from bandscope_analysis import _native +from bandscope_analysis.sections.segmenter import _checkerboard_novelty_reference + + +def _identity_similarity() -> np.ndarray: + """Return a deterministic square self-similarity matrix.""" + return np.eye(3, dtype=np.float64) + + +def test_checkerboard_reference_zero_kernel_preserves_legacy_zeros() -> None: + """A zero-sized reference kernel retains the prior all-zero curve.""" + novelty = _checkerboard_novelty_reference(_identity_similarity(), kernel_size=0) + + np.testing.assert_array_equal(novelty, np.zeros(3, dtype=np.float64)) + + +@pytest.mark.skipif( + not _native.HAVE_RUST or _native._checkerboard_novelty_rust is None, + reason="Rust numeric extension is not installed", +) +def test_checkerboard_native_zero_kernel_matches_reference() -> None: + """The Rust kernel must not panic or emit an n+1 position for size zero.""" + assert _native._checkerboard_novelty_rust is not None + + novelty = _native._checkerboard_novelty_rust(_identity_similarity(), 0) + + np.testing.assert_array_equal(novelty, np.zeros(3, dtype=np.float64))