feat: enforce matching decisions for playlist suggestions

[Iranman]

Summary

Migrates one additional matching consumer -- playlist missing-track suggestions -- to the shared backend-authoritative matching contract PR #19 established for Import Review's attach-recording endpoint.

• Base SHA: 14c7f02d778326792cf9984aed9d2a8224536dfc (main, includes merged PR fix: enforce matching decisions for import review attachment #19 and PR fix: bump Next.js to 16.2.11 for high-severity advisory cluster #30/Next.js 16.2.11)
• Head SHA: 4884819094a57f0aa54b7efbebcd4fbb1ffe7145

Update (review-correction pass, head 4884819): independent review found the first pass (1880970) accepted a missing decision_version, allowed a batch to partially apply when one row was stale/conflicted, and verified/rolled back persisted rows by matching artist/title text instead of an exact row identity (both false-positive- and cross-row-prone). All three are fixed below -- see "Review-correction summary."

Endpoints migrated

GET  /api/playlists/<name>/suggestions
POST /api/playlists/<name>/apply-safe-suggestions

POST /api/playlists/<name>/resolve-track (manual resolution) is untouched -- it stays an explicit, separately-audited user action, not folded into automatic eligibility (per scope).

Trust boundary

The backend rebuilds candidates and the matching decision fresh on every request; the browser never supplies or is trusted for safety, confidence, conflicts, or eligibility.

• POST apply-safe-suggestions now accepts only { suggestions: [{ track_key, mb_trackid?, item_id?, decision_version }] }. Any other client-supplied field (safe, confidence, conflicts, action_eligibility, ...) is parsed and ignored.
• GET suggestions responses are enriched with the same decision shape Import Review uses: mb_trackid / mb_albumid / mb_releasegroupid, decision_version (drv2:), and a full decision object (confidence_score, safety_key, review_required, requires_confirmation, conflicts, warnings, eligibility_reason, action_eligibility), plus an evidence block. The legacy top-level confidence / safe / reason fields remain for compatibility but are now derived only from the decision and marked @deprecated in TypeScript.

Matching-contract changes (backend/matching_contract.py)

build_recording_matching_decision() gained two additive, opt-in parameters -- Import Review's own call sites never pass them, so its behavior is bit-for-bit unchanged:

• library_identity_verified: bool = False -- lets a caller assert "this candidate is an existing Beets library item whose artist/title deterministically match the query." When true and the match is strong (title/artist similarity >= 0.94/0.90, no conflicts), the safety ladder reaches safety_key: "safe" without requiring a MusicBrainz Recording ID at all -- playlist resolution never needs to write a Recording ID, only correct the playlist's own desired-track entry.
  • New action_eligibility.playlist_resolve_without_review field reflects this (attach_without_review OR library_eligible).
  • attach_without_review was re-scoped from safety_key == "safe" to the underlying recording-ID-specific boolean directly, so a library-identity-only "safe" decision can never be misread as "safe to attach a Recording ID" (there may be none).
• extra_conflicts: Sequence[str] = () -- lets a caller fold an adapter-level conflict (e.g. a competing-candidate tie) into the same conflicts list the safety ladder and decision_version hash both already consume, so a forced-review state can never be applied after the version was computed (which would desync the hash from what was displayed).

Safe eligibility rules (playlist_resolve_without_review)

• Existing Beets library item: deterministic artist+title match (>=0.94/0.90 similarity), no recording-ID conflicts, no competing top candidate, any available fingerprint evidence supports rather than contradicts. No MusicBrainz Recording ID required.
• MusicBrainz-only recording candidate: identical bar to Import Review's attach_without_review (resolved Recording ID, release-group identity present, no conflicts, evidence-supported). A plain MB text-search hit without release-group resolution stays review-required -- this PR does not add release-group resolution to the playlist MB path (scoped out; see Known limitations).
• Candidate list order and raw score never substitute for these checks; the decision function computes eligibility from the same evidence regardless of how the adapter ranks/displays candidates.

AcoustID / AI

• AcoustID is attempted via the existing _acoustid_fingerprint_ids() helper whenever the top library candidate has a resolvable file path; when it doesn't, fingerprint state is truthfully not_attempted (never reported as matched/verified).
• No AI involvement was added to playlist suggestions in this PR -- the shared contract's AI fields simply report "not evaluated at this boundary," so AI configuration/availability can never block or influence playlist suggestions.

Review-correction summary

• Mandatory decision_version: every submitted row must include a nonempty decision_version or the whole batch is rejected 400 decision_version_required (or the generic 400 playlist_batch_rejected when mixed with other malformed rows) before any candidate lookup happens. A row that supplies some version (even forged) is no longer "malformed" -- it is correctly evaluated and rejected for staleness instead, never silently trusted.
• Strict all-or-nothing batches: the whole submitted batch is validated -- structural checks, then duplicate-submission checks, then fresh candidate/staleness/eligibility checks -- before anything is written. Any row that is malformed, stale, untrusted, still review-required, or conflicted rejects the entire batch (409 for business-rule rejections, 400 for structurally malformed ones) with zero manifest writes and no transaction created. Only a batch where every row is either already-resolved (idempotent no-op) or newly safe-to-apply proceeds, and then exactly one manifest write covers all of it.
• Duplicate submissions: the same track_key submitted twice with different candidates/versions is rejected (playlist_duplicate_submission, folds into the same all-or-nothing rejection); an exact duplicate of the same row is silently deduplicated before validation (not applied twice, no duplicate rollback record).
• Stable per-row identity (row_id): the desired-track manifest schema gained a row_id field, minted once (self-healing/idempotent migration for manifests written before this existed) and preserved through every read/write/merge. track_key in the API is this row_id. Persistence verification and rollback now locate and confirm the exact row by row_id -- never "any row whose text happens to match" -- so two rows resolving to the same artist/title, or a pre-existing unrelated row that already has the target text, can never be confused with each other or double-counted as proof of a change.
• True idempotent replay: replaying an already-applied submission now looks the row up by its stable row_id in the full manifest (not just the "still missing" subset) and compares its current identity to what the submission would resolve to. An exact match is a genuine 200 {"changed": false, "unchanged": [...], "reason": "already_resolved"} no-op -- no second write, no second transaction -- rather than the previous behavior of reporting a 409 stale rejection just because the row had moved out of the "missing" list.
• Truthful AcoustID evidence: the playlist candidate adapter now reports one of not_attempted / lookup_failed / no_match / matched / conflict / mapped_unverified instead of collapsing a real fpcalc/network exception into "no evidence," or reporting a real mapped recording id while simultaneously claiming no_result. conflict (AcoustID disagrees with the item's own Recording ID) reaches the shared contract as a genuine fingerprint mismatch and blocks eligibility, exactly like Import Review's own AcoustID handling.
• Fresh library-identity re-read: the playlist candidate adapter now re-reads the exact Beets item by item_id (lib.get_item) rather than trusting the (possibly stale-cached) library index payload; a deleted/moved item is dropped from the candidate set entirely instead of being offered as a safe suggestion from cached data.
• Truthful transaction failure handling: a manifest-write exception after a transaction was created now marks it Failed (sanitized message, no raw paths/exceptions) and returns 500/playlist_persistence_failed -- it can never end a request left Running.
• Frontend structured-error handling: handleApplySafeSuggestions now inspects the response's structured code via the existing apiErrorBody() helper instead of treating every rejection identically. playlist_suggestions_stale reloads suggestions; playlist_update_in_progress and batch-rejection codes (playlist_batch_rejected, playlist_review_required, playlist_conflict, candidate_not_in_trusted_set, decision_version_required, playlist_duplicate_submission) show counts/reasons without clearing the suggestion rows; only a fully atomic success clears rows and reloads playlist detail.
• Truthful derived-state accounting: the response now includes manifest_updated / m3u_updated (always false) / sync_required, so the UI never implies the M3U or Plex sync already reflects a manifest-only change (consistent with the pre-existing, unaudited resolve-track action's own scope).

Stale-decision / staleness behavior

At apply time the backend: acquires a per-playlist reservation -> reloads the manifest (self-healing row_id migration happens here if needed) -> rebuilds the trusted candidate set fresh for every row -> verifies every row's structure (nonempty decision_version) -> verifies every row's candidate is in the trusted set -> checks idempotency (row already resolved to this identity) -> verifies decision_version matches -> verifies playlist_resolve_without_review is still true. Any single row failing any of these checks (other than the idempotent-no-op case) rejects the entire batch: 409 playlist_suggestions_stale/playlist_review_required/playlist_conflict/etc. (the specific code when every blocking row shares one cause, else the generic playlist_batch_rejected), or 400 decision_version_required/playlist_batch_rejected for structurally malformed rows. Zero rows are written and no transaction is created for a rejected batch.

Batch / atomicity behavior

apply-safe-suggestions validates the entire submitted batch in two phases -- structural (dict shape, nonempty track_key/decision_version, duplicate-submission detection) then business (trusted-set membership, idempotency, staleness, eligibility) -- before writing anything. Any blocking row anywhere in the batch rejects it wholesale with zero mutation. Only when every row is either an idempotent no-op or newly safe does it proceed: exactly one manifest write (_playlist_replace_rows_by_id, keyed by each row's exact row_id) covers every applied row. The manifest is re-read afterward and each applied row's row_id is individually confirmed to hold its exact intended identity before the transaction is marked Completed; if any row fails this exact-identity verification the transaction is marked Failed and the response is 500 playlist_persistence_failed (never a false partial success). Reapplying an already-resolved row is a true no-op (unchanged, reason already_resolved) with no duplicate manifest write or transaction.

Concurrency

Reuses the same process-local, per-key-reservation pattern PR #19 established for attach-recording (_reserve_attach_recording_item / _ATTACH_RECORDING_RESERVATIONS_LOCK), scoped to playlist name: a second apply-safe-suggestions request for the same playlist gets 409 playlist_update_in_progress immediately; different playlists proceed fully in parallel. The reservation is released on success, every rejection path, and any unhandled exception. This deployment runs a single python app.py process (see Dockerfile), so process-local locking is sufficient here, matching the existing attach-recording assumption -- documented as a known limitation if that ever changes.

Transaction / audit behavior

Each successful batch creates one TransactionStore record, operation_type: "Playlist Match" (added to backend/transaction_engine.py's TRANSACTION_TYPES), with one changes[] entry per resolved row keyed by row_id: current_metadata / new_metadata / metadata_diff, candidate_identity, persisted_identity (populated only from the post-write, per-row_id re-read -- never left {} on a Completed transaction), decision_version, confidence, reason, warnings, evidence. Rollback data (playlist_track_restore operations, one per row, each carrying that row's exact row_id plus its previous/applied identity) is recorded before the manifest write. Any exception during the write/verify sequence marks the transaction Failed with a sanitized message -- it is never left Running.

Rollback behavior

playlist_track_restore is a new operation type wired into the existing generic /api/transactions/<id>/rollback dispatcher (_run_playlist_track_restore). It locates the exact row by its recorded row_id (never by artist/title text), verifies the row's current identity still matches the recorded applied state (playlist_rollback_state_changed if something else changed it since -- refuses to overwrite blindly), restores the previous identity via the same _playlist_replace_rows_by_id path, then re-reads the manifest and confirms that exact row_id now holds the previous identity (playlist_rollback_persistence_failed if not; playlist_rollback_row_missing if the row no longer exists at all). A restore that doesn't verify contributes to Partially Rolled Back, never a false Rolled Back. This is a metadata/manifest rollback only -- like attach-recording's own rollback, it does not restore downloaded audio or deleted files.

Frontend changes

• frontend/src/api/types.ts: new PlaylistSuggestionDecision / PlaylistActionEligibility / PlaylistSuggestionSubmission / PlaylistSuggestionOutcomeRow types; PlaylistTrackSuggestion gained the decision-shaped fields and marked confidence / safe / reason @deprecated; PlaylistApplySuggestionsResponse now matches the actual bucketed response shape.
• frontend/src/api/client.ts: applySafePlaylistSuggestions(name, suggestions) now takes and sends only the minimal submission list.
• frontend/src/views/Playlists.tsx: handleApplySafeSuggestions builds submissions only from track_key / mb_trackid / item_id / decision_version, filters by decision.action_eligibility.playlist_resolve_without_review, reports all five outcome buckets to the user, and re-fetches playlist detail afterward (the apply response no longer carries a full detail payload). safeSuggestionCount reads the decision field directly. No redesign of the page, no new state library, existing pipeline controls untouched.

Structured error codes

playlist_not_found, playlist_track_not_found, playlist_candidate_required, candidate_not_in_trusted_set, decision_version_required, playlist_suggestions_stale, playlist_batch_rejected, playlist_duplicate_submission, playlist_review_required, playlist_conflict, playlist_update_in_progress, playlist_persistence_failed, playlist_rollback_row_missing, playlist_rollback_state_changed, playlist_rollback_persistence_failed.
(invalid_recording_id, playlist_evidence_unavailable are reserved for future use; not reachable in this slice's scope.)

Explicitly out of scope (untouched)

Playlist download methods (SLSKD/SpotiFLAC/yt-dlp/SoundCloud), staged-folder naming, import-downloaded workflow, Beets import behavior, Plex path mapping/sync, playlist source parsing/syncing, quality cleanup, Library Cleanup, Replacement jobs, submission workflows, album matching, Import Review behavior, PR #19 attachment behavior, AI provider configuration, manual resolve-track (still a separate, unaudited-in-this-PR explicit action).

Files changed

app.py
backend/matching_contract.py
backend/transaction_engine.py            (added "Playlist Match" to TRANSACTION_TYPES)
frontend/src/api/client.ts
frontend/src/api/types.ts
frontend/src/views/Playlists.tsx
tests/test_matching_contract.py          (existing action_eligibility key-set assertion updated)
tests/test_playlist_backend_job.py       (existing source-presence assertion updated for the renamed helper)
tests/test_transaction_integration.py    (existing rollback-route source assertion updated for the op-type set refactor)
tests/test_playlist_matching_contract.py       (new)
tests/test_playlist_safe_suggestions.py        (new)
tests/test_playlist_suggestion_integrity.py    (new)
tests/test_playlist_suggestion_frontend.py     (new)

No package/dependency files, screenshots, or documentation changed.

Tests

• New playlist matching-contract suite (69 tests across 4 files, up from 49) run 5x consecutively: OK every time.
• PR fix: enforce matching decisions for import review attachment #19 + related regression modules (test_import_review_attach_enforcement, test_import_review_attach_integrity, test_security_hardening, test_transaction_engine, test_transaction_integration, test_ai_batch_retry_race, test_ai_batch_import_reliability, test_matching_contract, test_routes_setup, test_review_queue_singleton_items): 367 tests, OK.
• Full existing playlist regression modules (test_playlist_saved_discovery, test_playlist_backend_job, test_playlist_pipeline, test_playlist_match_quality, test_playlist_resume, test_playlist_provider_album_guard, test_playlist_detail_performance): 87 tests, OK (pre-existing static-assertion tests updated only for the intentional rename/refactor, not weakened).
• Full unittest discover -s tests -p "test_*.py" run twice: 1286 tests, OK (1 skipped) both times (up from 1264; +22 new tests this pass covering mandatory decision_version, atomic-batch rejection, duplicate submissions, exact-row identity, AcoustID states, fresh library-identity re-read, and true idempotent replay).
• py_compile, security_secret_scan.py, validate_compose_security.py, git diff --check, git status --short: all clean.
• Frontend (npm ci / typecheck / lint / build): all clean. npm audit --audit-level=high: 0 vulnerabilities.
• Docker (RUN_DOCKER_SMOKE=1, BeetsFreshInstallDockerSmokeTests): OK.

Known limitations

• MB-only (non-library) candidates are eligible only at the same bar as Import Review's attach_without_review (full release-group resolution required); this PR does not add MusicBrainz release-group resolution to the playlist search path, so a plain MB text-search hit always requires review here. Documented rather than expanded, to keep this slice narrow.
• Concurrency reservation is process-local (matches the existing attach-recording assumption and this deployment's single-process python app.py entrypoint); if the deployment ever moves to multiple web workers, this (and attach-recording's own reservation) would need a persistent/shared reservation instead.
• M3U regeneration and pipeline "checkpoint" state are untouched by this operation, consistent with the pre-existing resolve-track action's own scope -- suggestion application only ever rewrites the desired-track manifest, never the M3U file or pipeline checkpoint. The response now says so truthfully (manifest_updated / m3u_updated: false / sync_required) instead of implying otherwise.
• Manual resolve-track remains unaudited via TransactionStore in this PR (out of scope, per spec) and still uses the original text-based _playlist_apply_manifest_replacements path (unchanged); it stays entirely separate from automatic eligibility and from the new row_id-based apply/rollback path.
• The row_id manifest-schema migration is self-healing (assigned on first read/write of a legacy manifest) but not instantaneous: two concurrent first-time migrations of the same never-yet-migrated manifest could each mint different ids for the same logical row before either write lands; this fails closed (a mismatched row_id is reported stale/not found, never applied to the wrong row) rather than silently misapplying anything.