diff --git a/README.md b/README.md index 61b95d1..3e884bc 100644 --- a/README.md +++ b/README.md @@ -94,8 +94,10 @@ never needed for anything else in the app. Event link imports provide snapshot previews for Spotify/Deezer tracks, albums and playlists, YouTube/YouTube Music links and SoundCloud tracks/sets. -Direct-source rows preserve the linked recording; Spotify retains its existing -Deezer association. YouTube and SoundCloud use the separate, opt-in web-audio +Direct-source rows preserve the linked recording: an existing collection track +is reused only when it carries the same ISRC, never because its title or artist +looks alike, so a remix is not taken for its original. Spotify retains its +existing Deezer association. YouTube and SoundCloud use the separate, opt-in web-audio component, distributed alongside the application with pinned dependency notices. See [component setup and limits](web-audio-component/README.md). @@ -169,8 +171,8 @@ The full pipeline is documented in ## Install -1. Download `Syncbox-0.9.0-macos-arm64.dmg` from the - [latest release](https://github.com/Adridot/syncbox/releases/tag/v0.9.0), +1. Download `Syncbox-0.10.0-macos-arm64.dmg` from the + [latest release](https://github.com/Adridot/syncbox/releases/tag/v0.10.0), open it, and drag `Syncbox.app` into your Applications folder. A `.zip` of the same build is published alongside it if you prefer. 2. Open it once. Because the app is ad-hoc signed rather than signed with an diff --git a/docs/PRIVACY.md b/docs/PRIVACY.md index ef2f77a..e253cb3 100644 --- a/docs/PRIVACY.md +++ b/docs/PRIVACY.md @@ -1,6 +1,6 @@ # Syncbox Provider Data and Privacy -Includes the event link-import change under development for Syncbox 0.9.0. +Includes the event link-import change under development for Syncbox 0.10.0. Syncbox is a local, single-user macOS application. It has no Syncbox cloud backend and does not create a Syncbox account. diff --git a/docs/SPEC-UNIFIED.md b/docs/SPEC-UNIFIED.md index 9dd5d83..d85ec5a 100644 --- a/docs/SPEC-UNIFIED.md +++ b/docs/SPEC-UNIFIED.md @@ -157,11 +157,11 @@ Entities that survive regardless of the technology (details SPEC-01 §6): **5.6 Library sync.** Diffing by track: duplicate Spotify in a playlist → `ignored`; `ignored`/`ready` carried over unchanged; `imported`/`matched` reconciled; fresh match → `matched`/`conflict` (if ambiguous)/`new`; absent from playlist → `removed_from_source`. Default tags inherited from `source.tags`. Spotify snapshot (`snapshot_id`) detects changes. Apply: only `matched`/`ready` imported/tagged (otherwise 409); **library MyTags must pre-exist**. Removing a source = stop following only (RB tracks + MyTags preserved). **D9**: remove legacy `tag_rules` table (cause of B4); concept preserved via `source.tags`. -**5.7 Events.** 3 modes (from playlist / empty / by link). Unique atomic folder (`mkdir(exist_ok=False)`, slug collision → `-2`…). `default_tag` = event name (“Situation” category). Event matching: `ambiguous`→`ambiguous` (not `conflict`), no default tags. Staging/claim for Spotify/manual rows: one file shared only between two tracks with the **same non-empty ISRC**. Direct provider links require verified same-item provenance or an explicit local-file association instead. Apply: creates/repairs a **smart playlist** under “Event Imports”, restores the XML after commit. **Smart playlist**: `SmartList = ":"` (operator 8 = contains); IDs > 2³¹ converted to **signed 32-bit** — **load-bearing**. RB write: new rows with **string ID** (mixed int+string PKs crash SQLAlchemy on flush); self-heal of a soft-deleted artist. Delete preview read **inside** the mutation session. **D10**: remove Live Import M3U8 (UI + `live_import.py` + route); always require RB closed and write the collection directly (eliminates B12). **D11/D23**: delete event **always with exact preview** + **guarded on `mutationAllowed`** like apply (fixes B11). **Cleanup (fixes T8/T12)**: event deletion removes its **disk artifacts** (staging folder, audio, `.xml.bak` snapshots) — no orphans. +**5.7 Events.** 3 modes (from playlist / empty / by link). Unique atomic folder (`mkdir(exist_ok=False)`, slug collision → `-2`…). `default_tag` = event name (“Situation” category). Event matching: `ambiguous`→`ambiguous` (not `conflict`), no default tags. Staging/claim for Spotify/manual rows: one file shared only between two tracks with the **same non-empty ISRC**. Direct provider links require verified same-item provenance, an ISRC-equal collection content, or an explicit local-file association instead. Apply: creates/repairs a **smart playlist** under “Event Imports”, restores the XML after commit. **Smart playlist**: `SmartList = ":"` (operator 8 = contains); IDs > 2³¹ converted to **signed 32-bit** — **load-bearing**. RB write: new rows with **string ID** (mixed int+string PKs crash SQLAlchemy on flush); self-heal of a soft-deleted artist. Delete preview read **inside** the mutation session. **D10**: remove Live Import M3U8 (UI + `live_import.py` + route); always require RB closed and write the collection directly (eliminates B12). **D11/D23**: delete event **always with exact preview** + **guarded on `mutationAllowed`** like apply (fixes B11). **Cleanup (fixes T8/T12)**: event deletion removes its **disk artifacts** (staging folder, audio, `.xml.bak` snapshots) — no orphans. **Event link snapshots (2026-09-07).** Existing events accept Spotify/Deezer tracks, albums and playlists; YouTube videos and playlists, including YouTube Music album representations; and SoundCloud tracks and sets/albums. A video with playlist context requires an explicit scope choice. A bounded background worker resolves a persisted preview, keeping source order and disclosed unavailable entries. Confirmation atomically adds only selected entries as manual rows, deduplicates provider/item identity, and stores the result for replay. Removed/ignored items require explicit re-addition. Preview, dismissal and confirmation create no audio jobs and never write Rekordbox. Imports do not change the event's synchronized Spotify playlist. -Direct Deezer/YouTube/SoundCloud rows preserve the exact linked recording through matching, staging, acquisition, restart, apply and managed-file migration. Similar titles or matching ISRCs do not establish identity. Automatic reuse requires recorded same-item provenance plus a matching file hash. Explicit local selection records the selected path and hash. Spotify retains existing ISRC/fuzzy matching and automatic/manual Deezer association. Each acquisition job stores its provider, server-owned selector, effective item, actual final path, source quality, output properties and processing. The shared FIFO continues after per-item failures; published-output recovery verifies owner, selector and hash. Web audio requires its own enablement/component, independently of Deezer credentials. Spotify disconnect removes Spotify previews/provenance while preserving direct-source jobs and imports. +Direct Deezer/YouTube/SoundCloud rows preserve the exact linked recording through matching, staging, acquisition, restart, apply and managed-file migration. Automatic reuse requires recorded same-item provenance plus a matching file hash or, failing that, an active local collection content with the same ISRC that passes the ISRC duration guard; apply re-verifies that ISRC before any Rekordbox write. Title/artist/duration similarity and staged-file metadata scoring never establish identity, so a remix (its own ISRC) never lands on its original. Health > Backups previews and executes a trash-first reconciliation of downloads that duplicate an ISRC-equal collection content. Explicit local selection records the selected path and hash. Spotify retains existing ISRC/fuzzy matching and automatic/manual Deezer association. Each acquisition job stores its provider, server-owned selector, effective item, actual final path, source quality, output properties and processing. The shared FIFO continues after per-item failures; published-output recovery verifies owner, selector and hash. Web audio requires its own enablement/component, independently of Deezer credentials. Spotify disconnect removes Spotify previews/provenance while preserving direct-source jobs and imports. Previews are bounded to 1,000 entries, 100 catalogue pages, a 4 MiB stored manifest and a 120-second resolution budget; provider access failures and limits are explicit. Authenticated Spotify track hydration uses bounded individual requests with partial-result preservation and the existing anonymous title-only fallback. Spotify collections require a connected account with access to their items. Web acquisition prefers original/native supported audio, remuxes when possible, otherwise converts to MP3; complete short audio needs no artwork. Source and output quality are displayed separately. Ordinary Rekordbox apply/reapply guards remain mandatory. diff --git a/docs/USER_GUIDE.md b/docs/USER_GUIDE.md index b94c681..f52c0c2 100644 --- a/docs/USER_GUIDE.md +++ b/docs/USER_GUIDE.md @@ -103,7 +103,9 @@ The Health hub contains five tabs: - **Missing**: repair missing Rekordbox file references; - **Untagged**: review structurally untagged or suspicious entries; - **Smart Fixes**: preview conservative metadata corrections before applying; -- **Backups**: inspect, rotate, and restore guarded Rekordbox backups. +- **Backups**: inspect, rotate, and restore guarded Rekordbox backups, and + reconcile event downloads whose recording (same ISRC) was already in the + collection. Audio-quality diagnostics are local and read-only. An uncertain spectral cutoff is keeper-neutral; it is not presented as proof of a lossy transcode. diff --git a/openspec/changes/reconcile-deezer-event-tracks-by-isrc/.openspec.yaml b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/.openspec.yaml new file mode 100644 index 0000000..8126432 --- /dev/null +++ b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/.openspec.yaml @@ -0,0 +1,2 @@ +schema: spec-driven +created: 2026-10-09 diff --git a/openspec/changes/reconcile-deezer-event-tracks-by-isrc/design.md b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/design.md new file mode 100644 index 0000000..709af3b --- /dev/null +++ b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/design.md @@ -0,0 +1,81 @@ +## Context + +See proposal.md - Why. The relevant code today: + +- `events_service.match_event_tracks` short-circuits exact-source rows into `source_identity.known_file`, which only accepts an `acquisition_jobs` row with the same `provider` + `effective_source_item_id`, the same published path and an unchanged SHA-256. `effective_source_item_id` was added by migration 0011 with no backfill, and the Spotify-to-ISRC path records whatever Deezer ID streamrip picked, so the proof rarely exists. +- `events_service.apply_event` demotes an exact-source `matched` row to `missing` when `source_identity.eligible_file` finds no provenance for the content's file. An ISRC match would be undone here. +- `matching.match` already does ISRC-first with the guarded duration/title collision policy and returns `method == "isrc"`; fuzzy follows only when ISRC fails. +- Deezer link previews carry the public-API ISRC into `event_tracks.isrc`; `rb.py` exposes the collection's ISRC column; `rb_write.add_content` tags ISRC on files it adds. +- `acquisition_migration` is the precedent for a previewed repair of existing data: `build_plan` returns a JSON plan with file states and the Rekordbox fingerprint, `execute` requires the identical plan, runs inside `mutate()` and deletes trash-first via `platform_os.delete_file` with the consent contract. `event_remove` already holds the primitives to untag, soft-delete and classify a content's other usages. +- `event_tracks` has no match-method column; `ready` rows are never re-run through the matcher. + +## Goals / Non-Goals + +**Goals:** restore recognition of ISRC-identical recordings for exact-source rows without reopening fuzzy matching; repair the duplicates 0.9.0 already produced with the same safety envelope as the other repairs; make the ISRC assumption verifiable on production before anything is executed. + +**Non-Goals:** backfilling provenance for old jobs (nothing to compare against), changing library/collection matching, scoring staged files by metadata for exact-source rows, deduplicating contents that Syncbox did not create, a generic "rematch ready rows" feature. + +## Decisions + +### 1. ISRC equality is an identity proof, second to provenance + +In the exact-source branch of `match_event_tracks`: provenance first (unchanged); when it yields neither a collection candidate nor a staged path, run `matching.match` over the same candidate list and accept the result only when `method == "isrc"`; any other result leaves the row `missing`. This keeps the duration/title collision guard and the configured `isrc_collision_policy`, and keeps a provenance `ready` row as `ready` (that row is the reconciliation's job, not the matcher's). + +Alternatives: treating Deezer like Spotify (full fuzzy) contradicts the remix/original decision; storing a match method column to distinguish ISRC matches adds schema for something the apply guard can re-verify statelessly (decision 2). + +### 2. Apply re-verifies the ISRC instead of demanding provenance + +`apply_event` keeps demoting exact-source `matched` rows that fail `eligible_file`, but first accepts the row when the current snapshot row for `content_id` has a normalized ISRC equal to the track's. The comparison is the matcher's own `_isrc` normalization, so match and apply agree. No new column, no new state. + +### 3. Reconciliation is a sibling of the acquisition storage migration + +New module `event_source_reconcile.py` with `build_plan(conn, storage_root, db_path)` and `execute(conn, db_path, backups_root, cache, storage_root, plan, *, app_db_path, retention, consent_to_permanent_delete)`, exposed as `GET/POST /api/events/source-reconciliation` through the same handler shape as `acquisition_storage_migration`. The plan carries `plan_version`, the Rekordbox fingerprint, each item's row id, status, ISRC, candidate content id, file state (path, size, mtime, SHA-256) and action, plus `skipped` entries with a reason and the `isrc_coverage` block. Execution rebuilds the plan and refuses on any difference (same stale-plan rule as the migration and track removal). + +Items are classified per row: + +| Row | Condition | Action | +| --- | --- | --- | +| `ready`, exact-source, ISRC | staged file inside the event staging dir, regular file; exactly one active local content with equal ISRC (guard passes) | `rematch`: trash the file first, then app DB only - status `matched`, `content_id`, confidence 100, `staging_file_path` NULL | +| `applied`, exact-source, ISRC | its content's file is the row's staged download inside the event staging dir; exactly one other active local content with equal ISRC; the duplicate carries no other tag, no other playlist membership and no other event row | `replace`: inside `mutate()` tag the original with the event tag, untag the duplicate, soft-delete the duplicate; then app DB `content_id` = original; then trash the file | +| any | zero candidates, more than one candidate, guard rejects, file missing/symlink/outside staging, duplicate used elsewhere, no ISRC | `skipped` with reason | +| `applied`, exact-source, ISRC | earlier `replace` repaired the row but its download is still on disk and referenced by no active content | `cleanup`: trash the file, clear `staging_file_path` | + +Candidate selection for the reconciliation is stricter than the matcher: more than one ISRC-equal content is `ambiguous` and skipped, because this path deletes files. The matcher keeps its first-candidate rule. + +Alternatives: a startup auto-repair was rejected because Rekordbox writes and deletions in this project are always previewed and consented; composing `event_remove.remove_tracks` + re-add was rejected because it leaves a `removed` row per track and runs two mutations per event instead of one. + +### 4. Order of effects inside execute + +Three actions, each with its own order: + +- `rematch` (no Rekordbox write): trash the download first, then set the row `matched`. A refused trash leaves row and file unchanged, so the spec's "row and file left unchanged" scenario holds and nothing can flip the row back through provenance while the file still exists. +- `replace`: one `mutate()` for all `replace` items of all events (a single backup is the pre-reconciliation restore point and a long run never rotates it out of the retention), then the application-DB transaction repointing each row, then trash. The row keeps its `staging_file_path` until the trash succeeds, so a kept file stays referenced and is never re-adopted. +- `cleanup`: an `applied` row repaired by an earlier `replace` whose download is still on disk and referenced by no active content. Trash, then clear `staging_file_path`. + +`PermanentDeleteConsentRequired` is raised (HTTP 428) only while nothing has changed in the run, so the client's consent loop replays the identical plan. After a change, a refused trash is reported in `cleanup_pending` with `consent_required`, and the next preview offers the kept file as a `cleanup` item. The acquisition job rows keep their `published_path`; `eligible_file` already ignores a path that no longer exists. + +### 5. ISRC coverage is computed in `build_plan`, not a separate script + +`isrc_coverage` = counts of exact-source rows with/without ISRC, ISRC rows with a candidate, the list of active contents with an empty ISRC whose file is the published or output path of a Deezer acquisition job (so files the owner dropped into a staging folder never trip the stop check), and the count of ISRCs containing non-alphanumerics. It costs one pass over data the plan already reads. The owner runs the preview against production and records the figures in `verification.md` before executing; a non-empty "without ISRC" list is a stop signal to investigate the tagging, not a reason to loosen matching. + +### 6. UI: one block in Health > Backups + +Same component pattern as the storage migration block: load the preview, show item count per action and the coverage figures, a button to execute with the plan, consent prompt on `PermanentDeleteConsentRequired`, result toast. No event-screen changes. + +## Risks / Trade-offs + +- [Deezer downloads lack an ISRC tag in Rekordbox] → coverage report makes it visible before execution; such rows simply stay as today. +- [Collection already holds two originals with one ISRC] → reconciliation skips as ambiguous; the Duplicates screen is the tool for that. +- [ISRC formatting differs (dashes) between Deezer API and Rekordbox tags] → the coverage report also counts ISRCs containing non-alphanumerics; if non-zero, extend `matching._isrc` to strip them in a follow-up. +- [Plan drifts while the user reads it] → fingerprint + file-state equality refuses execution, same as the other repairs. +- [Duplicate content soft-deleted while Rekordbox is open] → `mutate()` already enforces the closed-Rekordbox guard used by every write. + +## Migration Plan + +1. Ship matcher and apply changes; new imports stop downloading ISRC-known recordings immediately. +2. The owner opens Health > Backups on production, reads the preview and the coverage figures, and executes once satisfied. Backups are taken by `mutate()`; trashed files are recoverable from the OS trash. +3. Rollback: revert the build; reconciled rows remain valid `matched`/`applied` rows for the old code, since neither state depends on the new logic. + +## Open Questions + +- Whether YouTube/SoundCloud rows should ever carry an ISRC is moot for now: yt-dlp does not supply one, so the rule applies to Deezer in practice without naming the provider. diff --git a/openspec/changes/reconcile-deezer-event-tracks-by-isrc/proposal.md b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/proposal.md new file mode 100644 index 0000000..22dad36 --- /dev/null +++ b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/proposal.md @@ -0,0 +1,28 @@ +## Why + +Since 0.9.0 a track added to an event from a Deezer link is only recognised as already in the Rekordbox collection when Syncbox holds recorded provenance for that exact Deezer item: a published acquisition job with the same Deezer ID, the same path and an unchanged SHA-256. In the owner's and the users' libraries that proof almost never exists (files acquired before 0.9.0, files acquired through the Spotify-to-ISRC path under another Deezer ID, files retagged by Rekordbox, files acquired on another machine), so tracks that are plainly in the collection show as missing, get downloaded again and end up as duplicates in Rekordbox. Spotify additions never regressed because they still use the ISRC-first matcher. + +## What Changes + +- An exact-source event track (Deezer, YouTube, SoundCloud) that carries an ISRC SHALL be matched to an active local collection content with the same ISRC when no recorded provenance resolves it. The existing ISRC duration guard applies. Fuzzy title/artist/duration matching and staged-file metadata scoring remain forbidden for exact-source tracks: an ISRC identifies a recording, so a remix never matches its original. +- Event apply accepts an exact-source `matched` row either through recorded provenance (unchanged) or by re-verifying that the matched content still carries the same ISRC, instead of demoting ISRC matches to missing. +- A previewed, trash-first reconciliation repairs the data already produced by 0.9.0: a `ready` exact-source row whose download duplicates an ISRC-equal collection content becomes `matched` to that content and its download is trashed; an `applied` exact-source row whose Rekordbox content was created from such a download gets its event tag moved to the ISRC-equal original, the duplicate content soft-deleted and its file trashed. Ambiguous or otherwise-used contents are reported, never touched. +- The reconciliation preview reports ISRC coverage (exact-source rows without ISRC, collection candidates found, managed Deezer downloads in the collection lacking an ISRC tag) so the owner can confirm on the production database that ISRC matching actually covers the affected libraries before executing anything. + +## Capabilities + +### New Capabilities + +- `event-source-reconciliation`: previewed repair of existing exact-source event rows that duplicate ISRC-equal collection contents, including ISRC coverage reporting, trash-first file cleanup and guarded Rekordbox writes. + +### Modified Capabilities + +- `event-source-acquisition`: the requirement "Direct links retain their exact provider item" changes so that recorded provenance OR ISRC equality with the duration guard may automatically associate an existing collection content; approximate (fuzzy, staged-file scoring) association remains forbidden. This capability was introduced by `add-event-multiplatform-link-import`, which is complete but not yet archived; its spec must be synced to `openspec/specs/` before this delta is archived. + +## Impact + +- Sidecar: `events_service.match_event_tracks` and `apply_event` (exact-source branch), `source_identity`, one new reconciliation module following `acquisition_migration` (plan/execute, fingerprint freshness, `mutate()`, trash-first consent), two API routes, tests. +- UI: Health > Backups tab gains a reconciliation preview/execute block next to the storage migration, with its localized strings and API types. +- Data: no schema change. Existing acquisition job rows keep their published path; a trashed download simply stops being an eligible provenance file. +- Rekordbox: writes only inside `mutate()` with backups, reusing the existing tag/untag/soft-delete primitives from event track removal. +- Out of scope: library/collection acquisition policy, staged-file claiming by metadata for exact-source rows, YouTube/SoundCloud items without ISRC (they keep today's behaviour), the Duplicates screen. diff --git a/openspec/changes/reconcile-deezer-event-tracks-by-isrc/specs/event-source-acquisition/spec.md b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/specs/event-source-acquisition/spec.md new file mode 100644 index 0000000..69bf3ef --- /dev/null +++ b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/specs/event-source-acquisition/spec.md @@ -0,0 +1,31 @@ +## MODIFIED Requirements + +### Requirement: Direct links retain their exact provider item + +Deezer, YouTube, and SoundCloud link additions SHALL acquire the identified provider item. The system SHALL NOT substitute another item, another provider, or an approximate local/staged match automatically. This constraint SHALL apply to initial matching, later rematching, staged-file claiming, retries, and manual acquisition requests. An unavailable exact item SHALL remain unresolved with an explanation. + +An existing file SHALL be associated automatically only through one of two proofs of identity: recorded provenance for the same provider item and a valid file, or an active local collection content whose ISRC equals the linked item's ISRC and passes the existing ISRC duration guard. Recorded provenance takes precedence. Title, artist, or duration similarity, and staged-file metadata scoring, SHALL NOT associate a file with an exact-source track; other local associations SHALL require explicit user selection. When an ISRC association is applied to Rekordbox, the system SHALL re-verify that the matched content still carries the same ISRC and SHALL otherwise return the row to unresolved before writing. + +#### Scenario: Remix resembles a local original +- **WHEN** a linked remix resembles a local original by title, artist, or duration and the two carry different ISRCs +- **THEN** the original is not automatically selected and the remix retains its exact-source acquisition target + +#### Scenario: Linked item already in the collection under another acquisition +- **WHEN** a linked Deezer item carries an ISRC, no recorded provenance resolves it, and an active local collection content carries the same ISRC within the duration guard +- **THEN** the track is matched to that content with full confidence and no acquisition is offered + +#### Scenario: Linked item without ISRC +- **WHEN** a linked item carries no ISRC and no recorded provenance resolves it +- **THEN** the track remains unresolved and no fuzzy or staged-file association is made + +#### Scenario: ISRC association no longer valid at apply time +- **WHEN** an ISRC-matched exact-source row is applied and the matched content is no longer active or no longer carries that ISRC +- **THEN** the row is returned to unresolved before any Rekordbox write and the rest of the event applies normally + +#### Scenario: Exact Deezer item is unavailable +- **WHEN** a linked Deezer item is unavailable but another item has the same ISRC +- **THEN** acquisition fails for the linked item without automatically downloading the other item + +#### Scenario: Subsequent rematch or staged-file scan +- **WHEN** a linked SoundCloud item is still missing and a later matching or staging scan finds an approximate candidate +- **THEN** the scan does not silently associate that file with the linked item diff --git a/openspec/changes/reconcile-deezer-event-tracks-by-isrc/specs/event-source-reconciliation/spec.md b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/specs/event-source-reconciliation/spec.md new file mode 100644 index 0000000..0926303 --- /dev/null +++ b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/specs/event-source-reconciliation/spec.md @@ -0,0 +1,57 @@ +## Purpose + +Repair exact-source event rows that were downloaded or imported while an ISRC-equal recording already existed in the Rekordbox collection, through a previewed, trash-first, guarded operation. + +## ADDED Requirements + +### Requirement: Reconciliation is previewed before execution + +The system SHALL expose a read-only reconciliation preview covering every exact-source event row that carries an ISRC. The preview SHALL list each row with its proposed action, the ISRC-equal collection content it would be associated with, and the file that would be trashed. Execution SHALL require the exact preview it was built from and SHALL refuse a preview that no longer matches the current Rekordbox snapshot, application rows, or file states. + +#### Scenario: Preview lists a redundant download +- **WHEN** a `ready` exact-source row holds a staged download and an active local collection content carries the same ISRC +- **THEN** the preview proposes matching the row to that content and trashing the download + +#### Scenario: Preview is stale at execution +- **WHEN** the Rekordbox snapshot, the row, or the download changed after the preview was produced +- **THEN** execution is refused and the user must reopen the preview + +### Requirement: Redundant downloads are matched to the existing collection content + +For a `ready` exact-source row whose staged download duplicates exactly one active local collection content with the same ISRC, execution SHALL set the row to matched with that content, clear its staged file reference, and send the download to the operating-system trash. No Rekordbox write SHALL occur for this case. + +#### Scenario: Ready row becomes matched +- **WHEN** execution processes a `ready` exact-source row with one ISRC-equal collection content +- **THEN** the row is matched to that content with full confidence, its download is trashed, and the next apply tags the existing content + +#### Scenario: Trash is unavailable without consent +- **WHEN** trashing the download fails and explicit permanent-delete consent was not provided +- **THEN** the row and the file are left unchanged and the system requests consent for a retry + +### Requirement: Duplicate applied contents are replaced by the original + +For an `applied` exact-source row whose Rekordbox content was created from its staged download and that duplicates exactly one other active local collection content with the same ISRC, execution SHALL, inside one guarded Rekordbox mutation with a backup: tag the original content with the event tag, remove the event tag from the duplicate, soft-delete the duplicate content, and then repoint the row to the original and trash the download file. The event SHALL keep its status, tag, and smart playlist. + +#### Scenario: Applied duplicate is replaced +- **WHEN** execution processes an applied exact-source row whose content duplicates an older ISRC-equal content +- **THEN** the original content carries the event tag, the duplicate content is soft-deleted, the row references the original, and the download is trashed + +#### Scenario: Duplicate content is used elsewhere +- **WHEN** the duplicate content carries other tags, belongs to other playlists, or is referenced by another event row +- **THEN** the row is reported as blocked and nothing is changed for it + +### Requirement: Ambiguous or unproven candidates are never reconciled + +The system SHALL skip and report a row when more than one distinct collection content carries its ISRC, when the ISRC duration guard rejects the candidate, when the row's file is not inside the event's managed staging directory, or when the file is a symbolic link. Fuzzy metadata SHALL NOT be used to choose a candidate. + +#### Scenario: Two originals share the ISRC +- **WHEN** two active collection contents other than the row's own carry the same ISRC +- **THEN** the row is reported as ambiguous and left unchanged + +### Requirement: ISRC coverage is reported + +The preview SHALL report, for the whole application database, how many exact-source event rows carry an ISRC and how many do not, how many ISRC rows found a collection candidate, and which managed Deezer downloads present in the collection carry no ISRC tag. The system SHALL NOT change behaviour based on this report; it exists so the owner can confirm on production that ISRC matching covers the affected libraries before executing. + +#### Scenario: Coverage shows a gap +- **WHEN** a managed Deezer download present in the collection has an empty ISRC +- **THEN** the preview lists that content so the owner can inspect it before execution diff --git a/openspec/changes/reconcile-deezer-event-tracks-by-isrc/tasks.md b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/tasks.md new file mode 100644 index 0000000..32e3986 --- /dev/null +++ b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/tasks.md @@ -0,0 +1,25 @@ +## 1. ISRC matching for exact-source rows + +- [x] 1.1 In `events_service.match_event_tracks`, after the provenance lookup yields no candidate and no path, run `matching.match` and accept only `method == "isrc"`; verify with tests in `sidecar/tests/test_events_service.py` covering: ISRC hit → `matched`/confidence 100, fuzzy-only hit → stays `missing`, remix with a different ISRC → `missing`, provenance path still wins. +- [x] 1.2 In `events_service.apply_event`, accept an exact-source `matched` row whose current snapshot content has the same normalized ISRC before the `eligible_file` demotion; verify with a test that an ISRC-matched row is tagged on apply and that a row whose content lost its ISRC is reclassified `missing` without aborting the apply. +- [x] 1.3 Confirm `claim_staged_files` still skips metadata scoring for exact-source rows; verify the existing claim tests pass unchanged. + +## 2. Reconciliation module + +- [x] 2.1 Create `sidecar/src/syncbox/event_source_reconcile.py` with `build_plan` producing `plan_version`, fingerprint, items (`rematch`/`replace`), `skipped` with reasons, and `isrc_coverage`; verify with unit tests over a fixture app DB + Rekordbox fixture for each table row of design decision 3 (ready hit, applied hit, ambiguous, no candidate, file outside staging, duplicate tagged elsewhere, missing ISRC). +- [x] 2.2 Implement `execute`: stale-plan refusal, one `mutate()` for all `replace` items reusing `rb_write.tag_content`/`untag_content`/`soft_delete_content`, app-DB transaction for every item, trash-first cleanup via `platform_os.delete_file` with the consent contract and `cleanup_pending` reporting; verify with tests for the happy path, stale plan rejection, trash refused before any change (row and file unchanged, 428, retry with consent), and trash refused after the commit (row repaired, file kept, consent requested, `cleanup` item, retry with consent). +- [x] 2.3 Add `GET/POST /api/events/source-reconciliation` in `api.py` following `acquisition_storage_migration` (dry_run default, plan echo, consent flag); verify with an API test in `sidecar/tests/test_api.py` exercising preview then execute. + +## 3. Production ISRC check + +- [x] 3.1 Run the preview against the owner's production application DB and Rekordbox (read-only GET) and record in `verification.md`: counts with/without ISRC, candidates found, the list of managed Deezer downloads lacking ISRC, and the count of ISRCs containing non-alphanumerics; verify the "without ISRC" list is empty before any execution. If it is not empty, stop and report the affected files instead of executing. + +## 4. UI + +- [x] 4.1 Add the reconciliation block to `ui/src/screens/health/BackupsTab.vue` (preview counts per action, coverage figures, execute button, consent prompt, result toast) with API types and `en`/`fr` strings; verify with a spec in `ui/src/screens/__tests__/health-tabs.spec.ts` covering preview rendering, execute call with the plan, and the consent retry. + +## 5. Verification and docs + +- [x] 5.1 Run the full sidecar and UI test suites; verify green. +- [ ] 5.2 Execute the reconciliation on production after the task 3.1 check, confirm in Rekordbox that each replaced event track points at the original content and that duplicates are gone, and record the outcome in `verification.md`. +- [x] 5.3 Update `docs/SPEC-UNIFIED.md` and the user docs sentence on exact-source matching to state the ISRC rule; verify the docs mention both the ISRC proof and the fuzzy exclusion. diff --git a/openspec/changes/reconcile-deezer-event-tracks-by-isrc/verification.md b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/verification.md new file mode 100644 index 0000000..1dc3dbf --- /dev/null +++ b/openspec/changes/reconcile-deezer-event-tracks-by-isrc/verification.md @@ -0,0 +1,34 @@ +# Verification — 2026-10-09 + +## 3.1 Production ISRC check (read-only preview) + +Run against the owner's production data with the development build of +`event_source_reconcile.build_plan`, the application database opened with +SQLite `mode=ro` (no migration, no write) and master.db through +`rb.open_readonly`. Syncbox 0.9.0 was running, Rekordbox was closed. + +- Application DB: `~/Library/Application Support/Syncbox/syncbox.db` +- Rekordbox: `~/Library/Pioneer/rekordbox/master.db` +- ISRC collision policy: `guarded` + +| Figure | Value | +| --- | --- | +| Exact-source rows with an ISRC | 27 | +| Exact-source rows without an ISRC | 0 | +| ISRC rows with a collection candidate | 16 | +| Managed Deezer downloads in the collection without ISRC | 0 (list empty) | +| ISRCs containing non-alphanumerics (rows + collection) | 0 | +| Proposed `rematch` | 16 | +| Proposed `replace` / `cleanup` | 0 / 0 | +| Skipped `no_candidate` | 11 | + +The "without ISRC" list is empty, so the stop condition does not apply. All 16 +proposals are `ready` rows of the event "Thibault & Alix" (nothing from 0.9.0 +was applied yet, hence no `replace`). Each was compared by hand with its +candidate: same title and artist in every case, 15 in `Collection` and 1 in +`Collection manuelle`. "Sweet Caroline (Party Remix)" by DJ Ötzi resolves to +the remix content, not to the Neil Diamond original: the ISRC keeps the two +recordings apart. + +Not executed. Task 5.2 (execution on production and the Rekordbox check) +belongs to the owner, from Health > Backups on the released build. diff --git a/optional-component/pyproject.toml b/optional-component/pyproject.toml index 682230a..16cf947 100644 --- a/optional-component/pyproject.toml +++ b/optional-component/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "syncbox-deezer-component" -version = "0.9.0" +version = "0.10.0" description = "Separately distributed optional Deezer acquisition component for Syncbox" license = "MIT" requires-python = ">=3.13,<3.14" diff --git a/optional-component/uv.lock b/optional-component/uv.lock index 6f2ec60..cdff3b3 100644 --- a/optional-component/uv.lock +++ b/optional-component/uv.lock @@ -673,7 +673,7 @@ dependencies = [ [[package]] name = "syncbox-deezer-component" -version = "0.9.0" +version = "0.10.0" source = { virtual = "." } dependencies = [ { name = "certifi" }, diff --git a/release-build.json b/release-build.json index eccf917..0da2d86 100644 --- a/release-build.json +++ b/release-build.json @@ -34,11 +34,11 @@ }, "release": { "source_date_epoch_basis": "versioned-release-metadata", - "source_date_epoch_utc": "2026-09-07T00:00:00Z", - "version": "0.9.0" + "source_date_epoch_utc": "2026-10-09T00:00:00Z", + "version": "0.10.0" }, "schema": 1, - "source_date_epoch": 1788739200, + "source_date_epoch": 1791504000, "toolchain": { "apple_clang": "Apple clang version 21.0.0 (clang-2100.0.123.102)", "apple_ld": "@(#)PROGRAM:ld PROJECT:ld-1266.8", diff --git a/release/licenses/base/THIRD_PARTY_NOTICES.txt b/release/licenses/base/THIRD_PARTY_NOTICES.txt index fd9c659..20b08e1 100644 --- a/release/licenses/base/THIRD_PARTY_NOTICES.txt +++ b/release/licenses/base/THIRD_PARTY_NOTICES.txt @@ -115,7 +115,7 @@ Dependency inventory - [python] send2trash 2.1.0 — BSD-3-Clause — https://files.pythonhosted.org/packages/c5/f0/184b4b5f8d00f2a92cf96eec8967a3d550b52cf94362dad1100df9e48d57/send2trash-2.1.0.tar.gz - [python] six 1.17.0 — MIT — https://files.pythonhosted.org/packages/94/e7/b2c673351809dca68a0e064b6af791aa332cf192da575fd474ed7d6f16a2/six-1.17.0.tar.gz - [python] sqlalchemy 2.0.51 — MIT — https://files.pythonhosted.org/packages/02/f1/a7a892f18d4d224e6b26f706531eafccc41e37594d37d304786969ee13cb/sqlalchemy-2.0.51.tar.gz -- [python] sqlcipher3-wheels 0.6.2+syncbox.commoncrypto.1 — Zlib — https://github.com/Adridot/syncbox/tree/v0.9.0/sidecar/vendor/sqlcipher3-commoncrypto +- [python] sqlcipher3-wheels 0.6.2+syncbox.commoncrypto.1 — Zlib — https://github.com/Adridot/syncbox/tree/v0.10.0/sidecar/vendor/sqlcipher3-commoncrypto - [python] sse-starlette 3.4.5 — BSD-3-Clause — https://files.pythonhosted.org/packages/d2/1b/bc9e3e7a72dcdad7dc7888758f5d00f56f8909ed5cfdff822bd72bb4c520/sse_starlette-3.4.5.tar.gz - [python] starlette 1.3.1 — BSD-3-Clause — https://files.pythonhosted.org/packages/eb/e3/7c1dc7381d9f8ab7d854328ebfa884e62cb3f3d8549ddfd37c7814f42afa/starlette-1.3.1.tar.gz - [python] typing-extensions 4.16.0 — PSF-2.0 — https://files.pythonhosted.org/packages/f6/cc/6253133b5bb138fc3306cebfbda2c520f545d36b5be2c7255cc528bb45d6/typing_extensions-4.16.0.tar.gz diff --git a/release/licenses/base/dependency-inventory.json b/release/licenses/base/dependency-inventory.json index 2ded93a..6ce727f 100644 --- a/release/licenses/base/dependency-inventory.json +++ b/release/licenses/base/dependency-inventory.json @@ -1,6 +1,6 @@ { "artifact": "base", - "artifact_version": "0.9.0", + "artifact_version": "0.10.0", "entries": [ { "ecosystem": "bundled-native", @@ -1605,7 +1605,7 @@ ], "name": "sqlcipher3-wheels", "scope": "base-runtime", - "source": "https://github.com/Adridot/syncbox/tree/v0.9.0/sidecar/vendor/sqlcipher3-commoncrypto", + "source": "https://github.com/Adridot/syncbox/tree/v0.10.0/sidecar/vendor/sqlcipher3-commoncrypto", "source_path": "sidecar/vendor/sqlcipher3-commoncrypto", "source_tree_sha256": "c9c957aee64c8722543814e6fe9e52f9c363bd5bd89aa52dfabcaa3749eafb7c", "upstream_declared_license": "MIT", @@ -6819,10 +6819,10 @@ } ], "inputs": { - "cargo_lock_sha256": "292bb1ece81134a63dc2a2850ef2b3dffdcca00f812fb1a3edb64d852e153f3e", + "cargo_lock_sha256": "cadd18afc0fa44cc57f2ecee2c2ffa17ca3dfd6124929457b604a90d71dec46e", "pnpm_lock_sha256": "9167ad4c2062672bcbf637e48f05ff963f68bcd387b3d429f358e5da5e016c47", - "python_lock_sha256": "6f171f1b7383b9531fd0d14b5677fa0548a2e3a1429538ebdf309892c4ea0880", - "release_build_sha256": "d247e1887f88b369e33dbeec1b0f8cb22ac854c85ba0976b2760a06275e16f9d" + "python_lock_sha256": "7c02c45496f3987c52fa0315c6b64deae2f407eeba7be95d93abd15417bf12cb", + "release_build_sha256": "0626f4c389864cad6b74193442d19f1522cc6bc7a1db3e786a88963082cd2c3a" }, "project": { "assets": [ diff --git a/release/licenses/optional/dependency-inventory.json b/release/licenses/optional/dependency-inventory.json index 58f0deb..a0d215e 100644 --- a/release/licenses/optional/dependency-inventory.json +++ b/release/licenses/optional/dependency-inventory.json @@ -1,6 +1,6 @@ { "artifact": "optional", - "artifact_version": "0.9.0", + "artifact_version": "0.10.0", "entries": [ { "ecosystem": "bundled-native", @@ -1182,8 +1182,8 @@ "inputs": { "cargo_lock_sha256": null, "pnpm_lock_sha256": null, - "python_lock_sha256": "26c1d88100f4ba6141933fa807cd71e91f17481cf16302593958a2026df5bb8f", - "release_build_sha256": "d247e1887f88b369e33dbeec1b0f8cb22ac854c85ba0976b2760a06275e16f9d" + "python_lock_sha256": "2130686d7522cc0f8650a9969667e2b6c6580536e9603e11b74bd691ff6c41a4", + "release_build_sha256": "0626f4c389864cad6b74193442d19f1522cc6bc7a1db3e786a88963082cd2c3a" }, "project": { "assets": [], diff --git a/shell/package.json b/shell/package.json index 5d4dae9..c76fbb9 100644 --- a/shell/package.json +++ b/shell/package.json @@ -1,7 +1,7 @@ { "name": "syncbox-shell", "private": true, - "version": "0.9.0", + "version": "0.10.0", "packageManager": "pnpm@10.29.3", "engines": { "node": "24.13.0", diff --git a/shell/src-tauri/Cargo.lock b/shell/src-tauri/Cargo.lock index 77c6662..5eac2d6 100644 --- a/shell/src-tauri/Cargo.lock +++ b/shell/src-tauri/Cargo.lock @@ -3225,7 +3225,7 @@ dependencies = [ [[package]] name = "syncbox-shell" -version = "0.9.0" +version = "0.10.0" dependencies = [ "libc", "objc2", diff --git a/shell/src-tauri/Cargo.toml b/shell/src-tauri/Cargo.toml index 6ae1db4..d8efa67 100644 --- a/shell/src-tauri/Cargo.toml +++ b/shell/src-tauri/Cargo.toml @@ -1,6 +1,6 @@ [package] name = "syncbox-shell" -version = "0.9.0" +version = "0.10.0" license = "MIT" edition = "2021" diff --git a/sidecar/pyproject.toml b/sidecar/pyproject.toml index dd5c175..8f232a4 100644 --- a/sidecar/pyproject.toml +++ b/sidecar/pyproject.toml @@ -5,7 +5,7 @@ name = "syncbox" # Pinned to ui/package.json (the canonical version source, 6.11) by # tests/test_version_single_source.py. -version = "0.9.0" +version = "0.10.0" description = "Syncbox sidecar - lawful Rekordbox collection maintenance for DJs" requires-python = ">=3.14,<3.15" license = "MIT" diff --git a/sidecar/src/syncbox/api.py b/sidecar/src/syncbox/api.py index a58d160..f91740b 100644 --- a/sidecar/src/syncbox/api.py +++ b/sidecar/src/syncbox/api.py @@ -49,6 +49,7 @@ appdb, dedup, event_remove, + event_source_reconcile, events_service, library_service, link_imports, @@ -2206,6 +2207,28 @@ def acquisition_storage_migration(deps, request, body): ) +def events_source_reconciliation(deps, request, body): + _require_rekordbox(deps) + _require_storage(deps) + policy = deps.settings.get("isrc_collision_policy") + if request.method == "GET" or bool(body.get("dry_run", True)): + return event_source_reconcile.build_plan( + deps.conn, deps.storage_root, deps.db_path, isrc_collision_policy=policy + ) + return event_source_reconcile.execute( + deps.conn, + deps.db_path, + deps.backups_root, + deps.cache(), + deps.storage_root, + body.get("plan"), + app_db_path=deps.app_db_path, + retention=deps.retention, + consent_to_permanent_delete=bool(body.get("consent_to_permanent_delete")), + isrc_collision_policy=policy, + ) + + # --- duplicates (5.4, A3 per 5.12) --------------------------------------------------- @@ -3021,6 +3044,11 @@ def r(path: str, handler, methods: list[str], *, owns_lock=False) -> Route: r("/api/library/tracks/{track_id:int}/missing", track_mark_missing, ["POST"]), r("/api/library/tracks/{track_id:int}/ignore", track_ignore, ["POST"]), r("/api/library/tracks/{track_id:int}/restore", track_restore, ["POST"]), + r( + "/api/events/source-reconciliation", + events_source_reconciliation, + ["GET", "POST"], + ), r("/api/events", events_list, ["GET"]), r("/api/events", events_create, ["POST"]), r("/api/events/{event_id:int}", events_get, ["GET"]), diff --git a/sidecar/src/syncbox/event_source_reconcile.py b/sidecar/src/syncbox/event_source_reconcile.py new file mode 100644 index 0000000..05ac877 --- /dev/null +++ b/sidecar/src/syncbox/event_source_reconcile.py @@ -0,0 +1,314 @@ +"""Previewed repair of exact-source event rows whose download duplicates an +ISRC-equal collection content (reconcile-deezer-event-tracks-by-isrc). + +Sibling of acquisition_migration: build_plan only reads, execute requires +the identical plan. One action per row: + +- rematch: a 'ready' download duplicates one collection content. The + download is trashed FIRST, then the row is matched to that content; no + Rekordbox write, and a refused trash leaves row and file unchanged. +- replace: an 'applied' row whose Rekordbox content IS its download. One + mutate() moves the event tag to the original and soft-deletes the + duplicate, the row is repointed, then the download is trashed. +- cleanup: a replace whose download is still on disk (trash refused after + the commit). The row keeps referencing the file until it is gone, so the + staged-file adoption never picks it up again. + +A PermanentDeleteConsentRequired is raised (428) only while nothing has +changed yet, so the client's consent retry with the same plan is exact. +After that, a refused trash is reported as cleanup_pending and the next +preview offers it as a cleanup item. +""" + +from __future__ import annotations + +import re +from pathlib import Path + +from syncbox.event_delete import ( + _OTHER_TAGS_SQL, + _TAG_SQL, + EventMigrationError, + _file_state, + _fingerprint_tuple, + _inside_staging, + _serial_fingerprint, + event_staging, +) +from syncbox.events_service import SITUATION_CATEGORY, _now, _xml_snapshot +from syncbox.matching import _isrc, _isrc_collision +from syncbox.platform_os import PermanentDeleteConsentRequired, delete_file +from syncbox.rb import open_readonly +from syncbox.rb_write import open_rekordbox, soft_delete_content, tag_content, untag_content +from syncbox.safety.mutate import StaleSnapshotError, mutate +from syncbox.safety.paths import canonical_key +from syncbox.spotify import spotify_id_from_path + +PLAN_VERSION = 1 +_NON_ALNUM = re.compile(r"[^A-Z0-9]") +_CONTENTS_SQL = ( + "SELECT ID, Title, Length, ISRC, FolderPath FROM djmdContent " + "WHERE rb_local_deleted = 0 ORDER BY ID" +) +_LINK_SQL = ( + "SELECT 1 FROM djmdSongMyTag WHERE ContentID = :content_id " + "AND MyTagID = :tag_id AND rb_local_deleted = 0" +) +_PLAYLIST_SQL = ( + "SELECT 1 FROM djmdSongPlaylist WHERE ContentID = :content_id AND rb_local_deleted = 0" +) + + +def build_plan(conn, storage_root, db_path, *, isrc_collision_policy="guarded") -> dict: + before = _serial_fingerprint(db_path) + ro = open_readonly(db_path) + try: + def query(sql, params=()): + return ro.execute(sql, params).fetchall() + + contents, by_isrc, by_key = {}, {}, {} + for content_id, title, length, isrc, folder_path in query(_CONTENTS_SQL): + local = bool(folder_path) and spotify_id_from_path(folder_path) is None + content = { + "content_id": str(content_id), + "title": title, + "duration_ms": int(length * 1000) if length else 0, + "isrc": _isrc(isrc), + "path": folder_path, + "key": canonical_key(folder_path, storage_root) if local else None, + } + contents[content["content_id"]] = content + if content["key"]: + by_key.setdefault(content["key"], set()).add(content["content_id"]) + if content["isrc"]: + by_isrc.setdefault(content["isrc"], []).append(content) + + live = [dict(row) for row in conn.execute( + "SELECT t.*, e.name AS event_name, e.default_tag, e.staging_dir " + "FROM event_tracks t JOIN events e ON e.id = t.event_id " + "WHERE t.status != 'removed' AND e.delete_phase IS NULL ORDER BY t.id" + )] + file_holders, content_holders = {}, {} + for row in live: + for path in (row["staging_file_path"], row["selected_local_path"]): + if path: + file_holders.setdefault(canonical_key(path, storage_root), set()).add(row["id"]) + if row["content_id"]: + content_holders.setdefault(str(row["content_id"]), set()).add(row["id"]) + + tags, stagings = {}, {} + + def event_tag(row): + if row["event_id"] not in tags: + found = query(_TAG_SQL, {"tag": row["default_tag"], "category": SITUATION_CATEGORY}) + tags[row["event_id"]] = str(found[0][0]) if found else None + return tags[row["event_id"]] + + def staging_of(row): + if row["event_id"] not in stagings: + try: + stagings[row["event_id"]] = event_staging(row, storage_root) + except EventMigrationError: + stagings[row["event_id"]] = None + return stagings[row["event_id"]] + + def classify(row, isrc, file_key, candidates): + """An item dict, or the reason the row is left alone.""" + if not isrc: + return "missing_isrc" + path = Path(row["staging_file_path"]) + state = _file_state(path, with_hash=True) + if state.get("kind") != "file" or not _inside_staging(path, staging_of(row)): + return "unsafe_file" + if file_holders.get(file_key, set()) - {row["id"]}: + return "file_in_use" + on_file = by_key.get(file_key, set()) # active contents playing the download + item = { + "track_id": row["id"], "event_id": row["event_id"], "event_name": row["event_name"], + "title": row["title"], "artist": row["artist"], "isrc": isrc, + "status": row["status"], "duplicate_content_id": None, "tag_id": None, "file": state, + } + if row["status"] == "ready": + if on_file: + return "file_in_use" + item["action"] = "rematch" + else: + own = contents.get(str(row["content_id"])) if row["content_id"] else None + if own is None: + return "not_a_download" + if not on_file and own["isrc"] == isrc: + return {**item, "action": "cleanup", "content_id": own["content_id"]} + if on_file != {own["content_id"]}: + return "not_a_download" + tag_id = event_tag(row) + params = {"content_id": own["content_id"], "tag_id": tag_id} + if tag_id is None or not query(_LINK_SQL, params): + return "not_tagged" + if (content_holders.get(own["content_id"], set()) - {row["id"]} + or query(_OTHER_TAGS_SQL, params) or query(_PLAYLIST_SQL, params)): + return "duplicate_in_use" + item.update(action="replace", duplicate_content_id=own["content_id"], tag_id=tag_id) + if not candidates: + return "no_candidate" + if len(candidates) > 1: + return "ambiguous" + if _isrc_collision(row, candidates[0], isrc_collision_policy or "guarded"): + return "guard_rejected" + return {**item, "content_id": candidates[0]["content_id"]} + + items, skipped = [], [] + coverage = {"rows_with_isrc": 0, "rows_without_isrc": 0, "rows_with_candidate": 0} + odd_isrcs = sum(bool(_NON_ALNUM.search(c["isrc"])) for c in contents.values() if c["isrc"]) + for row in live: + if row["source_provider"] not in ("deezer", "youtube", "soundcloud"): + continue + isrc = _isrc(row["isrc"]) + path = row["staging_file_path"] + file_key = canonical_key(path, storage_root) if path else None + # The row's own download is never its own candidate. + candidates = [c for c in by_isrc.get(isrc, []) if c["key"] != file_key] if isrc else [] + coverage["rows_with_isrc" if isrc else "rows_without_isrc"] += 1 + coverage["rows_with_candidate"] += bool(candidates) + odd_isrcs += bool(isrc and _NON_ALNUM.search(isrc)) + if row["status"] not in ("ready", "applied") or not path: + continue + outcome = classify(row, isrc, file_key, candidates) + if isinstance(outcome, dict): + items.append(outcome) + else: + skipped.append({ + "track_id": row["id"], "event_id": row["event_id"], "event_name": row["event_name"], + "title": row["title"], "artist": row["artist"], "isrc": isrc or None, + "status": row["status"], "reason": outcome, + }) + + # Managed Deezer downloads Rekordbox knows without an ISRC tag: ISRC + # matching cannot see them, so the owner inspects them before executing. + deezer_files = { + canonical_key(path, storage_root) + for (path,) in conn.execute( + "SELECT COALESCE(published_path, output_path) FROM acquisition_jobs " + "WHERE provider = 'deezer' AND COALESCE(published_path, output_path) IS NOT NULL" + ) + } + coverage["downloads_without_isrc"] = [ + {"content_id": c["content_id"], "title": c["title"], "path": c["path"]} + for c in contents.values() + if c["key"] in deezer_files and not c["isrc"] + ] + coverage["non_alphanumeric_isrcs"] = odd_isrcs + finally: + ro.close() + if before != _serial_fingerprint(db_path): + raise StaleSnapshotError( + "master.db changed while the reconciliation preview was built; run it again" + ) + return { + "dry_run": True, + "plan_version": PLAN_VERSION, + "fingerprint": before, + "items": items, + "skipped": skipped, + "isrc_coverage": coverage, + } + + +def execute( + conn, + db_path, + backups_root, + cache, + storage_root, + plan, + *, + app_db_path=None, + retention: int = 20, + consent_to_permanent_delete: bool = False, + isrc_collision_policy="guarded", +) -> dict: + if not isinstance(plan, dict) or plan.get("plan_version") != PLAN_VERSION: + raise ValueError("a valid source reconciliation preview is required") + if build_plan(conn, storage_root, db_path, isrc_collision_policy=isrc_collision_policy) != plan: + raise StaleSnapshotError("the source reconciliation preview is stale; reopen it") + trashed, pending = [], [] + + def trash(item, *, may_raise) -> bool: + path = Path(item["file"]["path"]) + reason = None + if _file_state(path, with_hash=True) != item["file"]: + reason = "changed" + else: + try: + delete_file(path, consent_to_permanent_delete=consent_to_permanent_delete) + except PermanentDeleteConsentRequired: + if may_raise: + raise # nothing changed yet: the 428 retry replays this exact plan + reason = "permanent_delete_consent" + if reason: + pending.append({"track_id": item["track_id"], "path": str(path), "reason": reason}) + return False + trashed.append(str(path)) + return True + + # No Rekordbox write: trash first, so a refused trash changes nothing. + for item in plan["items"]: + if item["action"] == "replace" or not trash(item, may_raise=not trashed and not pending): + continue + if item["action"] == "rematch": + conn.execute( + "UPDATE event_tracks SET status = 'matched', content_id = ?, confidence = 100," + " staging_file_path = NULL, updated_at = ? WHERE id = ?", + (item["content_id"], _now(), item["track_id"]), + ) + else: + conn.execute( + "UPDATE event_tracks SET staging_file_path = NULL, updated_at = ? WHERE id = ?", + (_now(), item["track_id"]), + ) + + replaces = [item for item in plan["items"] if item["action"] == "replace"] + if replaces: + # One mutate() for every event: one backup is THE pre-reconciliation + # restore point, and a long run never rotates it out of the retention. + xml_path, xml_bytes = _xml_snapshot(db_path, None) + with mutate( + db_path, + backups_root, + retention=retention, + expected_fingerprint=_fingerprint_tuple(plan["fingerprint"]), + open_db=open_rekordbox, + invalidate_cache=cache.invalidate, + app_db_path=app_db_path, + backup_reason="event_source_reconciliation", + ) as db: + for item in replaces: + tag_content(db, item["content_id"], item["tag_id"]) + untag_content(db, item["duplicate_content_id"], item["tag_id"]) + soft_delete_content(db, item["duplicate_content_id"]) + if xml_bytes is not None: + xml_path.write_bytes(xml_bytes) # byte-identical restore (SPEC-01 1.6) + now = _now() + conn.execute("BEGIN") + try: + for item in replaces: + conn.execute( + "UPDATE event_tracks SET content_id = ?, updated_at = ? WHERE id = ?", + (item["content_id"], now, item["track_id"]), + ) + conn.execute("COMMIT") + except BaseException: + conn.execute("ROLLBACK") + raise + for item in replaces: + if trash(item, may_raise=False): + conn.execute( + "UPDATE event_tracks SET staging_file_path = NULL, updated_at = ? WHERE id = ?", + (_now(), item["track_id"]), + ) + return { + **plan, + "dry_run": False, + "trashed_files": trashed, + "cleanup_pending": pending, + "consent_required": any(p["reason"] == "permanent_delete_consent" for p in pending), + } diff --git a/sidecar/src/syncbox/events_service.py b/sidecar/src/syncbox/events_service.py index 8911af9..763b1f0 100644 --- a/sidecar/src/syncbox/events_service.py +++ b/sidecar/src/syncbox/events_service.py @@ -20,7 +20,7 @@ from pathlib import Path from syncbox import event_delete, library_service, relink, source_identity -from syncbox.matching import match +from syncbox.matching import _isrc, match from syncbox.rb_write import ( add_content, audio_metadata, @@ -64,10 +64,6 @@ def _now() -> str: return datetime.now(timezone.utc).isoformat(timespec="seconds") -def _isrc(value) -> str: - return (value or "").strip().upper() - - def slugify(name: str) -> str: """ASCII slug: NFKD-folded, lowercase, non-alphanumerics collapsed to '-'.""" text = unicodedata.normalize("NFKD", name or "").encode("ascii", "ignore").decode() @@ -355,9 +351,18 @@ def match_event_tracks(conn, event, cache, storage_root, **thresholds) -> list[d path = source_identity.known_file(conn, track) candidate = next((r for r in candidates if path and r.get("file_path") and resolve_stored_path(r["file_path"], storage_root) == Path(path)), None) + if path is None and _isrc(track["isrc"]): + # Second identity proof after provenance: an equal ISRC under + # the matcher's duration guard. Fuzzy stays forbidden here, so + # a remix (its own ISRC) never lands on its original. + same = [r for r in candidates if _isrc(r.get("isrc")) == _isrc(track["isrc"])] + result = match({"title": track["title"], "duration_ms": track["duration_ms"], + "isrc": track["isrc"]}, same, **thresholds) + if result.method == "isrc": + candidate = {"content_id": result.content_id} status = "matched" if candidate else "ready" if path else "missing" content_id = candidate["content_id"] if candidate else None - confidence = 100 if path else None + confidence = 100 if candidate or path else None conn.execute( "UPDATE event_tracks SET status = ?, content_id = ?, confidence = ?, staging_file_path = ?, updated_at = ? WHERE id = ?", (status, content_id, confidence, path, now, track["id"]), @@ -589,10 +594,13 @@ def apply_event( tracks = list_event_tracks(conn, event["id"]) applicable = [t for t in tracks if t["status"] in ("matched", "ready")] direct_matches = [track for track in applicable if track["status"] == "matched" and source_identity.exact_source(track)] - current_paths = {str(row["content_id"]): row.get("file_path") for row in cache.get(storage_root)} if direct_matches else {} + current = {str(row["content_id"]): row for row in cache.get(storage_root)} if direct_matches else {} invalid_matches = [] for track in direct_matches: - stored_path = current_paths.get(str(track["content_id"])) + content = current.get(str(track["content_id"]), {}) + if _isrc(track["isrc"]) and _isrc(content.get("isrc")) == _isrc(track["isrc"]): + continue # the ISRC identity proof still holds on the active content + stored_path = content.get("file_path") path = resolve_stored_path(stored_path, storage_root) if stored_path else None if not source_identity.eligible_file(conn, track, path): conn.execute("UPDATE event_tracks SET status = 'missing', content_id = NULL, confidence = NULL WHERE id = ?", (track["id"],)) diff --git a/sidecar/src/syncbox/optional_component.json b/sidecar/src/syncbox/optional_component.json index 4a93e96..4728412 100644 --- a/sidecar/src/syncbox/optional_component.json +++ b/sidecar/src/syncbox/optional_component.json @@ -1,10 +1,10 @@ { "architecture": "arm64", - "archive": "syncbox-deezer-component-0.9.0-macos-arm64.zip", + "archive": "syncbox-deezer-component-0.10.0-macos-arm64.zip", "certifi_version": "2026.6.17", "component": "syncbox-deezer-component", - "component_version": "0.9.0", - "download_url": "https://github.com/Adridot/syncbox/releases/download/v0.9.0/syncbox-deezer-component-0.9.0-macos-arm64.zip", + "component_version": "0.10.0", + "download_url": "https://github.com/Adridot/syncbox/releases/download/v0.10.0/syncbox-deezer-component-0.10.0-macos-arm64.zip", "executable": "syncbox-deezer-component", "pillow_version": "10.4.0", "pillow_wheel": "pillow-10.4.0-cp313-cp313-macosx_11_0_arm64.whl", @@ -13,8 +13,8 @@ "python_version": "3.13.11", "root": "syncbox-deezer-component", "schema": 1, - "sha256": "acd8be0c6ff870372befbe00511a8e79625f82d48835d97d8c80e00ee569d072", - "size": 17467373, + "sha256": "7693ee30fa9adbc6166f7796d7f11535d7cb7b5dc802d073a95c7bf45fc34dee", + "size": 17467380, "streamrip_commit": "189acda489927719aa8591f6acdd7d67aecf929b", "streamrip_version": "2.2.0" } diff --git a/sidecar/src/syncbox/web_audio.py b/sidecar/src/syncbox/web_audio.py index f9fd0e8..776704a 100644 --- a/sidecar/src/syncbox/web_audio.py +++ b/sidecar/src/syncbox/web_audio.py @@ -12,7 +12,7 @@ from syncbox import acquisition NAME = "syncbox-web-audio-component" -VERSION = "0.9.0" +VERSION = "0.10.0" def component_root(data_dir): diff --git a/sidecar/src/syncbox/web_audio_component.json b/sidecar/src/syncbox/web_audio_component.json index 386920e..b3cb44d 100644 --- a/sidecar/src/syncbox/web_audio_component.json +++ b/sidecar/src/syncbox/web_audio_component.json @@ -1,13 +1,13 @@ { "component": "syncbox-web-audio-component", - "version": "0.9.0", + "version": "0.10.0", "protocol_version": 1, "platform": "macos-arm64", - "archive": "syncbox-web-audio-component-0.9.0-macos-arm64.zip", + "archive": "syncbox-web-audio-component-0.10.0-macos-arm64.zip", "size": 77656556, - "sha256": "61a85eef7ced0ae7788a53acbea8ad0b3d9788926acd1822efff65f0882fe880", + "sha256": "b24f738ba7700510b229b269b5ea92400b7d056e4f7df2052162cf749763ec77", "license_inventory_complete": true, - "component_version": "0.9.0", + "component_version": "0.10.0", "root": "syncbox-web-audio-component", "executable": "syncbox-web-audio-component", "versions": { @@ -16,5 +16,5 @@ }, "ffmpeg": "ffmpeg version 9.0.1 Copyright (c) 2000-2026 the FFmpeg developers", "deno": "deno 2.9.5 (stable, release, aarch64-apple-darwin)", - "download_url": "https://github.com/Adridot/syncbox/releases/download/v0.9.0/syncbox-web-audio-component-0.9.0-macos-arm64.zip" + "download_url": "https://github.com/Adridot/syncbox/releases/download/v0.10.0/syncbox-web-audio-component-0.10.0-macos-arm64.zip" } diff --git a/sidecar/tests/test_api.py b/sidecar/tests/test_api.py index 175f0d5..a542b06 100644 --- a/sidecar/tests/test_api.py +++ b/sidecar/tests/test_api.py @@ -915,6 +915,34 @@ def fake_delete( assert seen[-1] == {"dry_run": False, "plan": preview_plan, "consent": True} +def test_source_reconciliation_previews_then_executes_with_consent(tmp_path, monkeypatch): + env = make_env(tmp_path) + env.deps.settings.update({"isrc_collision_policy": "strict"}) + preview_plan = {"dry_run": True, "plan_version": 1, "items": [{"track_id": 1}]} + seen = [] + monkeypatch.setattr(api.event_source_reconcile, "build_plan", + lambda conn, storage, db, *, isrc_collision_policy: ( + seen.append(("plan", isrc_collision_policy)) or preview_plan)) + + def fake_execute(conn, db, backups, cache, storage, plan, *, consent_to_permanent_delete, **kwargs): + seen.append(("execute", plan, consent_to_permanent_delete, kwargs["isrc_collision_policy"])) + if not consent_to_permanent_delete: + raise PermanentDeleteConsentRequired(Path("/vol/x.mp3"), OSError("no trash")) + return {**plan, "dry_run": False, "trashed_files": ["/vol/x.mp3"]} + + monkeypatch.setattr(api.event_source_reconcile, "execute", fake_execute) + url = "/api/events/source-reconciliation" + assert env.client.get(url).json() == preview_plan + assert env.client.post(url, json={}).json() == preview_plan # dry run is the default + blocked = env.client.post(url, json={"dry_run": False, "plan": preview_plan}) + assert blocked.status_code == 428 and blocked.json()["consent"] == "permanent_delete" + ok = env.client.post(url, json={"dry_run": False, "plan": preview_plan, + "consent_to_permanent_delete": True}) + assert ok.status_code == 200 and ok.json()["dry_run"] is False + assert seen == [("plan", "strict"), ("plan", "strict"), + ("execute", preview_plan, False, "strict"), ("execute", preview_plan, True, "strict")] + + def _staged_event_track(env, event, title, *, status, content_id=None, filename=None): """One event track row, with its staged file on disk when named.""" path = None diff --git a/sidecar/tests/test_event_source_reconcile.py b/sidecar/tests/test_event_source_reconcile.py new file mode 100644 index 0000000..e284765 --- /dev/null +++ b/sidecar/tests/test_event_source_reconcile.py @@ -0,0 +1,247 @@ +"""Source reconciliation (reconcile-deezer-event-tracks-by-isrc): the plan per +design decision 3 and the trash-first execution with its consent contract. + +master.db is a plain SQLite file carrying the columns the planner reads, so +a Rekordbox write made through the faked rb_write helpers is seen by the +next preview exactly like a real commit would be. +""" + +import sqlite3 +from contextlib import contextmanager +from pathlib import Path + +import pytest + +from syncbox import appdb, event_source_reconcile as reconcile +from syncbox.events_service import add_track, create_event, list_event_tracks +from syncbox.platform_os import PermanentDeleteConsentRequired +from syncbox.safety.mutate import StaleSnapshotError, fingerprint + + +class Cache: + def invalidate(self): + pass + + +@pytest.fixture +def env(tmp_path, monkeypatch): + conn = appdb.open_app_db(tmp_path / "app.db") + storage = tmp_path / "storage" + event = create_event(conn, storage, "Gig") + master = tmp_path / "master.db" + rb = sqlite3.connect(master) + rb.executescript( + "CREATE TABLE djmdContent (ID TEXT, Title TEXT, Length INTEGER, ISRC TEXT," + " FolderPath TEXT, rb_local_deleted INTEGER DEFAULT 0);" + "CREATE TABLE djmdMyTag (ID TEXT, Name TEXT, ParentID TEXT, rb_local_deleted INTEGER DEFAULT 0);" + "CREATE TABLE djmdSongMyTag (ContentID TEXT, MyTagID TEXT, rb_local_deleted INTEGER DEFAULT 0);" + "CREATE TABLE djmdSongPlaylist (ContentID TEXT, rb_local_deleted INTEGER DEFAULT 0);" + "INSERT INTO djmdMyTag VALUES ('cat', 'Situation', 'root', 0), ('T1', 'Gig', 'cat', 0)," + " ('T9', 'Peak', 'cat', 0);" + ) + rb.commit() + rb.close() + monkeypatch.setattr(reconcile, "open_readonly", lambda path: sqlite3.connect(path)) + trash = {"works": True, "calls": []} + + def fake_delete(path, *, consent_to_permanent_delete=False): + trash["calls"].append((str(path), consent_to_permanent_delete)) + if not trash["works"] and not consent_to_permanent_delete: + raise PermanentDeleteConsentRequired(Path(path), OSError("no trash")) + Path(path).unlink() + return "trashed" + + @contextmanager + def fake_mutate(db_path, backups_root, *, expected_fingerprint, invalidate_cache, **kwargs): + assert expected_fingerprint == fingerprint(db_path) + db = sqlite3.connect(db_path) + yield db + db.commit() + db.close() + invalidate_cache() + + monkeypatch.setattr(reconcile, "delete_file", fake_delete) + monkeypatch.setattr(reconcile, "mutate", fake_mutate) + monkeypatch.setattr(reconcile, "tag_content", lambda db, c, t: db.execute( + "INSERT INTO djmdSongMyTag VALUES (?, ?, 0)", (c, t))) + monkeypatch.setattr(reconcile, "untag_content", lambda db, c, t: db.execute( + "UPDATE djmdSongMyTag SET rb_local_deleted = 1 WHERE ContentID = ? AND MyTagID = ?", (c, t))) + monkeypatch.setattr(reconcile, "soft_delete_content", lambda db, c: db.execute( + "UPDATE djmdContent SET rb_local_deleted = 1 WHERE ID = ?", (c,))) + + class Env: + pass + + e = Env() + e.conn, e.storage, e.event, e.master, e.trash, e.tmp = conn, storage, event, master, trash, tmp_path + e.audio = Path(event["staging_dir"]) / "audio" + + def rb_sql(sql, *params): + db = sqlite3.connect(master) + db.execute(sql, params) + db.commit() + db.close() + + def content(content_id, isrc, path, title="Song", length=200): + rb_sql("INSERT INTO djmdContent VALUES (?, ?, ?, ?, ?, 0)", content_id, title, length, isrc, str(path)) + + def row(item_id, status, isrc, filename=None, content_id=None, path=None): + track = add_track(conn, event, resolved_source={ + "provider": "deezer", "item_id": item_id, "url": f"https://www.deezer.com/track/{item_id}", + "title": "Song", "artist": "Artist", "duration_ms": 200_000, "isrc": isrc, + }) + if filename: + path = e.audio / filename + path.write_bytes(b"download " + filename.encode()) + conn.execute("UPDATE event_tracks SET status = ?, staging_file_path = ?, content_id = ? WHERE id = ?", + (status, str(path) if path else None, content_id, track["id"])) + return track["id"] + + e.rb_sql, e.content, e.row = rb_sql, content, row + e.plan = lambda: reconcile.build_plan(conn, storage, master) + e.run = lambda plan, **kw: reconcile.execute(conn, master, tmp_path / "b", Cache(), storage, plan, **kw) + e.track = lambda track_id: next(t for t in list_event_tracks(conn, event["id"]) if t["id"] == track_id) + yield e + conn.close() + + +def _replace_case(env): + """An applied download (D2, tagged by the event) duplicating original O2.""" + env.content("O2", "FR0000000002", env.storage / "music" / "two.mp3") + track = env.row("2", "applied", "FR0000000002", "b.mp3", content_id="D2") + env.content("D2", "FR0000000002", env.audio / "b.mp3") + env.rb_sql("INSERT INTO djmdSongMyTag VALUES ('D2', 'T1', 0)") + return track + + +def test_plan_classifies_every_design_case_and_reports_coverage(env): + env.content("O1", "FR0000000001", env.storage / "music" / "one.mp3") + ready_hit = env.row("1", "ready", "fr0000000001 ", "a.mp3") + applied_hit = _replace_case(env) + env.content("O3a", "FR0000000003", env.storage / "music" / "three.mp3") + env.content("O3b", "FR0000000003", env.storage / "music" / "three (2).mp3") + ambiguous = env.row("3", "ready", "FR0000000003", "c.mp3") + no_candidate = env.row("4", "ready", "FR0000000004", "d.mp3") + outside = env.tmp / "outside.mp3" + outside.write_bytes(b"not ours") + outside_staging = env.row("5", "ready", "FR0000000001", path=outside) + env.content("O6", "FR0000000006", env.storage / "music" / "six.mp3") + tagged_elsewhere = env.row("6", "applied", "FR0000000006", "f.mp3", content_id="D6") + env.content("D6", "FR0000000006", env.audio / "f.mp3") + env.rb_sql("INSERT INTO djmdSongMyTag VALUES ('D6', 'T1', 0), ('D6', 'T9', 0)") + no_isrc = env.row("7", "ready", None, "g.mp3") + # Coverage-only material: a Deezer download Rekordbox holds without ISRC, + # and one dashed ISRC. + env.content("H", "", env.audio / "h.mp3") + env.conn.execute("INSERT INTO acquisition_jobs (provider, scope, ref, status, phase, published_path)" + " VALUES ('deezer', 'event', '1', 'downloaded', 'published', ?)", (str(env.audio / "h.mp3"),)) + env.content("X", "FR-000-0000099", env.storage / "music" / "x.mp3") + + plan = env.plan() + + items = {item["track_id"]: item for item in plan["items"]} + assert set(items) == {ready_hit, applied_hit} + assert (items[ready_hit]["action"], items[ready_hit]["content_id"]) == ("rematch", "O1") + assert items[ready_hit]["file"]["sha256"] + assert (items[applied_hit]["action"], items[applied_hit]["content_id"], + items[applied_hit]["duplicate_content_id"], items[applied_hit]["tag_id"]) == ("replace", "O2", "D2", "T1") + reasons = {entry["track_id"]: entry["reason"] for entry in plan["skipped"]} + assert reasons == { + ambiguous: "ambiguous", + no_candidate: "no_candidate", + outside_staging: "unsafe_file", + tagged_elsewhere: "duplicate_in_use", + no_isrc: "missing_isrc", + } + coverage = plan["isrc_coverage"] + assert (coverage["rows_with_isrc"], coverage["rows_without_isrc"], coverage["rows_with_candidate"]) == (6, 1, 5) + assert [entry["content_id"] for entry in coverage["downloads_without_isrc"]] == ["H"] + assert coverage["non_alphanumeric_isrcs"] == 1 + assert plan["fingerprint"] == [list(part) for part in fingerprint(env.master)] + + +def test_plan_skips_a_guard_rejected_candidate_and_a_file_held_elsewhere(env): + env.content("O1", "FR0000000001", env.storage / "music" / "one.mp3", title="Other Thing", length=500) + guarded = env.row("1", "ready", "FR0000000001", "a.mp3") + env.content("O2", "FR0000000002", env.storage / "music" / "two.mp3") + shared = env.row("2", "ready", "FR0000000002", "b.mp3") + holder = env.row("3", "missing", None) + env.conn.execute("UPDATE event_tracks SET staging_file_path = ? WHERE id = ?", (str(env.audio / "b.mp3"), holder)) + + reasons = {entry["track_id"]: entry["reason"] for entry in env.plan()["skipped"]} + + assert reasons == {guarded: "guard_rejected", shared: "file_in_use"} + + +def test_execute_rematches_and_replaces_then_trashes(env): + env.content("O1", "FR0000000001", env.storage / "music" / "one.mp3") + ready = env.row("1", "ready", "FR0000000001", "a.mp3") + applied = _replace_case(env) + + result = env.run(env.plan()) + + assert result["dry_run"] is False and result["cleanup_pending"] == [] and not result["consent_required"] + assert sorted(result["trashed_files"]) == [str(env.audio / "a.mp3"), str(env.audio / "b.mp3")] + assert not (env.audio / "a.mp3").exists() and not (env.audio / "b.mp3").exists() + row = env.track(ready) + assert (row["status"], row["content_id"], row["confidence"], row["staging_file_path"]) == ("matched", "O1", 100, None) + row = env.track(applied) + assert (row["status"], row["content_id"], row["staging_file_path"]) == ("applied", "O2", None) + db = sqlite3.connect(env.master) + assert db.execute("SELECT rb_local_deleted FROM djmdContent WHERE ID = 'D2'").fetchone() == (1,) + assert db.execute("SELECT ContentID FROM djmdSongMyTag WHERE MyTagID = 'T1' AND rb_local_deleted = 0").fetchall() == [("O2",)] + db.close() + assert env.plan()["items"] == [] + + +def test_execute_refuses_a_stale_plan(env): + env.content("O1", "FR0000000001", env.storage / "music" / "one.mp3") + ready = env.row("1", "ready", "FR0000000001", "a.mp3") + plan = env.plan() + (env.audio / "a.mp3").write_bytes(b"retagged since the preview") + + with pytest.raises(StaleSnapshotError): + env.run(plan) + + assert env.track(ready)["status"] == "ready" and env.trash["calls"] == [] + plan = env.plan() + env.content("O1b", "FR0000000001", env.storage / "music" / "one (2).mp3") + with pytest.raises(StaleSnapshotError): + env.run(plan) + + +def test_refused_trash_before_any_change_leaves_row_and_file_for_the_consent_retry(env): + env.content("O1", "FR0000000001", env.storage / "music" / "one.mp3") + ready = env.row("1", "ready", "FR0000000001", "a.mp3") + env.trash["works"] = False + plan = env.plan() + + with pytest.raises(PermanentDeleteConsentRequired): + env.run(plan) + assert env.track(ready)["status"] == "ready" and (env.audio / "a.mp3").exists() + + env.run(plan, consent_to_permanent_delete=True) + assert env.track(ready)["status"] == "matched" and not (env.audio / "a.mp3").exists() + + +def test_refused_trash_after_the_commit_is_cleanup_pending_then_retried_with_consent(env): + applied = _replace_case(env) + env.trash["works"] = False + + result = env.run(env.plan()) + + assert result["consent_required"] is True + assert result["cleanup_pending"] == [ + {"track_id": applied, "path": str(env.audio / "b.mp3"), "reason": "permanent_delete_consent"} + ] + row = env.track(applied) # repaired, and the kept file stays referenced + assert (row["content_id"], row["staging_file_path"]) == ("O2", str(env.audio / "b.mp3")) + assert (env.audio / "b.mp3").exists() + + plan = env.plan() + assert [(item["action"], item["content_id"]) for item in plan["items"]] == [("cleanup", "O2")] + with pytest.raises(PermanentDeleteConsentRequired): + env.run(plan) + env.run(plan, consent_to_permanent_delete=True) + assert env.track(applied)["staging_file_path"] is None and not (env.audio / "b.mp3").exists() + assert env.plan()["items"] == [] diff --git a/sidecar/tests/test_events_service.py b/sidecar/tests/test_events_service.py index db48c62..e0a625f 100644 --- a/sidecar/tests/test_events_service.py +++ b/sidecar/tests/test_events_service.py @@ -256,6 +256,51 @@ def test_match_event_tracks_retries_acquisition_failure(conn, tmp_path): assert list_event_tracks(conn, event["id"])[0]["status"] == "matched" +def _deezer(conn, event, item_id, title, isrc): + return add_track(conn, event, resolved_source={ + "provider": "deezer", "item_id": item_id, "url": f"https://www.deezer.com/track/{item_id}", + "title": title, "artist": "Artist", "duration_ms": 200_000, "isrc": isrc, + }) + + +def test_exact_source_matches_by_isrc_never_by_fuzzy(conn, tmp_path): + storage = tmp_path / "storage" + event = create_event(conn, storage, "Deezer Night") + isrc_hit = _deezer(conn, event, "1", "Song", "fraaa0000001 ") + fuzzy_only = _deezer(conn, event, "2", "Song", None) + remix = _deezer(conn, event, "3", "Song (Remix)", "FRBBB0000002") + proven = _deezer(conn, event, "4", "Song", "FRAAA0000001") + staged = Path(event["staging_dir"]) / "audio" / "proven.mp3" + staged.write_bytes(b"provenance bytes") + conn.execute( + "INSERT INTO acquisition_jobs (provider, scope, ref, status, phase, effective_source_item_id," + " published_path, published_sha256) VALUES ('deezer', 'event', ?, 'downloaded', 'published', '4', ?, ?)", + (str(proven["id"]), str(staged), _sha256(staged)), + ) + cache = FakeCache([{"content_id": "orig", "title": "Song", "artist": "Artist", + "duration_ms": 200_000, "isrc": "FRAAA0000001", "file_path": "/music/song.mp3"}]) + + match_event_tracks(conn, event, cache, storage) + rows = {t["id"]: t for t in list_event_tracks(conn, event["id"])} + + assert (rows[isrc_hit["id"]]["status"], rows[isrc_hit["id"]]["content_id"], + rows[isrc_hit["id"]]["confidence"]) == ("matched", "orig", 100) + assert rows[fuzzy_only["id"]]["status"] == "missing" + assert rows[remix["id"]]["status"] == "missing" + assert (rows[proven["id"]]["status"], rows[proven["id"]]["staging_file_path"], + rows[proven["id"]]["content_id"]) == ("ready", str(staged.resolve()), None) + + +def test_exact_source_isrc_match_respects_the_duration_guard(conn, tmp_path): + storage = tmp_path / "storage" + event = create_event(conn, storage, "Guarded") + _deezer(conn, event, "1", "Song", "FRAAA0000001") + cache = FakeCache([{"content_id": "other", "title": "Totally Different", "artist": "X", + "duration_ms": 400_000, "isrc": "FRAAA0000001", "file_path": "/m.mp3"}]) + match_event_tracks(conn, event, cache, storage) + assert list_event_tracks(conn, event["id"])[0]["status"] == "missing" + + # --- staging claims (5.7 claim rule) ----------------------------------------------- @@ -1002,6 +1047,36 @@ def fake_mutate( assert stale["id"] in missing_ids +def test_apply_reverifies_isrc_for_exact_source_matches(conn, tmp_path, monkeypatch): + storage = tmp_path / "storage" + event = create_event(conn, storage, "Apply ISRC") + kept = _deezer(conn, event, "1", "Kept", "FRAAA0000001") + lost = _deezer(conn, event, "2", "Lost", "FRAAA0000002") + for track, content_id in ((kept, "C1"), (lost, "C2")): + conn.execute("UPDATE event_tracks SET status = 'matched', content_id = ?, confidence = 100" + " WHERE id = ?", (content_id, track["id"])) + + @contextmanager + def fake_mutate(db_path, backups_root, **kwargs): + yield "db" + + _fake_apply_helpers(monkeypatch, fake_mutate) + tagged = [] + monkeypatch.setattr(events_service, "tag_content", lambda db, c, t: tagged.append(c)) + cache = FakeCache([ + {"content_id": "C1", "isrc": "fraaa0000001", "file_path": "/music/kept.mp3"}, + {"content_id": "C2", "isrc": "", "file_path": "/music/lost.mp3"}, # retagged in Rekordbox + ]) + + result = apply_event(conn, tmp_path / "master.db", tmp_path / "b", cache, storage, event) + + assert tagged == ["C1"] + assert result["applied"] == 1 and result["reclassified_missing"] == [lost["id"]] + rows = {t["id"]: t for t in list_event_tracks(conn, event["id"])} + assert rows[kept["id"]]["status"] == "applied" + assert (rows[lost["id"]]["status"], rows[lost["id"]]["content_id"]) == ("missing", None) + + def test_delete_event_forwards_exact_plan_and_consent(conn, tmp_path, monkeypatch): event = create_event(conn, tmp_path / "storage", "Forward") plan = {"plan_version": 1, "event_id": event["id"]} diff --git a/sidecar/tests/test_link_imports.py b/sidecar/tests/test_link_imports.py index c3ea027..23b3d1f 100644 --- a/sidecar/tests/test_link_imports.py +++ b/sidecar/tests/test_link_imports.py @@ -128,7 +128,8 @@ def test_direct_source_never_claims_similar_audio_without_identity(conn, tmp_pat audio = tmp_path / "Similar original.mp3" audio.write_bytes(b"fixture original") track = events_service.add_track(conn, event, resolved_source={"provider": "deezer", "item_id": "101", "url": "https://www.deezer.com/track/101", "title": "Similar original", "isrc": "GBXXX1234567"}) - cache = SimpleNamespace(get=lambda _: [{"content_id": "999", "file_path": str(audio), "title": track["title"], "isrc": track["isrc"]}]) + # Same title, no ISRC on the collection side: only provenance may associate it. + cache = SimpleNamespace(get=lambda _: [{"content_id": "999", "file_path": str(audio), "title": track["title"], "isrc": None}]) assert events_service.match_event_tracks(conn, event, cache, tmp_path)[0]["status"] == "missing" assert events_service.claim_staged_files(conn, event) == [] conn.execute("UPDATE event_tracks SET status = 'ready', staging_file_path = ? WHERE id = ?", (str(audio), track["id"])) diff --git a/sidecar/tests/test_reproducible_archive.py b/sidecar/tests/test_reproducible_archive.py index d818b0a..64a458e 100644 --- a/sidecar/tests/test_reproducible_archive.py +++ b/sidecar/tests/test_reproducible_archive.py @@ -127,7 +127,7 @@ def _fixture(root: Path, mtime: int) -> Path: def test_tree_archives_are_identical_across_roots_and_metadata(monkeypatch, tmp_path): - monkeypatch.setenv("SOURCE_DATE_EPOCH", "1788739200") + monkeypatch.setenv("SOURCE_DATE_EPOCH", "1791504000") first_tree = _fixture(tmp_path / "first-root", 1_700_000_000) second_tree = _fixture(tmp_path / "second-root-with-a-longer-name", 1_750_000_000) first_archive = tmp_path / "first.zip" @@ -149,7 +149,7 @@ def test_tree_archives_are_identical_across_roots_and_metadata(monkeypatch, tmp_ "payload/empty/", "payload/link", ] - assert {info.date_time for info in infos} == {(2026, 9, 7, 0, 0, 0)} + assert {info.date_time for info in infos} == {(2026, 10, 9, 0, 0, 0)} modes = { info.filename: stat.S_IMODE(info.external_attr >> 16) for info in infos } @@ -217,11 +217,11 @@ def test_release_environment_uses_stable_source_metadata(monkeypatch): monkeypatch.setenv(name, value) metadata = json.loads((REPO / "release-build.json").read_text()) assert metadata["schema"] == 1 - assert metadata["source_date_epoch"] == 1788739200 + assert metadata["source_date_epoch"] == 1791504000 assert metadata["release"] == { "source_date_epoch_basis": "versioned-release-metadata", - "source_date_epoch_utc": "2026-09-07T00:00:00Z", - "version": "0.9.0", + "source_date_epoch_utc": "2026-10-09T00:00:00Z", + "version": "0.10.0", } assert metadata["toolchain"]["rustc"].startswith("rustc 1.96.1 ") assert metadata["toolchain"]["node"] == "v24.13.0" @@ -230,7 +230,7 @@ def test_release_environment_uses_stable_source_metadata(monkeypatch): ) environment = _load_release_builder()._release_environment() - assert environment["SOURCE_DATE_EPOCH"] == "1788739200" + assert environment["SOURCE_DATE_EPOCH"] == "1791504000" assert environment["PYTHONDONTWRITEBYTECODE"] == "1" assert environment["PYTHONHASHSEED"] == "0" assert environment["PYTHONNOUSERSITE"] == "1" @@ -424,7 +424,7 @@ def test_architecture_gate_accepts_only_arm64(monkeypatch): def test_artifact_comparator_reports_entry_level_mismatch(monkeypatch, tmp_path): - monkeypatch.setenv("SOURCE_DATE_EPOCH", "1788739200") + monkeypatch.setenv("SOURCE_DATE_EPOCH", "1791504000") comparator = _load_artifact_comparator() first_tree = _fixture(tmp_path / "first", 1_700_000_000) second_tree = _fixture(tmp_path / "second", 1_700_000_000) diff --git a/sidecar/uv.lock b/sidecar/uv.lock index a88b6ba..aeabf6e 100644 --- a/sidecar/uv.lock +++ b/sidecar/uv.lock @@ -530,7 +530,7 @@ wheels = [ [[package]] name = "syncbox" -version = "0.9.0" +version = "0.10.0" source = { virtual = "." } dependencies = [ { name = "certifi" }, diff --git a/ui/package.json b/ui/package.json index d31c7d3..cc472ba 100644 --- a/ui/package.json +++ b/ui/package.json @@ -1,7 +1,7 @@ { "name": "syncbox-ui", "private": true, - "version": "0.9.0", + "version": "0.10.0", "packageManager": "pnpm@10.29.3", "engines": { "node": "24.13.0", diff --git a/ui/src/api/types.ts b/ui/src/api/types.ts index f47c4ac..8ee4ba6 100644 --- a/ui/src/api/types.ts +++ b/ui/src/api/types.ts @@ -353,6 +353,45 @@ export interface PerformanceLive { tracks: PerformanceTrack[] } +/** GET/POST /api/events/source-reconciliation: exact-source downloads that + duplicate an ISRC-equal collection content. Execute echoes the plan. */ +export interface SourceReconciliationRow { + track_id: number + event_id: number + event_name: string + title: string | null + artist: string | null + isrc: string | null + status: string +} +export interface SourceReconciliationPlan { + dry_run: boolean + plan_version: number + fingerprint: unknown + items: Array< + SourceReconciliationRow & { + action: 'rematch' | 'replace' | 'cleanup' + content_id: string + duplicate_content_id: string | null + tag_id: string | null + file: { path: string } + } + > + skipped: Array + isrc_coverage: { + rows_with_isrc: number + rows_without_isrc: number + rows_with_candidate: number + downloads_without_isrc: Array<{ content_id: string; title: string | null; path: string }> + non_alphanumeric_isrcs: number + } +} +export interface SourceReconciliationResult extends SourceReconciliationPlan { + trashed_files: string[] + cleanup_pending: Array<{ track_id: number; path: string; reason: string }> + consent_required: boolean +} + export interface BackupInfo { name: string files: string[] diff --git a/ui/src/i18n/en.ts b/ui/src/i18n/en.ts index 5d7fa8a..ff3f428 100644 --- a/ui/src/i18n/en.ts +++ b/ui/src/i18n/en.ts @@ -755,6 +755,7 @@ export const en = { untagged_remove: 'Untagged cleanup', smart_fixes: 'Smart fixes', acquisition_storage_migration: 'Download storage migration', + event_source_reconciliation: 'Event download reconciliation', legacy_metadata_backfill: 'Legacy metadata repair', pre_restore: 'Before restore', legacy: 'Unknown operation', @@ -789,6 +790,38 @@ export const en = { unverified_destination: 'The published destination is not referenced by its owner.', }, }, + reconcile: { + title: 'Event downloads already in your collection', + lead: 'Deezer, YouTube and SoundCloud downloads whose recording (same ISRC) is already in Rekordbox: the event uses the existing track and the download goes to the trash.', + confirm: 'Reconcile {n} track | Reconcile {n} tracks', + done: '{n} download moved to the trash | {n} downloads moved to the trash', + kept: '{n} file kept, run again to remove it | {n} files kept, run again to remove them', + rematch: '{n} download not imported yet | {n} downloads not imported yet', + replace: '{n} duplicate in Rekordbox | {n} duplicates in Rekordbox', + cleanup: '{n} leftover download | {n} leftover downloads', + coverage: 'ISRC: {with} tracks with, {without} without, {found} found in the collection', + downloadsWithoutIsrc: + '{n} Deezer download without ISRC in Rekordbox | {n} Deezer downloads without ISRC in Rekordbox', + oddIsrcs: '{n} ISRC with separators | {n} ISRCs with separators', + readyDetails: 'Tracks ready to reconcile', + downloadsWithoutIsrcDetails: 'Deezer downloads ISRC matching cannot see', + action: { + rematch: 'use the collection track', + replace: 'replace the duplicate with the collection track', + cleanup: 'move the leftover download to the trash', + }, + reason: { + missing_isrc: 'No ISRC proves the recording.', + unsafe_file: 'The download is missing, symbolic or outside the event folder.', + file_in_use: 'The download is also used by another track or Rekordbox entry.', + not_a_download: 'The Rekordbox entry is not this download.', + not_tagged: 'The Rekordbox entry no longer carries the event tag.', + duplicate_in_use: 'The duplicate has other tags, playlists or events.', + no_candidate: 'No collection track carries this ISRC.', + ambiguous: 'Several collection tracks carry this ISRC.', + guard_rejected: 'The collection track with this ISRC differs too much in duration and title.', + }, + }, empty: 'No backup yet — one is created before every write.', restoreNote: 'Each new restore point contains Rekordbox and Syncbox state. Restore snapshots the current state first and requires Rekordbox closed.', diff --git a/ui/src/i18n/fr.ts b/ui/src/i18n/fr.ts index 98b785f..a52078f 100644 --- a/ui/src/i18n/fr.ts +++ b/ui/src/i18n/fr.ts @@ -773,6 +773,7 @@ export const fr: typeof en = { untagged_remove: 'Nettoyage des sans-tag', smart_fixes: 'Smart fixes', acquisition_storage_migration: 'Migration du stockage des téléchargements', + event_source_reconciliation: 'Réconciliation des téléchargements d’événement', legacy_metadata_backfill: 'Réparation des métadonnées héritées', pre_restore: 'Avant restauration', legacy: 'Opération inconnue', @@ -807,6 +808,38 @@ export const fr: typeof en = { unverified_destination: 'La destination publiée n’est pas référencée par son propriétaire.', }, }, + reconcile: { + title: 'Téléchargements d’événement déjà dans votre collection', + lead: 'Téléchargements Deezer, YouTube et SoundCloud dont l’enregistrement (même ISRC) est déjà dans Rekordbox : l’événement utilise le titre existant et le téléchargement part à la corbeille.', + confirm: 'Réconcilier {n} titre | Réconcilier {n} titres', + done: '{n} téléchargement mis à la corbeille | {n} téléchargements mis à la corbeille', + kept: '{n} fichier conservé, relancez pour le supprimer | {n} fichiers conservés, relancez pour les supprimer', + rematch: '{n} téléchargement pas encore importé | {n} téléchargements pas encore importés', + replace: '{n} doublon dans Rekordbox | {n} doublons dans Rekordbox', + cleanup: '{n} téléchargement résiduel | {n} téléchargements résiduels', + coverage: 'ISRC : {with} titres avec, {without} sans, {found} trouvés dans la collection', + downloadsWithoutIsrc: + '{n} téléchargement Deezer sans ISRC dans Rekordbox | {n} téléchargements Deezer sans ISRC dans Rekordbox', + oddIsrcs: '{n} ISRC avec séparateurs | {n} ISRC avec séparateurs', + readyDetails: 'Titres prêts à réconcilier', + downloadsWithoutIsrcDetails: 'Téléchargements Deezer invisibles pour la correspondance ISRC', + action: { + rematch: 'utiliser le titre de la collection', + replace: 'remplacer le doublon par le titre de la collection', + cleanup: 'mettre le téléchargement résiduel à la corbeille', + }, + reason: { + missing_isrc: 'Aucun ISRC ne prouve l’enregistrement.', + unsafe_file: 'Le téléchargement est absent, symbolique ou hors du dossier de l’événement.', + file_in_use: 'Le téléchargement sert aussi à un autre titre ou à une autre entrée Rekordbox.', + not_a_download: 'L’entrée Rekordbox n’est pas ce téléchargement.', + not_tagged: 'L’entrée Rekordbox ne porte plus le tag de l’événement.', + duplicate_in_use: 'Le doublon a d’autres tags, playlists ou événements.', + no_candidate: 'Aucun titre de la collection ne porte cet ISRC.', + ambiguous: 'Plusieurs titres de la collection portent cet ISRC.', + guard_rejected: 'Le titre de la collection avec cet ISRC diffère trop en durée et en titre.', + }, + }, empty: 'Aucun backup pour l’instant — un backup est créé avant chaque écriture.', restoreNote: 'Chaque nouveau point contient l’état Rekordbox et Syncbox. Restaurer sauvegarde d’abord l’état courant et requiert Rekordbox fermé.', diff --git a/ui/src/screens/__tests__/health-tabs.spec.ts b/ui/src/screens/__tests__/health-tabs.spec.ts index 91cfbd0..877bf66 100644 --- a/ui/src/screens/__tests__/health-tabs.spec.ts +++ b/ui/src/screens/__tests__/health-tabs.spec.ts @@ -2,6 +2,7 @@ import { flushPromises, mount } from '@vue/test-utils' import { createPinia, setActivePinia } from 'pinia' import { afterEach, beforeEach, expect, test, vi } from 'vitest' +import { setConsentBroker } from '../../api/client' import { i18n } from '../../i18n' import { router } from '../../router' import { useHealthStore } from '../../stores/health' @@ -188,6 +189,101 @@ test('a cleanup-only storage plan is visible and executable', async () => { ) }) +test('source reconciliation previews, executes the exact plan and retries after consent', async () => { + const path = '/storage/_syncbox/events/gig/audio/take-on-me.flac' + const plan = { + dry_run: true, + plan_version: 1, + fingerprint: [['1', '2']], + items: [ + { + track_id: 4, + event_id: 1, + event_name: 'Thibault & Alix', + title: 'Take on Me', + artist: 'a-ha', + isrc: 'NOA000000001', + status: 'ready', + action: 'rematch', + content_id: '77', + duplicate_content_id: null, + tag_id: null, + file: { path }, + }, + ], + skipped: [ + { + track_id: 5, + event_id: 1, + event_name: 'Thibault & Alix', + title: 'Brand New', + artist: 'X', + isrc: 'FR0000000005', + status: 'ready', + reason: 'no_candidate', + }, + ], + isrc_coverage: { + rows_with_isrc: 2, + rows_without_isrc: 0, + rows_with_candidate: 1, + downloads_without_isrc: [], + non_alphanumeric_isrcs: 0, + }, + } + const posts: Array> = [] + const fetchMock = vi.fn().mockImplementation((url: string, init?: RequestInit) => { + const route = new URL(url).pathname + if (route === '/api/events/source-reconciliation' && init?.method === 'POST') { + const body = JSON.parse(init.body as string) + posts.push(body) + if (!body.consent_to_permanent_delete) { + const refusal = { error: 'consent_required', consent: 'permanent_delete', message: 'no trash', path } + return Promise.resolve({ ok: false, status: 428, json: () => Promise.resolve(refusal) }) + } + const done = { ...plan, dry_run: false, trashed_files: [path], cleanup_pending: [], consent_required: false } + return Promise.resolve({ ok: true, status: 200, json: () => Promise.resolve(done) }) + } + const payload = + route === '/api/events/source-reconciliation' + ? plan + : route === '/api/acquisition/storage-migration' + ? { items: [], cleanup_directories: [], ignored: [] } + : route === '/api/doctor/backups' + ? { backups: [] } + : { configured: false, lines: [] } + return Promise.resolve({ ok: true, status: 200, json: () => Promise.resolve(payload) }) + }) + vi.stubGlobal('fetch', fetchMock) + const broker = vi.fn().mockResolvedValue(true) + setConsentBroker(broker) + + try { + const wrapper = mountBackups() + await flushPromises() + + const block = wrapper.get('.card.reconcile') + expect(block.text()).toContain('1 téléchargement pas encore importé') + expect(block.text()).toContain('ISRC : 2 titres avec, 0 sans, 1 trouvés dans la collection') + expect(block.text()).toContain('Take on Me') + expect(block.text()).toContain('Aucun titre de la collection ne porte cet ISRC.') + + await block.get('.restore').trigger('click') + await flushPromises() + + expect(broker).toHaveBeenCalledOnce() + expect(posts).toEqual([ + { dry_run: false, plan }, + { dry_run: false, plan, consent_to_permanent_delete: true }, + ]) + expect(wrapper.get('.banner[data-tone="success"]').text()).toContain( + '1 téléchargement mis à la corbeille', + ) + } finally { + setConsentBroker(null) + } +}) + test('B1: a failed scan click surfaces the backend message — never a silent no-op', async () => { stubScan({ fail: true }) const wrapper = mountTab(DuplicatesTab) diff --git a/ui/src/screens/health/BackupsTab.vue b/ui/src/screens/health/BackupsTab.vue index 94ed1e8..57b1958 100644 --- a/ui/src/screens/health/BackupsTab.vue +++ b/ui/src/screens/health/BackupsTab.vue @@ -6,7 +6,11 @@ import { ref } from 'vue' import { useI18n } from 'vue-i18n' import { ApiError, api } from '../../api/client' -import type { BackupInfo } from '../../api/types' +import type { + BackupInfo, + SourceReconciliationPlan, + SourceReconciliationResult, +} from '../../api/types' import LoadingState from '../../components/LoadingState.vue' import { useRefreshOnReturn } from '../../lib/refresh' import { revealInFolder } from '../../shell' @@ -49,6 +53,8 @@ interface StorageMigrationPlan { } const migration = ref(null) const migrationBusy = ref(false) +const reconciliation = ref(null) +const reconcileBusy = ref(false) const logs = ref<{ configured: boolean path?: string @@ -74,6 +80,13 @@ async function load() { } catch { migration.value = null } + try { + reconciliation.value = await api.get( + '/api/events/source-reconciliation', + ) + } catch { + reconciliation.value = null + } try { logs.value = await api.get<{ configured: boolean @@ -156,6 +169,37 @@ async function migrateStorage() { } } +// A 428 (trash unavailable) goes through the client's consent loop and is +// re-sent with the same plan; a refusal after the Rekordbox commit comes back +// as cleanup_pending and the reloaded preview offers it as a cleanup item. +async function reconcile() { + if (!reconciliation.value?.items.length) return + reconcileBusy.value = true + banner.value = null + try { + const result = await api.post( + '/api/events/source-reconciliation', + { dry_run: false, plan: reconciliation.value }, + ) + const kept = result.cleanup_pending.length + banner.value = { + tone: 'success', + text: + t('backups.reconcile.done', result.trashed_files.length) + + (kept ? ` · ${t('backups.reconcile.kept', kept)}` : ''), + } + await load() + } catch (cause) { + banner.value = { tone: 'error', text: describe(cause) } + } finally { + reconcileBusy.value = false + } +} + +function reconcileCount(action: 'rematch' | 'replace' | 'cleanup'): number { + return reconciliation.value?.items.filter((item) => item.action === action).length ?? 0 +} + function sizeLabel(bytes: number): string { if (bytes >= 1_000_000_000) return `${(bytes / 1_000_000_000).toFixed(1)} GB` if (bytes >= 1_000_000) return `${(bytes / 1_000_000).toFixed(1)} MB` @@ -182,6 +226,7 @@ function reasonLabel(reason?: string | null): string { 'untagged_remove', 'smart_fixes', 'acquisition_storage_migration', + 'event_source_reconciliation', 'legacy_metadata_backfill', 'pre_restore', ]) @@ -283,6 +328,105 @@ function reasonLabel(reason?: string | null): string { +
+
+
+

{{ t('backups.reconcile.title') }}

+

{{ t('backups.reconcile.lead') }}

+
+ +
+
+ {{ t('backups.reconcile.rematch', reconcileCount('rematch')) }} + {{ t('backups.reconcile.replace', reconcileCount('replace')) }} + + {{ t('backups.reconcile.cleanup', reconcileCount('cleanup')) }} + + + {{ t('backups.migration.ignored', reconciliation.skipped.length) }} + +
+
+ {{ + t('backups.reconcile.coverage', { + with: reconciliation.isrc_coverage.rows_with_isrc, + without: reconciliation.isrc_coverage.rows_without_isrc, + found: reconciliation.isrc_coverage.rows_with_candidate, + }) + }} + + {{ + t( + 'backups.reconcile.downloadsWithoutIsrc', + reconciliation.isrc_coverage.downloads_without_isrc.length, + ) + }} + + + {{ + t( + 'backups.reconcile.oddIsrcs', + reconciliation.isrc_coverage.non_alphanumeric_isrcs, + ) + }} + +
+
+ {{ t('backups.reconcile.readyDetails') }} +
+ {{ item.title || t('missing.untitled') }} — {{ item.artist || '—' }} · + {{ item.event_name }} · {{ t(`backups.reconcile.action.${item.action}`) }} +
{{ item.file.path }}
+
+
+
+ {{ t('backups.reconcile.downloadsWithoutIsrcDetails') }} +
+ {{ item.title || t('missing.untitled') }} +
{{ item.path }}
+
+
+
+ {{ t('backups.migration.ignoredDetails') }} +
+ {{ item.title || t('missing.untitled') }} — {{ item.event_name }} · + {{ t(`backups.reconcile.reason.${item.reason}`) }} +
+
+

{{ t('backups.title') }}

diff --git a/web-audio-component/pyproject.toml b/web-audio-component/pyproject.toml index e7262f8..cd10264 100644 --- a/web-audio-component/pyproject.toml +++ b/web-audio-component/pyproject.toml @@ -1,6 +1,6 @@ [project] name = "syncbox-web-audio-component" -version = "0.9.0" +version = "0.10.0" description = "Isolated YouTube and SoundCloud audio component for Syncbox" license = "MIT" requires-python = ">=3.13,<3.14" diff --git a/web-audio-component/uv.lock b/web-audio-component/uv.lock index c0bd034..052acca 100644 --- a/web-audio-component/uv.lock +++ b/web-audio-component/uv.lock @@ -339,7 +339,7 @@ wheels = [ [[package]] name = "syncbox-web-audio-component" -version = "0.9.0" +version = "0.10.0" source = { virtual = "." } dependencies = [ { name = "yt-dlp", extra = ["default"] },