feature: calibre root folder - #132
Open
benjitobz wants to merge 5 commits into
Open
Conversation
A root folder can be backed by a Calibre content server. Books imported into it are added through calibre rather than copied into place, calibre decides where the files live, and Chaptarr follows that naming instead of imposing its own. Deletions propagate: removing a book or an author removes the matching calibre records, and a scan reaps duplicate records left behind by earlier imports. Both are opt-in per root folder, off by default. Chaptarr no longer reads metadata back out of the calibre library. The library is a destination, not a source.
Add an action that writes Chaptarr's metadata into the calibre record behind a book, over one book or a selection, choosing which fields to write and previewing what each will carry. Title and author decide the folder calibre files a book under, so pushing them moves the files. That is left unticked by default, warned about when ticked, and the tracked paths follow the move rather than the next scan reading the vanished folder as a deletion. A field with nothing to write is skipped rather than sent empty, since Calibre-Web fills metadata in on ingest and a blank would erase it. The cover is reconciled against the monitored edition before it is read, covers are sent once their deferred download completes, and the connections are told when a push changes a book. The book page also shows a loading state instead of rendering against a book that is not in the store yet.
Refresh the stored cover when the monitored edition changes, restore proxied cover urls before persisting or downloading, and only auto-push a book's cover during its import window so later cover churn cannot overwrite artwork set in the library.
Only write fields that actually differ and remember writes the server refuses to persist, reuse existing calibre records for files already in the library, recognize series-branded titles as reapable duplicates and merge same-book records safely, publish AuthorScannedEvent for granular scans, keep book thumbnails in step with the pushed cover, cool down full-library path enumeration, and trim comments to the load-bearing whys.
3 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Lets a root folder be backed by a Calibre content server. Books imported into it are added through calibre rather than copied into place, calibre decides where the files live, and Chaptarr follows that naming instead of imposing its own. Deletions propagate — removing a book or author removes the matching calibre records — and a scan can reap duplicate records left by earlier imports. Both are opt-in per root folder and off by default. Chaptarr does not read metadata back out of the calibre library; the library is a destination, not a source.
The second commit adds an action to write Chaptarr's metadata into the calibre record behind a book, over one book or a selection, choosing which fields to write with a preview of what each will carry. Title and author decide the folder calibre files a book under, so pushing them moves the files on disk — that is unticked by default, warned about when ticked, and the tracked paths follow the move rather than letting the next disk scan read the vanished folder as a deletion. A field with nothing to write is skipped rather than sent empty, since Calibre-Web fills metadata in on ingest and a blank value erases it. Covers are sent once their deferred download completes, and reconciled against the monitored edition before being read, so a push cannot ship artwork the page doesn't show. This commit also adds a loading state to the book page, which previously crashed when rendered before its book was in the store.
The third commit fixes four faults that let a book's stored cover drift from its selected edition: proxy cover links were persisted over the real provider URLs on a client round-trip, the url-to-path cache resurrected another edition's artwork on a switch back, and neither a refresh that moved the monitored edition nor the file matching that pins an edition after import ever revisited the cover. The stored file now follows the monitored edition through all of those paths, with a post-scan sweep catching anything that slips through.
I have a few CWA PR's to draft but they should be up shortly to allow folks to use this if they want.
Related External PRs
CWA: crocodilestick/Calibre-Web-Automated#1530
CWA-NG: new-usemame/Calibre-Web-NextGen#2210
Fixes # (if applicable) - N/A
Database Migration
YES - RootFolders. Migration 108 adds two boolean columns, ReapCalibreDuplicates and AutoPushCalibreMetadata, both defaulting to false and guarded by existence checks so re-running is safe. (Numbered 108 to sit clear of develop's 104–107 but can be changed as needed)
How was this tested?
Docker on Ubuntu against Calibre-Web Automated (calibre content server) with a root folder pointed at the calibre library, plus AudioBookShelf and a separate mirror content server as connections.
Verified:
Note: Delays between the systems is to be expected. Chaptarr needs to scan and pickup the changes for some of the scenarios to take place.
Book uploaded through Calibre-Web, picked up by scan
Upload an epub into CWA's ingest; wait for a Chaptarr scan.
Expected: the file is adopted and mapped to the right book/edition, its calibre record id is stored on the file row, and only title/author are pushed (re-filing the folder to proper naming) — no other metadata in CWA changes. Verified with A Court of Thorns and Roses, Spell or High Water, The Two Towers, The Fellowship of the Ring.
Book uploaded through Calibre-Web, not picked up by scan - goes to unmapped files
Expected: the files are displayed in the unmapped files page of Chaptarr. Once manually fixed the rest of the previous scenario applies.
Auto-push gate (AutoPushCalibreMetadata)
Same pickup with the root-folder toggle on.
Expected: the pickup pushes all fields instead of just identity. Off by default; verified inert when off.
Import through Chaptarr (download path)
Expected: the file is added through calibre (AddAndConvert), calibre chooses the folder, identity is pushed, and the stored path is read back from calibre — including when the identity push moves the folder mid-import.
Field selection dialog
Open Calibre Push on a book page.
Expected: eleven selectable fields with per-field value previews; Title and Author unticked by default; ticking either shows the moves-files warning; the /books bulk dialog is identical minus previews, plus the how-to-preview tip.
Push writes only what it has
Push all fields on a book with no description/rating/tags in Chaptarr.
Expected: empty fields are skipped, not sent — CWA's ingest-set values survive. Before this guard, tags went as [] (clearing them), rating as 0, language as [""]. The log lists the fields actually written; skipped ones log at debug.
Identity push re-files the folder, Chaptarr follows
Pin an edition whose title differs only in capitalisation, push title/author.
Expected: calibre moves the folder (case-only rename = new directory on Linux); the tracked file paths update in place. A full RescanFolders afterwards leaves the book attached and fires no delete cascade. This is the exact scenario that previously unlinked the book and propagated deletions to every connection.
Bulk push over a selection
Select several books on /books, push.
Expected: one command carries all ids; each book pushes independently.
Explicit edition switch - Cover should match what is shown in Chaptarr
Change the monitored edition on the book page.
Expected: the stored cover re-downloads for the new edition. A client round-trip no longer persists /MediaCoverProxy/… links over the real provider URLs (verified: 14 proxied editions submitted, database stayed absolute).
Refresh moves the monitored edition - Cover should match what is shown in Chaptarr
Expected: the cover reconciles as part of the refresh.
Book with embedded cover is uploaded from CWA
Expected: Chaptarr will attempt to map the book, Update the path if needed and update CWA / Calibre accordingly to reference the new location. If enabled Chaptarr will swap the metadata and use the cover that is shown in Chaptarr.
Delete with files, book and author
Expected: the matching calibre records are removed. Gated on the Delete Files flag actually being set and on ebook media type; a delete without files touches calibre not at all.
Duplicate reaper (ReapCalibreDuplicates)
Scan with the toggle on, duplicates present.
Expected: redundant calibre records whose formats are a subset of the canonical record are removed (verified live: "Removed 1 duplicate calibre record(s) for canonical book 161: 167"). Off by default.
Rename/organize disabled for calibre roots
Preview Rename / organize on a calibre-managed root folder.
Expected: no-op - calibre owns naming there.
Propagating CWA changes to the connector.
Expected: Chaptarr will detect the changes and propagate the changes to the connector such as the calibre content server connector. Nothing will be written to Chaptarrs DB. Future state being ABS would also get the changes - another PR
Screenshots (UI changes only) - Adds a few:
Path: /books
Scenario: Calibre Root Folder and Calibre connector Enabled after clicking 'Book Editor'
Path: /books
Scenario: Calibre Root Folder Enabled after clicking 'Book Editor'
Path: /book/{id}
Scenario: Calibre Root Folder Enabled - Repushes all the selected metadata fields selected in the dialog into Calibre, updates the root folder and the connector if present. Second screenshot is of the dialog - Title and Author off by default since pushing those will move the books folder and linked applications won't follow it with manually repointing their path, warning shown if the fields are ticked.
Path: /settings/mediamanagement
Scenario: Adding a calibre managed root folder - only shows the calibre settings, all other setting from the root folder dialog remain the same - two additional checkboxes have been added to the calibre section, one helps with potential duplicates caused by calibre and one to allow Chaptarr to use it's metadata on initial push.
A note on AI: We know AI/agentic coding is everywhere and only getting
more popular. We won't insist that you disclose whether you used it or which
models you used, but in the same spirit, please don't take offense if your PR
is scrutinized and changes are requested.
Review time: The longer the PR and the more lines changed, the longer the
review will take. Small, focused PRs merge fastest. If yours is big, please be
patient.