feature: grimmory connector - #136
Open
benjitobz wants to merge 19 commits into
Open
Conversation
Queues affected Grimmory libraries on import, rename, delete and retag events and drains them via ProcessQueue after files land on disk, replacing the archived branch's in-handler sleep and all-libraries sync task. Book deletes now publish DeleteCompletedEvent after disk cleanup so queued notification work (Grimmory, Plex) drains at the right time.
Two connector toggles push Chaptarr's metadata (locking pushed fields) and cover to the matched Grimmory book on import, retag, and cover updates. A Grimmory Push toolbar button on book details and a bulk button in the book editor open a field-picker dialog driving the PushGrimmoryMetadata command; both disable without files on disk. A poller forwards edits made in Grimmory (audit log for metadata, cover stamps for covers, own-user echoes excluded) to any connection implementing the new IExternalLibraryEditTarget seam, keeping this branch mergeable without the ABS/CCS branches.
Grimmory rewrites <book>.metadata.json (and .cover.jpg) next to the book on every edit when sidecar write-on-update is enabled, so the forwarder now watches the root folders for those writes - the same mechanism the calibre forwarder uses on metadata.db - replacing the 2-minute audit-log poll and its admin requirement. Chaptarr's own pushes are filtered through a recent-push registry rather than by username, so edits made in Grimmory under the connection's account are forwarded too.
A push makes Grimmory rewrite the sidecar exactly once, so the suppression entry is spent on the first matching sidecar event; a person's edit made minutes later - previously discarded for the whole 10-minute window - forwards again.
Cover-only pushes leave no sidecar echo to consume, so recording them could swallow the next real edit made in Grimmory.
A push's echo and a person's edit made inside the same debounce window coalesced into one batch entry and were discarded together. Each filesystem event is now checked on arrival: the first event after a push consumes the entry, duplicates inside a short shadow are absorbed, and anything later is queued and forwarded.
The mount guard skipped cleanup whenever a scan found no media files, so a granular scan of a book folder whose only file was deleted externally (e.g. removed in Grimmory) never pruned the tracked rows and the deletion never reached connections. An empty subfolder scan now cleans up when the root folder itself provably has content; root-level empty scans keep the guard.
Grimmory never updates a locked field, even for the writer who locked it, and applies a request's values before its lock flags - so the first pushed value froze forever and corrected re-pushes silently no-oped. Each push now clears its target locks via toggle-field-locks first; the update itself re-locks them. Publish date also prefers the monitored edition's release date over the book's.
Reverts the unlock-before-write pass: a locked field in Grimmory now stays exactly as locked, including against Chaptarr's own re-pushes. Updating a locked value means unlocking it in Grimmory first. The monitored-edition release date preference stays.
Restores the archived quickstart section; the card drives the standard notification modal off the schema, so it picks up the current library dropdowns and push/forward toggles as-is.
Grimmory now implements IExternalLibraryEditTarget, so an edit made in one instance reaches every other configured instance alongside the ABS and content server targets; the forwarder already excludes the source. Applies descriptive fields and covers per each connection's push toggles, resolves the book by root-relative path in that connection's own libraries, respects its locks, and records the push so the target's own sidecar rewrite is absorbed instead of ping- ponging - Grimmory's no-change detection ends the chain once values converge.
Grimmory writes its sidecar while the update request is still in flight, so the filesystem event could reach the forwarder before the registry entry existed and get forwarded back out as a real edit.
Strip the reviewer-facing narration from the Grimmory connector, keeping only the comments that record an external constraint: Grimmory rewriting the sidecar during a metadata update, its async refresh, its handling of locked fields, and the AudioBookShelf rescan the forwarder debounces past. Drop IGrimmoryProxy.GetLibraryBooks, which no caller outside the proxy used, and inline the single-use edition lookup in GrimmoryPushService. File the two Grimmory push strings in their alphabetical place in en.json instead of the middle of the GoTo* run, and translate the push dialog's field labels the way QuickstartMatchingSection does rather than hardcoding English. Publish Date reuses the existing PublishedDate key.
Grimmory rewrites its sidecar in response to a push and the forwarder drops that event as an echo, so Audiobookshelf and the other connections never saw covers or metadata that were pushed from Chaptarr.
Those three were written without their lock flag, so Grimmory's own metadata refresh could overwrite them. The cover is uploaded before the lock is set because Grimmory rejects an upload outright once the cover is locked, and an already-locked cover is left alone for the same reason.
12 tasks
A file adopted by a disk scan raises BookImportedEvent with NewDownload false, which NotificationService dropped before any provider saw it. A book added through Grimmory and picked up by a scan therefore got no library refresh and no metadata or cover push until something else touched it. Add the NotifyOnLibraryImports opt-in to INotification and NotificationBase and let NotificationService deliver library imports to the providers that declare it, along with an OnLibraryFileAdded hook for providers that work per file. Grimmory declares it whenever it is already configured to push, so a scan pickup goes through the existing OnReleaseImport path: refresh the matching library, then push the selected fields once Grimmory's own scan has the book. The three shared notification files are byte-identical to the ones on feature/calibre-content-server-connector so the two branches still merge without conflict.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Adds a Grimmory connection: a notification provider that refreshes Grimmory's libraries after
Chaptarr imports, renames, retags or deletes files, optionally pushes Chaptarr's metadata and
covers into Grimmory, and optionally forwards edits made in Grimmory back out to other
connections.
Library refreshes are queued at event time and drained in
ProcessQueue, per media type, so aburst of imports collapses into one refresh per library. Pushes are opt-in per connection
(Push Metadata / Push Covers) and lock the fields they write in Grimmory so its own metadata
refreshes don't overwrite them; a manual push dialog on Book Details and the book editor lets you
choose which fields to send.
Edit forwarding watches the root folders for Grimmory's sidecar writes (
.metadata.json/.cover.jpg) rather than polling, since Grimmory's database is remote. Chaptarr's own pushes alsorewrite the sidecar, so a small registry absorbs that echo and forwards only edits a person made.
Targets are discovered through
IExternalLibraryEditTarget, so no concrete provider is referencedand other connections can opt in independently.
Also includes a scan fix: an empty scan result for a subfolder of a root that provably still has
content is treated as a real delete rather than a dropped mount, so a book pruned from disk by an
external library app stops being tracked forever.
Fixes
Not a fix but resolves: #99
External Related PRs
grimmory-tools/grimmory#2567
This PR is not a blocker for this just the cover art won't get backed into the file by Grimmory. It's a small change. Confident it will get merged.
Database Migration
NO. Connection settings serialize into the existing
Notifications.SettingsJSON column. The newOnLibraryFileAdded/SupportsOnLibraryFileAddedmembers are interface-level only.How was this tested?
Docker on Ubuntu against Grimmory, plus a Calibre content server and AudioBookShelf
configured as sibling connections to receive forwarded edits. Edit forwarding requires Grimmory's
sidecar "write on update" setting to be on. Also see the External Related PRs
Verified:
Note: Delays between the systems are to be expected. Grimmory's library refresh is asynchronous,
and the edit forwarder debounces for 90s before sending, so several of these scenarios need a wait
rather than an immediate check.
Per-media-type library refresh
Import an ebook, then an audiobook, with both libraries selected on the connection.
Expected: each import refreshes only its own Grimmory library. Nothing is sent at event time —
refreshes queue and drain on
ProcessQueue, so a burst of imports collapses into one refresh perlibrary rather than one per file.
Book added in Grimmory is adopted by the scan
Add a book through Grimmory so the file lands in the shared root folder, then run a Chaptarr scan.
Expected: the file is adopted and mapped to the right book/edition if it matches one, otherwise it
shows on the unmapped files page for manual matching. Grimmory already holds the file, so no
refresh is needed for it to appear there.
Book deleted in Grimmory goes back to missing and redownloads
Delete a book in Grimmory so its file leaves the shared storage, then run a Chaptarr scan.
Expected: the file row is removed as MissingFromDisk and the book stays monitored — it drops back
to missing and is eligible for search again, rather than being unmonitored. An empty scan result
for a subfolder of a root that provably still has content is a real delete, not a dropped mount,
so only a root-level empty scan keeps the mount guard. Before this, the row lingered forever,
Chaptarr still believed it had the file, and the book was never re-searched.
Rename, retag, and delete trigger refresh
Rename an author's files, retag a book, delete a book with files, delete an author with files.
Expected: each queues a refresh for the right library. Deletes now drain the queue —
MediaFileDeletionServicepublishesDeleteCompletedEventon book deletes, which it previouslyonly did for author deletes, so a book delete no longer left the refresh sitting in the queue.
Field selection dialog
Open Grimmory Push on a book page.
Expected: ten selectable fields — Cover, Title, Author, Series, Description, Publisher, Published
Date, Language, Tags, Identifiers — each with the value that will be sent shown beside it. All are
ticked by default. Push is disabled if you untick everything. The /books bulk dialog is identical
minus the previews.
Push writes only what it has
Push all fields on a book with no description, publisher or genres in Chaptarr.
Expected: empty fields are skipped, not sent — whatever Grimmory already holds survives. Only the
fields with values appear in the request.
Pushed fields come back locked
Push title and description, then look at the book in Grimmory.
Expected: both land and both are locked there, so Grimmory's own metadata refreshes won't
overwrite them.
Re-push respects an existing lock
Change the description via an edition change in Chaptarr and pushing again without unlocking anything in Grimmory.
Expected: the field does not change — Grimmory skips locked fields even for the writer that locked
them. Unlock it in Grimmory, push again, and the new value lands. Locks stay authoritative.
Auto-push waits for Grimmory's scan
Enable Push Metadata and Push Covers, then import a new book.
Expected: the push is queued behind the refresh and retries until the book appears in Grimmory —
up to 90s, re-fetching the library list every 10s rather than trusting a cached one. Without the
wait the push lands before Grimmory has ingested the file and finds nothing.
Cover change in Chaptarr auto-pushes
Change a book's cover in Chaptarr with Push Covers on.
Expected: the new cover reaches Grimmory. Author-level cover events are deliberately ignored —
they fire during routine author refreshes and would otherwise fan out into a push for every book
of that author. Repeat pushes for the same book are suppressed for 5 minutes.
Bulk push over a selection
Select several books on /books, push.
Expected: one command carries all ids; each book pushes independently, and a failure on one does
not abort the rest.
Edit made in Grimmory is forwarded out
With Forward Grimmory Edits on, change a description or cover in Grimmory.
Expected: after the debounce, the edit reaches the other connections that accept library edits —
the Calibre content server and AudioBookShelf here. Identity fields stay Chaptarr's; only the
descriptive fields mirror. Nothing is written to Chaptarr's own database.
Chaptarr's own push is not echoed back
Push metadata to Grimmory with forwarding on.
Expected: Grimmory rewrites the sidecar in response to the push, and that write is recognised as
Chaptarr's own and dropped.
A real edit immediately after a push still forwards
Push to Grimmory, then edit the same book in Grimmory before the debounce window closes.
Expected: the push's echo is consumed once and the person's edit still comes out of the same
batch. This is the case that a per-batch decision would have collapsed — echo and genuine edit
coalescing and being discarded together.
Cover-only edit in Grimmory
Replace only the cover in Grimmory, touch no other field.
Expected: the cover reaches the sibling connections as image bytes, not as a URL, so a target that
cannot authenticate against Grimmory still gets it.
Sibling Grimmory connection receives the edit
Configure two Grimmory connections, edit a book in the first.
Expected: the second one receives the forwarded edit and converges; the source connection does not
receive its own edit back.
Screenshots (UI changes only)
Quick Start config option:
Connection config option:
Books ulk push 'Push Chaptarr metadata to Grimmory' button:
Books bulk push 'Push Chaptarr metadata to Grimmory' dialog after clicking:
Book details 'Grimmory Push' button:
Book details 'Push Chaptarr metadata to Grimmory' dialog with preview after clicking:
A note on AI: We know AI/agentic coding is everywhere and only getting
more popular. We won't insist that you disclose whether you used it or which
models you used, but in the same spirit, please don't take offense if your PR
is scrutinized and changes are requested.
Review time: The longer the PR and the more lines changed, the longer the
review will take. Small, focused PRs merge fastest. If yours is big, please be
patient.