diff --git a/CLAUDE.md b/CLAUDE.md index d6f09f4..82c2db8 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -26,13 +26,17 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co > Aspire-generated compose in the distribution milestone. > > **Known gaps (don't assume these exist):** -> - **The crawl work-list is hardcoded**, not Sonarr-driven. `CrawlOptions.Targets` binds from each -> agent's `Crawl` config section and falls back to seed shows in `Agents/{Ard,Zdf}/Program.cs` -> (`extra 3` / `Die Biene Maja` / `heute-show`). DR-010's reach-back — poll each Sonarr/Radarr -> instance for its `monitored` series — is **not implemented**, and neither is any -> Sonarr/Radarr-instance entity or config UI. `Settings` is download settings only. +> - **Search is not query-driven yet, and the crawl list is hardcoded.** `CrawlOptions.Targets` binds +> from each agent's `Crawl` config section and falls back to seed shows in `Agents/{Ard,Zdf}/Program.cs` +> (`extra 3` / `Die Biene Maja` / `heute-show`). So `t=tvsearch` only finds what has already been +> crawled — an unseeded show returns nothing. **DR-011** makes on-demand resolution the target and +> demotes the standing list to RSS-feed input; reach-back to Sonarr is an optional pre-warm, not a +> requirement (DR-010's work-list clause is retracted). No Sonarr/Radarr-instance config UI yet either +> — `Settings` is download settings only. > - `AppSettings.CatalogProviderKey` still defaults to `"mediathekview"`, and -> `Infrastructure/Catalog/MediathekView` survives from the DR-001 era — de-emphasised by DR-010. +> `Infrastructure/Catalog/MediathekView` survives from the DR-001 era. **Do not delete it yet** — +> per DR-011 the filmliste is the only full-catalog source available, and it is the fallback if +> per-show crawling proves too narrow (#49 is on hold). > - Auth is a single optional API key (see below); no user/identity model. > - `README.md` is **pre-DR-009 and wholly wrong** (describes the dropped role system, SQLite, > `Krautwatch.Worker`, the retired `/api/catalog` surface) — issue #25. @@ -236,7 +240,9 @@ DE proxy and `./build.cmd TestLive` does the real KiKA download (else it just pr ## Architecture decisions -`docs/architecture/` — **DR-009 (architecture reset) and DR-010 (arr-indexer direction) are current.** +`docs/architecture/` — **DR-009 (architecture reset), DR-010 (arr-indexer direction) and DR-011 +(search-driven indexing) are current.** DR-011 retracts DR-010's reach-back clause: the Sonarr monitored +list is *not* the crawl work-list. DR-002/004/008 are superseded; DR-001/003/005/006/007 still apply (some refined). Read the current DRs before structural changes. diff --git a/docs/architecture/DR-010-arr-indexer-direction.md b/docs/architecture/DR-010-arr-indexer-direction.md index e6aa8a0..c149ac4 100644 --- a/docs/architecture/DR-010-arr-indexer-direction.md +++ b/docs/architecture/DR-010-arr-indexer-direction.md @@ -2,10 +2,11 @@ | | | |---|---| -| **Status** | Accepted | +| **Status** | Accepted · **reach-back clause retracted by DR-011** | | **Date** | 2026-07-25 | | **Deciders** | Christian | | **Refines** | DR-001 (catalog provider abstraction) | +| **Refined by** | [DR-011](DR-011-search-driven-indexing.md) — the monitored list is *not* the crawl work-list | ## Context @@ -25,8 +26,12 @@ Krautwatch integrates with the `*arr` stack via the interfaces those apps alread catalog. - **Download client (SABnzbd-compatible):** queue / history / add-by-id, so Sonarr/Radarr treat Krautwatch as a usenet download client. Actual pulls are direct HTTP/HLS + ffmpeg, with subtitles. -- **Reach-back:** poll each configured Sonarr/Radarr instance for its `monitored` series list; that - is the crawl work-list. +- ~~**Reach-back:** poll each configured Sonarr/Radarr instance for its `monitored` series list; that + is the crawl work-list.~~ + > **Retracted by [DR-011](DR-011-search-driven-indexing.md).** Neither MediathekArr nor RundfunkArr + > reads Sonarr's monitored list — the Newznab contract is pull-by-query, so an indexer does not need + > it. Search becomes query-driven; the standing crawl list feeds only the RSS feed, and reach-back is + > an optional pre-warm. The **UI shrinks to configuration** (Sonarr/Radarr instances, connection test) — not a browse/search surface. diff --git a/docs/architecture/DR-011-search-driven-indexing.md b/docs/architecture/DR-011-search-driven-indexing.md new file mode 100644 index 0000000..af5557f --- /dev/null +++ b/docs/architecture/DR-011-search-driven-indexing.md @@ -0,0 +1,106 @@ +# DR-011 — Search-Driven Indexing (the monitored list is not the work-list) + +| | | +|---|---| +| **Status** | Accepted | +| **Date** | 2026-07-28 | +| **Deciders** | Christian | +| **Refines** | DR-010 (arr-indexer direction) — specifically retracts its **Reach-back** clause | + +## Context + +DR-010 established that Krautwatch is a Sonarr/Radarr **indexer + download client**, and that stands. +But one clause went further than the integration requires: + +> **Reach-back:** poll each configured Sonarr/Radarr instance for its `monitored` series list; that is +> the crawl work-list. + +That made the monitored list load-bearing: #6 was scoped as "the crawl work-list", #12 (RSS for +monitored shows) was blocked behind it, and #4/#5 existed largely to make it possible. Before building +it, we checked how the two comparable projects solve the same problem. + +### How the comparables actually do it + +**MediathekArr** keeps no catalog. It is a **live search proxy** — "pretending to be a usenet indexer, +but actually just fetching and parsing search results from **MediathekViewWeb**". TVDB supplies +metadata, and the effort goes into an "advanced filter and matching system for TV shows, seasons and +episodes" to survive "the horrendous lack of consistency and metadata in ARD/ZDF Mediatheken". + +**RundfunkArr** uses **curated configuration**: `data/shows.json` (show metadata incl. `tvdbId`, german +name, aliases, episodes) plus `data/rulesets.json` (per-show Mediathek topic, duration filters, regex +for season/episode extraction, matching strategy). Shows are added by hand or community pull request. +SQLite is cache and download history only. + +**Neither reads Sonarr's monitored series via the Sonarr API.** Both are pure Newznab providers. + +### Why they don't need to + +The Newznab contract is **pull-by-query, not push-by-work-list**. Sonarr already knows what it monitors +and issues `t=tvsearch&q=…&tvdbid=…&season=&ep=` for precisely what it wants. An indexer is not +supposed to know the monitored list — that is the client's job. + +The reason the question arose for us is a decision we made, not a property of being an `*arr` indexer: +DR-010 **de-emphasised MediathekView in favour of direct ARD/ZDF APIs**. MediathekViewWeb is +effectively a complete dump, so MediathekArr gets breadth for free. Our ARD crawler must walk A-Z +catalog → show page → episode list → item page, so it has to be told *which* shows to walk. +RundfunkArr has the identical constraint and answers it with human curation. + +**The work-list problem is self-inflicted by the direct-API choice.** + +## Decision + +**Retract the reach-back clause.** The monitored list is not the crawl work-list. Instead: + +1. **Search is query-driven.** `t=search`/`t=tvsearch` resolves against the broadcaster on demand and + caches the result, so any show Sonarr asks for works without having been pre-registered. This is + what makes Krautwatch behave like an indexer rather than a catalog with a search box. +2. **A standing crawl list exists only to feed the RSS feed.** RSS-Sync needs a recent-releases list, so + *something* must be crawled proactively. That list is configuration (today `CrawlOptions.Targets`), + optionally **pre-warmed** from a Sonarr/Radarr monitored list where one is configured. +3. **Reach-back becomes an optional convenience, never a prerequisite.** No feature may depend on a + configured `*arr` instance to function. +4. **Matching is where the effort goes.** Both comparables concentrate there, and it decides whether + Sonarr can match anything at all. Our equivalents are `EpisodeNumbering.Parse`, `SeriesType`, and the + currently-unset `Show.TvdbId`. + +### Why not reach-back as the primary mechanism + +- **It inverts the dependency.** Holding Sonarr credentials and calling back into it is a bidirectional + coupling neither comparable has, and it is the reason #4 and #48 became prerequisites for basic + indexing. +- **It does not satisfy search.** An interactive search for a show that is unmonitored, or monitored but + not yet crawled, returns nothing. A query-driven path is needed regardless — which demotes reach-back + to an optimisation by definition. + +## Consequences + +- ✅ Indexing works with **zero** `*arr` configuration, which is how a Newznab indexer should behave. +- ✅ Unblocks #12: the RSS feed can serve the standing crawl list without waiting on #6. +- ✅ #4/#5 stop being prerequisites for core function; they remain worthwhile for pre-warming and for + the setup experience (#54). +- ⚠️ On-demand search puts broadcaster latency in Sonarr's request path. The ARD flow is multi-hop + (A-Z widget → show page → episodes → item page), so caching and a sane timeout are mandatory, not + optional polish. +- ⚠️ Rate-limiting/politeness toward ARD/ZDF becomes a real concern once searches hit them live. +- ⚠️ **#49 (delete the MediathekView subsystem) is now on hold.** That filmliste parser is precisely the + full-catalog source MediathekArr depends on; it is the mechanism for breadth without per-show + curation. Decide the search model before deleting it. (The `SharpCompress` advisory that motivated #49 + was resolved by a version bump, so there is no security pressure to remove it.) +- ⚠️ DR-010 remains correct on everything else — product direction, the Newznab/SABnzbd surfaces, the + provider/port abstraction, and the UI shrinking to configuration. + +## Evidence and its limits + +Both READMEs were read directly (see Sources). Two gaps worth recording honestly: + +- MediathekArr's README does not *explicitly* state live-versus-cached; "fetching and parsing search + results" strongly implies live, but it is an inference. +- **Neither** project documents RSS feed support, so how they serve RSS-Sync is unknown — which is + precisely the part that still requires a standing crawl. Our RSS design cannot be copied from either + and has to be reasoned out. + +## Sources + +- and its README +- +- diff --git a/docs/architecture/README.md b/docs/architecture/README.md index 9ca25ca..975670a 100644 --- a/docs/architecture/README.md +++ b/docs/architecture/README.md @@ -11,4 +11,5 @@ | [DR-007](DR-007-target-framework-net10.md) | Target Framework: .NET 10 LTS | Accepted | | [DR-008](DR-008-single-binary-role-architecture.md) | Single-Binary Role-Switchable Worker | ~~Accepted~~ · superseded by DR-009 | | [DR-009](DR-009-architecture-reset.md) | Architecture Reset: Hexagonal + Vertical-Slice CQRS/A Microservices | **Accepted** | -| [DR-010](DR-010-arr-indexer-direction.md) | Product Direction: Sonarr/Radarr Indexer | **Accepted** | +| [DR-010](DR-010-arr-indexer-direction.md) | Product Direction: Sonarr/Radarr Indexer | **Accepted** · reach-back clause retracted by DR-011 | +| [DR-011](DR-011-search-driven-indexing.md) | Search-Driven Indexing (the monitored list is not the work-list) | **Accepted** |