Skip to content

Query-driven search: resolve t=tvsearch against the broadcaster on demand (DR-011) #58

Description

@ChrisonSimtian

Description

t=tvsearch / t=search currently only finds what a crawler has already written to Postgres, and the
crawl list is three hardcoded seed shows (extra 3, Die Biene Maja, heute-show in
Agents/{Ard,Zdf}/Program.cs). So Sonarr searching for anything else gets an empty feed — indistinguishable
from "not available in the Mediathek".

That is backwards for a Newznab indexer. Per
DR-011, search should resolve against
the broadcaster on demand
, which is how MediathekArr works: it is "pretending to be a usenet indexer, but
actually just fetching and parsing search results from MediathekViewWeb". Sonarr asks, the indexer goes and
looks.

This is the piece that makes Krautwatch usable with zero *arr configuration — no instance, no API key,
no pre-registered show list.

Scope

  • On a t=search/t=tvsearch miss, resolve live via IBroadcasterCrawler for the queried title,
    persist what comes back, and serve it in the same response.
  • Cache negative results too, with a shorter TTL. Sonarr retries the same failing query on a
    schedule; without this, every miss becomes a fresh multi-hop crawl of ARD.
  • Bound the request. The ARD path is A-Z widget → show page → episode list → item page, which is far
    slower than Sonarr's expectations for an indexer. Needs a hard timeout and a partial-results answer
    rather than hanging the search.
  • Be polite to ARD/ZDF: coalesce concurrent identical queries, and rate-limit outbound so a Sonarr
    library refresh cannot turn into a crawl storm.
  • Keep the standing crawl list for the RSS feed (DR-011) — this issue changes search, not RSS.
  • Decide whether a live-resolve miss should also add the show to the standing list, so RSS-Sync picks
    up future episodes automatically.

Usage Example

# Today: empty unless "Tatort" happens to be one of the seeded shows
GET /api?t=tvsearch&q=Tatort&apikey=…      → <rss> with no items

# Target: resolves against ARD on demand, caches, and returns
GET /api?t=tvsearch&q=Tatort&apikey=…      → <rss> with Tatort episodes

Alternative

Keep requiring every show to be pre-registered — either by hand in Crawl:Targets (RundfunkArr's model)
or pre-warmed from Sonarr (#6). Both work, but both mean an unregistered show silently returns nothing,
and neither behaves the way an operator expects an indexer to behave.

Notes

  • Latency is the central design risk, and it is the reason to do caching and timeouts as part of this
    issue rather than after it. Putting broadcaster round-trips in Sonarr's request path is the trade DR-011
    accepts consciously.
  • Matching quality decides whether results are usable at all. EpisodeNumbering.Parse and SeriesType
    exist; Show.TvdbId is reserved but never set. Both comparables invest heavily here — MediathekArr calls
    out "the horrendous lack of consistency and metadata in ARD/ZDF Mediatheken".
  • Neither comparable documents how it serves RSS-Sync, so that half cannot be copied and still needs a
    standing crawl.
  • Relates to Remove the dead MediathekView catalog subsystem (drops the SharpCompress NU1902 vulnerability) #49 (on hold): if per-show crawling proves too narrow, the MediathekView filmliste is the
    full-catalog alternative — which is exactly what MediathekArr leans on.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions