Skip to content

Custom provider live catalog is never refreshed in production (scoped catalog state rewrite) #4462

Description

@Hmbown

Defect

A custom provider registered via the custom provider path never gets its /v1/models catalog loaded in production: the model picker shows only the provider's default model instead of the endpoint's real offerings.

RuntimeClient::fetch_catalog_delta (crates/tui/src/client.rs:1464) and refresh_catalog_cache (crates/tui/src/client.rs:1559) exist, but every call site is in test code — the production startup path only refreshes the Models.dev catalog. Root-cause analysis credit: @baendlorel in #4370.

Why the naive fix is not acceptable

Simply invoking the refresh against today's single global LIVE_SNAPSHOT (crates/tui/src/provider_lake.rs:28) would let Models.dev and per-provider refreshes overwrite each other's rows. The v0.9.1 review of #4370 also flagged that copying price/capability metadata from a home provider onto a custom provider's same-named models fabricates facts the provider never stated — provider capability must remain route-effective and truthful.

Required shape

  • Source-layered catalog state: Models.dev layer plus one scoped layer per live provider fetch, merged at read time by (provider, wire_model_id) identity — no layer overwrites another.
  • Custom-provider rows carry only facts that provider's own /v1/models response supplied; unknown price/capability fields stay unknown rather than inherited from same-named home-provider models.
  • Production wiring that actually invokes per-provider refresh with failure-preserving semantics (keep stale rows on failure, surface health per v0.9.2: Distinguish configured providers and MCP servers from live health #4406).
  • Tests: layer coexistence, identity-merge correctness, unknown-metadata truthfulness, failure preservation.

Surfaced by #4370 (TelecomJS preset) — that PR remains the provider-preset vehicle; this issue tracks the catalog architecture it depends on.

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingenhancementNew feature or request

    Projects

    Status
    Backlog

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions