Skip to content

feat(playground): filter model picker to providers with active credentials#10

Open
gaurav0107 wants to merge 1 commit into
mainfrom
fix/playground-shows-configured-providers
Open

feat(playground): filter model picker to providers with active credentials#10
gaurav0107 wants to merge 1 commit into
mainfrom
fix/playground-shows-configured-providers

Conversation

@gaurav0107

Copy link
Copy Markdown
Collaborator

Why

The playground's ModelPicker is fed by a static catalog (web/src/lib/models.ts) and never looks at the credentials in /workspace/credentials. So:

  • A user who has only configured Anthropic still sees OpenAI / Gemini / Mistral / DeepSeek / Groq in the dropdown.
  • Picking any unconfigured provider's model is a guaranteed dispatch failure (no <provider> credential resolved) — late, confusing.
  • The "Custom..." escape hatch for new models the catalog hasn't picked up yet is buried inside an <optgroup label="Other"> — not discoverable.

What this changes

Surface the configured set up front, keep the catalog as the discoverability layer:

  1. /playground page — server-fetch /v1/llm-credentials?workspace_id=… in parallel with prompts + sessions; project to a sorted set of distinct providers where revoked_at IS NULL; pass into PlaygroundComposer.

  2. PlaygroundComposer + ModelCard — thread the configuredProviders array through to both ModelPicker instances (single + compare modes).

  3. ModelPicker — new optional availableProviders?: string[] prop:

    • undefined → show all providers (legacy; judges/etc keep working unchanged)
    • non-empty array → filter <optgroup>s to that set
    • empty array → only the Custom escape hatch + an inline No LLM credentials configured. Add one → link to /workspace/credentials

The "Custom…" option always stays available so a model the curated catalog hasn't picked up yet is one free-text input away.

Why no new backend endpoint

The existing GET /v1/llm-credentials?workspace_id=… already returns the rows with provider and revoked_at. Adding a derivative endpoint would be one more thing to maintain for no clear gain — the page just needs to know which providers have an active credential.

Verified locally

  • pnpm typecheck — clean
  • pnpm lint — No ESLint warnings or errors
  • pnpm build — succeeds (no static build regressions)

Test plan

  • CI green.
  • Merge.
  • On the live site at /playground:
    • With no credentials: picker shows only "Custom…" and a link to /workspace/credentials.
    • With only Anthropic configured: picker shows only Anthropic models + Custom.
    • With all 6 providers: picker behaves like today.
  • Other ModelPicker call sites (judges, etc.) are unaffected — they don't pass availableProviders so they get the legacy "show everything" behavior.

🤖 Generated with Claude Code

…tials

The playground's ModelPicker reads from a static catalog
(web/src/lib/models.ts) and never looked at the credentials configured
in /workspace/credentials. So a user with only Anthropic configured
still sees OpenAI / Gemini / Mistral / DeepSeek / Groq in the dropdown,
and picking any of those is a guaranteed dispatch failure with a
"no <provider> credential resolved" error.

Surface the configured set up front:

- web/src/app/playground/page.tsx: server-fetch
  /v1/llm-credentials?workspace_id=... in parallel with the prompts +
  sessions queries; project to a sorted set of distinct providers
  whose revoked_at is NULL; pass that into PlaygroundComposer.
- PlaygroundComposer + ModelCard: thread the configuredProviders
  array through to both ModelPicker instances (single + compare).
- ModelPicker: new optional `availableProviders?: string[]` prop.
    - undefined  -> show all providers (legacy behavior; judges and
                    elsewhere keep working unchanged)
    - non-empty  -> filter <optgroup>s to that set
    - empty      -> only the Custom escape hatch + a "No LLM
                    credentials configured. Add one ->" link to
                    /workspace/credentials

The Custom escape hatch always stays available so a model the curated
catalog hasn't picked up yet (e.g. a new Claude/Gemini preview) is one
free-text input away.

Verified locally:
- pnpm typecheck: clean
- pnpm lint: clean
- pnpm build: succeeds

Signed-off-by: Gaurav Dubey <gauravdubey0107@gmail.com>
Signed-off-by: gaurav0107 <gauravdubey0107@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant