List models from the server for OpenAI-compatible providers - #41
Open
sachin-detrax wants to merge 2 commits into
Open
List models from the server for OpenAI-compatible providers#41sachin-detrax wants to merge 2 commits into
sachin-detrax wants to merge 2 commits into
Conversation
Closes AtomicBot-ai#31. The OpenAI-compatible wizard asked operators to type a chat model id by hand, which nobody can do for a remote vLLM deployment whose model list changes without notice. The chat model step now GETs `{baseUrl}/v1/models` (bearer key from the wizard buffer or the resolved env key) and shows the ids as a windowed picker. Typing still works — any keystroke is treated as a deliberate override and hides the list; backspacing to empty brings it back. When the server refuses or is unreachable the step degrades to the old typed field with the error in the hint. Base URLs are also normalised on save: a pasted `https://host/v1/` used to be stored verbatim and produce `https://host/v1//v1/chat/completions` on every request.
Live check against openrouter with a base URL pasted as `.../api/v1` rendered `.../api/v1/v1/models` in the header while requesting the correct URL. Normalize at the single source both the fetch and the header read from.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #31.
Problem
The OpenAI-compatible provider wizard asked the operator to type a chat model id by hand. Against a remote vLLM/SGLang deployment — the case in #31 — nobody knows the ids offhand, and providers rotate them without notice. The server already publishes them at
GET {baseUrl}/v1/models.Change
src/llm/provider/openai/fetch-openai-compat-models.ts(new) —GET {baseUrl}/v1/modelswith the bearer key, sorted ids, 10s timeout, cached per normalized base URL. Same fetch-then-read-sync shape the OpenRouter picker already uses, so the sync key handler can read the result.↑/↓ move (21/341) · Enter select). Typing still works: any printable key is a deliberate override that hides the list, backspacing to empty brings it back. Arrows only for movement, so ids starting withj/kstay typable.model list unavailable (http 401) — type the id).https://host/v1/was stored verbatim and producedhttps://host/v1//v1/chat/completionson every later request. It is now normalized once, and the same helper feeds the fetch, the cache key and the displayed URL.The API key comes from the wizard buffer, falling back to
resolveLlmProviderApiKey(OPENAI_COMPAT_API_KEY/OPENAI_API_KEY/ATOMIC_AGENT_OPENAI_API_KEY), so a key already in.envneeds no retyping.Verification
15 new tests (fetch/cache/normalize, picker key flow, typed override, windowed render, refusal fallback). Full suite: 3293 passing, unchanged pre-existing failures on
main;tscclean.Also exercised against real servers, no mocks:
https://openrouter.ai/api— 341 ids listed and rendered; base pasted as.../api/v1/hit the cache instead of re-requesting; arrows + Enter selected a model and built the entry withbaseUrl: https://openrouter.ai/api.GET /v1/models auth=Bearer <key>, and the cold-cache render produced the windowed picker.https://api.openai.comwith no key →http 401, nothing cached, hint shown; unreachable host →fetch failedhint.Known limitation
Servers whose OpenAI base is a path prefix rather than a host root (e.g. DeepInfra's
https://api.deepinfra.com/v1/openai) 404 on discovery, because the codebase appends/v1/...and the normalizer only strips a trailing/v1. That shape already cannot serve chat completions through this provider, so it is pre-existing rather than a regression; the wizard falls back to typed entry there.