fix: discover models from an allowlisted configured gateway - #873
fix: discover models from an allowlisted configured gateway#873seonghobae wants to merge 1 commit into
Conversation
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Closing and replacing this PR because an organization-specific hostname appeared in an initial test fixture. The replacement will contain only generic fixtures and corrected model-info pricing evidence. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| allowed_hosts = { | ||
| host.strip().casefold() | ||
| for host in environ.get( | ||
| "CONTEXTUAL_ORCHESTRATOR_ALLOWED_PROVIDER_HOSTS", "" | ||
| ).split(",") | ||
| if host.strip() | ||
| } | ||
| if parsed.hostname.casefold() not in allowed_hosts: | ||
| raise ValueError("LLM gateway host must be present in the provider allowlist") |
There was a problem hiding this comment.
🔍 Discovery and runtime use two different host allowlists
configured_gateway_source allowlists the gateway host from the CONTEXTUAL_ORCHESTRATOR_ALLOWED_PROVIDER_HOSTS env var, but runtime egress uses the --allowed-provider-host CLI list (contextual_orchestrator/orchestrator.py:1630). With --auto-discover-model-agents, discovered gateway agents are created enabled (main.py); a CLI list that omits the gateway host then refuses routing to them at request time. Setting only the CLI list makes startup abort with ValueError.
Was this helpful? React with 👍 or 👎 to provide feedback.
| values = { | ||
| value.strip().rstrip("/") | ||
| for name in ("LLM_GATEWAY_API_URL", "LLM_GATEWAY_URL") | ||
| if isinstance((value := environ.get(name)), str) and value.strip() | ||
| } | ||
| if not values: | ||
| return None | ||
| if len(values) != 1: | ||
| raise ValueError("LLM gateway URL settings must identify the same endpoint") |
There was a problem hiding this comment.
📝 Info: Differently-written but equivalent gateway URLs get rejected
The URL set is normalized only by stripping whitespace and a trailing slash before len(values) != 1 is enforced; the /v1 canonicalization happens afterward. Setting LLM_GATEWAY_API_URL=https://host and LLM_GATEWAY_URL=https://host/v1, which resolve to the same endpoint, produces two values and raises, blocking discovery.
Was this helpful? React with 👍 or 👎 to provide feedback.
| readme = "README.md" | ||
| requires-python = ">=3.10" | ||
| dependencies = [ | ||
| "certifi>=2024.8.30", |
There was a problem hiding this comment.
📝 Info: certifi added to pyproject without a lockfile regeneration in the diff
pyproject.toml promotes certifi>=2024.8.30 to a direct dependency. requirements.lock already pins certifi==2026.7.22 transitively and satisfies the constraint, so hash-locked installs still work and nothing breaks; only the direct-vs-transitive classification in the lock may be stale.
Was this helpful? React with 👍 or 👎 to provide feedback.
1 participant
Problem
LLM_GATEWAY_API_URL/LLM_GATEWAY_URLandLLM_GATEWAY_API_KEYin the trusted bootstrap environment were not connected to anyProviderModelSource. The live/v1/modelsendpoint listed seven chat models plus one embedding model, butdiscover-modelsignored that gateway entirely.The same live endpoint also exposes a Sectigo chain trusted by certifi but absent from this uv Python installation system CA file, so a correctly wired source initially failed TLS with
unable to get local issuer certificate.Fix
CONTEXTUAL_ORCHESTRATOR_ALLOWED_PROVIDER_HOSTS.LLM_GATEWAY_API_KEYinto the credential KV during the one-shot bootstrap boundary; runtime discovery reads it back throughget_credential.--free-onlystill requires structured zero-price evidence; unknown pricing is never labeled free.Evidence
54 passed.113 passed.discovered_count=7,providers_with_errors=[]; models includegpt-4.1,gpt-4.1-mini,gpt-5.6-sol,gpt-5.6-terra,gpt-5.6-luna,gpt-5.5, andgpt-5.4.free_tier_count=0andpriced_count=0are intentionally honest.Stacked on #868 because that PR defines
orchestrator/auto,orchestrator/free, gateway-default, and omitted-model serving semantics.