-
Notifications
You must be signed in to change notification settings - Fork 0
fix(strix): route direct-OpenAI fallbacks through the OpenAI API base #1324
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
5ecbebb
84eb379
689212b
859ab5a
7c596f3
2d98237
9288031
91c79dc
c5f1c74
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
|
seonghobae marked this conversation as resolved.
seonghobae marked this conversation as resolved.
|
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -674,7 +674,10 @@ jobs: | |
| echo "LLM_API_BASE_FILE=$llm_api_base_file" >> "$GITHUB_ENV" | ||
|
|
||
| - name: Prepare GitHub Models fallback credentials | ||
| if: steps.gate.outputs.provider_mode == 'openai_direct' || steps.gate.outputs.provider_mode == 'openrouter' || steps.gate.outputs.provider_mode == 'nvidia_nim' | ||
| # github_models is included because its STRIX_FALLBACK_MODELS chain | ||
| # ends in openai-direct/gpt-5.4, which needs the direct-OpenAI key and | ||
| # API base to authenticate and route after the primary is exhausted. | ||
| if: steps.gate.outputs.provider_mode == 'openai_direct' || steps.gate.outputs.provider_mode == 'openrouter' || steps.gate.outputs.provider_mode == 'nvidia_nim' || steps.gate.outputs.provider_mode == 'github_models' | ||
|
Comment on lines
+677
to
+680
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📝 Info: github_models fallback chain now reachable Adding Was this helpful? React with 👍 or 👎 to provide feedback. |
||
| env: | ||
| GITHUB_MODELS_FALLBACK_TOKEN: ${{ secrets.STRIX_GITHUB_MODELS_TOKEN || github.token }} | ||
| OPENAI_FALLBACK_KEY: ${{ secrets.STRIX_OPENAI_API_KEY || secrets.OPENAI_API_KEY }} | ||
|
|
@@ -706,6 +709,9 @@ jobs: | |
| openai_fallback_key_file="$RUNNER_TEMP/openai_fallback_key.txt" | ||
| printf '%s' "$openai_trimmed" > "$openai_fallback_key_file" | ||
| echo "STRIX_OPENAI_FALLBACK_KEY_FILE=$openai_fallback_key_file" >> "$GITHUB_ENV" | ||
| openai_fallback_api_base_file="$RUNNER_TEMP/openai_fallback_api_base.txt" | ||
| printf '%s' 'https://api.openai.com/v1' > "$openai_fallback_api_base_file" | ||
| echo "STRIX_OPENAI_FALLBACK_API_BASE_FILE=$openai_fallback_api_base_file" >> "$GITHUB_ENV" | ||
|
coderabbitai[bot] marked this conversation as resolved.
seonghobae marked this conversation as resolved.
|
||
| fi | ||
|
|
||
| - name: Prepare Vertex AI credentials | ||
|
|
@@ -838,6 +844,7 @@ jobs: | |
| STRIX_GITHUB_MODELS_API_BASE_FILE: ${{ env.STRIX_GITHUB_MODELS_API_BASE_FILE }} | ||
| STRIX_GITHUB_MODELS_KEY_FILE: ${{ env.STRIX_GITHUB_MODELS_KEY_FILE }} | ||
| STRIX_OPENAI_FALLBACK_KEY_FILE: ${{ env.STRIX_OPENAI_FALLBACK_KEY_FILE }} | ||
| STRIX_OPENAI_FALLBACK_API_BASE_FILE: ${{ env.STRIX_OPENAI_FALLBACK_API_BASE_FILE }} | ||
| STRIX_FAIL_ON_PROVIDER_SIGNAL: "1" | ||
| STRIX_VERTEX_FALLBACK_MODELS: "" | ||
| NPM_CONFIG_IGNORE_SCRIPTS: "true" | ||
|
|
||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| # Strix direct-OpenAI fallback API-base routing: evidence and design record | ||
|
|
||
| ## Decision | ||
|
|
||
| A Strix cross-provider fallback to an explicit direct-OpenAI model | ||
| (`openai-direct/...` or `openai_direct/...`) must route through the OpenAI | ||
| inference endpoint, never through the primary provider's `LLM_API_BASE`. The | ||
| gate now prefers an explicit `STRIX_OPENAI_FALLBACK_API_BASE_FILE` for such | ||
| models and, when the override is absent, resolves no API-base override so | ||
| litellm uses its default `https://api.openai.com/v1` endpoint. | ||
|
|
||
| The central workflow writes `https://api.openai.com/v1` into | ||
| `$RUNNER_TEMP/openai_fallback_api_base.txt` and exports | ||
| `STRIX_OPENAI_FALLBACK_API_BASE_FILE` whenever it publishes the OpenAI | ||
| fallback key file, so every provider chain that ends in | ||
| `openai-direct/gpt-5.4` (NVIDIA NIM primary, OpenRouter primary, | ||
| GitHub Models primary) inherits correct routing automatically. | ||
|
|
||
| ## Failure this fixes | ||
|
|
||
| Required-CI evidence (BandScope PR #1021 strix run 32800796577, 2026-08-25) | ||
| showed the NVIDIA NIM primary and first fallback exhausting provider | ||
| availability, then the contracted final fallback `openai-direct/gpt-5.4` | ||
| failing with a plain-text gateway error: | ||
|
|
||
| ```text | ||
| LLM CONNECTION FAILED | ||
| Could not establish connection to the language model. | ||
| Error: 404 page not found | ||
| ``` | ||
|
|
||
| Root cause: with `provider_mode=nvidia_nim`, the workflow sets | ||
| `LLM_API_BASE_FILE=https://integrate.api.nvidia.com/v1`. The gate reused that | ||
| base for the openai-direct fallback child, so litellm sent OpenAI requests to | ||
| the NVIDIA NIM edge, whose Go gateway answered `404 page not found`. The | ||
| fallback key was already routed correctly (`STRIX_OPENAI_FALLBACK_KEY_FILE`); | ||
| only the base URL leaked from the primary provider. Because no vulnerability | ||
| report artifact was produced, the gate failed closed — correct policy on an | ||
| incomplete scan, but caused by routing rather than by any repository finding. | ||
|
|
||
| ## Trust boundary | ||
|
|
||
| The override is a runner-provisioned regular file under `$RUNNER_TEMP`, | ||
| resolved through the same `resolve_trusted_input_file` boundary as the other | ||
| API-base files: it must be a regular non-symlink file inside the trusted input | ||
| root, must trim to a single `https://` URL, and must not contain whitespace or | ||
| control characters. Absent or empty overrides fall back to litellm's default | ||
| endpoint instead of failing configuration, preserving standalone local gate | ||
| runs where no workflow provisioning exists. | ||
|
|
||
| ## Verification contract | ||
|
|
||
| Regression evidence proves that: | ||
|
|
||
| 1. with a NVIDIA NIM primary base configured, `openai-direct/gpt-5.4` | ||
| resolves through the explicit OpenAI fallback base when provided; | ||
| 2. without an override, the resolver returns no base so litellm defaults to | ||
| `https://api.openai.com/v1`; | ||
| 3. NVIDIA NIM primary attempts keep resolving through the NIM edge; | ||
| 4. `github_models/*` fallbacks keep their dedicated GitHub Models endpoint; | ||
| 5. a non-https override fails configuration (exit 2) instead of scanning; | ||
| 6. the workflow provisions the override file and passes it into the gate env; | ||
| 7. the required-workflow smoke contract pins both sides of the wiring; and | ||
| 8. the stale `gpt-5.6-luna` expectations left behind by the model rename are | ||
| aligned with the valid `gpt-5.4` contract in queue-contract tests. | ||
|
|
||
| ## Limitations | ||
|
|
||
| This change restores reachability of the final fallback; it does not create | ||
| OpenAI quota. If the OpenAI key is absent or exhausted after NIM exhaustion, | ||
| the gate still fails closed as provider-unavailable — by design, because no | ||
| complete authoritative scan exists. Hosted model catalogs may also change | ||
| independently of this repository; model-name updates remain manual contract | ||
| changes reviewed through CI. |
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📝 Info: OpenAI fallback still fails when its key is absent When the primary is NVIDIA NIM / OpenRouter / GitHub Models and no OpenAI key exists, the workflow writes neither the fallback key nor base file (strix.yml). The (Refers to this code) Was this helpful? React with 👍 or 👎 to provide feedback. |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -2415,9 +2415,23 @@ resolved_llm_api_base_for_model() { | |
| return 0 | ||
| fi | ||
|
|
||
| if is_explicit_openai_model "$model" && [ -z "${STRIX_OPENAI_FALLBACK_API_BASE_FILE:-}" ]; then | ||
| # Without workflow provisioning, an explicit direct-OpenAI model must | ||
| # still not inherit a foreign primary base: resolve no override so | ||
| # litellm defaults to https://api.openai.com/v1. | ||
| return 0 | ||
| fi | ||
|
seonghobae marked this conversation as resolved.
|
||
|
|
||
| local api_base_file="$LLM_API_BASE_FILE" | ||
| local api_base_file_name="LLM_API_BASE_FILE" | ||
| if is_github_models_model "$model" && [ -n "${STRIX_GITHUB_MODELS_API_BASE_FILE:-}" ]; then | ||
| if is_explicit_openai_model "$model" && [ -n "${STRIX_OPENAI_FALLBACK_API_BASE_FILE:-}" ]; then | ||
| # Cross-provider fallback: openai-direct/* candidates must reach the | ||
| # direct OpenAI API even when the primary provider selected a | ||
| # different LLM_API_BASE_FILE endpoint (e.g. NVIDIA NIM). Without | ||
| # this the fallback hits the primary gateway and 404s. | ||
| api_base_file="$STRIX_OPENAI_FALLBACK_API_BASE_FILE" | ||
| api_base_file_name="STRIX_OPENAI_FALLBACK_API_BASE_FILE" | ||
| elif is_github_models_model "$model" && [ -n "${STRIX_GITHUB_MODELS_API_BASE_FILE:-}" ]; then | ||
| # Cross-provider fallback: when the active primary provider uses a | ||
| # different API base (for example OpenRouter), github_models/* fallback | ||
| # attempts must still route through the GitHub Models inference endpoint. | ||
|
Comment on lines
+2418
to
2437
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📝 Info: Direct-OpenAI base resolution verified sound The new branches in (Refers to this code) Was this helpful? React with 👍 or 👎 to provide feedback. |
||
|
|
||
Uh oh!
There was an error while loading. Please reload this page.