Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 8 additions & 8 deletions .github/workflows/opencode-review-dispatch.yml
Original file line number Diff line number Diff line change
Expand Up @@ -4058,8 +4058,8 @@ jobs:
"apiKey": "{env:OPENAI_API_KEY}"
},
"models": {
"gpt-5.6-luna": {
"name": "OpenAI GPT-5.6 Luna (direct)",
"gpt-5.4": {
"name": "OpenAI GPT-5.4 (direct)",
"tool_call": true,
"reasoning": true,
"options": {
Expand Down Expand Up @@ -4471,17 +4471,17 @@ jobs:
# or used for product/model improvement, so private repositories
# include neither NIM nor anonymous free candidates and start at the
# existing keyed fallback list: OpenCode Zen GPT-5.6 Terra, DeepSeek
# V3, the direct GPT-5.6 Luna slot, and pinned PAID
# V3, the direct GPT-5.4 slot, and pinned PAID
# OpenRouter coder models (free-tier candidates hit the shared
# free-models-per-day cap and hung for the full candidate timeout,
# so the OpenRouter slots use cheap paid models billed against the
# org's OpenRouter credits), then the full-size GPT-4.1 long-context
# endpoint and provider-specific GPT/o3 fallbacks.
# The direct-OpenAI slot runs GPT-5.6 Luna: the newest family's
# cost-efficient tier, cheaper than the legacy gpt-5 it replaced
# ($1/$6 vs $1.25/$10 per 1M tokens) so the org OpenAI budget
# stretches further between top-ups.
OPENCODE_MODEL_CANDIDATES: "${{ needs.validate-pr-metadata.outputs.is_private == 'false' && 'nvidia-nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 nvidia-nim/nvidia/llama-3.1-nemotron-ultra-253b-v1 nvidia-nim/nvidia/nemotron-3-super-120b-a12b nvidia-nim/nvidia/nemotron-3-ultra-550b-a55b nvidia-nim/meta/llama-3.3-70b-instruct nvidia-nim/deepseek-ai/deepseek-v4-pro nvidia-nim/mistralai/codestral-22b-instruct-v0.1 opencode-free/nemotron-3-ultra-free opencode-free/deepseek-v4-flash-free opencode-free/north-mini-code-free opencode-free/laguna-s-2.1-free opencode-free/ling-3.0-flash-free opencode-free/big-pickle opencode-free/mimo-v2.5-free opencode-free/hy3-free opencode-free/minimax-m3-free opencode-free/glm-5-free opencode-free/kimi-k2.5-free opencode-free/qwen3.6-plus-free ' || '' }}opencode/gpt-5.6-terra github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/deepseek/deepseek-v3.2 openrouter/qwen/qwen3-coder github-models/openai/gpt-4.1 github-models/openai/gpt-5 github-models/openai/gpt-5-chat github-models/openai/o3 github-models/deepseek/deepseek-r1-0528 github-models/deepseek/deepseek-r1"
# The direct-OpenAI slot runs GPT-5.4: the current valid model name
# on the direct OpenAI platform (the retired gpt-5.6-luna catalog
# entry returned 404s), kept cost-efficient against the org's
# OpenAI budget between top-ups.
OPENCODE_MODEL_CANDIDATES: "${{ needs.validate-pr-metadata.outputs.is_private == 'false' && 'nvidia-nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 nvidia-nim/nvidia/llama-3.1-nemotron-ultra-253b-v1 nvidia-nim/nvidia/nemotron-3-super-120b-a12b nvidia-nim/nvidia/nemotron-3-ultra-550b-a55b nvidia-nim/meta/llama-3.3-70b-instruct nvidia-nim/deepseek-ai/deepseek-v4-pro nvidia-nim/mistralai/codestral-22b-instruct-v0.1 opencode-free/nemotron-3-ultra-free opencode-free/deepseek-v4-flash-free opencode-free/north-mini-code-free opencode-free/laguna-s-2.1-free opencode-free/ling-3.0-flash-free opencode-free/big-pickle opencode-free/mimo-v2.5-free opencode-free/hy3-free opencode-free/minimax-m3-free opencode-free/glm-5-free opencode-free/kimi-k2.5-free opencode-free/qwen3.6-plus-free ' || '' }}opencode/gpt-5.6-terra github-models/deepseek/deepseek-v3-0324 openai/gpt-5.4 openrouter/deepseek/deepseek-v3.2 openrouter/qwen/qwen3-coder github-models/openai/gpt-4.1 github-models/openai/gpt-5 github-models/openai/gpt-5-chat github-models/openai/o3 github-models/deepseek/deepseek-r1-0528 github-models/deepseek/deepseek-r1"
Comment thread
seonghobae marked this conversation as resolved.
# One attempt per model, then fall through to the next model. Retrying
# the SAME model 5x let a rate-limited/hung leader consume the whole
# step, so the pool never reached a healthy fallback model.
Expand Down
9 changes: 8 additions & 1 deletion .github/workflows/strix.yml
Comment thread
seonghobae marked this conversation as resolved.
Comment thread
seonghobae marked this conversation as resolved.
Original file line number Diff line number Diff line change
Expand Up @@ -674,7 +674,10 @@ jobs:
echo "LLM_API_BASE_FILE=$llm_api_base_file" >> "$GITHUB_ENV"

- name: Prepare GitHub Models fallback credentials
if: steps.gate.outputs.provider_mode == 'openai_direct' || steps.gate.outputs.provider_mode == 'openrouter' || steps.gate.outputs.provider_mode == 'nvidia_nim'
# github_models is included because its STRIX_FALLBACK_MODELS chain
# ends in openai-direct/gpt-5.4, which needs the direct-OpenAI key and
# API base to authenticate and route after the primary is exhausted.
if: steps.gate.outputs.provider_mode == 'openai_direct' || steps.gate.outputs.provider_mode == 'openrouter' || steps.gate.outputs.provider_mode == 'nvidia_nim' || steps.gate.outputs.provider_mode == 'github_models'
Comment on lines +677 to +680

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: github_models fallback chain now reachable

Adding github_models to the fallback-credentials step (strix.yml) plus the explicit-OpenAI base routing makes the openai-direct/gpt-5.4 fallback for a GitHub Models primary reachable; previously it would have inherited the GitHub Models base and tripped the compatibility guard at strix_quick_gate.sh.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

env:
GITHUB_MODELS_FALLBACK_TOKEN: ${{ secrets.STRIX_GITHUB_MODELS_TOKEN || github.token }}
OPENAI_FALLBACK_KEY: ${{ secrets.STRIX_OPENAI_API_KEY || secrets.OPENAI_API_KEY }}
Expand Down Expand Up @@ -706,6 +709,9 @@ jobs:
openai_fallback_key_file="$RUNNER_TEMP/openai_fallback_key.txt"
printf '%s' "$openai_trimmed" > "$openai_fallback_key_file"
echo "STRIX_OPENAI_FALLBACK_KEY_FILE=$openai_fallback_key_file" >> "$GITHUB_ENV"
openai_fallback_api_base_file="$RUNNER_TEMP/openai_fallback_api_base.txt"
printf '%s' 'https://api.openai.com/v1' > "$openai_fallback_api_base_file"
echo "STRIX_OPENAI_FALLBACK_API_BASE_FILE=$openai_fallback_api_base_file" >> "$GITHUB_ENV"
Comment thread
coderabbitai[bot] marked this conversation as resolved.
Comment thread
seonghobae marked this conversation as resolved.
fi

- name: Prepare Vertex AI credentials
Expand Down Expand Up @@ -838,6 +844,7 @@ jobs:
STRIX_GITHUB_MODELS_API_BASE_FILE: ${{ env.STRIX_GITHUB_MODELS_API_BASE_FILE }}
STRIX_GITHUB_MODELS_KEY_FILE: ${{ env.STRIX_GITHUB_MODELS_KEY_FILE }}
STRIX_OPENAI_FALLBACK_KEY_FILE: ${{ env.STRIX_OPENAI_FALLBACK_KEY_FILE }}
STRIX_OPENAI_FALLBACK_API_BASE_FILE: ${{ env.STRIX_OPENAI_FALLBACK_API_BASE_FILE }}
STRIX_FAIL_ON_PROVIDER_SIGNAL: "1"
STRIX_VERTEX_FALLBACK_MODELS: ""
NPM_CONFIG_IGNORE_SCRIPTS: "true"
Expand Down
12 changes: 11 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,17 @@ this file. The format follows Keep a Changelog, and versioned releases follow
Semantic Versioning where the repository publishes a release.

## [Unreleased]

- Route Strix cross-provider fallbacks to explicit direct-OpenAI models
(`openai-direct/...`) through the OpenAI inference endpoint instead of
inheriting the primary provider's `LLM_API_BASE`: the workflow now
provisions `STRIX_OPENAI_FALLBACK_API_BASE_FILE`
(`https://api.openai.com/v1`), the gate prefers it for explicit OpenAI
fallback models and otherwise resolves no override so litellm defaults to
OpenAI, and a non-https override fails configuration. This removes the
NVIDIA-NIM-edge `404 page not found` that made the contracted final
fallback unreachable after NIM exhaustion.
- Align stale `gpt-5.6-luna` test expectations with the valid `gpt-5.4`
contract left behind by the earlier model rename.
Comment thread
devin-ai-integration[bot] marked this conversation as resolved.
- Honor each trusted base project's exact, integrity-bearing pnpm
`packageManager` specification in OpenCode coverage images through the pinned
Node distribution's Corepack runtime, instead of admitting the specification
Expand Down
74 changes: 74 additions & 0 deletions docs/doctoring/strix-openai-fallback-api-base-routing.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# Strix direct-OpenAI fallback API-base routing: evidence and design record

## Decision

A Strix cross-provider fallback to an explicit direct-OpenAI model
(`openai-direct/...` or `openai_direct/...`) must route through the OpenAI
inference endpoint, never through the primary provider's `LLM_API_BASE`. The
gate now prefers an explicit `STRIX_OPENAI_FALLBACK_API_BASE_FILE` for such
models and, when the override is absent, resolves no API-base override so
litellm uses its default `https://api.openai.com/v1` endpoint.

The central workflow writes `https://api.openai.com/v1` into
`$RUNNER_TEMP/openai_fallback_api_base.txt` and exports
`STRIX_OPENAI_FALLBACK_API_BASE_FILE` whenever it publishes the OpenAI
fallback key file, so every provider chain that ends in
`openai-direct/gpt-5.4` (NVIDIA NIM primary, OpenRouter primary,
GitHub Models primary) inherits correct routing automatically.

## Failure this fixes

Required-CI evidence (BandScope PR #1021 strix run 32800796577, 2026-08-25)
showed the NVIDIA NIM primary and first fallback exhausting provider
availability, then the contracted final fallback `openai-direct/gpt-5.4`
failing with a plain-text gateway error:

```text
LLM CONNECTION FAILED
Could not establish connection to the language model.
Error: 404 page not found
```

Root cause: with `provider_mode=nvidia_nim`, the workflow sets
`LLM_API_BASE_FILE=https://integrate.api.nvidia.com/v1`. The gate reused that
base for the openai-direct fallback child, so litellm sent OpenAI requests to
the NVIDIA NIM edge, whose Go gateway answered `404 page not found`. The
fallback key was already routed correctly (`STRIX_OPENAI_FALLBACK_KEY_FILE`);
only the base URL leaked from the primary provider. Because no vulnerability
report artifact was produced, the gate failed closed — correct policy on an
incomplete scan, but caused by routing rather than by any repository finding.

## Trust boundary

The override is a runner-provisioned regular file under `$RUNNER_TEMP`,
resolved through the same `resolve_trusted_input_file` boundary as the other
API-base files: it must be a regular non-symlink file inside the trusted input
root, must trim to a single `https://` URL, and must not contain whitespace or
control characters. Absent or empty overrides fall back to litellm's default
endpoint instead of failing configuration, preserving standalone local gate
runs where no workflow provisioning exists.

## Verification contract

Regression evidence proves that:

1. with a NVIDIA NIM primary base configured, `openai-direct/gpt-5.4`
resolves through the explicit OpenAI fallback base when provided;
2. without an override, the resolver returns no base so litellm defaults to
`https://api.openai.com/v1`;
3. NVIDIA NIM primary attempts keep resolving through the NIM edge;
4. `github_models/*` fallbacks keep their dedicated GitHub Models endpoint;
5. a non-https override fails configuration (exit 2) instead of scanning;
6. the workflow provisions the override file and passes it into the gate env;
7. the required-workflow smoke contract pins both sides of the wiring; and
8. the stale `gpt-5.6-luna` expectations left behind by the model rename are
aligned with the valid `gpt-5.4` contract in queue-contract tests.

## Limitations

This change restores reachability of the final fallback; it does not create
OpenAI quota. If the OpenAI key is absent or exhausted after NIM exhaustion,
the gate still fails closed as provider-unavailable — by design, because no
complete authoritative scan exists. Hosted model catalogs may also change
independently of this repository; model-name updates remain manual contract
changes reviewed through CI.
Original file line number Diff line number Diff line change
Expand Up @@ -956,7 +956,7 @@ extract_strix_failed_check_block "$EVIDENCE_FILE" "$strix_evidence_file"

emit_known_missing_string_finding \
"$EVIDENCE_FILE" \
"steps.target_visibility.outputs.is_private == 'false' && 'nvidia_nim/nvidia/nemotron-3-ultra-550b-a55b' || 'gpt-5.6-luna'" \
"steps.target_visibility.outputs.is_private == 'false' && 'nvidia_nim/nvidia/nemotron-3-ultra-550b-a55b' || 'gpt-5.4'" \
"Strix public scans must default to NVIDIA NIM while private scans retain the contracted provider" \
".github/workflows/strix.yml" \
"scripts/ci/test_strix_quick_gate.sh"
Expand Down
16 changes: 15 additions & 1 deletion scripts/ci/strix_quick_gate.sh

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: OpenAI fallback still fails when its key is absent

When the primary is NVIDIA NIM / OpenRouter / GitHub Models and no OpenAI key exists, the workflow writes neither the fallback key nor base file (strix.yml). The openai-direct/gpt-5.4 fallback then resolves to the default OpenAI endpoint but keeps the primary provider's key (strix_quick_gate.sh), so it fails auth. This is not a regression and is documented as by-design fail-closed behavior.

(Refers to this code)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Original file line number Diff line number Diff line change
Expand Up @@ -2415,9 +2415,23 @@ resolved_llm_api_base_for_model() {
return 0
fi

if is_explicit_openai_model "$model" && [ -z "${STRIX_OPENAI_FALLBACK_API_BASE_FILE:-}" ]; then
# Without workflow provisioning, an explicit direct-OpenAI model must
# still not inherit a foreign primary base: resolve no override so
# litellm defaults to https://api.openai.com/v1.
return 0
fi
Comment thread
seonghobae marked this conversation as resolved.

local api_base_file="$LLM_API_BASE_FILE"
local api_base_file_name="LLM_API_BASE_FILE"
if is_github_models_model "$model" && [ -n "${STRIX_GITHUB_MODELS_API_BASE_FILE:-}" ]; then
if is_explicit_openai_model "$model" && [ -n "${STRIX_OPENAI_FALLBACK_API_BASE_FILE:-}" ]; then
# Cross-provider fallback: openai-direct/* candidates must reach the
# direct OpenAI API even when the primary provider selected a
# different LLM_API_BASE_FILE endpoint (e.g. NVIDIA NIM). Without
# this the fallback hits the primary gateway and 404s.
api_base_file="$STRIX_OPENAI_FALLBACK_API_BASE_FILE"
api_base_file_name="STRIX_OPENAI_FALLBACK_API_BASE_FILE"
elif is_github_models_model "$model" && [ -n "${STRIX_GITHUB_MODELS_API_BASE_FILE:-}" ]; then
# Cross-provider fallback: when the active primary provider uses a
# different API base (for example OpenRouter), github_models/* fallback
# attempts must still route through the GitHub Models inference endpoint.
Comment on lines +2418 to 2437

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Direct-OpenAI base resolution verified sound

The new branches in resolved_llm_api_base_for_model route explicit openai-direct/* fallbacks correctly: with STRIX_OPENAI_FALLBACK_API_BASE_FILE set they use api.openai.com; absent, they early-return no override so litellm defaults to OpenAI. The workflow writes the base file inside the same guard as the fallback key, so key and base stay paired. The explicit-openai and github-models classes are disjoint, so no branch conflict.

(Refers to this code)

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

Expand Down
2 changes: 2 additions & 0 deletions scripts/ci/test_strix_quick_gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -365,6 +365,8 @@ assert_strix_workflow_pr_trigger_hardened() {
assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.4'" "strix workflow gives NVIDIA NIM scans contracted fallbacks"
assert_file_not_contains "$workflow_file" "STRIX_FALLBACK_MODELS: \${{ steps.gate.outputs.provider_mode == 'github_models' && 'github_models/openai/o3" "strix workflow fallback list must not depend on GitHub Models, which is in platform-wide retirement"
assert_file_contains "$workflow_file" "Prepare GitHub Models fallback credentials" "strix workflow provisions GitHub Models fallback credentials for direct-OpenAI scans"
assert_file_contains "$workflow_file" "STRIX_OPENAI_FALLBACK_API_BASE_FILE" "strix workflow routes direct-OpenAI fallbacks through a trusted API base file"
assert_file_contains "$workflow_file" "https://api.openai.com/v1" "strix workflow uses the OpenAI platform endpoint for direct fallbacks"
assert_file_contains "$GATE_SCRIPT" "STRIX_GITHUB_MODELS_KEY_FILE" "strix gate reads the optional GitHub Models fallback key file"
assert_file_contains "$GATE_SCRIPT" "STRIX_GITHUB_MODELS_API_BASE_FILE" "strix gate routes github_models fallback models through the GitHub Models endpoint"
assert_file_not_contains "$workflow_file" 'github_models/deepseek/deepseek-r1-0528 | github_models/deepseek/deepseek-v3-0324)' "strix workflow keeps DeepSeek GitHub Models restricted to fallback-only routing"
Expand Down
8 changes: 4 additions & 4 deletions tests/test_opencode_agent_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -186,7 +186,7 @@ def test_opencode_model_pool_sets_high_effort_for_capable_candidates():
["opencode-free", "qwen3.6-plus-free"],
["opencode", "gpt-5.6-terra"],
["github-models", "deepseek/deepseek-v3-0324"],
["openai", "gpt-5.6-luna"],
["openai", "gpt-5.4"],
["openrouter", "deepseek/deepseek-v3.2"],
["openrouter", "qwen/qwen3-coder"],
["github-models", "openai/gpt-4.1"],
Expand All @@ -197,7 +197,7 @@ def test_opencode_model_pool_sets_high_effort_for_capable_candidates():
["github-models", "deepseek/deepseek-r1"],
]
assert zen_models == ["gpt-5.6-terra"]
assert direct_openai_models == ["gpt-5.6-luna"]
assert direct_openai_models == ["gpt-5.4"]
assert openrouter_models == [
"deepseek/deepseek-v3.2",
"qwen/qwen3-coder",
Expand Down Expand Up @@ -1740,7 +1740,7 @@ def test_workflow_provisions_sandbox_tool_and_reviewer_agent():
assert (
"opencode/gpt-5.6-terra "
"github-models/deepseek/deepseek-v3-0324 "
"openai/gpt-5.6-luna "
"openai/gpt-5.4 "
"openrouter/deepseek/deepseek-v3.2 "
"openrouter/qwen/qwen3-coder "
"github-models/openai/gpt-4.1 "
Expand Down Expand Up @@ -1887,7 +1887,7 @@ def test_workflow_provisions_sandbox_tool_and_reviewer_agent():
)
assert (
"github-models/deepseek/deepseek-v3-0324 "
"openai/gpt-5.6-luna "
"openai/gpt-5.4 "
"openrouter/deepseek/deepseek-v3.2 "
"openrouter/qwen/qwen3-coder "
"github-models/openai/gpt-4.1 "
Expand Down
2 changes: 1 addition & 1 deletion tests/test_pr_review_autofix_nvidia_nim_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,7 @@
DOCTORING_RECORD = Path("docs/doctoring/hourly-nvidia-nim-autofix.md")
CHANGELOG = Path("CHANGELOG.md")
REVIEW_DISPATCH_WORKFLOW = Path(".github/workflows/opencode-review-dispatch.yml")
REVIEW_DISPATCH_BLOB_SHA = "dd65d90e10e5040562b501ade1a40f89572f0984"
REVIEW_DISPATCH_BLOB_SHA = "478788d41ed3e0d8389292a059d225336a925830"


def _workflow_text(path: Path) -> str:
Expand Down
2 changes: 1 addition & 1 deletion tests/test_required_workflow_queue_contract.py
Original file line number Diff line number Diff line change
Expand Up @@ -505,7 +505,7 @@ def test_nvidia_nim_defaults_preserve_existing_fallbacks_without_secret(
assert strix.returncode == 0, strix.stderr
assert {
"provider_mode=openai_direct",
"strix_model=gpt-5.6-luna",
"strix_model=gpt-5.4",
} <= set(strix_output.read_text().splitlines())
assert (
"STRIX_MODEL: ${{ steps.gate.outputs.strix_model }}"
Expand Down
4 changes: 2 additions & 2 deletions tests/test_strix_nvidia_nim_not_found_fallback.py
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,7 @@ def test_workflow_uses_available_free_first_nvidia_plan(self) -> None:
workflow = STRIX_WORKFLOW.read_text(encoding="utf-8")
default_expression = (
"steps.target_visibility.outputs.is_private == 'false' && "
f"'{DEFAULT_NVIDIA_MODEL}' || 'gpt-5.6-luna'"
f"'{DEFAULT_NVIDIA_MODEL}' || 'gpt-5.4'"
)
self.assertIn(default_expression, workflow)
self.assertIn(
Expand All @@ -202,7 +202,7 @@ def test_workflow_uses_available_free_first_nvidia_plan(self) -> None:
)
self.assertIn(
"steps.gate.outputs.provider_mode == 'nvidia_nim' && "
f"'{FREE_NVIDIA_FALLBACK} openai-direct/gpt-5.6-luna'",
f"'{FREE_NVIDIA_FALLBACK} openai-direct/gpt-5.4'",
workflow,
)

Expand Down
Loading
Loading