Skip to content

ci(strix): bootstrap live fallback smoke contract #1355

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
seonghobae wants to merge 6 commits into from
+3 −10
Closed

ci(strix): bootstrap live fallback smoke contract #1355

Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
b4890d7
ci(strix): bootstrap live fallback smoke contract
seonghobae Aug 26, 2026
14348c9
Merge branch 'main' into fix/strix-live-fallback-smoke
seonghobae Aug 26, 2026
4c95299
test(strix): require direct OpenAI after live fallback
seonghobae Aug 26, 2026
f86ad1c
Merge origin/main into fix/strix-live-fallback-smoke
seonghobae Aug 26, 2026
b826ef2
Merge commit 'refs/codex/pr1355-main' into codex/pr1355-refresh
seonghobae Aug 26, 2026
6baa896
test(strix): distinguish live fallback smoke path
seonghobae Aug 26, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
13 changes: 3 additions & 10 deletions scripts/ci/strix_required_workflow_smoke.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -168,7 +168,6 @@ assert_file_contains "$gate_script" "NPM_CONFIG_IGNORE_SCRIPTS" "Strix gate disa
assert_file_contains "$full_gate_test" "assert_strix_workflow_pr_trigger_hardened" "Full Strix harness remains available outside the required path"

assert_file_contains "$workflow_file" "nvidia_nim/nvidia/nemotron-3-super-120b-a12b" "Strix defaults public scans to the current hosted NVIDIA NIM model"
assert_file_contains "$workflow_file" "openrouter/free openai-direct/gpt-5.4" "Strix crosses to OpenRouter's free router before direct OpenAI when NVIDIA is exhausted"
fallback_expression="$(grep 'STRIX_FALLBACK_MODELS:' "$workflow_file")"
if printf '%s' "$fallback_expression" | grep -Fq "nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5"; then
record_failure "Strix must not route to the retired NVIDIA fallback model"
Expand All @@ -177,15 +176,9 @@ assert_file_contains "$workflow_file" "STRIX_OPENROUTER_FALLBACK_KEY_FILE" "Stri
assert_file_contains "$workflow_file" "STRIX_OPENROUTER_FALLBACK_API_BASE_FILE" "Strix workflow provisions a trusted OpenRouter fallback API base file"
assert_file_contains_either \
"$workflow_file" \
"steps.resolve_nvidia_models.outputs.fallback" \
"nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5" \
"Strix accepts the legacy fallback or a live catalog-resolved fallback during migration"
assert_file_contains_either \
"$workflow_file" \
"openai_direct/gpt-5.4" \
"openai-direct/gpt-5.4" \
"Strix retains the cross-provider direct-OpenAI fallback"
# ponytail: transitional compatibility; require only the dynamic fallback after its workflow lands.
"steps.gate.outputs.provider_mode == 'nvidia_nim' && 'openrouter/free openai-direct/gpt-5.4'" \
"steps.gate.outputs.provider_mode == 'nvidia_nim' && format('{0} openrouter/free openai-direct/gpt-5.4', steps.resolve_nvidia_models.outputs.fallback)" \
"Strix accepts the current static fallback or a live catalog fallback crossing OpenRouter before direct OpenAI"
Comment thread
seonghobae marked this conversation as resolved.
Comment on lines 177 to +181

@devin-ai-integration devin-ai-integration Bot Aug 26, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

📝 Info: Contract relaxation stays fail-closed

The single assert_file_contains_either at strix_required_workflow_smoke.sh folds in the removed bare openrouter/free openai-direct/gpt-5.4 check: both needles embed that literal, and both end in openai-direct/gpt-5.4, so no coverage is lost. The first needle matches strix.yml:875 today; the second anticipates PR #1297's format() resolver form.

Open in Devin Review

Was this helpful? React with 👍 or 👎 to provide feedback.

assert_file_not_contains "$workflow_file" "github_models/openai/o3" "Strix fallback list must not depend on GitHub Models, which is in platform-wide retirement"
assert_file_contains "$workflow_file" "Nvidia_nimException" "Strix workflow recognizes provider-scoped NVIDIA NIM failures"
assert_file_contains "$gate_script" "is_nvidia_nim_not_found_error" "Strix gate classifies NVIDIA NIM model-catalog 404s"
Expand Down
Loading