-
Notifications
You must be signed in to change notification settings - Fork 0
fix(strix): route direct-OpenAI fallback models to the OpenAI platform endpoint #1295
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
7471787
3d18db7
26ab3e7
9482635
ba77755
0a4fbca
fa2370b
a65fe0c
d376c33
cdf1518
8634b77
d5fae85
c0e15c5
ecfa09d
a0c032f
5a8adb5
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change | ||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
|
@@ -396,8 +396,9 @@ fi | |||||||||||||||||
| # models (openai-direct/... or openai_direct/...). When the primary model runs | ||||||||||||||||||
| # against NVIDIA NIM, OpenRouter, or GitHub Models, its LLM_API_KEY cannot | ||||||||||||||||||
| # authenticate a direct-OpenAI fallback; this file carries the OpenAI key. | ||||||||||||||||||
| # Optional: without it, explicit direct-OpenAI models keep using LLM_API_KEY, | ||||||||||||||||||
| # which is correct whenever the primary already runs against direct OpenAI. | ||||||||||||||||||
| # When the primary already runs against direct OpenAI, its LLM_API_KEY is safe | ||||||||||||||||||
| # to reuse for direct-OpenAI fallback models. A cross-provider fallback without | ||||||||||||||||||
| # this key fails closed instead of receiving the primary provider's credential. | ||||||||||||||||||
| STRIX_OPENAI_FALLBACK_KEY_FILE="${STRIX_OPENAI_FALLBACK_KEY_FILE:-}" | ||||||||||||||||||
| if [ -n "$STRIX_OPENAI_FALLBACK_KEY_FILE" ] && { [ ! -f "$STRIX_OPENAI_FALLBACK_KEY_FILE" ] || [ -L "$STRIX_OPENAI_FALLBACK_KEY_FILE" ]; }; then | ||||||||||||||||||
| echo "ERROR: STRIX_OPENAI_FALLBACK_KEY_FILE must reference a regular file containing the API key." >&2 | ||||||||||||||||||
|
|
@@ -2432,6 +2433,15 @@ resolved_llm_api_base_for_model() { | |||||||||||||||||
| return 0 | ||||||||||||||||||
| fi | ||||||||||||||||||
|
|
||||||||||||||||||
| if is_explicit_openai_model "$model" && ! is_explicit_openai_model "$PRIMARY_MODEL"; then | ||||||||||||||||||
| # Cross-provider fallback: direct-OpenAI models must always use the | ||||||||||||||||||
| # OpenAI platform endpoint. Inheriting the primary provider's API | ||||||||||||||||||
| # base (NVIDIA NIM, OpenRouter, GitHub Models) sends an OpenAI | ||||||||||||||||||
| # model and key to a foreign host, which answers "404 page not | ||||||||||||||||||
| # found" and turns every rate-limit outage into a failed fallback. | ||||||||||||||||||
| return 0 | ||||||||||||||||||
| fi | ||||||||||||||||||
|
Comment on lines
+2436
to
+2443
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 🟡 Duplicate unreachable direct-OpenAI routing guard The cross-provider guard
Suggested change
Was this helpful? React with 👍 or 👎 to provide feedback. |
||||||||||||||||||
|
|
||||||||||||||||||
| local api_base_file="$LLM_API_BASE_FILE" | ||||||||||||||||||
| local api_base_file_name="LLM_API_BASE_FILE" | ||||||||||||||||||
| if is_github_models_model "$model" && [ -n "${STRIX_GITHUB_MODELS_API_BASE_FILE:-}" ]; then | ||||||||||||||||||
|
|
@@ -2562,11 +2572,21 @@ run_strix_once() { | |||||||||||||||||
| # with the GitHub Models token, not the direct-OpenAI key. | ||||||||||||||||||
| child_llm_api_key="$STRIX_GITHUB_MODELS_KEY" | ||||||||||||||||||
| fi | ||||||||||||||||||
| if is_explicit_openai_model "$model" && [ -n "$STRIX_OPENAI_FALLBACK_KEY" ]; then | ||||||||||||||||||
| # Cross-provider fallback: explicit direct-OpenAI models | ||||||||||||||||||
| # authenticate with the OpenAI key, not the primary provider's | ||||||||||||||||||
| # key (NVIDIA NIM, OpenRouter, or GitHub Models). | ||||||||||||||||||
| child_llm_api_key="$STRIX_OPENAI_FALLBACK_KEY" | ||||||||||||||||||
| if is_explicit_openai_model "$model"; then | ||||||||||||||||||
| if [ "$model" != "$PRIMARY_MODEL" ] && [ -n "$STRIX_OPENAI_FALLBACK_KEY" ]; then | ||||||||||||||||||
| # Cross-provider fallback: explicit direct-OpenAI models | ||||||||||||||||||
| # authenticate with the OpenAI key, not the primary provider's | ||||||||||||||||||
| # key (NVIDIA NIM, OpenRouter, or GitHub Models). The same | ||||||||||||||||||
| # dedicated key also takes precedence for same-provider fallback. | ||||||||||||||||||
| child_llm_api_key="$STRIX_OPENAI_FALLBACK_KEY" | ||||||||||||||||||
| elif is_explicit_openai_model "$PRIMARY_MODEL"; then | ||||||||||||||||||
| # Same-provider fallback: reuse the primary direct-OpenAI key | ||||||||||||||||||
| # only when no dedicated fallback key was configured. | ||||||||||||||||||
| : | ||||||||||||||||||
| else | ||||||||||||||||||
| echo "ERROR: direct-OpenAI fallback '$model' requires STRIX_OPENAI_FALLBACK_KEY_FILE when the primary model uses another provider." >&2 | ||||||||||||||||||
| return 2 | ||||||||||||||||||
| fi | ||||||||||||||||||
|
seonghobae marked this conversation as resolved.
Comment on lines
+2575
to
+2589
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. 📝 Info: Direct-OpenAI key and base routing stay consistent The key-selection block (strix_quick_gate.sh) and the base block (strix_quick_gate.sh) agree across all four cases: direct-OpenAI primary reuses its key and base; cross-provider fallback with a dedicated key routes to the OpenAI default base; same-provider fallback without a dedicated key reuses the primary key; cross-provider fallback without a key fails closed with rc 2. The base gate is false when the primary is itself direct-OpenAI, so same-provider fallbacks keep the primary base, matching the added tests. Was this helpful? React with 👍 or 👎 to provide feedback. |
||||||||||||||||||
| fi | ||||||||||||||||||
|
seonghobae marked this conversation as resolved.
seonghobae marked this conversation as resolved.
seonghobae marked this conversation as resolved.
|
||||||||||||||||||
| fi | ||||||||||||||||||
| set -o pipefail | ||||||||||||||||||
|
|
@@ -4233,6 +4253,8 @@ run_current_target_scan() { | |||||||||||||||||
| read -r -a FALLBACK_MODELS <<<"$FALLBACK_MODELS_RAW" | ||||||||||||||||||
|
|
||||||||||||||||||
| fallback_tried=0 | ||||||||||||||||||
| local fallback_attempts=0 | ||||||||||||||||||
| local fallback_config_failures=0 | ||||||||||||||||||
| for candidate_raw in "${FALLBACK_MODELS[@]}"; do | ||||||||||||||||||
| candidate="$(normalize_model "$candidate_raw")" | ||||||||||||||||||
| if [ -z "$candidate" ] || [ "$candidate" = "$PRIMARY_MODEL" ]; then | ||||||||||||||||||
|
|
@@ -4246,6 +4268,7 @@ run_current_target_scan() { | |||||||||||||||||
| fi | ||||||||||||||||||
|
|
||||||||||||||||||
| fallback_tried=1 | ||||||||||||||||||
| fallback_attempts=$((fallback_attempts + 1)) | ||||||||||||||||||
| if is_vertex_model "$PRIMARY_MODEL"; then | ||||||||||||||||||
| echo "Primary Vertex model unavailable; retrying with fallback '$candidate'." | ||||||||||||||||||
| else | ||||||||||||||||||
|
|
@@ -4264,7 +4287,9 @@ run_current_target_scan() { | |||||||||||||||||
| return 0 | ||||||||||||||||||
| fi | ||||||||||||||||||
| if [ "$fallback_scan_rc" -eq 2 ]; then | ||||||||||||||||||
| return 2 | ||||||||||||||||||
| fallback_config_failures=$((fallback_config_failures + 1)) | ||||||||||||||||||
| echo "Skipping fallback model '$candidate' because its provider configuration is invalid; trying the next configured fallback." >&2 | ||||||||||||||||||
| continue | ||||||||||||||||||
|
seonghobae marked this conversation as resolved.
seonghobae marked this conversation as resolved.
|
||||||||||||||||||
| fi | ||||||||||||||||||
|
|
||||||||||||||||||
| local strict_fallback_provider_signal=0 | ||||||||||||||||||
|
|
@@ -4332,6 +4357,10 @@ run_current_target_scan() { | |||||||||||||||||
| fi | ||||||||||||||||||
| return 1 | ||||||||||||||||||
| fi | ||||||||||||||||||
| if [ "$fallback_config_failures" -eq "$fallback_attempts" ]; then | ||||||||||||||||||
| echo "ERROR: All configured fallback models failed provider configuration." >&2 | ||||||||||||||||||
| return 2 | ||||||||||||||||||
| fi | ||||||||||||||||||
|
seonghobae marked this conversation as resolved.
seonghobae marked this conversation as resolved.
|
||||||||||||||||||
|
|
||||||||||||||||||
| if [ "$INFRA_ERROR_DETECTED" -eq 1 ] && | ||||||||||||||||||
| [ "$PR_FINDINGS_DECISION" = "allow_baseline" ]; then | ||||||||||||||||||
|
|
||||||||||||||||||
Uh oh!
There was an error while loading. Please reload this page.