Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .github/workflows/strix.yml
Original file line number Diff line number Diff line change
Expand Up @@ -834,7 +834,7 @@ jobs:
STRIX_LLM_MAX_RETRIES: 1
STRIX_TRANSIENT_RETRY_PER_MODEL: 2
STRIX_TRANSIENT_RETRY_BACKOFF_SECONDS: 60
STRIX_FALLBACK_MODELS: ${{ steps.gate.outputs.provider_mode == 'github_models' && 'openai-direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'openai_direct' && 'openai-direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'openrouter' && 'openai-direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.6-luna' || '' }}
STRIX_FALLBACK_MODELS: ${{ steps.gate.outputs.provider_mode == 'github_models' && 'openai_direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'openai_direct' && 'openai_direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'openrouter' && 'openai_direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai_direct/gpt-5.6-luna' || '' }}
Comment thread
seonghobae marked this conversation as resolved.
STRIX_GITHUB_MODELS_API_BASE_FILE: ${{ env.STRIX_GITHUB_MODELS_API_BASE_FILE }}
STRIX_GITHUB_MODELS_KEY_FILE: ${{ env.STRIX_GITHUB_MODELS_KEY_FILE }}
STRIX_OPENAI_FALLBACK_KEY_FILE: ${{ env.STRIX_OPENAI_FALLBACK_KEY_FILE }}
Expand Down
6 changes: 6 additions & 0 deletions .jules/bolt.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,3 +47,9 @@
## 2026-08-09 - [λŒ€μš©λŸ‰ 둜그 μŠ€μΊ” μ‹œ μ •κ·œν‘œν˜„μ‹ μ‹€ν–‰ μ „ O(N) μ„œλΈŒμŠ€νŠΈλ§ 검증 μ„ ν–‰]
**Learning:** `classify_testthat_failure`μ—μ„œ ν…ŒμŠ€νŠΈ μ‹€νŒ¨ 내역이 μ—†λŠ” 2MB 둜그 νŒŒμΌμ„ λŒ€μƒμœΌλ‘œ μ •κ·œν‘œν˜„μ‹μ„ μ‹€ν–‰ν•˜λ©΄ μ•½ 20msκ°€ μ†Œμš”λ˜μ§€λ§Œ, λ‹¨μˆœ λ¬Έμžμ—΄ 검색은 μ•½ 1ms만 μ†Œμš”λ©λ‹ˆλ‹€. λ¬Έμžμ—΄ 쑴재 μ—¬λΆ€κ°€ μ •κ·œν‘œν˜„μ‹ 맀칭의 μ „μ œ 쑰건일 λ•Œ, μ½œλ“œ 패슀(Cold Path)μ—μ„œ μˆœμ„œ μ΅œμ ν™”λŠ” 맀우 큰 μ„±λŠ₯ 차이λ₯Ό λ§Œλ“­λ‹ˆλ‹€.
**Action:** λŒ€μš©λŸ‰ ν…μŠ€νŠΈ μž…λ ₯(CI 둜그 λ“±)μ—μ„œ λ³΅μž‘ν•œ μ •κ·œν‘œν˜„μ‹μ„ νŒŒμ‹±ν•˜κΈ° 전에 항상 λΉ λ₯Έ O(N) λ¬Έμžμ—΄ 쑴재 μ—¬λΆ€ 확인을 λ¨Όμ € μˆ˜ν–‰ν•˜μ‹­μ‹œμ˜€.
## 2026-08-10 - Compile Combined Regexes for Multiple Substitutions
**Learning:** In `scripts/ci/redact_sensitive_log.py`, an iteration was looping over a tuple of pre-compiled regex objects (`PROVIDER_TOKEN_RES`) and repeatedly calling `.sub()` to redact strings. This resulted in O(M * N) overhead, where M is the number of regex patterns.
**Action:** When performing multiple regex replacements on the same text string where the replacements are identical (e.g., redacting text with a common marker), combine the regular expressions into a single compiled pattern using the `|` (alternation) operator. This allows `re.sub()` to process the string in a single O(N) pass, significantly reducing overhead in hot loops.
## 2026-08-11 - Match Bash String Manipulation Prefixes
**Learning:** In the `strix_quick_gate.sh` CI script, model prefix mapping translates `openai_direct/*` internally to litellm's expected `openai/*`. However, the GitHub Actions YAML defined the fallback models using hyphens (`openai-direct/gpt...`), causing the prefix matching to fail (`model#openai_direct/`) and passing the unknown literal `openai-direct/` string directly to the litellm provider engine.
**Action:** When configuring parameters for bash scripts that perform strict pattern matching and prefix replacement, ensure the configuration strings (like GitHub Actions environment variables) use the exact separator characters (e.g., underscores instead of hyphens) expected by the downstream string manipulation logic.
4 changes: 2 additions & 2 deletions PR_GOVERNANCE_AUDIT.md
Original file line number Diff line number Diff line change
Expand Up @@ -173,11 +173,11 @@ warning already flags this model as "not a recommended frontier model...
weaker models may miss vulnerabilities or produce lower-quality findings",
and this run is a concrete instance of that risk materializing as a false
required-check failure, not a missed finding. Fix: `STRIX_FALLBACK_MODELS`
now falls back to `openai-direct/gpt-5.6-luna` (Strix's own top-recommended
now falls back to `openai_direct/gpt-5.6-luna` (Strix's own top-recommended
model, already wired via `STRIX_OPENAI_API_KEY`/`OPENAI_API_KEY`) instead of
the dead GitHub Models pair, on all four provider-mode branches. The
`nvidia_nim` branch keeps its NVIDIA-hosted fallback as an interim retry
before this openai-direct fallback; that retains the existing free/low-cost
before this openai_direct fallback; that retains the existing free/low-cost
NVIDIA-first policy and is a separate cost/quality tradeoff this fix does not
revisit. GitHub Models remains a selectable `github_models` primary mode for
now (unchanged scope) but is no longer relied on as a silent universal
Expand Down
14 changes: 7 additions & 7 deletions scripts/ci/redact_sensitive_log.py
Original file line number Diff line number Diff line change
Expand Up @@ -24,11 +24,12 @@
r"[^\s\"'\\]+",
re.IGNORECASE,
)
PROVIDER_TOKEN_RES = (
re.compile(r"\b(?:gh[pousr]_[A-Za-z0-9_]{20,}|github_pat_[A-Za-z0-9_]{20,})\b"),
re.compile(r"\bsk-[A-Za-z0-9_-]{20,}\b"),
re.compile(r"\bxox[baprs]-[A-Za-z0-9-]{20,}\b"),
re.compile(r"\bAKIA[0-9A-Z]{16}\b"),
# Combined provider token regexes to optimize redaction loop by reducing string parsing passes
PROVIDER_TOKEN_RE = re.compile(
r"\b(?:gh[pousr]_[A-Za-z0-9_]{20,}|github_pat_[A-Za-z0-9_]{20,})\b|"
r"\bsk-[A-Za-z0-9_-]{20,}\b|"
r"\bxox[baprs]-[A-Za-z0-9-]{20,}\b|"
r"\bAKIA[0-9A-Z]{16}\b"
Comment thread
seonghobae marked this conversation as resolved.
)


Expand Down Expand Up @@ -118,8 +119,7 @@ def _redact_unstructured(text: str) -> str:
cleaned = _redact_assignments(text)
cleaned = BEARER_RE.sub(lambda match: f"{match.group('prefix')}{REDACTED}", cleaned)
cleaned = JWT_RE.sub(REDACTED, cleaned)
for pattern in PROVIDER_TOKEN_RES:
cleaned = pattern.sub(REDACTED, cleaned)
cleaned = PROVIDER_TOKEN_RE.sub(REDACTED, cleaned)
return cleaned


Expand Down
2 changes: 1 addition & 1 deletion scripts/ci/strix_required_workflow_smoke.sh
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ assert_file_contains "$gate_script" "NPM_CONFIG_IGNORE_SCRIPTS" "Strix gate disa
assert_file_contains "$full_gate_test" "assert_strix_workflow_pr_trigger_hardened" "Full Strix harness remains available outside the required path"

assert_file_contains "$workflow_file" "nvidia_nim/nvidia/nemotron-3-super-120b-a12b" "Strix defaults public scans to the current hosted NVIDIA NIM model"
assert_file_contains "$workflow_file" "nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.6-luna" "Strix tries another NVIDIA hosted model before falling back to direct OpenAI"
assert_file_contains "$workflow_file" "nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai_direct/gpt-5.6-luna" "Strix tries another NVIDIA hosted model before falling back to direct OpenAI"
assert_file_not_contains "$workflow_file" "github_models/openai/o3" "Strix fallback list must not depend on GitHub Models, which is in platform-wide retirement"
assert_file_contains "$workflow_file" "Nvidia_nimException" "Strix workflow recognizes provider-scoped NVIDIA NIM failures"
assert_file_contains "$gate_script" "is_nvidia_nim_not_found_error" "Strix gate classifies NVIDIA NIM model-catalog 404s"
Expand Down
6 changes: 3 additions & 3 deletions scripts/ci/test_strix_quick_gate.sh
Original file line number Diff line number Diff line change
Expand Up @@ -360,9 +360,9 @@ assert_strix_workflow_pr_trigger_hardened() {
assert_file_contains "$workflow_file" "https://integrate.api.nvidia.com/v1" "strix workflow routes NVIDIA NIM scans to the hosted endpoint"
assert_file_contains "$workflow_file" "LLM_API_BASE_FILE" "strix workflow passes the GitHub Models API base through a trusted input file"
assert_file_not_contains "$workflow_file" '${{ secrets.STRIX_OPENAI_API_KEY || github.token }}' "strix workflow must not use fallback-secret syntax for LLM API keys"
assert_file_contains "$workflow_file" "openai-direct/gpt-5.6-luna" "strix workflow keeps a direct-OpenAI fallback on a tool-capable, Strix-recommended model without GPT-4.1 downgrade"
assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'openai_direct' && 'openai-direct/gpt-5.6-luna'" "strix workflow gives direct-OpenAI scans a same-provider fallback so transient errors degrade instead of skipping"
assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.6-luna'" "strix workflow gives NVIDIA NIM scans contracted fallbacks"
assert_file_contains "$workflow_file" "openai_direct/gpt-5.6-luna" "strix workflow keeps a direct-OpenAI fallback on a tool-capable, Strix-recommended model without GPT-4.1 downgrade"
assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'openai_direct' && 'openai_direct/gpt-5.6-luna'" "strix workflow gives direct-OpenAI scans a same-provider fallback so transient errors degrade instead of skipping"
assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai_direct/gpt-5.6-luna'" "strix workflow gives NVIDIA NIM scans contracted fallbacks"
assert_file_not_contains "$workflow_file" "STRIX_FALLBACK_MODELS: \${{ steps.gate.outputs.provider_mode == 'github_models' && 'github_models/openai/o3" "strix workflow fallback list must not depend on GitHub Models, which is in platform-wide retirement"
assert_file_contains "$workflow_file" "Prepare GitHub Models fallback credentials" "strix workflow provisions GitHub Models fallback credentials for direct-OpenAI scans"
assert_file_contains "$GATE_SCRIPT" "STRIX_GITHUB_MODELS_KEY_FILE" "strix gate reads the optional GitHub Models fallback key file"
Expand Down
2 changes: 1 addition & 1 deletion tests/test_strix_nvidia_nim_not_found_fallback.py
Original file line number Diff line number Diff line change
Expand Up @@ -202,7 +202,7 @@ def test_workflow_uses_available_free_first_nvidia_plan(self) -> None:
)
self.assertIn(
"steps.gate.outputs.provider_mode == 'nvidia_nim' && "
f"'{FREE_NVIDIA_FALLBACK} openai-direct/gpt-5.6-luna'",
f"'{FREE_NVIDIA_FALLBACK} openai_direct/gpt-5.6-luna'",
workflow,
)

Expand Down
Loading