From 9a47c10e5837288db99d9d0ad3d98c615228b605 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 00:05:41 +0000 Subject: [PATCH 01/13] fix(e2e): restrict readiness polling to loopback destinations Land the #1244 loopback boundary on current main without mixing Strix classifier changes. Accept literal localhost and standard-library loopback addresses, unwrap IPv4-mapped IPv6, and reject userinfo, missing hosts, unspecified binds, and metadata destinations before any request is opened. Co-authored-by: Seongho Bae --- ARCHITECTURE.md | 5 ++ CHANGELOG.md | 6 ++ ...ndboxed-web-readiness-loopback-boundary.md | 9 ++ ...ndboxed-web-readiness-loopback-boundary.md | 82 +++++++++++++++++++ scripts/ci/sandboxed_web_e2e.py | 35 +++++++- tests/test_sandboxed_web_e2e.py | 35 +++++++- 6 files changed, 169 insertions(+), 3 deletions(-) create mode 100644 docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md create mode 100644 docs/doctoring/sandboxed-web-readiness-loopback-boundary.md diff --git a/ARCHITECTURE.md b/ARCHITECTURE.md index 6310abcfe..67813bc61 100644 --- a/ARCHITECTURE.md +++ b/ARCHITECTURE.md @@ -126,6 +126,9 @@ sequenceDiagram invents a hypothesis, observed result, or verdict. - Sandbox helpers copy the workspace, drop secret environment values unless explicitly allowlisted by **name**, and run subprocesses with `shell=False`. + Web E2E readiness URLs are loopback-only; see + [`docs/doctoring/sandboxed-web-readiness-loopback-boundary.md`](docs/doctoring/sandboxed-web-readiness-loopback-boundary.md) + and [`docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md`](docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md). - Logs and review receipts redact credential shapes (tokens, bearer values, known provider prefixes). They do not mask operational PII that the control plane must process. @@ -173,3 +176,5 @@ resolver conflict. — product-specific psychometric repair heartbeat and scientific gates. - [`docs/doctoring/exact-artifact-sbom-attestation.md`](docs/doctoring/exact-artifact-sbom-attestation.md) — current increment's attestation decision and APA 7th citations. +- [`docs/doctoring/sandboxed-web-readiness-loopback-boundary.md`](docs/doctoring/sandboxed-web-readiness-loopback-boundary.md) + — loopback-only web E2E readiness polling and APA 7th citations. diff --git a/CHANGELOG.md b/CHANGELOG.md index 1630c32d4..2fe137baa 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -70,6 +70,12 @@ Semantic Versioning where the repository publishes a release. ### Fixed +- Restricted sandboxed web E2E readiness polling to standard-library-classified + loopback addresses and literal `localhost`, including IPv6 `::1` and the + complete IPv4 loopback block, while rejecting userinfo, missing hosts, + IPv4-mapped public addresses, unspecified bind addresses, and metadata + destinations before any request is opened. + - Publish only the sanitized cumulative Strix report tree, avoiding a later copy of relative scanner output that could reintroduce known internal warning text into uploaded security evidence. diff --git a/docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md b/docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md new file mode 100644 index 000000000..403ce7fe0 --- /dev/null +++ b/docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md @@ -0,0 +1,9 @@ +# ADR-0003: Sandboxed web readiness loopback boundary + +- Status: accepted +- Date: 2026-08-25 +- Scope: ContextualWisdomLab/.github control-plane E2E sandbox +- Decision: Poll `--backend-ready-url` and `--frontend-ready-url` only after the URL is proven to be HTTP(S) loopback. Accept literal `localhost` or a standard-library loopback address, unwrap IPv4-mapped IPv6, reject userinfo and missing hosts, and keep redirects disabled. +- Ownership: `.github` owns the sandbox helper. Product repositories keep pointing readiness at their own loopback services. +- Figma File ID: N/A. This repository has no customer UI. +- Consequence: A review run cannot use the sandbox poller as an SSRF trampoline to metadata services or public hosts. Operators fix a rejected URL by pointing it at `127.0.0.1` or `::1`. Papers live in `docs/doctoring/sandboxed-web-readiness-loopback-boundary.md`. diff --git a/docs/doctoring/sandboxed-web-readiness-loopback-boundary.md b/docs/doctoring/sandboxed-web-readiness-loopback-boundary.md new file mode 100644 index 000000000..51071556d --- /dev/null +++ b/docs/doctoring/sandboxed-web-readiness-loopback-boundary.md @@ -0,0 +1,82 @@ +# Sandboxed web readiness loopback boundary + +## Decision + +`sandboxed_web_e2e.py` opens a readiness URL only after +`require_loopback_readiness_url` accepts it. The accepted destinations are +literal `localhost` (a trailing FQDN dot is stripped) or an address that +Python's standard-library `ipaddress` module classifies as loopback after +IPv4-mapped IPv6 addresses are unwrapped. Redirects remain disabled. + +This supports the complete IPv4 loopback block, including `127.0.0.2`, and +IPv6 `::1`. It rejects `0.0.0.0`, `::`, public hosts, `.localhost` +subdomains, cloud-metadata link-local addresses, missing hosts, and +userinfo-confused URLs such as `http://user@127.0.0.1/`. A mapped public +address such as `::ffff:8.8.8.8` cannot pass merely because it is IPv6. + +The boundary uses the standard library rather than a second address table. +It therefore follows the runtime's maintained special-purpose definitions and +keeps one fail-closed validation point before any network request. Do not add +individual non-loopback exceptions. + +This successor lands the same buyer-facing repair as ContextualWisdomLab/.github#1244 +on current `main` and keeps Strix classifier ownership out of the SSRF slice +(unlike ContextualWisdomLab/.github#1313). + +## Operator next action + +Point `--backend-ready-url` and `--frontend-ready-url` at the sandboxed +service on loopback. If readiness fails with `URL cannot target external +hostname`, replace the destination with `http://127.0.0.1:/...` or +`http://[::1]:/...` instead of opening the firewall or adding a +hostname exception. + +## Verification + +The regression exercises literal `localhost`, a trailing-dot `localhost.`, +`127.0.0.1`, another address in `127.0.0.0/8`, IPv6 `::1`, mapped loopback +`::ffff:127.0.0.1`, an unspecified address, a `.localhost` subdomain, a +public hostname, the common cloud metadata address, mapped public IPv6, +userinfo, and a missing host. Only the literal name avoids another DNS +resolution boundary. The existing no-redirect test continues to prove that +an allowed readiness endpoint cannot redirect the poller across the +boundary. + +```mermaid +flowchart TD + Url["Readiness URL"] + Scheme{"http or https?"} + Userinfo{"userinfo present?"} + Host{"localhost or loopback IP after mapped unwrap?"} + Open["Poll with redirects disabled"] + Reject["Fail closed before any request"] + + Url --> Scheme + Scheme -->|"no"| Reject + Scheme -->|"yes"| Userinfo + Userinfo -->|"yes"| Reject + Userinfo -->|"no"| Host + Host -->|"no"| Reject + Host -->|"yes"| Open +``` + +## References + +Berners-Lee, T., Fielding, R., & Masinter, L. (2005). *Uniform Resource +Identifier (URI): Generic syntax* (RFC 3986). Internet Engineering Task +Force. https://doi.org/10.17487/RFC3986 + +Cotton, B., Vegoda, L., Bonica, R., & Haberman, B. (2013). *Special-purpose +IP address registries* (RFC 6890). Internet Engineering Task Force. +https://doi.org/10.17487/RFC6890 + +Internet Assigned Numbers Authority. (2026). *IANA IPv4 special-purpose +address registry*. Retrieved August 25, 2026, from +https://www.iana.org/assignments/iana-ipv4-special-registry/iana-ipv4-special-registry.xhtml + +OWASP Foundation. (n.d.). *Server-side request forgery prevention cheat +sheet*. Retrieved August 25, 2026, from +https://cheatsheetseries.owasp.org/cheatsheets/Server_Side_Request_Forgery_Prevention_Cheat_Sheet.html + +Python Software Foundation. (2026). *ipaddress — IPv4/IPv6 manipulation +library*. https://docs.python.org/3/library/ipaddress.html diff --git a/scripts/ci/sandboxed_web_e2e.py b/scripts/ci/sandboxed_web_e2e.py index ae0c3105a..3332be709 100644 --- a/scripts/ci/sandboxed_web_e2e.py +++ b/scripts/ci/sandboxed_web_e2e.py @@ -3,6 +3,7 @@ from __future__ import annotations import argparse +import ipaddress import json import os import signal @@ -13,6 +14,7 @@ import tempfile import time import urllib.error +import urllib.parse import urllib.request from collections.abc import Sequence from dataclasses import dataclass @@ -115,12 +117,41 @@ def start_service(label: str, command: str, cwd: Path, env: dict[str, str], logs return Service(label=label, command=command, process=process, log_path=log_path) +def require_loopback_readiness_url(url: str) -> None: + """Reject a readiness URL that is not a local loopback HTTP(S) target. + + Operators should point ``--backend-ready-url`` and ``--frontend-ready-url`` + at the sandboxed service itself. Public hosts, cloud metadata addresses, + unspecified bind addresses, DNS names other than literal ``localhost``, + and userinfo-confused URLs are rejected before any request is opened. + IPv4-mapped IPv6 addresses are unwrapped and re-checked so + ``::ffff:8.8.8.8`` cannot bypass the loopback rule. + """ + parsed = urllib.parse.urlparse(url) + if parsed.scheme.lower() not in {"http", "https"}: + raise ValueError(f"URL must start with http:// or https://, got: {url}") + if parsed.username or parsed.password: + raise ValueError("URL cannot include userinfo") + hostname = (parsed.hostname or "").lower().rstrip(".") + if not hostname: + raise ValueError("URL must include a loopback hostname") + if hostname == "localhost": + return + try: + address = ipaddress.ip_address(hostname) + except ValueError as exc: + raise ValueError(f"URL cannot target external hostname: {hostname}") from exc + if address.version == 6 and address.ipv4_mapped is not None: + address = address.ipv4_mapped + if not address.is_loopback: + raise ValueError(f"URL cannot target external hostname: {hostname}") + + def wait_for_url(url: str, timeout: int, service: Service) -> bool: """Poll a readiness URL until it responds or the service exits.""" if not url: return True - if not (url.startswith("http://") or url.startswith("https://")): - raise ValueError(f"URL must start with http:// or https://, got: {url}") + require_loopback_readiness_url(url) deadline = time.monotonic() + timeout opener = urllib.request.build_opener(NoRedirectHandler()) while time.monotonic() < deadline: diff --git a/tests/test_sandboxed_web_e2e.py b/tests/test_sandboxed_web_e2e.py index 6e092c293..edd9b683c 100644 --- a/tests/test_sandboxed_web_e2e.py +++ b/tests/test_sandboxed_web_e2e.py @@ -105,13 +105,19 @@ def test_sandboxed_web_e2e_runs_services_and_does_not_mutate_source(tmp_path, ca def test_wait_helpers_and_service_cleanup_edges(monkeypatch, tmp_path): - """Small helper branches handle empty URLs, exited services, and hard cleanup.""" + """Small helper branches handle empty URLs, loopback readiness, and hard cleanup.""" exited = subprocess.Popen([sys.executable, "-c", ""], text=True) exited.wait(timeout=5) exited_service = sandboxed_web_e2e.Service("done", "true", exited, tmp_path / "missing.log") assert sandboxed_web_e2e.wait_for_url("", 1, exited_service) is True assert sandboxed_web_e2e.wait_for_url("http://127.0.0.1:1/", 1, exited_service) is False + assert sandboxed_web_e2e.wait_for_url("http://localhost:1/", 1, exited_service) is False + assert sandboxed_web_e2e.wait_for_url("http://localhost./health", 1, exited_service) is False + assert sandboxed_web_e2e.wait_for_url("http://127.0.0.2:1/", 1, exited_service) is False + assert sandboxed_web_e2e.wait_for_url("http://[::1]:1/health", 1, exited_service) is False + assert sandboxed_web_e2e.wait_for_url("HTTP://[::ffff:127.0.0.1]:1/", 1, exited_service) is False + assert sandboxed_web_e2e.wait_for_url("https://127.0.0.1:1/", 1, exited_service) is False with pytest.raises(ValueError, match="URL must start with http:// or https://"): sandboxed_web_e2e.wait_for_url("file:///etc/passwd", 1, exited_service) sandboxed_web_e2e.stop_service(exited_service) @@ -228,6 +234,33 @@ def open(self, url, timeout): assert sandboxed_web_e2e.tail_text(log_path).splitlines()[0] == "line-10" +def test_wait_for_url_rejects_non_loopback_and_confused_deputy_targets(tmp_path): + """Readiness polling must fail closed on public, metadata, and userinfo targets.""" + exited = subprocess.Popen([sys.executable, "-c", ""], text=True) + exited.wait(timeout=5) + exited_service = sandboxed_web_e2e.Service("done", "true", exited, tmp_path / "missing.log") + + with pytest.raises(ValueError, match="URL cannot target external hostname: example\\.com"): + sandboxed_web_e2e.wait_for_url("http://example.com/health", 1, exited_service) + with pytest.raises(ValueError, match="URL cannot target external hostname: app\\.localhost"): + sandboxed_web_e2e.wait_for_url("http://app.localhost:8000/health", 1, exited_service) + with pytest.raises(ValueError, match="URL cannot target external hostname: 169\\.254\\.169\\.254"): + sandboxed_web_e2e.wait_for_url("http://169.254.169.254/latest/meta-data/", 1, exited_service) + with pytest.raises(ValueError, match="URL cannot target external hostname: 0\\.0\\.0\\.0"): + sandboxed_web_e2e.wait_for_url("http://0.0.0.0:8000/health", 1, exited_service) + with pytest.raises(ValueError, match="URL cannot target external hostname: ::"): + sandboxed_web_e2e.wait_for_url("http://[::]/", 1, exited_service) + with pytest.raises(ValueError, match="URL cannot target external hostname: ::ffff:8\\.8\\.8\\.8"): + sandboxed_web_e2e.wait_for_url("http://[::ffff:8.8.8.8]/", 1, exited_service) + with pytest.raises(ValueError, match="URL cannot include userinfo"): + sandboxed_web_e2e.wait_for_url("http://user@127.0.0.1/", 1, exited_service) + with pytest.raises(ValueError, match="URL cannot include userinfo"): + sandboxed_web_e2e.wait_for_url("http://:pass@127.0.0.1/", 1, exited_service) + with pytest.raises(ValueError, match="URL must include a loopback hostname"): + sandboxed_web_e2e.wait_for_url("http:///health", 1, exited_service) + sandboxed_web_e2e.stop_service(exited_service) + + def test_no_redirect_handler_raises_httperror_without_following(): """Readiness checks must raise HTTPError on redirects to prevent attacker-controlled internal URLs.""" import urllib.error From 3f0c3ac5c018770c32a80663a28c1f499ddfa1ba Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 00:14:53 +0000 Subject: [PATCH 02/13] fix(e2e): resolve localhost and require loopback answers A poisoned hosts file could map the literal name to a public A/AAAA record. Resolve localhost before opening the readiness URL and fail closed unless every answer is loopback after IPv4-mapped unwrap. Co-authored-by: Seongho Bae --- CHANGELOG.md | 7 +-- ...ndboxed-web-readiness-loopback-boundary.md | 2 +- ...ndboxed-web-readiness-loopback-boundary.md | 15 +++--- scripts/ci/sandboxed_web_e2e.py | 41 +++++++++++---- tests/test_sandboxed_web_e2e.py | 50 +++++++++++++++++++ 5 files changed, 95 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2fe137baa..302072af7 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -72,9 +72,10 @@ Semantic Versioning where the repository publishes a release. - Restricted sandboxed web E2E readiness polling to standard-library-classified loopback addresses and literal `localhost`, including IPv6 `::1` and the - complete IPv4 loopback block, while rejecting userinfo, missing hosts, - IPv4-mapped public addresses, unspecified bind addresses, and metadata - destinations before any request is opened. + complete IPv4 loopback block, resolving `localhost` so every answer must + remain loopback, while rejecting userinfo, missing hosts, IPv4-mapped + public addresses, unspecified bind addresses, and metadata destinations + before any request is opened. - Publish only the sanitized cumulative Strix report tree, avoiding a later copy of relative scanner output that could reintroduce known internal warning diff --git a/docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md b/docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md index 403ce7fe0..543270414 100644 --- a/docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md +++ b/docs/adr/0003-sandboxed-web-readiness-loopback-boundary.md @@ -3,7 +3,7 @@ - Status: accepted - Date: 2026-08-25 - Scope: ContextualWisdomLab/.github control-plane E2E sandbox -- Decision: Poll `--backend-ready-url` and `--frontend-ready-url` only after the URL is proven to be HTTP(S) loopback. Accept literal `localhost` or a standard-library loopback address, unwrap IPv4-mapped IPv6, reject userinfo and missing hosts, and keep redirects disabled. +- Decision: Poll `--backend-ready-url` and `--frontend-ready-url` only after the URL is proven to be HTTP(S) loopback. Accept literal `localhost` or a standard-library loopback address, resolve `localhost` and require every answer to be loopback, unwrap IPv4-mapped IPv6, reject userinfo and missing hosts, and keep redirects disabled. - Ownership: `.github` owns the sandbox helper. Product repositories keep pointing readiness at their own loopback services. - Figma File ID: N/A. This repository has no customer UI. - Consequence: A review run cannot use the sandbox poller as an SSRF trampoline to metadata services or public hosts. Operators fix a rejected URL by pointing it at `127.0.0.1` or `::1`. Papers live in `docs/doctoring/sandboxed-web-readiness-loopback-boundary.md`. diff --git a/docs/doctoring/sandboxed-web-readiness-loopback-boundary.md b/docs/doctoring/sandboxed-web-readiness-loopback-boundary.md index 51071556d..65438e75d 100644 --- a/docs/doctoring/sandboxed-web-readiness-loopback-boundary.md +++ b/docs/doctoring/sandboxed-web-readiness-loopback-boundary.md @@ -6,7 +6,10 @@ `require_loopback_readiness_url` accepts it. The accepted destinations are literal `localhost` (a trailing FQDN dot is stripped) or an address that Python's standard-library `ipaddress` module classifies as loopback after -IPv4-mapped IPv6 addresses are unwrapped. Redirects remain disabled. +IPv4-mapped IPv6 addresses are unwrapped. Literal `localhost` is then +resolved; every A/AAAA answer must itself be loopback, so a poisoned hosts +file cannot smuggle a public address through the name allowlist. Redirects +remain disabled. This supports the complete IPv4 loopback block, including `127.0.0.2`, and IPv6 `::1`. It rejects `0.0.0.0`, `::`, public hosts, `.localhost` @@ -37,17 +40,17 @@ The regression exercises literal `localhost`, a trailing-dot `localhost.`, `127.0.0.1`, another address in `127.0.0.0/8`, IPv6 `::1`, mapped loopback `::ffff:127.0.0.1`, an unspecified address, a `.localhost` subdomain, a public hostname, the common cloud metadata address, mapped public IPv6, -userinfo, and a missing host. Only the literal name avoids another DNS -resolution boundary. The existing no-redirect test continues to prove that -an allowed readiness endpoint cannot redirect the poller across the -boundary. +userinfo, a missing host, and poisoned localhost resolution (public A, +mapped public AAAA, empty answers, resolver errors, and non-IP answers). +The existing no-redirect test continues to prove that an allowed readiness +endpoint cannot redirect the poller across the boundary. ```mermaid flowchart TD Url["Readiness URL"] Scheme{"http or https?"} Userinfo{"userinfo present?"} - Host{"localhost or loopback IP after mapped unwrap?"} + Host{"loopback IP, or localhost whose every resolved answer is loopback?"} Open["Poll with redirects disabled"] Reject["Fail closed before any request"] diff --git a/scripts/ci/sandboxed_web_e2e.py b/scripts/ci/sandboxed_web_e2e.py index 3332be709..978113e69 100644 --- a/scripts/ci/sandboxed_web_e2e.py +++ b/scripts/ci/sandboxed_web_e2e.py @@ -9,6 +9,7 @@ import signal import shutil import shlex +import socket import subprocess import sys import tempfile @@ -117,6 +118,30 @@ def start_service(label: str, command: str, cwd: Path, env: dict[str, str], logs return Service(label=label, command=command, process=process, log_path=log_path) +def _require_loopback_ip_text(ip_text: str, hostname: str) -> None: + """Reject a literal or resolved address that is not loopback.""" + try: + address = ipaddress.ip_address(ip_text) + except ValueError as exc: + raise ValueError(f"URL cannot target external hostname: {hostname}") from exc + if address.version == 6 and address.ipv4_mapped is not None: + address = address.ipv4_mapped + if not address.is_loopback: + raise ValueError(f"URL cannot target external hostname: {hostname}") + + +def _require_resolved_loopback_hostname(hostname: str) -> None: + """Resolve a literal localhost name and require every answer to be loopback.""" + try: + results = socket.getaddrinfo(hostname, None) + except OSError as exc: + raise ValueError(f"URL cannot target unresolved hostname: {hostname}") from exc + if not results: + raise ValueError(f"URL cannot target unresolved hostname: {hostname}") + for result in results: + _require_loopback_ip_text(result[4][0], hostname) + + def require_loopback_readiness_url(url: str) -> None: """Reject a readiness URL that is not a local loopback HTTP(S) target. @@ -124,8 +149,10 @@ def require_loopback_readiness_url(url: str) -> None: at the sandboxed service itself. Public hosts, cloud metadata addresses, unspecified bind addresses, DNS names other than literal ``localhost``, and userinfo-confused URLs are rejected before any request is opened. - IPv4-mapped IPv6 addresses are unwrapped and re-checked so - ``::ffff:8.8.8.8`` cannot bypass the loopback rule. + Literal ``localhost`` is resolved and every answer must be loopback, so a + poisoned hosts file cannot smuggle a public A/AAAA record through the + name allowlist. IPv4-mapped IPv6 addresses are unwrapped and re-checked + so ``::ffff:8.8.8.8`` cannot bypass the loopback rule. """ parsed = urllib.parse.urlparse(url) if parsed.scheme.lower() not in {"http", "https"}: @@ -136,15 +163,9 @@ def require_loopback_readiness_url(url: str) -> None: if not hostname: raise ValueError("URL must include a loopback hostname") if hostname == "localhost": + _require_resolved_loopback_hostname(hostname) return - try: - address = ipaddress.ip_address(hostname) - except ValueError as exc: - raise ValueError(f"URL cannot target external hostname: {hostname}") from exc - if address.version == 6 and address.ipv4_mapped is not None: - address = address.ipv4_mapped - if not address.is_loopback: - raise ValueError(f"URL cannot target external hostname: {hostname}") + _require_loopback_ip_text(hostname, hostname) def wait_for_url(url: str, timeout: int, service: Service) -> bool: diff --git a/tests/test_sandboxed_web_e2e.py b/tests/test_sandboxed_web_e2e.py index edd9b683c..39339a316 100644 --- a/tests/test_sandboxed_web_e2e.py +++ b/tests/test_sandboxed_web_e2e.py @@ -261,6 +261,56 @@ def test_wait_for_url_rejects_non_loopback_and_confused_deputy_targets(tmp_path) sandboxed_web_e2e.stop_service(exited_service) +def test_localhost_resolution_must_stay_loopback(monkeypatch, tmp_path): + """Literal localhost is allowed only when every resolved address is loopback.""" + exited = subprocess.Popen([sys.executable, "-c", ""], text=True) + exited.wait(timeout=5) + exited_service = sandboxed_web_e2e.Service("done", "true", exited, tmp_path / "missing.log") + + monkeypatch.setattr( + sandboxed_web_e2e.socket, + "getaddrinfo", + lambda host, port: [(0, 0, 0, "", ("8.8.8.8", 0))], + ) + with pytest.raises(ValueError, match="URL cannot target external hostname: localhost"): + sandboxed_web_e2e.wait_for_url("http://localhost:1/", 1, exited_service) + + monkeypatch.setattr(sandboxed_web_e2e.socket, "getaddrinfo", lambda host, port: []) + with pytest.raises(ValueError, match="URL cannot target unresolved hostname: localhost"): + sandboxed_web_e2e.wait_for_url("http://localhost:1/", 1, exited_service) + + def _unresolved(host, port): + raise socket.gaierror("name not known") + + monkeypatch.setattr(sandboxed_web_e2e.socket, "getaddrinfo", _unresolved) + with pytest.raises(ValueError, match="URL cannot target unresolved hostname: localhost"): + sandboxed_web_e2e.wait_for_url("http://localhost:1/", 1, exited_service) + + monkeypatch.setattr( + sandboxed_web_e2e.socket, + "getaddrinfo", + lambda host, port: [(0, 0, 0, "", ("not-an-ip", 0))], + ) + with pytest.raises(ValueError, match="URL cannot target external hostname: localhost"): + sandboxed_web_e2e.wait_for_url("http://localhost:1/", 1, exited_service) + + monkeypatch.setattr( + sandboxed_web_e2e.socket, + "getaddrinfo", + lambda host, port: [(0, 0, 0, "", ("::ffff:8.8.8.8", 0))], + ) + with pytest.raises(ValueError, match="URL cannot target external hostname: localhost"): + sandboxed_web_e2e.wait_for_url("http://localhost:1/", 1, exited_service) + + monkeypatch.setattr( + sandboxed_web_e2e.socket, + "getaddrinfo", + lambda host, port: [(0, 0, 0, "", ("::ffff:127.0.0.1", 0))], + ) + assert sandboxed_web_e2e.wait_for_url("http://localhost:1/", 1, exited_service) is False + sandboxed_web_e2e.stop_service(exited_service) + + def test_no_redirect_handler_raises_httperror_without_following(): """Readiness checks must raise HTTPError on redirects to prevent attacker-controlled internal URLs.""" import urllib.error From 0ef296dcb88d2740bb390b181bfd0fe49979ac84 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 03:05:13 +0000 Subject: [PATCH 03/13] fix(strix): strip MODEL QUALITY WARNING boxes without fail-open #1311@7c6d80c used a cross-box regex that also deleted a preceding Fatal box. Classify a sanitized console copy so gate-last-attempt.log stays raw, and keep only same-box banner removal for reports. Co-authored-by: Seongho Bae --- CHANGELOG.md | 3 + scripts/ci/strix_quick_gate.sh | 101 +++++++++++++- scripts/ci/test_strix_quick_gate.sh | 196 ++++++++++++++++++++++++++++ 3 files changed, 296 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 302072af7..1d55cfbdc 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -76,6 +76,9 @@ Semantic Versioning where the repository publishes a release. remain loopback, while rejecting userinfo, missing hosts, IPv4-mapped public addresses, unspecified bind addresses, and metadata destinations before any request is opened. +- Strip only a complete Strix `MODEL QUALITY WARNING` box before + infrastructure-signal matching, keep a preceding Fatal/Denied box, and + classify a sanitized console copy so `gate-last-attempt.log` stays raw. - Publish only the sanitized cumulative Strix report tree, avoiding a later copy of relative scanner output that could reintroduce known internal warning diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index 36ec3e5f8..8bd498ff5 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -151,6 +151,63 @@ preserve_attempt_log() { fi } +# Write a classification copy of the raw console transcript. Never mutate +# $STRIX_LOG: publish_artifact_reports copies it to gate-last-attempt.log. +# Remove only a complete box that itself contains MODEL QUALITY WARNING. +# A cross-box wildcard from the first ╭ to a later ╰ would erase a +# preceding Fatal/Denied box (fail-open). +sanitize_strix_console_log() { + local src="$1" + local dest="$2" + if [ -z "$src" ] || [ -z "$dest" ] || [ ! -f "$src" ] || [ -L "$src" ]; then + return 0 + fi + if [ -L "$dest" ]; then + return 0 + fi + python3 - "$src" "$dest" <<'PY' +from pathlib import Path +import sys + +src = Path(sys.argv[1]) +dest = Path(sys.argv[2]) + + +def strip_model_quality_warning_boxes(text: str) -> str: + pieces: list[str] = [] + index = 0 + length = len(text) + while index < length: + start = text.find("╭", index) + if start < 0: + pieces.append(text[index:]) + break + pieces.append(text[index:start]) + close = text.find("╰", start) + if close < 0: + pieces.append(text[start:]) + break + line_end = text.find("\n", close) + if line_end < 0: + box = text[start:] + index = length + else: + box = text[start : line_end + 1] + index = line_end + 1 + if "MODEL QUALITY WARNING" in box: + continue + pieces.append(box) + return "".join(pieces) + + +try: + text = src.read_text(encoding="utf-8") +except UnicodeDecodeError: + raise SystemExit(0) +dest.write_text(strip_model_quality_warning_boxes(text), encoding="utf-8") +PY +} + sanitize_known_strix_report_warnings() { local report_root for report_root in "$@"; do @@ -174,6 +231,33 @@ known_internal_warning = re.compile( ) +def strip_model_quality_warning_boxes(text: str) -> str: + pieces: list[str] = [] + index = 0 + length = len(text) + while index < length: + start = text.find("╭", index) + if start < 0: + pieces.append(text[index:]) + break + pieces.append(text[index:start]) + close = text.find("╰", start) + if close < 0: + pieces.append(text[start:]) + break + line_end = text.find("\n", close) + if line_end < 0: + box = text[start:] + index = length + else: + box = text[start : line_end + 1] + index = line_end + 1 + if "MODEL QUALITY WARNING" in box: + continue + pieces.append(box) + return "".join(pieces) + + def iter_report_logs(root: Path): for current_root, dir_names, file_names in os.walk(root, topdown=True, followlinks=False): current_path = Path(current_root) @@ -191,12 +275,15 @@ def iter_report_logs(root: Path): for log_path in iter_report_logs(root): try: - lines = log_path.read_text(encoding="utf-8").splitlines(keepends=True) + original = log_path.read_text(encoding="utf-8") except UnicodeDecodeError: continue + text = strip_model_quality_warning_boxes(original) + lines = text.splitlines(keepends=True) filtered = [line for line in lines if not known_internal_warning.match(line)] - if filtered != lines: - log_path.write_text("".join(filtered), encoding="utf-8") + text = "".join(filtered) + if text != original: + log_path.write_text(text, encoding="utf-8") PY done } @@ -2790,6 +2877,12 @@ PY fi fi preserve_attempt_log "$model" "$rc" + local classified_log="${STRIX_LOG}.classified" + sanitize_strix_console_log "$STRIX_LOG" "$classified_log" + local inspect_log="$STRIX_LOG" + if [ -f "$classified_log" ] && [ ! -L "$classified_log" ]; then + inspect_log="$classified_log" + fi sanitize_known_strix_report_warnings "$ACTIVE_REPORTS_DIR" "${resolved_target_path%/}/strix_runs" local report_failure_signal=0 @@ -2798,7 +2891,7 @@ PY echo "Strix report artifacts emitted warning/fatal/denied/timeout output; failing closed." | tee -a "$STRIX_LOG" >&2 fi - if [ "$report_failure_signal" -eq 1 ] || has_detected_infrastructure_error; then + if [ "$report_failure_signal" -eq 1 ] || STRIX_LOG="$inspect_log" has_detected_infrastructure_error; then INFRA_ERROR_DETECTED=1 if [ "$rc" -eq 0 ] && provider_signal_fail_closed_enabled; then echo "Strix run emitted provider infrastructure or failure-signal output; failing closed." >&2 diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 945eb3fb3..40bab16c2 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -4542,6 +4542,80 @@ EOS echo "scan ok but unknown report warning remains" exit 0 ;; + console-model-quality-warning-banner-sanitized) + # Cosmetic startup banner on a non-frontier model. Classification + # must ignore only that box so a clean 0-finding scan succeeds. + cat <<'EOS' +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ │ +│ MODEL QUALITY WARNING │ +│ │ +│ 'vertex_ai/console-model-quality-warning-banner-sanitized' is not a │ +│ recommended frontier model for Strix. │ +│ │ +│ You can continue, but weaker models may miss vulnerabilities or produce │ +│ lower-quality findings. │ +│ │ +╰──────────────────────────────────────────────────────────────────────────────╯ +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ │ +│ Penetration test completed │ +│ │ +│ Vulnerabilities 0 │ +│ │ +╰──────────────────────────────────────────────────────────────────────────────╯ +EOS + exit 0 + ;; + console-model-quality-warning-preserves-prior-fatal-box) + # A genuine Fatal box must survive when it precedes the cosmetic + # banner. A cross-box regex would delete both and fail open. + cat <<'EOS' +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ Fatal: LLM CONNECTION FAILED after 3 retries │ +╰──────────────────────────────────────────────────────────────────────────────╯ +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ │ +│ MODEL QUALITY WARNING │ +│ │ +│ 'vertex_ai/console-model-quality-warning-preserves-prior-fatal-box' is not │ +│ a recommended frontier model for Strix. │ +│ │ +╰──────────────────────────────────────────────────────────────────────────────╯ +Penetration test completed +Vulnerabilities 0 (No exploitable vulnerabilities detected) +EOS + exit 0 + ;; + report-model-quality-warning-banner-sanitized) + mkdir -p "$STRIX_REPORTS_DIR/fake-model-quality-banner" + cat >"$STRIX_REPORTS_DIR/fake-model-quality-banner/strix.log" <<'EOS' +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ MODEL QUALITY WARNING │ +│ 'vertex_ai/report-model-quality-warning-banner-sanitized' is not a │ +│ recommended frontier model for Strix. │ +╰──────────────────────────────────────────────────────────────────────────────╯ +2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) +EOS + echo "scan ok with sanitized model-quality report banner" + exit 0 + ;; + report-model-quality-warning-preserves-prior-fatal-box) + mkdir -p "$STRIX_REPORTS_DIR/fake-model-quality-fatal" + cat >"$STRIX_REPORTS_DIR/fake-model-quality-fatal/strix.log" <<'EOS' +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ Fatal: LLM CONNECTION FAILED after 3 retries │ +╰──────────────────────────────────────────────────────────────────────────────╯ +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ MODEL QUALITY WARNING │ +│ 'vertex_ai/report-model-quality-warning-preserves-prior-fatal-box' is not │ +│ a recommended frontier model for Strix. │ +╰──────────────────────────────────────────────────────────────────────────────╯ +2026-06-18 13:10:44.089 INFO strix-pr-scope-example - strix.tools.finish.tool: finish_scan: completed scan with 0 vulnerability report(s) +EOS + echo "scan ok but prior fatal report box must remain" + exit 0 + ;; bare-timeout-with-provider-marker) # Emit bare "Connection timed out" alongside a provider marker so # is_timeout_error() matches the Tier 3 branch gated on @@ -5835,6 +5909,52 @@ PY "scenario=$scenario does not rewrite logs through symlinked report directories" fi + if [ "$scenario" = "console-model-quality-warning-banner-sanitized" ]; then + assert_file_contains \ + "$repo_root_dir/strix_runs/gate-last-attempt.log" \ + "MODEL QUALITY WARNING" \ + "scenario=$scenario publishes the raw console including the cosmetic banner" + local attempt_log="" + attempt_log="$(find "$repo_root_dir/strix_runs/gate-attempts" -type f -name '*.log' -print -quit 2>/dev/null || true)" + if [ -z "$attempt_log" ]; then + record_failure "scenario=$scenario should preserve a raw per-attempt log" + else + assert_file_contains \ + "$attempt_log" \ + "MODEL QUALITY WARNING" \ + "scenario=$scenario keeps raw per-attempt logs" + fi + fi + + if [ "$scenario" = "console-model-quality-warning-preserves-prior-fatal-box" ]; then + assert_file_contains \ + "$repo_root_dir/strix_runs/gate-last-attempt.log" \ + "Fatal: LLM CONNECTION FAILED after 3 retries" \ + "scenario=$scenario keeps the raw Fatal box in the published last-attempt log" + fi + + if [ "$scenario" = "report-model-quality-warning-banner-sanitized" ]; then + assert_file_not_contains \ + "$repo_root_dir/strix_runs/fake-model-quality-banner/strix.log" \ + "MODEL QUALITY WARNING" \ + "scenario=$scenario strips only the cosmetic report banner" + assert_file_contains \ + "$repo_root_dir/strix_runs/fake-model-quality-banner/strix.log" \ + "finish_scan: completed scan with 0 vulnerability report(s)" \ + "scenario=$scenario keeps non-banner report evidence" + fi + + if [ "$scenario" = "report-model-quality-warning-preserves-prior-fatal-box" ]; then + assert_file_contains \ + "$repo_root_dir/strix_runs/fake-model-quality-fatal/strix.log" \ + "Fatal: LLM CONNECTION FAILED after 3 retries" \ + "scenario=$scenario keeps a preceding Fatal report box" + assert_file_not_contains \ + "$repo_root_dir/strix_runs/fake-model-quality-fatal/strix.log" \ + "MODEL QUALITY WARNING" \ + "scenario=$scenario still strips the cosmetic report banner" + fi + if [ "$scenario" = "report-known-internal-warning-variant-sanitized" ]; then assert_file_not_contains \ "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ @@ -6368,6 +6488,46 @@ run_filtered_gate_case_if_requested() { "vertex_ai/report-known-internal-warning-sanitized" \ "" ;; + console-model-quality-warning-banner-sanitized) + run_gate_case "$STRIX_TEST_CASE_FILTER" \ + "vertex_ai/console-model-quality-warning-banner-sanitized" \ + "" \ + "0" \ + "Strix run succeeded for model 'vertex_ai/console-model-quality-warning-banner-sanitized'" \ + "1" \ + "vertex_ai/console-model-quality-warning-banner-sanitized" \ + "" + ;; + console-model-quality-warning-preserves-prior-fatal-box) + run_gate_case "$STRIX_TEST_CASE_FILTER" \ + "vertex_ai/console-model-quality-warning-preserves-prior-fatal-box" \ + "" \ + "1" \ + "Strix run emitted provider infrastructure or failure-signal output; failing closed." \ + "1" \ + "vertex_ai/console-model-quality-warning-preserves-prior-fatal-box" \ + "" + ;; + report-model-quality-warning-banner-sanitized) + run_gate_case "$STRIX_TEST_CASE_FILTER" \ + "vertex_ai/report-model-quality-warning-banner-sanitized" \ + "" \ + "0" \ + "Strix run succeeded for model 'vertex_ai/report-model-quality-warning-banner-sanitized'" \ + "1" \ + "vertex_ai/report-model-quality-warning-banner-sanitized" \ + "" + ;; + report-model-quality-warning-preserves-prior-fatal-box) + run_gate_case "$STRIX_TEST_CASE_FILTER" \ + "vertex_ai/report-model-quality-warning-preserves-prior-fatal-box" \ + "" \ + "1" \ + "Strix report artifacts emitted warning/fatal/denied/timeout output; failing closed." \ + "1" \ + "vertex_ai/report-model-quality-warning-preserves-prior-fatal-box" \ + "" + ;; provider-fatal-success-signal | provider-warning-success-signal) run_gate_case "$STRIX_TEST_CASE_FILTER" \ "vertex_ai/$STRIX_TEST_CASE_FILTER" \ @@ -10434,6 +10594,42 @@ run_gate_case "report-known-internal-warning-sanitized" \ "" \ "1" +run_gate_case "console-model-quality-warning-banner-sanitized" \ + "vertex_ai/console-model-quality-warning-banner-sanitized" \ + "" \ + "0" \ + "Strix run succeeded for model 'vertex_ai/console-model-quality-warning-banner-sanitized'" \ + "1" \ + "vertex_ai/console-model-quality-warning-banner-sanitized" \ + "" + +run_gate_case "console-model-quality-warning-preserves-prior-fatal-box" \ + "vertex_ai/console-model-quality-warning-preserves-prior-fatal-box" \ + "" \ + "1" \ + "Strix run emitted provider infrastructure or failure-signal output; failing closed." \ + "1" \ + "vertex_ai/console-model-quality-warning-preserves-prior-fatal-box" \ + "" + +run_gate_case "report-model-quality-warning-banner-sanitized" \ + "vertex_ai/report-model-quality-warning-banner-sanitized" \ + "" \ + "0" \ + "Strix run succeeded for model 'vertex_ai/report-model-quality-warning-banner-sanitized'" \ + "1" \ + "vertex_ai/report-model-quality-warning-banner-sanitized" \ + "" + +run_gate_case "report-model-quality-warning-preserves-prior-fatal-box" \ + "vertex_ai/report-model-quality-warning-preserves-prior-fatal-box" \ + "" \ + "1" \ + "Strix report artifacts emitted warning/fatal/denied/timeout output; failing closed." \ + "1" \ + "vertex_ai/report-model-quality-warning-preserves-prior-fatal-box" \ + "" + run_gate_case "report-known-internal-warning-variant-sanitized" \ "vertex_ai/report-known-internal-warning-variant-sanitized" \ "" \ From 9397bf4a2317127807ad7942fed88a7057db3ca1 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 04:07:14 +0000 Subject: [PATCH 04/13] fix(strix): keep same-box failure evidence when stripping the banner heading Deleting a whole MODEL QUALITY WARNING box hid a Fatal or provider Warning emitted on the next line of that same box. Strip only the cosmetic heading line, keep the classified console copy so last-attempt stays raw, and cover the same-box fail-closed path. Co-authored-by: Seongho Bae --- CHANGELOG.md | 7 +-- scripts/ci/strix_quick_gate.sh | 80 ++++++++++------------------- scripts/ci/test_strix_quick_gate.sh | 80 +++++++++++++++++++++++++++++ 3 files changed, 111 insertions(+), 56 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1d55cfbdc..163cad909 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -76,9 +76,10 @@ Semantic Versioning where the repository publishes a release. remain loopback, while rejecting userinfo, missing hosts, IPv4-mapped public addresses, unspecified bind addresses, and metadata destinations before any request is opened. -- Strip only a complete Strix `MODEL QUALITY WARNING` box before - infrastructure-signal matching, keep a preceding Fatal/Denied box, and - classify a sanitized console copy so `gate-last-attempt.log` stays raw. +- Strip only the Strix `MODEL QUALITY WARNING` heading line before + infrastructure-signal matching so a Fatal/Denied/Warning in the same + box stays visible, keep a preceding Fatal/Denied box, and classify a + sanitized console copy so `gate-last-attempt.log` stays raw. - Publish only the sanitized cumulative Strix report tree, avoiding a later copy of relative scanner output that could reintroduce known internal warning diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index 8bd498ff5..1eb4cd137 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -153,9 +153,10 @@ preserve_attempt_log() { # Write a classification copy of the raw console transcript. Never mutate # $STRIX_LOG: publish_artifact_reports copies it to gate-last-attempt.log. -# Remove only a complete box that itself contains MODEL QUALITY WARNING. -# A cross-box wildcard from the first ╭ to a later ╰ would erase a -# preceding Fatal/Denied box (fail-open). +# Remove only the cosmetic MODEL QUALITY WARNING heading line. Deleting a +# whole box would hide a Fatal/Denied/Warning emitted in that same box +# (fail-open). A cross-box wildcard from the first ╭ to a later ╰ would +# also erase a preceding Fatal/Denied box. sanitize_strix_console_log() { local src="$1" local dest="$2" @@ -167,44 +168,30 @@ sanitize_strix_console_log() { fi python3 - "$src" "$dest" <<'PY' from pathlib import Path +import re import sys src = Path(sys.argv[1]) dest = Path(sys.argv[2]) +model_quality_heading = re.compile(r"│[ \t]*MODEL QUALITY WARNING[ \t]*│") +ansi_csi = re.compile(r"\x1b\[[0-?]*[ -/]*[@-~]") -def strip_model_quality_warning_boxes(text: str) -> str: - pieces: list[str] = [] - index = 0 - length = len(text) - while index < length: - start = text.find("╭", index) - if start < 0: - pieces.append(text[index:]) - break - pieces.append(text[index:start]) - close = text.find("╰", start) - if close < 0: - pieces.append(text[start:]) - break - line_end = text.find("\n", close) - if line_end < 0: - box = text[start:] - index = length - else: - box = text[start : line_end + 1] - index = line_end + 1 - if "MODEL QUALITY WARNING" in box: +def strip_model_quality_warning_heading(text: str) -> str: + kept: list[str] = [] + for line in text.splitlines(keepends=True): + plain = ansi_csi.sub("", line.rstrip("\r\n")) + if model_quality_heading.fullmatch(plain): continue - pieces.append(box) - return "".join(pieces) + kept.append(line) + return "".join(kept) try: text = src.read_text(encoding="utf-8") except UnicodeDecodeError: raise SystemExit(0) -dest.write_text(strip_model_quality_warning_boxes(text), encoding="utf-8") +dest.write_text(strip_model_quality_warning_heading(text), encoding="utf-8") PY } @@ -231,31 +218,18 @@ known_internal_warning = re.compile( ) -def strip_model_quality_warning_boxes(text: str) -> str: - pieces: list[str] = [] - index = 0 - length = len(text) - while index < length: - start = text.find("╭", index) - if start < 0: - pieces.append(text[index:]) - break - pieces.append(text[index:start]) - close = text.find("╰", start) - if close < 0: - pieces.append(text[start:]) - break - line_end = text.find("\n", close) - if line_end < 0: - box = text[start:] - index = length - else: - box = text[start : line_end + 1] - index = line_end + 1 - if "MODEL QUALITY WARNING" in box: +model_quality_heading = re.compile(r"│[ \t]*MODEL QUALITY WARNING[ \t]*│") +ansi_csi = re.compile(r"\x1b\[[0-?]*[ -/]*[@-~]") + + +def strip_model_quality_warning_heading(text: str) -> str: + kept = [] + for line in text.splitlines(keepends=True): + plain = ansi_csi.sub("", line.rstrip("\r\n")) + if model_quality_heading.fullmatch(plain): continue - pieces.append(box) - return "".join(pieces) + kept.append(line) + return "".join(kept) def iter_report_logs(root: Path): @@ -278,7 +252,7 @@ for log_path in iter_report_logs(root): original = log_path.read_text(encoding="utf-8") except UnicodeDecodeError: continue - text = strip_model_quality_warning_boxes(original) + text = strip_model_quality_warning_heading(original) lines = text.splitlines(keepends=True) filtered = [line for line in lines if not known_internal_warning.match(line)] text = "".join(filtered) diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 40bab16c2..b64762fe3 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -4616,6 +4616,30 @@ EOS echo "scan ok but prior fatal report box must remain" exit 0 ;; + console-model-quality-warning-preserves-same-box-failure) + # A whole-box delete would drop this Fatal with the heading + # and fail open. Strip only the cosmetic heading line. + cat <<'EOS' +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ MODEL QUALITY WARNING │ +│ Fatal: LLM CONNECTION FAILED after 3 retries │ +╰──────────────────────────────────────────────────────────────────────────────╯ +Penetration test completed +Vulnerabilities 0 (No exploitable vulnerabilities detected) +EOS + exit 0 + ;; + report-model-quality-warning-preserves-same-box-failure) + mkdir -p "$STRIX_REPORTS_DIR/fake-model-quality-same-box-failure" + cat >"$STRIX_REPORTS_DIR/fake-model-quality-same-box-failure/strix.log" <<'EOS' +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ MODEL QUALITY WARNING │ +│ Provider WARNING: report evidence is incomplete │ +╰──────────────────────────────────────────────────────────────────────────────╯ +EOS + echo "scan returned zero findings with incomplete evidence in the banner box" + exit 0 + ;; bare-timeout-with-provider-marker) # Emit bare "Connection timed out" alongside a provider marker so # is_timeout_error() matches the Tier 3 branch gated on @@ -5955,6 +5979,24 @@ PY "scenario=$scenario still strips the cosmetic report banner" fi + if [ "$scenario" = "console-model-quality-warning-preserves-same-box-failure" ]; then + assert_file_contains \ + "$repo_root_dir/strix_runs/gate-last-attempt.log" \ + "Fatal: LLM CONNECTION FAILED after 3 retries" \ + "scenario=$scenario keeps the raw same-box Fatal line in last-attempt" + fi + + if [ "$scenario" = "report-model-quality-warning-preserves-same-box-failure" ]; then + assert_file_contains \ + "$repo_root_dir/strix_runs/fake-model-quality-same-box-failure/strix.log" \ + "Provider WARNING: report evidence is incomplete" \ + "scenario=$scenario keeps same-box provider warning evidence" + assert_file_not_contains \ + "$repo_root_dir/strix_runs/fake-model-quality-same-box-failure/strix.log" \ + "MODEL QUALITY WARNING" \ + "scenario=$scenario still strips only the cosmetic heading" + fi + if [ "$scenario" = "report-known-internal-warning-variant-sanitized" ]; then assert_file_not_contains \ "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ @@ -6528,6 +6570,26 @@ run_filtered_gate_case_if_requested() { "vertex_ai/report-model-quality-warning-preserves-prior-fatal-box" \ "" ;; + console-model-quality-warning-preserves-same-box-failure) + run_gate_case "$STRIX_TEST_CASE_FILTER" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-failure" \ + "" \ + "1" \ + "Strix run emitted provider infrastructure or failure-signal output; failing closed." \ + "1" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-failure" \ + "" + ;; + report-model-quality-warning-preserves-same-box-failure) + run_gate_case "$STRIX_TEST_CASE_FILTER" \ + "vertex_ai/report-model-quality-warning-preserves-same-box-failure" \ + "" \ + "1" \ + "Strix report artifacts emitted warning/fatal/denied/timeout output; failing closed." \ + "1" \ + "vertex_ai/report-model-quality-warning-preserves-same-box-failure" \ + "" + ;; provider-fatal-success-signal | provider-warning-success-signal) run_gate_case "$STRIX_TEST_CASE_FILTER" \ "vertex_ai/$STRIX_TEST_CASE_FILTER" \ @@ -10630,6 +10692,24 @@ run_gate_case "report-model-quality-warning-preserves-prior-fatal-box" \ "vertex_ai/report-model-quality-warning-preserves-prior-fatal-box" \ "" +run_gate_case "console-model-quality-warning-preserves-same-box-failure" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-failure" \ + "" \ + "1" \ + "Strix run emitted provider infrastructure or failure-signal output; failing closed." \ + "1" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-failure" \ + "" + +run_gate_case "report-model-quality-warning-preserves-same-box-failure" \ + "vertex_ai/report-model-quality-warning-preserves-same-box-failure" \ + "" \ + "1" \ + "Strix report artifacts emitted warning/fatal/denied/timeout output; failing closed." \ + "1" \ + "vertex_ai/report-model-quality-warning-preserves-same-box-failure" \ + "" + run_gate_case "report-known-internal-warning-variant-sanitized" \ "vertex_ai/report-known-internal-warning-variant-sanitized" \ "" \ From c1ab9574465379626cabfe940d8873fd0f299006 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 04:08:33 +0000 Subject: [PATCH 05/13] fix(strix): match the banner heading anywhere on the line A trailing-space fullmatch left MODEL QUALITY WARNING in the classified copy, so a clean 0-finding scan still fail-closed. Search the heading after stripping ANSI so same-box failures stay visible. Co-authored-by: Seongho Bae --- scripts/ci/strix_quick_gate.sh | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index 1eb4cd137..c67de0a03 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -180,8 +180,8 @@ ansi_csi = re.compile(r"\x1b\[[0-?]*[ -/]*[@-~]") def strip_model_quality_warning_heading(text: str) -> str: kept: list[str] = [] for line in text.splitlines(keepends=True): - plain = ansi_csi.sub("", line.rstrip("\r\n")) - if model_quality_heading.fullmatch(plain): + plain = ansi_csi.sub("", line) + if model_quality_heading.search(plain): continue kept.append(line) return "".join(kept) @@ -225,8 +225,8 @@ ansi_csi = re.compile(r"\x1b\[[0-?]*[ -/]*[@-~]") def strip_model_quality_warning_heading(text: str) -> str: kept = [] for line in text.splitlines(keepends=True): - plain = ansi_csi.sub("", line.rstrip("\r\n")) - if model_quality_heading.fullmatch(plain): + plain = ansi_csi.sub("", line) + if model_quality_heading.search(plain): continue kept.append(line) return "".join(kept) From f02eb90a68aa7258f39a5957435930b85feb32a2 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 04:10:10 +0000 Subject: [PATCH 06/13] fix(strix): delete cosmetic-only banner boxes, keep same-box failures Heading-only stripping left fixture or model text such as -warning- in the leftover banner, which the generic Warn matcher fail-closed. Delete a box that is only MODEL QUALITY WARNING. If the same box also has Fatal, Denied, Timeout, or Provider WARNING, strip the heading and keep the failure line. Co-authored-by: Seongho Bae --- CHANGELOG.md | 9 ++-- scripts/ci/strix_quick_gate.sh | 88 +++++++++++++++++++++++++++++----- 2 files changed, 81 insertions(+), 16 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 163cad909..7663e02e0 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -76,10 +76,11 @@ Semantic Versioning where the repository publishes a release. remain loopback, while rejecting userinfo, missing hosts, IPv4-mapped public addresses, unspecified bind addresses, and metadata destinations before any request is opened. -- Strip only the Strix `MODEL QUALITY WARNING` heading line before - infrastructure-signal matching so a Fatal/Denied/Warning in the same - box stays visible, keep a preceding Fatal/Denied box, and classify a - sanitized console copy so `gate-last-attempt.log` stays raw. +- Strip a cosmetic-only Strix `MODEL QUALITY WARNING` box before + infrastructure-signal matching; if that same box also carries + Fatal/Denied/Timeout or Provider WARNING, keep those lines. Keep a + preceding Fatal/Denied box, and classify a sanitized console copy so + `gate-last-attempt.log` stays raw. - Publish only the sanitized cumulative Strix report tree, avoiding a later copy of relative scanner output that could reintroduce known internal warning diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index c67de0a03..23d76c6b5 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -153,10 +153,10 @@ preserve_attempt_log() { # Write a classification copy of the raw console transcript. Never mutate # $STRIX_LOG: publish_artifact_reports copies it to gate-last-attempt.log. -# Remove only the cosmetic MODEL QUALITY WARNING heading line. Deleting a -# whole box would hide a Fatal/Denied/Warning emitted in that same box -# (fail-open). A cross-box wildcard from the first ╭ to a later ╰ would -# also erase a preceding Fatal/Denied box. +# Delete a cosmetic-only MODEL QUALITY WARNING box. If that same box also +# contains Fatal/Denied/Timeout or Provider WARNING, strip only the heading +# so the failure stays visible. A cross-box wildcard from the first ╭ to a +# later ╰ would erase a preceding Fatal/Denied box (fail-open). sanitize_strix_console_log() { local src="$1" local dest="$2" @@ -175,23 +175,55 @@ src = Path(sys.argv[1]) dest = Path(sys.argv[2]) model_quality_heading = re.compile(r"│[ \t]*MODEL QUALITY WARNING[ \t]*│") ansi_csi = re.compile(r"\x1b\[[0-?]*[ -/]*[@-~]") +other_failure = re.compile( + r"(?:^|[^A-Za-z])(?:Fatal|Denied|Timeout)(?:[^A-Za-z]|$)|Provider WARNING", + re.I, +) -def strip_model_quality_warning_heading(text: str) -> str: +def strip_heading_lines(text: str) -> str: kept: list[str] = [] for line in text.splitlines(keepends=True): - plain = ansi_csi.sub("", line) - if model_quality_heading.search(plain): + if model_quality_heading.search(ansi_csi.sub("", line)): continue kept.append(line) return "".join(kept) +def strip_model_quality_warning_boxes(text: str) -> str: + pieces: list[str] = [] + index = 0 + length = len(text) + while index < length: + start = text.find("╭", index) + if start < 0: + pieces.append(text[index:]) + break + pieces.append(text[index:start]) + close = text.find("╰", start) + if close < 0: + pieces.append(text[start:]) + break + line_end = text.find("\n", close) + if line_end < 0: + box = text[start:] + index = length + else: + box = text[start : line_end + 1] + index = line_end + 1 + if "MODEL QUALITY WARNING" in box: + if other_failure.search(box): + pieces.append(strip_heading_lines(box)) + continue + pieces.append(box) + return "".join(pieces) + + try: text = src.read_text(encoding="utf-8") except UnicodeDecodeError: raise SystemExit(0) -dest.write_text(strip_model_quality_warning_heading(text), encoding="utf-8") +dest.write_text(strip_model_quality_warning_boxes(text), encoding="utf-8") PY } @@ -220,18 +252,50 @@ known_internal_warning = re.compile( model_quality_heading = re.compile(r"│[ \t]*MODEL QUALITY WARNING[ \t]*│") ansi_csi = re.compile(r"\x1b\[[0-?]*[ -/]*[@-~]") +other_failure = re.compile( + r"(?:^|[^A-Za-z])(?:Fatal|Denied|Timeout)(?:[^A-Za-z]|$)|Provider WARNING", + re.I, +) -def strip_model_quality_warning_heading(text: str) -> str: +def strip_heading_lines(text: str) -> str: kept = [] for line in text.splitlines(keepends=True): - plain = ansi_csi.sub("", line) - if model_quality_heading.search(plain): + if model_quality_heading.search(ansi_csi.sub("", line)): continue kept.append(line) return "".join(kept) +def strip_model_quality_warning_boxes(text: str) -> str: + pieces = [] + index = 0 + length = len(text) + while index < length: + start = text.find("╭", index) + if start < 0: + pieces.append(text[index:]) + break + pieces.append(text[index:start]) + close = text.find("╰", start) + if close < 0: + pieces.append(text[start:]) + break + line_end = text.find("\n", close) + if line_end < 0: + box = text[start:] + index = length + else: + box = text[start : line_end + 1] + index = line_end + 1 + if "MODEL QUALITY WARNING" in box: + if other_failure.search(box): + pieces.append(strip_heading_lines(box)) + continue + pieces.append(box) + return "".join(pieces) + + def iter_report_logs(root: Path): for current_root, dir_names, file_names in os.walk(root, topdown=True, followlinks=False): current_path = Path(current_root) @@ -252,7 +316,7 @@ for log_path in iter_report_logs(root): original = log_path.read_text(encoding="utf-8") except UnicodeDecodeError: continue - text = strip_model_quality_warning_heading(original) + text = strip_model_quality_warning_boxes(original) lines = text.splitlines(keepends=True) filtered = [line for line in lines if not known_internal_warning.match(line)] text = "".join(filtered) From 51291c3a46fa23296c825c62b1c1476bde011c15 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 04:19:49 +0000 Subject: [PATCH 07/13] fix(strix): align overlay to gpt-5.4 and keep same-box rate-limit evidence The PR overlay still named the removed gpt-5.6-luna fallback, so the trusted required-path smoke fail-closed before the sanitizer could matter. Match the protected-main gpt-5.4 contract. Also keep RateLimitError and related infra tokens when they share a MODEL QUALITY WARNING box, so deleting a cosmetic-only banner cannot hide them. Co-authored-by: Seongho Bae --- .github/workflows/strix.yml | 6 +-- scripts/ci/strix_quick_gate.sh | 10 ++++- scripts/ci/strix_required_workflow_smoke.sh | 4 +- scripts/ci/test_strix_quick_gate.sh | 45 +++++++++++++++++++-- 4 files changed, 54 insertions(+), 11 deletions(-) diff --git a/.github/workflows/strix.yml b/.github/workflows/strix.yml index c8c054e42..f89119070 100644 --- a/.github/workflows/strix.yml +++ b/.github/workflows/strix.yml @@ -453,7 +453,7 @@ jobs: - name: Gate Strix secrets id: gate env: - STRIX_MODEL: ${{ github.event.client_payload.strix_llm || (steps.target_visibility.outputs.is_private == 'false' && 'nvidia_nim/nvidia/nemotron-3-super-120b-a12b' || 'gpt-5.6-luna') }} + STRIX_MODEL: ${{ github.event.client_payload.strix_llm || (steps.target_visibility.outputs.is_private == 'false' && 'nvidia_nim/nvidia/nemotron-3-super-120b-a12b' || 'gpt-5.4') }} STRIX_MODEL_REQUESTED: ${{ github.event.client_payload.strix_llm || '' }} STRIX_OPENAI_API_KEY: ${{ secrets.STRIX_OPENAI_API_KEY || secrets.OPENAI_API_KEY }} STRIX_OPENROUTER_API_KEY: ${{ secrets.OPENROUTER_API_KEY }} @@ -464,7 +464,7 @@ jobs: run: | strix_model="$(printf '%s' "$STRIX_MODEL" | sed 's/^[[:space:]]*//;s/[[:space:]]*$//')" if [ -z "$STRIX_MODEL_REQUESTED" ] && [ "$strix_model" = "nvidia_nim/nvidia/nemotron-3-super-120b-a12b" ] && [ -z "${STRIX_NVIDIA_NIM_API_KEY:-}" ]; then - strix_model="gpt-5.6-luna" + strix_model="gpt-5.4" fi echo "strix_model=$strix_model" >> "$GITHUB_OUTPUT" case "$strix_model" in @@ -834,7 +834,7 @@ jobs: STRIX_LLM_MAX_RETRIES: 1 STRIX_TRANSIENT_RETRY_PER_MODEL: 2 STRIX_TRANSIENT_RETRY_BACKOFF_SECONDS: 60 - STRIX_FALLBACK_MODELS: ${{ steps.gate.outputs.provider_mode == 'github_models' && 'openai-direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'openai_direct' && 'openai-direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'openrouter' && 'openai-direct/gpt-5.6-luna' || steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.6-luna' || '' }} + STRIX_FALLBACK_MODELS: ${{ steps.gate.outputs.provider_mode == 'github_models' && 'openai-direct/gpt-5.4' || steps.gate.outputs.provider_mode == 'openai_direct' && 'openai-direct/gpt-5.4' || steps.gate.outputs.provider_mode == 'openrouter' && 'openai-direct/gpt-5.4' || steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.4' || '' }} STRIX_GITHUB_MODELS_API_BASE_FILE: ${{ env.STRIX_GITHUB_MODELS_API_BASE_FILE }} STRIX_GITHUB_MODELS_KEY_FILE: ${{ env.STRIX_GITHUB_MODELS_KEY_FILE }} STRIX_OPENAI_FALLBACK_KEY_FILE: ${{ env.STRIX_OPENAI_FALLBACK_KEY_FILE }} diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index 23d76c6b5..24c082940 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -176,7 +176,10 @@ dest = Path(sys.argv[2]) model_quality_heading = re.compile(r"│[ \t]*MODEL QUALITY WARNING[ \t]*│") ansi_csi = re.compile(r"\x1b\[[0-?]*[ -/]*[@-~]") other_failure = re.compile( - r"(?:^|[^A-Za-z])(?:Fatal|Denied|Timeout)(?:[^A-Za-z]|$)|Provider WARNING", + r"(?:^|[^A-Za-z])(?:Fatal|Denied|Timeout)(?:[^A-Za-z]|$)" + r"|Provider WARNING" + r"|RateLimitError|Nvidia_nimException|LLM CONNECTION FAILED" + r"|APIConnectionError|Too Many Requests", re.I, ) @@ -253,7 +256,10 @@ known_internal_warning = re.compile( model_quality_heading = re.compile(r"│[ \t]*MODEL QUALITY WARNING[ \t]*│") ansi_csi = re.compile(r"\x1b\[[0-?]*[ -/]*[@-~]") other_failure = re.compile( - r"(?:^|[^A-Za-z])(?:Fatal|Denied|Timeout)(?:[^A-Za-z]|$)|Provider WARNING", + r"(?:^|[^A-Za-z])(?:Fatal|Denied|Timeout)(?:[^A-Za-z]|$)" + r"|Provider WARNING" + r"|RateLimitError|Nvidia_nimException|LLM CONNECTION FAILED" + r"|APIConnectionError|Too Many Requests", re.I, ) diff --git a/scripts/ci/strix_required_workflow_smoke.sh b/scripts/ci/strix_required_workflow_smoke.sh index 8539afdfe..76aec7910 100755 --- a/scripts/ci/strix_required_workflow_smoke.sh +++ b/scripts/ci/strix_required_workflow_smoke.sh @@ -170,8 +170,8 @@ assert_file_contains "$full_gate_test" "assert_strix_workflow_pr_trigger_hardene assert_file_contains "$workflow_file" "nvidia_nim/nvidia/nemotron-3-super-120b-a12b" "Strix defaults public scans to the current hosted NVIDIA NIM model" assert_file_contains_either \ "$workflow_file" \ - "nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai_direct/gpt-5.6-luna" \ - "nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.6-luna" \ + "nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai_direct/gpt-5.4" \ + "nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.4" \ "Strix tries another NVIDIA hosted model before falling back to direct OpenAI" assert_file_not_contains "$workflow_file" "github_models/openai/o3" "Strix fallback list must not depend on GitHub Models, which is in platform-wide retirement" assert_file_contains "$workflow_file" "Nvidia_nimException" "Strix workflow recognizes provider-scoped NVIDIA NIM failures" diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index b64762fe3..2d09d6957 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -303,7 +303,7 @@ assert_strix_workflow_pr_trigger_hardened() { assert_file_not_contains "$workflow_file" "STRIX_TOTAL_TIMEOUT_SECONDS:" "strix workflow must not expose total timeout env names in GitHub logs" assert_file_not_contains "$workflow_file" "STRIX_PR_SCOPE_MAX_FILES_PER_BATCH" "strix workflow must not split Strix PR evidence into separate scanner runs" assert_file_not_contains "$workflow_file" "secrets.STRIX_LLM == 'vertex_ai/gemini-3.1-pro-preview-customtools' && 'vertex_ai/gemini-2.5-flash'" "strix workflow must not quarantine the approved Vertex preview model after organization secret visibility is fixed" - assert_file_contains "$workflow_file" "steps.target_visibility.outputs.is_private == 'false' && 'nvidia_nim/nvidia/nemotron-3-super-120b-a12b' || 'gpt-5.6-luna'" "strix workflow defaults public scans to NVIDIA NIM and keeps private scans on the contracted provider" + assert_file_contains "$workflow_file" "steps.target_visibility.outputs.is_private == 'false' && 'nvidia_nim/nvidia/nemotron-3-super-120b-a12b' || 'gpt-5.4'" "strix workflow defaults public scans to NVIDIA NIM and keeps private scans on the contracted provider" assert_file_contains "$workflow_file" 'if [ -z "$STRIX_MODEL_REQUESTED" ] && [ "$strix_model" = "nvidia_nim/nvidia/nemotron-3-super-120b-a12b" ] && [ -z "${STRIX_NVIDIA_NIM_API_KEY:-}" ]' "strix workflow falls back to the contracted provider when the NVIDIA secret is absent" assert_file_contains "$workflow_file" 'STRIX_MODEL: ${{ steps.gate.outputs.strix_model }}' "strix workflow propagates the gate-selected fallback model to the scanner" assert_file_not_contains "$workflow_file" "secrets.STRIX_LLM ||" "strix workflow must not let the legacy STRIX_LLM secret override PR defaults" @@ -360,9 +360,9 @@ assert_strix_workflow_pr_trigger_hardened() { assert_file_contains "$workflow_file" "https://integrate.api.nvidia.com/v1" "strix workflow routes NVIDIA NIM scans to the hosted endpoint" assert_file_contains "$workflow_file" "LLM_API_BASE_FILE" "strix workflow passes the GitHub Models API base through a trusted input file" assert_file_not_contains "$workflow_file" '${{ secrets.STRIX_OPENAI_API_KEY || github.token }}' "strix workflow must not use fallback-secret syntax for LLM API keys" - assert_file_contains "$workflow_file" "openai-direct/gpt-5.6-luna" "strix workflow keeps a direct-OpenAI fallback on a tool-capable, Strix-recommended model without GPT-4.1 downgrade" - assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'openai_direct' && 'openai-direct/gpt-5.6-luna'" "strix workflow gives direct-OpenAI scans a same-provider fallback so transient errors degrade instead of skipping" - assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.6-luna'" "strix workflow gives NVIDIA NIM scans contracted fallbacks" + assert_file_contains "$workflow_file" "openai-direct/gpt-5.4" "strix workflow keeps a direct-OpenAI fallback on a tool-capable, Strix-recommended model without GPT-4.1 downgrade" + assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'openai_direct' && 'openai-direct/gpt-5.4'" "strix workflow gives direct-OpenAI scans a same-provider fallback so transient errors degrade instead of skipping" + assert_file_contains "$workflow_file" "steps.gate.outputs.provider_mode == 'nvidia_nim' && 'nvidia_nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 openai-direct/gpt-5.4'" "strix workflow gives NVIDIA NIM scans contracted fallbacks" assert_file_not_contains "$workflow_file" "STRIX_FALLBACK_MODELS: \${{ steps.gate.outputs.provider_mode == 'github_models' && 'github_models/openai/o3" "strix workflow fallback list must not depend on GitHub Models, which is in platform-wide retirement" assert_file_contains "$workflow_file" "Prepare GitHub Models fallback credentials" "strix workflow provisions GitHub Models fallback credentials for direct-OpenAI scans" assert_file_contains "$GATE_SCRIPT" "STRIX_GITHUB_MODELS_KEY_FILE" "strix gate reads the optional GitHub Models fallback key file" @@ -4640,6 +4640,17 @@ EOS echo "scan returned zero findings with incomplete evidence in the banner box" exit 0 ;; + console-model-quality-warning-preserves-same-box-ratelimit) + cat <<'EOS' +╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ +│ MODEL QUALITY WARNING │ +│ RateLimitError: Too Many Requests │ +╰──────────────────────────────────────────────────────────────────────────────╯ +Penetration test completed +Vulnerabilities 0 (No exploitable vulnerabilities detected) +EOS + exit 0 + ;; bare-timeout-with-provider-marker) # Emit bare "Connection timed out" alongside a provider marker so # is_timeout_error() matches the Tier 3 branch gated on @@ -5997,6 +6008,13 @@ PY "scenario=$scenario still strips only the cosmetic heading" fi + if [ "$scenario" = "console-model-quality-warning-preserves-same-box-ratelimit" ]; then + assert_file_contains \ + "$repo_root_dir/strix_runs/gate-last-attempt.log" \ + "RateLimitError: Too Many Requests" \ + "scenario=$scenario keeps the raw same-box rate-limit line in last-attempt" + fi + if [ "$scenario" = "report-known-internal-warning-variant-sanitized" ]; then assert_file_not_contains \ "$repo_root_dir/strix_runs/fake-known-internal-warning-variant/strix.log" \ @@ -6590,6 +6608,16 @@ run_filtered_gate_case_if_requested() { "vertex_ai/report-model-quality-warning-preserves-same-box-failure" \ "" ;; + console-model-quality-warning-preserves-same-box-ratelimit) + run_gate_case "$STRIX_TEST_CASE_FILTER" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit" \ + "" \ + "1" \ + "Strix run emitted provider infrastructure or failure-signal output; failing closed." \ + "1" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit" \ + "" + ;; provider-fatal-success-signal | provider-warning-success-signal) run_gate_case "$STRIX_TEST_CASE_FILTER" \ "vertex_ai/$STRIX_TEST_CASE_FILTER" \ @@ -10710,6 +10738,15 @@ run_gate_case "report-model-quality-warning-preserves-same-box-failure" \ "vertex_ai/report-model-quality-warning-preserves-same-box-failure" \ "" +run_gate_case "console-model-quality-warning-preserves-same-box-ratelimit" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit" \ + "" \ + "1" \ + "Strix run emitted provider infrastructure or failure-signal output; failing closed." \ + "1" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit" \ + "" + run_gate_case "report-known-internal-warning-variant-sanitized" \ "vertex_ai/report-known-internal-warning-variant-sanitized" \ "" \ From 6b06a9239adce28532cd2d225a8f346184592f04 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 04:23:59 +0000 Subject: [PATCH 08/13] fix(strix): align gpt-5.4 contracts and expect rate-limit retries Path-policy still pinned the removed gpt-5.6-luna secret-absent default after the overlay matched protected main. RateLimitError in a MODEL QUALITY WARNING box is retryable, so the keep-path must expect the three-model Vertex sequence; one call would mean the box was deleted. Co-authored-by: Seongho Bae --- scripts/ci/test_strix_quick_gate.sh | 20 +++++++++++-------- .../test_required_workflow_queue_contract.py | 2 +- ...est_strix_nvidia_nim_not_found_fallback.py | 4 ++-- 3 files changed, 15 insertions(+), 11 deletions(-) diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 2d09d6957..e3d65a85b 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -4641,6 +4641,10 @@ EOS exit 0 ;; console-model-quality-warning-preserves-same-box-ratelimit) + # RateLimitError is retryable. Keeping it in the classified copy + # therefore tries the two default Vertex fallbacks. Deleting the + # whole box would hide the token, inspect would pass, and the + # gate would stop after one successful call. cat <<'EOS' ╭─ STRIX ──────────────────────────────────────────────────────────────────────╮ │ MODEL QUALITY WARNING │ @@ -6611,12 +6615,12 @@ run_filtered_gate_case_if_requested() { console-model-quality-warning-preserves-same-box-ratelimit) run_gate_case "$STRIX_TEST_CASE_FILTER" \ "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit" \ - "" \ + "vertex_ai/gemini-2.5-pro vertex_ai/gemini-2.5-flash" \ "1" \ "Strix run emitted provider infrastructure or failure-signal output; failing closed." \ - "1" \ - "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit" \ - "" + "3" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit|vertex_ai/gemini-2.5-pro|vertex_ai/gemini-2.5-flash" \ + "||" ;; provider-fatal-success-signal | provider-warning-success-signal) run_gate_case "$STRIX_TEST_CASE_FILTER" \ @@ -10740,12 +10744,12 @@ run_gate_case "report-model-quality-warning-preserves-same-box-failure" \ run_gate_case "console-model-quality-warning-preserves-same-box-ratelimit" \ "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit" \ - "" \ + "vertex_ai/gemini-2.5-pro vertex_ai/gemini-2.5-flash" \ "1" \ "Strix run emitted provider infrastructure or failure-signal output; failing closed." \ - "1" \ - "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit" \ - "" + "3" \ + "vertex_ai/console-model-quality-warning-preserves-same-box-ratelimit|vertex_ai/gemini-2.5-pro|vertex_ai/gemini-2.5-flash" \ + "||" run_gate_case "report-known-internal-warning-variant-sanitized" \ "vertex_ai/report-known-internal-warning-variant-sanitized" \ diff --git a/tests/test_required_workflow_queue_contract.py b/tests/test_required_workflow_queue_contract.py index e58f5e6c0..1d79f1daa 100644 --- a/tests/test_required_workflow_queue_contract.py +++ b/tests/test_required_workflow_queue_contract.py @@ -505,7 +505,7 @@ def test_nvidia_nim_defaults_preserve_existing_fallbacks_without_secret( assert strix.returncode == 0, strix.stderr assert { "provider_mode=openai_direct", - "strix_model=gpt-5.6-luna", + "strix_model=gpt-5.4", } <= set(strix_output.read_text().splitlines()) assert ( "STRIX_MODEL: ${{ steps.gate.outputs.strix_model }}" diff --git a/tests/test_strix_nvidia_nim_not_found_fallback.py b/tests/test_strix_nvidia_nim_not_found_fallback.py index 990269725..17f0e9a30 100644 --- a/tests/test_strix_nvidia_nim_not_found_fallback.py +++ b/tests/test_strix_nvidia_nim_not_found_fallback.py @@ -192,7 +192,7 @@ def test_workflow_uses_available_free_first_nvidia_plan(self) -> None: workflow = STRIX_WORKFLOW.read_text(encoding="utf-8") default_expression = ( "steps.target_visibility.outputs.is_private == 'false' && " - f"'{DEFAULT_NVIDIA_MODEL}' || 'gpt-5.6-luna'" + f"'{DEFAULT_NVIDIA_MODEL}' || 'gpt-5.4'" ) self.assertIn(default_expression, workflow) self.assertIn( @@ -202,7 +202,7 @@ def test_workflow_uses_available_free_first_nvidia_plan(self) -> None: ) self.assertIn( "steps.gate.outputs.provider_mode == 'nvidia_nim' && " - f"'{FREE_NVIDIA_FALLBACK} openai-direct/gpt-5.6-luna'", + f"'{FREE_NVIDIA_FALLBACK} openai-direct/gpt-5.4'", workflow, ) From d4ea752830df1c6e25bc38147fe08a9b3480f058 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Tue, 25 Aug 2026 05:58:48 +0000 Subject: [PATCH 09/13] fix(strix): keep OpenCode dispatch-pool harness needles on gpt-5.6-luna Merging 8fd471a3 brought #1318's global luna-to-gpt-5.4 replace in test_strix_quick_gate.sh. Three OpenCode dispatch-pool assertions now expected openai/gpt-5.4, but opencode-review-dispatch.yml still lists openai/gpt-5.6-luna. Restore those needles so the harness matches the unchanged dispatch workflow. Keep the Strix quota fixtures on gpt-5.4. Do not fold the dispatch-pool model change from #1316. Co-authored-by: Seongho Bae --- scripts/ci/test_strix_quick_gate.sh | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index c54915b4c..a97820267 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -771,7 +771,7 @@ assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" ' assert_file_contains "$workflow_file" "Run OpenCode PR Review model pool" "opencode review includes a broad catalog fallback pool" assert_file_not_contains "$workflow_file" "steps.opencode_review_model_pool.outcome == 'success'" "opencode approval gate still runs after model pool failure to publish a reason" assert_file_contains "$workflow_file" "opencode-free/north-mini-code-free" "opencode review starts public repository reviews with a free coding model" - assert_file_contains "$workflow_file" "opencode/gpt-5.6-terra github-models/deepseek/deepseek-v3-0324 openai/gpt-5.4 openrouter/deepseek/deepseek-v3.2 openrouter/qwen/qwen3-coder github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review retains paid Zen and DeepSeek V3 before full-size GPT fallbacks" + assert_file_contains "$workflow_file" "opencode/gpt-5.6-terra github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/deepseek/deepseek-v3.2 openrouter/qwen/qwen3-coder github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review retains paid Zen and DeepSeek V3 before full-size GPT fallbacks" assert_file_contains "$workflow_file" "The publish gate re-runs source-backed validation against PR-head data" "opencode review publish gate validates model output against the PR-head worktree" assert_file_contains "$workflow_file" '"openai/o3"' "opencode config declares OpenAI o3 fallback" assert_file_contains "$workflow_file" '"openai/o4-mini"' "opencode config declares OpenAI o4-mini fallback" @@ -922,7 +922,7 @@ assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" ' assert_file_contains "$workflow_file" 'OPENCODE_RUN_TIMEOUT_SECONDS: "5400"' "opencode catalog fallback preserves legitimate full-hour provider sessions" assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" "OpenCode %s attempt %s/%s failed" "opencode catalog fallback records per-model retry failures" assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" "exponential backoff" "opencode model retry paths use exponential backoff instead of fixed sleeps" - assert_file_contains "$workflow_file" "opencode/gpt-5.6-terra github-models/deepseek/deepseek-v3-0324 openai/gpt-5.4 openrouter/deepseek/deepseek-v3.2 openrouter/qwen/qwen3-coder github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review tries paid Zen and DeepSeek V3 before OpenAI fallbacks" + assert_file_contains "$workflow_file" "opencode/gpt-5.6-terra github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/deepseek/deepseek-v3.2 openrouter/qwen/qwen3-coder github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review tries paid Zen and DeepSeek V3 before OpenAI fallbacks" assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-r1-0528 github-models/deepseek/deepseek-r1" "opencode review keeps DeepSeek reasoning fallback coverage after OpenAI candidates" assert_file_contains "$workflow_file" "coverage-source-tree:" "opencode workflow materializes coverage source before running PR-head tests" assert_file_contains "$workflow_file" "coverage-evidence:" "opencode workflow measures coverage before review" @@ -1273,7 +1273,7 @@ assert_file_contains "$REPO_ROOT/scripts/ci/run_opencode_review_model_pool.sh" ' assert_file_contains "$workflow_file" 'repos/${GH_REPOSITORY}' "opencode review workflow uses env-backed repository context in shell commands" assert_file_contains "$workflow_file" "Run OpenCode PR Review model pool" "opencode review starts the central model pool" assert_file_contains "$workflow_file" "nvidia-nim/nvidia/llama-3.3-nemotron-super-49b-v1.5 nvidia-nim/nvidia/llama-3.1-nemotron-ultra-253b-v1 nvidia-nim/nvidia/nemotron-3-super-120b-a12b nvidia-nim/nvidia/nemotron-3-ultra-550b-a55b nvidia-nim/meta/llama-3.3-70b-instruct nvidia-nim/deepseek-ai/deepseek-v4-pro nvidia-nim/mistralai/codestral-22b-instruct-v0.1 opencode-free/nemotron-3-ultra-free" "opencode review keeps all NVIDIA NIM candidates inside the public-repository pool" - assert_file_contains "$workflow_file" "opencode/gpt-5.6-terra github-models/deepseek/deepseek-v3-0324 openai/gpt-5.4 openrouter/deepseek/deepseek-v3.2 openrouter/qwen/qwen3-coder github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review keeps paid Zen, DeepSeek V3, and full-size GPT fallbacks" + assert_file_contains "$workflow_file" "opencode/gpt-5.6-terra github-models/deepseek/deepseek-v3-0324 openai/gpt-5.6-luna openrouter/deepseek/deepseek-v3.2 openrouter/qwen/qwen3-coder github-models/openai/gpt-4.1 github-models/openai/gpt-5" "opencode review keeps paid Zen, DeepSeek V3, and full-size GPT fallbacks" assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-r1-0528" "opencode review keeps a reachable DeepSeek R1 reasoning fallback model" assert_file_contains "$workflow_file" "github-models/deepseek/deepseek-v3-0324" "opencode review has a reachable DeepSeek V3 fallback model" assert_file_not_contains "$workflow_file" "secrets.NVIDIA_NIM_API_KEY || secrets.NVIDIA_API_KEY" "opencode review never falls back from the scoped NVIDIA NIM secret to the legacy provider secret" From 7d3dddd34fe578a10cc52b008ac554f4a49ebc8a Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 26 Aug 2026 08:03:40 +0900 Subject: [PATCH 10/13] fix(strix): ignore known internal console warnings --- scripts/ci/strix_quick_gate.sh | 16 +++++++++++- scripts/ci/test_strix_quick_gate.sh | 40 +++++++++++++++++++++++++++++ 2 files changed, 55 insertions(+), 1 deletion(-) diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index ea016d5bf..9c3d2407e 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -179,6 +179,14 @@ known_scanner_warning = re.compile( r"^(?:│ MODEL QUALITY WARNING\s+│|" r"Warning: You are sending unauthenticated requests to the HF Hub\.)" ) +known_internal_warning = re.compile( + r"^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}\.\d+ WARNING " + r"[^ ]+ - strix\.core\.execution: agent [0-9a-f]+ " + r"(?:" + r"produced non-lifecycle final output in non-interactive mode" + r"|ended a turn without a lifecycle tool call \(interactive=False\)" + r"); forcing tool continuation \(\d+/\d+\): " +) other_failure = re.compile( r"(?:^|[^A-Za-z])(?:Fatal|Denied|Timeout)(?:[^A-Za-z]|$)" r"|Provider WARNING" @@ -229,12 +237,18 @@ def strip_model_quality_warning_boxes(text: str) -> str: try: text = src.read_text(encoding="utf-8") except UnicodeDecodeError: + # Never let a previous attempt's classified copy be inspected after a + # malformed console transcript. The raw source remains untouched. + dest.write_text("", encoding="utf-8") raise SystemExit(0) text = strip_model_quality_warning_boxes(text) text = "".join( line for line in text.splitlines(keepends=True) - if not known_scanner_warning.match(ansi_csi.sub("", line)) + if not ( + known_scanner_warning.match(ansi_csi.sub("", line)) + or known_internal_warning.match(ansi_csi.sub("", line)) + ) ) dest.write_text(text, encoding="utf-8") PY diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 1151bf746..f09ec06a4 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -331,6 +331,8 @@ assert_strix_workflow_pr_trigger_hardened() { assert_file_contains "$GATE_SCRIPT" '[[ "$normalized_changed_file" == scripts/ci/test_*.sh || "$normalized_changed_file" == scripts/ci/*_test.sh ]]' "strix gate excludes large CI test harness scripts from model scan input" assert_file_contains "$GATE_SCRIPT" "Materialized PR-head changed-file scope for Strix scan" "strix gate avoids copying the full PR head tree into privileged scan targets by default" assert_file_contains "$GATE_SCRIPT" "sanitize_known_strix_report_warnings" "strix gate sanitizes only known internal Strix report warnings" + assert_file_contains "$GATE_SCRIPT" "known_internal_warning = re.compile" "strix gate sanitizes known internal console warnings" + assert_file_contains "$GATE_SCRIPT" 'dest.write_text("", encoding="utf-8")' "strix gate clears stale classified console output after decode errors" assert_file_contains "$GATE_SCRIPT" 'MODEL QUALITY WARNING' "strix gate accepts the scanner's informational fallback-model banner" assert_file_contains "$GATE_SCRIPT" 'unauthenticated requests to the HF Hub' "strix gate accepts the scanner dependency's non-fatal download warning" assert_file_not_contains "$GATE_SCRIPT" 'known_scanner_warning = re.compile(r".*Warn' "strix gate does not broadly suppress warning-class evidence" @@ -4534,6 +4536,14 @@ EOS echo "Warning: provider response included incomplete scan state" exit 0 ;; + console-known-internal-warning-sanitized) + cat <<'EOS' +2026-06-18 13:08:05.986 WARNING strix-pr-scope-example - strix.core.execution: agent a9fb4033 produced non-lifecycle final output in non-interactive mode; forcing tool continuation (1/500): internal console coordination note +2026-08-22 09:53:26.193 WARNING strix-pr-scope-example - strix.core.execution: agent 673f770f ended a turn without a lifecycle tool call (interactive=False); forcing tool continuation (1/500): +scan completed with 0 vulnerability report(s) +EOS + exit 0 + ;; provider-denied-success-signal) echo "Denied: provider credentials were rejected" exit 0 @@ -6026,6 +6036,17 @@ PY fi fi + if [ "$scenario" = "console-known-internal-warning-sanitized" ]; then + assert_file_contains \ + "$repo_root_dir/strix_runs/gate-last-attempt.log" \ + "produced non-lifecycle final output" \ + "scenario=$scenario preserves the raw non-lifecycle console warning" + assert_file_contains \ + "$repo_root_dir/strix_runs/gate-last-attempt.log" \ + "ended a turn without a lifecycle tool call" \ + "scenario=$scenario preserves the raw lifecycle console warning" + fi + if [ "$scenario" = "console-model-quality-warning-preserves-prior-fatal-box" ]; then assert_file_contains \ "$repo_root_dir/strix_runs/gate-last-attempt.log" \ @@ -6630,6 +6651,16 @@ run_filtered_gate_case_if_requested() { "vertex_ai/excluded-dir-primary" \ "" ;; + console-known-internal-warning-sanitized) + run_gate_case "$STRIX_TEST_CASE_FILTER" \ + "vertex_ai/console-known-internal-warning-sanitized" \ + "" \ + "0" \ + "Strix run succeeded for model 'vertex_ai/console-known-internal-warning-sanitized'" \ + "1" \ + "vertex_ai/console-known-internal-warning-sanitized" \ + "" + ;; pull-request-target-changed-backend-context) run_pull_request_target_changed_backend_context_scope_case ;; @@ -10828,6 +10859,15 @@ run_gate_case "provider-warning-success-signal" \ "" \ "1" +run_gate_case "console-known-internal-warning-sanitized" \ + "vertex_ai/console-known-internal-warning-sanitized" \ + "" \ + "0" \ + "Strix run succeeded for model 'vertex_ai/console-known-internal-warning-sanitized'" \ + "1" \ + "vertex_ai/console-known-internal-warning-sanitized" \ + "" + run_gate_case "provider-report-rate-limit-fallback-success" \ "vertex_ai/report-rate-limit-primary" \ "vertex_ai/fallback-one vertex_ai/fallback-two" \ From 908f794748bf3241328e3bd93890bb116f753b3b Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 26 Aug 2026 08:22:36 +0900 Subject: [PATCH 11/13] fix(strix): preserve malformed console failure markers --- scripts/ci/strix_quick_gate.sh | 8 +++++--- scripts/ci/test_strix_quick_gate.sh | 2 +- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index 9c3d2407e..7c1017326 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -237,9 +237,11 @@ def strip_model_quality_warning_boxes(text: str) -> str: try: text = src.read_text(encoding="utf-8") except UnicodeDecodeError: - # Never let a previous attempt's classified copy be inspected after a - # malformed console transcript. The raw source remains untouched. - dest.write_text("", encoding="utf-8") + # Preserve byte-level evidence so ASCII failure markers in an otherwise + # malformed transcript still reach the fail-closed grep classifiers. This + # also replaces any previous attempt's classified copy without mutating + # the raw source. + dest.write_bytes(src.read_bytes()) raise SystemExit(0) text = strip_model_quality_warning_boxes(text) text = "".join( diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index f09ec06a4..5806609b8 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -332,7 +332,7 @@ assert_strix_workflow_pr_trigger_hardened() { assert_file_contains "$GATE_SCRIPT" "Materialized PR-head changed-file scope for Strix scan" "strix gate avoids copying the full PR head tree into privileged scan targets by default" assert_file_contains "$GATE_SCRIPT" "sanitize_known_strix_report_warnings" "strix gate sanitizes only known internal Strix report warnings" assert_file_contains "$GATE_SCRIPT" "known_internal_warning = re.compile" "strix gate sanitizes known internal console warnings" - assert_file_contains "$GATE_SCRIPT" 'dest.write_text("", encoding="utf-8")' "strix gate clears stale classified console output after decode errors" + assert_file_contains "$GATE_SCRIPT" "dest.write_bytes(src.read_bytes())" "strix gate preserves byte-level console evidence after decode errors" assert_file_contains "$GATE_SCRIPT" 'MODEL QUALITY WARNING' "strix gate accepts the scanner's informational fallback-model banner" assert_file_contains "$GATE_SCRIPT" 'unauthenticated requests to the HF Hub' "strix gate accepts the scanner dependency's non-fatal download warning" assert_file_not_contains "$GATE_SCRIPT" 'known_scanner_warning = re.compile(r".*Warn' "strix gate does not broadly suppress warning-class evidence" From 9f267b8b5b297401dbb4157ff47b872d1b84930c Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 26 Aug 2026 08:50:08 +0900 Subject: [PATCH 12/13] fix(strix): invalidate stale classified console copies --- scripts/ci/strix_quick_gate.sh | 9 ++++++++- scripts/ci/test_strix_quick_gate.sh | 1 + 2 files changed, 9 insertions(+), 1 deletion(-) diff --git a/scripts/ci/strix_quick_gate.sh b/scripts/ci/strix_quick_gate.sh index 7c1017326..cee3aa193 100755 --- a/scripts/ci/strix_quick_gate.sh +++ b/scripts/ci/strix_quick_gate.sh @@ -160,7 +160,14 @@ preserve_attempt_log() { sanitize_strix_console_log() { local src="$1" local dest="$2" - if [ -z "$src" ] || [ -z "$dest" ] || [ ! -f "$src" ] || [ -L "$src" ]; then + if [ -z "$src" ] || [ -z "$dest" ]; then + return 0 + fi + if [ ! -f "$src" ] || [ -L "$src" ]; then + # A missing source must not leave a prior attempt's classified copy active. + if [ -f "$dest" ] && [ ! -L "$dest" ]; then + rm -f -- "$dest" + fi return 0 fi if [ -L "$dest" ]; then diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 5806609b8..4b5df0967 100755 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -333,6 +333,7 @@ assert_strix_workflow_pr_trigger_hardened() { assert_file_contains "$GATE_SCRIPT" "sanitize_known_strix_report_warnings" "strix gate sanitizes only known internal Strix report warnings" assert_file_contains "$GATE_SCRIPT" "known_internal_warning = re.compile" "strix gate sanitizes known internal console warnings" assert_file_contains "$GATE_SCRIPT" "dest.write_bytes(src.read_bytes())" "strix gate preserves byte-level console evidence after decode errors" + assert_file_contains "$GATE_SCRIPT" "A missing source must not leave a prior attempt's classified copy active" "strix gate removes stale classified console copies" assert_file_contains "$GATE_SCRIPT" 'MODEL QUALITY WARNING' "strix gate accepts the scanner's informational fallback-model banner" assert_file_contains "$GATE_SCRIPT" 'unauthenticated requests to the HF Hub' "strix gate accepts the scanner dependency's non-fatal download warning" assert_file_not_contains "$GATE_SCRIPT" 'known_scanner_warning = re.compile(r".*Warn' "strix gate does not broadly suppress warning-class evidence" From e294587328bb07fb1d89ec9fd34c448481945b74 Mon Sep 17 00:00:00 2001 From: Seongho Bae Date: Wed, 26 Aug 2026 10:47:47 +0900 Subject: [PATCH 13/13] docs(copy): make web verification guidance actionable --- CHANGELOG.md | 17 ++++++----------- 1 file changed, 6 insertions(+), 11 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5a75cf9d8..9f1740339 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -81,17 +81,12 @@ Semantic Versioning where the repository publishes a release. ### Fixed -- Restricted sandboxed web E2E readiness polling to standard-library-classified - loopback addresses and literal `localhost`, including IPv6 `::1` and the - complete IPv4 loopback block, resolving `localhost` so every answer must - remain loopback, while rejecting userinfo, missing hosts, IPv4-mapped - public addresses, unspecified bind addresses, and metadata destinations - before any request is opened. -- Strip a cosmetic-only Strix `MODEL QUALITY WARNING` box before - infrastructure-signal matching; if that same box also carries - Fatal/Denied/Timeout or Provider WARNING, keep those lines. Keep a - preceding Fatal/Denied box, and classify a sanitized console copy so - `gate-last-attempt.log` stays raw. +- Web verification now checks services through local readiness addresses only. + Start the backend and frontend on this computer and use their local health + URLs when running the check. +- Review results now separate cosmetic notices from blocking failures. Open the + failure details and correct the requested issue before running the check + again. - Resolve Strix visibility from the trusted GitHub event for ordinary push, schedule, and pull-request runs, reserving API retries for cross-repository dispatches whose workflow token may not see the target repository.