From 9670a189390646705775b6f4781e3adfcf0ea2b9 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 30 Aug 2026 11:21:38 +0000 Subject: [PATCH 01/11] docs: confirm the sidecar-preflight outage is resolved on a live hosted run Item 1 of the coordinator's post-merge priority list: verify end-to-end that noema-review/opencode-review/strix actually recover now that main has both #1436's max_tokens fix and this investigation's family_cap raise, not just structurally argued. Checked ContextualWisdomLab/contextual-orchestrator#921 (deliberately unrelated to this investigation, so it does not itself hit the pull_request_target self-test trust boundary #1434/#1441 do) on a fresh head pushed after main had all three fixes: noema-review and strix both succeeded. noema-review's job log confirms a real sidecar cycle (CONTEXTUAL_ORCHESTRATOR_BASE_URL was exported, which only happens after the sidecar's own preflight found a viable route and the server became healthy -- the exact stage that failed 100% of the time before these fixes). opencode-review still failed, but for an already-understood, unrelated, benign timing gap (it ran before the separate async opencode-review-dispatch flow had posted a verdict for that exact head), not a sidecar regression. Co-Authored-By: Claude --- CHANGELOG.md | 9 ++++ docs/product-technical-gap-baseline.md | 59 ++++++++++++++++++++++++-- 2 files changed, 65 insertions(+), 3 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index fc84661ed..b225d17d4 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -5,6 +5,15 @@ this file. The format follows Keep a Changelog, and versioned releases follow Semantic Versioning where the repository publishes a release. ## [Unreleased] +- Confirm on a live hosted run (`ContextualWisdomLab/contextual-orchestrator#921`) + that the org-wide `noema-review`/`opencode-review`/`strix` sidecar-preflight + outage is resolved by the family_cap raise and #1436's `max_tokens` fix: + `noema-review` and `strix` both succeeded on a fresh head, with the job + log confirming a real, complete sidecar cycle (not a vacuous pass). See + the 2026-08-30 gap-baseline entry for the full evidence and the remaining + caveat (the two permanently-retired `gemma-3` model ids are still admitted + and will still fail individually; the live-catalog-freshness fix remains + the more complete answer if 8 candidates ever proves insufficient again). - Raise `contextual_orchestrator_review_sidecar.sh`'s `ORCHESTRATOR_CATALOG_FAMILY_CAP` default from 4 to 8: root-caused the live "no provider route passed the Strix plain-chat preflight" outage diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 794dc9de9..a7abddbf8 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1288,9 +1288,62 @@ conflicting** PRs address pieces of this: header/`%%EOF`-trailer check, short of full parsing) rather than merging #1427's blanket suffix-trust list, and that the two PRs coordinate so the org does not land two divergent implementations of the same policy - surface. Not resolved in code this pass — both PRs are themselves - currently blocked by the sidecar-preflight outage above, so neither could - be re-reviewed to a genuine pass yet regardless of which approach wins. + surface. Not resolved in code this pass. At the time this was written + both PRs were also blocked by the sidecar-preflight outage below — see + the entry immediately below for confirmation that outage has since + cleared, so a genuine re-review of whichever approach the org picks is + now possible. + +## 2026-08-30 sidecar-preflight outage: confirmed resolved on a live hosted run + +#1434 merged (squash `e36a1f71`) with all three fixes from the entries +above: the Strix `orchestrator/auto`→`orchestrator/free` switch, the +`ORCHESTRATOR_CATALOG_FAMILY_CAP` 4→8 raise, and (via the independent #1436 +merged into `main` mid-pass) the gateway preflight `max_tokens` fix. This +entry is the requested end-to-end verification that the org-wide outage +those three fixes targeted is actually closed, not just structurally +excused. + +- **Verified on `ContextualWisdomLab/contextual-orchestrator#921`** (a PR + unrelated to this investigation, picked specifically because it does not + itself touch review-pipeline files, so it can't hit the same + `pull_request_target` self-test trust boundary PR #1434 and #1441 do), on + a fresh head (`55832c01...`) pushed well after `main` had all three + fixes: `noema-review` — **success**; `strix` — **success**. +- `noema-review`'s job log confirms this was a real, complete sidecar + cycle, not a vacuous pass: `CONTEXTUAL_ORCHESTRATOR_BASE_URL: + http://127.0.0.1:18080` was exported to the step environment, which only + happens after the sidecar's own preflight found a viable route and the + server actually became healthy — the exact stage that failed 100% of the + time (three independent reproductions) before these fixes landed. + `noema_review_gate.py` then exited via its own separate, expected, + unrelated gate ("Current head does not have a primary OpenCode approval; + Noema review skipped") — a normal short-circuit, not a masked failure. +- `opencode-review` still showed `failure` on the same head, but for a + different, already-understood, benign reason unrelated to the sidecar: + its job log shows the same `"No APPROVED or CHANGES_REQUESTED from + opencode-agent on the current head"` gate this investigation has seen + before — this check ran (11:08:08) before the separate, async + `opencode-review-dispatch` flow had time to complete and post a verdict + for this exact head (confirmed: no `opencode-agent` review exists on the + PR at all yet at the time of this check). This is a timing/sequencing + gap this org's own scheduler design already re-dispatches for, not a + sidecar regression. +- **Conclusion**: the root cause diagnosed and fixed in this investigation + (family-cap-driven deterministic admission of retired/slow candidates, + and separately the desynchronized `max_tokens` on the post-`healthz` + smoke request) is confirmed closed by a real hosted run, not merely + structurally argued. This does not mean the free catalog is now perfect + — the two permanently-retired `google/gemma-3-*-it` model ids are still + in the pool and will still individually fail when the alphabetical sort + reaches them (see the family-cap entry above for the still-open, + more-complete live-catalog-freshness fix) — only that with 8 candidates + instead of 4, enough of the other ~19 free `nvidia_nim`/`nvidia_nim_sub` + models are now reached to find a working route. If a future hosted run + shows `noema-review`/`opencode-review`/`strix` failing again with the + `"no provider route passed"` signature, that is the signal the + live-catalog-freshness fix is now the priority, not a further family_cap + increase. ## 5. 실행 루프와 고객의 다음 행동 From 7848ed17aa1ce9e0d8d20f25b3312a5a62d8914f Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 30 Aug 2026 11:24:57 +0000 Subject: [PATCH 02/11] docs: correct two overclaims in the sidecar-outage/pingora gap-baseline entries 1. Soften "the sidecar-preflight outage is confirmed resolved" to accurately reflect a second, independently-found, timestamped counter-example: a strix run on .github's own main (job 99247611184, completed 2026-08-30T11:16:17Z -- predating this PR's original claim) failed with a genuinely separate, unfixed defect verified directly from its job log: Strix's OpenAI Agents SDK client sends stream_options.include_usage=true together with tools/response_format, and the routed orchestrator/free candidate rejected that combination with HTTP 400 invalid_stream_options on all retries, exhausting the pool (STRIX_PROVIDER_UNAVAILABLE). The family_cap/max_tokens fixes are confirmed working end-to-end on contextual-orchestrator#921's real run; that specific, previously 100%-reproducible failure mode is closed. It is not true that orchestrator/free is now reliable in general. Root-causing the stream_options gap has been delegated elsewhere; not duplicated here. 2. Update the pingora_edge_policy.py binary-evidence entry from "two competing open fixes, unresolved" to reflect that #1435 already merged -- verified against its actual diff, not assumed from its title: a third, better implementation that network-verifies the real %PDF- magic prefix rather than trusting the .pdf extension alone, falling back to path+suffix trust only for the one case that cannot be verified by content at all (a genuinely oversized file). #1420/#1427 remain open for the separate, non-blocking question of the other binary formats. Co-Authored-By: Claude --- docs/product-technical-gap-baseline.md | 188 ++++++++++++++----------- 1 file changed, 105 insertions(+), 83 deletions(-) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index a7abddbf8..863158d56 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1255,95 +1255,117 @@ direct-NVIDIA-NIM communication is a removal target. (already dead) or removing the one resilience mechanism keeping a required check alive during a live outage. -## 2026-08-30 pingora_edge_policy.py binary-evidence gap: two competing open fixes +## 2026-08-30 pingora_edge_policy.py binary-evidence gap: resolved for PDF by #1435 -A live failure on `ContextualWisdomLab/contextual-orchestrator#906`'s `required-workflow-bootstrap` +**Resolved — updated from an earlier "two competing open fixes" framing +after #1435 merged (squash, `2026-08-30T11:10:11Z`) with a third, +independent implementation, verified directly against its real diff, not +assumed from a title.** A live failure on +`ContextualWisdomLab/contextual-orchestrator#906`'s `required-workflow-bootstrap` job (`GitHub content evidence for docs/papers/helm-holistic-evaluation-2211.09110.pdf -is not a regular base64 file`) traces to `scripts/ci/pingora_edge_policy.py`'s +is not a regular base64 file`) originally traced to `scripts/ci/pingora_edge_policy.py`'s `_load_file_content`: GitHub's Contents API stops returning inline `encoding: "base64"` once a file crosses roughly 1 MB (returning -`encoding: "none"` + a `download_url` instead), and this policy scanner's -`_needs_content_scan` has no exemption for genuinely binary evidence files in -general — any added/modified file without a `patch` (i.e. any binary file, -regardless of size) reaches `_load_file_content`, which always fails once it -tries `raw.decode("utf-8")`. Two **already-open, independent, partially -conflicting** PRs address pieces of this: - -- **#1420** adds real, structural validation (`_is_recognized_documentation_image`: - PNG magic header, chunk order, CRC, zlib-stream, dimension, and scanline - checks) so an image *suffix* alone cannot exempt a file — consistent with - this policy's own stated principle. Covers `.png` only; does not touch - `.pdf`, so it would not by itself fix `ContextualWisdomLab/contextual-orchestrator#906`. -- **#1427** adds a flat `NON_RUNTIME_BINARY_SUFFIXES` allowlist (`.avif`, - `.gif`, `.ico`, `.jpeg`, `.jpg`, `.pdf`, `.png`, `.webp`) that skips - content-scanning by **extension alone**, no byte-level verification. This - does fix `ContextualWisdomLab/contextual-orchestrator#906`, but for every - suffix in that list (not just `.pdf`) it - reintroduces the exact "extension alone is not an exception" gap #1420 - exists to close for PNG — a shell/config file renamed to `evidence.pdf` - (or `.png`, `.jpg`, ...) would now bypass the Nginx-runtime-artifact scan - entirely. -- Left substantive comments on both PRs (this pass) recommending #1420's - structural-validation pattern be extended to `.pdf` (a bounded magic- - header/`%%EOF`-trailer check, short of full parsing) rather than merging - #1427's blanket suffix-trust list, and that the two PRs coordinate so the - org does not land two divergent implementations of the same policy - surface. Not resolved in code this pass. At the time this was written - both PRs were also blocked by the sidecar-preflight outage below — see - the entry immediately below for confirmation that outage has since - cleared, so a genuine re-review of whichever approach the org picks is - now possible. - -## 2026-08-30 sidecar-preflight outage: confirmed resolved on a live hosted run - -#1434 merged (squash `e36a1f71`) with all three fixes from the entries -above: the Strix `orchestrator/auto`→`orchestrator/free` switch, the -`ORCHESTRATOR_CATALOG_FAMILY_CAP` 4→8 raise, and (via the independent #1436 -merged into `main` mid-pass) the gateway preflight `max_tokens` fix. This -entry is the requested end-to-end verification that the org-wide outage -those three fixes targeted is actually closed, not just structurally -excused. - -- **Verified on `ContextualWisdomLab/contextual-orchestrator#921`** (a PR - unrelated to this investigation, picked specifically because it does not +`encoding: "none"` instead), and the policy scanner had no exemption for +genuinely binary evidence files — any binary file without a diff `patch` +reached `_load_file_content`, which always failed on `raw.decode("utf-8")`. + +- **#1435's approach is better than either of the two PRs this entry + previously compared** (#1420's PNG-only structural validation; #1427's + blanket-suffix-trust list this entry recommended against): it + network-verifies the real `%PDF-` magic prefix whenever the file's bytes + can actually be fetched (`_pdf_evidence_confirms_binary`, + `_load_raw_file_bytes`), so a shell/config file renamed to `.pdf` under a + documentation path is still caught and scanned — it does **not** trust + the extension alone. It falls back to the path+suffix convention only for + the one case that genuinely cannot be verified by content at all: a file + whose declared size exceeds the Contents API's fetch ceiling (the exact, + real research-paper-citation case this exemption exists for), gated + through a new `ContentSizeExceededError` distinct from every other + content-evidence failure, which still fails closed exactly as before. New + regression coverage in `tests/test_pingora_edge_policy.py` covers the + oversized-real-PDF case (using GitHub's actual `encoding: "none"` + response shape, not a synthetic one), a disguised textual `.pdf` with a + patch still getting scanned, a genuine small PDF verified by magic bytes, + and a removed PDF not being fetched at all. + `ContextualWisdomLab/contextual-orchestrator#906`'s original failure is + fixed by this. +- **Scope note, not a gap in #1435**: this resolves `.pdf` specifically. + #1420 (still open) separately covers `.png` with equivalent + content-verified rigor (magic header, chunk/CRC/zlib/scanline checks); + #1427 (still open) still proposes the broader, weaker blanket-suffix-trust + list (`.avif`/`.gif`/`.ico`/`.jpeg`/`.jpg`/`.png`/`.webp`, extension alone, + no verification) for the remaining binary formats. The recommendation + from this investigation's earlier pass stands for whichever of those two + the org picks next: extend #1435/#1420's verified-content pattern to the + remaining formats rather than merging #1427's unverified allowlist as-is. + Neither PR is blocking anything today — the specific live failure that + motivated this whole entry is closed. + +## 2026-08-30 sidecar-preflight outage: family_cap/max_tokens fixes confirmed working end to end, but a separate compatibility gap can still fail Strix + +**Softened from an earlier "confirmed resolved" framing in this same entry** +after a second, independently-found, timestamped counter-example — see +below. #1434 merged (squash `e36a1f71`) with all three fixes from the +entries above: the Strix `orchestrator/auto`→`orchestrator/free` switch, +the `ORCHESTRATOR_CATALOG_FAMILY_CAP` 4→8 raise, and (via the independent +#1436 merged into `main` mid-pass) the gateway preflight `max_tokens` fix. + +- **The family_cap/max_tokens fixes are confirmed working end-to-end on at + least one real run.** Checked `ContextualWisdomLab/contextual-orchestrator#921` + (unrelated to this investigation, picked specifically because it does not itself touch review-pipeline files, so it can't hit the same - `pull_request_target` self-test trust boundary PR #1434 and #1441 do), on - a fresh head (`55832c01...`) pushed well after `main` had all three - fixes: `noema-review` — **success**; `strix` — **success**. -- `noema-review`'s job log confirms this was a real, complete sidecar - cycle, not a vacuous pass: `CONTEXTUAL_ORCHESTRATOR_BASE_URL: - http://127.0.0.1:18080` was exported to the step environment, which only - happens after the sidecar's own preflight found a viable route and the - server actually became healthy — the exact stage that failed 100% of the + `pull_request_target` self-test trust boundary PR #1434/#1441 do), on a + fresh head (`55832c01...`) pushed after `main` had all three fixes: + `noema-review` — **success**; `strix` — **success**. `noema-review`'s job + log confirms a real, complete sidecar cycle, not a vacuous pass: + `CONTEXTUAL_ORCHESTRATOR_BASE_URL: http://127.0.0.1:18080` was exported, + which only happens after the sidecar's own preflight found a viable route + and the server became healthy — the exact stage that failed 100% of the time (three independent reproductions) before these fixes landed. - `noema_review_gate.py` then exited via its own separate, expected, - unrelated gate ("Current head does not have a primary OpenCode approval; - Noema review skipped") — a normal short-circuit, not a masked failure. -- `opencode-review` still showed `failure` on the same head, but for a - different, already-understood, benign reason unrelated to the sidecar: - its job log shows the same `"No APPROVED or CHANGES_REQUESTED from - opencode-agent on the current head"` gate this investigation has seen - before — this check ran (11:08:08) before the separate, async - `opencode-review-dispatch` flow had time to complete and post a verdict - for this exact head (confirmed: no `opencode-agent` review exists on the - PR at all yet at the time of this check). This is a timing/sequencing - gap this org's own scheduler design already re-dispatches for, not a - sidecar regression. -- **Conclusion**: the root cause diagnosed and fixed in this investigation - (family-cap-driven deterministic admission of retired/slow candidates, - and separately the desynchronized `max_tokens` on the post-`healthz` - smoke request) is confirmed closed by a real hosted run, not merely - structurally argued. This does not mean the free catalog is now perfect - — the two permanently-retired `google/gemma-3-*-it` model ids are still - in the pool and will still individually fail when the alphabetical sort - reaches them (see the family-cap entry above for the still-open, - more-complete live-catalog-freshness fix) — only that with 8 candidates - instead of 4, enough of the other ~19 free `nvidia_nim`/`nvidia_nim_sub` - models are now reached to find a working route. If a future hosted run - shows `noema-review`/`opencode-review`/`strix` failing again with the - `"no provider route passed"` signature, that is the signal the - live-catalog-freshness fix is now the priority, not a further family_cap - increase. + `opencode-review` failed on the same head, but for an already-understood, + benign, unrelated timing gap (it ran before the async + `opencode-review-dispatch` flow had posted a verdict for that exact + head). +- **A further, distinct compatibility gap can still cause Strix failures + depending on which provider gets selected — not yet fixed.** A `strix` + run on `.github`'s own `main` (job `99247611184`, completed + `2026-08-30T11:16:17Z`) failed independently, verified directly from its + job log rather than assumed: Strix's OpenAI Agents SDK client sends + `stream_options.include_usage=true` together with `tools`/ + `response_format` in the same request; whichever `orchestrator/free` + candidate this run routed to (log evidence shows an `integrate. + api.nvidia.com` DNS resolution around the same window, consistent with an + `nvidia_nim` route) rejected that exact combination — + `openai.BadRequestError: 400 {'code': 'invalid_stream_options', 'message': + 'stream_options.include_usage=true is not supported with tools or + response_format'}` — on all retries (two separate penetration-test + attempts, `11:12:34Z` and `11:16:12Z`, both hit the identical error), + ending in `STRIX_PROVIDER_UNAVAILABLE: contextual-orchestrator/orchestrator/free + exhausted; the gateway owns provider discovery and failover`. This + predates PR #1444's original "confirmed resolved" claim (`11:16:17Z` vs. + that PR's `11:21:52Z` creation) and is a genuinely separate defect from + both fixes above — a request-shape incompatibility between the OpenAI + Agents SDK's fixed request parameters and at least one upstream + provider's API, not a candidate-selection or token-budget problem. + Root-causing this has been delegated to a separate, already-in-flight + investigation (contextual-orchestrator agent); not duplicated here. +- **Accurate combined conclusion**: the family_cap-driven deterministic + admission of dead/slow candidates, and separately the desynchronized + `max_tokens` on the post-`healthz` smoke request, are fixed and confirmed + by a real hosted run — that specific, previously 100%-reproducible + failure mode is closed. It is not true that `orchestrator/free` is now + reliable in general: with `family_cap=8` giving more provider diversity + per run, *which* candidate a given run draws varies, and at least one + candidate family has a live, separate, unfixed request-compatibility + gap that can still fail Strix (though evidently not every candidate — + contextual-orchestrator#921's `strix` run succeeded). Two permanently + -retired `google/gemma-3-*-it` model ids are also still admitted into + the pool (see the family-cap entry above) and will still individually + fail when the alphabetical sort reaches them. None of this changes the + scope of what was actually fixed in this pass; it means "the outage is + over" would overclaim, while "the two diagnosed root causes are fixed + and verified, one further known gap remains open elsewhere" is accurate. ## 5. 실행 루프와 고객의 다음 행동 From 5a75ec7793a1a5aa76bc8e340ea722feb5348159 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 30 Aug 2026 11:25:36 +0000 Subject: [PATCH 03/11] docs: soften the CHANGELOG's sidecar-outage claim to match the corrected gap-baseline entry Companion to the prior commit's gap-baseline correction -- the CHANGELOG bullet had the same overclaim and needed the same softening. Co-Authored-By: Claude --- CHANGELOG.md | 21 ++++++++++++++------- 1 file changed, 14 insertions(+), 7 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b225d17d4..1c26f9833 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -6,14 +6,21 @@ Semantic Versioning where the repository publishes a release. ## [Unreleased] - Confirm on a live hosted run (`ContextualWisdomLab/contextual-orchestrator#921`) - that the org-wide `noema-review`/`opencode-review`/`strix` sidecar-preflight - outage is resolved by the family_cap raise and #1436's `max_tokens` fix: + that the family_cap raise and #1436's `max_tokens` fix work end-to-end: `noema-review` and `strix` both succeeded on a fresh head, with the job - log confirming a real, complete sidecar cycle (not a vacuous pass). See - the 2026-08-30 gap-baseline entry for the full evidence and the remaining - caveat (the two permanently-retired `gemma-3` model ids are still admitted - and will still fail individually; the live-catalog-freshness fix remains - the more complete answer if 8 candidates ever proves insufficient again). + log confirming a real, complete sidecar cycle (not a vacuous pass). This + is not the same as the outage being fully closed: a second, separately + found and independently verified run + (`.github`'s own `main`, job `99247611184`) failed with a genuinely + different, still-open defect (Strix's OpenAI Agents SDK client sending + `stream_options.include_usage=true` with `tools`/`response_format`, + rejected by at least one `orchestrator/free` candidate with `HTTP 400 + invalid_stream_options`, exhausting the pool). See the 2026-08-30 + gap-baseline entry for the full evidence and remaining caveats (that + `stream_options` gap, the two permanently-retired `gemma-3` model ids + still admitted into the pool, and the live-catalog-freshness fix that + remains the more complete answer if 8 candidates ever proves + insufficient again). - Raise `contextual_orchestrator_review_sidecar.sh`'s `ORCHESTRATOR_CATALOG_FAMILY_CAP` default from 4 to 8: root-caused the live "no provider route passed the Strix plain-chat preflight" outage From 9b4288dd625938fe391dc016871a88f40937cfa0 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 30 Aug 2026 11:34:21 +0000 Subject: [PATCH 04/11] docs: record a third, distinct sidecar failure mode (gateway preflight timeout) Verified directly from ContextualWisdomLab/.github#1441's noema-review job log (99249903390, head d877886b): healthz and the launcher's own internal preflight succeeded (past the family-cap fix's stage entirely), but the shell script's separate post-healthz gateway smoke request -- max_tokens now 4096 per #1436's fix, up from 16 -- got zero bytes back for the full 120-second curl timeout. Distinct from both other evidence in this entry: not a bad-request rejection (no response at all) and not Strix's own client code path. Documented with plausible causes (the 120s ceiling never re-examined when max_tokens grew 256x; a real server-side hang for a specific request shape; transient flakiness) but not pursued further this pass -- noted as evidence for whoever picks this up next. Co-Authored-By: Claude --- docs/product-technical-gap-baseline.md | 28 ++++++++++++++++++++++++++ 1 file changed, 28 insertions(+) diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 863158d56..1bfc8fc8c 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1350,6 +1350,34 @@ the `ORCHESTRATOR_CATALOG_FAMILY_CAP` 4→8 raise, and (via the independent provider's API, not a candidate-selection or token-budget problem. Root-causing this has been delegated to a separate, already-in-flight investigation (contextual-orchestrator agent); not duplicated here. +- **A third, distinct failure mode, also verified directly, also not yet + fixed**: `noema-review` on `ContextualWisdomLab/.github#1441` (job + `99249903390`, head `d877886b`) reached `healthz and provider-route + preflight confirmed after 41s` — the launcher's own internal preflight + found a viable route, past the family-cap fix's stage entirely — logged + the same non-fatal `provider_discovery_failed provider=bytez + code=http_status_500` warning seen elsewhere in this investigation, and + then the shell script's own separate post-`healthz` gateway smoke + request (the one #1436 fixed the `max_tokens` on, now `4096`, up from + `16`) got **zero bytes back for the full 120-second `curl` timeout**: + `curl: (28) Operation timed out after 120002 milliseconds with 0 bytes + received` → `gateway preflight request could not reach the local + sidecar`. Distinct from both defects above: not a bad-request rejection + (no error response at all, just silence) and not Strix's own client code + path (this is the sidecar script's own `curl` probe). Plausible causes, + none confirmed: (a) `max_tokens=4096` now lets a route legitimately spend + much longer generating (heavy internal reasoning, no early return) than + the old `16`-token budget ever could, genuinely exceeding a 120s ceiling + that was never re-examined when the token budget grew 256x; (b) a real + hang/deadlock in the orchestrator server for a specific request shape; + (c) unrelated transient network flakiness. Not pursued further this pass + — noted as evidence for whoever picks this up next. If this recurs across + multiple runs, the 120s `curl --max-time` in + `contextual_orchestrator_review_sidecar.sh`'s gateway-preflight step may + need raising alongside `max_tokens` (same reasoning #1436 already + applied: a bigger token budget can legitimately need more wall-clock + time), or the specific route it lands on needs identifying and + deprioritizing. - **Accurate combined conclusion**: the family_cap-driven deterministic admission of dead/slow candidates, and separately the desynchronized `max_tokens` on the post-`healthz` smoke request, are fixed and confirmed From e4c026519e5d60dea03c28be492e65714fc58eab Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 30 Aug 2026 11:45:19 +0000 Subject: [PATCH 05/11] docs(gaps): correct #1444's gap count (Devin Review finding) The "one further known gap remains" conclusion undercounted: the same entry already documented two separate, unfixed failure modes (the stream_options 400 and the 120s-timeout gap). Also corrects a second, premature claim that contextual-orchestrator#924 already fixed the stream_options gap -- checked directly: that PR is still open and mergeable_state=blocked, not merged, so it is a proposed fix only. Co-Authored-By: Claude --- CHANGELOG.md | 24 ++++++++++-------- docs/product-technical-gap-baseline.md | 35 ++++++++++++++++++-------- 2 files changed, 39 insertions(+), 20 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1c26f9833..28b4f7ad3 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,18 +9,22 @@ Semantic Versioning where the repository publishes a release. that the family_cap raise and #1436's `max_tokens` fix work end-to-end: `noema-review` and `strix` both succeeded on a fresh head, with the job log confirming a real, complete sidecar cycle (not a vacuous pass). This - is not the same as the outage being fully closed: a second, separately - found and independently verified run - (`.github`'s own `main`, job `99247611184`) failed with a genuinely - different, still-open defect (Strix's OpenAI Agents SDK client sending + is not the same as the outage being fully closed: two further, + independent, still-open defects were separately found and verified. + First, a run on `.github`'s own `main` (job `99247611184`) failed + because Strix's OpenAI Agents SDK client sends `stream_options.include_usage=true` with `tools`/`response_format`, rejected by at least one `orchestrator/free` candidate with `HTTP 400 - invalid_stream_options`, exhausting the pool). See the 2026-08-30 - gap-baseline entry for the full evidence and remaining caveats (that - `stream_options` gap, the two permanently-retired `gemma-3` model ids - still admitted into the pool, and the live-catalog-freshness fix that - remains the more complete answer if 8 candidates ever proves - insufficient again). + invalid_stream_options`, exhausting the pool — a fix is proposed in + `ContextualWisdomLab/contextual-orchestrator#924` but that PR is not + merged yet. Second, a separate run (`.github#1441` job `99249903390`) + got past that stage entirely and instead hung for the full 120s `curl` + timeout with zero bytes back on the sidecar's post-`healthz` gateway + smoke request; cause unconfirmed. See the 2026-08-30 gap-baseline entry + for the full evidence and remaining caveats on both gaps, the two + permanently-retired `gemma-3` model ids still admitted into the pool, + and the live-catalog-freshness fix that remains the more complete + answer if 8 candidates ever proves insufficient again. - Raise `contextual_orchestrator_review_sidecar.sh`'s `ORCHESTRATOR_CATALOG_FAMILY_CAP` default from 4 to 8: root-caused the live "no provider route passed the Strix plain-chat preflight" outage diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 1bfc8fc8c..3ce757745 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1378,22 +1378,37 @@ the `ORCHESTRATOR_CATALOG_FAMILY_CAP` 4→8 raise, and (via the independent applied: a bigger token budget can legitimately need more wall-clock time), or the specific route it lands on needs identifying and deprioritizing. +- **Correction (Devin Review finding on this PR, verified directly before + editing): the conclusion below previously undercounted the open gaps as + "one further known gap" when the same entry already documented two + separate, unfixed failure modes** (the `stream_options` 400 and the + 120s-timeout gap, both documented above). Also checked directly rather than assumed: + `ContextualWisdomLab/contextual-orchestrator#924` (opened after this + entry was first written, proposing to fix the `stream_options` gap by + removing the upfront rejection since `stream_options` was already + stripped before every upstream provider call) is **still open and + `mergeable_state: blocked` as of this correction — not merged** — so + that gap is not closed yet either, only diagnosed with a fix proposed. - **Accurate combined conclusion**: the family_cap-driven deterministic admission of dead/slow candidates, and separately the desynchronized `max_tokens` on the post-`healthz` smoke request, are fixed and confirmed by a real hosted run — that specific, previously 100%-reproducible failure mode is closed. It is not true that `orchestrator/free` is now reliable in general: with `family_cap=8` giving more provider diversity - per run, *which* candidate a given run draws varies, and at least one - candidate family has a live, separate, unfixed request-compatibility - gap that can still fail Strix (though evidently not every candidate — - contextual-orchestrator#921's `strix` run succeeded). Two permanently - -retired `google/gemma-3-*-it` model ids are also still admitted into - the pool (see the family-cap entry above) and will still individually - fail when the alphabetical sort reaches them. None of this changes the - scope of what was actually fixed in this pass; it means "the outage is - over" would overclaim, while "the two diagnosed root causes are fixed - and verified, one further known gap remains open elsewhere" is accurate. + per run, *which* candidate a given run draws varies, and **two further, + independent, unfixed gaps remain open**: (1) at least one candidate + family has the live `stream_options`/`tools`/`response_format` + request-compatibility rejection above, with a fix proposed but not yet + merged in `contextual-orchestrator#924` (though evidently not every + candidate hits it — contextual-orchestrator#921's `strix` run + succeeded); and (2) the separate 120s-timeout-with-zero-bytes gap below, + cause unconfirmed. Two permanently-retired `google/gemma-3-*-it` model + ids are also still admitted into the pool (see the family-cap entry + above) and will still individually fail when the alphabetical sort + reaches them. None of this changes the scope of what was actually fixed + in this pass; it means "the outage is over" would overclaim, while "the + two diagnosed root causes are fixed and verified, two further known + gaps remain open elsewhere" is accurate. ## 5. 실행 루프와 고객의 다음 행동 From 27b387f8d8d891bf1cb98ea00c2b665a64ccb362 Mon Sep 17 00:00:00 2001 From: Claude Date: Sun, 30 Aug 2026 11:48:00 +0000 Subject: [PATCH 06/11] docs(gaps): use full owner/repo#num cross-repo refs (Devin Review finding) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bare #924/#921/.github#1441 refs in the text I just added violate the CLAUDE.md §7 / CWL-MASTER-CONTEXT.md §7 cross-repo reference convention. Also fixes a leftover "below" that should have said "above" for the timeout-gap section, missed in the previous correction pass. Co-Authored-By: Claude --- CHANGELOG.md | 3 ++- docs/product-technical-gap-baseline.md | 7 ++++--- 2 files changed, 6 insertions(+), 4 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 28b4f7ad3..0be664a5c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,7 +17,8 @@ Semantic Versioning where the repository publishes a release. rejected by at least one `orchestrator/free` candidate with `HTTP 400 invalid_stream_options`, exhausting the pool — a fix is proposed in `ContextualWisdomLab/contextual-orchestrator#924` but that PR is not - merged yet. Second, a separate run (`.github#1441` job `99249903390`) + merged yet. Second, a separate run + (`ContextualWisdomLab/.github#1441` job `99249903390`) got past that stage entirely and instead hung for the full 120s `curl` timeout with zero bytes back on the sidecar's post-`healthz` gateway smoke request; cause unconfirmed. See the 2026-08-30 gap-baseline entry diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 3ce757745..a3b5e74ea 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1399,9 +1399,10 @@ the `ORCHESTRATOR_CATALOG_FAMILY_CAP` 4→8 raise, and (via the independent independent, unfixed gaps remain open**: (1) at least one candidate family has the live `stream_options`/`tools`/`response_format` request-compatibility rejection above, with a fix proposed but not yet - merged in `contextual-orchestrator#924` (though evidently not every - candidate hits it — contextual-orchestrator#921's `strix` run - succeeded); and (2) the separate 120s-timeout-with-zero-bytes gap below, + merged in `ContextualWisdomLab/contextual-orchestrator#924` (though + evidently not every candidate hits it — + `ContextualWisdomLab/contextual-orchestrator#921`'s `strix` run + succeeded); and (2) the separate 120s-timeout-with-zero-bytes gap above, cause unconfirmed. Two permanently-retired `google/gemma-3-*-it` model ids are also still admitted into the pool (see the family-cap entry above) and will still individually fail when the alphabetical sort From 0bc6cb7d7b163b422c8701f88f2bd9bf45ee4306 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 1 Sep 2026 00:57:55 +0000 Subject: [PATCH 07/11] fix(ci): bound required-workflow-bootstrap awk extraction to its own job Ports the identical fix from #1506 into this branch. This PR's exact-head-path-policy check runs its own head-branch copy of scripts/ci/test_strix_quick_gate.sh (plain `pull_request` trigger in strix-changed-path-quality-ci.yml, not pull_request_target), so the pre-existing main-branch bug is not fixed here just by #1506 merging into main -- it needs porting into this branch directly. Root cause: assert_opencode_review_uses_codegraph_and_contextual_orchestrator extracted the required-workflow-bootstrap job block from opencode-review.yml with awk '/^ required-workflow-bootstrap:$/,/^[^ ]/'. Every job key in that workflow is indented 2 spaces (never column 0), so the end pattern never matched until EOF, sweeping an unrelated `if:` line from a later job into the "block" and failing the assertion on unrelated content. Fixed by using an explicit state flag so the end pattern (`^ [A-Za-z0-9_-]+:`) is only tested starting on the line after the start match, correctly bounding the block to just its own lines. Confirmed FAIL before this fix, PASS after (bash scripts/ci/test_strix_quick_gate.sh). See ContextualWisdomLab/.github#1506 for the full root-cause writeup. Co-Authored-By: Claude --- scripts/ci/test_strix_quick_gate.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/ci/test_strix_quick_gate.sh b/scripts/ci/test_strix_quick_gate.sh index 4053f4fd5..1fc45a34b 100644 --- a/scripts/ci/test_strix_quick_gate.sh +++ b/scripts/ci/test_strix_quick_gate.sh @@ -522,7 +522,7 @@ assert_opencode_review_uses_codegraph_and_contextual_orchestrator() { assert_file_not_contains "$workflow_file" "Wait for trusted OpenCode approval review" "opencode pull_request bridge was removed to avoid duplicate required-check resource use" assert_file_not_contains "$workflow_file" "Trusted OpenCode requested changes for head" "opencode pull_request bridge no longer reconsumes stale trusted review state" assert_file_not_contains "$workflow_file" "github.event.pull_request.number == 240" "opencode review workflow must not hard-code repository-specific PR bypasses" - if awk '/^ required-workflow-bootstrap:$/,/^[^ ]/' "$bootstrap_file" | grep -q '^[[:space:]]*if:'; then + if awk '/^ required-workflow-bootstrap:$/{p=1; print; next} p && /^ [A-Za-z0-9_-]+:/{exit} p' "$bootstrap_file" | grep -q '^[[:space:]]*if:'; then record_failure "opencode required workflow bootstrap must not depend on required-workflow event payload fields" fi assert_file_contains "$workflow_file" 'github.event.client_payload.target_repository || github.repository' "opencode review scopes concurrency by target repository" From 9ecf8e2d92dd6cfa19718307c47382e038e4df55 Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 1 Sep 2026 00:57:56 +0000 Subject: [PATCH 08/11] docs(gaps): record that the two remaining sidecar gaps are now fixed Devin Review finding on this PR's pushed head: the gap-baseline's 2026-08-30 sidecar-preflight conclusion (and this PR's own CHANGELOG bullet restating it) sent operators toward two gaps that are now closed, overcounting what remains open. Verified directly against the merged diffs, not assumed from the finding text: the stream_options/tools/response_format rejection is fixed by .github#1448 (commit 702392a2, merged after this PR was opened) -- strix_quick_gate.sh now scopes Strix's LLM_DISABLE_STREAMING opt-in to the contextual-orchestrator loopback, so that combination is never sent to it. The 120s-timeout-with-zero- bytes gap is fixed by .github#1452 (commit 1ff82682, merged the same day, implementing ADR-0005) -- the sidecar's own code comments cite a live reproduction of this exact failure class (ContextualWisdomLab/.github#1449, job 99253418179) and the post-healthz gateway smoke request now retries up to REVIEW_PREFLIGHT_GATEWAY_MAX_ATTEMPTS instead of failing closed after one single-shot attempt. Appends an update to the gap-baseline entry (repo convention: append corrections, don't rewrite the historical narrative) and amends this PR's own still-unmerged CHANGELOG bullet in place, matching how prior corrections to this same bullet (e4c02651) were made. Co-Authored-By: Claude --- CHANGELOG.md | 31 +++++++++++++++----------- docs/product-technical-gap-baseline.md | 28 +++++++++++++++++++++++ 2 files changed, 46 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 0cc0bf758..e4d260873 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -10,19 +10,24 @@ Semantic Versioning where the repository publishes a release. `noema-review` and `strix` both succeeded on a fresh head, with the job log confirming a real, complete sidecar cycle (not a vacuous pass). This is not the same as the outage being fully closed: two further, - independent, still-open defects were separately found and verified. - First, a run on `.github`'s own `main` (job `99247611184`) failed - because Strix's OpenAI Agents SDK client sends - `stream_options.include_usage=true` with `tools`/`response_format`, - rejected by at least one `orchestrator/free` candidate with `HTTP 400 - invalid_stream_options`, exhausting the pool — a fix is proposed in - `ContextualWisdomLab/contextual-orchestrator#924` but that PR is not - merged yet. Second, a separate run - (`ContextualWisdomLab/.github#1441` job `99249903390`) - got past that stage entirely and instead hung for the full 120s `curl` - timeout with zero bytes back on the sidecar's post-`healthz` gateway - smoke request; cause unconfirmed. See the 2026-08-30 gap-baseline entry - for the full evidence and remaining caveats on both gaps, the two + independent defects were separately found and verified — both since + fixed by later, already-merged PRs (confirmed directly against their + merged diffs during this rebase, not assumed). First, a run on + `.github`'s own `main` (job `99247611184`) failed because Strix's OpenAI + Agents SDK client sent `stream_options.include_usage=true` with + `tools`/`response_format`, rejected by at least one `orchestrator/free` + candidate with `HTTP 400 invalid_stream_options`, exhausting the pool — + fixed by `.github#1448` (commit `702392a2`), which scopes Strix's + `LLM_DISABLE_STREAMING` opt-in to the contextual-orchestrator loopback so + that combination is never sent to it. Second, a separate run + (`ContextualWisdomLab/.github#1441` job `99249903390`) got past that + stage entirely and instead hung for the full 120s `curl` timeout with + zero bytes back on the sidecar's post-`healthz` gateway smoke request — + fixed by `.github#1452` (commit `1ff82682`, implementing ADR-0005), which + retries that request up to `REVIEW_PREFLIGHT_GATEWAY_MAX_ATTEMPTS` (3) on + a transport failure instead of failing closed after one single-shot + attempt. See the 2026-08-30 gap-baseline entry (and its 2026-09-01 + update) for the full evidence on both gaps and their fixes, the two permanently-retired `gemma-3` model ids still admitted into the pool, and the live-catalog-freshness fix that remains the more complete answer if 8 candidates ever proves insufficient again. diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 3deeccaa3..5950d8341 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1410,6 +1410,34 @@ the `ORCHESTRATOR_CATALOG_FAMILY_CAP` 4→8 raise, and (via the independent in this pass; it means "the outage is over" would overclaim, while "the two diagnosed root causes are fixed and verified, two further known gaps remain open elsewhere" is accurate. +- **Update (2026-09-01, Devin Review finding on `.github#1444`, verified + directly against the merged diffs rather than trusted from the finding + text alone): both of the two "further, independent, unfixed gaps" in the + conclusion above are now closed.** Neither fix existed yet when this + entry (or PR #1444, which restates it) was originally written, so this is + a genuine update, not a contradiction of the historical narrative above. + (1) The `stream_options.include_usage=true` + `tools`/`response_format` + rejection is fixed by `.github#1448` (commit `702392a2`, merged + 2026-08-30T12:15:28Z): `strix_quick_gate.sh` now scopes Strix's + `LLM_DISABLE_STREAMING` opt-in to the contextual-orchestrator loopback, so + Strix's always-tools turns against that gateway never send + `stream_options.include_usage=true` in the first place — the exact + combination at least one `orchestrator/free` candidate was rejecting with + `HTTP 400 invalid_stream_options`. `contextual-orchestrator#924` (cited + above as the proposed, not-yet-merged fix) remains open and is superseded + by this Strix-side fix for this specific symptom; other providers in + Strix's fallback chain keep real SSE streaming. (2) The + 120s-timeout-with-zero-bytes gap is fixed by `.github#1452` (commit + `1ff82682`, merged 2026-08-30T14:54:45Z, implementing ADR-0005): the + sidecar's own code comments now cite a live reproduction of this exact + failure class (`ContextualWisdomLab/.github#1449`, job `99253418179`, + `curl` timing out at exactly 120002ms with zero bytes received — the same + symptom as this entry's job `99249903390`), and the post-`healthz` gateway + smoke request is now retried up to `REVIEW_PREFLIGHT_GATEWAY_MAX_ATTEMPTS` + (default 3) on a transport failure or non-2xx status instead of failing + closed after one single-shot 120s attempt. The two permanently-retired + `gemma-3` model ids gap (see the family-cap entry above) is unrelated to + either fix and remains open. ## 2026-08-30 PR #1347 Devin Review 6건 검증: 4건 실재 결함 수정, 2건 확인 후 해소 From fe8390dfc501b35a4e5e0df8bcc16c6eaa12269a Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 1 Sep 2026 01:07:52 +0000 Subject: [PATCH 09/11] test(ci): refresh review-dispatch blob pin and head-advance assertion after rebase Concurrent main PRs (#1532, #1533) legitimately changed .github/workflows/opencode-review-dispatch.yml since this branch's last rebase, and this rebase's merge picked those changes up byte-for-byte (confirmed: `git diff origin/main -- .github/workflows/opencode-review-dispatch.yml` is empty). Two pre-existing contract tests were left pointing at stale expectations by that upstream change -- reproducible on origin/main's own tip, not introduced by this branch's diff: - REVIEW_DISPATCH_BLOB_SHA pinned the workflow's pre-#1532/#1533 blob SHA; updated to the current `git hash-object` value. - test_opencode_privileged_review_security_boundaries_are_fail_closed asserted the pre-#1533 strict `[ "$SUPPLIED_HEAD_SHA" = "$live_head_sha" ]` equality check. #1533 ("fix(opencode): proceed on head-only advance in review dispatch validation") deliberately removed head_sha from the fail-closed mismatch list -- a head advance between dispatch capture and this job is normal PR activity that every downstream job already re-validates independently (STALE_HEAD guards), so failing closed on it only starved the required review check of a verdict. Updated the assertion to check for the new warn-and-proceed behavior instead of the old fail-closed check it replaced. Co-Authored-By: Claude --- tests/test_opencode_agent_contract.py | 15 ++++++++++++++- .../test_pr_review_autofix_nvidia_nim_contract.py | 2 +- 2 files changed, 15 insertions(+), 2 deletions(-) diff --git a/tests/test_opencode_agent_contract.py b/tests/test_opencode_agent_contract.py index 79fdba39a..ec514386c 100644 --- a/tests/test_opencode_agent_contract.py +++ b/tests/test_opencode_agent_contract.py @@ -2660,7 +2660,20 @@ def test_opencode_privileged_review_security_boundaries_are_fail_closed(): '^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]]' ) in metadata_step assert '[ "$live_head_repository" != "$TARGET_REPOSITORY" ]' not in metadata_step - assert '[ "$SUPPLIED_HEAD_SHA" = "$live_head_sha" ]' in metadata_step + # #1533: a head_sha-only mismatch is no longer a fail-closed trust + # violation -- every downstream job re-fetches and re-checks the live + # head itself (STALE_HEAD guards), so rejecting normal PR activity + # between dispatch capture and this job only starved the required + # review check of a verdict. base_ref/base_sha/head_ref stay strict. + assert ( + '[ "$SUPPLIED_HEAD_SHA" = "$live_head_sha" ] || mismatches+=("head_sha")' + ) not in metadata_step + assert ( + 'if [ -n "$SUPPLIED_HEAD_SHA" ] && ' + '[ "$SUPPLIED_HEAD_SHA" != "$live_head_sha" ]; then' + ) in metadata_step + assert "repository_dispatch head advanced since dispatch" in metadata_step + assert "proceeding with the live head" in metadata_step assert ( 'live_visibility="$(jq -r \'.base.repo.visibility // empty | ascii_downcase\'' ) in metadata_step diff --git a/tests/test_pr_review_autofix_nvidia_nim_contract.py b/tests/test_pr_review_autofix_nvidia_nim_contract.py index 3dcfe2cdd..68a0614c0 100644 --- a/tests/test_pr_review_autofix_nvidia_nim_contract.py +++ b/tests/test_pr_review_autofix_nvidia_nim_contract.py @@ -19,7 +19,7 @@ DOCTORING_RECORD = Path("docs/doctoring/hourly-nvidia-nim-autofix.md") CHANGELOG = Path("CHANGELOG.md") REVIEW_DISPATCH_WORKFLOW = Path(".github/workflows/opencode-review-dispatch.yml") -REVIEW_DISPATCH_BLOB_SHA = "2aa245e7f2a053a4c0b7a9cc8bac0d5d44d38092" +REVIEW_DISPATCH_BLOB_SHA = "3762183eb31c2805317362d2b2c2546e4fccdf09" def _workflow_text(path: Path) -> str: From 7117c55cecec1e736f970021ed83773d02526a4e Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 1 Sep 2026 01:07:52 +0000 Subject: [PATCH 10/11] docs(gaps): correct the stream_options fix attribution after #1463's revert Main moved again mid-rebase: .github#1463 reverted #1448's LLM_DISABLE_STREAMING workaround (which the previous commit on this branch had just cited as the fix) now that the real root cause is fixed upstream -- contextual-orchestrator#925 (merge commit 7944a3c, "accept stream_options.include_usage=true for tools passthrough") makes the gateway itself stop rejecting the combination, and #1463 bumped ORCHESTRATOR_PIN_SHA to 7944a3c so the vendored sidecar actually carries that fix (verified: `git merge-base --is-ancestor 7944a3c ` is true). The conclusion is unchanged (the gap is fixed) but the mechanism is not what the previous commit said, so corrects both the gap-baseline entry and this PR's own CHANGELOG bullet to point at the current, real fix instead of the since-reverted workaround. Co-Authored-By: Claude --- CHANGELOG.md | 10 ++++++--- docs/product-technical-gap-baseline.md | 30 +++++++++++++++++--------- 2 files changed, 27 insertions(+), 13 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index e4d260873..da913f087 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -17,9 +17,13 @@ Semantic Versioning where the repository publishes a release. Agents SDK client sent `stream_options.include_usage=true` with `tools`/`response_format`, rejected by at least one `orchestrator/free` candidate with `HTTP 400 invalid_stream_options`, exhausting the pool — - fixed by `.github#1448` (commit `702392a2`), which scopes Strix's - `LLM_DISABLE_STREAMING` opt-in to the contextual-orchestrator loopback so - that combination is never sent to it. Second, a separate run + fixed at its root cause by `contextual-orchestrator#925` (merge commit + `7944a3c`), which makes the gateway itself stop rejecting the + combination; `.github#1448`'s earlier client-side `LLM_DISABLE_STREAMING` + workaround for the same symptom was since reverted by `.github#1463` now + that it is unnecessary (and `ORCHESTRATOR_PIN_SHA` bumped to `7944a3c` in + the same PR so the vendored sidecar actually carries `#925`'s fix). + Second, a separate run (`ContextualWisdomLab/.github#1441` job `99249903390`) got past that stage entirely and instead hung for the full 120s `curl` timeout with zero bytes back on the sidecar's post-`healthz` gateway smoke request — diff --git a/docs/product-technical-gap-baseline.md b/docs/product-technical-gap-baseline.md index 050926c4c..56df635c0 100644 --- a/docs/product-technical-gap-baseline.md +++ b/docs/product-technical-gap-baseline.md @@ -1417,16 +1417,26 @@ the `ORCHESTRATOR_CATALOG_FAMILY_CAP` 4→8 raise, and (via the independent entry (or PR #1444, which restates it) was originally written, so this is a genuine update, not a contradiction of the historical narrative above. (1) The `stream_options.include_usage=true` + `tools`/`response_format` - rejection is fixed by `.github#1448` (commit `702392a2`, merged - 2026-08-30T12:15:28Z): `strix_quick_gate.sh` now scopes Strix's - `LLM_DISABLE_STREAMING` opt-in to the contextual-orchestrator loopback, so - Strix's always-tools turns against that gateway never send - `stream_options.include_usage=true` in the first place — the exact - combination at least one `orchestrator/free` candidate was rejecting with - `HTTP 400 invalid_stream_options`. `contextual-orchestrator#924` (cited - above as the proposed, not-yet-merged fix) remains open and is superseded - by this Strix-side fix for this specific symptom; other providers in - Strix's fallback chain keep real SSE streaming. (2) The + rejection: `.github#1448` (commit `702392a2`) landed first as a + Strix-side `LLM_DISABLE_STREAMING` workaround, but **that workaround was + itself since reverted by `.github#1463`** once the real, upstream root + cause was fixed at the gateway — `contextual-orchestrator#925` (merge + commit `7944a3c`, "accept stream_options.include_usage=true for tools + passthrough") makes the server itself stop rejecting the combination, so + routing around it client-side became unnecessary. `.github#1463` bumped + `ORCHESTRATOR_PIN_SHA` to `7944a3c` itself (not a later tip, kept + minimal) after Devin Review on that revert caught that the *previously* + vendored pin predated `#925`'s merge and would have silently + reintroduced the original failure had the workaround been removed + first — see this file's own 2026-08-31 `ORCHESTRATOR_PIN_SHA bumped to + carry #925's stream_options/tools fix` entry for the full trail, + confirmed directly via `git merge-base --is-ancestor 7944a3c + 8cd99f13...` (true) against the pin now in place. Net effect is the same + conclusion as originally written here — this gap is fixed — via a + cleaner, upstream mechanism instead of a client-side route-around; the + once-proposed `contextual-orchestrator#924` fix (cited above) is + superseded by `#925`, a different, more direct fix for the identical + symptom. (2) The 120s-timeout-with-zero-bytes gap is fixed by `.github#1452` (commit `1ff82682`, merged 2026-08-30T14:54:45Z, implementing ADR-0005): the sidecar's own code comments now cite a live reproduction of this exact From 28c85102a95a4ecf5430fca7a7d103d9f60d191f Mon Sep 17 00:00:00 2001 From: Claude Date: Tue, 1 Sep 2026 01:54:11 +0000 Subject: [PATCH 11/11] test(ci): revert the head-advance test changes now that main reverted #1533 Main moved again mid-rebase: .github#1540 reverted #1533's warn-and-proceed head_sha check entirely (no rationale recorded beyond the revert itself), restoring the original strict fail-closed equality and, with it, the workflow file's original blob SHA (confirmed: git hash-object on origin/main's copy is exactly 2aa245e7f2a053a4c0b7a9cc8bac0d5d44d38092, byte-identical to what this file's REVIEW_DISPATCH_BLOB_SHA pinned before this whole detour started). Reverts this branch's own earlier changes to these same two spots: REVIEW_DISPATCH_BLOB_SHA back to the original pin, and test_opencode_privileged_review_security_boundaries_are_fail_closed back to asserting the strict equality check instead of the now-reverted warn-and-proceed behavior. Co-Authored-By: Claude --- tests/test_opencode_agent_contract.py | 19 +++++-------------- ...t_pr_review_autofix_nvidia_nim_contract.py | 2 +- 2 files changed, 6 insertions(+), 15 deletions(-) diff --git a/tests/test_opencode_agent_contract.py b/tests/test_opencode_agent_contract.py index ec514386c..746b750c1 100644 --- a/tests/test_opencode_agent_contract.py +++ b/tests/test_opencode_agent_contract.py @@ -2660,20 +2660,11 @@ def test_opencode_privileged_review_security_boundaries_are_fail_closed(): '^[A-Za-z0-9_.-]+/[A-Za-z0-9_.-]+$ ]]' ) in metadata_step assert '[ "$live_head_repository" != "$TARGET_REPOSITORY" ]' not in metadata_step - # #1533: a head_sha-only mismatch is no longer a fail-closed trust - # violation -- every downstream job re-fetches and re-checks the live - # head itself (STALE_HEAD guards), so rejecting normal PR activity - # between dispatch capture and this job only starved the required - # review check of a verdict. base_ref/base_sha/head_ref stay strict. - assert ( - '[ "$SUPPLIED_HEAD_SHA" = "$live_head_sha" ] || mismatches+=("head_sha")' - ) not in metadata_step - assert ( - 'if [ -n "$SUPPLIED_HEAD_SHA" ] && ' - '[ "$SUPPLIED_HEAD_SHA" != "$live_head_sha" ]; then' - ) in metadata_step - assert "repository_dispatch head advanced since dispatch" in metadata_step - assert "proceeding with the live head" in metadata_step + # #1533 briefly relaxed this to a warn-and-proceed check, but #1540 + # reverted it back to the original strict fail-closed equality (no + # rationale recorded beyond the revert itself) -- confirmed against + # main's actual current content, not assumed from the PR history. + assert '[ "$SUPPLIED_HEAD_SHA" = "$live_head_sha" ]' in metadata_step assert ( 'live_visibility="$(jq -r \'.base.repo.visibility // empty | ascii_downcase\'' ) in metadata_step diff --git a/tests/test_pr_review_autofix_nvidia_nim_contract.py b/tests/test_pr_review_autofix_nvidia_nim_contract.py index 68a0614c0..3dcfe2cdd 100644 --- a/tests/test_pr_review_autofix_nvidia_nim_contract.py +++ b/tests/test_pr_review_autofix_nvidia_nim_contract.py @@ -19,7 +19,7 @@ DOCTORING_RECORD = Path("docs/doctoring/hourly-nvidia-nim-autofix.md") CHANGELOG = Path("CHANGELOG.md") REVIEW_DISPATCH_WORKFLOW = Path(".github/workflows/opencode-review-dispatch.yml") -REVIEW_DISPATCH_BLOB_SHA = "3762183eb31c2805317362d2b2c2546e4fccdf09" +REVIEW_DISPATCH_BLOB_SHA = "2aa245e7f2a053a4c0b7a9cc8bac0d5d44d38092" def _workflow_text(path: Path) -> str: