docs: confirm the family_cap/max_tokens fixes work end-to-end (one further gap remains open) - #1444
Open
seonghobae wants to merge 8 commits into
Open
docs: confirm the family_cap/max_tokens fixes work end-to-end (one further gap remains open)#1444seonghobae wants to merge 8 commits into
seonghobae wants to merge 8 commits into
Conversation
…ed run Item 1 of the coordinator's post-merge priority list: verify end-to-end that noema-review/opencode-review/strix actually recover now that main has both #1436's max_tokens fix and this investigation's family_cap raise, not just structurally argued. Checked ContextualWisdomLab/contextual-orchestrator#921 (deliberately unrelated to this investigation, so it does not itself hit the pull_request_target self-test trust boundary #1434/#1441 do) on a fresh head pushed after main had all three fixes: noema-review and strix both succeeded. noema-review's job log confirms a real sidecar cycle (CONTEXTUAL_ORCHESTRATOR_BASE_URL was exported, which only happens after the sidecar's own preflight found a viable route and the server became healthy -- the exact stage that failed 100% of the time before these fixes). opencode-review still failed, but for an already-understood, unrelated, benign timing gap (it ran before the separate async opencode-review-dispatch flow had posted a verdict for that exact head), not a sidecar regression. Co-Authored-By: Claude <noreply@anthropic.com>
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
…ne entries 1. Soften "the sidecar-preflight outage is confirmed resolved" to accurately reflect a second, independently-found, timestamped counter-example: a strix run on .github's own main (job 99247611184, completed 2026-08-30T11:16:17Z -- predating this PR's original claim) failed with a genuinely separate, unfixed defect verified directly from its job log: Strix's OpenAI Agents SDK client sends stream_options.include_usage=true together with tools/response_format, and the routed orchestrator/free candidate rejected that combination with HTTP 400 invalid_stream_options on all retries, exhausting the pool (STRIX_PROVIDER_UNAVAILABLE). The family_cap/max_tokens fixes are confirmed working end-to-end on contextual-orchestrator#921's real run; that specific, previously 100%-reproducible failure mode is closed. It is not true that orchestrator/free is now reliable in general. Root-causing the stream_options gap has been delegated elsewhere; not duplicated here. 2. Update the pingora_edge_policy.py binary-evidence entry from "two competing open fixes, unresolved" to reflect that #1435 already merged -- verified against its actual diff, not assumed from its title: a third, better implementation that network-verifies the real %PDF- magic prefix rather than trusting the .pdf extension alone, falling back to path+suffix trust only for the one case that cannot be verified by content at all (a genuinely oversized file). #1420/#1427 remain open for the separate, non-blocking question of the other binary formats. Co-Authored-By: Claude <noreply@anthropic.com>
…ted gap-baseline entry Companion to the prior commit's gap-baseline correction -- the CHANGELOG bullet had the same overclaim and needed the same softening. Co-Authored-By: Claude <noreply@anthropic.com>
…t timeout) Verified directly from #1441's noema-review job log (99249903390, head d877886): healthz and the launcher's own internal preflight succeeded (past the family-cap fix's stage entirely), but the shell script's separate post-healthz gateway smoke request -- max_tokens now 4096 per #1436's fix, up from 16 -- got zero bytes back for the full 120-second curl timeout. Distinct from both other evidence in this entry: not a bad-request rejection (no response at all) and not Strix's own client code path. Documented with plausible causes (the 120s ceiling never re-examined when max_tokens grew 256x; a real server-side hang for a specific request shape; transient flakiness) but not pursued further this pass -- noted as evidence for whoever picks this up next. Co-Authored-By: Claude <noreply@anthropic.com>
The "one further known gap remains" conclusion undercounted: the same entry already documented two separate, unfixed failure modes (the stream_options 400 and the 120s-timeout gap). Also corrects a second, premature claim that contextual-orchestrator#924 already fixed the stream_options gap -- checked directly: that PR is still open and mergeable_state=blocked, not merged, so it is a proposed fix only. Co-Authored-By: Claude <noreply@anthropic.com>
…ding) Bare #924/#921/.github#1441 refs in the text I just added violate the CLAUDE.md §7 / CWL-MASTER-CONTEXT.md §7 cross-repo reference convention. Also fixes a leftover "below" that should have said "above" for the timeout-gap section, missed in the previous correction pass. Co-Authored-By: Claude <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Item 1 of the post-#1434-merge priority list: verify end-to-end that
noema-review/opencode-review/strixactually recover now thatmainhas both #1436'smax_tokensfix and #1434's family_cap raise — not just structurally argued.Update (softened from this PR's original "confirmed resolved" title/framing): after opening this PR, a second, independently-found, timestamped counter-example surfaced and was verified directly against its own job log — a
strixrun on.github's ownmain(job99247611184, completed2026-08-30T11:16:17Z, predating this PR's original claim) failed with a genuinely separate, unfixed defect: Strix's OpenAI Agents SDK client sendsstream_options.include_usage=truetogether withtools/response_format, and whicheverorchestrator/freecandidate it routed to rejected that exact combination withHTTP 400 invalid_stream_optionson all retries, exhausting the pool (STRIX_PROVIDER_UNAVAILABLE). This is a request-shape incompatibility, not a candidate-selection or token-budget problem — genuinely distinct from what #1434/#1436 fixed. Root-causing it has been delegated elsewhere and is not duplicated in this PR.Accurate combined claim, checked against both pieces of evidence:
ContextualWisdomLab/contextual-orchestrator#921(unrelated to this investigation, picked because it doesn't itself hit thepull_request_targetself-test trust boundary fix(zdr): cite NVIDIA's own Trial ToS for the nvidia_nim not-ZDR classification #1434/fix(strix): add missing github-token fallback for self-target status publish #1441 do):noema-reviewandstrixboth succeeded on a fresh head.noema-review's job log confirms a real, complete sidecar cycle (not vacuous):CONTEXTUAL_ORCHESTRATOR_BASE_URLwas exported, which only happens after the sidecar's own preflight found a viable route and the server became healthy — the exact stage that failed 100% of the time (three independent reproductions) before these fixes landed.orchestrator/freeis not reliable in general yet. Withfamily_cap=8giving more provider diversity per run, which candidate a given run draws varies, and at least one candidate family has the separate, live, unfixedstream_optionsgap above that can still fail Strix (though evidently not every candidate — contextual-orchestrator#921's ownstrixrun succeeded). Two permanently-retiredgoogle/gemma-3-*-itmodel ids are also still admitted into the pool and will still individually fail when the alphabetical sort reaches them.Also includes a second correction (real Devin finding, verified against the actual merged diff): updates the
pingora_edge_policy.pybinary-evidence entry from "two competing open fixes, unresolved" to reflect that #1435 already merged — a third, better implementation that network-verifies the real%PDF-magic prefix rather than trusting the.pdfextension alone.Test plan
Docs-only change (plus CHANGELOG bullets).
tests/test_pr_governance_audit_contract.pypasses.🤖 Generated with Claude Code
https://claude.ai/code/session_015Gs7KmNvH75nxz1sL8mKjw