Incident
Since Strix moved to contextual-orchestrator's orchestrator/free pool today (#1434), every Strix scan org-wide has been failing closed with openai.BadRequestError: 400 invalid_stream_options — stream_options.include_usage=true is not supported with tools or response_format. Example: https://github.com/ContextualWisdomLab/.github/actions/runs/33307905354/job/99247611184
Root cause (confirmed by reading actual source, not guessing):
- strix-agent 1.5.3 hardcodes
ModelSettings(include_usage=True) for every turn (strix/core/inputs.py::make_model_settings()), with no supported env var, CLI flag, or config key to disable just that flag while keeping streaming. Verified against strix v1.5.3 and openai-agents v0.19.4 source directly.
contextual_orchestrator/server.py (around line 6583-6589) unconditionally rejects stream=true + stream_options.include_usage=true whenever tools or response_format are present — a check introduced in commit e7618a3 alongside an unrelated fix, justified only by a test docstring ("Structured passthrough cannot emit usage SSE") that is factually wrong: _chat_response_sse_chunks already builds a genuine usage_source: "reported" SSE chunk from the provider's real usage field, and the tool-loop path (proxy_completion, orchestrator.py) already forces the upstream call to non-streaming and already captures real per-call usage — the honest-usage plumbing this check thinks doesn't exist already exists and already runs on this exact path.
Mitigation shipped now
This PR/commit sets LLM_DISABLE_STREAMING: "true" in the Strix job env in .github/workflows/strix.yml. This is a genuine, documented strix env var (strix/config/models.py::_NonStreamingModel) that makes every turn a single non-streaming call, so stream_options is never sent and the gateway's check never fires. This restores the required Strix gate org-wide immediately, at the cost of Strix losing real-time SSE streaming (all turns become synchronous).
What must happen before this is reverted
- A PR on
ContextualWisdomLab/contextual-orchestrator that removes or narrows the stale guard at server.py (~line 6583-6589), per evidence that both the tool-loop and non-tool-loop branches already source real, honest usage from the provider response (never fabricated) — no orchestrator.py change is required, the plumbing already works.
- That PR flips the two tests currently pinning the rejection (
tests/test_chat_tools_passthrough_controls_http_honesty.py::test_http_structured_stream_usage_fails_closed_before_execution, tests/test_stream_options_null_flags_noop_http_honesty.py::test_http_chat_structured_streams_include_usage) to assert a real usage_source: "reported" SSE chunk, adds a positive honesty-invariant test, and corrects the now-stale CHANGELOG.md entry describing the old rejection as intentional.
- Once that PR is merged and deployed, verify with a manual Strix
workflow_dispatch run (or a direct request reproducing Strix's exact request shape) showing 200 + usage_source: "reported", then remove LLM_DISABLE_STREAMING: "true" from strix.yml here so Strix regains real streaming.
Do not let this mitigation quietly become permanent — it trades away working infrastructure for no architectural reason once the real fix lands.
Incident
Since Strix moved to
contextual-orchestrator'sorchestrator/freepool today (#1434), every Strix scan org-wide has been failing closed withopenai.BadRequestError: 400 invalid_stream_options — stream_options.include_usage=true is not supported with tools or response_format. Example: https://github.com/ContextualWisdomLab/.github/actions/runs/33307905354/job/99247611184Root cause (confirmed by reading actual source, not guessing):
ModelSettings(include_usage=True)for every turn (strix/core/inputs.py::make_model_settings()), with no supported env var, CLI flag, or config key to disable just that flag while keeping streaming. Verified against strix v1.5.3 and openai-agents v0.19.4 source directly.contextual_orchestrator/server.py(around line 6583-6589) unconditionally rejectsstream=true+stream_options.include_usage=truewhenevertoolsorresponse_formatare present — a check introduced in commite7618a3alongside an unrelated fix, justified only by a test docstring ("Structured passthrough cannot emit usage SSE") that is factually wrong:_chat_response_sse_chunksalready builds a genuineusage_source: "reported"SSE chunk from the provider's realusagefield, and the tool-loop path (proxy_completion,orchestrator.py) already forces the upstream call to non-streaming and already captures real per-call usage — the honest-usage plumbing this check thinks doesn't exist already exists and already runs on this exact path.Mitigation shipped now
This PR/commit sets
LLM_DISABLE_STREAMING: "true"in the Strix job env in.github/workflows/strix.yml. This is a genuine, documented strix env var (strix/config/models.py::_NonStreamingModel) that makes every turn a single non-streaming call, sostream_optionsis never sent and the gateway's check never fires. This restores the required Strix gate org-wide immediately, at the cost of Strix losing real-time SSE streaming (all turns become synchronous).What must happen before this is reverted
ContextualWisdomLab/contextual-orchestratorthat removes or narrows the stale guard atserver.py(~line 6583-6589), per evidence that both the tool-loop and non-tool-loop branches already source real, honest usage from the provider response (never fabricated) — no orchestrator.py change is required, the plumbing already works.tests/test_chat_tools_passthrough_controls_http_honesty.py::test_http_structured_stream_usage_fails_closed_before_execution,tests/test_stream_options_null_flags_noop_http_honesty.py::test_http_chat_structured_streams_include_usage) to assert a realusage_source: "reported"SSE chunk, adds a positive honesty-invariant test, and corrects the now-staleCHANGELOG.mdentry describing the old rejection as intentional.workflow_dispatchrun (or a direct request reproducing Strix's exact request shape) showing 200 +usage_source: "reported", then removeLLM_DISABLE_STREAMING: "true"fromstrix.ymlhere so Strix regains real streaming.Do not let this mitigation quietly become permanent — it trades away working infrastructure for no architectural reason once the real fix lands.