feat(llm): support ChatGPT/Codex subscription auth for local reviews - #1106
feat(llm): support ChatGPT/Codex subscription auth for local reviews#1106acoliver wants to merge 15 commits into
Conversation
The Codex backend requires stream:true and rejects temperature and max_output_tokens, so the non-streaming Responses path cannot serve it. Accumulate output from response.output_item.done rather than the final response, which Codex sends with an empty output array. Items are sorted by output_index so reasoning precedes the function_call it belongs to, which is required for the alibaba#1070 encrypted-reasoning replay to be valid input. The accumulated items are spliced back into the terminal event envelope and re-unmarshalled so RawJSON is populated and usage extraction takes the same path as every other provider. Three failures that would otherwise read as success are now surfaced: a mid-stream error event (ssestream does not set stream.Err for it), a truncated stream with no terminal event, and non-terminal response status, via a checkResponseStatus shared with the non-streaming path. Codex returns {"detail": ...} with no top-level error key, which the SDK error parser drops, so every gateway rejection appeared as a bare 400. A middleware rewrites it into the shape the SDK reads. Refs alibaba#1105
Review found the refresh path could hang indefinitely. It ran with context.Background() against http.DefaultClient, which has no timeout, and the cross-process lock spun every 50ms with no staleness check. A process killed mid-refresh leaves the lock behind, because SIGINT does not run deferred functions, and every later review then blocked silently before producing any output. The refresh now runs under a bounded context with an HTTP timeout, and a lock older than the staleness window is broken. checkResponseStatus had grown a default arm that errored on any status outside the six named constants, and it ran on the non-streaming path too. Upstream behaviour was to let an empty or vendor status fall through to "stop", so any openai-responses gateway that omits status would have started failing. The shared check is restored to its previous semantics and the strict terminal-status check now lives only in the stream accumulator. NeedsRefresh treated a zero expiry as "refresh now" while RefreshIfNeeded treated it as "already fresh", so the resolver would decide to refresh, call a function that did nothing, and use a stale token. Both now use the single predicate. RequiresStreaming was gating three unrelated behaviours. It is split into RequiresStreaming, RejectsSamplingParams and DetailErrorEnvelope so a future streaming provider does not silently inherit Codex gateway quirks. The credential is no longer loaded when the provider entry overrides url or protocol. Without that guard a redirected endpoint would have received a live ten-day OAuth token that the user never pasted into the config. logout now states the ten-day access-token window on success as well as on failure, since revoking a refresh token does not invalidate an access token already issued. Refs alibaba#1105
…haping A second review pass over this branch found three more issues. A refresh response is not obliged to repeat the id_token, and when it was omitted the stored record was rebuilt from the response alone, blanking the account id and plan derived from it. The stored id_token is now carried forward, mirroring the existing refresh-token handling. The gateway shaping flags were gated on a protocol override but not a url override, while the credential guard checks both. A user with an explicit api_key and a redirected url would therefore have had Codex request shaping applied to a gateway that never asked for it. Both now use one predicate. The callback listener bound the name localhost rather than the loopback address. A modified hosts file or resolver could point that name at a routable address and expose the authorization code beyond the machine. Refs alibaba#1105
What the review tools found in this branchThis branch was reviewed by Open Code Review and by Kodus before submission. Recording Open Code Review, reviewing itselfOCR reviewed this branch using the very feature it adds: First pass, 11 findings. Eight were real and are fixed in
Three did not survive checking:
Second pass over the remediated branch, 9 findings. Three were real, fixed in
The remaining six were repeats of findings already disproved in the first pass, one Worth noting for its own sake: a stateless reviewer re-raises adjudicated findings on a Kodus, reviewing the companion branchKodus reviewed the parallel Kodus change (kodustech/kodus-ai#1806) and produced three Two of them, one root cause, claimed the credential compare-and-swap is key-order SELECT '{"a":1,"b":{"x":1,"y":2}}'::jsonb = '{"b":{"y":2,"x":1},"a":1}'::jsonb;
tThe third asked for path validation on an environment variable that names a credential Reliability noteEvery OCR run in this exercise returned |
Field results: reviewing unrelated pull requests through a subscriptionTo check this beyond self-review, I ran the built binary against two merged pull requests
The findings were substantive rather than stylistic. On #3390 it noted that the new Why
|
|
🔍 OpenCodeReview found 3 issue(s) in this PR.
|
A ChatGPT account is served the models its plan entitles it to, and that set differs between accounts and changes over time. The preset's Models list was being enforced as an allowlist, so a model the signed-in account can actually run was rejected locally before any request was made. The resolver already makes this exact allowance for ambient-auth providers, because Bedrock identifiers are scoped to an account. External-auth providers are the same case, so they now take the same path: the list remains a picker for `ocr config model` and no longer gates an override. Adds gpt-5.6-sol to the picker and replaces the test that asserted it must be rejected, which encoded a measurement taken against an account that was not entitled to it. Refs alibaba#1105
End-to-end verification of the login flowEarlier verification in this PR used a credential copied from an existing
The listener binds The credential lands with the intended permissions: And That Then, on the credential that login produced: Storage follows the existing layout rather than introducing a new one: |
Correction: the "pre-existing failures" note in this PR is wrongThe test plan says six tests fail on this checkout, unrelated to this branch: four They are not failing tests. They fail only when the run inherits a developer I identified the cause correctly and then described the symptom as if the repository Worth noting separately, since it is a genuine (pre-existing) sharp edge rather than a |
A review issues dozens of turns against one growing conversation, so nearly every request repeats the prior turn's context and depends on the provider's prompt cache to avoid paying for it twice. Measured over seven reviews on a ChatGPT subscription, only 36% of input was served from cache and 77% of the billed total was context the loop had already sent on the previous turn. Capturing the wire traffic of this client and of another Codex client that does not show the same loss, through the same recording endpoint, leaves one difference in the request: the other client sends prompt_cache_retention. The API default is "in_memory", which is short lived. The request is otherwise already shaped for caching, which the same capture confirms: prompt_cache_key is stable across a conversation, the tool list and instructions are byte-identical each turn, and the input array is a strict prefix extension. Gated on the preset so a redirected url, which may front a gateway that rejects the field, does not receive it. The cache improvement itself is not yet confirmed against the live endpoint; that needs a subscription with quota available. Refs alibaba#1105
… Codex" This reverts commit c584d2e. The endpoint rejects the parameter outright: prompt_cache_retention=24h -> 400 {"detail":"Unsupported parameter: prompt_cache_retention"} prompt_cache_retention=in_memory -> 400 {"detail":"Unsupported parameter: prompt_cache_retention"} same request without it -> 200 so the change did not improve caching, it broke every Codex request. A review run with it fails with all file reviews failing. The reasoning behind the original commit was wrong in its method. I compared this client's requests against another Codex client's requests, but both captures were taken against a permissive local recorder rather than against the real endpoint. The other client evidently does not send this field to Codex, so the difference I measured was an artefact of the test rig. The measured caching shortfall is real and still unexplained. Refs alibaba#1105
The poll request embeds server-derived values (device_auth_id, user_code). Building it with %q fmt-escapes as Go, which can emit sequences that are not valid JSON; json.Marshal keeps the wire form correct whatever the deviceauth backend returns. The client_id site stays %q — it is an ASCII package constant.
|
Review-thread dispositions (github-actions findings): Fixed — Declined, with reasons:
|
- provider TUI: clear stale api-key state when the selected provider authenticates without a key, so result() no longer reports the previously active provider's saved key as the new provider's key - logout now holds the cross-process refresh lock, so a concurrent refresh cannot save rotated credentials after the teardown's Clear - refresh lock: read the owner PID back after the exclusive create and release only while the lock still names this process, closing the two-waiter stale-lock race that let both co-hold the lock - device poll: retry transport failures on the poll interval instead of discarding a still-valid code, and poll once more after the final capped sleep before declaring expiry - loopback login: always print the authorization URL; a desktop opener can spawn successfully and still fail without reporting an error - OAuth error responses surface the allowlisted error code only, never free-text fields a server could echo back with token material - auth status distinguishes "expired; will refresh on next use" from a terminal expiry when a refresh token exists
|
Final OCR review (10 findings) adjudicated against the code; 8 accepted and fixed in aa1e2d3, 2 declined: Fixed (aa1e2d3):
Declined:
Verified: |
Follow-up review findings on aa1e2d3: - The stale-break os.Remove was unconditional, so a waiter paused between its staleness check and the remove could delete a lock another waiter had already broken and re-acquired, letting both co-hold. The remove is now conditional on the lock content still matching the bytes observed before the liveness probe, and the read-back comment no longer claims the window is fully closed (a single-read gap remains; kernel locks would close it). - runAuthLogout returned before Clear when the refresh lock could not be acquired, leaving tokens on disk because of a lock problem. Lock acquisition failure now falls through to a best-effort Clear with a revocation-skipped caveat; a Clear that failed inside the locked section still surfaces its error instead of being retried.
|
Findings-only follow-up review of aa1e2d3 (2 medium findings, both fixed in 03fd38a):
The follow-up also re-verified the remaining fix areas from aa1e2d3 (device final poll, transport-error retention, error-code allowlist, login URL printing, TUI stale-key clear) and confirmed them correct. |
|
Note this reviewed itself many times with OCR to test the thing :-) |
Summary
Adds local ChatGPT/Codex subscription authentication so
ocr reviewcan run against adeveloper's ChatGPT plan instead of a metered API key. Fixes #1105.
This is local only. The GitHub Action cannot use this credential path.
The endpoint contract was measured directly rather than inferred. #275 concluded a
Codex subscription "exposes no reusable endpoint+token pair to point OCR at"; the
measurements in #1105 show otherwise, and this PR is the implementation.
What was measured
POST https://chatgpt.com/backend-api/codex/responses, with a ChatGPT plan token:streamomitted orfalse400 {"detail":"Stream must be set to true"}store: trueor omitted400 {"detail":"Store must be set to false"}temperature400 {"detail":"Unsupported parameter: temperature"}max_output_tokens400 {"detail":"Unsupported parameter: max_output_tokens"}reasoning: {effort:"high"}200include: ["reasoning.encrypted_content"]200text.formatjson_schema200200, emitsfunction_callitemschatgpt-account-id/OpenAI-Beta/originator/User-Agent200for eachauth.openai.com/oauth/token200,expires_in=864000, refresh token rotatesModels are account-scoped. Which models a ChatGPT account can run depends on its plan
and changes over time, so the endpoint's refusal of a given model is a statement about
the account rather than about the API.
gpt-5.6-sol,gpt-5.6-lunaandgpt-5.6-terraall return
200on a Pro account.gpt-5.6-codex,gpt-5.1-codexandcodex-mini-latestare refused on the two accounts available to me, which I would notgeneralise further.
Because that set cannot be known upstream, the preset's model list is a picker for
ocr config modeland does not gate--model. This follows the resolver's existingtreatment of ambient-auth providers, where Bedrock identifiers are account-scoped for
the same reason.
Two results worth flagging because they contradict reasonable assumptions: every custom
header is optional, so the reserved-header list is not an obstacle; and the access token
lives ten days rather than an hour, so refresh is a robustness concern rather than a
correctness one.
Approach
Streaming Responses path. The existing client is non-streaming by construction and
deliberately drops
streamfromextra_body. Codex requires it, so this adds astreaming path gated on the provider, leaving every other
openai-responsesproviderbyte-for-byte unchanged.
Output is accumulated from
response.output_item.done, not from the final response.Codex sends
response.completedwithstatusandusagebut an emptyoutputarray, so reading it would produce an empty review. Items are sorted by
output_indexbecause reasoning must precede the
function_callit belongs to for #1070's replay to bevalid input. The accumulated items are spliced back into the terminal envelope and
re-unmarshalled so
RawJSON()is populated and usage extraction takes the same path asevery other provider rather than silently falling back.
Three failure modes that would otherwise read as success are surfaced: a mid-stream
{"type":"error"}event (ssestreamdoes not setstream.Err()for it, since it looksonly for a top-level
errorkey), a truncated stream with no terminal event (which wouldotherwise map to
finish_reason: "stop"), and a non-terminal response status.Error legibility. Codex emits two error shapes:
{"detail":...}for gatewayrejections and
{"error":{...}}for upstream errors. The SDK reads only the latter, soevery rejection in the table above surfaced as a bare
400 Bad Request. A middlewarerewrites the former into the shape the SDK reads.
Auth.
ocr auth login(PKCE + loopback),--device(device code, for headless orremote shells),
--no-browser(paste the URL), plusstatusandlogout. Constantsmatch the official Codex CLI. The loopback listener reuses the Host-header guard from
internal/viewer, since a callback carrying an authorization code is a worseDNS-rebinding target than the viewer.
Storage.
~/.opencodereview/auth/codex.json, mode0600in a0700directory. Theauth/subdirectory is deliberate:saveConfigcreates the parent at0755andos.MkdirAlldoes not chmod an existing directory, so a0700claim on the parent wouldsilently not hold. Writes are atomic (temp file, chmod, rename) because refresh rotates
the refresh token and a torn write would strand the user with neither credential. There
is no atomic-write precedent in the tree, so this is new code with its own tests.
Storage sits behind a small interface so an OS keyring backend is additive later, which
is what #236 asked for and what the official Codex CLI already does.
Not in scope
No new protocol constant:
openai-responsesis the correct wire protocol and only thetransport and credential source differ. No
extra_headersrelaxation, since every Codexheader turned out optional. No keyring in this PR.
Keeping it out of CI
The preset carries no
EnvVar, soaction.yml(whose entire credential surface isOCR_LLM_*) cannot supply this credential, andocr authis the only writer of thetoken file. Stated honestly, that is a convention rather than a structural guarantee:
nothing stops a workflow restoring the token file from
actions/cache.Test plan
go build ./...,go vet ./...,gofmt -s -lcleanmake check(license headers, english-only, LF,go mod tidy)internal/llmandinternal/codexauthsuites pass[]ResponseStreamEventUnion, covering allthree silent-failure modes and
output_indexorderingencrypted_contentsurvives into theNativeTurnreplaypayload, and that usage takes the raw-probe path (
CacheWriteTokens), which failsif the splice degrades to the struct fallback
TemperatureandMaxOutputTokensasserted absent for codex, present for a normalopenai-responsesprovider0700/0600, chmod-before-rename, original survives a failed rename,with the Windows guard
TestLookupProvider_CodexDetailsper.opencodereview/rule.json## ocr authsection added in all five localesocr auth logincompletes a real PKCE sign-in(
Signed in to ChatGPT account 79f9***bfe1), writes0600inside a0700directory, and
ocr auth statusthen reports the plan parsed from the id_tokenand exits 0
ocr llm testsucceeds ongpt-5.6-solagainstchatgpt.com/backend-api/codexusing the credential that login produced, and a full
ocr reviewcompletes on itThe full suite passes with a clean
HOME(HOME=$(mktemp -d) go test ./..., exit 0,24 packages, no failures). Six tests in
cmd/opencodereviewandinternal/config/rulesread the caller's real
~/.opencodereview/when one exists, so a contributor with anexisting OCR configuration will see them fail locally while CI is green. That is
pre-existing and not touched here.
Review notes
This branch was reviewed by both Open Code Review and Kodus before submission; findings
from each are summarised in a comment below.
Happy to split this if you would prefer: the streaming Responses path is independently
useful and could land ahead of the auth work.