Skip to content

feat(codex): ChatGPT Codex accounts as a backend protocol - #154

Open
DIodide wants to merge 23 commits into
KarpelesLab:masterfrom
DIodide:codex-protocol
Open

feat(codex): ChatGPT Codex accounts as a backend protocol#154
DIodide wants to merge 23 commits into
KarpelesLab:masterfrom
DIodide:codex-protocol

Conversation

@DIodide

@DIodide DIodide commented Aug 3, 2026

Copy link
Copy Markdown

Adds ChatGPT Codex as a backend protocol, so a ChatGPT subscription can serve requests alongside Claude accounts.

This is deliberately different from the existing third-party backend support. That path works because DeepSeek and GLM publish Anthropic-compatible endpoints, so upstream + modelMap is enough. Codex speaks the OpenAI Responses API — different request shape, different SSE event model, different auth — so it needs real translation in both directions.

I've kept everything outside the forwarding path protocol-agnostic. Selection, rotation, priority, routes, disable/enable, TC_ACCT pinning, quota thresholds, status rendering and state persistence are untouched; a codex account is a peer in the same fleet, not a parallel code path. The one place that needed protocol awareness outside the wire layer was accounts, which was fetching an Anthropic profile for every OAuth account.

What's here

  • login --codex — its own PKCE authorization against auth.openai.com, so TeamClaude's credential is independent of the Codex CLI's. import --codex also works but copies ~/.codex/auth.json, which means both hold one refresh token; OpenAI rotates it on every refresh, so whichever refreshes second is left with a dead one. The import path says so and points at login.
  • Request translation — Anthropic Messages → Responses API: system prompt to a developer message, content blocks to input items, tool schemas to function declarations, thinking budget to reasoning effort.
  • Response translation — a stateful Codex SSE → Anthropic SSE translator. The two event models don't line up: Anthropic needs content blocks opened and closed around a monotonic index, while Codex interleaves reasoning, text and tool calls freely.
  • Prompt caching — the cache is scoped by a key derived from session, sub-agent and model, set both as prompt_cache_key and on the Session_id header, mirroring CLIProxyAPI's ClaudeCodePromptCache. Without it the backend falls back to echoing Session_id, which quietly meant zero caching for any client that sends no session header. Measured on a 3k-token prompt: a stable key caches 2816 of 3016 input tokens from the second turn on, a per-request key caches nothing. cache_write_tokens is also mapped onto cache_creation_input_tokens, which was being dropped.
  • Quota — read from x-codex-* response headers and normalised onto the existing unified5h/unified7d fields, so rotation is predictive rather than waiting for a 429.
  • Token refresh — codex accounts are excluded from the prober (it reads an Anthropic endpoint that rejects a ChatGPT token) and from keep-warm (their plan has no 5h session to hold open), so they get a small scheduled refresher instead. It checks often and refreshes only inside a 30-minute window, which works out to roughly one refresh per token.
  • Model discovery/v1/models answers with the Codex catalog, so Claude Code lists GPT models in its own picker when CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY is set. Claude Code drops any listed id not starting with claude-, so ids are encoded into claude-prefixed ones and decoded before anything reads the model — the same encoding CLIProxyAPI uses (EnsureClaudeModelIDPrefix), so a setup written for one behaves the same on the other. The listing carries only what TeamClaude adds, since echoing Anthropic's catalog back duplicates every Claude entry in the picker; modelDiscovery.includeAnthropic restores the full list. blockedModels now filters the listing too — advertising a model the proxy would reject with a 400 puts a choice in the picker that fails the moment it is used. Selecting one just works: selection is protocol-aware, so a Codex model routes to a codex account and a Claude model never does. That also fixes the reverse case — a codex account with no modelMap used to be picked for claude-* requests and 400 on every one. modelMap is now only needed to make a codex account answer to Claude model names as a quota fallback. Only intercepted when a codex account exists, so an Anthropic-only fleet is untouched.
  • Observability — the serving account and resolved model are recorded per session and exposed at GET /teamclaude/status under sessions.byId, and the TUI activity line shows (claude-opus-5 → gpt-5.6-sol). This one is useful beyond codex: a DeepSeek or GLM account with a modelMap has the same blind spot today, and it's easy to split out if you'd rather take it separately.

How it was tested

The translators were ported from CLIProxyAPI's reference implementation and differential-tested against it: a small Go harness calls its translators directly, and the JS port is diffed against them. 38 request cases and 30 stream cases match byte-for-byte. Those outputs are committed as golden fixtures, so the suite asserts against ground truth without needing Go or a CLIProxyAPI checkout.

Both translators matched on the first run, which I didn't trust, so I mutation-tested the corpus — 12 plausible porting mistakes, of which the first pass killed only 9. The three survivors were real gaps (stale terminal arguments overwriting streamed deltas, missing deferral of items behind an active tool call, and the stop-reason path where Codex names a tool finish that produced no tool block). Cases were added to close them; it now kills all 12.

Fixtures only get you so far, though, and running a real Claude Code session against the live backend found four things they structurally could not:

  • Codex sends its SSE responses with no content-type header, so deciding "is this a stream?" from that header meant every response was buffered and returned as raw Responses-API events. My e2e mock had been setting the header — more polite than the real server, which is exactly how a mock hides a bug.
  • An error response is JSON, not SSE, so forcing the streaming path on non-200 fed the error body to the translator, which emitted nothing. The client saw 400 (no body) and the message explaining the failure was destroyed.
  • Claude Code calls Anthropic-only endpoints (bootstrap, oauth/usage, mcp_servers) alongside /v1/messages; refusing them broke those features whenever rotation landed on codex. They now fail over.
  • A thinking budget of ≤512 became effort minimal, which the backend rejects outright. The fixtures couldn't catch it because CLIProxyAPI clamps in a later pipeline stage they don't cover — a blind spot in testing function-against-function rather than pipeline-against-pipeline.

Verified live end to end: streaming, non-streaming, tool calls, a tool_result round trip, modelMap selection, and quota going from unknown to a recorded weekly bucket after one request.

Probing the backend for the model and reasoning-effort matrix also settled a disagreement between the only two sources available. The API's enum message advertises minimal, but sending it draws a model-specific rejection ('minimal' is not supported with the 'gpt-5.6-sol' model), so the documented enum is wider than any model accepts. Conversely the Codex CLI's model cache lists ultra as a supported level for sol and terra, but the API rejects it — that cache describes the CLI's menu, not the wire contract. The accepted set is none, low, medium, high, xhigh, max, which is what the clamp uses; ultra now lands on max rather than falling to the default.

All seven models the live catalog exposes are verified end to end through the proxy — each returning its own id in message_start, with no modelMap configured:

Model Description
gpt-5.6-sol Latest frontier agentic coding model
gpt-5.6-terra Balanced agentic coding model for everyday work
gpt-5.6-luna Fast and affordable agentic coding model
gpt-5.5, gpt-5.4, gpt-5.4-mini Previous generations
codex-auto-review Review-oriented alias

npm test is 581/582 and npm run lint is clean. The one failure is test/server-listen-error.test.js, which fails identically on an untouched upstream/master worktree here — environmental, not from this branch.

Known limits

  • Reasoning summaries aren't requested (matching CLIProxyAPI), so responses carry no thinking blocks. The thinking-block and signature-replay paths are fixture-verified but never exercised live, because the backend emits no reasoning events without an explicit summary opt-in.
  • Only /v1/messages is translated; everything else needs a Claude account in the fleet.
  • Auth impersonates the Codex CLI's client id and user agent, which is what makes a ChatGPT subscription usable here at all. Worth a deliberate decision on your side, and I'd understand a "no" on that basis alone.

Happy to restructure, split the observability commit out, or drop pieces — and equally happy to hear this is out of scope for the project, in which case it can live as a fork.

DIodide added 23 commits August 1, 2026 10:44
Codex credentials are ChatGPT OAuth tokens, not platform API keys: they
refresh against auth.openai.com with the Codex CLI's public client id and a
form-encoded body, where the Anthropic path uses platform.claude.com and JSON.

The refresher deliberately mirrors refreshAccessToken's retry and error
contract (err.status set on HTTP failures) because ensureTokenFresh keys off
that status to tell a dead refresh token from a transient blip. A codex
refresh that threw a bare Error would look transient forever and never
sideline a genuinely broken account.

The ChatGPT account id lives in a namespaced id_token claim rather than a
plain field, and is required on every upstream call, so it is derived on both
import and refresh.
Accounts gain `protocol` ('anthropic' by default) and `accountId`.
ensureTokenFresh dispatches on protocol so codex accounts refresh against
OpenAI while everything else keeps its existing path.

The point of putting protocol on the account record rather than branching
higher up is that selection, rotation, priority, routes, disable/enable and
TC_ACCT pinning stay protocol-agnostic — codex accounts are peers in the same
fleet, not a separate pool.

A codex refresh re-derives the account id from the fresh id_token so a
server-side account migration is picked up, guarded so a partial response
cannot erase a working id.
Codex accounts target ${base}/responses rather than the client's /v1/messages
path, and send headers built from scratch: the inbound set is Anthropic-
specific, and the backend expects the Codex CLI's identity. Endpoints with no
Responses-API equivalent (count_tokens and friends) return 404 rather than
being forwarded somewhere that would answer them wrongly.

The body translators are declared and tested-as-unimplemented rather than
faked, so the seam is real but cannot be mistaken for working code. They land
next.

isStreamingRequest full-parses instead of reusing TopLevelFieldFinder: that
machine only captures string values, so it reports null for `"stream": true`.
Imports ~/.codex/auth.json as a codex-protocol account, identified by ChatGPT
account id (there is no org dimension, and the email can change without the
account changing).

Two protocol-awareness fixes on the management surface: `accounts` skipped
the Anthropic profile fetch for codex accounts, which rejects a ChatGPT token
and rendered every codex account as a 401; and its inline refresh now
dispatches on protocol like ensureTokenFresh does.

Import warns that OpenAI rotates the refresh token on every refresh, so the
imported credential and the Codex CLI's copy are two holders of one rotating
family. Stating it at import time beats the failure surfacing much later as a
confusing re-login prompt.
Ported from CLIProxyAPI's reference implementation and verified byte-equal
against it across 38 cases (see test/fixtures/codex-requests.json): system
prompt to developer message, content blocks to Responses input items, tool
schemas to function declarations, thinking budget to reasoning effort.

Two things that are load-bearing and non-obvious:

Tool names and call ids are capped at 64 chars upstream. Claude Code's MCP
names routinely exceed that, so names are shortened with collision-avoiding
uniquification (two MCP tools differing only past the cut would otherwise
collapse onto one name and become unaddressable), and long call ids get a hash
suffix so tool results still pair with their calls.

Reasoning signatures are validated structurally before replay. Only reasoning
this backend issued can be replayed to it; a conversation that previously ran
on a Claude account carries Anthropic signatures, and forwarding one makes the
backend reject the entire request. Dropping the block degrades gracefully
instead.
The stateful half. The two event models do not line up: Anthropic requires
content blocks opened and closed around a monotonically increasing index,
while Codex emits output items that interleave reasoning, text and tool calls
freely. This owns that bookkeeping across the whole stream.

Verified frame-for-frame against CLIProxyAPI across 30 sequences covering
parallel tool calls, interleaved reasoning, deferred events, truncated
streams and every stop-reason path (test/fixtures/codex-streams.json).

Notable behaviours the fixtures pin down:
- Only one tool_use block may be open at a time, so calls are queued and
  anything that would open a competing block is deferred and replayed after.
- A multi-part reasoning summary stays one thinking block, because only
  output_item.done carries the item's final signature.
- Codex counts cached tokens inside input_tokens while Anthropic reports them
  alongside, so the cached count is subtracted to avoid double-counting.
- aggregateAnthropicStream folds the frames back into a Messages object for
  clients that asked for a non-streaming reply.
Replaces the milestone-1 stubs with the real path: translate the request body
after modelMap has chosen the upstream model, then stream-translate the
response. Usage accounting reads the translated frames, so quota tracking and
rotation work for codex accounts without a second code path.

Codex is always asked for a stream because the Responses API is
streaming-first here, so a client that wanted a plain response gets the frames
folded back into a Messages object and the content-type corrected before
headers go out.

A stream that ends without a terminal event now gets its open block closed
rather than leaving the client waiting.
…ked corpus

The fixtures are golden output captured from CLIProxyAPI via a Go harness that
calls its translators directly, so the suite asserts against ground truth
without needing Go or a CLIProxyAPI checkout. A failure means the port drifted
from what a real backend expects; regenerate only against a known-good
revision, never to make a failing test pass.

The corpus was validated by mutation testing rather than assumed sufficient:
12 plausible porting mistakes were injected and the first run killed only 9.
The three survivors — stale terminal arguments overwriting streamed deltas,
missing deferral of items behind an active tool call, and the stop-reason path
where Codex names a tool finish that produced no tool block — were each real
gaps, and the cases added to close them bring it to 12/12.

codex-e2e.test.js drives the actual proxy against a stand-in Codex backend to
cover the wiring the fixtures cannot: URL rewriting, Codex headers, modelMap
selection, usage accounting, endpoint refusal, and truncated streams.
The live Codex backend sends its SSE responses with no content-type header at
all. The streaming branch keyed off that header, so against the real backend
every codex response fell through to the buffered path and the client received
raw Responses-API events instead of translated Anthropic ones. Both the
streaming and non-streaming paths were affected.

A codex request is always sent with stream:true and accept: text/event-stream,
so its response is always SSE and no header sniffing is needed. The proxy now
also sets the content-type it is actually emitting, since upstream supplies
none: text/event-stream when the client asked to stream, application/json when
the frames are folded into a Messages object.

The e2e mock set content-type: text/event-stream and so passed throughout,
which is exactly how it hid this. It now omits the header like the real
backend, with regression cases for both the absent and present forms.

Found by running the proxy against the real backend; verified fixed there —
streaming, non-streaming, tool calls and a tool_result round trip all return
correct Anthropic events.
Makes rotation predictive for codex accounts instead of reactive. Without
this the proxy only learns an account is spent by getting a 429, so the first
request after exhaustion always fails before failing over — the same problem
anthropic-ratelimit-* solves for Claude accounts.

Quota is normalized onto the existing unified5h/unified7d fields rather than
getting codex-specific ones, so selection, the switch threshold, status
rendering and state persistence stay protocol-agnostic and codex accounts
remain peers in one fleet.

Buckets are classified by window DURATION, not by the primary/secondary
naming. The observed `plus` account reports its 7-day limit in the primary
slot and leaves secondary inactive; keying off the slot name would silently
mis-file the bucket on a plan that orders them the other way. A zero-length
window is skipped rather than recorded as 0% used, which would otherwise look
like abundant headroom on a limit that does not exist.

Three consequences elsewhere:
- A codex 429 is judged exhausted from utilization, since codex sends no
  per-bucket status header. Otherwise a spent weekly bucket would read as a
  transient throttle and the proxy would retry the same account for a week.
- The prober skips codex accounts: it reads Anthropic's usage endpoint, which
  rejects a ChatGPT token. They need no probe — quota arrives on every
  response.
- The warmer skips them too: it exists to hold a 5-hour session timer open,
  and the observed plan has only a rolling weekly window, so warming would
  spend real quota to accomplish nothing.

Header fixtures are captured verbatim from a live response. Verified against
the real backend: quota goes from unknown to 0% used with a reset 6.98 days
out after a single request.
Codex accounts had no scheduled refresh at all. Every other account type gets
one as a side effect of something else — a request refreshes on the way
through, and the prober and keep-warm scheduler both call ensureTokenFresh —
but codex accounts are excluded from both (the prober reads an Anthropic
endpoint that rejects a ChatGPT token; warming holds open a session window
this plan does not have). On an idle proxy nothing renewed them.

That was survivable, since the request path refreshes synchronously before
sending, but it left the proxy holding a stale credential for as long as it
stayed idle. OpenAI rotates the refresh token on every refresh, so the longer
teamclaude sits on an old one the likelier another holder rotates the family
out from under it — and then the refresh fails and the account drops out of
rotation until re-imported.

Scoped to codex accounts on purpose. Anthropic accounts already have three
refresh drivers, and this codebase deliberately avoids being an extra holder
rotating that family; adding unsolicited background refresh for them would
cause the very problem ensureTokenFresh's comments guard against.

On by default, unlike the prober and warmer, because it spends no quota: the
check is a clock comparison and a refresh only fires inside a 30-minute
lookahead. Against the observed ~10-day token lifetime that is roughly one
refresh per token, so it does not churn the refresh-token family any harder
than the lazy path already did. start() also checks immediately, so a proxy
started after days idle does not serve its first request on a dead token.

Verified live: a token forced to 578s from expiry was refreshed at startup and
the rotated pair — critically including the new refresh token — was persisted
to disk, so a restart picks up the live credential rather than a dead one.
Driving the codex path by hand means remembering to isolate the config, pick a
free port, cover the [1m] model-id spelling and find the log directory. This
wraps that up so the fork can be exercised without touching an installed
teamclaude or the real ~/.config/teamclaude.json.

The modelMap it writes covers both the bare and [1m]-suffixed spellings of each
model id, because rewriteModel matches the model string exactly and Claude Code
may send either. The `model` subcommand prints the ids actually observed in the
request logs, so an unmapped one is easy to spot rather than surfacing as a
confusing upstream rejection.
`import --codex` copies the Codex CLI's credential, so teamclaude and the CLI
end up holding one refresh token. OpenAI rotates it on every refresh, which
means whichever party refreshes second is left with a dead one — the conflict
the import path has been warning about.

That conflict is an artefact of copying a credential, not a property of the
provider. A separate authorization mints its own refresh-token lineage, so a
login gives teamclaude a grant that rotates independently of the CLI's and
neither invalidates the other.

The parameters that make the grant independent are `prompt=login` (forces a
fresh authorization rather than reusing the existing session) and
`offline_access` (without which no refresh token is issued at all); both are
pinned by tests. The redirect URI is registered against the Codex CLI's client
id, so the callback listener must bind port 1455 exactly rather than taking an
ephemeral one — a busy port now reports that plainly instead of EADDRINUSE.

Import stays supported for reusing an existing credential, and now points at
login as the way to avoid the sharing problem rather than only describing it.
The codex upsert fell back to matching an existing account BY NAME when no
codex account shared the new account id. That fallback was copied from the
Anthropic upsert, where every account speaks one protocol and matching by name
is safe. Here it meant a codex credential whose email equalled a Claude
account's name merged codex fields over that account and destroyed its
credential.

Not an edge case: one person's Claude and ChatGPT accounts routinely share an
email, so `login --codex` on a config holding the matching Claude account
silently replaced it. Observed on a real config, which is how it was found.

The name fallback is now scoped to codex accounts, and a default name that
collides with an account of another protocol is disambiguated instead of
reused — names are the user-facing key for remove/priority/TC_ACCT, so they
have to stay unique.

Covered end-to-end through the CLI, since the bug was in how the upsert
reconciled against accounts already on disk. Reverting the fix fails 3 of the
5 new tests.
A thinking budget of 512 or less translated to effort "minimal", which Codex
rejects outright:

  Unsupported value: 'minimal' is not supported with the 'gpt-5.6-sol' model.
  Supported values are: 'none', 'low', 'medium', 'high', 'xhigh', and 'max'.

So did "auto", from a budget of -1. Both took down the entire request.

The differential fixtures could not have caught this. They pin the translator
against CLIProxyAPI's translator, and CLIProxyAPI clamps in a later pipeline
stage (ApplyThinking) that the fixtures never exercised — a blind spot in
testing a single function against a single function rather than pipeline
against pipeline.

The clamp is therefore a separate step rather than folded into the translator,
mirroring where CLIProxyAPI does it and keeping the fixtures honest. `minimal`
becomes `low` rather than `none`, which would switch reasoning off instead of
reducing it; `auto` becomes `medium`, the backend default.

Verified live: the exact request that returned 400 now returns 200.
…rvable paths

Two problems found by running a real Claude Code session through the proxy.

An upstream error was fed to the SSE translator. isStreaming forced the
streaming path for every codex response regardless of status, but codex answers
with SSE only on success — an error is a plain JSON body. The translator found
no events in it and emitted nothing, so the client got "400 status code (no
body)" and the message explaining the failure was destroyed. That message was
the diagnosis for the second problem, so this one hid it.

Anthropic-only endpoints 404ed when rotation landed on codex. Claude Code calls
bootstrap, oauth/usage, mcp_servers and count_tokens alongside /v1/messages;
none has a Responses-API equivalent, and refusing them broke those features
outright. They now fail over to an account that can serve them, but only when
such an account exists — retrying unconditionally would exhaust the rotation
and surface as a 429, which misdiagnoses "no account speaks this API" as
"rate limited".
modelMap can redirect a request to a different model, but the client only
ever knows the model it asked for. Nothing recorded which one really ran, so
"why does this answer look different?" had no answer short of reading a
request log.

The proxy now records the serving account and resolved model per session and
exposes it at GET /teamclaude/status under sessions.byId, keyed by the
x-claude-code-session-id the client already sends. A status line holding that
id can ask what served ITS session rather than inferring from fleet-wide
state. The TUI activity line shows the same thing inline as
`(claude-opus-5 → gpt-5.6-sol)`.

Useful for any redirected backend, not just codex — a DeepSeek or GLM account
with a modelMap has exactly the same blind spot today.
Covers the setup flow, why `login --codex` is preferred over `import --codex`
(an import copies the Codex CLI's credential, so both hold one rotating
refresh token), the modelMap and priority a codex account needs, and the
limits worth knowing up front: Anthropic-only endpoints fail over to a Claude
account, reasoning effort is clamped to the levels Codex accepts, no reasoning
summaries come back, and a plan without Codex access is rejected outright.
Probing the live backend settled what the two available sources disagreed on.
The API's own enum message advertises `minimal`, but sending it draws a
model-specific rejection ("'minimal' is not supported with the 'gpt-5.6-sol'
model"), so the documented enum is wider than any model accepts. The Codex
CLI's model cache lists `ultra` as a supported level for sol and terra, but
the API rejects it outright — that cache describes the CLI's own menu, not the
wire contract.

What the backend actually answers to is `none`, `low`, `medium`, `high`,
`xhigh`, `max`. `ultra` previously fell through to the `medium` default, which
discarded far more of the caller's intent than needed; it now lands on `max`,
its nearest accepted neighbour.

All three Codex models — gpt-5.6-sol, gpt-5.6-terra, gpt-5.6-luna — are
verified end to end through the proxy, each returning its own id in
message_start. README now lists them with their descriptions and default
efforts instead of naming one id.
Claude Code reads /v1/models from the gateway at startup when
CLAUDE_CODE_ENABLE_GATEWAY_MODEL_DISCOVERY is set and offers whatever comes
back. Serving the Codex catalog there makes a GPT model a first-class choice
instead of something that has to be disguised as a Claude model via modelMap —
which is what CLIProxyAPI does too, detecting Claude Code by its
Anthropic-Version header.

The listing is the union of Anthropic's catalog, fetched with a real Claude
account's credential so it stays correct as models come and go, and each codex
account's, fetched from the backend's own /models endpoint (which requires a
client_version parameter and 400s without one). A failed Codex fetch returns
its stale cache or an empty list rather than throwing: a picker missing its
GPT entries is recoverable, one missing everything is not. The endpoint is
only intercepted when a codex account exists, so an Anthropic-only fleet is
untouched.

Selection is now protocol-aware, which is what makes picking a GPT model
actually work. A codex account is eligible for models Codex serves natively
plus anything its modelMap redirects; an Anthropic account is never eligible
for a Codex model. That also fixes the reverse case: a codex account with no
modelMap used to be picked for claude-* requests and 400 on every one.

Native models are matched by name rather than against the fetched catalog on
purpose — selection has to work before any catalog fetch has happened, and a
request naming an unknown gpt-* model is better rejected by the backend than
mis-routed to Anthropic.

All seven models the live catalog exposes verified end to end through the
proxy with no modelMap configured.
Serving the catalog at /v1/models was not enough on its own: Claude Code
discards any listed model whose id does not start with `claude-`, so all seven
GPT entries were silently dropped while the Claude ones came through.

CLIProxyAPI hit the same wall and solved it by encoding the real id into a
claude-prefixed one (internal/client/claude/models, EnsureClaudeModelIDPrefix:
`claude-fable-5-dd-` plus the id reversed, so the result cannot collide with or
be mistaken for a real Anthropic model). This mirrors that byte-for-byte, so a
setup written against one proxy behaves the same against the other. The id is
decoded before anything reads the model, so blocklists, routing, quota buckets,
the activity log and the upstream request all see the real name; only the
listing and the client's own picker ever hold the encoded form. The display
name is untouched, so the picker reads "GPT-5.6-Sol".

The listing now carries only what TeamClaude adds. Claude Code merges a gateway
listing with its own built-in models, so echoing Anthropic's catalog back
duplicated every Claude entry in the picker — one built-in and one "From
gateway". `modelDiscovery.includeAnthropic` restores the full list for a client
that needs it.

Verified live: all seven models selected by their cloaked ids, each routing to
Codex and reporting its own id back, with Claude models still served normally
and no modelMap configured.
blockedModels already rejects a request with a fast 400, so advertising such a
model put a choice in Claude Code's picker that failed the moment it was used.
One setting now means one thing: a model the proxy will not serve is neither
offered nor accepted.

Filtering happens on the real model id, before ids are cloaked for the picker,
so a pattern reads the way a user wrote it rather than against an encoded form.
Patterns keep their anchored semantics — `gpt-5.4` does not also remove
`gpt-5.4-mini`, so trimming a generation is explicit or uses a wildcard.
Codex caches on a prompt prefix but scopes the cache by a key, and we were
never setting one. The backend falls back to echoing the Session_id header as
the key, which happened to work for Claude Code because that header carried a
stable per-session id — but any client that sends no session header got a
fresh UUID per request and therefore cached nothing at all, ever.

Measured against the live backend on a 3k-token prompt: a stable key caches
2816 of 3016 input tokens from the second turn on, a per-request key caches
zero. So this was a silent, complete loss of caching for every non-Claude-Code
caller, and a dependency on undocumented header-echo behaviour for the rest.

The key is now derived — session, sub-agent and model, hashed — and set both
in the body as prompt_cache_key and on the Session_id header, mirroring what
CLIProxyAPI does (helps.ClaudeCodePromptCache). Deriving rather than storing
keeps it stable with no state to keep; hashing keeps a session identifier from
travelling to the backend in the clear. A caller with no session id falls back
to account plus model, which is what turns "never caches" into "caches".

Also maps input_tokens_details.cache_write_tokens onto Anthropic's
cache_creation_input_tokens. It was being dropped, so a turn that primed the
cache reported as cheaper than it was.

Verified end to end through the proxy: caching now engages both with and
without a client session id, and the key is confirmed identical across every
turn of a conversation.
@MagicalTux

Copy link
Copy Markdown
Member

Thanks for this — the engineering quality here is not in question. Differential-testing the translators against CLIProxyAPI, then mutation-testing the corpus when both passed first try and finding three real gaps, is a more rigorous validation story than most of this repo has. The four issues you found only by running a live session (the missing content-type on Codex SSE especially) are exactly the kind of thing fixtures structurally cannot catch, and calling that out is appreciated.

I am not merging it yet, and I want to be straight about why rather than leave it sitting silently.

This overlaps #95, which is my own in-progress multi-provider work, and the two take genuinely different architectures. #95 routes by host through MITM with one AccountManager per provider behind a router; this PR translates at /v1/messages and keeps a single pool. Both are defensible. They are not composable, so this is a design decision about where the provider seam belongs in this codebase, and that call is mine to make rather than something to settle by merge order.

The client-id point you raised in "Known limits" is the other half of it, and I am glad you surfaced it instead of burying it. It deserves a deliberate answer, not an implicit one.

So: leaving this open pending my architectural review against #95. Nothing is being asked of you right now — please do not rebase on my account. If I go with your approach, I would rather work from this branch than reimplement it. Two things I would likely take regardless of which way that lands:

  • the observability commit (serving account + resolved model per session, exposed on /teamclaude/status and in the TUI) — you are right that modelMap has this blind spot for DeepSeek and GLM today, and it stands alone
  • blockedModels filtering the /v1/models listing, for the same reason: advertising a model the proxy will 400 on is a bug independent of Codex

If you want to split either out into its own PR, I will review it on its own merits promptly. Otherwise I will come back to this thread once I have worked through the seam question.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants