Skip to content

codex: cost double-bills cached input and reasoning — both are subsets, not additions #27

Description

@lroolle

From the first cross-provider field eval (claude-code f8e49d74 vs codex 019fa810, same workspace, 2026-07-28).

Symptom. A 36-minute Codex investigation session traces at $101.99 — more than a full day of Claude work ($93.35) in the same workspace. The header reads 6.6m in/59k out, cache 6.3m r.

Cause. Two subset relationships in Codex usage that ccx treats as disjoint categories:

  1. input_tokens includes cached_input_tokens. Upstream codex-rs (protocol/src/protocol.rs): non_cached_input = input_tokens - cached_input(). ccx stores the raw values (internal/provider/codex/backend.go, token_count handling) and ComputeCost (internal/parser/pricing.go) bills InputTokens at the input rate plus CacheReadTokens at the cache rate — cached tokens are billed twice, once at full input rate.
  2. reasoning_output_tokens is a subset of output_tokens (OpenAI output_tokens_details.reasoning_tokens). ComputeCost adds ReasoningTokens * OutputPer1M on top of OutputTokens * OutputPer1M — reasoning billed twice.

Verification against the session: turn 2 shows 2.9m in, 31k out, cache 2.8m r → ccx math 2.9m*$10 + 31k*$80 + 2.8m*$5 = $45.5 ≈ the displayed $45.30. Honest math (subset semantics): ~100k*$10 + 31k*$80 + 2.8m*$5 ≈ $17.5. Session total honest ≈ $39 vs displayed $101.99 — ~2.6x overstated.

Also display semantics. Claude's in excludes cache reads (Anthropic usage semantics); Codex's in includes them. The same header line means different things per provider — 6.6m in on a session whose uncached input is ~300k.

Fix. Normalize at the Codex parse layer to the canonical exclusive semantics (Anthropic-style, which the rest of ccx already assumes):

  • InputTokens = input_tokens - cached_input_tokens (clamped), for both the session aggregate and per-message deltas
  • OutputTokens unchanged (keeps reasoning inside); ReasoningTokens stays informational
  • drop the reasoning term from ComputeCost (same rate as output, already counted)
  • document the contract on MessageUsage

Priority: P1 — cost is a headline number and it is wrong for every Codex session with cache hits, i.e. all of them.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions