Skip to content

fix: preserve cached and reasoning tokens without double billing - #140

Merged
tishachawla-jg merged 1 commit into
theagentplane:mainfrom
kevin-lozada-santos:fix/disjoint-provider-usage
Sep 16, 2026
Merged

tishachawla-jg merged 1 commit into
theagentplane:mainfrom
kevin-lozada-santos:fix/disjoint-provider-usage

Conversation

@kevin-lozada-santos

Copy link
Copy Markdown
Contributor

Cache-read and reasoning counts were dropped before reaching pricing. This inflated the issue's OpenAI-compatible high-cache fixture from 156,204 to 461,258 microdollars; native Anthropic instead lost the cache-read charge.

Normalize SDK responses and flat agent-step/dispatch counts into disjoint Usage buckets exactly once. input excludes cached reads and output excludes reasoning; flat ModelResponse/step counts remain inclusive totals. Populate both bundled adapters, account for native Anthropic's already-disjoint input, and preserve total prompt size in context-compaction trends. Document the contract and migration in Usage, ModelResponse, and the changelog. Pricing itself is unchanged.

Validation:

  • 13 new offline tests cover Chat Completions, Responses, native Anthropic, both actual adapters with mocked SDK calls, flat responses/steps, zero/missing details, legacy responses, reasoning pricing, the reported high-cache fixture, and rising cached context. 11 failed before implementation and all 13 pass afterward.
  • Full default python -m pytest -q: 244 passed, 43 skipped, 1 live test deselected; one dependency deprecation warning.
  • ruff check src tests examples, ruff format --check src tests examples, and mypy src/tokenops pass (58 source files).
  • git diff --check passes.

No live provider calls or original SWE-bench rerun. Cache-write premiums, streaming usage, additional adapters and context-compaction capabilities remain out of scope. The change to the compaction detector only retains its existing total-context semantics after separating cache reads.

Fixes #137. Follows the provider-aware approach discussed and approved in the issue.

@kevin-lozada-santos

Copy link
Copy Markdown
Contributor Author

The implementation and all local default tests/lint/type checks are complete. GitHub is waiting for maintainer approval to run the CI workflow for this contribution. Please enable it when reviewing the PR; I can address any resulting failures.

@tishachawla-jg
tishachawla-jg merged commit b5e58ab into theagentplane:main Sep 16, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cached tokens are billed at full input rate: Usage.cached is never populated (3x over-billing measured)

2 participants