Skip to content

Don't report transcript usage the proxy already priced, or a turn that hasn't finished (0.23.0) - #40

Merged
davidcrowe merged 1 commit into
mainfrom
fix/transcript-usage-proxy-and-partial
Sep 18, 2026
Merged

davidcrowe merged 1 commit into
mainfrom
fix/transcript-usage-proxy-and-partial

Conversation

@davidcrowe

Copy link
Copy Markdown
Collaborator

Two accounting defects in the hook's transcript collector, both found while auditing our own cost numbers on 2026-09-17.

1. A proxied session reported turns the proxy had already priced

collectTranscriptUsage ran on every PostToolUse with no check for where the model calls actually went. It exists for plain claude sessions that never touch the proxy — but a claude-acp session sends its calls through the proxy and fires this hook, so every call was priced twice.

Measured on our own workspace: llm.proxy.anthropic and llm.transcript.anthropic rows ~100ms apart carrying identical cost, on every sampled turn.

Gated on the base URL, not on ACP_KEY. launchedViaLauncher() exists next door and was the obvious call, but it also returns true when ACP_KEY alone is set — a credential someone can export while running plain claude, whose model calls still go straight to the provider. Suppressing those would delete the only record of their spend. Only the base URL says where the tokens actually went, and a pricing decision has to follow that rather than the credential. (It would also have silently emptied every existing test in transcript-model-usage.test.mjs, since the harness sets ACP_KEY — they'd have "passed" by doing nothing.)

2. A turn still streaming was reported from its partial record

One API call is appended to the transcript as many records, one per content block. cache_read / cache_creation / input_tokens are fixed at request time and identical on every record, but output_tokens is a placeholder — commonly 1–8 — until the final record, which is the only one carrying a terminal stop_reason.

PostToolUse fires between a turn's records, so a turn can straddle two read windows. The first window shipped the placeholder, and the gateway's first-wins dedupe locked it in, dropping the real total when it arrived.

Measured across a month of local transcripts: 2,748 calls sitting at output_tokens=3 and 2,263 at =2 on their partial rows. Worst case 39.9% of output tokens lost, while cache-read stayed exact — that asymmetry is the signature of the mechanism.

So a turn is reported only once its terminal stop_reason has been seen, and the offset is held at the first byte of the first incomplete turn so its remaining records are re-read next time. A turn that never completes — aborted or interrupted, ~0.7% of calls — is never reported; losing those beats billing a placeholder for them, and the stall is bounded because TRANSCRIPT_READ_CAP eventually slides the window past it.

Turns that complete after an incomplete one are still sent; because the offset holds, they may be re-sent next window, where the gateway's markSeen absorbs them. No double-count.

Relationship to the gateway fix

Defect 1 also has a gateway-side guard (gatewaystack-connect#1236, proxy/proxySessions.ts), which is what covers harnesses and already-installed plugins that never upgrade. This half stops the duplicate being generated at all. The two are independent — either can ship first.

Test note

One existing fixture gained a stop_reason: the id-fallback test built turns without one, which under the new rule are incomplete by definition. Its assertion — the requestId/uuid fallback chain — is unchanged. Four new tests cover the withhold-then-bill sequence, the mixed complete/incomplete case, the proxy gate, and a plain session still reporting (so the gate can't pass by disabling the feature).

node --test test/ → 247 passing.

Release

The hook's content hash changes with this. The registry needs the 0.23.0 hash after merge, never before:

ee8de59ae945688870070640d2cb204732c6a475a73e9796f303d83f47297627

…t hasn't finished (0.23.0)

Two accounting defects in the hook's transcript collector, both found while
auditing our own cost numbers on 2026-09-17.

1. A proxied session was reporting turns the proxy had already priced.

collectTranscriptUsage ran on every PostToolUse with no check for where the
model calls actually went. It exists for plain `claude` sessions that never
touch the proxy — but a `claude-acp` session sends its calls through the
proxy AND fires this hook, so every call was priced twice. On our own
workspace: llm.proxy.anthropic and llm.transcript.anthropic rows ~100ms
apart carrying identical cost, on every sampled turn.

Gated on the base URL, not on ACP_KEY. launchedViaLauncher() exists next
door and would have been the obvious call, but it also returns true when
ACP_KEY alone is set — a credential someone can export while running plain
`claude`, whose model calls still go straight to the provider. Suppressing
those would lose the only record of their spend. Only the base URL says
where the tokens actually went, and a pricing decision has to follow that
rather than the credential.

2. A turn still streaming was reported from its partial record.

One API call is appended to the transcript as many records, one per content
block. cache_read / cache_creation / input_tokens are fixed at request time
and identical on every record, but output_tokens is a placeholder — commonly
1-8 — until the final record, which is the only one carrying a terminal
stop_reason. PostToolUse fires between a turn's records, so a turn can
straddle two read windows; the first window shipped the placeholder and the
gateway's first-wins dedupe locked it in, dropping the real total when it
arrived. Measured across a month of local transcripts: 2,748 calls sitting
at output_tokens=3 and 2,263 at =2 on their partial rows, worst case 39.9%
of output tokens lost while cache-read stayed exact.

So a turn is reported only once its terminal stop_reason has been seen, and
the offset is held at the first byte of the first incomplete turn so its
remaining records are re-read next time. A turn that never completes — an
aborted or interrupted request, ~0.7% of calls — is never reported; losing
those beats billing a placeholder for them, and the stall is bounded because
TRANSCRIPT_READ_CAP eventually slides the window past it.

Defect 1 also has a gateway-side guard (proxy/proxySessions.ts), which is
what covers harnesses and already-installed plugins that never upgrade.
This half is what stops the duplicate being generated at all.

One existing test fixture gained a stop_reason: the id-fallback test built
turns without one, which under the new rule are incomplete by definition.
Its assertion — the requestId/uuid fallback chain — is unchanged.

Note for release: the hook's content hash changes with this, so the registry
needs the 0.23.0 hash AFTER merge, never before:
  ee8de59ae945688870070640d2cb204732c6a475a73e9796f303d83f47297627
@davidcrowe
davidcrowe merged commit 4787e85 into main Sep 18, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant