Send transcript model usage on PostToolUse so plain claude sessions get costed (0.18.0) - #36
Merged
Merged
Conversation
… get costed (0.18.0) gatewaystack-connect#932 shipped the gateway-side collector on 2026-09-02 (transcriptUsage.ts: sanitizeModelUsage + recordTranscriptUsage, fully live, no rollout gate) together with a client-side collectTranscriptUsage — but only inside the marketing installer's bundled OFFLINE fallback engine, which runs when the GitHub fetch fails at install time. Online installs fetch this file, and for Claude Code the installer removes the direct govern.mjs wiring once the marketplace plugin is in, so ~/.acp/govern.mjs is not even invoked. `git log --all -S'model_usage'` on this repo returned nothing through 0.17.0. The gateway has been waiting two weeks for a field no shipped client ever sent; every external harness tenant has zero costed runs. This is the port. - Every hook payload carries transcript_path; the session JSONL records each assistant turn's model + usage (input, cache-read, cache-write, output tokens). PostToolUse reads what arrived since the last report (a per-transcript byte offset in ~/.acp/transcript-offsets.json) and sends the turns as `model_usage` on the /govern/tool-output call it already makes. Only complete lines are consumed; a torn trailing line waits. - Shape is exactly what transcriptUsage.ts validates: string id (message id, else requestId, else uuid) and model, four non-negative integer counts (missing = 0), optional ISO ts. Same-id streamed repeats collapse to the last. At most 2 MB read and 50 turns sent per call; the offsets file drops transcripts that no longer exist and keeps at most 40 others. - Absent → absent: no transcript_path, a missing / empty / unreadable / malformed transcript, or nothing new since last time all leave the key out entirely (never an empty array). The whole collector is wrapped, so no failure in it can reach the decision or delay the call; PreToolUse is untouched. - Version 0.18.0 (plugin.json, marketplace.json, PLUGIN_VERSION) so the attestation hash is compared against a fresh version, not flagged as an edited 0.17.0 hook. Tests (test/transcript-model-usage.test.mjs, spawns the real hook against a stub gateway): well-formed turns sent in the gateway's shape; missing counts default to 0; malformed / non-assistant / id-less / model-less lines skipped and same-id repeats keep the last; id fallback order; no transcript_path → no key; missing / empty / user-only transcript → no key; offset advances and nothing-new → no key; partial trailing line waits; 50-turn cap keeps the newest; corrupt offsets file ignored; a directory or non-UTF-8 bytes at transcript_path leaves the PostToolUse outcome unchanged; PreToolUse body unchanged. Also verified by hand against a real 2.4 MB local transcript: 50 entries, every one passes the gateway's check.
davidcrowe
added a commit
that referenced
this pull request
Sep 16, 2026
…licy half it lacks #36 made plain `claude` priced — from the transcript, at API-rate equivalents — so "neither priced nor policy-checked" became false the moment it merged. Rewritten to lead with what the session has and name only what still needs the launcher: [ACP] Tool calls in this session are checked and logged, and model-call cost is estimated from the session transcript (API-rate equivalent, not a metered charge). Model calls are not policy-checked: plain `claude` sends them straight to the provider, so tool-result redaction and model routing are off. For those, launch with `claude-acp` (~/.acp/bin/claude-acp). Shown once per session. The estimate-vs-metered distinction is stated up front so it does not arrive later as a support question. The wording is chosen by the same condition that makes the session priced — a readable transcript_path on the payload — so a harness that hands over no transcript still gets the plainer "neither priced nor policy-checked" form rather than a claim the hook cannot back. Every mechanical guarantee is unchanged: allow path only, once per session via the exclusive marker, silence with no session_id, never in --local, never with ACP_KEY or a proxy base URL. Tests: wording assertions updated; new case for a payload with no (or a missing) transcript; transcript-model-usage.test.mjs runs as a launched session so its PreToolUse assertion stays about the body. 220/220.
davidcrowe
added a commit
that referenced
this pull request
Sep 16, 2026
…-checked (0.19.0) (#37) * Tell plain-launch sessions their model calls are not priced or policy-checked (0.19.0) Production 2026-09-16: 0 of 69 external tenants have a single llm.proxy.* row, and every external harness tenant has zero costed runs. Everyone runs the plain binary; nobody runs <harness>-acp, and nothing tells them — the only surface that mentions cost is the launcher's own exit line, which only prints after you already used the launcher. The hook runs under the plain binary, so the hook says it. Port of the installer's bundled-engine change (agenticcontrolplane.com PR #178) into the govern.mjs that online installs fetch and the marketplace plugin runs — #178 alone reaches nobody in today's data. One systemMessage, once per session, on the first ALLOWED call: [ACP] Tool calls in this session are checked and logged. Model calls are not: plain `claude` sends them straight to the provider, so they are neither priced nor policy-checked (tool-result redaction, model routing). For the cost X-ray and model-call policy, launch with `claude-acp` (~/.acp/bin/claude-acp). Shown once per session. - Detection: every launcher is generated from one template whose first act is `export ACP_KEY`; the hook inherits that env. No ACP_KEY, and no provider base URL at the ACP proxy, means the session was not started by a launcher. Bias to silence on doubt. - Once per session: marker ~/.acp/session-notices/unpriced-<session_id> (id sanitised to [A-Za-z0-9._-], capped at 120), created with an exclusive flag so parallel hook processes for the same session cannot both fire. No session_id in the payload means silence, no marker. Markers older than 7 days are pruned; the directory is capped at 200. - Every harness with a launcher is named by ACP_CLIENT (claude-acp, codex-acp for both "codex" and "codex-plugin", qwen-acp, opencode-acp, pi-acp, hermes-acp, prime-acp, grok-acp, dsh-acp). Cursor has no launcher and gets nothing. - Marketplace-only installs (`claude plugin install`) have this hook but no launcher on disk; for them the notice names the installer instead of a path that does not exist. One deviation from #178, in favour of saying something true. - Cloud mode only: the local path never reaches it. No-credential and managed-unenrolled paths never reach it either. - Purely additive: rides the allow path as one JSON object (merged with the tier notice and the wire warning when they apply — the updatedInput branch now reads the message once instead of twice), never a deny or an ask, and the whole thing is wrapped so any failure is silence. - Version 0.19.0 so the attestation hash is compared against a fresh version. Branched off main independently of the transcript-usage PR (0.18.0); the two conflict in bin/govern.mjs and this one rebases. Tests (test/unpriced-session-notice.test.mjs, real hook against a stub gateway): notice on a plain launch with nothing that could change the call; silent on the 2nd/3rd call of the same session; fires once for a new session; nothing without a session id; nothing with ACP_KEY in env or a provider base URL at the proxy; deny and ask byte-identical with no notice, then the notice rides the session's first allowed call; wire warning + notice as ONE object; codex names codex-acp, cursor gets nothing; six parallel processes yield exactly one notice; stale markers pruned and the directory capped; hostile session ids sanitised; launcher absent names the installer; --local silent. wire-warning.test.mjs now runs as a launched session so its assertions stay about the warning. * Notice: say what a plain-launch session HAS after 0.18.0, then the policy half it lacks #36 made plain `claude` priced — from the transcript, at API-rate equivalents — so "neither priced nor policy-checked" became false the moment it merged. Rewritten to lead with what the session has and name only what still needs the launcher: [ACP] Tool calls in this session are checked and logged, and model-call cost is estimated from the session transcript (API-rate equivalent, not a metered charge). Model calls are not policy-checked: plain `claude` sends them straight to the provider, so tool-result redaction and model routing are off. For those, launch with `claude-acp` (~/.acp/bin/claude-acp). Shown once per session. The estimate-vs-metered distinction is stated up front so it does not arrive later as a support question. The wording is chosen by the same condition that makes the session priced — a readable transcript_path on the payload — so a harness that hands over no transcript still gets the plainer "neither priced nor policy-checked" form rather than a claim the hook cannot back. Every mechanical guarantee is unchanged: allow path only, once per session via the exclusive marker, silence with no session_id, never in --local, never with ACP_KEY or a proxy base URL. Tests: wording assertions updated; new case for a payload with no (or a missing) transcript; transcript-model-usage.test.mjs runs as a launched session so its PreToolUse assertion stays about the body. 220/220.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
gatewaystack-connect#932 (2026-09-02) shipped the gateway-side collector —
transcriptUsage.ts:sanitizeModelUsage+recordTranscriptUsage, fully live, no rollout gate — and a client-sidecollectTranscriptUsage. But the client half landed only in the marketing installer's bundled offline fallback engine (install.shheredoc,PLUGIN_VERSION = "0.5.0"), which runs only when the GitHub fetch fails at install time. Online installs fetch this file, and for Claude Code the installer removes the directgovern.mjswiring once the marketplace plugin is in — so~/.acp/govern.mjsis not even invoked.git log --all -S'model_usage'on this repo returns nothing through 0.17.0. The gateway has been waiting two weeks for a field no shipped client has ever sent; every external harness tenant has zero costed runs.This ports the collector into the hook that actually executes. Once shipped, cost starts working for every existing plain-
claudeuser with no change to how they launch and no proxy.What
transcript_path; the session JSONL records each assistant turn'smodel+usage. On PostToolUse the hook reads what arrived since its last report (per-transcript byte offset in~/.acp/transcript-offsets.json) and sends the turns asmodel_usageon the/govern/tool-outputcall it already makes. Only complete lines are consumed; a torn trailing line waits for the next call.transcriptUsage.ts:78–98validates: stringid(message id →requestId→uuid) andmodel; four non-negative integer counts (missing = 0); optional ISOts. Same-id streamed repeats collapse to the last. At most 2 MB read and 50 turns sent per call. The offsets file drops transcripts that no longer exist and keeps at most 40 others.transcript_path, a missing / empty / unreadable / malformed transcript, or nothing new since last time all leave the key out entirely — never an empty array. The whole collector is wrapped in try/catch, so nothing in it can reach the decision or delay the call. PreToolUse is untouched.plugin.json,marketplace.json,PLUGIN_VERSION) so attestation compares the hash against a fresh version rather than flagging an edited 0.17.0 hook.Semantics mirror the reference implementation in the installer heredoc (
install.sh:730–785,:803), including advancing the offset before the request goes out: a failed POST loses that window rather than risking a double-count (the gateway's(session, id)dedupe is in-process only). Worth revisiting if under-reporting shows up in the data.Tests
node --test— 204 pass, 0 fail (192 existing + 12 new intest/transcript-model-usage.test.mjs, which spawns the real hook against a stub gateway):transcript_path→ no key; missing / empty / user-only transcript → no keytranscript_pathleaves the PostToolUse outcome unchanged; PreToolUse body unchangedAlso verified by hand against a real 2.4 MB local Claude Code transcript: 50 entries, every one passes the gateway's shape check (
claude-opus-5, cache-read counts in the 200k range, real message ids and timestamps).Conflicts / merge order
This PR and the unpriced-session notice PR (
feat/unpriced-session-notice) both touchbin/govern.mjsand both bump the version; they are branched independently offmain, not stacked. Merge this one first (it fixes the cost gap for everyone; the notice only tells people about it), then rebase the notice PR — its bump is already 0.19.0 so the rebase is mechanical.Not merged, not published to npm, not deployed. Publishing is yours.