refactor(usage): give the model-call read model real pricing columns - #4869
Conversation
…odel The Usage read model copied each AgentRun `model_call_attempt_recorded` payload verbatim into `usage_model_call_attempts`. The authority was never wrong; the projection selected too much. On a real workspace that made `requestObservation` 97% of the table: 383 rows holding 10.45 MB of record_json, averaging 28.6 KB each, while every cost answer reads a few hundred bytes of it. `ModelCallPricingRecord` now names what a Usage answer reads, and `ModelCallAttempt` extends it, so the read path depends on the narrow type while the authority keeps the whole record. The ledger writes rows through `projectModelCallPricingRecord` and reads them back through a codec held to that exact shape. Rows written before this are folded in place through the same function rather than rebuilt from the authority: deleting a Session drops its `core_agent_runs` rows and cascades their events while its ledger rows stay, so a wipe-and-replay would erase that spend from the all-time totals. On the same workspace the table drops from 10.66 MB to 0.89 MB with every pricing number unchanged. Generated-by: Claude Code
…re dead helpers Cleanup pass over the previous commit. `projectModelCallPricingRecord` named all 21 fields a third time, next to the interface and the shape. Only the shape is exhaustively checked at compile time, so an added optional field would have been silently dropped. It now projects through `pickShape`, a new `record-schema` helper that keeps the keys a shape allows — the shape becomes the one list, and the special-cased token loop goes with it. The migration selected every row into JS and used a thrown decode error to mean "already narrow", so its steady state was a full-table round trip that could never do anything. `schemaVersion` discriminates the two shapes in both directions — required on an attempt, rejected by the pricing decoder — so SQLite now selects exactly the rows still to fold, and the remaining catch means what it says. `assertPricingInvariants` took a label only to vary an error prefix no caller reads. `sumModelCallCostUsd` and `isModelCallAttempt` had no production consumers; the former was being retyped here for nobody. Records the invariant this all rests on where it can be found: deleting a Session cascades its runs and events but deliberately leaves the ledger rows, so for a deleted Session the projection is the last copy of that spend. Generated-by: Claude Code
Each test the narrowing added now stands for one obligation nothing else covers. Dropped: a core idempotency assertion the storage "already narrowed is untouched" test subsumes; an absent-optional assertion over a `pickShape` detail that neither JSON nor the decoder can observe; a byte-ratio assertion that measured the fixture's size rather than the contract. The paging test now uses the small fixture — it needs many wide rows, not large ones. Comments state the rule a reader needs and stop arguing for the change; the argument is this branch's commit and PR history. Generated-by: Claude Code
81d2f1d to
7ac9367
Compare
The read model stored each attempt as a JSON blob and answered every Usage question by handing the caller every matching record to fold in JS. An all-time total therefore materialized a workspace's entire model-call history, and the blob carried request diagnostics that grow with the conversation rather than with spend: 383 rows held 10.9 MB of `record_json` in a real workspace. The fields a cost answer reads are now columns, so a total is a SUM the table computes and the reads return answers instead of records. Damaged rows keep the three columns they already had — attempt_id, completed_at, session_id — and leave the pricing columns empty; "unreadable" is now `cost_basis IS NULL`, held sound by an all-or-nothing CHECK, so a row whose pricing was lost is still counted and reported rather than dropped, and one of them cannot fail the query. Migration rewrites the table in place. It cannot wipe and replay the AgentRun stream: deleting a Session cascades its events away while these rows deliberately survive, so for those rows this table is the last copy. Generated-by: Claude Code
Astro-Han
changed the title
github-actions
Bot
added
effort/XXL
jackwener
left a comment
jackwener
left a comment
There was a problem hiding this comment.
Reviewed at exact head c841b629. One [P1]: the migration prices rows the old reader refused, so spend that was excluded from every total now enters it — including negative amounts. The rest of the read-model rewrite holds up, and for valid rows the migration is faithful field for field.
[P1] "Readable" got looser than the decoder it replaced, and the difference lands in the bill
The old path decided a row was usable through decodeModelCallAttempt, which enforced a vocabulary, an exact shape and sane values. v7 decides the same question structurally: nine columns IS NOT NULL plus the cost/usage CHECKs. No vocabulary, no exact-shape check, no sign check.
So a blob the old reader rejected — but from which json_extract can still pull those nine fields — is migrated as cost_basis = 'priced'. countableFilter is cost_basis IS NOT NULL, so it enters SUM; and because unreadableRecords is cost_basis IS NULL, it is not reported as unreadable either. It is silently counted.
Measured on production SQLite:
| stored blob | old JS reader | after v7 |
|---|---|---|
invalid JSON, or {sessionId} only |
reject | tombstone (correct) |
status: 'COMPLETE' |
reject | priced, $0.004, 100 in |
callKind: 'not-a-kind' |
reject | priced, $0.004 |
inputTokens: -5 |
reject | priced, input −5 |
costUsd: -0.01 |
reject | priced, −$0.01 |
| an extra JSON key | reject (hasExactShape) |
priced, $0.004 |
logicalCallId: '' |
reject | priced, $0.004 |
The stated rationale is right; it is the second half that is missing. Not putting a CHECK on status or call_kind so that one damaged row cannot fail a whole workspace's migration is a good decision, and the tombstone contract genuinely holds for rows that fail extraction. But "must not fail the migration" and "must be counted as spend" are different requirements, and right now the first is being satisfied by doing the second. A row the reader would have refused belongs in unreadableRecords, not in the total — that is the #1638 contract this PR says it is holding structurally.
The shape of the fix stays inside the design: keep the table CHECK-free on those vocabularies, and apply the decoder's own rules — vocabulary, exact shape, non-negative tokens and cost — in the migration's classification step, so a row that fails any of them lands as a tombstone. That preserves both properties at once: the migration still cannot fail on bad data, and bad data still cannot become money.
Worth being explicit about severity, since this is a read model: the authority is untouched and nothing is destroyed, so the numbers are recoverable by fixing the classifier and re-running. What makes it P1 rather than P2 is that until then every all-time total silently includes rows that were never priced, and a negative costUsd moves the total in a direction no user can explain.
Faithful where it matters most
For blobs the old reader accepted, extraction matches the JS decode map with fieldDiffs: [] — priced 0.004 with every token column equal, and the unpriced/missing-usage cases equal too. The concern with a json_extract rewrite is that a row extracts but yields a different number; that did not happen.
The corrupt-row contract holds as described: a damaged JSON row beside a good one migrates without failing, lands cost_basis NULL, and leaves the good row at 0.004. The nine-column all-or-nothing CHECK is real — a half-record INSERT throws.
The reason for creating the new table under its final name is also real, not stylistic: ALTER TABLE … RENAME leaves the old column list in sqlite_schema.sql, so a renamed table keeps a CREATE text still mentioning record_json and assertCurrentOperationalTargetSchema's exact comparison would fail. Confirmed on SQLite. Version accounting checks out — SQLITE_USAGE_SCHEMA_VERSION = 7 is what the registry writes, the jump is 5→7 with no v6 ever shipped, and assertSupportedOperationalSchemaVersion throws when observed exceeds supported.
The SQL mirrors agree with the TypeScript
Since the same rules are now written twice, in two languages, they were compared rather than trusted:
- Status is a faithful three-way.
success→status = 'completed',error→status = 'failed', and everything else →status NOT IN ('completed','failed'), which is exactlyusageStatusForAttempt'sreturn 'aborted'catch-all. - The places where SQL looks more permissive are unreachable for any row that reaches a total.
CACHE_READ_TOKENScarries aninput_tokens IS NULLbranch andCOALESCE(…, 0)where the TypeScript is a bareMath.min. Butinput_tokensis one of the nine all-or-nothing columns, so a row withcost_basisset always has it — those branches can only be reached by tombstones, which totals exclude. Defensive, not divergent.
One property worth stating plainly because it shapes how the numbers must be presented: SUM(PRICED_COST) returns 0, not NULL, when nothing in the window is priced. The only thing separating "these calls were free" from "their price was never resolved" is the coverage pair beside it. Wherever a total is shown, its coverage has to be shown with it — which is also why the P1 above matters: it moves rows from the coverage side of that pair onto the money side.
简体中文
在 exact head c841b629 上评审。一条 [P1]:迁移把旧读者拒绝过的行判为已定价,于是本来被排除在所有总额之外的花费进入了总额 —— 其中包括负数金额。 读模型重写的其余部分成立,而且对合法行,迁移是逐字段忠实的。
[P1] 「可读」比它所取代的解码器更松,而这个差值落进了账单
旧路径通过 decodeModelCallAttempt 判定一行是否可用,它强制词表、精确形状与合理取值。v7 用结构判定同一个问题:九列 IS NOT NULL 加上 cost/usage 的 CHECK。没有词表、没有精确形状、没有符号检查。
于是一个旧读者会拒绝、但 json_extract 仍能取出那九个字段的 blob,会被迁移成 cost_basis = 'priced'。countableFilter 是 cost_basis IS NOT NULL,所以它进入 SUM;而 unreadableRecords 是 cost_basis IS NULL,所以它也不会被报为不可读。它是被静默计入的。
在生产 SQLite 上实测:无效 JSON 或只有 {sessionId} → 墓碑(正确);status: 'COMPLETE'、callKind: 'not-a-kind'、多一个 JSON 键、logicalCallId: '' → 均按已定价计入 $0.004;inputTokens: -5 → 已定价,输入 −5;costUsd: -0.01 → 已定价,−$0.01。
它给出的理由是对的,缺的是后半句。 不给 status / call_kind 加 CHECK、以免一行坏数据让整个工作区迁移失败,这是个好决定,而且对抽取失败的行,墓碑合同确实成立。但「不能让迁移失败」与「必须计入花费」是两个不同的要求,而现在前者是靠做到后者来满足的。一行旧读者会拒绝的记录,应当进 unreadableRecords,而不是进总额 —— 那正是本 PR 自称要结构性地守住的 #1638 合同。
修法留在原设计之内:表上仍然不加那些词表 CHECK,而是在迁移的分类步骤里套用解码器自己的规则(词表、精确形状、token 与金额非负),任何一条不过就落成墓碑。这样两个性质同时保住:迁移仍然不会因坏数据而失败,坏数据也仍然不会变成钱。
关于严重度值得说明:这是读模型,权威未动、没有东西被销毁,所以修好分类器再跑一次就能恢复这些数字。它之所以是 P1 而不是 P2,在于在修好之前,每一个 all-time 总额都静默包含了从未被定价的行,而一个负的 costUsd 会把总额推向没有用户能解释的方向。
最要紧的地方是忠实的
对旧读者接受的 blob,抽取结果与 JS 解码映射 fieldDiffs: [] —— 已定价的 0.004 与每一个 token 列都相等,未定价/缺用量的情形也相等。json_extract 重写最该担心的是「抽得出来但数字不同」,而这没有发生。
损坏行合同如描述成立;九列同生共死的 CHECK 是真的(半条记录的 INSERT 会抛)。以最终名建新表的理由也是真的而非风格问题:ALTER TABLE … RENAME 会在 sqlite_schema.sql 里留下旧的列清单,于是改名后的表其 CREATE 文本仍提到 record_json,assertCurrentOperationalTargetSchema 的逐字比对会失败 —— 已在 SQLite 上确认。版本记账也对:SQLITE_USAGE_SCHEMA_VERSION = 7,5→7 跳过从未发布的 v6,且 observed 超过 supported 时 assertSupportedOperationalSchemaVersion 会抛。
SQL 镜像与 TypeScript 一致
由于同一套规则现在用两种语言各写了一遍,这里是比对而不是采信:
- 状态是忠实的三分:
success→status = 'completed',error→status = 'failed',其余 →status NOT IN ('completed','failed'),恰好等于usageStatusForAttempt的return 'aborted'兜底。 - SQL 中看起来更宽松的地方,对任何能进入总额的行都不可达。
CACHE_READ_TOKENS带有input_tokens IS NULL分支与COALESCE(…, 0),而 TypeScript 只是一句Math.min;但input_tokens正是那九个同生共死列之一,所以只要cost_basis有值它就有值 —— 那些分支只有墓碑行走得到,而总额排除墓碑。是防御,不是分歧。
有一条性质值得直说,因为它决定了这些数字必须怎样呈现:当窗口内没有任何已定价行时,SUM(PRICED_COST) 返回 0 而不是 NULL。区分「这些调用是免费的」与「它们的价格从未被解析出来」的,只有旁边那对 coverage 计数。凡是展示总额的地方,都必须同时展示它的 coverage —— 这也正是上面那条 P1 要紧的原因:它把行从那一对里的 coverage 一侧,搬到了金钱一侧。
Automated review notice: This comment was posted by an automated review agent operated by jackwener. It is not an independent human review and does not replace one.
|
Not merging a change for this one — the divergence is real but the rows it describes cannot exist.
Each row in your table — What does change, and is worth stating: the old path re-decoded on every read, which incidentally guarded against a locally tampered database. v7 drops that. The consequence is one local number, the AgentRun authority is untouched, and it is recoverable by re-running a fixed classifier — P3, not P1. I looked at tightening the SQL classifier anyway and decided against it. Vocabulary and sign checks are expressible; exact shape is not, so SQL would hold a second, approximate definition of "readable" next to the decoder's — a standing synchronization cost for an input class with no producer. The only self-consistent alternative is classifying with Your closing point stands and is already the contract: |
|
Withdrawing the [P1] — it should be [P3], and your argument is right. I checked the load-bearing premise rather than taking it, and it holds on every axis. What I verifiedYour case rests on one claim: the decoder has only ever loosened, so no shipped Maka could write a row today's decoder would refuse. Checked against the tagged writers rather than against
With a single writer that inserts only after What the finding actually isYour reframing is the accurate one, and it is worth keeping in the record rather than dropping entirely: the old path re-decoded on every read, which incidentally validated the database on each query. v7 replaces that with a structural test done once at migration. For any row the product can produce these agree, and your real-workspace run — 383 rows, field-for-field identical, zero tombstones — is the evidence that they do. What changes is narrow: a row corrupted at rest by something outside Maka is now priced into a local total instead of being reported as unreadable. The AgentRun authority is untouched, nothing is destroyed, and a fixed classifier plus a re-run restores the number. That is a P3, and I am recording it at P3 rather than as a defect you should fix now. I also accept the reason for not tightening the SQL classifier. Vocabulary and sign are expressible in SQL; exact shape is not — so the classifier would hold a second, approximate definition of "readable" beside the decoder's, with a standing obligation to keep them in step, bought for an input class with no producer. Decoding every blob at upgrade time buys the same nothing at a much higher price. Leaving the structural test alone is the right call. On my side of itThe probe rows were synthesised by writing blobs directly, which is exactly the step that needed a producer argument before the grade was set. Establishing that a shape is reachable is part of the finding, not a detail after it — I graded first and would have owed you that check either way. The rest of the earlier review is unaffected: extraction is field-for-field faithful for valid rows, the tombstone contract and the nine-column CHECK hold, creating the table under its final name is necessary for 简体中文撤回那条 [P1] —— 它应当是 [P3],你的论证是对的。 我没有采信,而是去核了它的承重前提,结果每一条都成立。 我核了什么 你的论证只压在一句话上:解码器只曾放宽,所以任何已发布的 Maka 都写不出今天的解码器会拒绝的行。我是对照已发布 tag 的写入方核的,不是对照
再加上「唯一写入方且只在 这条发现真正是什么 你的重新表述才是准确的,而且值得留在记录里而不是整条丢掉:旧路径每次读取都重新解码,顺带在每次查询时校验了数据库;v7 用一次迁移时的结构判定取代了它。对产品能产出的任何行,两者一致 —— 你在真实工作区上的那次运行(383 行、逐字段相同、零墓碑)就是它们一致的证据。 改变的部分很窄:一行在静止状态下被 Maka 之外的东西损坏的记录,现在会被计入本地总额,而不是被报为不可读。AgentRun 权威未动,没有东西被销毁,修好分类器再跑一次即可恢复该数字。这是 P3,我按 P3 记录,而不是当作你现在必须修的缺陷。 不收紧 SQL 分类器的理由我也接受。 词表与符号在 SQL 里可表达,精确形状不可 —— 那样分类器就会在解码器之外持有第二份近似的「可读」定义,并背上让两者保持同步的长期义务,而买来的只是一个没有产出方的输入类。在升级时解码每一个 blob,花更高的代价买到同样的空。保持这个结构判定不动是对的。 我这一侧 那些探针行是直接写 blob 合成出来的,而这恰恰是在定级之前需要一个「产出方论证」的步骤。证明一种形状是可达的,是这条发现的一部分,而不是它之后的细节 —— 我先定了级,无论如何都欠你这一步核查。 先前评审的其余部分不受影响:合法行的抽取逐字段忠实、墓碑合同与九列 CHECK 成立、以最终名建表对
|
jackwener
left a comment
jackwener
left a comment
There was a problem hiding this comment.
Approving at exact head c841b629. The P1 is withdrawn, one [P3] remains, and nothing left blocks. Required test is green on this head; the PR is still a draft, and under this repository's rule this approval stays valid across the pushes that follow.
The review is complete across both lanes it was split into — migration fidelity, damaged rows and version accounting; and the agreement between the new SQL and the TypeScript rules it mirrors.
What the change gets right, in the order that matters for a read model that holds money:
- Extraction is faithful where faithfulness is the whole question. For every blob the old reader accepted,
json_extractproduces the same values as the JS decode map —fieldDiffs: [], priced and unpriced cases alike, and a real workspace migrates 383 rows field-for-field with zero tombstones. The characteristic failure of ajson_extractrewrite is a row that extracts but yields a different number; it does not happen here. - The damaged-row contract is structural, as claimed. A corrupt row beside a good one migrates without failing the workspace, lands
cost_basis NULL, and is counted intounreadableRecordsrather than dropped. The nine-column all-or-nothing CHECK is real — a half record throws — and leavingstatusandcall_kindunconstrained is what keeps one bad row from failing an upgrade. - The SQL mirrors match the TypeScript. Status is a faithful three-way:
success→status = 'completed',error→status = 'failed', everything else →status NOT IN ('completed','failed'), which is exactlyusageStatusForAttempt's catch-all. Where the SQL looks more permissive —CACHE_READ_TOKENS'sinput_tokens IS NULLbranch and itsCOALESCEagainst a bareMath.min— those branches are unreachable for any row that reaches a total, becauseinput_tokensis one of the nine columns that stand or fall together. Defensive, not divergent. - Creating the table under its final name is necessary, not stylistic.
ALTER TABLE … RENAMEleaves the old column list insqlite_schema.sql, so a renamed table keeps aCREATEtext still namingrecord_jsonand the exact comparison inassertCurrentOperationalTargetSchemawould fail. Version accounting is right: 5→7 with no v6 ever shipped, and an older Maka refuses a newer schema rather than writing a blob row back.
[P3] — recorded, not a defect to fix now. The old path re-decoded on every read, which incidentally revalidated the database on each query; v7 replaces that with one structural test at migration time. For any row the product can produce these agree. What changes is that a row corrupted at rest, by something outside Maka, is now priced into a local total instead of being reported as unreadable. The authority is untouched and a fixed classifier plus a re-run restores the number. Tightening the SQL classifier to close it would put a second, approximate definition of "readable" beside the decoder's — vocabulary and sign are expressible in SQL, exact shape is not — and buy that standing synchronization cost for an input class with no producer. Leaving it alone is the right trade.
One property to carry into whatever renders these numbers: SUM over a window with nothing priced returns 0, not NULL. A total is only meaningful beside its coverage, which is why ModelCallCoverage travelling with every answer is load-bearing rather than decoration.
简体中文
在 exact head c841b629 上批准。P1 已撤回,余下一条 [P3],没有东西再阻塞。 本 head 的必需 test 已绿;PR 仍是草稿,而按本仓库规则,此批准会跨其后的 push 继续有效。
评审的两条车道都已完成 —— 迁移保真度、损坏行与版本记账;以及新 SQL 与它所镜像的 TypeScript 规则之间的一致性。
这次改动做对的地方,按「一个承载金钱的读模型」该有的轻重排序:
- 在「忠实」就是全部问题的地方,抽取是忠实的。 对旧读者接受的每一个 blob,
json_extract产出与 JS 解码映射相同的值 ——fieldDiffs: [],已定价与未定价两种情形皆然;真实工作区 383 行逐字段迁移、零墓碑。json_extract重写的典型失败是「抽得出来但数字不同」,这里没有发生。 - 损坏行合同如其所称是结构性的。 一行损坏记录与一行正常记录并存时,迁移不会让整个工作区失败,它落为
cost_basis NULL,并被计入unreadableRecords而不是被丢弃。九列同生共死的 CHECK 是真的(半条记录会抛),而不约束status与call_kind正是让一行坏数据无法搞垮升级的原因。 - SQL 镜像与 TypeScript 相符。 状态是忠实的三分,恰好等于
usageStatusForAttempt的兜底。SQL 看起来更宽松之处(CACHE_READ_TOKENS的input_tokens IS NULL分支与COALESCE,对应 TS 里裸的Math.min),对任何能进入总额的行都不可达 —— 因为input_tokens正是那九个同生共死列之一。是防御,不是分歧。 - 以最终名建表是必需的,不是风格问题。
ALTER TABLE … RENAME会在sqlite_schema.sql里留下旧列清单,于是改名后的表其CREATE文本仍写着record_json,assertCurrentOperationalTargetSchema的逐字比对就会失败。版本记账正确:5→7,从未发布过 v6,且旧版本会拒绝更新的 schema 而不是把 blob 行写回去。
[P3] —— 记录在案,不是现在要修的缺陷。 旧路径每次读取都重新解码,顺带在每次查询时重新校验了数据库;v7 用迁移时的一次结构判定取代它。对产品能产出的任何行,两者一致。 改变的是:一行在静止态被 Maka 之外的东西损坏的记录,现在会被计入本地总额,而不是被报为不可读。权威未动,修好分类器并重跑即可恢复该数字。为堵它而收紧 SQL 分类器,会在解码器之外放上第二份近似的「可读」定义 —— 词表与符号在 SQL 里可表达,精确形状不可 —— 并为一个没有产出方的输入类背上长期同步成本。保持原样是正确的取舍。
有一条性质请带进任何呈现这些数字的地方:窗口内没有任何已定价行时,SUM 返回 0 而非 NULL。总额只有与它的 coverage 并列时才有意义,这也是 ModelCallCoverage 随每个答案一起返回属于承重设计而非装饰的原因。
Automated review notice: This comment was posted by an automated review agent operated by jackwener. It is not an independent human review and does not replace one.
2 participants
Summary
usage_model_call_attemptsis a read model, not a second authority: every row is projected from the AgentRun authority'smodel_call_attempt_recordedevents throughcatchUpProjection, andModelCallLedgerWriterhas no independent write entry point. The authority has been right the whole time. What was wrong is the shape of the read model and, following from it, the shape of every read.The row stored the authority event verbatim as one
record_jsonblob. Two consequences:ModelCallLedgerReader.readreturned records, not answers;projectModelCallUsageSummaryfolded them afterwards. A summary overrange: 'all'resolves to{ from: 0, to: now }, so asking a workspace for its all-time spend read every row it has ever written, parsed every blob, and summed in a loop. SQLite can do that sum without materializing anything.requestObservationandpromptCompositiondescribe the shape of a request and are read only by the Session Inspector, context diagnostics, and Desktop execution diagnostics — all of which decode the AgentRun stream directly and never touch this table. On a real workspace they were 97% of it: 383 rows holding 10.45 MB ofrecord_json, 28.6 KB per row.So the fields a cost answer reads become columns, and the reads become aggregates.
MODEL_CALL_COLUMNSis 21 columns;ModelCallLedgerReadernow exposessummary,bucketsandlogs, each of which is oneSELECTreturning one answer.packages/storage/src/model-call-usage-sql.tsholds the fragments those three share, and each fragment names the core function whose rule it mirrors (clampCacheReadTokens,usageStatusForAttempt) so the two do not drift silently. Time-bucket keys are still built in JS byusageBucketKey, the same function the legacy Usage source uses — SQL only decides which rows group together, so a day boundary is defined in exactly one place.Damaged rows.
attempt_id,completed_atandsession_idwere already columns, so a row whose pricing cannot be recovered keeps its identity and its window and leaves the pricing columns empty. "Unreadable" is nowcost_basis IS NULL, and a table CHECK makes that sound: the nine required columns are all present or all absent together, and no CHECK constrains thestatusorcall_kindvocabularies — one damaged row must not turn into a failed migration for a whole workspace. This is the#1638contract held structurally rather than by a decoder: such a row is still counted intounreadableRecords, still reported, never silently dropped, and cannot fail the query.Because the answers are computed in SQL, the JS that used to compute them is gone:
projectModelCallUsageSummary/Buckets/Logs, plusmodelCallAttemptsFromRunEvents,summarizeModelCallCoverage,sumModelCallCostUsd,settledAttemptandisModelCallAttempt, which had no production consumers left once the fold moved.pickShapestays inrecord-schemawith one consumer:toTraceAttemptnow derives the Inspector's wire shape fromMODEL_ATTEMPT_SHAPEinstead of restating 20 field names by hand.Nothing about the AgentRun authority changes, and no table is added.
Migration
Usage schema version 7 rewrites the table in place: the old table is renamed aside, the canonical table is created under its final name, and every row is re-inserted with
json_extractpulling each column out of the blob it still carries. A row the extraction cannot read — invalid JSON, or a shape with no pricing in it — lands as a tombstone: identity and window kept, pricing columns empty.It cannot wipe and replay the authority instead, which is the usual move for a read model.
usage_model_call_attemptshas no foreign key tocore_agent_runs, whilecore_agent_run_eventsand the projection checkpoints both cascade from it. Deleting a Session therefore removes the authority for its calls and deliberately leaves these rows standing, so spend does not vanish from all-time totals when a conversation is deleted. For those rows this table is the last copy, and a replay would erase exactly them. That invariant is now written down at both ends — the ledger's header comment andConversationOperationalStateStore.purge.The new table is created under its final name rather than renamed into place:
assertCurrentOperationalTargetSchemacompares the storedCREATEtext exactly, and a table renamed into place keeps a rewritten statement that no longer matches.Compatibility boundary: an older Maka cannot write a blob row back, because
assertSupportedOperationalSchemaVersionrefuses to open a workspace whose schema is newer than it supports.Refs #1679
Verification
Measured by copying this machine's real workspace and opening the copy with this branch. The original was never opened — read-only copy, migration run against the copy.
dbstat)record_jsonVACUUMEvery number the projection answers is identical across the migration: 383 rows, 7 distinct Sessions,
sum(inputTokens)55,186,834,sum(outputTokens)204,626, clampedsum(cacheReadInputTokens)412,160, priced-attempt count andsum(costUsd). Zero rows became tombstones on this workspace.The 18.5 ms → 0.17 ms line is the same all-time total taken both ways on the same data: the old path (read every
record_json,JSON.parse, fold in JS) against the newSUM. The gap is a factor of the row count, so it widens on a workspace with more history — that is the defect being removed, not a micro-optimization.Suites, each as
npm --workspace <name> run buildthennode --test --test-concurrency=4 "dist/**/*.test.js"in the package directory:@maka/core— 814 pass, 0 fail@maka/storage— 1122 pass, 8 skipped, 0 fail@maka/runtime— 3247 pass, 0 fail@maka/runtime-host— 1705 pass, 0 failnpm run format,npm run lint— cleanCoverage, one test per obligation:
packages/storage/src/__tests__/model-call-usage-query.test.ts): unpriced spend is never counted as zero and is reported in coverage instead; a genuinely free call reads apart from an unpriced one; usage-missing is reported separately from unpriced; a malformed cache reading cannot inflate the cache total; cache-only evidence survives without inventing an input total; the Session/window/provider/model/status filters;interruptedcounts as aborted, not as an error; buckets by provider and model; day buckets are named by the sameusageBucketKeyboth Usage sources derive; logs page newest-first and carry coverage for the whole match; a replayedattemptIdis one call.sqlite-usage-schema.test.ts): a wide blob and a narrow blob both convert; a corrupt row and a row with no pricing in it become tombstones that keep their Session; a second run is a no-op.Size
effort/XXLbyscripts/pr-effort.mjs— 2,889 readable lines, of which 1,522 are tests. It is one change and does not split: removing a representation and installing its replacement is a single unit, and the migration, the columns, the SQL reads and the callers that stop folding in JS have no merge constraint separating them — landing any subset leaves the tree with two ways to answer a Usage question.AI use
Select exactly one:
Tool(s) and scope: Claude Code — wrote the schema, migration, SQL reads and tests in this branch under review, and ran the measurements above.
Checklist
Does this PR entail a change in behavior?