diff --git a/planflow/archive/2026-08-20-current-context-size/design.md b/planflow/archive/2026-08-20-current-context-size/design.md new file mode 100644 index 0000000..36c4b34 --- /dev/null +++ b/planflow/archive/2026-08-20-current-context-size/design.md @@ -0,0 +1,110 @@ +# current-context-size Design + +## Summary + +Add a per-trace `current_context_size` metric to the Langfuse generation observation's `usageDetails`, reproducing the number shown in the OpenCode TUI "Context" panel (e.g. `69,406`). For OpenCode it is `input + output + reasoning + cache.read + cache.write` of the trace's last assistant message with `output > 0`; for Claude Code the same code path reduces to Anthropic's four usage fields. Implementation threads `reasoning_tokens` through the existing normalised `message.usage` seam (opencode.rs), adds a source-agnostic `get_context_size` accessor (transcript.rs), and appends one conditional key to the emitted `usageDetails` (emit.rs). Existing summed token reporting is untouched; the metric is omitted (never zero) when token data is absent; Pi traces are unaffected. + +## Definition of Done + +- [ ] Each Langfuse trace produced by code-trace from an OpenCode session carries a generation `usageDetails.current_context_size` equal to the value the opencode TUI "Context" panel would show at that point in the session. +- [ ] The same applies to Claude Code traces, using the Anthropic-usage equivalent formula. +- [ ] Existing `usageDetails` keys (`input`, `output`, `cache_creation_input_tokens`, `cache_read_input_tokens`) and their summed values are byte-for-byte unchanged. +- [ ] Traces with no usable token data produce no `current_context_size` key (no zeroes). +- [ ] Pi-agent traces are unaffected. +- [ ] Covered by unit tests (normaliser + emit) and integration tests for both OpenCode and Claude Code fixtures. + +## Architecture + +Both OpenCode and Claude Code sources converge on the normalised Claude-style `message.usage` block before `emit.rs` builds the Langfuse batch. The metric is computed **once, source-agnostically**, from that block. + +Approved approach: **A — carry `reasoning` as a `reasoning_tokens` key in the normalised `message.usage` Value**, so a single formula serves both sources. + +### Component changes + +1. **`src/opencode.rs`** — `extract_opencode_usage` (currently src/opencode.rs:206) additionally emits `"reasoning_tokens": get("reasoning")` in the returned usage Value. `get()` already defaults to 0 when the key is absent, so v1-format messages (`info.metadata.assistant.tokens`) need no handling. No other change. + +2. **`src/transcript.rs`** — new function alongside `get_usage` (src/transcript.rs:119): + + ```rust + pub fn get_context_size(msg: &Value) -> Option { + let u = msg.get("message")?.get("usage")?; + let get = |k: &str| u.get(k).and_then(Value::as_u64).unwrap_or(0); + Some(get("input_tokens") + get("output_tokens") + get("reasoning_tokens") + + get("cache_read_input_tokens") + get("cache_creation_input_tokens")) + } + ``` + + Returns `None` when no usage block exists — same "absent ≠ zero" contract as `get_usage`. `Usage` struct and `get_usage` are **untouched**. + +3. **`src/emit.rs`** — in `build_ingestion_batch`, computed next to `total_usage` (currently src/emit.rs:106): + + - scan `turn.assistant_msgs` in reverse for the first message with a usage block and `output_tokens > 0`, then `get_context_size` it. + - if found, add `"current_context_size"` to the existing `usageDetails` json! object (src/emit.rs:190) — only when `usageDetails` is already being emitted (i.e. `total_usage` is `Some`). + +### Data flow + +``` +OpenCode SDK message info.tokens {input, output, reasoning, cache{read,write}} + → extract_opencode_usage (adds reasoning_tokens) + → normalised message.usage + ┌→ get_usage (summed, existing 4 keys) — unchanged +Claude Code JSONL message.usage ────────┤ + └→ get_context_size (last msg with output>0) + → build_ingestion_batch → generation usageDetails { input, output, + cache_creation_input_tokens, cache_read_input_tokens, current_context_size } +``` + +No changes to `src/turns.rs`, `src/main.rs`, `src/pi_agent.rs`, or either TypeScript plugin. + +### Semantics & edge cases + +- **TUI parity (OpenCode):** per turn (one turn = one trace), `current_context_size` = `input + output + reasoning + cache.read + cache.write` from the turn's **last assistant message with `output > 0`** — exactly the OpenCode TUI sidebar formula (`packages/opencode/src/cli/cmd/tui/routes/session/sidebar.tsx`, the `context()` memo). In a multi-step tool-call turn this is the final step — the largest context of the turn and the number on screen at idle. +- **Claude Code:** same code path; the formula reduces to `input_tokens + output_tokens + cache_read_input_tokens + cache_creation_input_tokens` (`reasoning_tokens` absent → 0). Anthropic's API exposes no reasoning field; this is straight accounting of the final call's window. +- **`output_tokens > 0` guard:** skips degenerate/cancelled assistant steps, mirroring the TUI's `findLast(x => x.role === "assistant" && x.tokens.output > 0)`. Applied uniformly; every real API reply has output > 0, so for Claude Code it only filters pathological entries. If the last assistant message *lacks usage entirely*, the reverse scan falls back to an earlier qualifying message — matching the TUI's backward scan. +- **Absent ≠ zero:** when no qualifying assistant message exists, `current_context_size` is omitted from `usageDetails` — never emitted as `0`. When the turn has no usage at all, no `usageDetails` object is created (existing behaviour). Deliberate asymmetry: a turn whose only assistant steps have `output_tokens == 0` still gets summed `usageDetails` but no `current_context_size`; this mirrors the TUI and must carry a code comment. +- **Pi:** no usage block → `get_context_size` returns `None` → nothing emitted. `pi_agent.rs` untouched. +- **Privacy:** the metric is a token count only; no new content leaves the machine. + +## Existing Patterns followed + +- **Normalise-then-emit:** sources translate to the Claude-format `message.usage` Value; emit stays source-agnostic (same seam `extract_opencode_usage` already uses). +- **`get_usage` contract:** `Option`-style absence semantics; `?`-short-circuit on missing blocks; `unwrap_or(0)` per key. +- **Omit-don't-zero:** the existing comment at src/emit.rs:184–186 ("Omitted entirely (not zero-filled)… so Langfuse never prices a generation at $0") is the precedent for omitting `current_context_size` rather than emitting 0. +- Keeping the `Usage` struct fixed while adding a parallel accessor avoids touching its `Add` impl and the summed-pricing path. + +## Implementation Phases + +1. **Normaliser seam** — `opencode.rs` emits `reasoning_tokens`; unit test updates (`extracts_v2_tokens_into_usage` with non-zero `reasoning`, v1 path asserting `reasoning_tokens == 0`). +2. **Metric computation** — `transcript::get_context_size` + unit tests; `emit.rs` reverse-scan + conditional `current_context_size` in `usageDetails`; emit unit tests (last-not-summed regression, omitted-when-absent, omitted-when-zero-output, exact key set). +3. **Integration coverage** — extend `end_to_end_opencode_transcript` (multi-assistant turn, non-zero reasoning, assert TUI-formula value ≠ summed input), extend `end_to_end_simple_transcript` (Claude Code assertion), Pi fixture asserting key absence. `cargo test` green. + +## Additional Considerations + +- **v1 OpenCode messages:** `info.metadata.assistant.tokens` predates the `reasoning` field; `get()`'s `unwrap_or(0)` covers it. +- **usageDetails key set:** after the change, `usageDetails` contains exactly `input`, `output`, `cache_creation_input_tokens`, `cache_read_input_tokens`, and (when data exists) `current_context_size`. `reasoning_tokens` must **not** leak into emitted `usageDetails` — it exists only in the internal normalised Value. +- **Out of scope:** `tests/concurrency_test.rs` (Track 2 race suite), `harness/` (Track 1 container), any change to summed usage semantics, any front-end/Langfuse dashboard work, exposing `reasoning` as its own usageDetails key (possible later, not required by the DoD). + +## Acceptance Criteria + +- **current-context-size.AC1.1** *(success)* — OpenCode turn with assistant messages carrying v2 `tokens` including `reasoning`: the generation's `usageDetails.current_context_size` equals `input + output + reasoning + cache.read + cache.write` of the last assistant message with `output > 0`. +- **current-context-size.AC1.2** *(failure — regression guard)* — multi-assistant tool-loop turn: `current_context_size` is taken from the last qualifying step and **differs from** the summed per-step usage; it must not equal `total_usage`'s summed input. +- **current-context-size.AC1.3** *(failure — compat)* — OpenCode v1-format message without `reasoning`: `reasoning` contributes 0 and the formula still matches the TUI equivalent. +- **current-context-size.AC2.1** *(success)* — Claude Code trace: `current_context_size` equals `input_tokens + output_tokens + cache_read_input_tokens + cache_creation_input_tokens` of the last assistant message with `output_tokens > 0`. +- **current-context-size.AC2.2** *(failure — fallback)* — last assistant message missing a usage block entirely: metric falls back to an earlier assistant message with `output_tokens > 0`. +- **current-context-size.AC3.1** *(regression)* — the existing four `usageDetails` keys' values are byte-for-byte identical to before the change; all pre-existing `emit.rs` and integration tests pass unmodified. +- **current-context-size.AC3.2** *(failure — key hygiene)* — emitted `usageDetails` contains no `reasoning` / `reasoning_tokens` key. +- **current-context-size.AC4.1** *(failure — absent ≠ zero)* — turn with no usage data at all: no `usageDetails` object is created; no `current_context_size` key anywhere. +- **current-context-size.AC4.2** *(failure — guard)* — assistant usage present but `output_tokens == 0` on every step: `usageDetails` may exist but must not contain `current_context_size` (no zeroes). +- **current-context-size.AC5.1** *(regression)* — Pi traces carry no `current_context_size` key and existing Pi fixtures/tests pass unchanged. +- **current-context-size.AC6.1** *(verification)* — `cargo test` is green, including the new unit tests (`opencode.rs`, `transcript.rs`, `emit.rs`) and both extended integration tests. + +## Glossary + +- **`current_context_size`** — the new metric: token count of the context window at the end of a trace's final assistant step, per the formulas in Semantics. +- **turn** — one user message plus all following assistant messages (a tool-call loop counts as one turn); each turn produces one Langfuse trace with one generation observation. See `src/turns.rs`. +- **`message.usage`** — the normalised Claude-format usage block both sources converge on: `{input_tokens, output_tokens, cache_creation_input_tokens, cache_read_input_tokens}` (+ `reasoning_tokens` after this change). +- **normalisation seam** — the point where a source-specific message format (OpenCode SDK, Pi entries) is translated into the Claude-format Values consumed by `emit.rs`; implemented in `src/opencode.rs` / `src/pi_agent.rs`. +- **`usageDetails`** — Langfuse's numeric usage map on a generation observation (`input`, `output`, …). Accepts arbitrary extra keys such as `current_context_size`. +- **TUI context panel** — OpenCode's sidebar display ("Context: N tokens, X% used"), computed in `sidebar.tsx` as `input + output + reasoning + cache.read + cache.write` of the last assistant message with `output > 0`. +- **OpenCode v1 / v2 formats** — two SDK message shapes for token data: v1 nests under `info.metadata.assistant.tokens`; v2 uses `info.tokens` (with `reasoning`). Both are handled by `extract_opencode_usage`. +- **absent ≠ zero** — the codebase convention of omitting a metric entirely when its source data is missing, rather than emitting 0 (which Langfuse would treat as a real measurement). diff --git a/planflow/archive/2026-08-20-current-context-size/implementation.md b/planflow/archive/2026-08-20-current-context-size/implementation.md new file mode 100644 index 0000000..35bb080 --- /dev/null +++ b/planflow/archive/2026-08-20-current-context-size/implementation.md @@ -0,0 +1,352 @@ +# current-context-size — Implementation Plan + +Source design: `planflow/changes/current-context-size/design.md` +Slug: `current-context-size` + +All file paths and line numbers below were verified against the current code (HEAD) by a read-only scout pass on 2026-08-20. Discrepancies found: none material (see Verification Notes). Two plan-level gaps the scout surfaced are handled here as explicit new tests, not extensions of non-existent ones. + +Verification command (all tasks): `cargo test` +Expected: `test result: ok.` for every test binary, zero failures. + +--- + +## Phase 1 — Normaliser seam: carry `reasoning_tokens` + +### Task 1.1 — Emit `reasoning_tokens` from `extract_opencode_usage` + +**File:** `src/opencode.rs` (function `extract_opencode_usage`, lines 206–231) + +**Change:** In the `json!` block at lines 225–230, add one key after `cache_read_input_tokens`: + +```rust + Some(json!({ + "input_tokens": get("input"), + "output_tokens": get("output"), + "cache_creation_input_tokens": cache_write, + "cache_read_input_tokens": cache_read, + "reasoning_tokens": get("reasoning"), + })) +``` + +No other change. `get` (line 214) already returns `0` for absent keys via `unwrap_or(0)`, so v1 messages (`info.metadata.assistant.tokens`, which predate `reasoning`) automatically yield `reasoning_tokens: 0`. The v1/v2 `or_else` fallback at lines 208–213 is untouched. + +**Depends on:** nothing. +**Commit:** `feat(opencode): carry reasoning_tokens through normaliser` +**ACs covered:** contributes to AC1.1, AC1.3. +**Test:** Task 1.2. + +--- + +### Task 1.2 — Unit tests for `reasoning_tokens` extraction + +**File:** `src/opencode.rs` (`#[cfg(test)] mod tests`, starts line 233) + +**Change (a) — extend `extracts_v2_tokens_into_usage` (line 302):** add `"reasoning": 7` to the `tokens` block in the fixture (line 307) and add an assertion that the normalised `usage.reasoning_tokens == 7`. The existing four assertions (lines 312–315) stay unchanged. + +**Change (b) — new test `extracts_v1_tokens_without_reasoning` (after line 326):** a v1-format assistant message whose `info.metadata.assistant.tokens = { input: 5, output: 9, cache: { read: 1, write: 2 } }` (no `reasoning` key). Assert: `usage.reasoning_tokens == 0` and `usage.input_tokens == 5` (proves v1 path still works and reasoning defaults to 0 — satisfies AC1.3). This is a new test, not an extension — no existing v1 usage test exists (scout flag #1). + +**Verify:** +``` +cargo test opencode +``` +Expected: `extracts_v2_tokens_into_usage` and `extracts_v1_tokens_without_reasoning` both pass; `omits_usage_when_no_tokens_block` (line 319) still passes. + +**Depends on:** 1.1. +**Commit:** `test(opencode): reasoning_tokens extraction (v2 + v1)` +**ACs covered:** AC1.1, AC1.3. + +--- + +## Phase 2 — Source-agnostic metric: `get_context_size` + emit + +### Task 2.1 — Add `get_context_size` to `transcript.rs` + +**File:** `src/transcript.rs` (insert after `get_usage`, which ends at line 133; before `extract_text` at line 136) + +**Change:** new public function: + +```rust +/// Context-window size for one assistant message, as the OpenCode TUI +/// "Context" panel reports it: input + output + reasoning + cache read + +/// cache write. Returns None when the message has no usage block, matching +/// `get_usage`'s "absent ≠ zero" contract. +pub fn get_context_size(msg: &Value) -> Option { + let u = msg.get("message")?.get("usage")?; + let get = |k: &str| u.get(k).and_then(Value::as_u64).unwrap_or(0); + Some( + get("input_tokens") + + get("output_tokens") + + get("reasoning_tokens") + + get("cache_read_input_tokens") + + get("cache_creation_input_tokens"), + ) +} +``` + +`Usage` struct (line 94), its `Add` impl (lines 102–114), and `get_usage` (line 119) are **not modified**. + +**Depends on:** nothing (independent of Phase 1; `reasoning_tokens` defaults to 0 when absent). +**Commit:** `feat(transcript): get_context_size accessor` +**ACs covered:** contributes to AC1.1, AC2.1. +**Test:** Task 2.2. + +--- + +### Task 2.2 — Unit tests for `get_context_size` + +**File:** `src/transcript.rs` (`mod tests`, starts line 197) + +**Change — new tests (after `get_usage_absent_block_returns_none` at line 273):** + +- `get_context_size_sums_all_five_fields`: a message whose `usage` has `input_tokens: 10, output_tokens: 20, reasoning_tokens: 5, cache_read_input_tokens: 3, cache_creation_input_tokens: 2` → returns `Some(40)`. +- `get_context_size_missing_fields_default_to_zero`: usage with only `input_tokens: 7` → returns `Some(7)` (other four default 0; proves Claude Code's `reasoning_tokens`-absent case reduces to the four-field Anthropic formula). +- `get_context_size_absent_block_returns_none`: no `usage` key → returns `None` (mirrors the existing `get_usage_absent_block_returns_none` contract — AC4.1 at the accessor level). + +**Verify:** +``` +cargo test transcript +``` +Expected: all three new tests pass; existing `get_usage` tests (lines 248, 261, 273) still pass. + +**Depends on:** 2.1. +**Commit:** `test(transcript): get_context_size (sum, defaults, absent)` +**ACs covered:** AC1.1, AC2.1, AC4.1 (accessor). + +--- + +### Task 2.3 — Emit `current_context_size` in `build_ingestion_batch` + +**File:** `src/emit.rs` (`build_ingestion_batch`, lines 78–269) + +**Change (a) — compute the metric (insert after the `total_usage` fold, lines 106–114, before `trace_id` at line 116):** + +```rust + // Mirror the OpenCode TUI "Context" panel: the last assistant step of + // the turn with output > 0. Omitted (not zero) when no qualifying step + // exists, so Langfuse never records a synthetic 0 context size. + let context_size = turn + .assistant_msgs + .iter() + .rev() + .find(|m| { + matches!(transcript::get_usage(m), Some(u) if u.output_tokens > 0) + }) + .and_then(transcript::get_context_size); +``` + +`turn.assistant_msgs` is `Vec` in chronological order (`src/turns.rs:8`, verified — reverse yields last-first; duplicate streamed fragments are already merged by `build_turns`). + +**Change (b) — emit the key (modify the `usageDetails` block, lines 187–197):** + +Replace the `json!` macro with a mutable `serde_json::Map` so the extra key is added only when present: + +```rust + if let Some(usage) = total_usage { + let mut details = serde_json::Map::new(); + details.insert("input".to_string(), json!(usage.input_tokens)); + details.insert("output".to_string(), json!(usage.output_tokens)); + details.insert( + "cache_creation_input_tokens".to_string(), + json!(usage.cache_creation_input_tokens), + ); + details.insert( + "cache_read_input_tokens".to_string(), + json!(usage.cache_read_input_tokens), + ); + if let Some(cs) = context_size { + details.insert("current_context_size".to_string(), json!(cs)); + } + gen_body.insert("usageDetails".to_string(), Value::Object(details)); + } +``` + +The four existing keys keep identical values (AC3.1). `current_context_size` is added only inside the `if let Some(usage) = total_usage` block — so a turn with no usage at all still gets no `usageDetails` object (AC4.1), and a turn whose only steps have `output_tokens == 0` gets `usageDetails` without `current_context_size` (AC4.2). `reasoning_tokens` never appears in `usageDetails` (AC3.2). + +**Depends on:** 2.1 (for `get_context_size`). +**Commit:** `feat(emit): current_context_size in generation usageDetails` +**ACs covered:** AC1.1, AC1.2, AC2.1, AC2.2, AC3.1, AC3.2, AC4.1, AC4.2. +**Tests:** 2.4. + +--- + +### Task 2.4 — Unit tests for emit + +**File:** `src/emit.rs` (`#[cfg(test)] mod tests`) + +**Change (a) — extend `generation_event_carries_usage_details` (line 422):** after the existing per-key assertions (lines 444–448), add an exact-key-set assertion to catch leaks (scout flag #2, satisfies AC3.2): + +```rust + let keys: Vec<&str> = usage.as_object().unwrap().keys().map(|k| k.as_str()).collect(); + assert_eq!(keys, vec!["input", "output", "cache_creation_input_tokens", "cache_read_input_tokens"]); + // current_context_size: single assistant with output>0 → present, equals the 4-field sum. + // (This fixture has no reasoning_tokens, so the metric = 10+20+3+5 = 38.) + assert_eq!(usage["current_context_size"], 38); +``` + +Note: the existing fixture (lines 433–438) has no `reasoning_tokens`, so the expected value is `input + output + cache_read + cache_creation = 10+20+3+5 = 38` — this also proves the Claude-Code-style 4-field reduction (AC2.1). + +**Change (b) — new test `current_context_size_uses_last_assistant_not_sum` (after line 484):** a turn with two assistant messages, both with `output_tokens > 0`: +- msg A: `input_tokens: 10, output_tokens: 5, cache_creation: 0, cache_read: 0, reasoning_tokens: 0` +- msg B (later): `input_tokens: 40, output_tokens: 8, cache_creation: 2, cache_read: 1, reasoning_tokens: 3` + +Assert: +- `usageDetails.input == 50` (summed — AC3.1 unchanged behaviour) +- `usageDetails.current_context_size == 54` (40+8+1+2+3 from msg B, the **last** — NOT 50+13+... summed — this is the AC1.2 regression guard) +- key set includes `current_context_size` and the four standard keys, nothing else. + +**Change (c) — new test `current_context_size_omitted_when_no_output` (after (b)):** a turn with one assistant message whose usage has `output_tokens: 0` (and input 5). Assert `usageDetails` exists (has `input: 5`) but **has no `current_context_size` key** (AC4.2). Optionally assert key set is exactly the four standard keys. + +**Change (d) — new test `current_context_size_falls_back_to_earlier_step` (after (c)):** two assistant messages: msg A `output_tokens: 4, input 10`, msg B (later) **no usage block at all**. Assert `current_context_size` is present and equals msg A's 4-field value (AC2.2 fallback — reverse scan skips msg B, finds msg A). + +**Verify:** +``` +cargo test emit +``` +Expected: `generation_event_carries_usage_details` and `generation_event_sums_usage_across_assistant_messages` still pass (AC3.1 regression); four new assertions/tests pass. + +**Depends on:** 2.3. +**Commit:** `test(emit): current_context_size (last-not-sum, omitted, fallback, key set)` +**ACs covered:** AC1.2, AC2.1, AC2.2, AC3.1, AC3.2, AC4.2. + +--- + +## Phase 3 — Integration coverage for both sources + Pi regression + +### Task 3.1 — Extend OpenCode integration test + +**File:** `tests/integration_test.rs` (`end_to_end_opencode_transcript`, lines 70–113) + +**Change (a) — add `reasoning` to msg_2's tokens (line 77):** +```rust +"tokens": { "input": 12, "output": 34, "reasoning": 6, "cache": { "read": 0, "write": 0 } } +``` + +**Change (b) — add a second qualifying assistant step (msg_4) after msg_3, with larger input (simulating context growth after the tool result), output > 0, and reasoning. Insert after line 88 (before the closing `]` of `msgs` at line 89):** +```rust + json!({ + "info": { "id": "msg_4", "role": "assistant", "providerID": "anthropic", "modelID": "claude-sonnet-4-20250514", "tokens": { "input": 200, "output": 15, "reasoning": 9, "cache": { "read": 4, "write": 1 } } }, + "parts": [{ "type": "text", "text": "Done!" }] + }), +``` + +After normalisation, `turn.assistant_msgs` = [msg_2, msg_3(no usage), msg_4] (msg_3 is the tool_result carrier, no tokens). `build_turns` still yields 1 turn (the `turns.len() == 1` assertion at line 93 still holds — msg_4 is another assistant step in the same turn). + +**Change (c) — update/add assertions (after line 112):** +```rust + // Existing summed usage (unchanged): input 12+200=212, output 34+15=49. + assert_eq!(events[1]["body"]["usageDetails"]["input"], 212); + assert_eq!(events[1]["body"]["usageDetails"]["output"], 49); + // current_context_size = last step (msg_4) TUI formula: + // 200 + 15 + 9 + 4 + 1 = 229. Differs from summed input (212) → AC1.2. + assert_eq!(events[1]["body"]["usageDetails"]["current_context_size"], 229); + // AC3.2: no reasoning_tokens leaked into usageDetails. + assert!(events[1]["body"]["usageDetails"].as_object().unwrap().get("reasoning_tokens").is_none()); +``` + +Update the existing two assertions at lines 111–112 from `12`/`34` to `212`/`49` (the fixture now sums two assistant steps). + +**Verify:** +``` +cargo test --test integration_test end_to_end_opencode_transcript +``` +Expected: pass. + +**Depends on:** 2.3 (emit must emit the key), 1.1 (opencode must carry reasoning). +**Commit:** `test(integration): opencode current_context_size (multi-assistant, reasoning)` +**ACs covered:** AC1.1, AC1.2, AC3.2 (integration level). + +--- + +### Task 3.2 — Extend Claude Code integration test + +**File:** `tests/integration_test.rs` (`end_to_end_simple_transcript`, lines 6–37) + +**Change:** the existing assistant fixture (line 9) has usage `{input_tokens: 12, output_tokens: 34, cache_creation_input_tokens: 0, cache_read_input_tokens: 0}`. Add an assertion after line 36: + +```rust + // current_context_size = 12 + 34 + 0 + 0 = 46 (no reasoning_tokens in + // Claude Code transcripts → 4-field Anthropic formula). AC2.1. + assert_eq!(events[1]["body"]["usageDetails"]["current_context_size"], 46); +``` + +No fixture change needed — the existing single-assistant turn already has `output_tokens: 34 > 0` and is the last assistant message. + +**Verify:** +``` +cargo test --test integration_test end_to_end_simple_transcript +``` +Expected: pass. + +**Depends on:** 2.3. +**Commit:** `test(integration): claude code current_context_size` +**ACs covered:** AC2.1 (integration level). + +--- + +### Task 3.3 — Pi regression assertion + +**File:** `tests/integration_test.rs` (`end_to_end_pi_agent_transcript`, lines 116–157) + +**Change:** add an assertion that the Pi trace's generation event has **no** `current_context_size` key (and no `usageDetails` at all, since Pi carries no usage). After the existing metadata assertions (around line 157): + +```rust + // AC5.1: Pi traces carry no usage block → no usageDetails, no current_context_size. + assert!(events[1]["body"].get("usageDetails").is_none()); +``` + +(Pi's `normalize_pi_agent_messages` emits assistant messages without a `usage` key — verified at `src/pi_agent.rs:110–118` — so `total_usage` is `None`, no `usageDetails` object is created, and `current_context_size` cannot appear.) + +**Verify:** +``` +cargo test --test integration_test end_to_end_pi_agent_transcript +``` +Expected: pass. + +**Depends on:** 2.3. +**Commit:** `test(integration): pi trace has no current_context_size (regression)` +**ACs covered:** AC5.1. + +--- + +### Task 3.4 — Full test suite green + +**Command:** +``` +cargo test +``` + +**Expected:** all test binaries report `test result: ok.`; zero failures. This is the AC6.1 verification gate. No code change — this task is the verification step that closes the plan. + +**Depends on:** 3.1, 3.2, 3.3 (and transitively all of Phases 1–2). +**Commit:** none (verification only). If any test fails, fix in the responsible task before re-running. + +**ACs covered:** AC6.1. + +--- + +## AC → Task traceability + +| AC | Tasks | +|----|------| +| AC1.1 | 1.1, 1.2, 2.1, 2.3, 3.1 | +| AC1.2 | 2.3, 2.4(b), 3.1 | +| AC1.3 | 1.2(b) | +| AC2.1 | 2.1, 2.2, 2.3, 2.4(a), 3.2 | +| AC2.2 | 2.3, 2.4(d) | +| AC3.1 | 2.3, 2.4(a), 2.4(b), 3.1 | +| AC3.2 | 2.3, 2.4(a), 3.1 | +| AC4.1 | 2.2, 2.3 | +| AC4.2 | 2.3, 2.4(c) | +| AC5.1 | 3.3 | +| AC6.1 | 3.4 | + +Every AC maps to at least one task. Every functionality task (1.1, 2.1, 2.3) has a paired test task (1.2, 2.2, 2.4). No task depends on "this will exist somehow" — dependencies are explicit and ordered. + +## Verification Notes (from scout pass) + +- All design line references confirmed accurate: opencode.rs:206, transcript.rs:94/119, emit.rs:78/106/187–197, turns.rs:6–10. +- `extract_opencode_usage` returns `Option` (wrapped in `Some(json!(...)))` — design's "emits a key" wording matches. +- `turns.rs` collects `assistant_msgs` in first-seen (chronological) order; streamed duplicates are merged to their final content (lines 75–78) — reverse `.rev()` yields the true last step. +- `tests/support/fake_langfuse.rs` stores events as raw `serde_json::Value` (line 41, `Mutex>`) with no schema validation (lines 278–283) — extra `usageDetails` keys pass through and are queryable via pointer/index. No fake changes needed. +- `Cargo.toml` version 0.5.1 — unchanged by this plan. +- Scout flags handled: #1 (v1 test is new, Task 1.2b), #2 (exact-key assertion is new, Task 2.4a), #3 (opencode fixture extended with reasoning + second step, Task 3.1). Flags #4/#5 are cosmetic, no action. diff --git a/planflow/archive/2026-08-20-current-context-size/specs/emit/spec.md b/planflow/archive/2026-08-20-current-context-size/specs/emit/spec.md new file mode 100644 index 0000000..fdb8082 --- /dev/null +++ b/planflow/archive/2026-08-20-current-context-size/specs/emit/spec.md @@ -0,0 +1,61 @@ +# Delta for emit (trace emission & usage metrics) + +## ADDED Requirements + +### Requirement: Per-trace current context size metric +The system SHALL emit a `current_context_size` numeric key on each generation observation's `usageDetails` when token usage data is available for the trace's final assistant step. + +#### Scenario: OpenCode turn with reasoning-bearing usage +- GIVEN an OpenCode session turn whose last assistant message with `output > 0` carries `tokens` including `reasoning` +- WHEN code-trace builds the Langfuse ingestion batch for that turn +- THEN the generation observation's `usageDetails` SHALL contain `current_context_size` equal to `input + output + reasoning + cache.read + cache.write` of that last assistant message (matching the OpenCode TUI "Context" panel) + +#### Scenario: Claude Code turn +- GIVEN a Claude Code transcript turn whose last assistant message with `output_tokens > 0` carries Anthropic `message.usage` +- WHEN code-trace builds the ingestion batch +- THEN `usageDetails.current_context_size` SHALL equal `input_tokens + output_tokens + cache_read_input_tokens + cache_creation_input_tokens` of that last assistant message (`reasoning_tokens` absent → 0) + +#### Scenario: Multi-step tool-call turn uses the last step, not the sum +- GIVEN a turn with multiple assistant messages each carrying usage with `output > 0` +- WHEN the batch is built +- THEN `current_context_size` SHALL reflect only the last qualifying assistant message, and SHALL NOT equal the summed per-step usage + +#### Scenario: Last assistant step lacks usage, earlier step has it +- GIVEN a turn whose final assistant message has no `usage` block but an earlier assistant message has usage with `output > 0` +- WHEN the batch is built +- THEN `current_context_size` SHALL fall back to that earlier qualifying message's value + +## ADDED Requirements + +### Requirement: Omit metric when data is absent +The system SHALL NOT emit `current_context_size` when no qualifying assistant message exists, and SHALL NOT emit it as `0`. + +#### Scenario: Turn with no usage data at all +- GIVEN a turn whose assistant messages carry no `usage` blocks +- WHEN the batch is built +- THEN no `usageDetails` object SHALL be created and no `current_context_size` key SHALL appear anywhere + +#### Scenario: All assistant steps have zero output +- GIVEN a turn whose only assistant messages with usage have `output_tokens == 0` +- WHEN the batch is built +- THEN `usageDetails` MAY be present (from summed usage) but SHALL NOT contain `current_context_size` + +## MODIFIED Requirements + +### Requirement: Existing usage keys are unchanged +The system SHALL preserve the exact values of the existing `usageDetails` keys (`input`, `output`, `cache_creation_input_tokens`, `cache_read_input_tokens`) and their per-turn summed computation. `current_context_size` is additive only. + +#### Scenario: Regression — four existing keys +- GIVEN any turn that produced `usageDetails` before this change +- WHEN the batch is built after this change +- THEN the four existing keys SHALL have byte-for-byte identical values to before, and `reasoning_tokens` SHALL NOT appear as a `usageDetails` key + +## ADDED Requirements + +### Requirement: Pi traces unaffected +The system SHALL NOT emit `current_context_size` on traces produced from Pi agent sessions, since Pi normalised messages carry no `usage` block. + +#### Scenario: Pi session +- GIVEN a Pi agent session turn +- WHEN the batch is built +- THEN the generation observation SHALL have no `usageDetails` object and no `current_context_size` key diff --git a/planflow/archive/2026-08-20-current-context-size/work.md b/planflow/archive/2026-08-20-current-context-size/work.md new file mode 100644 index 0000000..a55a36d --- /dev/null +++ b/planflow/archive/2026-08-20-current-context-size/work.md @@ -0,0 +1,42 @@ +# current-context-size — Work Plan + +Source: `planflow/changes/current-context-size/implementation.md` + +## Summary + +Two parallel entry streams converge at the integration tests. All tasks are agent-owned — the work is mechanical, well-specified, and test-backed with `cargo test` as the gate. + +**Stream A — Normaliser (opencode.rs):** +1.1 → 1.2 — add `reasoning_tokens` to `extract_opencode_usage`, then unit-test it (v2 + v1). + +**Stream B — Metric (transcript.rs + emit.rs):** +2.1 → 2.2 → 2.3 → 2.4 — add `get_context_size`, test it, wire it into `build_ingestion_batch`, test the emit behaviour (last-not-sum, omitted, fallback, key hygiene). + +**Convergence — Integration tests:** +3.1 (needs 1.1 + 2.3), 3.2 (needs 2.3), 3.3 (needs 2.3) can run in parallel once their deps land. 3.4 runs `cargo test` as the final gate. + +**Checkpoint:** after 3.4 — human reviews the full suite result before merge. + +## Parallelism + +- 1.1 and 2.1 start immediately and independently. +- 1.2 starts when 1.1 lands; 2.2 starts when 2.1 lands. +- 2.3 starts when 2.1 lands (doesn't need 2.2). +- 2.4 starts when 2.3 lands. +- 3.1, 3.2, 3.3 start when 2.3 lands (3.1 also needs 1.1). +- 3.4 starts when 3.1, 3.2, 3.3 all land. + +## Task table + +| id | title | owner | depends_on | size | checkpoint | +|----|-------|-------|------------|------|------------| +| 1.1 | Emit reasoning_tokens from extract_opencode_usage | agent | | S | false | +| 1.2 | Unit tests for reasoning_tokens extraction (v2 + v1) | agent | 1.1 | S | false | +| 2.1 | Add get_context_size to transcript.rs | agent | | S | false | +| 2.2 | Unit tests for get_context_size | agent | 2.1 | S | false | +| 2.3 | Emit current_context_size in build_ingestion_batch | agent | 2.1 | M | false | +| 2.4 | Unit tests for emit (last-not-sum, omitted, fallback, key set) | agent | 2.3 | M | false | +| 3.1 | Extend OpenCode integration test (multi-assistant, reasoning) | agent | 1.1, 2.3 | S | false | +| 3.2 | Extend Claude Code integration test | agent | 2.3 | S | false | +| 3.3 | Pi regression assertion | agent | 2.3 | S | false | +| 3.4 | Full cargo test suite green | agent | 3.1, 3.2, 3.3 | S | true | diff --git a/planflow/specs/emit/spec.md b/planflow/specs/emit/spec.md new file mode 100644 index 0000000..7bbfd55 --- /dev/null +++ b/planflow/specs/emit/spec.md @@ -0,0 +1,59 @@ +# emit Specification + +## Purpose + +Defines the observable behaviour of trace emission to Langfuse: what usage and metric keys appear on each generation observation's `usageDetails`, how per-trace values are computed, and the omission (never-zero) rules when token data is absent. + +Living spec maintained by `planflow-archive`; do not hand-edit. Change history: `planflow/archive/2026-08-20-current-context-size/`. + +### Requirement: Per-trace current context size metric +The system SHALL emit a `current_context_size` numeric key on each generation observation's `usageDetails` when token usage data is available for the trace's final assistant step. + +#### Scenario: OpenCode turn with reasoning-bearing usage +- GIVEN an OpenCode session turn whose last assistant message with `output > 0` carries `tokens` including `reasoning` +- WHEN code-trace builds the Langfuse ingestion batch for that turn +- THEN the generation observation's `usageDetails` SHALL contain `current_context_size` equal to `input + output + reasoning + cache.read + cache.write` of that last assistant message (matching the OpenCode TUI "Context" panel) + +#### Scenario: Claude Code turn +- GIVEN a Claude Code transcript turn whose last assistant message with `output_tokens > 0` carries Anthropic `message.usage` +- WHEN code-trace builds the ingestion batch +- THEN `usageDetails.current_context_size` SHALL equal `input_tokens + output_tokens + cache_read_input_tokens + cache_creation_input_tokens` of that last assistant message (`reasoning_tokens` absent → 0) + +#### Scenario: Multi-step tool-call turn uses the last step, not the sum +- GIVEN a turn with multiple assistant messages each carrying usage with `output > 0` +- WHEN the batch is built +- THEN `current_context_size` SHALL reflect only the last qualifying assistant message, and SHALL NOT equal the summed per-step usage + +#### Scenario: Last assistant step lacks usage, earlier step has it +- GIVEN a turn whose final assistant message has no `usage` block but an earlier assistant message has usage with `output > 0` +- WHEN the batch is built +- THEN `current_context_size` SHALL fall back to that earlier qualifying message's value + +### Requirement: Omit metric when data is absent +The system SHALL NOT emit `current_context_size` when no qualifying assistant message exists, and SHALL NOT emit it as `0`. + +#### Scenario: Turn with no usage data at all +- GIVEN a turn whose assistant messages carry no `usage` blocks +- WHEN the batch is built +- THEN no `usageDetails` object SHALL be created and no `current_context_size` key SHALL appear anywhere + +#### Scenario: All assistant steps have zero output +- GIVEN a turn whose only assistant messages with usage have `output_tokens == 0` +- WHEN the batch is built +- THEN `usageDetails` MAY be present (from summed usage) but SHALL NOT contain `current_context_size` + +### Requirement: Existing usage keys are unchanged +The system SHALL preserve the exact values of the existing `usageDetails` keys (`input`, `output`, `cache_creation_input_tokens`, `cache_read_input_tokens`) and their per-turn summed computation. `current_context_size` is additive only. + +#### Scenario: Regression — four existing keys +- GIVEN any turn that produced `usageDetails` before this change +- WHEN the batch is built after this change +- THEN the four existing keys SHALL have byte-for-byte identical values to before, and `reasoning_tokens` SHALL NOT appear as a `usageDetails` key + +### Requirement: Pi traces unaffected +The system SHALL NOT emit `current_context_size` on traces produced from Pi agent sessions, since Pi normalised messages carry no `usage` block. + +#### Scenario: Pi session +- GIVEN a Pi agent session turn +- WHEN the batch is built +- THEN the generation observation SHALL have no `usageDetails` object and no `current_context_size` key diff --git a/src/emit.rs b/src/emit.rs index 503c43f..2f842b0 100644 --- a/src/emit.rs +++ b/src/emit.rs @@ -185,15 +185,41 @@ pub fn build_ingestion_batch( // all, so Langfuse never prices a generation at $0 for a source that // simply doesn't report usage. if let Some(usage) = total_usage { - gen_body.insert( - "usageDetails".to_string(), - json!({ - "input": usage.input_tokens, - "output": usage.output_tokens, - "cache_creation_input_tokens": usage.cache_creation_input_tokens, - "cache_read_input_tokens": usage.cache_read_input_tokens, - }), + let mut details = serde_json::Map::new(); + details.insert("input".to_string(), json!(usage.input_tokens)); + details.insert("output".to_string(), json!(usage.output_tokens)); + details.insert( + "cache_creation_input_tokens".to_string(), + json!(usage.cache_creation_input_tokens), ); + details.insert( + "cache_read_input_tokens".to_string(), + json!(usage.cache_read_input_tokens), + ); + // Mirror the OpenCode TUI "Context" panel: the last assistant step of + // the turn with output > 0. Omitted (not zero) when no qualifying step + // exists, so Langfuse never records a synthetic 0 context size. The + // scan only runs at all because the turn has usage (`total_usage` is + // Some); the guard itself reads one message and never synthesises 0. + let context_size = turn + .assistant_msgs + .iter() + .rev() + .find_map(|m| { + let usage = transcript::get_usage(m)?; + // The ? above keeps the "absent ≠ zero" contract intact: if + // get_context_size ever diverges from get_usage's guard, the + // metric disappears rather than synthesising a 0. + if usage.output_tokens > 0 { + transcript::get_context_size(m) + } else { + None + } + }); + if let Some(cs) = context_size { + details.insert("current_context_size".to_string(), json!(cs)); + } + gen_body.insert("usageDetails".to_string(), Value::Object(details)); } events.push(json!({ "id": uuid::Uuid::new_v4().to_string(), @@ -446,6 +472,12 @@ mod tests { assert_eq!(usage["output"], 20); assert_eq!(usage["cache_creation_input_tokens"], 5); assert_eq!(usage["cache_read_input_tokens"], 3); + let mut keys: Vec<&str> = usage.as_object().unwrap().keys().map(|k| k.as_str()).collect(); + keys.sort_unstable(); + assert_eq!(keys, vec!["cache_creation_input_tokens", "cache_read_input_tokens", "current_context_size", "input", "output"]); + // current_context_size = 4-field Anthropic sum (no reasoning_tokens in this + // fixture): input 10 + output 20 + cache_read 3 + cache_creation 5 = 38. + assert_eq!(usage["current_context_size"], 38); } #[test] @@ -483,6 +515,112 @@ mod tests { assert_eq!(usage["output"], 13); } + #[test] + fn current_context_size_uses_last_assistant_not_sum() { + use crate::source::Source; + let turn = Turn { + user_msg: json!({"type":"user","message":{"role":"user","content":"Do something"}}), + assistant_msgs: vec![ + json!({ + "type":"assistant", + "message":{ + "id":"m1", + "role":"assistant", + "model":"claude", + "content":[{"type":"tool_use","id":"tu_1","name":"Bash","input":{"command":"ls"}}], + "usage":{"input_tokens":10,"output_tokens":5,"cache_creation_input_tokens":0,"cache_read_input_tokens":0,"reasoning_tokens":0}, + } + }), + json!({ + "type":"assistant", + "message":{ + "id":"m2", + "role":"assistant", + "model":"claude", + "content":[{"type":"text","text":"Done"}], + "usage":{"input_tokens":40,"output_tokens":8,"cache_creation_input_tokens":2,"cache_read_input_tokens":1,"reasoning_tokens":3}, + } + }), + ], + tool_results_by_id: HashMap::new(), + }; + let events = build_ingestion_batch("sess1", 1, &turn, Path::new("/tmp/t.jsonl"), &["claude-code".to_string()], Source::ClaudeCode, None); + let usage = &events[1]["body"]["usageDetails"]; + // Input is summed across both assistant messages (AC3.1 unchanged). + assert_eq!(usage["input"], 50); + // current_context_size mirrors the OpenCode Context panel: the last + // assistant step's own context size (40+8+1+2+3), NOT the sum (AC1.2). + assert_eq!(usage["current_context_size"], 54); + // Exactly the standard keys — msg B's reasoning_tokens (3) must not leak + // into the emitted usageDetails. + let mut keys: Vec<&str> = usage.as_object().unwrap().keys().map(|k| k.as_str()).collect(); + keys.sort_unstable(); + assert_eq!(keys, vec!["cache_creation_input_tokens", "cache_read_input_tokens", "current_context_size", "input", "output"]); + } + + #[test] + fn current_context_size_omitted_when_no_output() { + use crate::source::Source; + let turn = Turn { + user_msg: json!({"type":"user","message":{"role":"user","content":"Hello"}}), + assistant_msgs: vec![json!({ + "type":"assistant", + "message":{ + "id":"m1", + "role":"assistant", + "model":"claude", + "content":[{"type":"text","text":"Done"}], + "usage":{"input_tokens":5,"output_tokens":0}, + } + })], + tool_results_by_id: HashMap::new(), + }; + let events = build_ingestion_batch("sess1", 1, &turn, Path::new("/tmp/t.jsonl"), &["claude-code".to_string()], Source::ClaudeCode, None); + let usage = &events[1]["body"]["usageDetails"]; + // usageDetails still exists (summed input) but carries no current_context_size (AC4.2). + assert_eq!(usage["input"], 5); + assert!(usage.get("current_context_size").is_none()); + // Exactly the four standard keys, nothing leaked. + let mut keys: Vec<&str> = usage.as_object().unwrap().keys().map(|k| k.as_str()).collect(); + keys.sort_unstable(); + assert_eq!(keys, vec!["cache_creation_input_tokens", "cache_read_input_tokens", "input", "output"]); + } + + #[test] + fn current_context_size_falls_back_to_earlier_step() { + use crate::source::Source; + let turn = Turn { + user_msg: json!({"type":"user","message":{"role":"user","content":"Do something"}}), + assistant_msgs: vec![ + json!({ + "type":"assistant", + "message":{ + "id":"m1", + "role":"assistant", + "model":"claude", + "content":[{"type":"tool_use","id":"tu_1","name":"Bash","input":{"command":"ls"}}], + "usage":{"input_tokens":10,"output_tokens":4,"cache_creation_input_tokens":0,"cache_read_input_tokens":0}, + } + }), + json!({ + "type":"assistant", + "message":{ + "id":"m2", + "role":"assistant", + "model":"claude", + "content":[{"type":"text","text":"Done"}], + } + }), + ], + tool_results_by_id: HashMap::new(), + }; + let events = build_ingestion_batch("sess1", 1, &turn, Path::new("/tmp/t.jsonl"), &["claude-code".to_string()], Source::ClaudeCode, None); + let usage = &events[1]["body"]["usageDetails"]; + // Reverse scan skips msg B (no usage block) and falls back to msg A + // (10+4+0+0 = 14), even though msg B is chronologically last (AC2.2). + assert_eq!(usage["current_context_size"], 14); + } + #[test] fn trace_event_carries_user_id_when_set() { use crate::source::Source; diff --git a/src/opencode.rs b/src/opencode.rs index 080ddf4..72a4809 100644 --- a/src/opencode.rs +++ b/src/opencode.rs @@ -227,6 +227,7 @@ fn extract_opencode_usage(info: &Value) -> Option { "output_tokens": get("output"), "cache_creation_input_tokens": cache_write, "cache_read_input_tokens": cache_read, + "reasoning_tokens": get("reasoning"), })) } @@ -303,7 +304,7 @@ mod tests { let msgs = vec![json!({ "info": { "id": "msg2", "role": "assistant", "modelID": "claude", - "tokens": { "input": 100, "output": 50, "cache": { "read": 10, "write": 20 } } + "tokens": { "input": 100, "output": 50, "reasoning": 7, "cache": { "read": 10, "write": 20 } } }, "parts": [{ "type": "text", "text": "hi" }] })]; @@ -313,6 +314,7 @@ mod tests { assert_eq!(usage["output_tokens"], 50); assert_eq!(usage["cache_read_input_tokens"], 10); assert_eq!(usage["cache_creation_input_tokens"], 20); + assert_eq!(usage["reasoning_tokens"], 7); } #[test] @@ -325,6 +327,24 @@ mod tests { assert!(normalized[0]["message"].get("usage").is_none()); } + #[test] + fn extracts_v1_tokens_without_reasoning() { + // v1 nests token usage under info.metadata.assistant.tokens and + // predates the `reasoning` key, so reasoning_tokens must default to 0. + let msgs = vec![json!({ + "info": { + "id": "msg2", "role": "assistant", + "metadata": { "assistant": { "modelID": "claude-3-5-sonnet", "tokens": { "input": 5, "output": 9, "cache": { "read": 1, "write": 2 } } } } + }, + "parts": [{ "type": "text", "text": "hi" }] + })]; + let normalized = normalize_opencode_messages(msgs); + let usage = &normalized[0]["message"]["usage"]; + assert_eq!(usage["reasoning_tokens"], 0); + assert_eq!(usage["input_tokens"], 5); + assert_eq!(usage["output_tokens"], 9); + } + #[test] fn pending_tool_results_attached_to_previous_assistant() { let msgs = vec![ diff --git a/src/transcript.rs b/src/transcript.rs index fc05463..bd7fd2b 100644 --- a/src/transcript.rs +++ b/src/transcript.rs @@ -132,6 +132,22 @@ pub fn get_usage(msg: &Value) -> Option { }) } +/// Context-window size for one assistant message, as the OpenCode TUI +/// "Context" panel reports it: input + output + reasoning + cache read + +/// cache write. Returns None when the message has no usage block, matching +/// `get_usage`'s "absent ≠ zero" contract. +pub fn get_context_size(msg: &Value) -> Option { + let u = msg.get("message")?.get("usage")?; + let get = |k: &str| u.get(k).and_then(Value::as_u64).unwrap_or(0); + Some( + get("input_tokens") + + get("output_tokens") + + get("reasoning_tokens") + + get("cache_read_input_tokens") + + get("cache_creation_input_tokens"), + ) +} + /// Extract plain text from content (string or array of text blocks). pub fn extract_text(content: Option<&Value>) -> String { let Some(c) = content else { @@ -275,6 +291,35 @@ mod tests { assert!(get_usage(&v).is_none()); } + #[test] + fn get_context_size_sums_all_five_fields() { + let v: Value = serde_json::json!({ + "message": {"role": "assistant", "usage": { + "input_tokens": 10, + "output_tokens": 20, + "reasoning_tokens": 5, + "cache_read_input_tokens": 3, + "cache_creation_input_tokens": 2 + }} + }); + assert_eq!(get_context_size(&v), Some(40)); + } + + #[test] + fn get_context_size_missing_fields_default_to_zero() { + let v: Value = serde_json::from_str( + r#"{"message":{"role":"assistant","usage":{"input_tokens":7}}}"#, + ) + .unwrap(); + assert_eq!(get_context_size(&v), Some(7)); + } + + #[test] + fn get_context_size_absent_block_returns_none() { + let v: Value = serde_json::from_str(r#"{"message":{"role":"assistant"}}"#).unwrap(); + assert!(get_context_size(&v).is_none()); + } + #[test] fn is_tool_result_detects_correctly() { let v: Value = serde_json::json!({ diff --git a/tests/integration_test.rs b/tests/integration_test.rs index 008970d..a3fcd49 100644 --- a/tests/integration_test.rs +++ b/tests/integration_test.rs @@ -34,6 +34,9 @@ fn end_to_end_simple_transcript() { assert_eq!(events[1]["body"]["model"], "claude-sonnet-4-20250514"); assert_eq!(events[1]["body"]["usageDetails"]["input"], 12); assert_eq!(events[1]["body"]["usageDetails"]["output"], 34); + // current_context_size = 12 + 34 + 0 + 0 = 46 (no reasoning_tokens in + // Claude Code transcripts → 4-field Anthropic formula). AC2.1. + assert_eq!(events[1]["body"]["usageDetails"]["current_context_size"], 46); } #[test] @@ -74,7 +77,7 @@ fn end_to_end_opencode_transcript() { "parts": [{ "type": "text", "text": "Hello" }] }), json!({ - "info": { "id": "msg_2", "role": "assistant", "providerID": "anthropic", "modelID": "claude-sonnet-4-20250514", "tokens": { "input": 12, "output": 34, "cache": { "read": 0, "write": 0 } } }, + "info": { "id": "msg_2", "role": "assistant", "providerID": "anthropic", "modelID": "claude-sonnet-4-20250514", "tokens": { "input": 12, "output": 34, "reasoning": 6, "cache": { "read": 0, "write": 0 } } }, "parts": [ { "type": "text", "text": "Hi there!" }, { "type": "tool_use", "id": "tu_1", "name": "Bash", "input": { "command": "ls" } } @@ -86,11 +89,25 @@ fn end_to_end_opencode_transcript() { { "type": "tool_result", "tool_use_id": "tu_1", "content": "file1.txt" } ] }), + json!({ + "info": { "id": "msg_4", "role": "assistant", "providerID": "anthropic", "modelID": "claude-sonnet-4-20250514", "tokens": { "input": 200, "output": 15, "reasoning": 9, "cache": { "read": 4, "write": 1 } } }, + "parts": [{ "type": "text", "text": "Done!" }] + }), ]; let normalized = code_trace::opencode::normalize_opencode_messages(msgs); let turns = code_trace::turns::build_turns(normalized); assert_eq!(turns.len(), 1); + // All three assistant steps survive as separate messages in order. msg_3 is + // the tool_result carrier (its content is folded into msg_2's + // tool_results, but it still emits its own empty assistant message with no + // usage). msg_4 is the final text-only step carrying its own usage. + let assistant_ids: Vec<&str> = turns[0] + .assistant_msgs + .iter() + .filter_map(|m| m["message"]["id"].as_str()) + .collect(); + assert_eq!(assistant_ids, vec!["msg_2", "msg_3", "msg_4"]); let tags = vec!["opencode".to_string()]; let events = code_trace::emit::build_ingestion_batch( @@ -108,8 +125,15 @@ fn end_to_end_opencode_transcript() { assert_eq!(events[0]["body"]["name"], "OpenCode - Turn 1"); assert_eq!(events[0]["body"]["metadata"]["source"], "opencode"); assert_eq!(events[1]["body"]["model"], "claude-sonnet-4-20250514"); - assert_eq!(events[1]["body"]["usageDetails"]["input"], 12); - assert_eq!(events[1]["body"]["usageDetails"]["output"], 34); + // Summed usage across the two assistant steps with usage (msg_2 + msg_4): + // input 12+200=212, output 34+15=49 (msg_3 carries no tokens → no usage). + assert_eq!(events[1]["body"]["usageDetails"]["input"], 212); + assert_eq!(events[1]["body"]["usageDetails"]["output"], 49); + // current_context_size = last step (msg_4) TUI formula: 200+15+9+4+1 = 229. + // Differs from summed input 212 → AC1.2 (last-not-sum). + assert_eq!(events[1]["body"]["usageDetails"]["current_context_size"], 229); + // AC3.2: no reasoning_tokens leaked into emitted usageDetails. + assert!(events[1]["body"]["usageDetails"].get("reasoning_tokens").is_none()); } #[test] @@ -155,4 +179,7 @@ fn end_to_end_pi_agent_transcript() { assert_eq!(events[0]["type"], "trace-create"); assert!(events[0]["body"]["name"].as_str().unwrap().starts_with("Pi Agent")); assert_eq!(events[0]["body"]["metadata"]["source"], "pi-agent"); + + // AC5.1: Pi traces carry no usage block → no usageDetails, no current_context_size. + assert!(events[1]["body"].get("usageDetails").is_none()); }