Compaction should measurably minify context while preserving the prompt cache

[tishachawla-jg]

Summary

context_compaction fires near the context ceiling and issues a MUTATE, but the live rewrite only removes exact-duplicate messages. It doesn't measurably shrink cost, and it isn't aware of the prompt cache — which is where the real money is. This issue makes compaction actually reduce spend: preserve the cache discount, minify stale context, react to cache collapse (not just size), and prove the saving.

Background: cost is fresh tokens, not prompt size

Providers bill cached prefix tokens far cheaper than fresh ones (Anthropic ~0.1x; OpenAI/Gemini automatic prefix caching). TokenOps's own price book already prices cached at roughly half the input rate (control/pricing.py), and after #140 the Usage buckets carry cached / reasoning disjointly. So the highest-value compaction move is not shrinking the prompt — it's not breaking the cache. A single volatile value ahead of the static prefix busts the whole discount.

Current behavior (what's there today)

• Detector (src/tokenops/control/policies/context_compaction.py): trips when est_input >= ctx_max, or earlier when input is rising across recent LLM steps past ctx_max/2. est_input is a chars/4 estimate; the trend reads usage.input + usage.cached (total context, post-fix: preserve cached and reasoning tokens without double billing #140). Size-based only.
• Rewrite (_compact_messages, src/tokenops/control/integration.py, called at the controls.call.compact branch in wrap_complete): pins system messages, drops duplicate non-system messages by (role, content). No summarization, no cache-prefix handling, no size measurement. feat(compaction): hoist system messages into a stable cache-friendly prefix #124 (in review) additionally hoists system into a stable leading prefix.

Proposal

Four moves, ordered by value and safety (each shippable on its own):

1. Cache-preserving rewrite (lossless, highest ROI)

Keep the static block (system, tool/schema definitions, constraints) as a byte-stable leading prefix across calls; keep volatile content (latest turn, timestamps, IDs) after it. #124 starts this by hoisting system; extend to keep the prefix stable call-to-call so the provider cache stays a hit.

• File: src/tokenops/control/integration.py (_compact_messages).

2. Cache-aware detection (a cache_guard-style arm)

Today the detector reacts to size. Add an arm that reacts to the cache discount collapsing: with disjoint counts from #140, watch the fresh-vs-cached ratio across recent LLM steps and trip when caching breaks, even if the prompt didn't grow. This is the cache_guard policy sketched in #66; it can live as a sibling detector or a second arm here.

• File: src/tokenops/control/policies/context_compaction.py (or a new policies/cache_guard.py).

3. Real minification of stale context (lossy, gated)

Beyond exact-duplicate dedup: fold or truncate old, large tool outputs (e.g. a 50KB fetched page down to the span that mattered) and summarize the middle band while pinning system/schema/constraints/live state and keeping the last K turns verbatim. Because summarization is lossy and can cost a call, gate it behind steps 1-2.

• File: _compact_messages (or a helper it calls).

4. Measure and never regress

• Record estimated tokens (and cost via the PriceFn) before and after each compaction into the ledger event, so the dashboard shows tokens/cost saved per run.
• Guard: never take a rewrite that busts a healthy cache or whose projected saving is below its own cost. A compaction that makes things worse is worse than none.
• Files: src/tokenops/control/integration.py, ledger event (src/tokenops/control/ledger.py).

Acceptance criteria

• With a stable system block, the cacheable prefix is identical across successive governed calls (a test asserts prefix stability).
• The detector trips on cache-discount collapse with a non-growing prompt (cache-aware arm), in addition to the existing size arm.
• A compaction event records tokens_before / tokens_after (and priced delta) readable by the dashboard.
• A guard prevents a rewrite whose projected saving <= 0.
• Never HALTs; degrades to telemetry when no assembly hook (compaction_supported), per Design smell: context_compaction's has_hook is host capability in a governance record, and ships inert by default #138/fix: derive context_compaction capability from controls, remove has_hook config #139.
• make lint and make test green.

Files

What	Where
Detector (size arm + new cache-aware arm)	src/tokenops/control/policies/context_compaction.py
Rewrite (cache-preserving + minify + measure)	src/tokenops/control/integration.py
Ledger event (before/after tokens)	src/tokenops/control/ledger.py
Tests	tests/test_context_compaction.py, integration wrap tests

Out of scope

• Cache-write premium accounting (a separate follow-up, per fix: preserve cached and reasoning tokens without double billing #140).
• Streaming compaction (wrap_stream) — Multimodal stream governance: implement wrap_stream and time-windowed accrual (spec) #110's territory.
• Provider-specific cache-breakpoint APIs; keep the strategy provider-agnostic behind the adapter for now.

Related

#124 (cache-prefix reorder), #140 (disjoint cached/reasoning tokens), #66 (cache_guard), #110 (streaming), #138/#139 (compaction capability derivation).

[RjyavardhanSingh]

on it