feat(coding-agent): cap inline transport images at 24 MiB#303
Open
amsminn wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #302
Problem
Image base64 accumulates in the session and the full history is replayed into every request. Once the HTTP body exceeds a provider's size limit, the request is rejected with 413 and the session wedges permanently (real incident: 63 images = 101 MiB → endless Kimi nginx 413s). Token-axis compaction never intervenes because it doesn't look at body bytes (last usage was only 34% of the window). Details: #302.
Change
Cap the total inline image base64 at 24 MiB in the transport conversion layer. Anything over the budget is replaced, oldest first, with a re-read hint placeholder. Stateless and transport-only — persisted session storage is untouched.
packages/coding-agent/src/core/messages.ts—TRANSPORT_IMAGE_BUDGET_BYTES = 24 MiB,IMAGE_ELISION_PLACEHOLDER,elideOldImages()(newest→oldest hard recency cutoff; the newest block is always kept regardless of size but consumes the budget, so an oversized newest block elides everything older → the total stays bounded; returns the same reference when nothing is elided),convertToLlmForTransport()(moves the existing sdk.ts blockImages closure verbatim + chains elision)packages/coding-agent/src/core/sdk.ts— closure replaced with a delegation to the function above (constant budget — no mutable state)packages/coding-agent/test/suite/harness.ts— the harness Agent uses the same conversion, with a small-budget injection option (transportImageBudget) for testspackages/coding-agent/src/core/changes.md— fork-change recordUntouched:
packages/ai,agent-session.ts, compaction, session storage format.Why 24 MiB
The only documented provider wall is the Anthropic Messages API's 32 MB request limit (413
request_too_large). 24 MiB of images + up to ~4 MiB of text (1M tokens) + overhead ≈ 29 MB < 32 MB. Since the cap sits below the smallest known wall, a single budget is safe on every provider, including mid-session provider fallback. (Kimi's ~100 MiB is an unverified observation and was excluded as a design input.) At the average FHD screenshot size (~1.7 MiB) the cap retains the most recent 12–15 screenshots — plenty for computer-use continuity. Previously wedged sessions pass from the very first request after reopening.Why there is no retry/recovery layer
elideOldImagestakes budget/keep as options, so a recovery layer can be added in a follow-up PR if it ever proves necessary.Trade-offs (deliberate)
Tests / QA
test/image-elision.test.ts): same-reference no-op, cutoff boundaries, keep-consumes-budget, sibling text preserved, consecutive placeholder dedupe, later-is-newer, blockImages paritytest/transport-image-budget.test.ts): with a faux provider, the very first request is elided / under-budget context untouchedblock-images.test.ts,compaction.test.tspassnpm run checkgreenlocal-ignore/qa-evidence/20260723-transport-image-budget/)Out of scope (follow-ups)
413 request-size classification (
overflow.ts— upstream contribution candidate) · speculative//btw/emergency paths ·read_videoingestion cap · budget configurability