feat(context): compact and continue instead of dropping history

[boeschj]

Summary

Fixes the bug behind "it kept saying it had no recollection of the session": a long conversation was silently truncated to fit the context window, so the model lost history nobody told it about.

• window.completeMessages keeps every message since the newest checkpoint or returns error.ContextTooLarge. It never drops silently.
• The loop compacts once and resumes the turn on overflow instead of refusing it. The result is visible in the transcript as a compaction card, so context loss is never invisible.
• recovered_from_overflow bounds recovery to one attempt per submit, and resets on each new user message.
• preserve_context keeps unsummarised messages past the in-memory display cap, so the display cap can no longer cost the model real context.
• requestBudget folds system, tools and instruction cost into the budget, and reserves less output room while summarising.

Extracted from #23, which is being closed. The Codex provider is not part of this: all provider_state plumbing and the cached-token field were left behind.

Two consequences worth calling out, both caused by this change rather than carried over:

• Ollama's shrink-on-overflow retry is removed. It halved the budget and retried, which can only make a complete-or-refuse request more likely to be refused. It was unreachable, not load bearing.
• systemText keeps its dropped argument and its note. Nothing passes a non-zero count now that requests are complete-or-refuse, so it cannot fire today, but deleting it was a behaviour change this PR did not need to make. Restored in 693be74.

Also narrows four window.zig helpers with no callers outside that file from pub fn to fn.

Test plan

• zig build test (515 passed, 5 skipped, up from 506 on master)
• zig build clean
• zig fmt --check clean
• Both commits build independently, so git bisect works across this branch
• Run a conversation past the model's context window and confirm it compacts and continues rather than stalling
• Confirm the compaction card appears in the transcript

Deviations from the ticket text

No ticket. Scope is the context and compaction work from #23, minus anything Codex specific.

Out of scope (flagged for follow-up)

• loop.zig estimates tokens at a hardcoded four bytes per token. Worth a named constant.
• An unknown model still falls back to a flat 32K budget (window.zig, pre-existing). A user on a model synth cannot resolve metadata for gets compacted every ~24K tokens with no explanation of why.
• max_transcript_messages in tui/model.zig documents an invariant nothing enforces now that compaction is token driven.
• Cached prompt token reporting was left out deliberately. Extracting it without the Codex provider would add a field that is always zero, since no remaining provider populates it. It needs openai.zig to read usage.prompt_tokens_details.cached_tokens, which is new work rather than extraction.

🤖 Generated with Claude Code

[pullfrog]

Important

The keep-all-or-refuse contract works, but recovery from a server-rejected overflow is gated by the same local estimate that caused it — worth addressing before merge.

Reviewed changes — full diff of both commits against master, plus the surrounding loop/provider code each path touches.

• window.zig keep-all-or-refuse — new completeMessages returns the entire active context (floored at the newest checkpoint) or error.ContextTooLarge; requestBudget folds system, tools, and instruction cost into one budget with a 90% reserve while summarising; contextTokens estimates the active transcript. Four now-file-private helpers lost pub with no remaining external callers (verified by grep).
• Loop compacts and resumes — ask() now checks budget, then auto-compact threshold, then fit, recovering from one overflow per submit via startCompaction(true) (recovered_from_overflow, reset per submit); turn("") sends a continuation instruction when the last message is the checkpoint; stop() and the cancel path now append settled tool results so the wire history stays complete on halted turns.
• Providers — ollama's halve-and-retry overflow branch is removed (waitAndRetry now only retries transient failures); both backends serialize through requestBudget/completeMessages and systemText loses its never-fired dropped-messages note; provider.explain special-cases ContextTooLarge.
• Persistence/display — resumeSession loads @max(limit, contextMessageCount) so the unsummarised context always loads; Conversation.preserve_context stops trim from dropping anything back to the newest checkpoint; usage.context_tokens is now prompt+completion.
• Tests — new coverage for overflow recovery, compaction-failure, resume-past-the-page, and complete tool exchanges after cancel; existing loop tests tightened (every call answered exactly once, turns continue after a halt).

⚠️ A server-rejected request has no recovery path when the local estimate was low

The refusal contract has two overflow surfaces: the pre-send estimate and the provider's own rejection. Both recoveries are gated by the same 4-bytes-per-token estimate — ask()'s recovery requires contextFits to fail, and manual /compact is checked by contextFits too — and a server rejection never updates usage.context_tokens, so the loop cannot learn the true size. On content the estimator undercounts (CJK text, per-message tool-JSON overhead) the first request of a session — or the first after compaction, where finishCompaction zeroes usage.context_tokens — can be rejected by the server, fail the turn, and then the suggested remedy /compact refuses with the same estimate: the two advice messages (ollama's "Try /compact" and the loop's "too large to compact safely … switch to a model with a larger context window") point at each other with no exit. The removed ollama halve-and-retry was the only path that reacted to server feedback rather than the estimate.

Technical details

# Overflow recovery is gated by the same estimate that failed

## Affected sites
- src/agent/loop.zig:923-932 (`ask`) — overflow recovery only triggers when the local `contextFits` estimate says the request does not fit; a server-side rejection bypasses this path entirely.
- src/agent/loop.zig:722-730 (`contextFits`) — same estimate gates manual `/compact` via `startCompaction`, so the remedy the failure message suggests fails for the same reason.
- src/agent/loop.zig:734-745 (`finishCompaction`) — zeroes `usage.context_tokens`, so the first request after compaction is estimate-only again.
- src/agent/loop.zig:1168-1181 (`pollRequest` failure path) — a rejected request never records provider-reported usage, so nothing corrects `usage.context_tokens` after a rejection.

## Required outcome
- A server-rejected overflow must lead to either a forced compaction attempt or a corrected size estimate, not a dead end where the error advice and the compaction gate reference each other.

## Suggested approach (optional)
- Update `usage.context_tokens` from provider-reported usage on the failure path when the rejection carries it, and/or let an overflow-classified provider error force `startCompaction(true)` once, mirroring the `recovered_from_overflow` bound.

## Open questions for the human (optional)
- Is "switch to a larger-context model" the intended terminal UX when the estimate is wrong, or should the server's own token count drive compaction?

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using openai-compatible/glm-5.3-flash:cloud ｜ 𝕏}

[pullfrog]

ℹ️ No new issues in the delta — but prior review feedback is unchanged, so this is not an approval.

Reviewed changes — the incremental delta since the prior pullfrog review (head 1673d6c), one commit:

• Restored systemText's dropped-messages note — the dropped: usize parameter and the <note> block that 1673d6c removed are back, byte-identical to master's version (verified against 271777c), including the "the dropped-messages note still gets through" test. Both production call sites (src/provider/ollama.zig:897, src/provider/openai.zig:833) now pass an explicit 0, so the note cannot fire while requests are complete-or-refuse — a deliberate scope decision documented in the PR body.

The delta itself is clean. What keeps the PR short of mergeable is prior, still-open feedback: the ⚠️ server-rejected-overflow recovery gap from the prior review, and the open inline thread on the stale shrink-retry doc comment at src/provider/ollama.zig:532-534.

  ^{｜ Fix it ➔ ｜ View workflow run ｜ Using openai-compatible/glm-5.3-flash:cloud ｜ 𝕏}

[pullfrog]

ℹ️ Minor suggestions only — one recall gap in the new overflow classifier, inline; everything else in the delta checks out.

Reviewed changes — the incremental delta since the prior pullfrog review (head 71d9bf8), one commit:

• Server-rejected overflow now recovers — pollRequest answers a provider error.ContextTooLarge with one compaction attempt per submit (recovered_from_overflow, gated on auto_compact_at), mirroring the bound the predicted-overflow path in ask() already had; a compaction the backend also refuses falls through and reports instead of trying again.
• Overflow classification centralized — the needle list ollama previously kept locally for advice text moved into Provider.mentionsOverflow (provider.zig); openai now maps a matching non-success body to error.ContextTooLarge (previously error.HttpError) and ollama delegates through its isOverflow wrapper.
• Request.start snapshots turn.compacting — the worker's copy of the turn keeps the summarisation flag.
• Two new tests pin the recovery (refuse once → compact → carry on) and its one-attempt bound (a compaction the backend also refuses reports after two calls, not three).

The prior ⚠️ (no recovery path for a server-rejected overflow when the local estimate was low) is resolved by the first bullet, and the stale ollama doc comment was closed by 71d9bf8; what remains is minor. CI passes on the head commit.

  ^{｜ Fix all ➔ ｜ Fix 👍s ➔ ｜ View workflow run ｜ Using openai-compatible/glm-5.3-flash:cloud ｜ 𝕏}

[pullfrog]

Verified recall gap in the new classifier: TGI's OpenAI-compatible route refuses genuine overflows with Input validation error: \inputs` must have less than 2048 tokens. Given: 4194— zero needle matches — so it degrades toerror.HttpErrorand the new recovery never fires for TGI users (huggingface/text-generation-inference issues #937, #1519, #1572; the last from thechat_completionsroute). Fail-safe rather than destructive, so this is recall only. Adding"must have less than"closes it and cannot collide with quota bodies; keep"exceeded"` out — OpenAI's 403 quota body ("You exceeded your current quota") would match it.

Suggested change

	const needles: []const []const u8 = &.{
	"context length", "context window", "too long", "too large",
	"too many token", "exceeds", "input length", "token limit",
	"maximum context",
	};
	const needles: []const []const u8 = &.{
	"context length", "context window", "too long", "too large",
	"too many token", "exceeds", "input length", "token limit",
	"maximum context", "must have less than",
	};