Skip to content

feat(desktop): composer context gauge should update per settled provider request, not just at turn end #4717

Description

@me2seeks

Problem

The desktop composer's context gauge (ContextUsageIndicator, next to the model picker in the composer) only updates once per turn, when the turn fully ends:

  • It is fed by selectLatestRequestUsage (apps/desktop/src/renderer/chat-composer-region.tsx, ~L157), which scans the transcript for the newest token_usage message's lastRequestAnchor and refuses anything whose anchor names another route.
  • That message is appended once per send, in the Final usage event block after the agent loop breaks (packages/runtime/src/ai-sdk-backend.ts, ~L3086): one record spans every Runtime loop step and retry, and its anchor carries only the LAST step's input/output.
  • So during a long agentic turn (dozens of tool steps, minutes — exactly when context grows fastest and the user most needs the signal to /compact or wrap up), the gauge sits stale at the previous turn's end value.

Meanwhile the session inspector's context bar in the same window updates per settled provider request. Two indicators, one session, different numbers mid-turn — and the one sitting next to the model picker is the stale one.

Prior art: the inspector already does better

  • The Host seals the latest_context projection in the same storage transaction that commits a completed request's canonical attempt (packages/runtime/src/context-diagnostics.ts) — i.e., per settled provider request, not per turn.
  • The inspector subscribes to the live session event stream and re-reads context.diagnostics.query on a 400 ms-debounced trigger (apps/desktop/src/renderer/features/workbar/tools/inspector/use-session-trace.ts, session-trace-refresh.ts), only while visible.

Proposal: feed the composer gauge from the same snapshot

Verified feasible with no protocol changes:

  1. context.diagnostics.query is a plain read (packages/runtime-host/src/server/context-coordinator.ts #queryDiagnostics): header snapshot + run-store read, no execution authority, no busy gate. It is already bridged to the renderer as inspector.context(sessionId) (apps/desktop/src/preload/preload.ts, ~L3179).
  2. The renderer already streams the active session's live events for the transcript; hook a 400 ms-debounced pull on the trace-relevant event set at the app-shell/composer level and feed inputTokens to the gauge as "used".
  3. Keep the gauge's window semantics: declared context window for the active route first, then the snapshot's contextWindow (frozen at call time) / model metadata as fallback.
  4. Preserve the existing route-guard semantics: today the gauge refuses a count whose anchor names another model/connection rather than showing model A's tokens against model B's window (selectLatestRequestUsage's three refusals). The snapshot carries providerId/modelId but no connectionId, so the guard needs an equivalent decision — match the snapshot's route against the composer's active route, and fall back to the per-turn anchor (or ) when they diverge.
  5. Same protections as the inspector: revision counter against out-of-order reads, keep-last-value on query failure, discard when the session changed mid-flight.

The end-of-turn token_usage stays the authoritative persisted record; the pull only enriches the live turn. Both indicators read the same snapshot row, so they cannot disagree.

Why pull over a new push event

Same reasoning as the TUI sibling #4545:

  • Zero protocol surface: no core event schema, backend emission, host mapper, or persistence-semantics changes.
  • Single source of truth: composer gauge and inspector read the same latest-context snapshot row — no second derivation path to drift on resume/backfill/compact edges.
  • All changes contained in the desktop renderer (plus tests).

Scope

  • Composer gauge data path: per-settled-request refresh via debounced context.diagnostics.query, keyed to the active session's live event stream.
  • Route-guard decision for snapshot-fed values, with the current anchor-based path as fallback.
  • Tests: mid-turn refresh, debounce coalescing, out-of-order discard, failure keeps last value, session switch discards, turn-end reconciliation, cross-route refusal.

Non-goals

Refs: #4545 (TUI sibling), desktop pull model #1625 #2323, per-turn behavior #4019.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

enhancementNew feature or request

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions