Skip to content

fix(inspector): expose usage and context evidence - #122

Merged
phodal merged 3 commits into
mainfrom
fix/harness-inspector-usage-context
Aug 28, 2026
Merged

fix(inspector): expose usage and context evidence#122
phodal merged 3 commits into
mainfrom
fix/harness-inspector-usage-context

Conversation

@phodal

@phodal phodal commented Aug 28, 2026

Copy link
Copy Markdown
Member

Summary

  • Add one provider-aware Usage and context disclosure to standalone Harness Inspector and Studio Session Detail.
  • Normalize Codex cumulative token snapshots, preserve Claude cache-creation tokens, and attach matching Cursor Context Usage Canvas evidence.
  • Keep Codex developer/system/reasoning text and raw context items out of dialogue and portable reports.

Why

  • Issue/Story: None; justified maintenance from a live Harness Inspector evidence audit.
  • User or maintainer outcome: Inspector can answer which token and bounded context evidence was observed for Codex, Claude, and Cursor without inventing billing totals or exporting prompt text.

Traceability and Scope

  • Spec/ADR, if applicable: docs/specs/2026-08-28-harness-inspector-usage-context.md
  • Acceptance criteria addressed: AC-1 through AC-7.
  • Canonical owners changed: Codex, Claude, and Cursor session adapters; shared session summary/report projection; standalone Inspector and Studio Session Detail.
  • Explicit non-goals: raw prompt/context export, billing or account claims, a stable public host-schema contract, or a universal provider transcript IR.

Change Type

  • Feature
  • Bug fix
  • Tests only
  • Documentation/community
  • Refactor with no intended behavior change
  • Dependency, packaging, or infrastructure

Test and Review Evidence

Check Result
npm run check on Node 24.19.0 Passed: root 1548 passed, 2 skipped; Harness 172; Harness UI 31; Studio 293; generated sources and package/archive verification passed
npx vitest run test/sessions/session-analysis-providers.test.mjs test/sessions/session-analysis.test.mjs test/sessions/commit-session-link.test.mjs Passed: 143 tests
npx vitest run test/reporting/harness-inspector.test.mjs Passed: 29 tests
npm run test:browser -w @qoder-ai/harness-studio -- --grep "opens a project workspace and compares Inspector-discovered Sessions" Passed: 1 Playwright scenario
node scripts/harness-inspector/visual-contract-check.mjs Passed: 15 surface and viewport states

Manual or visual evidence:

  • Studio usage/context detail checked at 1440x900, 1024x768, and 390x844 with no console/page errors or document-level horizontal overflow.
  • Standalone Inspector checked across wide, compact, and narrow layouts with no clipped facts, below-floor targets, page errors, or horizontal overflow.
  • A live local render found 58 sessions and 7,989 tool calls; usage was observed for 44 of 45 Codex sessions and all 13 Claude sessions. This workspace had no matching Cursor Session, so Cursor context matching is fixture-backed.

Risk and Recovery

  • Compatibility and cross-platform impact: filesystem work uses node:path and temporary-directory fixtures; Windows and Linux remain authoritative through PR CI.
  • Package, plugin, schema, or generated-file impact: additive report/session fields only; no generated files or dependency changes.
  • Rollback or recovery path: revert commit 13d5f68.
  • Residual risk or unverified boundary: local provider files are private implementation formats and may drift; Cursor live-session matching was not available in the current workspace.

AI Involvement

  • Level: Generated by Codex, with explicit Co-authored-by metadata.
  • Human review and validation: Human review is pending in this PR; automated, browser, visual, and live-host evidence is recorded above.

Checklist

  • I followed AGENTS.md, CONTRIBUTING.md, and the relevant canonical-owner guidance.
  • The change is focused and does not include unrelated local or generated state.
  • Tests and documentation match the behavior actually delivered.
  • Markdown links were checked when documentation moved or changed.
  • Cross-platform behavior was considered for Windows, macOS, and Linux.
  • Package/runtime verification was run when shipped files or dependencies changed.
  • User-facing or compatibility changes are recorded in CHANGELOG.md. Intentionally unchanged because repository instructions prohibit proactive release-metadata edits without explicit authorization.
  • I have the right to contribute this work under the repository MIT License. Maintainer confirmation requested.

phodal and others added 2 commits August 28, 2026 09:24
Normalize Codex cumulative token snapshots and keep developer/system context out of dialogue. Preserve Claude cache creation and Cursor context/source-time evidence, then expose a provider-aware usage panel in Inspector and Studio.

This justified maintenance follows docs/specs/2026-08-28-harness-inspector-usage-context.md and was validated with npm run check on Node 24, focused Playwright, and the Inspector visual contract.

Co-authored-by: Codex (GPT 5.6 Sol) <codex@openai.com>
Preserve provider-observed inference usage in Turn order and render token and context-window occupancy in standalone Inspector and Studio. Missing per-response context stays explicitly unavailable instead of inheriting a session snapshot.

Implements AC-8 and AC-9 in docs/specs/2026-08-28-harness-inspector-usage-context.md. Validated with the Node 24 npm run check gate, focused Playwright coverage, and the 15-surface visual contract.

Co-authored-by: Codex (GPT 5.6 Sol) <codex@openai.com>
@phodal

phodal commented Aug 28, 2026

Copy link
Copy Markdown
Member Author

Follow-up: per-response usage and context-window evidence

Each retained model inference now appears in Turn order as Model response N:

  • provider-observed token breakdown for that response
  • same-event context used / window / percent full when available
  • explicit not observed for this response when the provider did not emit a window

Live local evidence: Codex projected 5,341 rows across 44/45 sessions, all with same-event context-window evidence. Claude projected 981 rows across 13/13 sessions, with no context-window field available. Cursor historical rows remain evidence-gated; the current Context Usage Canvas is not copied onto older responses.

wide-session-usage

Count every matched workspace transcript as relevant when no time filter was requested, even when a transcript lacks an observable native timestamp. This keeps coverage denominators stable across Windows and POSIX filesystems.

Implements AC-10 in docs/specs/2026-08-28-harness-inspector-usage-context.md. Validated with the 53-test Cursor/provider suite and the full Node 24 npm run check gate.

Co-authored-by: Codex (GPT 5.6 Sol) <codex@openai.com>
@phodal
phodal merged commit c84e620 into main Aug 28, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant