Skip to content

feat: add Agent Lens for subagent traces - #12

Merged
cosmtrek merged 27 commits into
cosmtrek:masterfrom
yearth:feat/agent-lens
Jul 17, 2026
Merged

feat: add Agent Lens for subagent traces#12
cosmtrek merged 27 commits into
cosmtrek:masterfrom
yearth:feat/agent-lens

Conversation

@yearth

@yearth yearth commented Jul 15, 2026

Copy link
Copy Markdown
Contributor

Why

Mindwalk currently replays one normalized session trace at a time. When a root Codex or Claude session launches subagents, their work is stored in separate session artifacts, so the root view cannot answer a basic question: where did each subagent spend its attention?

This PR adds an Agent Lens V1. It keeps Main and every child as independent traces and lets the user choose one observation lens at a time. It deliberately does not merge actors into a synthetic timeline or infer lifecycle state that the source logs do not provide.

User flow

  1. Open a root session as usual.
  2. Open the new Agents sheet in the right-side dock.
  3. See Main plus each discovered child with truthful availability and link-quality information.
  4. Select an available child to switch the map, HUD, and timeline to that child trace.
  5. Switch back to Main without losing the remembered playhead for either actor.

Missing, failed, derived, and zero-event children remain visible as explicit states instead of silently disappearing.

What changed

1. Versioned agent graph contract

  • Adds a lightweight AgentGraph / AgentNode model with stable actor IDs.
  • Stores identity, parent/depth, role and instruction preview, launch metadata, trace availability, and link quality.
  • Keeps the existing Trace contract actor-scoped; graphs reference traces instead of embedding or merging them.

2. Adapter-owned session correlation

  • Codex: correlates spawn_agent call/output metadata with child session_meta fields such as parent thread ID, agent ID, depth, role, and nickname.
  • Claude: correlates root subagents/ artifacts and sidecar metadata, preserving exact vs derived quality.
  • Auxiliary child sessions stay hidden from the ordinary left session rail and are reachable only through their owning root graph.
  • Cache invalidation includes Claude sidecars so graph state does not remain stale after metadata changes.

3. Root-scoped API and ownership checks

Adds:

  • GET /api/sessions/{root}/agents
  • GET /api/sessions/{root}/agents/{agent}/trace

Child traces are loaded on demand. The server verifies graph ownership before returning a child trace, including ambiguous duplicate Codex root IDs, so a node cannot be fetched through a different root session. Missing or unavailable traces use explicit conflict/error responses rather than guessed fallback data.

4. Agent Lens UI

  • Adds an Agents dock sheet with Main, available children, unavailable/failed states, event counts, roles, previews, and retry behavior.
  • Switching actors replaces the map, HUD, and timeline with that actor trace while the parent task remains selected in the session rail.
  • Each actor remembers its own playhead; switching clears the selected file to avoid carrying stale file state across traces.
  • Report evidence belongs to Main and switches back to Main before jumping.
  • Clicking a subagent launch mark opens the Agents sheet instead of pretending the child events live in the root timeline.

5. Viewport-level agent details

  • Child instructions and correlation details render through a document-level fixed portal instead of inside the dock row, so long content is not clipped by the HUD or responsible for panel-level horizontal scrolling.
  • Hover or keyboard focus opens a quick preview. A separate accessible info control pins a non-modal detail card without switching the active lens.
  • Pinned details are vertically scrollable and can be dismissed through the same info control, the Close button, Escape, or an outside click.
  • Positioning prefers the canvas beside the Agents sheet, clamps to the viewport, and recomputes after resize, capture-phase scroll, and the dock entrance animation.
  • Child rows use sibling selection/detail buttons rather than nested interactive controls; missing or failed traces remain non-selectable while their diagnostic details stay available.

6. Async and export safety

  • Generation/request guards prevent a delayed child response from overwriting a newer session or actor selection.
  • Video export owns the active session, actor, canvas, and playhead until recording ends.
  • Session switching, actor switching, report evidence navigation, Inspector history jumps, Rescan, view changes, and timeline controls are locked or guarded during export.

Review guide

The easiest review order is:

  1. Contract: internal/model/agent.go
  2. Discovery/correlation: internal/adapter/codex/agents.go and internal/adapter/claudecode/agents.go
  3. Authorization and trace projection: internal/server/server.go
  4. Lens state and UX: web/src/App.tsx and web/src/ui/AgentsPanel.tsx
  5. End-to-end behavior: web/e2e/agent-lens.spec.ts

The adapter tests are intentionally fixture-heavy because each source format owns its linking semantics. The browser suite uses the production React/API path with deterministic fixtures for races and unavailable states.

Deliberate V1 boundaries

Testing

  • go test ./... -count=1
  • go test -race -count=1 ./internal/adapter/... ./internal/server
  • npm --prefix web run build
  • npm --prefix web run test:e2e — 15/15
  • make test
  • git diff --check

The browser cases cover truthful rows, Main/child playhead memory, a zero-event child, report-to-Main navigation, rapid stale-response switching, failed child retry, export locking, viewport-level detail placement, dock overflow protection, pinned long-content scrolling, and all supported dismissal paths.

Manual QA

  • Real Codex: Main trace with five available children and one failed child; verified zero-event and non-empty children, return to Main, and stable browser state.
  • Real Claude: verified Main/child switching and return to Main.
  • Rechecked the UI with a tmc-pc-v2 root containing 2,149 files, 834 events, and six subagent entries.
  • Rechecked the viewport detail card on a frozen 654-event real Codex root with two children: the portal stayed within the viewport, escaped dock clipping, preserved Main while pinned, and produced no browser console errors.
  • Final whole-branch independent QA gate: PASS.

Known evidence gap

The historical real Claude missing-parent artifact used during discovery is no longer present on the local machine. That state is covered by deterministic adapter, API, and browser fixtures, but the exact original live artifact could not be revalidated.

Related to #9.

@yearth
yearth marked this pull request as draft July 15, 2026 10:57
@yearth
yearth marked this pull request as ready for review July 15, 2026 11:05
@cosmtrek

Copy link
Copy Markdown
Owner

@yearth Looks cool. I'll take some time to try this feature and share my feedback.

Copy link
Copy Markdown
Owner

Thanks for the work on Agent Lens. I verified the PR in a fresh worktree, exercised the feature in the browser, and reviewed the adapter, server, caching, contract, packaging, and UI paths. The core interaction works well, but I think the following issues are worth addressing. I have grouped them by area and included the concrete failure mode and code location.

1. AgentGraph correlation correctness

1.1 Legacy Codex spawn output can produce duplicate and incorrectly failed nodes

Location

  • internal/adapter/codex/agents.go:21-24 only models agent_id and nickname.
  • parseAgentLaunchOutput at lines 279-284 rejects legacy output such as {"task_name": "..."}.
  • unlinkedCodexLaunchNode at lines 235-255 treats any observed, non-empty, unparseable output as failed.
  • The later derived-child pass at lines 119-139 can then add the real child again through parent_thread_id correlation.

Failure mode

A single legacy spawn can appear as two rows: one failed/unavailable launch node and one derived/available child node. Non-empty output is not sufficient evidence that the launch failed; it may simply be a different success format.

Suggested fix

  • Recognize task_name as a legacy success shape, or at minimum keep an unrecognized non-empty result as unknown rather than failed.
  • Correlate the launch with catalog child metadata when possible and merge the unlinked launch with the derived child.
  • Only mark failed when there is explicit failure evidence.

Acceptance case

A legacy Codex session containing one task_name spawn should produce exactly one usable AgentNode.

1.2 Claude launch matching should use a stable call identity

Location

internal/adapter/claudecode/agents.go:101-113 stores matched launches in map[*claudeAgentLaunch]bool.

Failure mode

If the same callID is read from more than one transcript, the two parsed values have different pointer identities. One can be matched to the real artifact while the other is emitted as a ghost unavailable row.

Suggested fix

Track matched launches by callID, or by a stable composite identity if additional ownership context is required.

Acceptance case

The same callID appearing across transcript inputs should still produce only one node.

1.3 Depth-first sorting is required for nested agents

Location

  • internal/adapter/codex/agents.go:306-323
  • internal/adapter/claudecode/agents.go:366-383
  • web/src/ui/AgentsPanel.tsx:60 renders the returned array directly.

Failure mode

Both comparators sort by depth before launch order. For a graph Main -> A, B and A -> A1, the UI receives A, B, A1. Because indentation is the only hierarchy cue, A1 visually appears under B.

Suggested fix

Return a stable preorder traversal: each parent immediately followed by its complete subtree, with siblings ordered by launchSeq and then stable label/ID fallbacks. The two adapters can share this ordering helper.

Acceptance case

At every nesting depth, each subtree remains contiguous in the returned AgentGraph and in the panel.

2. Refresh consistency and caching

2.1 Rescan leaves child trace data stale

Location

  • web/src/App.tsx:102 stores child traces in actorTraceCache.
  • The cache is only cleared by resetLens at lines 123-140.
  • A fresh scan of the current session only reloads the graph at lines 254-257.
  • Lens switching prefers the cached trace at lines 326-330.
  • loadSession can combine a cached child trace with the newly loaded citymap at lines 191-203.

Failure mode

After a running subagent grows from, for example, 21 to 30 events, Rescan can update the row to “30 events” while entering the lens still displays the cached 21-event trace. If the repository layout changed, the cached child trace can also retain file IDs assigned against the old citymap.

Suggested fix

  • On a fresh scan, invalidate child traces while preserving per-lens playheads.
  • If a child lens is active, reload that child trace or safely return to Main.
  • Never reuse a child trace whose file-ID projection belongs to a previous citymap.

Acceptance case

After a child trace or repository tree changes, one Rescan updates the graph count, lens events, and file-to-building mapping together.

2.2 AgentGraph is rebuilt for every API request

Location

  • GET /agents calls agentGraph at internal/server/server.go:242.
  • GET /agents/{id}/trace calls it again for ownership validation at line 260.
  • agentGraph at lines 659-672 copies the complete session catalog and invokes the adapter on every request.
  • The adapters then reread the root and related child files.

Impact

On the local corpus, a 66 MB session took roughly 0.5 seconds per graph build. Concurrent requests repeated the same parsing work, adding CPU time and increasing RSS. This is currently acceptable for small sessions but scales poorly for long traces and nested teams.

Suggested fix

  • Cache AgentGraph by root session key.
  • Fingerprint every graph input: root trace, related child traces, and Claude sidecars.
  • Reuse the trace cache’s fingerprint/inflight pattern so concurrent requests share one build.
  • Invalidate on fingerprint changes and fresh scans.

Acceptance case

Repeated and concurrent requests with the same fingerprints should perform one graph build; a changed input should automatically rebuild it.

3. Contract and packaging completeness

3.1 AgentGraph has no mirrored JSON Schema

Location

  • The Go contract is defined in internal/model/agent.go:30-52.
  • The TypeScript contract is defined in web/src/types.ts:49-70.
  • schema/ currently contains only trace, citymap, and report schemas.

Suggested fix

Add schema/agent-graph.schema.json covering version 1, node kinds and statuses, trace availability, link quality/method, parent/depth fields, and optional metadata. Add a contract test that validates representative exported graphs.

3.2 The embedded frontend was not regenerated

Location

  • The normal server embeds internal/server/static through internal/server/server.go:32.
  • This PR changes web/src extensively but does not update internal/server/static.
  • make test only builds web; make embed-static performs the copy into the embedded directory.

Failure mode

The feature works through the Vite dev server, but a normally built mindwalk binary can continue serving the old UI without Agent Lens.

Suggested fix

  • Run make embed-static and commit the generated assets.
  • Add a CI check that regenerates embedded assets and fails if the worktree changes.
  • Smoke-test Agent Lens using a non-dev binary.

Acceptance case

mindwalk serve and mindwalk serve --dev should expose the same Agent Lens functionality.

4. UI feedback and discoverability

4.1 Child-trace errors and Retry are rendered below the entire list

Location

  • The list starts at web/src/ui/AgentsPanel.tsx:41.
  • Loading and error blocks are rendered after the complete list at lines 80-98.
  • The whole Dock panel scrolls through web/src/styles.css:831-842.

Observed behavior

At 1280x720, the panel had 583 px of visible height while the 12-row list was 1,275 px tall. When loading a top-row agent failed, the row spinner disappeared but the error and Retry appeared more than 1,300 px below the top. To a sighted user, the click appeared to do nothing.

Loading itself is not silent—the clicked row already shows a spinner and “Loading trace…”. The real gap is failure recovery.

Suggested fix

  • Render a child-trace error and Retry directly below the affected row.
  • Render graph-level errors at the top of the list.
  • Keep the row-level spinner and remove the redundant global “Loading trace…” block.
  • Avoid scrollIntoView, which would move users away from the row they selected.

4.2 The feature entry points are hard to discover and the list is too tall

Location

  • web/src/ui/Hud.tsx:74-77 renders Lens as a non-interactive div.
  • web/src/ui/Hud.tsx:144-148 renders N subagents as a non-interactive span.
  • web/src/ui/AgentsPanel.tsx:131-138 renders title, status, role, and a two-line instruction preview as separate rows.
  • The current title tooltip only says “Open trace”; it does not expose the instruction.

Suggested fix

  • Make N subagents and Lens open the Agents sheet. The Lens control is especially important because the subagent count disappears after switching into a child trace.
  • Compress each agent to two visual lines: title plus right-aligned event count, then role plus a one-line instruction preview.
  • Provide hover and keyboard-focus access to the truncated instruction.
  • Add the root event count to Main for alignment.
  • Keep derived-link and launch-correlation details available, but move this engineering terminology into a tooltip or secondary detail.

A separate Dock status dot and a lens-cycling keyboard shortcut do not seem necessary for this pass.

4.3 A zero-event child trace displays the wrong empty state

Location

web/src/ui/Timeline.tsx:420 displays “Select a session to start the walk” whenever there is no current event.

Failure mode

A valid selected child trace with zero events is presented as though no session were selected.

Suggested fix

Distinguish “no session selected” from “selected trace has no events.” The latter could say: “No recorded activity for this agent.”

Suggested validation set

  • A legacy Codex task_name spawn does not create failed + derived duplicates.
  • Repeated Claude callIDs produce one node.
  • Nested AgentGraph fixtures preserve contiguous subtrees.
  • Rescan updates graph counts, child events, and file-ID projection together.
  • Eight concurrent graph requests trigger one build.
  • A top-row child request failure exposes the error and Retry without scrolling.
  • Zero-event, missing, unavailable, failed, and derived states remain accurate and distinguishable.
  • Agent Lens works in both the development server and the embedded production binary.

The core design remains strong: adapter ownership boundaries are clean, child traces are authorized through the root graph, link quality is explicit, and the lens/playhead behavior is solid. The items above are mainly about making that model reliable across legacy data, refreshes, larger sessions, nested graphs, and the packaged binary.

@yearth
yearth force-pushed the feat/agent-lens branch from 966fb2d to 4a21403 Compare July 17, 2026 07:46
@yearth

yearth commented Jul 17, 2026

Copy link
Copy Markdown
Contributor Author

@cosmtrek Thanks for the detailed review. I have addressed all ten items in the latest revision.

Correlation correctness

  • Legacy Codex task_name results are now recognized and merged with their catalog child instead of producing failed + derived duplicates. Unknown JSON results remain unknown unless explicit failure evidence exists.
  • Claude launches are matched by stable call ID.
  • Both adapters now use a shared deterministic preorder traversal, keeping every nested subtree contiguous.

Refresh and caching

  • Rescan now invalidates child trace projections while preserving per-lens playheads, so graph counts, child events, and file mappings refresh together.
  • AgentGraph now has fingerprint-based caching and inflight request sharing, with invalidation for changed inputs, sidecars, and fresh scans.

Contract and packaging

  • Added the versioned AgentGraph JSON Schema and contract validation.
  • Regenerated the embedded frontend, added a CI drift check, and smoke-tested the production binary.

UI and discoverability

  • Child failures and Retry are row-local.
  • Lens and subagent-count HUD elements now open the Agents sheet.
  • Rows are compact two-line entries with Main event-count alignment.
  • Zero-event children now show the correct empty state.
  • Agent details were further moved into a viewport-level portal with hover/focus preview and a pinnable, scrollable detail card, avoiding dock clipping and horizontal overflow.

Validation now includes the focused acceptance cases, the full Go suite, race tests, Playwright 15/15, production build/static parity, real Codex browser QA, and a passing CI run.

One evidence gap remains: the historical Claude missing-parent artifact used during discovery is no longer available locally, so that exact artifact is covered by deterministic fixtures rather than a fresh live replay.

@cosmtrek
cosmtrek merged commit 05081bc into cosmtrek:master Jul 17, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants