Skip to content

v0.90: make agent observability rich and capability-aware - #89

Open
KAVentures wants to merge 35 commits into
mainfrom
feat/rich-agent-observability-v090-rebased
Open

KAVentures wants to merge 35 commits into
mainfrom
feat/rich-agent-observability-v090-rebased

Conversation

@KAVentures

@KAVentures KAVentures commented Sep 26, 2026 •

Copy link
Copy Markdown
Owner

What this changes

This upgrades agent observation across the supported integrations instead of special-casing Claude Code.

Claude Code

  • Combines existing safe lifecycle hooks with Claude Code's stable OTel log events.
  • Correlates hook prompt_id with OTel prompt.id so one user request becomes one structural execution.
  • Adds observed model calls, model identity, token usage, human decision outcomes, retry exhaustion and subagent handoffs.
  • SubagentStart now creates an explicit parent handoff plus child run boundary.
  • Adds a dedicated write-only /agent-ingest/v1/claude-otel endpoint with a strict structural allowlist.
  • One-click Connect now installs hooks + logs-only OTel configuration while explicitly leaving prompt/response/tool-content logging disabled.
  • Existing foreign telemetry settings are never overwritten; Disconnect removes only values that still exactly match what OWG wrote.

Codex

  • One-click setup now exports both Codex structural OTel logs and traces to the same write-only endpoint; prompt, agent-response and guardian-rationale content logging remains disabled.
  • Uses turn IDs when available, extracts model/token usage from safe structural attributes when emitted and maps multi-agent spawn communication to handoffs.
  • Handles current Codex telemetry where semantic event names can be replaced by tracing call-site names using only low-cardinality structural fields.
  • Detects older OWG trace-only managed blocks as partial so Connect upgrades them rather than falsely reporting full coverage.

OpenAI Agents SDK

  • Adds a stable adapter sensor identity.
  • Response spans now contribute model identity and token usage without serializing response/input content.
  • Existing tool, hierarchy and native handoff support is retained.

Generic OTel / custom agents

  • Adds capability metadata instead of pretending every integration can observe every signal.
  • Portable GenAI model/tool/usage/timing remains supported; provider-specific handoffs and approvals are explicitly unavailable unless a native adapter supplies them.

Dashboard + MCP

  • 0 now means an observable signal was observed zero times.
  • — means the active integration cannot observe that signal (or has not declared the capability).
  • Rich reports include model calls, tools, handoffs, approval requests, token totals, model identity and telemetry depth without productivity scoring.
  • Claude hook + OTel duplicate tool evidence is de-duplicated at read time.
  • Compact MCP get_agent_runs exposes the same capability-aware summaries instead of dropping the richer signals.

Privacy

No prompts, assistant responses, reasoning, tool arguments/results, transcript paths, raw log bodies or arbitrary OTel attributes are added to canonical evidence. Provider-native payloads are treated as untrusted input and pass through strict allowlists plus the existing canonical content-field rejection.

Verification added

  • Claude OTel privacy + structural projection
  • Claude hook/OTel turn correlation and de-duplication
  • Claude one-click telemetry conflict/reversibility behavior
  • Codex logs+trace configuration and trace-only managed-block upgrade
  • Codex turn/model/token + multi-agent spawn projection
  • Codex semantic fallback without reading message content
  • OpenAI Agents response model/usage projection
  • write-only Claude OTel route auth/privacy
  • capability-aware dashboard syntax/semantics

Version bumped consistently to 0.90.0.

Merge the provider observability work into the rebased feature branch while retaining PR #86 one-click setup changes.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant