Skip to content

Capture coverage: Copilot, Gemini CLI and Cursor observe; clipboard writes; structural web-agent detection - #103

Merged
KAVentures merged 31 commits into
fix/capture-correctnessfrom
feat/capture-coverage
Sep 28, 2026
Merged

KAVentures merged 31 commits into
fix/capture-correctnessfrom
feat/capture-coverage

Conversation

@KAVentures

Copy link
Copy Markdown
Owner

Release 2 of the capture audit. Stacked on KAVentures/openworkgraph#102: the base is fix/capture-correctness, so this diff shows only release 2. Merge #102 first; GitHub then retargets this PR to main.

What changes

Observe for GitHub Copilot, Gemini CLI and Cursor. The same switch, backup, preserve and refuse rules as Claude Code and Codex. Remove takes out only OpenWorkGraph's own entries.

  • Copilot (VS Code):
    • Settings: github.copilot.chat.otel.* in the user settings.json, with captureContent: false.
    • Mapping: the existing generic GenAI-span adapter maps invoke_agent / chat / execute_tool to run / model call / tool call. It never reads gen_ai.tool.call.arguments/result or gen_ai.input.messages.
    • Refuses when:
      • Copilot telemetry already goes elsewhere;
      • content capture is on;
      • settings.json has comments (VS Code's JSONC). Manual setup shows the keys.
  • Gemini CLI:
    • Settings: a telemetry block with target: local, otlpProtocol: http and logPrompts: false. Gemini's default is true, which would put prompts, request/response text and tool arguments into its log events.
    • Adapter: a new log adapter maps user_prompt / api_response / api_error / tool_call (+ human accept/reject/modify decisions; auto_accept is policy, not a person). It is an allowlist, so content stays out even if prompt logging is re-enabled.
    • Refuses when the user has their own telemetry settings.
  • Cursor:
    • Hooks: ~/.cursor/hooks.json gets sessionStart/End, beforeSubmitPrompt/stop (turn = generation_id), postToolUse(Failure) and subagentStop.
    • Never blocks: Cursor runs hooks synchronously, so the bridge replies first ({"continue": true} / {}), then records, and always exits 0.
    • No permission hooks: preToolUse, beforeShellExecution, beforeMCPExecution, beforeReadFile and subagentStart are never used.
    • Never read: prompt, text, tool input/output, edits, commands, error messages, email, workspace and transcript paths.

Endpoint for clients without headers

  • Why: Copilot and Gemini build their exporters from a settings-file base URL and can't send Authorization from there. (Verified in their sources: exporter-*-otlp-http with {url: base + '/v1/…'}.)
  • Route: /agent-ingest/otlp/{copilot|gemini}/{token}/v1/{traces|logs|metrics}. The token is a separate write-only secret.
  • Hardening:
    • JSON only, with 415 and a diagnostic for protobuf;
    • bounded gzip;
    • the Observe switch is honoured;
    • metrics, and Copilot's logs (which repeat its spans), are counted and dropped.
  • Access log: a filter replaces the token with [redacted]. Verified with a real uvicorn.
  • Diagnostics: new channels copilot_otel, gemini_otel and cursor_hooks.

Clipboard writes

  • What: the collector notices clipboard writes without a shortcut (menu, right-click, drag, apps) from the OS change counter, which never exposes contents, and records them as clipboard_write.
  • Links: Cmd/Ctrl+C changes are absorbed rather than double-counted, and a later paste links to the most recent write.
  • Limits: a write is never inferred as a paste, nothing is attributed while away, and clipboard_write_detection_enabled: false opts out.
  • Timestamp fix: click, scroll and clipboard events now carry their occurrence time, not the worker-processing time.

Web agents in any UI language

  • Primary signals: aria-busy, data-is-streaming, data-testid send/stop tokens, a submit button in the composer's form, and Enter in the message box (not Shift+Enter or IME composition). The text is never read.
  • Fallback: English labels only. There are still no class-name selectors.
  • Version: browser sensor 1.13.0. The version check derives from the manifest, so the dashboard asks to reload older extensions.

Smaller

Verification

  • New tests:
    • tests/test_agent_coverage_v099.py (14): endpoint, auth, 415, gzip, metrics, Observe off, redaction, all three presets including refusals, Cursor mapping, and the real hook process replying correctly with OpenWorkGraph unreachable.
    • tests/test_capture_coverage_v099.py (2): the real collector loop covering write vs shortcut, paste linking and away.
    • tests/js/agent_surface_structural.test.mjs (4): including a Swedish UI.
  • Updated tests for the intended contract changes: Cursor/VS Code/Gemini are now Observe-capable, and the browser sensor version.
  • Full suite: 801 passed, 1 skipped, run twice. JS 73/73. check_injected_dashboard_js.py passes.
  • End to end with the official @opentelemetry/exporter-{trace,logs}-otlp-http 0.222.0 packages against a real server:
    • Copilot-shaped spans and Gemini log records were stored as structural events;
    • no planted content was stored;
    • the token never appeared in the log.

Not verified here / not included

  • Not run inside real VS Code, Gemini CLI or Cursor installs. The contracts come from their current source and docs.
  • Cursor's CLI reportedly sends only shell hooks, so Observe covers the Cursor app.
  • Not included:
    • Claude metrics;
    • Copilot CLI (env-var configuration only);
    • platform rework (osascript, UWP, Wayland), still planned as its own PR.

…rites; structural web-agent detection

- Observe presets for GitHub Copilot (VS Code OTel, content capture off),
  Gemini CLI (OTLP/HTTP logs, logPrompts forced false) and Cursor (non-blocking
  hooks only; never a permission hook). Backup/preserve/refuse rules as for
  the other connectors; manual setup material in the dashboard.
- OTLP endpoint with a separate write-only path token for clients that cannot
  send headers from a settings file; JSON only (415 for protobuf), bounded
  gzip; token redacted from access logs; per-app diagnostics channels.
- Gemini CLI log adapter and Cursor hook adapter: structural allowlists,
  turn = prompt/generation, never falling back to the session.
- Clipboard writes without a shortcut from the OS change counter (contents
  never read, never a paste, nothing while away); interaction events stamped
  with their occurrence time.
- Web agents detected from structure first (aria-busy, streaming markers,
  data-testid send/stop, submit in composer form, Enter in message box);
  English labels only as fallback. Browser sensor 1.13.0.
- Agents tab names only frameworks whose runs were actually hidden.
@KAVentures

Copy link
Copy Markdown
Owner Author

Follow-up from end-to-end verification:

  • Merged the Capture correctness: one collector, away spans, document boundaries, honest health #102 fixes (ephemeral history, lease revoke on quit, empty span, Windows test).
  • Verified through the full local app: Observe on/remove via /v1/connections for Copilot, Gemini and Cursor (files restored exactly on remove). Official @opentelemetry exporters posting to the endpoints as written into the settings files. The exact Cursor hook command from hooks.json. Stored events are structural only, and the path token never appears in the log.
  • Added a regression test: ephemeral history keeps Cursor and Copilot sessions across turns.
  • Fixed: the Cursor manual-setup modal showed the path-token note (Cursor uses hooks).
  • Fixed test isolation: the ephemeral test now uses its own data folder.

@KAVentures
KAVentures merged commit 0629753 into fix/capture-correctness Sep 28, 2026
10 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant