Capture correctness: one collector, away spans, document boundaries, honest health

[KAVentures]

Release 1 of the capture audit: fixes where OpenWorkGraph's own evidence was misleading. It changes no schema, no field meaning and no sharing rule, and it adds no new organizational signal.

Why

On a real install (21,852 events, Sep 15–28):

• Idle counted as focus: 198 h of foreground time, but only 12.3 h with any input, and single spans ran 13 h overnight.
• Duplicated time: two collectors recorded at the same time, duplicating 16 h (8%). Some days showed 26 h and 37.5 h.
• Noise warnings: 12% of all events were capture_health rows caused by routine checkpoints.
• Sensors claimed "ON" on macOS while the OS withheld events.

What changes

One collector per data folder

• The lock: an OS advisory lock (flock / msvcrt.locking) on <data>/.collector.lock, held for the collector's lifetime. It is released on exit or crash, with no stale PID file.
• A second collector: it exits with code 75. The supervisor backs off for 15 s instead of respawning every second.
• Separate folders: data/live and data/demo are separate.
• Verified live on macOS: a second collector on the same folder exited 75.

Away spans (not a new meaning for duration_seconds)

• When a span ends: after away_after_seconds (default 300, minimum 60) with no input, or when the screen locks. The rest becomes away_span (app "Away").
• Reading time kept: the first 300 s remain with the app, which covers reading and thinking.
• Idle source: the OS input clock (CGEventSourceSecondsSinceLastEventType / GetLastInputInfo). It needs no permission and never sees keys. The collector's own sensors are a fallback only when they can see input.
• No signal: without one (Linux) there is no away detection, instead of guesses.
• Never shared: away_span is dropped in prepare_event_for_gateway regardless of org policy, since sharing it would be presence monitoring. Analytics already count app time from focus_span only.

Debounced same-app document boundaries

• The new default: application_and_document. A materially different title must persist for 2 polls. The boundary is backdated to when the new title first appeared.
• Ignored noise: unread badges, unsaved markers, "Edited"/"Redigerad", percentages, "Not Responding", one-poll dialogs, titles going empty.
• Privacy modes: titles are keyed in memory only (never stored).
  • none: no title information is used.
  • hash: normalizes before comparing.
  • Excluded apps: never tracked.
• Existing configs: every config.json was created from the example with "application", so that value now follows the default. "application_only" is the explicit opt-out.

capture_health = degradation

• Degradation keys: worker errors, queue drops, an unreadable foreground window, and missing permissions.
• Reporting: a new kind of problem or a permission change is reported immediately; repeats at most every 10 minutes.
• Routine counters (checkpoints, gaps, document boundaries, away) move to diagnostics.

Truthful macOS permissions

• How it checks: AXIsProcessTrusted and IOHIDCheckAccess preflight, which never prompt, rechecked every minute.
• What each sensor needs: clicks need Accessibility; keyboard counts and copy/paste shortcuts need Accessibility and Input Monitoring. This was observed on a real untrusted process, where pynput refuses both listeners.
• Where it shows: startup output says BLOCKED per sensor, the heartbeat carries permissions and a truthful keyboard_sensor, capture health records missing permissions, and the Recording pill says "macOS … permission missing".

Claude Code turn boundaries

• Mapping: UserPromptSubmit → run_started and Stop → run_finished/success, keyed by prompt_id (shared with Claude's tool hooks and OTel). SessionStart/SessionEnd still bound the session.
• Missing prompt_id: the turn hook is ignored, never confused with the session.
• Content: these payloads carry the prompt and the last reply, and the adapter reads neither (tested).
• Existing installs: outdated OpenWorkGraph hook entries are refreshed at launch in enterprise_runner only when Claude Code Observe is on and OpenWorkGraph's hooks already exist. A backup is written, and user hooks and env are kept. It is deliberately not done at app startup, so tests never touch a real ~/.claude.
• PreCompact: deferred. It needs a new agent operation, which older Gateways would reject on sync.

Agent telemetry diagnostics

• Where: GET /v1/agent-telemetry/diagnostics (API/dashboard auth) plus a line in the Connections rows.
• What it shows: per channel, requests with times, rejections by reason code, OTel records seen and ignored, and events stored, plus which exporters and hook events are configured.
• No content: no payloads, headers or error text.
• What it answers: whether zero Claude model calls means Claude isn't exporting, the request was rejected, or the adapter ignored it.

Lease-gated agent spool

• When it spools: on a transport failure or 5xx, a hook event may be spooled to data/auth/agent_spool. A 4xx is never spooled.
• The lease gate: only while a 90 s lease is valid, and the lease's data folder capture state reads "recording" directly (never defaulted). The lease is issued only by a running, recording, non-demo OpenWorkGraph and revoked on Pause, Stop and exit.
• After Stop or quit: events are dropped.
• On delivery: the flush runs the normal ingest path, so Observe switches, deletions, retention and pause windows apply (tested: a paused-time event is dropped at flush).
• Limits: 2,000 files × 256 KB, 24 h max age. Other data folders' files are left alone.

Tests

• New files: tests/test_capture_correctness_v098.py (16) drives the real collector loop with a scripted clock, window and OS idle clock. tests/test_agent_delivery_v098.py (12) covers the lease, flush, client, diagnostics, hook refresh and status. tests/js/capture_correctness.test.mjs (2).
• Updated: the Claude adapter/control-plane tests now assert the new hook contract, including no content in turn events.
• Full suite: 785 passed, 1 skipped, run twice. JS 69/69. check_injected_dashboard_js.py passes.
• Smoke run of the real collector on macOS in an untrusted shell:
  • the lock held;
  • every blocked sensor was reported;
  • an away span opened while the user was away.

Not in this PR (release 2)

• Clipboard-write monitoring (writes only; no inferred pastes).
• Copilot OTel, Gemini CLI OTel (with prompt logging forced off) and Cursor hooks presets.
• Structural/multilingual web-agent detection.
• Claude metrics.
• Platform rework (macOS without osascript, Windows UWP, Linux/Wayland).
• Known issue, unchanged: Claude subagent events share the parent turn's run_id, so they are grouped into the parent turn rather than linked as children.

[KAVentures]

Follow-up from end-to-end verification (isolated full stack on macOS: real launcher, collector supervisor, hook processes, config writes):

• Fixed: ephemeral agent history. A turn's Stop (or a subagent finishing) purged and tombstoned the whole running session, so ephemeral users lost every later turn. Now only the session's own end (run_id == session_id) closes it. Regression test added.
• Fixed: clean quit left the spool lease valid for up to 90 s. uvicorn re-raises SIGTERM after shutdown, so finally/atexit never ran. The lease is now revoked in the app's shutdown step and never re-issued after stop. Verified live: lease gone right after quit, hook spools nothing.
• Fixed: empty 0-second focus span when a document change first appeared in the same poll as a checkpoint (~1.7% of document switches). Regression test added.
• Fixed: Windows-only test flake. The pause-window test depended on the clock tick; it now uses explicit timestamps.
• Fixed: long Recording pill overlapped the header; permission names moved to the hover text.