Build background session ingestion watcher

[geier]

Context

The long-term memory system uses a background watcher process that automatically ingests every OpenCode session into the Graphiti knowledge graph. The AI never writes to memory — it only reads. This watcher is the write pipeline that ensures every session is captured, every time, without relying on the AI to "remember" to save anything.

OpenCode's backend emits SSE events including session.status (with idle/busy/retry states). When a session transitions from busy → idle, the watcher fetches the full transcript and posts it to Graphiti's REST API for entity/fact extraction.

Goal

Build a standalone watcher script (TypeScript/Bun) that:

1. Connects to the OpenCode SSE event stream
2. Detects when sessions complete (busy → idle transitions)
3. Fetches the full session transcript via the OpenCode API
4. Posts the transcript to Graphiti's REST API for ingestion
5. Tracks which sessions have been ingested to avoid duplicates

Requirements

SSE Event Monitoring

• Connect to GET http://<opencode-host>:4096/event?directory=<dir> with Accept: text/event-stream
• Parse session.status events (see app-prefixable/src/sdk/gen/types.gen.ts line 594)
• Track busy → idle transitions per session ID
• Handle SSE reconnection on disconnect (exponential backoff)
• Support monitoring multiple project directories

Transcript Fetching

• On idle transition, call GET /session/{sessionID}/message to retrieve full transcript
• Format messages into a coherent transcript (user messages, assistant messages, tool calls/results)
• Include metadata: session ID, project directory, timestamp, model used

Graphiti Ingestion

• Post transcript to Graphiti REST API: POST http://<graphiti-host>:8000/v1/episodes
• Use source_description and group_id for scoping (e.g., group_id = project name)
• Handle Graphiti's async processing (the API returns immediately, extraction happens in background)
• Log ingestion status and any errors

Deduplication

• Maintain a local state file (JSON or SQLite) tracking:
  • Session ID → last ingested message timestamp
  • Avoid re-ingesting sessions that haven't changed since last ingestion
• On restart, check all existing sessions against the state file and ingest any missed ones

Configuration

• Environment variables:
  • OPENCODE_API_URL — OpenCode backend URL (default: http://127.0.0.1:4096)
  • OPENCODE_DIRECTORIES — comma-separated list of project directories to monitor
  • GRAPHITI_API_URL — Graphiti REST API URL (default: http://localhost:8000)
  • GRAPHITI_GROUP_ID — default group ID for scoping (default: derived from directory name)
  • STATE_FILE — path to deduplication state file (default: ~/.local/share/opencode-memory/state.json)

Files to create

• memory/watcher/index.ts — main watcher script
• memory/watcher/sse.ts — SSE client with reconnection logic
• memory/watcher/transcript.ts — transcript fetching and formatting
• memory/watcher/graphiti.ts — Graphiti REST API client
• memory/watcher/state.ts — deduplication state management
• memory/watcher/package.json — dependencies (minimal, Bun-native where possible)

Reference

• SSE event types: app-prefixable/src/sdk/gen/types.gen.ts (lines 580-607, 839-881)
• SSE handling pattern: app-prefixable/src/context/events.tsx (subscriber pattern)
• SSE parsing: app-prefixable/src/utils/sse.ts
• Busy → idle detection: app-prefixable/src/pages/layout.tsx (lines 1688-1713, busyTracker pattern)
• Session API: GET /session/{id}/message returns Array<{ info: Message, parts: Part[] }>
• Graphiti REST API: https://github.com/getzep/graphiti (see server/ directory)
• OpenCode session export format: opencode export CLI command outputs full session JSON

Acceptance Criteria

• Watcher starts and connects to OpenCode SSE stream
• Detects session completion (busy → idle) reliably
• Fetches and formats full session transcript
• Successfully posts transcript to Graphiti REST API
• Deduplication prevents re-ingesting the same session content
• Handles SSE disconnections with automatic reconnection
• Logs ingestion events (session ID, timestamp, status)
• Works with multiple project directories
• Catches up on missed sessions on restart