feat: recall quality, provenance, keyless graph, and connector parity

.env.example

@@ -26,22 +26,24 @@
 # The detection order is OPENAI_API_KEY → MINIMAX_API_KEY → ANTHROPIC_API_KEY
 # → GEMINI_API_KEY → OPENROUTER_API_KEY → noop.
 
-# OPENAI_API_KEY=sk-...                          # Used for OpenAI-compatible embeddings today. PR #307 will extend this to chat completions (DeepSeek, SiliconFlow, vLLM, LM Studio, Ollama via `/v1`).
+# OPENAI_API_KEY=sk-...                          # Activates both the OpenAI-compatible LLM provider (DeepSeek, SiliconFlow, vLLM, LM Studio, Ollama via `/v1`) and OpenAI embeddings. Set OPENAI_API_KEY_FOR_LLM=false to scope it to embeddings only.
 # OPENAI_BASE_URL=https://api.openai.com         # Override for OpenAI-compatible providers
+# OPENAI_MODEL=gpt-5.6-luna                      # Default OpenAI-compatible chat model
+# OPENAI_API_KEY_FOR_LLM=false                   # Skip OpenAI auto-detection for LLM; key stays active for embeddings
 
 # ANTHROPIC_API_KEY=sk-ant-...
-# ANTHROPIC_MODEL=claude-sonnet-4-20250514       # Default Anthropic model
+# ANTHROPIC_MODEL=claude-sonnet-5                # Default Anthropic model
 # ANTHROPIC_BASE_URL=https://api.anthropic.com   # Override for Anthropic-compatible proxies / Azure AI Foundry
 
 # GEMINI_API_KEY=...                             # Either env name works; GEMINI_API_KEY takes precedence
 # GOOGLE_API_KEY=...                             # Alias for GEMINI_API_KEY when set alone (emits a one-time stderr hint)
-# GEMINI_MODEL=gemini-2.5-flash                  # Default Gemini model (auto-detected GA model)
+# GEMINI_MODEL=gemini-3.7-flash                  # Default Gemini model (current stable Flash)
 
 # OPENROUTER_API_KEY=sk-or-...
-# OPENROUTER_MODEL=anthropic/claude-sonnet-4-20250514
+# OPENROUTER_MODEL=anthropic/claude-sonnet-5
 
 # MINIMAX_API_KEY=...
-# MINIMAX_MODEL=MiniMax-M2.7
+# MINIMAX_MODEL=MiniMax-M3
 
 # MAX_TOKENS=4096                                # Cap LLM completion tokens for compression / summarise calls
 
@@ -111,6 +113,13 @@
 # CONSOLIDATION_DECAY_DAYS=30                    # Age (days) after which non-reinforced memories decay during consolidation
 # GRAPH_EXTRACTION_ENABLED=true                  # Extract concept-graph edges on remember; powers the graph-traversal recall path
 # GRAPH_EXTRACTION_BATCH_SIZE=8                  # Memories per graph-extraction batch
+
+# Local reasoning models only: set to 1 to ask the model to skip its
+# hidden thinking pass during graph extraction. Extraction runs several
+# times faster; relation quality can drop slightly. Leave unset to let
+# the model think (default).
+# AGENTMEMORY_LLM_NOTHINK=1
+
 # AGENTMEMORY_REFLECT=true                       # Periodically auto-synthesize lessons from memories
 # AGENTMEMORY_DROP_STALE_INDEX=true              # Drop on-disk BM25 / vector index on startup if dim guard fires (#248). Recovery toggle for stuck-state debugging.
 # AGENTMEMORY_IMAGE_EMBEDDINGS=true              # Enable image embeddings when an image provider is present (experimental).
@@ -119,7 +128,7 @@
 # 6. CLI / runtime knobs
 # -----------------------------------------------------------------------------
 
-# AGENTMEMORY_TOOLS=all                          # core (7 tools, default) | all (51 tools) — surface exposed to MCP clients
+# AGENTMEMORY_TOOLS=core                         # all (54 tools, default) | core (8 tools): surface exposed to MCP clients
 # AGENTMEMORY_SLOTS=memory                       # Comma-separated plugin slot names the CLI should claim
 # AGENTMEMORY_DEBUG=1                            # Trace MCP shim probe + standalone fallback decisions to stderr
 # AGENTMEMORY_FORCE_PROXY=1                      # Skip the MCP shim livez probe and trust AGENTMEMORY_URL (for sandboxed MCP clients that can't reach localhost)

CHANGELOG.md

@@ -4,11 +4,9 @@ All notable changes to agentmemory will be documented in this file.
 
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [0.9.29] — 2026-08-15
 
-## [0.9.29] — 2026-08-02
-
-Patch release: the `.env` file now actually applies everywhere, imports become searchable, consolidation runs on session stop, twelve MCP-only agents get activated on connect, and every capture surface finally agrees on what "project" means. No breaking changes; read the upgrade notes below for four behavior changes you will notice.
+Release wave in two parts. Recall quality: hybrid ranking reaches the primary recall path, lessons get a real index, every record learns where it came from, the knowledge graph populates keyless, and agent scoping threads through all save paths — plus connector parity for pi and Codex, a new DeepSeek Harness connector, current provider model defaults, and a viewer clarity pass. Foundation: the `.env` file now applies everywhere, imports become searchable, consolidation runs on session stop, twelve MCP-only agents get activated on connect, and every capture surface agrees on what "project" means. No breaking changes; read the upgrade notes for behavior changes you will notice.
 
 ### Upgrade notes
 
@@ -19,6 +17,14 @@ Patch release: the `.env` file now actually applies everywhere, imports become s
 
 ### Added
 
+- **Write-time provenance on every record.** Each observation and memory carries an immutable origin block (channel `user` / `agent` / `tool` / `import` / `shared`, detail, capturedAt) stamped at capture, save, and import, and inherited through both compression paths. The base for trust-aware retrieval and ingest screening.
+- **`similarTo` advisory hint on save.** `mem::remember` reports a near-miss similarity match (0.4 to 0.7) back to the caller so agents can spot near-duplicates without the write being blocked.
+- **`AGENTMEMORY_LLM_NOTHINK=1`** (opt-in): asks local reasoning models to skip their hidden thinking pass during graph extraction. Extraction runs faster; relation quality can drop slightly. Default behavior unchanged; documented in `.env.example`.
+- **Keyless graph extraction.** `mem::graph-extract` always runs a deterministic structural pass first: files and concepts on compressed observations become nodes, and co-occurrence within an observation becomes a `related_to` edge. The graph now populates without any LLM key; `GRAPH_EXTRACTION_ENABLED` plus a provider key gates only the LLM pass that layers typed relations (fixes, depends_on, causes) on top. Session end fires extraction unconditionally.
+- **pi extension: capture parity with the Claude Code plugin.** Session registration on start (after the health check populates reachability, so it fires on the first session of a fresh process), prompt capture on submit (deduped in a 5-minute client window against auto-retry re-submissions; stored with user-channel provenance), per-tool observations from `tool_result` (server `inferType` classifies command_run / file_edit / file_read; `AGENTMEMORY_TOOL_OBSERVE=0` opts out), turn capture slices raised 500/4000 → 8000/8000, `memory_save` scoped to the current project instead of the global bucket, session end + one cross-session consolidate run on real quit only (`/new`, `/resume`, `/fork`, reloads excluded; no client-side summarize call — `session/end` already fans out the summary, avoiding the double-summarize the Stop hook had), status checks accept `status: "ok"`, and the status refresh no longer throws a stale-context error when the session is replaced mid-health-check. Live-verified on pi v0.84.2: prompt + turn observations landed and the session closed as `completed` on quit. Codex executes only hooks with a recorded `trusted_hash` and shows its "Hooks need review" approval prompt exclusively in the interactive TUI, so a `codex exec`-only workflow left the freshly installed hooks silently inert. `connect codex --with-hooks` now warns to launch `codex` once and choose "Trust all and continue" (and to re-approve after upgrades, since refreshed paths change the hash). Found by live-testing the documented flow end to end.
+- **`connect pi` actually installs.** The pi adapter was a stub that printed manual copy instructions because `integrations/` never shipped in the npm package. The extension source now ships, and `connect pi` copies it into `~/.pi/agent/extensions/agentmemory/`, which pi auto-discovers — no settings.json edit, `/reload` picks it up live. Idempotent by content compare; `--force` and stale copies refresh with a backup. `integrations/pi` is also a proper pi package now (`pi-package` keyword, `pi.extensions` manifest, peer deps on `@earendil-works/pi-coding-agent` + `typebox`; private, local installs only — `pi install ./integrations/pi` from a checkout), and the extension's type import moved off the renamed upstream package name.
+- **DeepSeek Harness connector.** `agentmemory connect dsh` appends an `@deepseek-ai/dsh-mcp-client` row to the home-level `$DSH_HOME/cordis.patch.yml`, the machine-local patch layer every Harness profile loads. `--with-hooks` adds full auto-capture: the bundled Claude Code hook scripts run through Harness's first-party `@deepseek-ai/dsh-hooks-claude-code` bridge (SessionStart, UserPromptSubmit, PreToolUse, PostToolUse, Stop) via a manifest written to `$DSH_HOME/agentmemory.hooks.json` with absolute script paths. Idempotent, `--force` replaces the rows, honors `DSH_HOME`.
+- **Viewer clarity pass.** Two-pane session explorer (list beside a sticky detail panel on wide screens), dashboard stat cards that navigate to their tabs, memory and lesson rows that expand to the full stored record with raw JSON and origin provenance, type-clustered graph layout with label collision avoidance when relations are sparse, health notes translated from machine slugs into sentences, honest zero states for consolidation and graph, and the official icon as the favicon.
 - `--data-dir` flag and `AGENTMEMORY_DATA_DIR` so iii-engine state lives outside repositories, with gated legacy `./data` adoption and Docker-volume preservation (#314)
 - Native hooks adapter for Droid via `~/.factory/hooks.json`, reusing the bundled hook scripts (#1130)
 - Native hooks adapter for Antigravity CLI (agy) via a stdin bridge that normalizes agy's hook payloads onto the bundled hook scripts, with an explicit PreToolUse allow decision (#1146, thanks @berthojoris)
@@ -28,8 +34,23 @@ Patch release: the `.env` file now actually applies everywhere, imports become s
 - `AGENTMEMORY_PROJECT_NAME` override in the OpenCode plugin (#1125)
 - Provider fetches retry 429/503 honoring `Retry-After` under a total-elapsed budget capped below the iii invocation timeout (#1136)
 
+### Changed
+
+- **Provider default models bumped to current generations.** OpenAI `gpt-4o-mini` → `gpt-5.6-luna`, Anthropic `claude-sonnet-4-20250514` (deprecated upstream, retires 2026-06-15) → `claude-sonnet-5`, Gemini `gemini-2.5-flash` → `gemini-3.7-flash` (current stable Flash), MiniMax `MiniMax-M2.7` → `MiniMax-M3`, OpenRouter `anthropic/claude-sonnet-4-20250514` → `anthropic/claude-sonnet-5`. The premium-model cost warning now also matches OpenAI's Sol flagship tier, and its cheap-alternative hint leads with `deepseek/deepseek-v4-flash-0731`. Explicit `*_MODEL` env overrides are unaffected. Embedding defaults are unchanged (`text-embedding-3-small`, `gemini-embedding-001`, local MiniLM are all current). README local-model picks refreshed to the Qwen 3 / gpt-oss / DeepSeek R1 generation.
+- Local embeddings migrate from `@xenova/transformers` to `@huggingface/transformers` v4 with Node 22+ support; CI now tests Node 20, 22, 24, and 26 (#479, #1096)
+
 ### Fixed
 
+- **Hybrid ranking on the primary recall path.** `mem::search` (behind `memory_recall`) now ranks through the full BM25 + vector + graph fusion when the vector index is populated; it was keyword-only while only smart-search got hybrid ranking. Fusion weights normalize per item over the streams that actually ranked it, with an explicit cross-stream agreement bonus, replacing the every-enabled-stream denominator that permanently penalized single-stream hits. Result order is deterministic (score, best rank, id).
+- **Indexed lesson recall.** Lessons get a dedicated in-memory BM25 index built lazily from one KV list and maintained incrementally on save, delete, and decay; recall previously listed and substring-scanned the whole corpus per query. A record cache beside the index takes recall to zero KV round-trips.
+- **Superseded versions leave recall.** Superseded memory versions are removed from the BM25 and vector indexes; the version chain stays in KV for history, but recall no longer returns an outdated fact as if current. `mem::remember` also finds supersession candidates through the search index (top 50) instead of walking every memory per save, with a full-scan fallback while the index is cold.
+- **`agentId` threads through every save path** ([#1159](https://github.com/rohitg00/agentmemory/issues/1159), [#1160](https://github.com/rohitg00/agentmemory/issues/1160), [#1197](https://github.com/rohitg00/agentmemory/issues/1197)). REST `/agentmemory/remember` forwards `agentId` instead of dropping it; `memoryToObservation()` carries the memory's `agentId` into the search-index shape so saved memories are visible to agent-scoped search; the MCP `memory_save` schema exposes `agentId` and the standalone stdio package forwards both `agentId` and `project`.
+- **Per-session project attribution in the OpenCode plugin** ([#1188](https://github.com/rohitg00/agentmemory/issues/1188)). Project and cwd resolve from each session's own directory at `session.created` (pruned on session end) instead of module-level state that filed every session in a multi-directory OpenCode process under whichever repo loaded the plugin first.
+- **Prompt dedup no longer swallows prompts** ([#1173](https://github.com/rohitg00/agentmemory/issues/1173)). Hooks hash the payload when `tool_input` is absent, so prompt_submit, notification, and lifecycle events dedup on content instead of collapsing onto one shared key that silently dropped every prompt after the first in a TTL window.
+- **Stop hook no longer summarizes twice** ([#1203](https://github.com/rohitg00/agentmemory/issues/1203)). The direct `/agentmemory/summarize` POST is gone; `/session/end` already fans out `event::session::stopped`, which runs `mem::summarize`.
+- **Safe Docker-mode stop** ([#1151](https://github.com/rohitg00/agentmemory/issues/1151)). The CLI refuses to adopt or signal Docker/VM port holders (com.docker.backend, vpnkit, colima) as the native engine unless `--force`; Docker-mode teardown is scoped to agentmemory's own compose services instead of an unscoped `down`; the native worker is reaped before Docker teardown instead of deleting `worker.pid` with the process still running.
+- **Viewer live stream and freshness.** The stream WebSocket target resolves from `/agentmemory/livez` (new `streamsPort` field) instead of viewerPort-1 arithmetic, which pointed at the wrong server whenever the viewer bound a fallback port and silently degraded live updates to polling. Tab data refetches on entry (with a freshness gate), so a memory saved by the agent appears without a hard reload.
+- **Hermetic tests** ([#1178](https://github.com/rohitg00/agentmemory/issues/1178)). HOME/USERPROFILE are isolated for the whole vitest run so suites stop reading the developer's real `~/.agentmemory/.env`.
 - Boot hydrates `~/.agentmemory/.env` into `process.env`, closing the class of "env var in .env is ignored" bugs (#1136)
 - Imported and replayed observations are indexed into BM25 and the vector index, so imports are searchable (#1072, via #1136)
 - Snapshot timer actually runs, non-positive intervals clamp to the default, and snapshot creation is serialized across timer, REST, and MCP (#1006, via #1136)
@@ -47,10 +68,6 @@ Patch release: the `.env` file now actually applies everywhere, imports become s
 - Viewer surfaces health status from non-2xx health responses (#1046)
 - Documented REST endpoint count matches the registered routes again (130)
 
-### Changed
-
-- Local embeddings migrate from `@xenova/transformers` to `@huggingface/transformers` v4 with Node 22+ support; CI now tests Node 20, 22, 24, and 26 (#479, #1096)
-
 ## [0.9.28] — 2026-07-19
 
 Patch release: hardens the hook runner against malformed payloads and closes a cross-agent context leak. No breaking changes; drop-in upgrade.