feat: SessionWeaver Phase 2 retrofit (B-OS → B5) [integration] - #18
Draft
NetDevAutomate wants to merge 31 commits into
Draft
feat: SessionWeaver Phase 2 retrofit (B-OS → B5) [integration]#18NetDevAutomate wants to merge 31 commits into
NetDevAutomate wants to merge 31 commits into
Conversation
Freeze the SessionWeaver Phase 2 retrofit's spec and design ahead of any Phase B code, per EXECUTION-ERRATA.md correction #3 and council ruling R2 ("acceptance is not spec-check alone"). The design document normatively fixes the cross-machine standing order for replicated concept lifecycle events and its two-copy test matrix, the v48/v49 migration contracts and their rollback strategy, the non-fatal ontology refresh-failure seam, seed sanitization, the byte-for-byte memory_recall contract, the fresh-install scope diagnostic, and the ConceptService compatibility seam -- so B1-B6 can implement against a reviewed contract instead of inventing one under implementation pressure. Delta specs add Gherkin-scenario requirements to six existing capabilities (harness-session-memory, data-store-and-sync, mcp-server, session-export, health-and-diagnostics, configuration-and-secrets); no capability is newly created. openspec validate --specs --all: 25 passed, 0 failed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ange - proposal.md: the request_scope() call-site list has eight entries; the source plan mislabels it "seven" (design.md and tasks.md already said eight). - design.md: B3 must test whether the reference allocator's table-wide MAX(logical_time) already satisfies two-copy matrix item 4 before adding an explicit advance-on-import step. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…and diagnostics Lift SessionWeaver's frozen tier-1 ontology extraction/rebuild/status module (extraction version tier1-v2-canonical-messages, same logical-hash algorithm, so A2's baseline stays comparable). Migration v48 installs the six schema objects additively; install_schema is tolerant of a pre-existing ad hoc ontology schema (found on the real sessions.db) so a real upgrade converges instead of crashing -- the first rebuild replaces it regardless of its prior shape. Wire an incremental ontology refresh into export_sessions._run_export, behind a named, monkeypatchable seam (refresh_ontology_after_export) called after the capture commit: a refresh failure is caught, logged as a structured ontology_refresh_failed warning, surfaced in the export summary, and never rolls back captured sessions (EXECUTION-ERRATA.md #7). Add `session-maint ontology-rebuild [--incremental]` and `ontology-status` for manual/maintenance recovery. Add the sync boundary: ontology tables never join SYNC_TABLES or GLOBAL_SYNC_TABLES (positive-control tested), and sync._seed_remote_db strips every ontology row from a seed snapshot before scp so a freshly seeded remote derives its own ontology rather than inheriting the source's. Add a report-only "harness" doctor check (present/coverage/freshness/ extraction-version), never fatal, with a fixture-inserted red-path test. Add the ontology_live safety harness (Online Backup only, real DB never touched, new opt-in live_ontology pytest marker) and R7 migration-safety tests: fresh creation, interrupted-migration recovery, downgrade, and idempotent re-migration. Retains real acceptance receipts (docs/data/) from a live Online Backup run: coverage 100%, zero integrity violations, identical hash across two full rebuilds and one incremental no-op, cold rebuild 3.47s. Fixes a v47-pinned assertion in test_replica_coordinator.py that this version bump would otherwise break, and regenerates .secrets.baseline (detect-secrets 1.5.0) for the two evidence files' SHA-256 hashes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ed scope
A virgin install crashed or leaked ad-hoc errors the moment any scope-dependent
path ran: `studyloop study` propagated a bare ScopeError as a generic exit-1
ClickException, each of the seven unguarded request_scope() MCP tool sites let
FastMCP wrap it in ad-hoc text, and session-db-mcp's open_context()/
_get_connection() let sqlite3's "unable to open database file" leak through a
distinct, unhelpful error shape on a database that simply doesn't exist yet.
Add ScopeUnconfiguredError (a ScopeError subclass identifying exactly the
"nothing configured" case) and scope_setup_diagnostic(), one shared
{code: "scope_unconfigured", message, remediation} shape. Every entry point
that can hit it now reports that same payload instead of a traceback: the
CLI exits 2, both MCP servers' tool-call boundaries return isError with the
JSON payload embedded in the text content, and a missing database is treated
identically to a missing scope rather than as its own error class.
Both packages' config writers (generate_default_config(), ensure_config_dir())
now write memory.default_scope: unclassified explicitly for a brand-new
config.yaml, so a fresh install is immediately usable. The runtime default for
an absent file or a file that omits the key stays unset (errata #9) -- this
changes only what a *generated* file contains, not what an absent setting means.
Tests: packages/studyloop/tests/test_fresh_install_scope.py drives a virgin
HOME (no STUDYLOOP_CONFIG, no SESSION_CONTEXT_SCOPE) as a real subprocess
against the source tree -- the CLI, all seven MCP tool sites, and
session-db-mcp's session_search/memory_search -- so this suite's own
config-isolation fixtures cannot hide the defect the way the existing fixture
already did. test_fresh_install_scope_installed.py (marked integration)
repeats every check against a real `uv build` wheel install per plan ruling
R10. Unit tests cover scope_setup_diagnostic, ScopeUnconfiguredError, the two
config writers' round trip, and open_context()/_get_connection() on a missing
database.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Add a "Tier-1 ontology (derived, never synced)" section to docs/session-memory.md covering what it captures, why it never syncs, the automatic export-time refresh, and session-maint ontology-rebuild/ontology-status for manual recovery. Record the feature under CHANGELOG.md's [Unreleased] Added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Mark tasks.md's B2 tasks 2.1 (migration v48 + lift), 2.2 (refresh hook + session-maint ontology-rebuild), and 2.3 (sync boundary + seed sanitization) complete. Leave 2.4 unchecked with an inline note: its tests/hash/timing/count verification and a green preflight are done, but code/build-ontology.py's retirement is explicitly B6's (task-B2-brief.md) and the independent review is the reviewer's step. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Task B1 of openspec/changes/sessionweaver-phase2-retrofit: generated config writes memory.default_scope: unclassified; one structured ScopeError diagnostic at the CLI and every MCP tool in both servers; missing-database path covered; virgin-HOME tests on source-tree and installed-package paths. Review: .superpowers/sdd/sessionweaver-phase2/task-B1-review.md (0 findings). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…d agents CI run 34167164262 (jobs test (3.12) and test (3.13)) failed test_fresh_install_scope.py::test_studyloop_study_exits_2_with_the_diagnostic_on_a_virgin_home because on the GitHub runner `studyloop study 'Test Topic'` exits 1 with "No AI agent found. Install one of: Kiro CLI, Codex, Claude Code, OpenCode, or pi" before the scope check ever runs. session/start.py calls detect_agents() (studyloop.agent_launcher) before start_study_session() raises ScopeUnconfiguredError, so the expected exit 2 + structured scope diagnostic is unreachable unless some agent binary is resolvable via shutil.which on the subprocess's PATH. Locally the test only passed because this machine has agent CLIs installed. Fix the test, not production code: add a `_fake_agent_bin()` helper that writes a no-op executable named `claude` (matching studyloop.adapters.claude.ADAPTER.binary) into a throwaway bin dir and prepends it to the subprocess PATH via `_usable_path()`/`_virgin_env()`'s new `agent_bin` parameter. detect_agents() only calls shutil.which(), so the script is never actually executed -- start_study_session() still raises ScopeUnconfiguredError immediately after agent selection, well before any launch command is built. Applied to both test_studyloop_study_exits_2_with_the_diagnostic_on_a_virgin_home and its installed-wheel sibling in test_fresh_install_scope_installed.py, which had the same dependency. Reproduced RED locally by running both files with a PATH stripped of every real agent CLI (venv bin + tmux + /usr/bin:/bin only), then confirmed GREEN with that same stripped PATH after the fix. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Replace permission-bit failure injection with the backup helper's real None failure contract. This keeps the abort-before-stream invariant reliable under both ordinary users and root CI containers.
Preserve direct coverage that a nonzero SSH/sqlite command result is translated into None after the push-contract test moved to the orchestration seam.
Bring the independently reviewed migration v48, deterministic ontology evidence, and CI-parity sync test correction into the SessionWeaver Phase 2 integration branch. # Conflicts: # .secrets.baseline # docs/session-memory.md
Lift SessionWeaver's concept_schema.py unchanged (SCHEMA_VERSION = 2, SCHEMA_FINGERPRINT byte-identical to the reference, af95685e6e39e166148006519862bee3be1a15219d76772236a82890fe11011d) and add migrate_v49, which installs exactly that DDL: context_concepts, context_concept_events, context_concept_clock, context_concept_fts and context_concept_schema, plus their indexes/triggers and the two context_citations guard triggers. UPSTREAM_SCHEMA_VERSION is pinned to 49, the migration number that installs the sidecar. Why: tasks.md 3.1 -- B3 owns v49 (v48 stays B2's); the sidecar must be additive-only, with context_assertions.proposed_state left as execution state (errata #3), and adopt a byte-identical pre-existing PoC sidecar while refusing drift, because sidecar rows are authored data. Migration-safety per B2's R7 pattern: fresh create to v49, additive-only shape check on context_assertions, exact-adopt, drift refusal, interrupted-migration recovery, repeated-open idempotence, downgrade contract, and a real Online Backup v47->v49 upgrade retained as docs/data/concept-sidecar-migration-v49-receipt.json (aggregates only; source sentinels asserted unchanged; live DB still v47, 5813 sessions). The detect-secrets baseline is regenerated for the receipt's hex-hash false positives, matching B2's precedent. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ection Lift SessionWeaver v0.2.0's concepts.py, winddown.py, okf.py (as okf_import.py), projection.py (A3b2's fixed Publisher with its identity-guarded rollback), authorization.py and safe_fs.py into agent_session_tools.context, adapting only import paths to the package (relative imports; okf -> okf_import). All frozen contracts are kept byte-identical: the sessionweaver://session/ source-URI prefix, the session_weaver_projection frontmatter key, the projection marker and manifest filenames, and ConceptService's public surface (project, winddown, transition, bind_legacy, import_okf). Why: tasks.md 3.1 -- B4 consumes ConceptService as a caller, never a second implementation, and B6 deletes the SessionWeaver copies, so the lifted behaviour must be the reference behaviour, provable by the reference's own tests. Tests: the reference suites lifted with the same import rewrites (223 tests: schema exactness/adoption, wind-down parsing, lifecycle transitions, bind closure, OKF scan/classification precedence, projection publish/rollback, adversarial SQL integrity). Two adaptations, both consequences of v49 installing the sidecar up front: the upstream-version pin asserts 49, and a rejected import call now asserts zero sidecar rows instead of no sidecar table. One targeted nosec B506 documents that _StrictSafeLoader narrows yaml.SafeLoader. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Concept roots and their complete append-only event history join the context replication protocol: snapshot selection follows the authorization seam's two visibility shapes (bound roots travel with their assertion's citation closure, legacy roots with their claimed session), the apply path inserts only rows it does not already have, and the receiving side's own schema triggers re-validate every root. The local clock, the FTS read model and the schema marker never travel. Standing is exactly the frozen order (design.md, ruling R2): standing = max(events, key=(lamport, machine_id, event_id)) with machine_id = context_access_state.instance and lamport = logical_time. The three read-model sites drop the reference's standing-kind precedence, which contradicted the frozen pure-triple order under replication ties while being locally unobservable (local lamports are strictly increasing); the lifted total-order test is adapted to the frozen order with the reasoning recorded inline. Duplicate machine_id is diagnosed and refused, never merged: negotiate() already refuses two live peers with one instance, and the new ConceptReplicaIdentityError refuses incoming events that claim the receiver's own instance for history it never wrote, or that bind one (origin_instance, origin_seq) slot to two different events. Lamport advance on import (design.md's B3 verification note): matrix item 4 was run against the unmodified reference allocator first, and passes -- _allocate's table-wide MAX(logical_time) already advances the next local event past every imported lamport, so no explicit advance-on-import step was added. Also wired: v49 adds the three replica-content-generation triggers for the two replicated tables (v46 froze its own list); staged transport indexes both tables by id; the legacy-sync guard treats the sidecar's seeded singletons and FTS shadow tables as bookkeeping, not populated context; tiering's archive-completeness proof excludes the per-database clock and concept-FTS shadow tables while keeping roots/events in the retention proof; compaction seeds a fresh clock/marker and copies roots after their citation closure so the sidecar triggers re-validate. Tests: the normative two-copy matrix (7 scenarios) on fixtures in both replication orders, replay idempotence, post-convergence Lamport advance, read-model hash equivalence, concurrent accept/retire resolved to the computed winner in either order, causal accept-then-retire, and three duplicate-identity refusals; plus the same matrix once on two real Online Backup copies (live_concepts marker, copy B honestly re-identified before any concept history exists, source sentinels asserted unchanged). Full package suite: 1999 passed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…MCP tool session-context gains winddown --session ID --from FILE|--stdin and concept accept|retire ID --reason, concept bind LEGACY_ID --from FILE --reason, concept import-okf DIR [--dry-run] [--report PATH], and concept project --out DIR [--project ID] [--json], all thin wrappers over the frozen ConceptService seam with the reference CLI's bounded input reading (wind-down byte limit, descriptor-anchored non-symlink files), atomic report writes, deterministic JSON payloads, and the reference exit-code contract (2 validation, 1 runtime/write failure). mcp_server.py gains memory_winddown with the same validation: a field-level error list raises a structured ToolError and writes nothing; a valid batch commits atomically. Why: tasks.md 3.2 -- one seam, one validation surface, no second implementation of concept transitions; context_assertions keeps its execution-state vocabulary (verified in the round-trip test). Also frozen (tasks.md 3.4): ConceptService's public API is pinned by an exact-signature regression test (method set, every signature string, and the four result shapes), so B4 consumes a checked contract. This commit also carries the live_concepts harness extensions and their opt-in tests (they type-check as one unit with the CLI): the live two-copy matrix on two real Online Backup copies, and the full 2,033-record legacy OKF import runner whose sanitized report feeds the OpenSpec evidence attachment. Tests: lossless round trip, stdin input, field-level failures for malformed documents/invalid JSON/symlinked input, lifecycle verbs with retired-terminal diagnostics, legacy bind, OKF dry-run/write/report, projection output, MCP happy path and loud structured failure with zero partial writes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…evidence docs/context-memory.md gains the wind-down/concept command reference, the two trust labels (model-proposed vs legacy-unbound, never blended), the frozen cross-machine standing order in one paragraph, and the frozen ConceptService surface table. CHANGELOG records the concept sidecar under [Unreleased] Added. OpenSpec tasks 3.1-3.5 are checked with their evidence pointers (3.5's independent review remains the reviewer's step). The evidence file is added with -f: openspec/ is gitignored for new files, but this change's artifacts are tracked and the brief requires the report attached to the change directory. Evidence: openspec/changes/sessionweaver-phase2-retrofit/evidence/ legacy-okf-import-report.json -- the full legacy OKF import on a real Online Backup (live_concepts marker): 2,035 scanned, 2,033 parseable records imported legacy-unbound with zero write failures and the exact A3b1 baseline FTS content hash (0c036eae...); the two extra files are post-baseline non-records classified invalid_schema (reported, never dropped); dry-run/write classification identical; idempotent re-import already_present=2,033 with zero writes; OKF tree sentinel and source database sentinels unchanged. The visibility sub-classification split differs from the baseline because this run retains every project unclassified on a newer corpus (5,813 sessions vs 5,678); the report retains the split (no_exact_match 1559, no_visible_evidence 429, oversized_evidence 45) for the reviewer. The detect-secrets baseline is regenerated for the evidence hashes. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ence Completes the previous docs commit: the gitignore advice for openspec/ aborted the add chain, so the evidence attachment missed it. openspec/ ignores only new untracked files -- this change's artifacts are tracked, and the brief requires the sanitized legacy OKF import report attached to the change directory, so the evidence file is added with -f. The detect-secrets baseline is regenerated for the report's hex hashes, matching the established false-positive handling. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
pytest resolves rootdir (and its configfile) from the paths it is given, so any package-scoped invocation reads this package's pyproject.toml and never the workspace root's -m exclusions. Round 0 shipped the package config without them, so a plain 'pytest packages/agent-session-tools/...' silently ran the live_concepts suites: two 1 GB SQLite Online Backups of the owner's real sessions.db, a 140-second OKF import, and two committed evidence files rewritten -- the same trap fired during the independent review (B3 review round 1, Important #2). Add the -m exclusion for every opt-in marker this package declares (integration, live_ontology, live_concepts), matching the workspace-root convention and packages/studyloop's MUST-duplicate precedent, and pin it with a subprocess regression that proves package-scoped collection deselects every live-marked test by default while an explicit -m opt-in still selects them and ordinary tests stay selected. Co-Authored-By: Kiro <noreply@kiro.dev>
Round 0's committed migration receipt was captured at commit 1, before migrate_v49 gained its six replica_content_context_concept* triggers, so its whole-database schema_sha256 (5c2f2c56...) no longer described the shipped migration -- and nothing deterministic could catch that, because that hash covers every sqlite_master row and is therefore a property of the source corpus, not the migration (a fresh schema.sql install yields 81f3310a..., the real backup 6dfb4027...) (B3 review round 1, Important #1). Give the receipt a source-independent fingerprint instead: sidecar_migration_fingerprint() hashes the DDL of exactly the objects migrate_v49 installs (context_concept* objects, the two context_citations_bound_* guard triggers, the six replica_content_* triggers; FTS5 shadow tables excluded -- library-generated from the fingerprinted virtual-table declaration), emitted as sidecar_objects_sha256 (evidence_version 1 -> 2). Two always-on fixture regressions pin it: the name selection equals the complete non-shadow v49 delta (so a future migration edit cannot slip an object past the receipt check), and a fresh install of the shipped migration must reproduce the committed receipt's hash byte-for-byte. Receipt regenerated at HEAD on a fresh SQLite Online Backup (live DB untouched: v47, 5,813 sessions); its schema_sha256 is now 6dfb4027..., byte-identical to the review's independently reproduced value. Secrets baseline regenerated for the receipt's changed hex strings only. Co-Authored-By: Kiro <noreply@kiro.dev>
Bring the independently reviewed migration v49, lifted concept lifecycle/wind-down/import/projection, replication proofs, CLI/MCP surface, and full-corpus import evidence into the SessionWeaver Phase 2 integration branch.
Capture the existing public search contract before introducing the shared AND-to-OR planner. The fixture freezes row keys, deterministic ordering, null/default values, phrase/operator behavior, and the 300-character preview boundary.
Reuse the released SessionWeaver planner behind session_search while keeping its public row schema, SQL ranking, filters, and 300-character previews unchanged. Black-box tests prove the frozen cases remain identical and only the previously empty implicit-AND query widens through OR fallback.
Port SessionWeaver v0.2.0 concept-first recall onto StudyLoop's B3 authorization seam and shared query planner. The public MCP tool validates strict bounds, preserves the frozen report schema, deduplicates concept source sessions, and proves through behavioral traces that embeddings and ontology are not consulted.
Have studyloop install agents merge session-db and studyloop into Claude Code, Kiro, and Codex configuration without touching the real HOME. Format-preserving temp-HOME tests prove unrelated entries survive, repeated installation is byte-identical, missing configs are created, and doctor reports registration without mutation.
Repair an incorrect StudyLoop-owned JSON MCP entry by replacing only that value object, then append any missing owned entry. This avoids reserializing unrelated MCP entries while retaining byte-idempotence on subsequent installs.
Retain the exact 25-question corpus, aggregate-only v47 Online Backup evidence, and a reproducible released-reference acceptance runner. Document the frozen recall contract and MCP registration, add positive work/personal scope coverage, and mark OpenSpec B4 complete after a green 5,964-test preflight.
Route explicit phrase and boolean queries through the historical FTS escaping path so the shared recall planner remains frozen. Restrict AND-to-OR widening to empty implicit plain-text searches and pin compatibility with negative controls.
Replace owned JSON values and TOML subtrees by parser-validated byte spans rather than canonical-only regexes. This preserves unrelated configuration, prevents duplicate keys, handles CRLF and nested tables, and converges byte-identically on rerun.
Track TOML lexical state before recognizing candidate MCP table headers. This prevents owned-looking text inside multiline values from truncating unrelated config while retaining byte-preserving subtree repair.
Bring the independently reviewed shared planner, frozen-contract memory_recall, 25-question identity proof, and idempotent Claude/Kiro/Codex MCP registration into the SessionWeaver Phase 2 integration branch.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Draft integration PR for the SessionWeaver Phase 2 retrofit into StudyLoop. It exists so CI runs on every merge into
feat/sessionweaver-phase2-retrofit; it is not ready for review or merge until B5 and Council B complete.Tracking:
openspec/changes/sessionweaver-phase2-retrofit/(proposal, design, tasks). Stages land here as reviewed task branches: B1 fresh-install scope, B2 tier-1 ontology (migration v48), B3 concept lifecycle + replication (v49), B4memory_recall+ planner + MCP registration, B5 real-corpus validation, BL-1..BL-4, docs and ADRs.🤖 Generated with Claude Code