Merge pull request #4 from sdwolf4103/feat/memory-quality-cleanup

Release v1.4.0 memory quality cleanup

Author: sdwolf4103

.gitignore

@@ -51,3 +51,7 @@ pnpm-lock.yaml
 
 # Superpowers local planning artifacts
 docs/superpowers/plans/
+
+# Local dev/admin script inputs
+scripts/dev/run-migration-roots.local.txt
+scripts/dev/dry-run-roots.local.txt

CHANGELOG.md

@@ -5,6 +5,29 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.4.0] - 2026-04-28
+
+### Added
+
+- Local migration audit log for the `2026-04-28-quality-cleanup` migration:
+  `~/.local/share/opencode-working-memory/migration-logs/2026-04-28-quality-cleanup.jsonl`.
+- Local extraction rejection log for rejected compaction memory candidates:
+  `~/.local/share/opencode-working-memory/extraction-rejections.jsonl`.
+- Sanitized real-workspace regression fixtures for memory cleanup migration behavior.
+- Safe workspace residue cleanup tooling that dry-runs by default and quarantines definite temp/test workspace stores instead of deleting them.
+
+### Changed
+
+- Unified memory quality rules in a shared quality gate for compaction memory candidates and cleanup checks.
+- Rewritten compaction memory prompt to reduce over-production of low-quality memories.
+- Changed quality cleanup migration to be conservative: it supersedes only high-confidence garbage patterns, including progress snapshots, raw errors, commit/CI snapshots, temporary status notes, active file snapshots, code/API signatures, path-heavy entries, and empty entries.
+- Soft heuristic failures (`bad_feedback`, `bad_decision`) are intentionally excluded from automatic migration cleanup to protect durable declarative memories such as branding rules, API facts, release rules, user workflow preferences, and architecture decisions.
+- Isolated test runs under a temporary `XDG_DATA_HOME` so test workspaces no longer pollute real local workspace memory data.
+
+### Recovery note
+
+The cleanup migration changes matching entries to `status: "superseded"`; it does not delete the entry. If a useful memory is superseded, inspect the migration audit log and restore by changing that entry back to `status: "active"` in the workspace's `workspace-memory.json`. The migration runs once per workspace.
+
 ## [1.3.3] - 2026-04-28
 
 ### Fixed

README.md

@@ -174,6 +174,17 @@ It includes guards for:
 
 The goal is to remember durable facts, not every detail.
 
+Historical cleanup is intentionally conservative: extraction-time filtering may reject more aggressively, but one-time migration cleanup only supersedes high-confidence garbage patterns. This protects existing durable memories written in declarative style, such as "API endpoint is X" or "Product branding is Y".
+
+For local development cleanup, use:
+
+```bash
+npm run cleanup:workspaces -- --dry-run
+npm run cleanup:workspaces -- --quarantine
+```
+
+The cleanup command only quarantines definite temp/test workspace residues by default. It does not delete unknown missing-root workspaces.
+
 ## Configuration
 
 OpenCode Working Memory works out of the box.
@@ -210,6 +221,7 @@ cd opencode-working-memory
 npm install
 npm test
 npm run typecheck
+npm run cleanup:workspaces -- --dry-run
 ```
 
 ## Requirements

RELEASE_NOTES.md

@@ -1,5 +1,105 @@
 # Release Notes
 
+## 1.4.0 (2026-04-28)
+
+### Memory Quality Cleanup
+
+This release improves automatic workspace memory quality without risking broad cleanup of useful existing memories.
+
+The quality gate is now shared across compaction extraction and migration checks, the compaction prompt is stricter about what should become durable memory, and the one-time migration is intentionally conservative.
+
+### What Changed
+
+- **Unified quality rules**: memory quality checks now live in one shared module and apply consistently across feedback, decisions, project facts, and references.
+- **Stricter compaction output**: the compaction prompt now tells the model to save fewer memories and prefer durable facts, user preferences, architecture decisions, and hard-to-rediscover references.
+- **Conservative migration cleanup**: the `2026-04-28-quality-cleanup` migration only supersedes high-confidence garbage patterns, not every rejected memory.
+- **Audit logs**: automatic migration cleanup writes local JSONL audit records so superseded entries can be inspected and restored.
+- **Extraction rejection logs**: newly rejected compaction candidates are logged locally to help calibrate future quality rules.
+- **Regression coverage**: migration behavior is tested against sanitized real-workspace patterns to prevent mass false positives from coming back.
+- **Workspace cleanup tooling**: a dev/admin cleanup command can dry-run or quarantine definite temp/test workspace residues without deleting unknown missing-root workspaces.
+- **Test storage isolation**: test runs now use a temporary `XDG_DATA_HOME`, preventing fixture workspaces from polluting real local memory data.
+
+### What Gets Cleaned Up
+
+The migration may supersede existing `source: "compaction"` memories only when they match hard garbage patterns:
+
+- Empty entries
+- Progress snapshots, such as "Wave 1 completed successfully"
+- Test or suite count snapshots, such as "180 tests passed"
+- Raw errors and stack traces
+- Commit or CI snapshots
+- Temporary status notes, such as "Currently running npm test"
+- Active file snapshots
+- Code or API signatures
+- Path-heavy entries that are just rediscoverable file lists
+
+### What Is Protected
+
+The migration does not supersede entries whose only issue is a soft heuristic failure, such as:
+
+- `bad_feedback`
+- `bad_decision`
+
+This protects useful declarative memories like:
+
+- Product branding rules
+- API facts
+- Release rules
+- Architecture decisions
+- User workflow preferences
+
+Explicit and manual memories are also protected.
+
+### Migration Behavior
+
+- Runs once per workspace.
+- Only affects active `source: "compaction"` entries.
+- Marks matching entries as `status: "superseded"` instead of deleting them.
+- Adds `quality_cleanup` and `quality:<reason>` tags to superseded entries.
+- Writes audit logs to:
+  `~/.local/share/opencode-working-memory/migration-logs/2026-04-28-quality-cleanup.jsonl`
+- Writes extraction rejection logs to:
+  `~/.local/share/opencode-working-memory/extraction-rejections.jsonl`
+
+### Recovery
+
+If a useful memory is superseded, inspect the migration audit log and restore the entry by changing its status back to `"active"` in the workspace's `workspace-memory.json`.
+
+### Workspace Residue Cleanup
+
+If old test/temp workspace stores already exist locally, inspect them first:
+
+```bash
+npm run cleanup:workspaces -- --dry-run
+```
+
+To move definite temp/test residues into a local quarantine folder instead of deleting them:
+
+```bash
+npm run cleanup:workspaces -- --quarantine
+```
+
+The cleanup command skips existing workspace roots and unknown missing-root workspaces by default.
+
+### Upgrade Notes
+
+- No configuration changes required.
+- Existing workspace memory files remain compatible.
+- The OpenCode config entry stays the same:
+
+```json
+{
+  "plugin": ["opencode-working-memory"]
+}
+```
+
+### Validation
+
+- `npm test`
+- `npm run typecheck`
+
+---
+
 ## 1.3.2 (2026-04-27)
 
 ### CI Compatibility Patch

package.json

@@ -1,6 +1,6 @@
 {
   "name": "opencode-working-memory",
-  "version": "1.3.3",
+  "version": "1.4.0",
   "description": "Three-layer memory architecture for OpenCode with workspace memory and hot session state",
   "type": "module",
   "main": "index.ts",
@@ -16,7 +16,8 @@
   "scripts": {
     "build": "node -e \"console.log('No build step required: OpenCode loads index.ts directly')\"",
     "typecheck": "tsc --noEmit",
-    "test": "node --test --experimental-strip-types tests/*.test.ts",
+    "test": "node --import ./tests/setup-xdg-data-home.ts --test --experimental-strip-types tests/*.test.ts",
+    "cleanup:workspaces": "node --experimental-strip-types scripts/dev/cleanup-workspaces.ts",
     "check:compat": "npm install --no-save @opencode-ai/plugin@latest && npm run typecheck && npm test"
   },
   "keywords": [

scripts/dev/cleanup-workspaces.ts

@@ -0,0 +1,100 @@
+#!/usr/bin/env node
+/**
+ * Safely inspect or quarantine stale test/temp workspace memory stores.
+ *
+ * Default mode is dry-run. Quarantine moves only definite temp/test residues.
+ * Unknown missing roots are reported but skipped unless --include-orphans is set.
+ */
+
+import { cleanupWorkspaceResidues } from "../../src/workspace-cleanup.ts";
+
+type CliOptions = {
+  mode: "dry-run" | "quarantine";
+  dataHome?: string;
+  olderThanDays?: number;
+  includeOrphans: boolean;
+};
+
+function usage(): string {
+  return `Usage:
+  npm run cleanup:workspaces -- --dry-run
+  npm run cleanup:workspaces -- --quarantine
+  npm run cleanup:workspaces -- --quarantine --older-than-days 1
+
+Options:
+  --dry-run             List candidates without moving anything (default)
+  --quarantine          Move definite temp/test residues to quarantine
+  --data-home <path>    Override XDG data home for testing/admin work
+  --older-than-days <n> Only consider workspace dirs older than n days
+  --include-orphans     Also quarantine missing non-temp roots (off by default)
+  --help                Show this help
+`;
+}
+
+function parseArgs(argv: string[]): CliOptions {
+  const options: CliOptions = { mode: "dry-run", includeOrphans: false };
+
+  for (let i = 0; i < argv.length; i++) {
+    const arg = argv[i];
+    switch (arg) {
+      case "--dry-run":
+        options.mode = "dry-run";
+        break;
+      case "--quarantine":
+        options.mode = "quarantine";
+        break;
+      case "--data-home":
+        options.dataHome = argv[++i];
+        if (!options.dataHome) throw new Error("--data-home requires a path");
+        break;
+      case "--older-than-days": {
+        const value = Number(argv[++i]);
+        if (!Number.isFinite(value) || value < 0) throw new Error("--older-than-days requires a non-negative number");
+        options.olderThanDays = value;
+        break;
+      }
+      case "--include-orphans":
+        options.includeOrphans = true;
+        break;
+      case "--help":
+      case "-h":
+        console.log(usage());
+        process.exit(0);
+      default:
+        throw new Error(`Unknown option: ${arg}\n${usage()}`);
+    }
+  }
+
+  return options;
+}
+
+const options = parseArgs(process.argv.slice(2));
+const result = await cleanupWorkspaceResidues({
+  dataHome: options.dataHome,
+  mode: options.mode,
+  includeOrphans: options.includeOrphans,
+  minAgeMs: options.olderThanDays === undefined ? undefined : options.olderThanDays * 24 * 60 * 60 * 1_000,
+});
+
+console.log(`Mode: ${result.mode}`);
+console.log(`Scanned: ${result.results.length}`);
+console.log(`Candidates: ${result.candidates.length}`);
+
+if (result.candidates.length > 0) {
+  console.log("\nCandidates:");
+  for (const candidate of result.candidates) {
+    console.log(`- ${candidate.workspaceKey} ${candidate.classification} root=${candidate.root ?? "<missing>"}`);
+    console.log(`  reasons=${candidate.reasons.join(",")}`);
+  }
+}
+
+if (result.quarantined.length > 0) {
+  console.log(`\nQuarantined: ${result.quarantined.length}`);
+  console.log(`Quarantine dir: ${result.quarantineDir}`);
+}
+
+const unknownOrphans = result.results.filter(item => item.classification === "orphan_unknown");
+if (unknownOrphans.length > 0 && !options.includeOrphans) {
+  console.log(`\nUnknown missing-root workspaces skipped: ${unknownOrphans.length}`);
+  console.log("Use --include-orphans only after manually confirming they are safe to quarantine.");
+}

scripts/dev/run-migration.ts

@@ -0,0 +1,50 @@
+/**
+ * Local helper to trigger migration on workspace roots.
+ *
+ * Usage:
+ *   MIGRATION_RUN_ROOTS=/path/a:/path/b bun run scripts/dev/run-migration.ts
+ *
+ * Or create a local file (gitignored):
+ *   echo "/path/to/workspace1" > scripts/dev/run-migration-roots.local.txt
+ *   echo "/path/to/workspace2" >> scripts/dev/run-migration-roots.local.txt
+ *   bun run scripts/dev/run-migration.ts
+ */
+
+import { existsSync } from "node:fs";
+import { readFile } from "node:fs/promises";
+import { join } from "node:path";
+import { loadWorkspaceMemory } from "../../src/workspace-memory.ts";
+
+async function getRoots(): Promise<string[]> {
+  // Priority 1: environment variable
+  const envRoots = process.env.MIGRATION_RUN_ROOTS;
+  if (envRoots) {
+    return envRoots.split(":").filter(root => root.length > 0);
+  }
+
+  // Priority 2: local file
+  const localFile = join(import.meta.dirname, "run-migration-roots.local.txt");
+  if (existsSync(localFile)) {
+    const content = await readFile(localFile, "utf8");
+    return content.trim().split("\n").filter(root => root.length > 0);
+  }
+
+  // No roots configured
+  console.log("No workspace roots configured.");
+  console.log("Set MIGRATION_RUN_ROOTS=/path/a:/path/b or create run-migration-roots.local.txt");
+  return [];
+}
+
+const roots = await getRoots();
+
+if (roots.length === 0) {
+  process.exit(0);
+}
+
+for (const root of roots) {
+  console.log(`Loading workspace memory: ${root}`);
+  const store = await loadWorkspaceMemory(root);
+  const active = store.entries.filter(entry => entry.status !== "superseded").length;
+  const superseded = store.entries.filter(entry => entry.status === "superseded").length;
+  console.log(`  active=${active} superseded=${superseded} migrations=${(store.migrations ?? []).join(",")}`);
+}