Skip to content

Commit 60b9ca7

Browse files
committed
fix(memory): isolate test workspace cleanup
1 parent 8da39c7 commit 60b9ca7

14 files changed

Lines changed: 692 additions & 70 deletions

.gitignore

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -52,5 +52,6 @@ pnpm-lock.yaml
5252
# Superpowers local planning artifacts
5353
docs/superpowers/plans/
5454

55-
# Local migration dry-run roots
55+
# Local dev/admin script inputs
56+
scripts/dev/run-migration-roots.local.txt
5657
scripts/dev/dry-run-roots.local.txt

CHANGELOG.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,15 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
1414
- Local extraction rejection log for rejected compaction memory candidates:
1515
`~/.local/share/opencode-working-memory/extraction-rejections.jsonl`.
1616
- Sanitized real-workspace regression fixtures for memory cleanup migration behavior.
17+
- Safe workspace residue cleanup tooling that dry-runs by default and quarantines definite temp/test workspace stores instead of deleting them.
1718

1819
### Changed
1920

2021
- Unified memory quality rules in a shared quality gate for compaction memory candidates and cleanup checks.
2122
- Rewritten compaction memory prompt to reduce over-production of low-quality memories.
2223
- Changed quality cleanup migration to be conservative: it supersedes only high-confidence garbage patterns, including progress snapshots, raw errors, commit/CI snapshots, temporary status notes, active file snapshots, code/API signatures, path-heavy entries, and empty entries.
2324
- Soft heuristic failures (`bad_feedback`, `bad_decision`) are intentionally excluded from automatic migration cleanup to protect durable declarative memories such as branding rules, API facts, release rules, user workflow preferences, and architecture decisions.
25+
- Isolated test runs under a temporary `XDG_DATA_HOME` so test workspaces no longer pollute real local workspace memory data.
2426

2527
### Recovery note
2628

README.md

Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -176,6 +176,15 @@ The goal is to remember durable facts, not every detail.
176176

177177
Historical cleanup is intentionally conservative: extraction-time filtering may reject more aggressively, but one-time migration cleanup only supersedes high-confidence garbage patterns. This protects existing durable memories written in declarative style, such as "API endpoint is X" or "Product branding is Y".
178178

179+
For local development cleanup, use:
180+
181+
```bash
182+
npm run cleanup:workspaces -- --dry-run
183+
npm run cleanup:workspaces -- --quarantine
184+
```
185+
186+
The cleanup command only quarantines definite temp/test workspace residues by default. It does not delete unknown missing-root workspaces.
187+
179188
## Configuration
180189

181190
OpenCode Working Memory works out of the box.
@@ -212,6 +221,7 @@ cd opencode-working-memory
212221
npm install
213222
npm test
214223
npm run typecheck
224+
npm run cleanup:workspaces -- --dry-run
215225
```
216226

217227
## Requirements

RELEASE_NOTES.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -16,6 +16,8 @@ The quality gate is now shared across compaction extraction and migration checks
1616
- **Audit logs**: automatic migration cleanup writes local JSONL audit records so superseded entries can be inspected and restored.
1717
- **Extraction rejection logs**: newly rejected compaction candidates are logged locally to help calibrate future quality rules.
1818
- **Regression coverage**: migration behavior is tested against sanitized real-workspace patterns to prevent mass false positives from coming back.
19+
- **Workspace cleanup tooling**: a dev/admin cleanup command can dry-run or quarantine definite temp/test workspace residues without deleting unknown missing-root workspaces.
20+
- **Test storage isolation**: test runs now use a temporary `XDG_DATA_HOME`, preventing fixture workspaces from polluting real local memory data.
1921

2022
### What Gets Cleaned Up
2123

@@ -63,6 +65,22 @@ Explicit and manual memories are also protected.
6365

6466
If a useful memory is superseded, inspect the migration audit log and restore the entry by changing its status back to `"active"` in the workspace's `workspace-memory.json`.
6567

68+
### Workspace Residue Cleanup
69+
70+
If old test/temp workspace stores already exist locally, inspect them first:
71+
72+
```bash
73+
npm run cleanup:workspaces -- --dry-run
74+
```
75+
76+
To move definite temp/test residues into a local quarantine folder instead of deleting them:
77+
78+
```bash
79+
npm run cleanup:workspaces -- --quarantine
80+
```
81+
82+
The cleanup command skips existing workspace roots and unknown missing-root workspaces by default.
83+
6684
### Upgrade Notes
6785

6886
- No configuration changes required.

package.json

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -16,7 +16,8 @@
1616
"scripts": {
1717
"build": "node -e \"console.log('No build step required: OpenCode loads index.ts directly')\"",
1818
"typecheck": "tsc --noEmit",
19-
"test": "node --test --experimental-strip-types tests/*.test.ts",
19+
"test": "node --import ./tests/setup-xdg-data-home.ts --test --experimental-strip-types tests/*.test.ts",
20+
"cleanup:workspaces": "node --experimental-strip-types scripts/dev/cleanup-workspaces.ts",
2021
"check:compat": "npm install --no-save @opencode-ai/plugin@latest && npm run typecheck && npm test"
2122
},
2223
"keywords": [

scripts/dev/cleanup-workspaces.ts

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
#!/usr/bin/env node
2+
/**
3+
* Safely inspect or quarantine stale test/temp workspace memory stores.
4+
*
5+
* Default mode is dry-run. Quarantine moves only definite temp/test residues.
6+
* Unknown missing roots are reported but skipped unless --include-orphans is set.
7+
*/
8+
9+
import { cleanupWorkspaceResidues } from "../../src/workspace-cleanup.ts";
10+
11+
type CliOptions = {
12+
mode: "dry-run" | "quarantine";
13+
dataHome?: string;
14+
olderThanDays?: number;
15+
includeOrphans: boolean;
16+
};
17+
18+
function usage(): string {
19+
return `Usage:
20+
npm run cleanup:workspaces -- --dry-run
21+
npm run cleanup:workspaces -- --quarantine
22+
npm run cleanup:workspaces -- --quarantine --older-than-days 1
23+
24+
Options:
25+
--dry-run List candidates without moving anything (default)
26+
--quarantine Move definite temp/test residues to quarantine
27+
--data-home <path> Override XDG data home for testing/admin work
28+
--older-than-days <n> Only consider workspace dirs older than n days
29+
--include-orphans Also quarantine missing non-temp roots (off by default)
30+
--help Show this help
31+
`;
32+
}
33+
34+
function parseArgs(argv: string[]): CliOptions {
35+
const options: CliOptions = { mode: "dry-run", includeOrphans: false };
36+
37+
for (let i = 0; i < argv.length; i++) {
38+
const arg = argv[i];
39+
switch (arg) {
40+
case "--dry-run":
41+
options.mode = "dry-run";
42+
break;
43+
case "--quarantine":
44+
options.mode = "quarantine";
45+
break;
46+
case "--data-home":
47+
options.dataHome = argv[++i];
48+
if (!options.dataHome) throw new Error("--data-home requires a path");
49+
break;
50+
case "--older-than-days": {
51+
const value = Number(argv[++i]);
52+
if (!Number.isFinite(value) || value < 0) throw new Error("--older-than-days requires a non-negative number");
53+
options.olderThanDays = value;
54+
break;
55+
}
56+
case "--include-orphans":
57+
options.includeOrphans = true;
58+
break;
59+
case "--help":
60+
case "-h":
61+
console.log(usage());
62+
process.exit(0);
63+
default:
64+
throw new Error(`Unknown option: ${arg}\n${usage()}`);
65+
}
66+
}
67+
68+
return options;
69+
}
70+
71+
const options = parseArgs(process.argv.slice(2));
72+
const result = await cleanupWorkspaceResidues({
73+
dataHome: options.dataHome,
74+
mode: options.mode,
75+
includeOrphans: options.includeOrphans,
76+
minAgeMs: options.olderThanDays === undefined ? undefined : options.olderThanDays * 24 * 60 * 60 * 1_000,
77+
});
78+
79+
console.log(`Mode: ${result.mode}`);
80+
console.log(`Scanned: ${result.results.length}`);
81+
console.log(`Candidates: ${result.candidates.length}`);
82+
83+
if (result.candidates.length > 0) {
84+
console.log("\nCandidates:");
85+
for (const candidate of result.candidates) {
86+
console.log(`- ${candidate.workspaceKey} ${candidate.classification} root=${candidate.root ?? "<missing>"}`);
87+
console.log(` reasons=${candidate.reasons.join(",")}`);
88+
}
89+
}
90+
91+
if (result.quarantined.length > 0) {
92+
console.log(`\nQuarantined: ${result.quarantined.length}`);
93+
console.log(`Quarantine dir: ${result.quarantineDir}`);
94+
}
95+
96+
const unknownOrphans = result.results.filter(item => item.classification === "orphan_unknown");
97+
if (unknownOrphans.length > 0 && !options.includeOrphans) {
98+
console.log(`\nUnknown missing-root workspaces skipped: ${unknownOrphans.length}`);
99+
console.log("Use --include-orphans only after manually confirming they are safe to quarantine.");
100+
}
Lines changed: 7 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,12 +2,12 @@
22
* Local helper to trigger migration on workspace roots.
33
*
44
* Usage:
5-
* MIGRATION_DRY_RUN_ROOTS=/path/a:/path/b bun run scripts/dev/dry-run-migration.ts
5+
* MIGRATION_RUN_ROOTS=/path/a:/path/b bun run scripts/dev/run-migration.ts
66
*
77
* Or create a local file (gitignored):
8-
* echo "/path/to/workspace1" > scripts/dev/dry-run-roots.local.txt
9-
* echo "/path/to/workspace2" >> scripts/dev/dry-run-roots.local.txt
10-
* bun run scripts/dev/dry-run-migration.ts
8+
* echo "/path/to/workspace1" > scripts/dev/run-migration-roots.local.txt
9+
* echo "/path/to/workspace2" >> scripts/dev/run-migration-roots.local.txt
10+
* bun run scripts/dev/run-migration.ts
1111
*/
1212

1313
import { existsSync } from "node:fs";
@@ -17,21 +17,21 @@ import { loadWorkspaceMemory } from "../../src/workspace-memory.ts";
1717

1818
async function getRoots(): Promise<string[]> {
1919
// Priority 1: environment variable
20-
const envRoots = process.env.MIGRATION_DRY_RUN_ROOTS;
20+
const envRoots = process.env.MIGRATION_RUN_ROOTS;
2121
if (envRoots) {
2222
return envRoots.split(":").filter(root => root.length > 0);
2323
}
2424

2525
// Priority 2: local file
26-
const localFile = join(import.meta.dirname, "dry-run-roots.local.txt");
26+
const localFile = join(import.meta.dirname, "run-migration-roots.local.txt");
2727
if (existsSync(localFile)) {
2828
const content = await readFile(localFile, "utf8");
2929
return content.trim().split("\n").filter(root => root.length > 0);
3030
}
3131

3232
// No roots configured
3333
console.log("No workspace roots configured.");
34-
console.log("Set MIGRATION_DRY_RUN_ROOTS=/path/a:/path/b or create dry-run-roots.local.txt");
34+
console.log("Set MIGRATION_RUN_ROOTS=/path/a:/path/b or create run-migration-roots.local.txt");
3535
return [];
3636
}
3737

src/extractors.ts

Lines changed: 2 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,7 @@ import type { ActiveFile, LongTermMemoryEntry, LongTermType, OpenError } from ".
55
import { LONG_TERM_LIMITS } from "./types.ts";
66
import { assessMemoryQuality } from "./memory-quality.ts";
77
import { extractionRejectionLogPath } from "./paths.ts";
8+
import { redactCredentials } from "./redaction.ts";
89

910
function id(prefix: string): string {
1011
return `${prefix}_${Date.now()}_${Math.random().toString(36).slice(2, 8)}`;
@@ -248,15 +249,6 @@ async function logExtractionRejection(entry: ExtractionRejectionLogEntry): Promi
248249
}
249250
}
250251

251-
function redactSensitiveText(text: string): string {
252-
return text
253-
.replace(/bearer\s+[a-zA-Z0-9._-]+/gi, "bearer [REDACTED]")
254-
.replace(/token[=:]\s*[a-zA-Z0-9._-]+/gi, "token=[REDACTED]")
255-
.replace(/password[=:]\s*[a-zA-Z0-9._-]+/gi, "password=[REDACTED]")
256-
.replace(/secret[=:]\s*[a-zA-Z0-9._-]+/gi, "secret=[REDACTED]")
257-
.replace(/api[-_]?key[=:]\s*[a-zA-Z0-9._-]+/gi, "api_key=[REDACTED]");
258-
}
259-
260252
function shouldAcceptWorkspaceMemoryCandidate(
261253
entry: {
262254
type: LongTermType;
@@ -287,7 +279,7 @@ function shouldAcceptWorkspaceMemoryCandidate(
287279
void logExtractionRejection({
288280
timestamp: new Date().toISOString(),
289281
type: entry.type,
290-
text: redactSensitiveText(text),
282+
text: redactCredentials(text),
291283
reasons: quality.reasons,
292284
source: "compaction",
293285
});

src/redaction.ts

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
/**
2+
* Shared redaction utilities for sensitive credential patterns.
3+
* Used by both workspace memory normalization and extraction rejection logging.
4+
*/
5+
6+
// Password labels in multiple languages
7+
const PASSWORD_LABELS = /password|passwd|pwd|||||contraseña|mot de passe|passwort/i;
8+
9+
// Username labels in multiple languages
10+
const USERNAME_LABELS = /username|user name|||||usuario|utilisateur|benutzer/i;
11+
12+
// Sensitive key labels
13+
const SENSITIVE_LABELS = /api[_-]?key|token|bearer|secret|credential|auth|auth[_-]?key|private[_-]?key/i;
14+
15+
// Secret value pattern (excludes common delimiters and brackets)
16+
const SECRET_VALUE = String.raw`[^` + "`" + String.raw`'",,,\s\[]+`;
17+
18+
// Prefix patterns for different credential types
19+
const PIN_PREFIX = String.raw`(\bPIN\b(?:\s*(?:是|=|:|:)\s*|\s+(?![是=::])))`;
20+
const PASSWORD_PREFIX = String.raw`((?:${PASSWORD_LABELS.source})(?:\s*(?:是|=|:|:)\s*|\s+(?![是=::])))`;
21+
const USERNAME_PREFIX = String.raw`((?:${USERNAME_LABELS.source})(?:\s*(?:是|=|:|:)\s*|\s+(?![是=::])))`;
22+
const SENSITIVE_PREFIX = String.raw`((?:${SENSITIVE_LABELS.source})(?:\s*(?:推|是|=|:|:)\s*|[::]\s*))`;
23+
const BEARER_PREFIX = String.raw`(Bearer\s+)`;
24+
25+
/**
26+
* Redacts sensitive credentials from text.
27+
* Handles:
28+
* - PINs in multiple formats
29+
* - Username/password pairs
30+
* - Standalone passwords
31+
* - Bearer tokens
32+
* - API keys, secrets, credentials, auth tokens, private keys
33+
*
34+
* Supports multiple languages and delimiters (ASCII and CJK).
35+
*/
36+
export function redactCredentials(text: string): string {
37+
let result = text;
38+
39+
// 1. PIN
40+
result = result.replace(
41+
new RegExp(String.raw`${PIN_PREFIX}[\`'"]?(${SECRET_VALUE})`, "gi"),
42+
"$1[REDACTED]",
43+
);
44+
45+
// 2. Username+password pair
46+
result = result.replace(
47+
new RegExp(
48+
String.raw`${USERNAME_PREFIX}[\`'"]?(${SECRET_VALUE})((?:,|,)\s*)${PASSWORD_PREFIX}[\`'"]?(${SECRET_VALUE})`,
49+
"gi",
50+
),
51+
"$1[REDACTED]$3$4[REDACTED]",
52+
);
53+
54+
// 3. Standalone password
55+
result = result.replace(
56+
new RegExp(String.raw`${PASSWORD_PREFIX}[\`'"]?(${SECRET_VALUE})`, "gi"),
57+
"$1[REDACTED]",
58+
);
59+
60+
// 4. Bearer tokens (but not "bearer token:" labels)
61+
result = result.replace(
62+
new RegExp(String.raw`${BEARER_PREFIX}(?!token\s*[:=:])[\`'"]?(${SECRET_VALUE})`, "gi"),
63+
"$1[REDACTED]",
64+
);
65+
66+
// 5. Sensitive keys/tokens
67+
result = result.replace(
68+
new RegExp(String.raw`${SENSITIVE_PREFIX}[\`'"]?(${SECRET_VALUE})`, "gi"),
69+
"$1[REDACTED]",
70+
);
71+
72+
return result;
73+
}

0 commit comments

Comments
 (0)