Skip to content

fix(debug-retention): collect key-log-only proxy run dirs from new runs only; existing key logs untouched - #1388

Merged
Juliusolsson05 merged 11 commits into
integration/batch-2026-09-27-rfrom
fix/retention-collects-keylog-run-dirs
Sep 27, 2026
Merged

Juliusolsson05 merged 11 commits into
integration/batch-2026-09-27-rfrom
fix/retention-collects-keylog-run-dirs

Conversation

@Juliusolsson05

@Juliusolsson05 Juliusolsson05 commented Sep 27, 2026 •

Copy link
Copy Markdown
Owner

Fixes #1385.

Plan: docs/plans/2026-09-27-retention-collects-keylog-run-dirs.md.

Problem

collectProxyRunDirs recognised a proxy run dir only by proxy-events.jsonl. A run dir holding just session-meta.json + sslkeylog.log was walked into and never collected, budgeted or removed. Those are plaintext TLS session secrets. #1380 review c recounted names and sizes only (contents never read): 23 such dirs on the owner's machine, 5.18 MB, May–September 2026.

What merges: FUTURE runs only

  • A key-log-only dir is recognised as a run dir, but collected only when it is NOT in the BASELINE: the set of key-log-only dirs that existed when this build first started. keyLogBaseline() captures it at run start (holdDebugStoragePruneUntilRecovered), strictly (any unlistable directory means no baseline, and nothing is written), and writes it once to STATE_DIR/debug-retention-keylog-baseline.json. With no baseline, NO key-log-only dir is collected. Timestamps were tried twice and dropped (review a): a date constant and a first-prune marker each excluded new runs forever, and any clock comparison admits a pre-upgrade run after a clock step back.
  • The existing dirs, including the owner's 23, are NOT touched: they are in the baseline, so they are never collected and never walked into, whatever their names. Deleting them stays the owner's decision (q91), now tracked in Owner decision (q91): delete the existing pre-upgrade key-log-only proxy run dirs? #1460; this PR does not make it.
  • A dir with an events file is collected whole as before, key log included (main's existing behaviour; the q91 ruling, tracked in Owner decision (q91): delete the existing pre-upgrade key-log-only proxy run dirs? #1460, covers only the key-log-only dirs). Metadata alone and _shared-conf are still skipped.
  • q115 ("unknown is never empty"): dirStats no longer skips a child it cannot read. Only ENOENT means absent; any other error leaves the whole dir uncollected that pass.
  • The manual-bundle ledger loader is NOT changed: W4's fix: bound the small unbounded-growth items from the C6 hunt (#1278) #1417 owns it (q118), and this PR will use it as-is.

Tests

  • debugRetention.keylog.test.ts, real directory shapes:
    • a new key-log-only dir is collected beside a normal run dir;
    • baseline members (the owner's shape, and one whose name sorts after a new run), _shared-conf and a metadata-only dir are not;
    • a run dir with an unreadable child keeps its key log through the locked pass and after two readable passes, and is removed once genuinely old. Red before the fix.
  • Pinned after B6's check: the birthtime-filter revert (a capture on a clock stepped back to 2020 still baselines every existing key log) and the baseline shape check.
  • Mutations, each killed: the birthtime filter re-added; baseline membership ignored; null collecting everything; lenient capture; capture recording nothing; returning an unsaved baseline.
  • Merged main (a real conflict with fix(claude): bump claude-code-headless to the rotating proxy-events log (#1273) #1376's rotated-generation marker): a dir holding proxy-events.jsonl OR proxy-events.1.jsonl is collected as on main; a key-log-only dir only when new.
  • Gate: npx tsc -b is clean; src/main/storage passes, 13 files / 164 tests at 35001c8a.
  • Residual (conservative, never a deletion): a run created during the few-millisecond startup scan, or before a delayed first successful capture, is baselined and never collected. A birthtime filter was tried and reverted: a clock step back could exclude an old key log from the baseline.

🤖 Generated with Claude Code

#1385 (q91 follow-up of #1380): a run dir holding only session-meta.json +
sslkeylog.log was walked into and never collected, so its plaintext TLS
secrets stayed on disk indefinitely (23 dirs, 5.18 MB on the owner's
machine). A run dir is now recognised by either evidence file. Red before.

HOLD for the owner's q91 decision: once collectable, the 48 h TTL pass
removes these months-old dirs on the first prune, so this IS the sweep.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05 Juliusolsson05 added type:bug Something works wrong class:C6-unbounded Unbounded growth sev:P2 Real bug with a workaround labels Sep 27, 2026
…never counts as empty (q109, q115)

dirStats skipped any child it could not read, so a run whose fresh data sat
in an unreadable child was dated by its oldest file and TTL-removed. A failed
read of the saved-bundles ledger returned an empty manual set, so manual
legacy bundles were bucketed as prunable autosave. Only ENOENT now means
absent; any other failure leaves the artifact uncollected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 and others added 2 commits September 27, 2026 07:48
W4's #1417 owns loadManualLegacyBundlePaths and treats an absent ledger as
unknown, stricter than this branch's ENOENT-means-empty. Keep only the
dirStats fix; use #1417's loader as-is after it merges.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…y; existing key logs stay (#1385)

The 23 existing key-log-only dirs are an owner decision (q91). Collection
now starts at runs dated on or after 2026-09-28; earlier or undated dirs
are left untouched and never walked into.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05 Juliusolsson05 changed the title HOLD(owner q91): debug retention collects key-log-only proxy run dirs fix(debug-retention): collect key-log-only proxy run dirs from new runs only; existing key logs untouched Sep 27, 2026
Juliusolsson05 and others added 4 commits September 27, 2026 10:59
…ain's rotated-generation run marker)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…s first pass, not a date constant (#1385, #1388 review a)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…tured at run start, not a timestamp (#1385, #1388 review a r2 + b)

A first-prune marker excluded runs made during the boot delay forever, and
any timestamp comparison admits a pre-upgrade run after a clock step back.
The baseline is the set of key-log-only dirs that existed when this build
first started, captured strictly and written once; with none, no key-log-only
dir is collected.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…n during capture stay out of it (#1385, #1388 review a round 3)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Juliusolsson05 and others added 3 commits September 27, 2026 11:45
…91 decision is tracked in #1460 (#1388 review c)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…uld exclude an old key log from the baseline (#1388 review b round 3)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ne shape check (#1388, B6 check)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@Juliusolsson05

Copy link
Copy Markdown
Owner Author

Disposition at 35001c8 (review cap reached; B6 manual inspection, then manager-verify a/b/c MERGE-READY):

  • a r1, same-day runs excluded by the date cutoff: fixed with a first-run marker (d8c62e9).
  • a r2, marker written after the boot delay, and a clock rollback: fixed with a baseline set captured at run start (8bd7e32).
  • a r3, a missing proxy root saved an empty baseline: fixed. There is no ENOENT exception now; an unlistable dir means no baseline.
  • b r3, the birthtime rule could exclude an old key log after a clock step back: fixed by reverting it (68baa3e). A test pins the revert with Date.now at 2020 (35001c8).
  • B6 verify: the wrong-shape baseline returns null. It is pinned, and fails without the shape guard.
  • Accepted residuals are conservative only (they keep, never delete): runs created during the startup capture are baselined. Dirs holding an events file keep main's existing behaviour. The key-log-only decision (q91) is tracked in Owner decision (q91): delete the existing pre-upgrade key-log-only proxy run dirs? #1460.
    Gates: tsc -b + 164 tests at 35001c8; CI green.

🤖 Generated with Claude Code

@Juliusolsson05
Juliusolsson05 marked this pull request as ready for review September 27, 2026 21:07
@Juliusolsson05
Juliusolsson05 changed the base branch from main to integration/batch-2026-09-27-r September 27, 2026 21:32
@Juliusolsson05
Juliusolsson05 merged commit b5a08b2 into integration/batch-2026-09-27-r Sep 27, 2026
2 checks passed
@Juliusolsson05
Juliusolsson05 deleted the fix/retention-collects-keylog-run-dirs branch September 27, 2026 21:32
Juliusolsson05 added a commit that referenced this pull request Sep 27, 2026
Real conflict with #1388 in debugRetention.ts, both sides kept:
- imports: this PR's rmdir/sep (parent sweep) + #1388's writeFile/relative
  (key-log baseline);
- collectArtifacts: this PR's cachedManualLegacyBundlePaths() stays the one
  ledger read (q118: absent or unreadable ledger = 'unknown', every legacy
  bundle protected, never cached), and #1388's keyLogOnlyBaseline capture
  is added after it. #1388 already calls this PR's loader; it keeps no loader
  of its own.

tsc -b 0 lines; storage, conversations, paste, window, lruMap and
performance suites 413/413.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

class:C6-unbounded Unbounded growth sev:P2 Real bug with a workaround type:bug Something works wrong

Projects

None yet

Development

Successfully merging this pull request may close these issues.

security(debug-retention): existing TLS key logs in key-log-only proxy run dirs are never cleaned up

1 participant