fix: summarize once idle, refresh on growth, and bound failed-summary retries

[minyek]

Description

Summaries are write-once. A conversation that gets summarized while still in
progress keeps a summary of only its early portion forever — every later turn
is invisible to the summary. For long-running or resumed conversations, the
one-line summary shown next to search results is therefore misleading.

The fix has two complementary halves: defer summarizing a conversation until
it has gone idle (so a still-active conversation is never frozen into a partial
snapshot), and refresh a summary when its transcript later grows. Note the
behavioral change from the defer half: a conversation is no longer summarized
until it has been idle for the quiescence window (EPISODIC_MEMORY_SUMMARY_QUIESCENCE_HOURS,
default 1h), so very recent conversations intentionally have no summary yet.

Because the refresh half makes us re-summarize, failures can now recur — so the
same change also makes failed (re-)summaries back off and eventually give up
instead of preserving a good summary but re-attempting on every sync. That retry
policy is unified with the existing first-failure error-sentinel path (see Fix 5).

(Search results themselves were always complete — search runs over indexed
exchanges, kept fresh via INSERT OR REPLACE. This fixes only the summary
displayed alongside them.)

Symptom

There is no error in the logs — the symptom is silently wrong output, plus
silently wasted work:

• <archive>/<project>/<session>-summary.txt for an active or long conversation
  describes only the first handful of exchanges.
• Continuing the conversation over hours/days never updates it.
• stats/verify count the file as "summarized", so it is never revisited.
• A conversation that can't be summarized (oversized transcript, a turn that
  always errors) re-invokes the summarizer on every sync, indefinitely.

Environment

• episodic-memory v1.4.2
• Claude Agent SDK summarizer
• All installs affected; most visible to anyone who keeps long or resumed
  sessions.

Steps to Reproduce

1. Start a conversation and let a sync summarize it early (a few exchanges).
2. Continue the conversation for many more turns over time.
3. Run sync again (or let the SessionStart background sync run).
4. Inspect <archive>/<project>/<session>-summary.txt.
5. Observe it still summarizes only the early portion — never re-summarized,
   even though the transcript has grown substantially.

Root Cause

shouldQueueForSummary only re-queued a conversation when its sentinel was
missing or a stale error marker. Once a real summary existed, the
conversation was treated as done permanently:

1. No coverage signal. Nothing recorded how much of the transcript a given
   summary reflected, so transcript growth was undetectable.
2. Append-only transcripts. A summary written at exchange 5 is never
   refreshed at exchange 500 — the file only ever grows, but nothing acted on
   that growth.
3. No idle awareness. Even if growth had been detected, summarizing an
   in-progress conversation would freeze a mid-session snapshot that is itself
   immediately stale.

And once re-summarization exists, a fourth gap appears in failure handling:

4. Unbounded retries. A failed re-summary correctly kept the prior summary
   but recorded nothing, so the file re-queued and re-invoked the LLM on every
   sync forever. First-ever failures (#96 error sentinels) backed off ~1h but
   never gave up. And since sync is SessionStart-driven, not a fixed timer, a
   1h floor rarely even skips a sync — the retry effectively fired every sync.

Fix Summary

1. Coverage header. Real summaries now carry a machine-readable first line —
   __COVERAGE__ {"bytes":<n>,"lastExchange":"<iso>","schema":1} — recording the
   archived JSONL byte size the summary reflects (formatSummaryFile /
   parseSummaryFile, buildCoverage, writeSummary).
2. Growth-driven re-queue. shouldQueueForSummary(path, currentArchiveBytes)
   re-queues a real summary when the archive has grown past its covered bytes
   (append-only ⇒ a size increase means new content).
3. Quiescence gate. isQuiescent + EPISODIC_MEMORY_SUMMARY_QUIESCENCE_HOURS
   (default 1) only (re)summarize after the conversation has been idle for the
   window, so a mid-session snapshot is never frozen. Idle is measured from the
   last exchange's timestamp — the archive copy's mtime is reset on every
   sync and is unreliable.
4. Gentle upgrade. Legacy header-less summaries are stamped with a
   current-size baseline (ensureCoverageBaseline) — a header rewrite, no LLM
   call — so existing installs gain a growth baseline without a mass
   re-summarization on upgrade.
5. Preserve, back off, then give up on failure. A failed (re-)summary keeps
   any prior real summary and records the attempt; one shared policy
   (shouldRetryAfterFailure) then holds the file until the retry floor
   (EPISODIC_MEMORY_SUMMARY_ERROR_RETRY_HOURS, default 1h) elapses and abandons
   it after EPISODIC_MEMORY_SUMMARY_MAX_ATTEMPTS (default 5) — keeping the last
   good summary rather than thrashing. This same policy governs both first-ever
   failures (#96 error sentinels, now attempt-counted) and re-summary failures
   (state in the coverage header's resummary field), via recordSummaryFailure.
   It replaces the prior mtime-based, never-give-up retry and the duplicated
   floor/cap logic.
6. Fair queue ordering. Never-summarized conversations are processed ahead of
   re-summary retries before the per-run summaryLimit, so a backlog of failing
   re-summaries can't starve fresh work (the #91 head-of-queue failure mode).
7. Display. search strips the coverage header from the summary shown
   alongside results.

Internally, the gate + write + failure logic is shared (needsSummary,
recordSummaryFailure, shouldRetryAfterFailure, summarizeIfQuiescent) across
the three indexer paths and the sync loop, and the _HOURS config defaults
convert to milliseconds at a single boundary (MS_PER_HOUR).

Configuration

• EPISODIC_MEMORY_SUMMARY_QUIESCENCE_HOURS (default 1) — idle time before a
  conversation is (re)summarized.
• EPISODIC_MEMORY_SUMMARY_ERROR_RETRY_HOURS (default 1) — minimum wait between
  failed summary attempts (the retry floor).
• EPISODIC_MEMORY_SUMMARY_MAX_ATTEMPTS (default 5) — failed attempts before
  giving up and keeping any existing summary. (new)

Error-sentinel format change

The #96 error sentinel's second line changes from a bare ISO timestamp to a
JSON failure-state line, so it can carry the attempt count:

before:  __ERRORED__\n2026-06-07T10:00:00.000Z\n<message>
after:   __ERRORED__\n{"attempts":2,"lastAttempt":1718040000000}\n<message>

Backward compatible: parseErrorSentinel reads the legacy v1.4.2 ISO form
(→ attempts: 0, lastAttempt from the timestamp), so sentinels already on disk
keep backing off correctly and are rewritten in the new form on their next
failure. An unrecognised second line degrades safely to {attempts: 0, lastAttempt: 0} → retried, never lost.

Tests added

• test/summary-coverage.test.ts — coverage header round-trip and legacy baseline stamping.
• test/summary-quiescence.test.ts — quiescence verdicts and _HOURS env parsing.
• test/summary-sentinel.test.ts — growth-aware gate decision table; unified
  backoff/give-up for both error sentinels and re-summaries; parseErrorSentinel
  round-trip + legacy ISO fallback; getMaxSummaryAttempts; recordSummaryFailure.
• test/summary-queue-order.test.ts — fresh-before-retry ordering (#91 guard).
• test/sync-resummary.test.ts — growth-driven re-summary, no-op when unchanged,
  preserve-on-failure (now also asserts the attempt is recorded).
• test/search-summary-strip.test.ts — the header never leaks into the displayed summary.
• Updates to sync-error-sentinel (mtime-aging → stored-timestamp; sentinel-format
  assertion), show, and sync tests for the new header/gate.

Fix Verified

• Full suite: 258 tests across 41 files pass.
• Real end-to-end on the built dist (isolated temp dirs, real summarizer):
  • Seed an 8 237-byte conversation → first sync writes a summary stamped "bytes":8237.
  • Grow the transcript to 41 069 bytes → next sync re-summarizes; coverage re-stamped "bytes":41069.
  • No further growth → next sync performs no re-summary.

Caveat / Notes

• Re-summary is triggered by byte growth, not an exchange-count delta. This
  is deliberate: transcripts are append-only and the per-sync coverage scan is
  ~23 ms even against a multi-GB archive, so no recency cutoff or minimum-delta
  threshold is needed. The quiescence gate bounds work to one re-summary per
  idle-after-growth episode.
• Give-up is permanent for a given conversation (it keeps its last good
  summary, if any): the attempt cap bounds cost on a transcript that never
  summarizes. The cap is the real bound, since the coarse, variable sync cadence
  makes the wall-clock floor too short to reliably skip a sync; the floor's job
  is to stop rapid session restarts from spending the budget on a transient blip.

[minyek]

PR update — bound failed-summary retries (eab6331)

Follow-up to the "preserve a prior summary on failure" behaviour here. Preserving
was right, but the failed re-summary had no backoff and no give-up, so an
un-summarizable transcript re-invoked the LLM on every sync, forever. The
first-failure #96 path had the mirror problem (backed off ~1h but never gave
up — and since sync is SessionStart-driven, the 1h floor rarely even skips a
sync).

Fix: one shared policy for both paths — back off until the retry floor
(EPISODIC_MEMORY_SUMMARY_ERROR_RETRY_HOURS, 1h), then give up after
EPISODIC_MEMORY_SUMMARY_MAX_ATTEMPTS (new, default 5), keeping any existing
summary. Failure state ({attempts, lastAttempt}) lives in the error sentinel and
in the coverage header's resummary field; success clears it. Also: never-summarized
conversations now sort ahead of re-summary retries before summaryLimit, so
failing retries can't starve fresh work (#91). Drops the mtime-based timing and
the duplicated logic.

Giving up first-failures too is deliberate: transient errors recover well within
5 attempts (≥5h); persistent ones never recover, so retrying forever only wastes
calls — and the exchanges stay indexed regardless.

⚠️ Reviewer note — on-disk format change. The #96 sentinel's second line
goes from a bare ISO timestamp to a JSON failure-state line (to carry the attempt
count). Backward compatible: parseErrorSentinel reads the legacy v1.4.2 form, so
existing sentinels keep working and rewrite on their next failure.

Supersedes description point 5 (writeErrorSentinelIfNew → recordSummaryFailure).
Full suite: 258 pass.