feat(shared): primitive history metadata storage and versioned import hashes

[zxch3n]

Related issue

Same-repository branch: no intake Issue required. Supersedes PR #584 (closed unmerged).

Problem / pressure

Two storage-layer defects in session history:

1. The history item catchall used schema.Any({ defaultLoroText: true }), so every new metadata string (toolCallId, status, title, kind, locations[].path, nested command/args/cwd, step command) became a LoroText container — thousands of never-streamed containers per long session. Measured on unmodified main 375c4c7: all of those were Text containers.
2. Import turn hashes had no version. After feat: read long conversations by window with shared history writes #376 moved cursor/replay types, hashing, baseline validation and import decisions into packages/shared/src/session-data, a stored cursor written by an old client (v1 hashes) would be compared against new hashes, manufacturing prefix_mismatch on an unchanged transcript.

This re-implements the necessary parts of #584 on current main; #584 was closed unmerged and its CLI-side ownership is obsolete after #376. #586 is untouched.

Summary

Commit 1 — storage policy (e8c11be2):

• schema.ts: item + nested catchalls default to primitive; streaming fields declared explicitly via storageSchema hints (text/thought text, plan markdown, tool content text/output + nested content.text, worktree step output). Diff oldText/newText, command, args, cwd, path, terminalId, input scalars, steps[].command stay primitive.
• Insertion-only: opening performs no writes; legacy Text keeps its container id on same-kind edits; primitives stay primitive; unknown/malformed stored shapes survive unrelated edits.

Commit 2 — versioned import in the current shared owner (9c773955):

• packages/shared/src/session-data/history-import.ts: HASH_VERSION_V1/V2; frozen v1 canonicalization pinned by a fixed byte fixture; v2 canonical item form so a sealed tool_call skeleton and the full call it came from hash identically. Documented tradeoff: v2 excludes tool payload (content/rawInput/rawOutput) from identity, so tool-output-only source drift does not by itself trigger a refresh.
• cursor.hashVersion versions importedTurnHashes; ExternalAcpHistorySyncMeta.hashVersion versions replayDigest; the stored baseline records its own version; absent means v1 (tested with a genuine pre-version shape — no hashVersion field anywhere — not a fabricated one).
• HistoryImportCursorSchema retains hashVersion across the Zod strip boundary; the planner and applyHistoryImport recompute replay hashes in the stored version when they differ; an unknown version or missing replay history refuses before any write; a cursor failure after the history write remains indeterminate; a metadata-only conflict marker never upgrades the cursor by implication.
• The CLI sync service materializes new imports at v2 and threads versions through its conflict pre-checks.

Visual explanation

replay (v2 hashes) ──► decideHistoryRefresh / decideHistoryConflictResolution
    cursor.hashVersion   ──► stored-version binding for prefix/suffix comparisons
    meta.hashVersion     ──► replayDigest compared at the metadata version
    baseline.hashVersion ──► accepted only when == cursor version (absent = v1)
                              │ mismatch + replay history → recompute at stored version
                              │ mismatch without it / unknown version → refuse pre-write
                              ▼
              applyHistoryImport: one synchronous commit block
              (history write → readStored baseline → versioned cursor write)

Storage: Any(defaultLoroText: false) + explicit storageSchema hints is a writer-side layout. Raw Mirror.setState is not a history writer, so hints cannot diverge from it; readers are unaffected.

Before / after

Before	After
Every new metadata string becomes a LoroText container	Metadata stays primitive; only declared streaming fields are Text
Unversioned hashes compared across canonical forms	v1/v2 explicit per cursor/meta/baseline; mismatches recompute from replay or refuse safely

Test plan

• Baseline on unmodified main 375c4c7: metadata fields proven Text containers via a probe, and frozen v1 bytes pinned (cc7ec54e218e6c9570af864fa57d76ee5677fcd0897c1b4919564e969f585935).
• New packages/shared/tests/history-storage-policy.test.ts (10 tests): full field matrix incl. empty values/arrays and unknown new fields, CID preservation on streaming edits, legacy Text + primitive layouts, malformed stored siblings, two-peer convergence, read-only open.
• Version matrix: decision level (v1/v1, v2/v2, cursor v1/meta v2, cursor v2/meta v1, skeleton parity, output-only drift, unknown version throws); port level through applyHistoryImport (genuine unversioned v1 cursor+baseline upgrade, untracked suffix conflict, already-resolved refusal, cursor failure stays indeterminate); real SessionDocument writer level (pre-version client shape accepted after upgrade).
• Suites at head 9c77395: shared 1211 passed (106 files); CLI 2809 passed with 1 pre-existing environment-dependent failure (tests/worktree-gc.test.ts, reproduces identically on unmodified main in this worktree environment); components 3616 passed; electron 112 passed; test:scripts passed.
• Static: tsgo typecheck clean for shared/components/CLI; oxlint 0 errors; prettier check clean; i18n keys complete; check:code-collab-imports / check:platform-boundaries / check:public-boundary pass; docs check exit 0 (packages/shared/AGENTS.md 8040/8192 bytes).
• CLI production build with the CI heap cap: NODE_OPTIONS=--max-old-space-size=2048 pnpm --filter lody build OK.

Limits

• No 3000-turn device performance, E2E acceptance, or payload-size reduction is claimed.
• The sealed-skeleton reader feature (ref payload fetch, hooks, UI) is not implemented; v2 is verified against synthetic fixtures only.
• The replay-version fallback in decideHistoryRefresh (absent = already in the stored version) is reachable only by legacy test doubles comparing opaque same-version hashes; production always carries the required HistoryImportReplay.hashVersion through planHistoryImport.

Review focus

• Challenge the v2 exclusion of tool payload from identity; the spec records the tradeoff explicitly.
• Challenge the genuine-v1 fixtures: no hashVersion field anywhere and baselines computed from actual stored content.
• Plausible failure to probe: a storageSchema hint diverging from raw Mirror.setState — HistoryWriter is the only history writer and hints are inert for readers.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9c77395502

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

[chatgpt-codex-connector]

Reject unknown replay hash versions before recomputing

When a refresh supplies an unknown replay version (for example, hashVersion: 3) against an existing v1/v2 cursor, this branch ignores the unsupported source version and successfully recomputes hashes using the stored version. The writer can then append history before createImportCursor eventually throws while hashing with version 3, so applyHistoryImport returns indeterminate with history and cursor out of sync instead of refusing before any write. Validate replayHashVersion as supported before recomputation.

AGENTS.md reference: packages/shared/src/session-data/AGENTS.md:L33-L38

Useful? React with 👍 / 👎.