fix: bound the decoded container value cache to stop wasm memory growth on handle reads

.changeset/bounded-container-value-cache.md

@@ -0,0 +1,24 @@
+---
+"loro-crdt": patch
+---
+
+Fix wasm memory retention when reading a document container by container
+(loro-dev/loro#1092).
+
+Every read through a container handle (`LoroMap.keys()`/`get()`,
+`LoroList.get()`, `LoroText.toJSON()`, ...) decoded the container's value into
+an in-memory cache that was pinned for the lifetime of the document — about
+4 KB per container, released only by `doc.free()`. Walking a large document
+this way (the pattern loro-mirror's initial state build uses) retained ~4 KB ×
+containers-ever-read and trapped wasm32 at the 4 GiB limit around one million
+containers.
+
+The decoded-value cache is now bounded (2048 entries, second-chance FIFO).
+Evicted entries are pure caches over the KV store and are re-decoded on the
+next read, so this only changes memory behavior, not API semantics.
+`doc.free()` semantics are unchanged.
+
+Measured on the issue's repro (570-turn document, 188k container handles,
+release build): the handle walk retains no per-container memory (external
+memory flat at ~84 MiB vs +641 MiB before) and runs ~10x faster
+(1.28 s vs 12.5 s) thanks to the smaller working set.

AGENTS.md

@@ -36,6 +36,9 @@ Loro is a Rust CRDT workspace with JS/WASM packaging and a MoonBit codec.
   [context/wasm-error-reporting.md](context/wasm-error-reporting.md).
 - WASM container id wrapper identity, lazy caching, and benchmark:
   [context/wasm-container-id-cache.md](context/wasm-container-id-cache.md).
+- Bounded decoded-value cache in `InnerStore` (second-chance FIFO, eviction
+  safety contract, loro-dev/loro#1092):
+  [context/container-value-cache.md](context/container-value-cache.md).
 - Context backlog: [context/CONTEXT-GAPS.md](context/CONTEXT-GAPS.md).
 
 ## Commands

context/container-value-cache.md

@@ -0,0 +1,90 @@
+# Container Value Cache (InnerStore)
+
+Verified against code 2026-09-04.
+
+## The funnel
+
+Every per-container read — handler reads like `map_get`, `list_get`,
+`map_len`, text reads, `contains_id` — goes through
+`InnerStore::with_container_for_read`
+(`crates/loro-internal/src/state/container_store/inner_store.rs`). On a KV
+miss the wrapper is created from the KV bytes and, if the read decoded the
+value (`ContainerWrapper::has_cached_value`), inserted into
+`InnerStore.store`. Bulk paths (`toJSON`, deep values, snapshot export) use
+`try_get_value_ephemeral` / `try_with_container_for_ephemeral_read`, which
+deliberately leave no residue.
+
+## The bounded cache (loro-dev/loro#1092)
+
+Before 2026-09, a decoded value stayed pinned in `store` until the doc was
+dropped: ~4 KB per container ever read, superlinear growth on
+container-by-container walks, wasm32 trap at 4 GiB near one million
+containers.
+
+`track_value_cache` now bounds the number of cached decoded values to
+`MAX_CACHED_CONTAINER_VALUES` (2048; 16 under `cfg(test)`) with a
+second-chance FIFO (`value_cache_queue` + per-wrapper
+`in_value_cache_queue` / `value_cache_referenced` bits on
+`ContainerWrapper`). The second-chance bit exists so a hot ancestor (e.g. the
+root list re-read once per item during a deep walk) survives eviction passes.
+
+## Eviction safety contract
+
+Only `flushed && Lazy && value.is_some()` wrappers are evictable
+(`ContainerWrapper::is_evictable_cached_value`): a flushed lazy wrapper is a
+pure cache over the KV bytes, so dropping it loses nothing. Once a container
+is mutated, `get_state_mut` converts the wrapper to `State` and clears
+`flushed`; `State` wrappers are never evicted, so unflushed edits can never be
+lost. Evicted entries are re-created from KV on the next read — eviction only
+costs a re-decode.
+
+Because of eviction, `store` is strictly a cache over `kv`: all lookup paths
+(`get_or_insert_with`, `ensure_container`, `get_mut`, `with_container_for_read`,
+the ephemeral reads, `contains_id`) fall back to `kv` on a `store` miss
+regardless of `load_state`. Do not reintroduce `load_state != AllLoaded` gates
+on those fallbacks — `load_all`/`decode_twice` (GC snapshot import) put the
+store in `AllLoaded` mode, and evicted entries must stay reachable there.
+
+## What `AllLoaded` means with eviction
+
+`LoadState::AllLoaded` now means "every `kv` entry was materialized into
+`store` at some point", NOT "`store` is complete right now". Eviction can
+remove entries afterwards, so `load_all()` tracks `evicted_since_full_load`
+(set on every eviction, cleared by `decode`/`decode_twice`/a full `load_all`
+scan) and re-scans `kv` when it is set instead of short-circuiting on
+`AllLoaded`. Without this, `iter_all_container_ids()` /
+`iter_all_containers_mut()` silently miss evicted containers; the shallow
+snapshot re-export path (`encoding/shallow_snapshot.rs`) enumerates containers
+that way and dropped them from the latest-state overlay — a silent data-loss
+on re-import. The "content in `store` is newer than `kv`" skip inside
+`load_all` stays sound: evicted entries are absent from `store`, so they are
+rebuilt from `kv`.
+
+Cost: one bool check per `load_all()` call, plus at most one `kv` re-scan
+after the first eviction following a full load (the scan's per-entry
+"already in `store`" skip makes repeat scans cheap).
+
+## Remaining pins
+
+- Tree containers materialize a full `State` on first value read
+  (`decode_value_from_bytes` returns `decoded_state` for trees), so tree walks
+  still pin state; they are not covered by the bound.
+- `decode_twice`/`load_all` insert one lightweight lazy shell per container
+  into `store` (no decoded value); those shells are small and stay.
+
+## Tests
+
+- `crates/loro-internal/src/state/container_store.rs` (`mod test`):
+  `handle_reads_bound_cached_container_values`,
+  `evicted_containers_stay_readable_and_editable`,
+  `evicted_entries_stay_readable_when_all_loaded` (also asserts
+  `iter_all_container_ids` stays complete after evictions),
+  `evicted_mergeable_child_and_tree_meta_survive_round_trip`,
+  `stale_queue_entry_after_lazy_to_state_conversion`.
+- `crates/loro-internal/src/encoding/shallow_snapshot.rs`:
+  `reexport_same_shallow_root_after_walk_eviction_keeps_overlay_containers`
+  (the silent data-loss regression).
+- `crates/loro-wasm/tests/walk_mem.test.ts`: ~100k-container handle walk keeps
+  `process.memoryUsage().external` within a small multiple of the `toJSON()`
+  delta (fails at ~286 MB on the pre-fix build, ~13 MB after); also asserts
+  the walk result equals `toJSON()` so an incomplete walk cannot pass.

crates/loro-internal/src/encoding/shallow_snapshot.rs

@@ -549,6 +549,7 @@ mod tests {
     use crate::encoding::fast_snapshot::_decode_snapshot_bytes;
     use crate::encoding::EncodeMode;
     use crate::encoding::ExportMode;
+    use crate::handler::MapHandler;
     use crate::handler::TextHandler;
     use crate::state::{ContainerCreationContext, FastStateSnapshot, RichtextState};
     use crate::HandlerTrait;
@@ -765,4 +766,75 @@ mod tests {
             LoroValue::Null
         );
     }
+
+    /// Regression test for the P1 found in review of the bounded container
+    /// value cache (loro-dev/loro#1092): after a shallow-snapshot import the
+    /// store is `AllLoaded`, and a container walk evicts most decoded values.
+    /// Re-exporting the same shallow root must still enumerate every container
+    /// created after the root — otherwise the latest-state overlay silently
+    /// drops the evicted ones and the next import loses them.
+    #[test]
+    fn reexport_same_shallow_root_after_walk_eviction_keeps_overlay_containers() {
+        // Doc A: base content behind the shallow root.
+        let a = LoroDoc::new_auto_commit();
+        a.set_peer_id(1).unwrap();
+        a.get_text("text")
+            .insert(0, "base", PosType::Unicode)
+            .unwrap();
+        a.commit_then_renew();
+        let start = a.oplog_frontiers();
+
+        // Doc B: import the shallow snapshot, then create containers that live
+        // only in the latest-state overlay (they are not in the root KV).
+        let b = LoroDoc::new_auto_commit();
+        b.set_peer_id(2).unwrap();
+        b.import(&a.export(ExportMode::shallow_snapshot(&start)).unwrap())
+            .unwrap();
+        let n = 64;
+        let list = b.get_list("list");
+        for i in 0..n {
+            let map = list
+                .insert_container(i, MapHandler::new_detached())
+                .unwrap();
+            map.insert("key", i as i64).unwrap();
+        }
+        b.commit_then_renew();
+        // 2 ops per container > MAX_OPS_NUM_TO_ENCODE_WITHOUT_LATEST_STATE
+        // (16 in test builds), so this export ships the latest-state overlay.
+        let blob = b.export(ExportMode::shallow_snapshot(&start)).unwrap();
+
+        // Doc C imports the blob (store is AllLoaded with lazy wrappers) and
+        // walks every overlay container, evicting most of them from the
+        // bounded value cache.
+        let c = LoroDoc::new_auto_commit();
+        c.import(&blob).unwrap();
+        let list = c.get_list("list");
+        for i in 0..n {
+            let child_id = list.get(i).unwrap().as_container().unwrap().clone();
+            assert_eq!(c.get_map(child_id).get("key"), Some((i as i64).into()));
+        }
+
+        // Re-exporting the same shallow root reuses the stored root bytes and
+        // rebuilds the overlay by enumerating all containers. With the broken
+        // `load_all` short-circuit this dropped every evicted container.
+        let reexported = c.export(ExportMode::shallow_snapshot(&start)).unwrap();
+        assert!(
+            shallow_sections(&reexported).state_bytes.is_some(),
+            "test setup must take the overlay export path"
+        );
+
+        let d = LoroDoc::new_auto_commit();
+        d.import(&reexported).unwrap();
+        let list = d.get_list("list");
+        assert_eq!(list.len(), n);
+        for i in 0..n {
+            let child_id = list
+                .get(i)
+                .unwrap_or_else(|| panic!("container {i} lost after shallow re-export"))
+                .as_container()
+                .unwrap()
+                .clone();
+            assert_eq!(d.get_map(child_id).get("key"), Some((i as i64).into()));
+        }
+    }
 }

crates/loro-internal/src/state/AGENTS.md

@@ -11,7 +11,10 @@ before changing mergeable child behavior.
 - `../state.rs`: `DocState`, checkout/path/deep-value traversal, state replay,
   lifecycle, and alive-container discovery.
 - `container_store/`: persisted KV-backed container snapshots and
-  `ContainerWrapper` encoding.
+  `ContainerWrapper` encoding. The decoded-value cache in `InnerStore` is
+  bounded and evicted wrappers must stay re-creatable from KV; read
+  [../../../../context/container-value-cache.md](../../../../context/container-value-cache.md)
+  before changing read/caching paths there.
 - `map_state.rs`, `list_state.rs`, `richtext_state.rs`, `tree_state.rs`,
   `movable_list_state.rs`, `counter_state.rs`: per-container state and snapshot
   codecs. `richtext_state.rs` also hosts `redact_dead_style_values`, used by