feat(core): add lazy list option and LazyList read view - #96
Merged
Merged
Conversation
schema.LoroList(item, idSelector, { lazy: { index, maxHydrated?, tailKeep? } })
keeps large lists out of memory: Mirror state holds a stable LazyList
instance at the lazy key instead of an array. Ids come from one
list.getShallowValue() call; the fields named in lazy.index are always
loaded (extracted from the bulk deep-value payload when available, one
map.getShallowValue() per item otherwise); items hydrate on demand via
hydrate(from, to) with LRU eviction (maxHydrated, default 200) that
exempts active subscribeRange windows and the trailing tailKeep items
(default 20). Hydrated data is keyed by item container id, never by
position, so positional views stay correct across concurrent edits.
- Loro events are intercepted before the immutable state walk:
structural deltas update ids/length/version and shift hydrated slots
by id; events inside hydrated items reuse applyEventBatchToState
(structural sharing identical to non-lazy lists); events inside
non-hydrated items only refresh index fields and bump version.
- setState updates touching a lazy path throw LazyListWriteError and
point to mirror.list(path); the write API
(push/insert/deleteById/updateById/updateAt) reuses the existing
diff/apply machinery so it produces exactly the Loro ops an
equivalent setState array edit would (round-trip tested).
- Lazy lists are excluded from the ephemeral compose;
checkStateConsistency verifies lazy paths by instance identity.
- InferType/InferInputType resolve lazy lists to
LazyList<Item, Partial<Item>>; LazyList, LazyListImpl,
LazyListWriteError, isLazyList and LazyListWriter are exported.
- React: useLazyRange(list, from, to) hydrates a window, re-renders on
in-range changes via subscribeRange, and releases on unmount.
- Benchmark (scripts/lazy-list-bench.mjs, 2000 items x ~30 nested
containers, ~58k containers): lazy init p50 20.5 ms, hydrate(0, 30)
p50 2.1 ms, eager control (bulk init) p50 117.7 ms.
zxch3n
changed the base branch from
perf/bulk-init-deep-value
to
perf/structured-read-state
September 6, 2026 06:46
Preserve slot provenance through reads and list deltas, and inherit Tree Ignore and depth-limit fallback repairs from the stacked branches.
zxch3n
added a commit
that referenced
this pull request
Sep 10, 2026
* perf(core): bulk-initialize Mirror state with one getDeepValueWithID call
Mirror initialization previously walked the document container by
container (~10 wasm crossings per container: id/kind/get/keys/toJSON).
buildRootStateSnapshot now reads the whole doc with a single
doc.getDeepValueWithID() call and builds state, $cid markers,
containerRegistry, inferOptionsByContainerId and rootPathById in one JS
walk that replicates registerChildContainer/containerToMirrorState
semantics exactly. No container handles are created during init; trees
and unhandled kinds fall back to the existing handle-based logic, and
the legacy traversal is kept as a fallback for loro-crdt versions
without getDeepValueWithID.
- Defensive cid parsing: tolerates both the Debug-like
"idx:N, id:cid:..." format and bare container ids.
- loro-crdt <1.15 drops the {value,cid} wrapper for empty containers,
making them indistinguishable from embedded plain values; the walk
disambiguates via the parent's cached getShallowValue() (one extra
call per affected parent only).
- Adds a seeded parity fuzz test (bulk vs legacy: state, $cid via
cidsEqual, registry, infer options) and a dependency-free benchmark
(pnpm bench:bulk-init, ~67k-container session-history-shaped doc):
wasm crossings during init drop from ~527k to 1.
* fix(core): preserve embedded values in bulk snapshots
Model: gpt-6
* perf(core): initialize from explicit readState nodes
* perf(core): initialize from selected container trees
* chore: validate bulk initialization on loro-crdt 1.16.0
* test(core): integrate published loro-crdt container trees
* perf(core): copy paths for single text events (#99)
* perf(core): copy paths for single text events
Avoid general Immer drafting for existing text leaves while retaining descriptors and immutable snapshots. Add real-replica parity checks, a deterministic fast-path guard, and a reproducible event-application benchmark.
Model: gpt-5
* perf(core): copy dense array ancestors without defining each property
Validate ordinary array descriptors before direct value copying and retain descriptor fallback for unusual arrays. Move differential assertions outside Wasm callbacks, add array parity and work guards, expand paired benchmarks, and move implementation notes below the quick start.
Model: gpt-5
* fix(core): route array subclasses through general event handling
Check array prototypes during path traversal, remove the redundant dense-copy guard, and verify subclass prototype parity with the general path. Removing the guard now fails the subclass regression.
Model: gpt-5
* chore: release main
* fix(core): fall back on container tree depth limits
* feat(core): add lazy list option and LazyList read view (#96)
* feat(core): add lazy list option and LazyList read view
schema.LoroList(item, idSelector, { lazy: { index, maxHydrated?, tailKeep? } })
keeps large lists out of memory: Mirror state holds a stable LazyList
instance at the lazy key instead of an array. Ids come from one
list.getShallowValue() call; the fields named in lazy.index are always
loaded (extracted from the bulk deep-value payload when available, one
map.getShallowValue() per item otherwise); items hydrate on demand via
hydrate(from, to) with LRU eviction (maxHydrated, default 200) that
exempts active subscribeRange windows and the trailing tailKeep items
(default 20). Hydrated data is keyed by item container id, never by
position, so positional views stay correct across concurrent edits.
- Loro events are intercepted before the immutable state walk:
structural deltas update ids/length/version and shift hydrated slots
by id; events inside hydrated items reuse applyEventBatchToState
(structural sharing identical to non-lazy lists); events inside
non-hydrated items only refresh index fields and bump version.
- setState updates touching a lazy path throw LazyListWriteError and
point to mirror.list(path); the write API
(push/insert/deleteById/updateById/updateAt) reuses the existing
diff/apply machinery so it produces exactly the Loro ops an
equivalent setState array edit would (round-trip tested).
- Lazy lists are excluded from the ephemeral compose;
checkStateConsistency verifies lazy paths by instance identity.
- InferType/InferInputType resolve lazy lists to
LazyList<Item, Partial<Item>>; LazyList, LazyListImpl,
LazyListWriteError, isLazyList and LazyListWriter are exported.
- React: useLazyRange(list, from, to) hydrates a window, re-renders on
in-range changes via subscribeRange, and releases on unmount.
- Benchmark (scripts/lazy-list-bench.mjs, 2000 items x ~30 nested
containers, ~58k containers): lazy init p50 20.5 ms, hydrate(0, 30)
p50 2.1 ms, eager control (bulk init) p50 117.7 ms.
* fix(core): preserve lazy item batches and hydration lifetimes
Model: gpt-6
* chore: validate lazy lists on loro-crdt 1.16.0
* perf(core): hydrate lazy items with container trees
* fix: preserve nested lazy updates and observe tail growth
* fix(core): normalize inserted lazy items and release deleted subtrees
---------
Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and behavior
Stacked on #98 (which is stacked on #95). Uses published loro-crdt 1.16.0 across core/react/jotai and the lockfile.
schema.LoroList(itemSchema, idSelector?, { lazy: { index, maxHydrated?, tailKeep? } })exposes a stable LazyList read view. Item identities and selected index fields stay resident; hydrate/release and LRU manage full item state.mirror.list(path)supplies writes, and React useLazyRange owns window subscriptions.Published container-tree integration
Ordinary lazy item hydration now calls
container.toContainerTree()and reuses the structured Mirror walker for identities, opaque values, registration and decoding. Roots containing lazy subtrees are excluded from the document-wide tree read, including mixed ordinary/lazy schemas. Items containing nested lazy lists retain shallow traversal so those lists remain unread until requested. Structured nodes never enter the legacy index-payload parser.Correctness
Hydrated item events are applied as a batch with ancestor deduplication. React subscribes before hydration and refreshes windows after structural edits. Writes pin targets by CID and release pins in finally. Existing cache-pressure, nested event, and StrictMode regressions remain covered.
A new published-package regression mixes ordinary and lazy roots and adds a 270-level unselected item: document-wide traversal would throw, but initialization and hydration of the selected first item succeed. The test disables legacy deep/text reads and verifies opaque values and registered child identities.
Validation
Build, 587 tests, lint (0 errors; existing warnings), and workspace typecheck pass on published loro-crdt 1.16.0. Default benchmark: 2,000 items, window 30, 3 warmups, 10 samples; p50 lazy init 32.011 ms, hydrate(0,30) 1.584 ms, eager init using the new container-tree path 80.122 ms. These are synthetic fixture results, not a universal speedup guarantee.
Remaining scope
Ephemeral overlays skip lazy slots; consistency checks verify LazyList instance identity rather than all item contents. Replacement guards under non-lazy list items remain best-effort. Explicit release overrides tailKeep but respects active subscriptions. Hydration uses per-item container trees rather than a new cross-item batching protocol.
Ignore integration
Includes #97 and its nested Ignore consistency fix. All three event entry points filter Ignore before registration/normalization and LazyList handling. Consistency snapshots preserve memory-only Ignore values and skip identical references (including stable LazyList views), while normal sibling values and identities remain checked. New local/remote mixed-batch regressions verify no ignored registration/notification, lazy item updates, subsequent setState, writer updates and consistency. The combined branch passes build, 587 tests, lint (0 errors), and typecheck on loro-crdt 1.16.0.