Skip to content

feat(wasm): add typed container trees with fixed JS construction - #1087

Merged
zxch3n merged 7 commits into
mainfrom
feat/wasm-json-text-export
Sep 6, 2026
Merged

zxch3n merged 7 commits into
mainfrom
feat/wasm-json-text-export

Conversation

@zxch3n

@zxch3n zxch3n commented Sep 3, 2026 •

Copy link
Copy Markdown
Member

This adds a typed, independent representation of visible document values that preserves real container identities without a path table or ambiguity with ordinary objects.

const roots = doc.toContainerTree();
const selected = doc.toContainerTree({ roots: ["settings", "profile"] });
const subtree = map.toContainerTree({ text: "delta" });
const window = list.toContainerTreeSlice(20, 40);
// window: { cid, start, totalLength, items }

Documents and all six attached container kinds expose toContainerTree(). Containers are { type, cid, value }; ordinary values are opaque Value nodes. The text option applies recursively, including Tree metadata. TypeScript infers the receiver kind and text format. Detached containers throw because they have no attached identity.

Root selection happens before value traversal; unknown/hidden roots are omitted without creating them. List/MovableList slices are separate from complete container nodes and include clamped coordinates and total length, read under one state lock. Bounds are nonnegative u32 integers, end-exclusive; obtaining the parent shallow list remains O(N).

The implementation traverses ephemeral shallow values and constructs JS objects with fixed helpers. Keys/peer strings are reused per read; binary buffers are owned. No public transport protocol, callback decoder, path table, or lifetime cache. Reads do not commit. The unpublished readState API is replaced; existing released getDeepValueWithID remains unchanged.

Validation: final release packaging for all targets; 372 Vitest tests passed (2 skipped, 2 todo), TypeScript, 4 Deno and 4 Bun tests passed. Coverage includes opaque data, recursive deltas, Tree metadata, IDs, ownership, special properties, root filtering without traversal of excluded deep subtrees, detached containers, slice bounds and inferred types. Mirror integration against the built package passes 453 core tests. Related Mirror integration: loro-dev/loro-mirror#98.

Performance experiments retain fixed construction: bounded UTF-8/UTF-16 string batching did not provide consistent end-to-end gains across Node and Chromium. Those prototypes are not included in this PR.

@github-actions

github-actions Bot commented Sep 4, 2026 •

Copy link
Copy Markdown
Contributor

WASM Size Report

  • Original size: 3145.28 KB
  • Gzipped size: 1041.30 KB
  • Brotli size: 730.39 KB

@zxch3n
zxch3n force-pushed the feat/wasm-json-text-export branch from 1430e55 to 80ac32f Compare September 4, 2026 12:17
@zxch3n
zxch3n force-pushed the feat/wasm-json-text-export branch from 80ac32f to 55093ca Compare September 4, 2026 16:38
Base automatically changed from feat/wasm-container-bulk-read to main September 5, 2026 05:14
Add getDeepValueJson(): string on LoroDoc and every container class —
serde_json serialization of the deep value in one WASM call, same content
as JSON.stringify(x.toJSON()) — and getDeepValueJsonWithIds():
{ json, cids } where cids lists container ids in pre-order DFS of the
serialized tree so a consumer can re-attach ids in a single JS walk.

The (json, cids) pair is produced by converting the with-id deep value to
a serde_json::Value and stripping { cid, value } nodes in one pass, so the
cids order always matches the key/item order a JS consumer sees after
JSON.parse, regardless of serde_json's preserve_order feature.

Benchmark on a ~70k-container doc (Map 15,632 / List 9,956 / Text 44,463,
4.1 MB JSON), release build: getDeepValueWithID 137.0 ms vs
getDeepValueJson()+JSON.parse 56.3 ms (2.4x). The 5x target is not
reachable from the JS side: profiling shows the Rust-side deep-value walk
and serialization dominate (getDeepValueJson alone is 46 ms; JSON.parse
of 4.1 MB is ~5 ms), not the boundary crossing.

Wasm size (dev build, with debug info): +524 KB (+0.52%).
@zxch3n
zxch3n force-pushed the feat/wasm-json-text-export branch from 55093ca to 7e99105 Compare September 5, 2026 05:15
Replace ambiguous shape-based id stripping with a sparse Uint32 position index. Stream ephemeral container values directly into JSON, preserve numeric-key order and plain data, and benchmark identity-preserving reads and projection costs on cold imported documents.

Model: gpt-6
@zxch3n zxch3n changed the title feat(wasm): JSON text export of deep values feat(wasm): streaming JSON deep reads with container positions Sep 5, 2026
Replace the unpublished positional JSON proposal with explicit nested container nodes and opaque ordinary values. See context/wasm-bulk-read.md for semantics and measurements.
@zxch3n zxch3n changed the title feat(wasm): streaming JSON deep reads with container positions feat(wasm): construct typed readState snapshots with fixed JS helpers Sep 5, 2026
@zxch3n zxch3n changed the title feat(wasm): construct typed readState snapshots with fixed JS helpers feat(wasm): add typed container trees with fixed JS construction Sep 5, 2026
@zxch3n
zxch3n merged commit c594ee0 into main Sep 6, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant