runtime: PIC package composition (decoupled allocator + dynamic-linking loader + unbounded/freed host returns)#33
Merged
Merged
Conversation
colinrozzi
force-pushed
the
pic-composition
branch
2 times, most recently
from
8abedb6 to
4d02643
Compare
…ic-linking loader, unbounded+freed host returns Replaces each package's baked-in dlmalloc with a shared, position-independent allocator side module, and teaches the runtime to act as a mini dynamic linker so an allocator + a package (and, ahead, package-to-package imports) share one linear memory without their stacks/data colliding — the substrate for packaging code and importing it across packages. - pack-guest: setup_guest! links an imported-allocator shim (pack:alloc) instead of embedding dlmalloc; __import_impl frees guest-allocated host-return buffers. - packages/pack-alloc: PIC dlmalloc side module (real free, bounded memory). - runtime: PIC loader (per-instance Memory+Table, disjoint __memory_base, package __stack_pointer, GOT heap globals, GOT.mem.__data_end resolution, __wasm_apply_data_relocs + __wasm_call_ctors) on both the sync path and the async/theater host-fn path; host marshalling moved off get_export(memory) to a host-owned Memory. - host returns: async host fns (theater's large returns) guest-allocate on the heap — UNBOUNDED — and are freed after decode via an ownership bit in the import status (1=guest-owned/free, 0=host-owned scratch); sync fns keep the bounded fixed buffer. Teeth-verified no-leak regression. Three fixes from theater-dev's soak of a real (mesh) actor: - Loader calls __wasm_apply_data_relocs before ctors — else stored data pointers (format!()'s static &str fragments) keep raw offsets and read blank. - pack-guest 'pic' feature defines placeholder __heap_base/__data_end so PIC side modules link on rustc > 1.92 (which injects those exports; lld can't satisfy them under -shared). PIC actors enable packr-guest/pic. - Loader resolves GOT.mem.__data_end (which real actors' dep trees import) to __memory_base + the module's exported offset; without it, instantiation fails. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
colinrozzi
force-pushed
the
pic-composition
branch
from
4d02643 to
d9983c9
Compare
This was referenced Jul 10, 2026
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
PIC package composition — decoupled allocator, dynamic-linking loader, unbounded+freed host returns
The substrate for packaging code and importing it across packages: replace each package's baked-in dlmalloc with a shared, position-independent (PIC) allocator side module, and make the runtime a mini dynamic linker so an allocator + a package share one linear memory without their stacks/data colliding.
What's in it
setup_guest!links an imported-allocator shim (pack:alloc) instead of embedding dlmalloc;__import_implfrees guest-allocated host-return buffers.free, bounded memory.Memory+Table, disjoint__memory_base, package__stack_pointer, GOT heap globals,__wasm_call_ctors) on both the sync path and the async/theater host-fn path; host marshalling moved offget_export("memory")to a host-ownedMemory.1=guest-owned/free,0=host-owned scratch). Sync fns keep the bounded fixed buffer.Verification
pic_composition(10 tests): every value shape round-trips through the shared in-wasm allocator, concurrent disjoint allocation, async large-return guest-allocation, and a teeth-verified no-leak regression (async_host_fn_large_returns_do_not_leak: 60 large async returns — memory grows ~15 MiB without the free, ~0 with it; verified it fails without the free).-D warnings, and fmt all green..cargo/configlink flags) before the test step.This is the fleet-committing PIC feature. Two gates before merge + release:
func_async*, unbounded). Sync host fns (func_typed*) keep the ~32KB fixed-buffer cap, which is PIC-layout-safe. If any theater host fn returns >32KB via a sync registration, it needs to move to async.Merging this does not publish (version stays 0.7.3); the release/publish is a separate, gated step after the soak.
🤖 Generated with Claude Code