feat(runtime): configurable garbage collector, memory leak fix, regression test#1248
Open
zaccharyfavere wants to merge 3 commits into
Open
feat(runtime): configurable garbage collector, memory leak fix, regression test#1248zaccharyfavere wants to merge 3 commits into
zaccharyfavere wants to merge 3 commits into
Conversation
4d54c4d to
87ae2cf
Compare
87ae2cf to
0e64d43
Compare
…, regression test Adds a GC configuration mechanism for orphan reactive resources, exposes Skip persistent memory usage as a metric, fixes a separate memory leak in the reactive read tracking, and ships a regression test that detects future memory leaks. GC configuration: - New GCConfig type and SkipService.gcConfig?: () => GCConfig in the public API. The config lives in user TypeScript and is invoked by the runtime each time it is needed, with no caching. - SkipRuntime_callGCConfigProvider FFI bridge (Skiplang -> JS via C++) following the same pattern as Mapper__mapEntry. - getGCConfig in Runtime.sk calls the lambda and parses the returned CJObject into a typed GCConfig, with fallback to defaults on error. - checkGarbage() sweeps expired resources from the garbage queue and evicts the oldest entries when the queue exceeds maxGarbageSize. - getGarbageQueueSize() returns the current queue depth. Memory metric: - New SKIP_get_persistent_size() in palloc.c (64-bit) aligned with the existing 32-bit version in runtime32_specific.c. - Exposed through the full FFI chain (Skiplang prelude binding, Runtime.sk wrapper, C++/WASM bindings, public TS method getSkipPersistentSize()). Reads leak fix: - Context.reads (SortedSet<Path>) accumulated read paths from root- level operations (createReactiveResource, closeReactiveResource) that were never released, unlike the per-mapper save/restore mechanism. - The set is now cleared at the end of each successful sweep in checkGarbage(). Regression test (skipruntime-ts/examples/stress-tests/gc-stress.ts): - Default mode (~30s) runs two tests on the two leak scenarios. - Full mode (-f) runs longer, exhaustive versions. - Computes a linear regression slope on memory measurements taken during a stable phase (after skipping the warmup window). - Fails if the slope exceeds 100 bytes/cycle threshold. - Uses getSkipPersistentSize() for byte-precise measurement. - Wrapped by a `test-mem-stress` Makefile target that handles LD_LIBRARY_PATH and SKIP_PLATFORM exports. Args pass-through: make test-mem-stress ARGS="--trace --full". Notes: - The "instantiate-close-with-map" sub-test currently FAILs on this branch alone because it exercises DeletedDir cleanup, which is in the separate fix-deleteddir-leak PR. The test passes once both PRs are merged together.
0e64d43 to
551acfc
Compare
…, regression test Adds a GC configuration mechanism for orphan reactive resources, exposes Skip persistent memory usage as a metric, fixes a separate memory leak in the reactive read tracking, and ships a regression test that detects future memory leaks. GC configuration: - New GCConfig type and SkipService.gcConfig?: () => GCConfig in the public API. The config lives in user TypeScript and is invoked by the runtime each time it is needed, with no caching. - SkipRuntime_callGCConfigProvider FFI bridge (Skiplang -> JS via C++) following the same pattern as Mapper__mapEntry. - getGCConfig in Runtime.sk calls the lambda and parses the returned CJObject into a typed GCConfig, with fallback to defaults on error. - checkGarbage() sweeps expired resources from the garbage queue and evicts the oldest entries when the queue exceeds maxGarbageSize. - getGarbageQueueSize() returns the current queue depth. Memory metric: - New SKIP_get_persistent_size() in palloc.c (64-bit) aligned with the existing 32-bit version in runtime32_specific.c. - Exposed through the full FFI chain (Skiplang prelude binding, Runtime.sk wrapper, C++/WASM bindings, public TS method getSkipPersistentSize()). Reads leak fix: - Context.reads (SortedSet<Path>) accumulated read paths from root- level operations (createReactiveResource, closeReactiveResource) that were never released, unlike the per-mapper save/restore mechanism. - The set is now cleared at the end of each successful sweep in checkGarbage(). Regression test (skipruntime-ts/examples/stress-tests/gc-stress.ts): - Default mode (~30s) runs two tests on the two leak scenarios. - Full mode (-f) runs longer, exhaustive versions. - Computes a linear regression slope on memory measurements taken during a stable phase (after skipping the warmup window). - Fails if the slope exceeds 100 bytes/cycle threshold. - Uses getSkipPersistentSize() for byte-precise measurement. - Wrapped by a `test-mem-stress` Makefile target that handles LD_LIBRARY_PATH and SKIP_PLATFORM exports. Args pass-through: make test-mem-stress ARGS="--trace --full". Notes: - The "instantiate-close-with-map" sub-test currently FAILs on this branch alone because it exercises DeletedDir cleanup, which is in the separate fix-deleteddir-leak PR. The test passes once both PRs are merged together.
bd1c802 to
f6071d4
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds a configurable GC sweep mechanism for orphan reactive resources, exposes Skip persistent memory (from palloc.c) usage as a metric, fixes a separate memory leak in the reactive read tracking, and ships a regression test that detects future memory leaks.
GC configuration:
Memory metric:
Reads leak fix:
Regression test (skipruntime-ts/examples/stress-tests/gc-stress.ts):
make test-mem-stress, with args pass-through viaARGS="--trace --full"Notes: