feat(storage): off-heap mmap tensor storage on Android — shared java.nio path + MappedGgufWeights (#921) - #966
Merged
Conversation
…nio path, MappedGgufWeights (#921) On Android every tensor lives on the hard-capped ART heap (256 MB default, 512 MB largeHeap), which caps practical model size regardless of kernels. The off-heap machinery already existed but was jvmMain-only. It is pure java.nio (FileChannel.map — API 1 on Android, no JNI), so this change shares the source between the JVM and Android compilations via a jvmAndroidMain shared source directory (each compilation builds against its full platform classpath; no intermediate source set needed): - skainet-lang-core: MmapFloatTensorData / FloatBufferTensorData / MmapTensorSource (one un-chained ByteBuffer call sequence for the pre-Java-9 android.jar nio API — behavior identical) - skainet-io-core: JvmMappedMemoryChunk, MappedRandomAccessSource, and JvmFileBackedResolver — BufferHandle.FileBacked now resolves to real mmap-backed accessors on Android too (#927-#931 seam) New MappedGgufWeights (skainet-io-gguf, JVM+Android): opens a GGUF, parses metadata through a positional source (heap cost O(metadata)), maps the file read-only once, and serves - dense F32 tensors as zero-heap mapped views (mappedFloatTensor), - any tensor as a FileBacked TensorStorage descriptor (mappedStorage), - packed payloads as heap bytes for existing kernels (packedBytes). Files >2 GB are rejected fast; windowed mapping is a SKEEP-003 follow-up. Verification (host-simulated — no device/emulator; stated per tracker): - MappedGgufHeapBudgetTest (jvmTest): a sparse 640 MB dense-F32 model — larger than the 512 MB ART budget — loads and sample-reads via mapped views with 1.4 MB managed-heap allocation (0.0022x of dense size, per-thread allocation counters). - androidHostTest (new withHostTest on skainet-io-gguf): the Android compilation of the identical shared source loads a 96 MB mapped tensor with ~240 KB used-heap growth; io-core androidHostTest covers chunk, mapped source and FileBacked resolver on the android variant. - Full jvmTest suites of all three modules + io linuxX64 tests green. lang-core apiCheck: jvm dump refreshed via apiDump — the diff is the pre-existing develop drift (Dim/Shape dynamic axes, #931 memory-report additions) that had jvmApiCheck failing on develop; this branch adds no JVM API change of its own (same classes, same target). apiCheck now passes. The api/android dump is not BCV-validated (jvm+klib only). Design-conformance: SKEEP-003 'Implementation Slices' — this is the #921 mobile destination-placement slice; deprecate-don't-delete respected (no public API removed or changed; sharing a source directory adds the same names to the Android variant). Refs #921 Refs SKEEP-002, SKEEP-003 (#932) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
📖 Documentation Preview The documentation has been built successfully for this PR. Generated Files:
Artifacts:
This comment will be updated automatically when the PR is updated. |
…dMain modules Dokka's pre-generation check refuses files that belong to two source sets (dokka#3701), which is exactly what the shared src/jvmAndroidMain source directory produces — the same files are compiled into both the jvm and android compilations by design. The android pages would duplicate the jvm ones anyway: document the shared API once via jvm. Fixes the build-docs and publish-validation legs on this PR; verified locally on all three modules (io-core, lang-core, io-gguf).
Contributor
Author
|
Both failing checks had the same root cause: Dokka's pre-generation validity check rejects the shared |
|
📖 Documentation Preview The documentation has been built successfully for this PR. Generated Files:
Artifacts:
This comment will be updated automatically when the PR is updated. |
aharakal
previously approved these changes
Aug 11, 2026
|
📖 Documentation Preview The documentation has been built successfully for this PR. Generated Files:
Artifacts:
This comment will be updated automatically when the PR is updated. |
aharakal
previously approved these changes
Aug 11, 2026
|
📖 Documentation Preview The documentation has been built successfully for this PR. Generated Files:
Artifacts:
This comment will be updated automatically when the PR is updated. |
aharakal
approved these changes
Aug 11, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements the Android off-heap/mmap slice of the storage roadmap: #921, per SKEEP-002 and the Implementation Slices section of SKEEP-003 (#932, amended in #963). On Android every tensor byte lives on the hard-capped ART heap (256 MB default / 512 MB
largeHeap), so practical model size is capped no matter how good the kernels are — while the file-backed pages llama.cpp/ORT/TFLite use sit outside that cap.What this does
The off-heap machinery already existed — it was just jvmMain-only, and it is pure
java.nio(FileChannel.mapis API 1 on Android; no JNI). This PR shares the source between the JVM and Android compilations via ajvmAndroidMainshared source directory (each compilation builds against its full platform classpath — no intermediate source set, no metadata-resolution risk):skainet-lang-core:MmapFloatTensorData/FloatBufferTensorData/MmapTensorSource(one ByteBuffer call sequence un-chained for the pre-Java-9android.jarnio API — behavior identical).skainet-io-core:JvmMappedMemoryChunk,MappedRandomAccessSource, andJvmFileBackedResolver—BufferHandle.FileBackednow resolves to real mmap-backed accessors on Android too, completing the TensorStorageFactory violates its own copy/borrow contracts: "zero-copy" borrowFloatArray copies, "borrowed" fromTensorData owns #927–Memory diagnostics: MemoryTracker.recordCopy discards the source label every caller passes; ActiveMemoryTracker is a mutable global #931 resolver seam on the second platform (issue ask MNIST data set loader #3: "make FileBacked real").skainet-io-gguf: newMappedGgufWeights(JVM+Android): opens a GGUF, parses metadata through a positional source (heap cost O(metadata)), maps the file read-only once, then servesmappedFloatTensor, wrap withctx.fromData),FileBackedTensorStoragedescriptor (mappedStorage),ByteArraykernels (packedBytes— their load transients are already streamed per tensor, GGUF DEQUANTIZE_TO_FP32 over-allocates: 1.1B Q4_K_M needs >12 GB heap transiently (~4.4 GB legit) #782/fix(io-gguf): stream GGUF dequantization — no boxed payloads, no defensive copies (#782) #965).Files >2 GB are rejected fast (single-region mapping uses int offsets); windowed mapping is a named follow-up under SKEEP-003's IO-pipeline improvement. ByteBuffer-backed packed tensor storage (kernels consuming mapped bytes zero-copy) is likewise SKEEP-003 territory, not this slice.
Verification — host-simulated (no device/emulator available; stated per the tracker)
MappedGgufHeapBudgetTest(jvmTest) — the 512 MB-budget gate: a sparse 640 MB dense-F32 model (larger than the ART large-heap cap) loads and sample-reads through mapped views with 1.4 MB of managed-heap allocation (0.0022x of the dense size), measured with per-thread allocation counters; sentinel floats round-trip through the mapping bit-exactly.androidHostTest(newwithHostTest {}onskainet-io-gguf) — the Android compilation of the identical shared source: 96 MB mapped tensor, ~240 KB used-heap growth, FileBacked→resolver→bytes round-trip on the android variant.skainet-io-coregainsMappedMemoryChunkAndroidHostTest(chunk reads/slices, mapped source, resolver).MappedGgufWeightsTest(jvmTest) — functional: mapped views vs file contents,copyMaterialize(resolver)FileBacked→Owned, packed-bytes fallback, fail-fast guards.apiCheck / compatibility
skainet-lang-core:apiChecknow passes: the jvm dump refresh in this PR consists entirely of pre-existing develop drift (Dim/Shapedynamic axes, Memory diagnostics: MemoryTracker.recordCopy discards the source label every caller passes; ActiveMemoryTracker is a mutable global #931 memory-report additions) that hadjvmApiCheckfailing on develop before this branch — called out here explicitly; this PR adds no JVM API of its own. (BCV validates jvm+klib; theapi/androiddump file is not part of validation.)SKEEP-003 conformance
This is the "Android off-heap / mmap" slice exactly as scoped in the SKEEP's Implementation Slices table: the mobile destination-placement stage of cross-cutting improvement 4, landing against the existing types without prejudging the storage-first/data-first end-state —
TensorDataview (MmapFloatTensorData) andTensorStoragedescriptor (FileBacked+ resolver) are both served, so whichever byte-owner survives the #932 decision keeps this path.Closes #921 (first slice: dense-F32 mmap + FileBacked resolution on Android; packed-zero-copy and >2 GB windowing tracked under SKEEP-003)
Refs #932, SKEEP-002, #963, #965
🤖 Generated with Claude Code