feat(decode): shared DecodeSession core + the Android activity leg (SKaiNET#1244) - #395
Merged
Merged
Conversation
… (SKaiNET#1244) The skainet-decode flow — GGUF peek, tokenizer, family-dispatched weight load, OptimizedLLMRuntime, and the traced prefill/decode/sample loop that feeds GenerationMetrics — moves verbatim into a new KMP module (jvm + android) so the Android activity leg reports the same numbers from the same loop. The JVM CLI becomes a thin caller owning only arg parsing, the JvmRandomAccessSource, and stdout; its output is unchanged. One addition over the JVM-only flow: MemoryProbe.sample().emitTo(sink) inside every decode span, which lights the page-fault/RSS rows on any platform with a /proc answer (Android, Linux) and stays "—" elsewhere. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…(SKaiNET#1244) First com.android.application in the repo: llm-apps:skainet-decode-android, a thin caller of skainet-decode-core's DecodeSession. Pre-flight AndroidGguf.fits verdict before a byte of payload is read, mapped weights via MappedRandomAccessSource (major-fault rate becomes meaningful), AndroidTraceSink composed for Perfetto, the whole session on one worker thread (RecordingTraceSink is single-threaded), report mirrored to logcat tag SkDecode and decode-report.md in the app's external-files dir for adb pull. largeHeap=true as shipped; the SKEEP-002 256-MB-heap measurement flips it off (documented in the README with the kernel-fallback caveat). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements SKaiNET#1244 — the Android half of skainet-decode (the JVM leg shipped in #369/#370).
llm-apps:skainet-decode-core(new KMP library, jvm + android): the decode flow extracted fromMain.ktinto a commonDecodeSession— sink/ctx construction (RecordingTraceSink,KernelDispatch.defaultSink, delegating context,ensureInstalled()), BOS logic, the traced prefill/decodeStep/sample loop,GenerationMetrics.from(events, peakBytesPerSecond).MemoryProbe.sample().emitTo(sink)runs inside every decode span, so the page-fault/RSS rows light up on Android AND Linux JVM. OptionalextraSinkcomposesAndroidTraceSinkfor Perfetto. The JVM CLI becomes a thin caller — output verified byte-identical with a real SmolLM2-135M Q8_0 run.llm-apps:skainet-decode-android(the repo's firstcom.android.application, AGP 9 built-in Kotlin): single classic-Views activity — path pre-filled to the app's external-files dir (adb-push workflow, README'd; no SAF sincecontent://can't mmap by path, no assets), pre-flightAndroidGguf.deviceMemory/fits(weightsMapped = true)verdict that refuses before allocating, whole load+decode on one dedicated thread (RecordingTraceSinkis not thread-safe), decode viaMappedRandomAccessSourceso the major-fault rate is meaningful, report → screen + logcatSkDecode+decode-report.md.largeHeap=truewith the SKEEP-002 config-split note;runtimeOnlyjni-cpu AAR, no nativeCpu/FFM; packaging keepsMETA-INF/services/**so the 0.52.0 self-healing SPI survives — verified in the APK: both.sovariants plus theViewKernelPack/KernelProviderservice entries.Also:
androidApplicationcatalog alias, settings/root wiring, architecture.adoc module tree, module README with the SKEEP-002 acceptance numbers ("≤ 40 MB managed heap for SmolLM2-135M Q8_0"; "~600 MB Q4_K on a 256 MB heap", measured largeHeap-off).Verified:
:llm-apps:skainet-decode:run(JVM regression gate) and:llm-apps:skainet-decode-android:assembleDebug+ APK inspection. Remaining for the device lane (no device attached here): install on a physical arm64 device, capture the report pair, record the numbers.