diff --git a/CHANGELOG.md b/CHANGELOG.md index dfa80f2e2..d6a35df33 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,14 @@ ## [Unreleased] +## [0.39.1] - 2026-08-11 + +Headline: **eager overhead off the JVM is gone.** The eager CPU ops gain +primitive FP32 fast paths, removing the per-element allocation/boxing overhead +that dominated on-device LLM decode (83% of end-to-end time on a Pixel 8a even +with NEON matmul), and `DirectCpuExecutionContext.ops` is cached instead of +rebuilt per access. The README now points LLM users to SKaiNET-transformers. + ### Documentation - **README points LLM users to SKaiNET-transformers** diff --git a/README.md b/README.md index c8cd4eef6..5d5af86c5 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.39.0")) + implementation(platform("sk.ainet:skainet-bom:0.39.1")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -297,7 +297,12 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.39.0 +## What's New in 0.39.1 + +- **Eager CPU ops run primitive FP32 fast paths.** The generic per-element paths (index-array allocations, boxed accessors, dtype dispatch) dominated on-device LLM decode — 83% of end-to-end SmolLM2-135M decode time on a Pixel 8a was non-matmul overhead even with the NEON backend. Hot ops (arithmetic, activations, unary math, softmax/logSoftmax, reductions, concat, reshape) now run flat primitive loops over the dense `FloatArray` buffer, benefiting every non-JVM target — Android, Kotlin/Native, JS/Wasm. `DirectCpuExecutionContext.ops` is also cached instead of rebuilt per access. +- **README points LLM users to SKaiNET-transformers.** A callout under "Start in 5 minutes" makes clear that LLM inference lives in the SKaiNET-transformers repository — this repo is the engine underneath. + +### Previously, in 0.39.0 - **On-device AI on Android — a NEON kernel backend.** New `skainet-backend-jni-cpu` module: the hand-tuned ARM matmul kernels reach Android through a JNI bridge (ART has no `java.lang.foreign`, so the FFM provider can never run there). Two `.so` tiers are built from the same sources and selected at load time from `/proc/cpuinfo` — a baseline `armv8-a` build that runs on every 64-bit core, and an `armv8.2-a+dotprod` build for the `vdotq_s32` Q4_K/Q6_K paths — so a single artifact is safe from Cortex-A53 up. Measured on a Pixel 8a: **~24 tok/s** SmolLM2-135M Q8_0 decode versus ~3.8 scalar (6.4x), clearing the on-device usability bar. The provider auto-registers via `ServiceLoader`; an app just adds the AAR. - **Android GGUF loading no longer OOMs.** `createRandomAccessSource` returned `null` on Android, forcing every model load through a full-file heap read that exhausted the ART heap on real devices. It now streams via positional `FileChannel` reads across `skainet-io-gguf` / `-safetensors` / `-onnx`. @@ -342,6 +347,10 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.39.1) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — primitive FP32 fast paths for the eager CPU ops (#949), README pointer to SKaiNET-transformers (#923) + ### Contributors (0.39.0) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — Android JNI NEON kernel backend with runtime dotprod dispatch (#943, #945), Android `createRandomAccessSource` streaming loads (#922), cinterop klib archive embedding (#942), Q4_0 NEON kernel (#939), GGUF loader fail-fast (#919), tensor-storage correctness fixes (#927, #928, #929, #930, #931), AAR release publishing (#947) diff --git a/docs/antora.yml b/docs/antora.yml index 08a0a93f3..32a7c123f 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.39.0 + skainet_version: 0.39.1 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index c79a502d9..8b67e2813 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.39.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.39.1`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). diff --git a/gradle.properties b/gradle.properties index 54e0d8c0d..579c2f7a0 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.39.0 +VERSION_NAME=0.39.1 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/