From 52b1c08c6f8d769972233329e52a16885c60801a Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Tue, 11 Aug 2026 12:07:05 +0200 Subject: [PATCH 1/2] Merge pull request #952 from SKaiNET-developers/fix/readme-transformers-pointer-923 docs(readme): point LLM users to SKaiNET-transformers (#923) --- CHANGELOG.md | 8 ++++++++ README.md | 9 +++++++++ 2 files changed, 17 insertions(+) diff --git a/CHANGELOG.md b/CHANGELOG.md index 04bc65f53..12737797c 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,14 @@ ## [Unreleased] +### Documentation + +- **README points LLM users to SKaiNET-transformers** + ([#923](https://github.com/SKaiNET-developers/SKaiNET/issues/923)): a callout under + "Start in 5 minutes" says plainly that LLM inference lives in the + SKaiNET-transformers repository — this repo is the engine underneath — and names + the `sk.ainet.transformers` artifacts and BOM to depend on. + ### Performance - **Primitive FP32 fast paths for the eager CPU ops** (`skainet-backend-cpu`, diff --git a/README.md b/README.md index fb1db1056..c8cd4eef6 100644 --- a/README.md +++ b/README.md @@ -32,6 +32,15 @@ matches what you want to try first. Working in Java? SKaiNET ships first-class Java support — see the [Java getting-started guide](docs/modules/ROOT/pages/tutorials/java-getting-started.adoc). +> [!NOTE] +> **Looking for LLM inference?** Llama, Qwen, Gemma, Apertus, BERT embeddings and +> GGUF chat models live in +> [**SKaiNET-transformers**](https://github.com/SKaiNET-developers/SKaiNET-transformers) — +> this repository is the engine underneath it (tensors, NN DSL, compiler, CPU/native +> backends, GGUF/SafeTensors IO). Depend on the `sk.ainet.transformers` artifacts, +> pinned together by the +> [transformers BOM](https://central.sonatype.com/artifact/sk.ainet.transformers/skainet-transformers-bom). + Use the version shown in this README as the source of truth for first-run snippets. If another page shows a different version, please open an issue or PR. From 31d6a5d1331baa3704e324fce76e1c5e35102c45 Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Tue, 11 Aug 2026 12:39:52 +0200 Subject: [PATCH 2/2] release: prepare 0.39.1 (scoped: #950 + #952) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bump VERSION_NAME 0.39.0 -> 0.39.1 and update all version-carrying docs for a release scoped to #950 and #952 only — #951 (native Q5_0/Q5_1 packed matmul kernels) is deferred to the next release: - CHANGELOG: consolidate [Unreleased] under [0.39.1] (2026-08-11); delta is #950 (primitive FP32 fast paths for the eager CPU ops + cached ctx.ops, #949) and #952 (README pointer to SKaiNET-transformers, #923). - README: BOM snippet -> 0.39.1, What's New in 0.39.1 (0.39.0 moves under "Previously"), Contributors (0.39.1). - docs/antora.yml: skainet_version attribute 0.39.0 -> 0.39.1. - kernel-support-matrix.adoc: regenerated via generateKernelMatrix on this base — Q5_0/Q5_1 rows stay panama-vector/panama-vector (no #951 kernels); version stamp -> 0.39.1. Branch prep only — no tag created. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 8 ++++++++ README.md | 13 +++++++++++-- docs/antora.yml | 2 +- .../ROOT/pages/reference/kernel-support-matrix.adoc | 2 +- gradle.properties | 2 +- 5 files changed, 22 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 12737797c..713a3c601 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,14 @@ ## [Unreleased] +## [0.39.1] - 2026-08-11 + +Headline: **eager overhead off the JVM is gone.** The eager CPU ops gain +primitive FP32 fast paths, removing the per-element allocation/boxing overhead +that dominated on-device LLM decode (83% of end-to-end time on a Pixel 8a even +with NEON matmul), and `DirectCpuExecutionContext.ops` is cached instead of +rebuilt per access. The README now points LLM users to SKaiNET-transformers. + ### Documentation - **README points LLM users to SKaiNET-transformers** diff --git a/README.md b/README.md index c8cd4eef6..5d5af86c5 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.39.0")) + implementation(platform("sk.ainet:skainet-bom:0.39.1")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -297,7 +297,12 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.39.0 +## What's New in 0.39.1 + +- **Eager CPU ops run primitive FP32 fast paths.** The generic per-element paths (index-array allocations, boxed accessors, dtype dispatch) dominated on-device LLM decode — 83% of end-to-end SmolLM2-135M decode time on a Pixel 8a was non-matmul overhead even with the NEON backend. Hot ops (arithmetic, activations, unary math, softmax/logSoftmax, reductions, concat, reshape) now run flat primitive loops over the dense `FloatArray` buffer, benefiting every non-JVM target — Android, Kotlin/Native, JS/Wasm. `DirectCpuExecutionContext.ops` is also cached instead of rebuilt per access. +- **README points LLM users to SKaiNET-transformers.** A callout under "Start in 5 minutes" makes clear that LLM inference lives in the SKaiNET-transformers repository — this repo is the engine underneath. + +### Previously, in 0.39.0 - **On-device AI on Android — a NEON kernel backend.** New `skainet-backend-jni-cpu` module: the hand-tuned ARM matmul kernels reach Android through a JNI bridge (ART has no `java.lang.foreign`, so the FFM provider can never run there). Two `.so` tiers are built from the same sources and selected at load time from `/proc/cpuinfo` — a baseline `armv8-a` build that runs on every 64-bit core, and an `armv8.2-a+dotprod` build for the `vdotq_s32` Q4_K/Q6_K paths — so a single artifact is safe from Cortex-A53 up. Measured on a Pixel 8a: **~24 tok/s** SmolLM2-135M Q8_0 decode versus ~3.8 scalar (6.4x), clearing the on-device usability bar. The provider auto-registers via `ServiceLoader`; an app just adds the AAR. - **Android GGUF loading no longer OOMs.** `createRandomAccessSource` returned `null` on Android, forcing every model load through a full-file heap read that exhausted the ART heap on real devices. It now streams via positional `FileChannel` reads across `skainet-io-gguf` / `-safetensors` / `-onnx`. @@ -342,6 +347,10 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.39.1) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — primitive FP32 fast paths for the eager CPU ops (#949), README pointer to SKaiNET-transformers (#923) + ### Contributors (0.39.0) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — Android JNI NEON kernel backend with runtime dotprod dispatch (#943, #945), Android `createRandomAccessSource` streaming loads (#922), cinterop klib archive embedding (#942), Q4_0 NEON kernel (#939), GGUF loader fail-fast (#919), tensor-storage correctness fixes (#927, #928, #929, #930, #931), AAR release publishing (#947) diff --git a/docs/antora.yml b/docs/antora.yml index 08a0a93f3..32a7c123f 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.39.0 + skainet_version: 0.39.1 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index c79a502d9..8b67e2813 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.39.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.39.1`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). diff --git a/gradle.properties b/gradle.properties index 54e0d8c0d..579c2f7a0 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.39.0 +VERSION_NAME=0.39.1 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/