From 21f2f0b60b2aec3ebe18c55b29615e56485cb695 Mon Sep 17 00:00:00 2001 From: michal harakal Date: Tue, 11 Aug 2026 23:33:29 +0200 Subject: [PATCH] release: prepare 0.40.0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bump VERSION_NAME 0.39.1 -> 0.40.0 and update all version-carrying docs. The tagged 0.39.1 release was scoped narrowly to #950 + #952; #708 (Q5_0/Q5_1 native kernels), #958 (Apple runtime FEAT_DotProd dispatch), and #959 (Apple iOS/macOS Kotlin/Native targets) were merged to develop afterward but their changelog entries had been appended under the already-shipped [0.39.1] header. This release corrects that: those entries move to [0.40.0] alongside the two items that were genuinely still Unreleased (#921 off-heap/mmap Android storage, #782 GGUF dequant over-allocation fix), and [0.39.1] is restored to match what the 0.39.1 tag actually shipped. - CHANGELOG: new [0.40.0] - 2026-08-11 section (headline + #921, #782, #708, #958, #959); [0.39.1] section reverted to its tagged content (#949, #923 only). - README: BOM snippet -> 0.40.0, What's New in 0.40.0 (0.39.1 moves under "Previously"), Contributors (0.40.0). - docs/antora.yml: skainet_version attribute 0.39.1 -> 0.40.0. - kernel-support-matrix.adoc: regenerated via generateKernelMatrix on this base — version stamp -> 0.40.0. Branch prep only — no tag created. Co-Authored-By: Claude Sonnet 5 --- CHANGELOG.md | 130 ++++++++++-------- README.md | 15 +- docs/antora.yml | 2 +- .../reference/kernel-support-matrix.adoc | 2 +- gradle.properties | 2 +- 5 files changed, 87 insertions(+), 64 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 66f810ee7..b639730d8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,18 @@ ## [Unreleased] +## [0.40.0] - 2026-08-11 + +Headline: **big models fit on real devices, and SKaiNET reaches iOS/macOS +natively.** Off-heap/mmap tensor storage lets Android load models beyond the +hard ART heap cap by paging weight bytes from mapped files instead of the +managed heap, and a compounding GGUF dequantization bug that transiently +needed >12 GB heap for a 1.1B Q4_K_M model is fixed down to a ~1.05x-of-dense +floor. Q5_0/Q5_1 packed matmul reaches the native tier (FFM, Kotlin/Native, +JNI) for the first time, and `skainet-backend-native-cpu` now publishes +iOS/macOS Kotlin/Native targets whose single Apple arm64 archive dispatches +FEAT_DotProd at runtime — one build serves A12 through M-series. + ### Added - **Off-heap / mmap tensor storage on Android** @@ -21,6 +33,31 @@ new `androidHostTest` suites (96 MB payload, ~240 KB used-heap growth). Files over 2 GB are rejected fast (single-region mapping); windowed mapping is a follow-up under SKEEP-003's IO pipeline improvement. +- **Native Q5_0 / Q5_1 packed matmul kernels (FFM, Kotlin/Native, JNI).** 0.39.0 shipped + packed GGUF *loading* for Q5_0/Q5_1 plus scalar + Panama kernels, but the native tier had + no Q5_x kernels — on the JVM the registry cascaded to Panama (50), and on Kotlin/Native and + Android the formats ran on the priority-0 scalar floor. New `skainet_q5_0_matmul` / + `skainet_q5_1_matmul` C kernels (plain NEON, no dotprod/i8mm requirement — runs on every + AArch64 core) expand the `qh` high-bit plane with a per-lane `vtstq_u8` bit test and fold + the dequant algebraically (`d*(dot - 16*Σx)` for Q5_0, `d*dot + m*Σx` for Q5_1) so the + per-block input sum hoists out of the output-row loop. Wired into all three consumers: + the FFM `NativeKernelProvider` (JVM), the cinterop `NativeKnKernelProvider` + (Kotlin/Native), and the Android JNI bridge (`JniKernels.q50Matmul`/`q51Matmul` + + `JniKernelProvider`), each with parity tests against the scalar references. Unblocks the + packed Q5_1 path for `functiongemma-270m` "Q5_K_M" checkpoints (whose attention/FFN + weights are Q5_1) under `NATIVE_OPTIMIZED` — see SKaiNET-transformers#170. (#708) +- **`skainet-backend-native-cpu` publishes Apple Kotlin/Native targets** — `iosArm64`, + `iosSimulatorArm64`, and `macosArm64` klibs with the Mach-O kernel static archive embedded + via cinterop, exactly like the Linux pair ([#959](https://github.com/SKaiNET-developers/SKaiNET/issues/959), + iOS kernel track of [#920](https://github.com/SKaiNET-developers/SKaiNET/issues/920)). + Archives are built by new Apple CMake lanes on a macOS host (platform SDKs, no `-march` — + the #958 runtime FEAT_DotProd dispatch serves A12 through M-series from one device archive) + or injected in CI via `-PskainetKernelsIosArm64Dir` / `-PskainetKernelsIosSimulatorArm64Dir` / + `-PskainetKernelsMacosArm64Dir`. The shared nativeTest parity suites now also run as + `macosArm64Test` (native) and `iosSimulatorArm64Test` (simulator) on the macos-14 PR lane. + On non-macOS hosts the Apple task family is disabled (ubuntu CI unaffected). Registration + on K/N remains manual via `installNativeKernels()` — Apple consumers call it once at startup, + same as Linux. ### Fixed @@ -43,6 +80,40 @@ (`NATIVE_OPTIMIZED`) keeps the loader's historical packed-block behavior bit-for-bit; a parity test pins the dequant path to the packed accessors bit-exactly across all seven supported quant formats. + +### Performance + +- **Apple arm64 runtime FEAT_DotProd dispatch for the Q4_K/Q6_K C kernels** + (`skainet-backend-native-cpu`, [#958](https://github.com/SKaiNET-developers/SKaiNET/issues/958), + part of the iOS kernel track of [#920](https://github.com/SKaiNET-developers/SKaiNET/issues/920)). + Apple builds now compile at the SDK-default arm64 baseline — a Kotlin/Native klib embeds + exactly one static archive, and Apple A12 (iPhone XS/XR, still iOS-supported) lacks + FEAT_DotProd while A13+/M-series have it — with the dotprod hot bodies compiled twice + (baseline + `target("dotprod")`-attributed) and selected once per matmul via a cached + `sysctlbyname("hw.optional.arm.FEAT_DotProd")` probe. Non-Apple builds keep the compile-time + `-march` guard as the only mechanism; Linux codegen is unchanged (qemu parity green, `sdot` + verified in the cross archive). The existing macOS FFM dylib moves from TU-level dotprod to + baseline+dispatch — runtime-equivalent on every Apple Silicon Mac. iOS builds are static-only + (`SKAINET_STATIC_ONLY`, auto-on for `CMAKE_SYSTEM_NAME=iOS`). + +### CI + +- **Releases embed the Apple kernel archives** ([#959](https://github.com/SKaiNET-developers/SKaiNET/issues/959)): + publish.yml's macos leg builds the three Mach-O static archives (with an `nm`/`objdump` + `sdot` assertion guarding the #958 dispatch body against clang's silent unknown-feature + ignore), uploads them fail-loud, and the publish job verifies and injects them via the + `-PskainetKernels{IosArm64,IosSimulatorArm64,MacosArm64}Dir` properties — the same + verified-artifact-or-fail contract as the Linux ELF archives. + +### Documentation + +- **Kernel support matrix gains the `native-cinterop` tier** (Native·linux + Native·apple, + the 7 packed-quant formats) — the Native·linux column was under-reported as `scalar` + before; both native columns now reflect `NativeKnKernelProvider` + ([#959](https://github.com/SKaiNET-developers/SKaiNET/issues/959)). The eager-backends + mindmap and the `installNativeKernels()` KDoc document the manual-registration contract + and the Apple A12 dispatch fallback. + ## [0.39.1] - 2026-08-11 Headline: **eager overhead off the JVM is gone.** The eager CPU ops gain @@ -58,21 +129,7 @@ rebuilt per access. The README now points LLM users to SKaiNET-transformers. "Start in 5 minutes" says plainly that LLM inference lives in the SKaiNET-transformers repository — this repo is the engine underneath — and names the `sk.ainet.transformers` artifacts and BOM to depend on. -### Added -- **Native Q5_0 / Q5_1 packed matmul kernels (FFM, Kotlin/Native, JNI).** 0.39.0 shipped - packed GGUF *loading* for Q5_0/Q5_1 plus scalar + Panama kernels, but the native tier had - no Q5_x kernels — on the JVM the registry cascaded to Panama (50), and on Kotlin/Native and - Android the formats ran on the priority-0 scalar floor. New `skainet_q5_0_matmul` / - `skainet_q5_1_matmul` C kernels (plain NEON, no dotprod/i8mm requirement — runs on every - AArch64 core) expand the `qh` high-bit plane with a per-lane `vtstq_u8` bit test and fold - the dequant algebraically (`d*(dot - 16*Σx)` for Q5_0, `d*dot + m*Σx` for Q5_1) so the - per-block input sum hoists out of the output-row loop. Wired into all three consumers: - the FFM `NativeKernelProvider` (JVM), the cinterop `NativeKnKernelProvider` - (Kotlin/Native), and the Android JNI bridge (`JniKernels.q50Matmul`/`q51Matmul` + - `JniKernelProvider`), each with parity tests against the scalar references. Unblocks the - packed Q5_1 path for `functiongemma-270m` "Q5_K_M" checkpoints (whose attention/FFN - weights are Q5_1) under `NATIVE_OPTIMIZED` — see SKaiNET-transformers#170. (#708) ### Performance - **Primitive FP32 fast paths for the eager CPU ops** (`skainet-backend-cpu`, @@ -92,51 +149,6 @@ rebuilt per access. The README now points LLM users to SKaiNET-transformers. - **`DirectCpuExecutionContext.ops` is cached.** The getter previously constructed a fresh ops instance on every access, re-running per-instance lazy kernel resolution in the eager hot loop ([#949](https://github.com/SKaiNET-developers/SKaiNET/issues/949)). -- **Apple arm64 runtime FEAT_DotProd dispatch for the Q4_K/Q6_K C kernels** - (`skainet-backend-native-cpu`, [#958](https://github.com/SKaiNET-developers/SKaiNET/issues/958), - part of the iOS kernel track of [#920](https://github.com/SKaiNET-developers/SKaiNET/issues/920)). - Apple builds now compile at the SDK-default arm64 baseline — a Kotlin/Native klib embeds - exactly one static archive, and Apple A12 (iPhone XS/XR, still iOS-supported) lacks - FEAT_DotProd while A13+/M-series have it — with the dotprod hot bodies compiled twice - (baseline + `target("dotprod")`-attributed) and selected once per matmul via a cached - `sysctlbyname("hw.optional.arm.FEAT_DotProd")` probe. Non-Apple builds keep the compile-time - `-march` guard as the only mechanism; Linux codegen is unchanged (qemu parity green, `sdot` - verified in the cross archive). The existing macOS FFM dylib moves from TU-level dotprod to - baseline+dispatch — runtime-equivalent on every Apple Silicon Mac. iOS builds are static-only - (`SKAINET_STATIC_ONLY`, auto-on for `CMAKE_SYSTEM_NAME=iOS`). - -### Added - -- **`skainet-backend-native-cpu` publishes Apple Kotlin/Native targets** — `iosArm64`, - `iosSimulatorArm64`, and `macosArm64` klibs with the Mach-O kernel static archive embedded - via cinterop, exactly like the Linux pair ([#959](https://github.com/SKaiNET-developers/SKaiNET/issues/959), - iOS kernel track of [#920](https://github.com/SKaiNET-developers/SKaiNET/issues/920)). - Archives are built by new Apple CMake lanes on a macOS host (platform SDKs, no `-march` — - the #958 runtime FEAT_DotProd dispatch serves A12 through M-series from one device archive) - or injected in CI via `-PskainetKernelsIosArm64Dir` / `-PskainetKernelsIosSimulatorArm64Dir` / - `-PskainetKernelsMacosArm64Dir`. The shared nativeTest parity suites now also run as - `macosArm64Test` (native) and `iosSimulatorArm64Test` (simulator) on the macos-14 PR lane. - On non-macOS hosts the Apple task family is disabled (ubuntu CI unaffected). Registration - on K/N remains manual via `installNativeKernels()` — Apple consumers call it once at startup, - same as Linux. - -### CI - -- **Releases embed the Apple kernel archives** ([#959](https://github.com/SKaiNET-developers/SKaiNET/issues/959)): - publish.yml's macos leg builds the three Mach-O static archives (with an `nm`/`objdump` - `sdot` assertion guarding the #958 dispatch body against clang's silent unknown-feature - ignore), uploads them fail-loud, and the publish job verifies and injects them via the - `-PskainetKernels{IosArm64,IosSimulatorArm64,MacosArm64}Dir` properties — the same - verified-artifact-or-fail contract as the Linux ELF archives. - -### Documentation - -- **Kernel support matrix gains the `native-cinterop` tier** (Native·linux + Native·apple, - the 7 packed-quant formats) — the Native·linux column was under-reported as `scalar` - before; both native columns now reflect `NativeKnKernelProvider` - ([#959](https://github.com/SKaiNET-developers/SKaiNET/issues/959)). The eager-backends - mindmap and the `installNativeKernels()` KDoc document the manual-registration contract - and the Apple A12 dispatch fallback. ## [0.39.0] - 2026-08-10 diff --git a/README.md b/README.md index 5d5af86c5..e22990639 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.39.1")) + implementation(platform("sk.ainet:skainet-bom:0.40.0")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -297,7 +297,14 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.39.1 +## What's New in 0.40.0 + +- **Android models grow past the ART heap cap.** Off-heap/mmap tensor storage shares the JVM's memory-mapped weight loading with Android — dense F32 tensors serve as zero-heap mapped views, and weight bytes live in OS-paged file-backed pages instead of the managed heap. A 640 MB dense model now loads with **1.4 MB** of heap allocation. +- **GGUF `DEQUANTIZE_TO_FP32` no longer over-allocates.** A 1.1B Q4_K_M GGUF transiently needed >12 GB heap against a ~4.4 GB dense-FP32 floor. Three compounding allocation sources in the loader and K-quant kernels are fixed, bringing peak live allocation to ~1.05x of the dense FP32 size. +- **Q5_0/Q5_1 packed matmul reaches the native tier.** New NEON C kernels for both formats are wired into the FFM (JVM), Kotlin/Native, and Android JNI providers — unblocking the packed Q5_1 path for Q5_K_M checkpoints under `NATIVE_OPTIMIZED`. +- **SKaiNET reaches iOS and macOS natively.** `skainet-backend-native-cpu` now publishes `iosArm64`, `iosSimulatorArm64`, and `macosArm64` Kotlin/Native targets with embedded kernel archives; a single Apple arm64 archive dispatches FEAT_DotProd at runtime, so one build serves A12 through M-series. + +### Previously, in 0.39.1 - **Eager CPU ops run primitive FP32 fast paths.** The generic per-element paths (index-array allocations, boxed accessors, dtype dispatch) dominated on-device LLM decode — 83% of end-to-end SmolLM2-135M decode time on a Pixel 8a was non-matmul overhead even with the NEON backend. Hot ops (arithmetic, activations, unary math, softmax/logSoftmax, reductions, concat, reshape) now run flat primitive loops over the dense `FloatArray` buffer, benefiting every non-JVM target — Android, Kotlin/Native, JS/Wasm. `DirectCpuExecutionContext.ops` is also cached instead of rebuilt per access. - **README points LLM users to SKaiNET-transformers.** A callout under "Start in 5 minutes" makes clear that LLM inference lives in the SKaiNET-transformers repository — this repo is the engine underneath. @@ -347,6 +354,10 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.40.0) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — off-heap/mmap tensor storage on Android (#921), GGUF `DEQUANTIZE_TO_FP32` over-allocation fix (#782), native Q5_0/Q5_1 packed matmul kernels (#708), Apple arm64 runtime FEAT_DotProd dispatch (#958), Apple iOS/macOS Kotlin/Native kernel targets (#959) + ### Contributors (0.39.1) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — primitive FP32 fast paths for the eager CPU ops (#949), README pointer to SKaiNET-transformers (#923) diff --git a/docs/antora.yml b/docs/antora.yml index 32a7c123f..4737fc66e 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.39.1 + skainet_version: 0.40.0 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index 798c0d3e3..6709ff4a5 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.39.1`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.40.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). diff --git a/gradle.properties b/gradle.properties index 579c2f7a0..dce5d82b2 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.39.1 +VERSION_NAME=0.40.0 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/