From 86ea2b17a02fd08f73fe6c0c38cf6b1515ea4ac7 Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Wed, 2 Sep 2026 15:05:58 +0200 Subject: [PATCH] =?UTF-8?q?release:=20prepare=200.53.0=20=E2=80=94=20lock-?= =?UTF-8?q?step=20with=20engine=200.53.0?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit CHANGELOG 0.53.0 block (skainet-decode on Android, SafeTensors loaders on the engine, the Gemma 3n export streaming fix, tool calling re-verified), README current-release section, and every published-coordinate reference bumped from 0.40.2 (docs tutorials, moonshine reference, gemma module README, gradle.properties VERSION_NAME). Co-Authored-By: Claude Fable 5.1 --- CHANGELOG.md | 57 +++++++++++++++++++ README.md | 57 +++++++++++-------- .../pages/reference/moonshine-encoder.adoc | 2 +- .../tutorials/android-getting-started.adoc | 2 +- .../pages/tutorials/getting-started-java.adoc | 4 +- .../pages/tutorials/getting-started-leaf.adoc | 2 +- .../pages/tutorials/llama3-tool-calling.adoc | 2 +- gradle.properties | 2 +- llm-inference/gemma/README.md | 6 +- 9 files changed, 101 insertions(+), 33 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 3ea8fff3..cd9d7068 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,63 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.53.0] — 2026-09-02 + +Version lock-step with the engine is restored: this release ships against **SKaiNET 0.53.0** +(#397), which brings the billion-parameter export fixes +([SKaiNET#1247](https://github.com/SKaiNET-developers/SKaiNET/issues/1247)) and the sharded +SafeTensors `ParametersLoader` ([SKaiNET#1246](https://github.com/SKaiNET-developers/SKaiNET/issues/1246)) +this repository's Gemma 3n export and family loaders were waiting on. Everything accumulated under +"Unreleased" since 0.40.2 — BitNet, the engine-loader migration, the Gemma 3n DSL path, the Qwen and +Apertus fixes — ships here too. + +### Added — skainet-decode on Android (SKaiNET#1244) + +- **`llm-apps:skainet-decode-core`** (#395): the decode-and-measure flow extracted from the JVM CLI + into a common `DecodeSession` (jvm + android) — traced prefill/decode/sample loop, + `MemoryProbe.sample().emitTo(sink)` inside every decode span so the page-fault/RSS rows of + `GenerationMetrics` populate on Android *and* Linux, optional extra `TraceSink` for Perfetto. + The JVM `skainet-decode` CLI is a thin caller with identical output. +- **`llm-apps:skainet-decode-android`** (#395): the repository's first `com.android.application` — + a single-activity app that loads a pushed GGUF through `MappedRandomAccessSource`, refuses via + `AndroidGguf.fits` before allocating, decodes on one dedicated thread, and reports + `GenerationMetrics` plus RSS/page-fault deltas to screen, logcat and `decode-report.md`. The APK + carries both JNI kernel variants and the `ViewKernelPack`/`KernelProvider` ServiceLoader entries, + so the engine's self-healing dispatch survives packaging. The physical-device measurement lane + (the SKEEP-002 numbers) is documented in the module README and still to be recorded. + +### Changed — SafeTensors loading rides the engine (SKaiNET#1246) + +- **Gemma** (#398), **the shared decoder loader** (#400), **Apertus and Gemma 3n** (#401): the + hand-rolled per-family SafeTensors materialization (bf16/f16 widening, byte decoding, size guards, + dead transposes) collapses onto the engine's `ShardedSafeTensorsParametersLoader` / + `SafeTensorsParametersLoader`. Each family keeps only its HF→GGUF slot table, name allowlist + (as the engine's `tensorFilter`) and any shape normalization; every dtype decision — including + `Require(BF16)`/`Require(FP16)` keep-native, now accepted on the SafeTensors lane — is the + engine's. Every collapsed loader gains a `dtypePolicy: DTypePolicy = Any` parameter (existing + call sites source-compatible) and a synthetic 2-shard fixture test written with the engine's + `SafeTensorsWriter`. Not yet collapsed: Voxtral (custom `QUANT4` format) and llm-core's legacy + Q4/non-float path, both waiting on a single-file `tensorFilter` + ([SKaiNET#1256](https://github.com/SKaiNET-developers/SKaiNET/issues/1256)). + +### Fixed — Gemma 3n export writes weights through the BufferResolver (SKaiNET#1247) + +- **`Gemma3nExportHarness.writeSafetensors`** (#396) no longer casts every constant to + `BufferHandle.Owned`: with engine 0.53.0, ≥2 GiB FP32 constants arrive as an aliased + `BufferHandle.Floats` (the tied embedding is exactly `Int.MAX_VALUE + 1` bytes), so the harness + streams every handle through `DefaultBufferResolver` in bounded chunks with the same chunked + bf16 conversion. The full 30-layer E2B export now emits a 15k-line StableHLO module with zero + failure comments and a 4.6 GB safetensors in under a minute, where it previously OOMed a + 46 GB heap. + +### Verified — tool calling against engine 0.53.0 + +- **`Gemma4E2BToolCallSmokeTest` re-enabled** (#399): the real Gemma 4 E2B Q4_K_M checkpoint now + emits `<|tool_call>call:calculator{expression:...}` and every assertion holds (it had been + `@Ignore`d for emitting prose without markup). Also re-run green in the same pass: + `FunctionGemmaOfficialGgufTest` (parsed `get_weather` call) and `QwenToolCallSmokeTest` with + Qwen3-1.7B-Q8 (well-formed `` calculator call). + ### Added — Gemma 3n StableHLO/IREE export harness + hybrid-AI design note - **`exportGemma3n`** (`Gemma3nExportHarness`, SmolLM2/FunctionGemma redecode pattern): diff --git a/README.md b/README.md index ae1a443f..50101691 100644 --- a/README.md +++ b/README.md @@ -108,28 +108,39 @@ Honest status — see the project-status note at the top of this README. ## Current release -The current release is **0.40.2** (against **SKaiNET 0.40.1**) — the release that rounds -out the **compiled on-device path**: FunctionGemma and SmolLM2 each get a standalone -DSL → StableHLO → IREE export module, and a new generic Android JNI runtime -(`llm-runtime/iree-android`) serves any compiled model the way `skainet-backend-jni-cpu` -already serves the eager path. - -**Tool-calling epic substrate lands.** Three stacked PRs complete the shared architecture -behind #35: HF-side chat-template auto-detection, registerable parser strategies, -resolution diagnostics, and validation against a real Qwen instruct checkpoint — -`AgentCli` now reports *why* a model resolved to native or generic tool-calling mode, in -every mode. - -**Packed-quant weights, one shared packer.** The GGUF-block → engine packing logic -gemma/llama/apertus each carried privately is hoisted into `BlockQuantPacking` -(transformer-core); Q5_0/Q5_1 join Q4_K/Q5_K/Q6_K/Q8_0 as kept-packed (instead of falling -back to FP32 dequant) now that engine 0.40.0 shipped native Q5 kernels. 0.40.2 also fixes -a real matmul-corruption regression the engine pin bump exposed on the classic -(non-pre-transposed) packed path — see [#311](https://github.com/SKaiNET-developers/SKaiNET-transformers/pull/311). - -**SmolLM2** joins the tool-calling families (`SmolLMChatTemplate` + parser strategy), and a -cross-target **SmolLM2-135M inference spike** in `kllama`'s commonTest gives directly comparable -load/tok-s numbers from one source on JVM, Linux native, and the iOS simulator. +The current release is **0.53.0** (against **SKaiNET 0.53.0**) — version lock-step with the engine +is restored, and the release ships everything accumulated since 0.40.2: BitNet b1.58, the +engine-loader migration (every GGUF weight loader a thin engine wrapper, MAPPED residency by +default), the Gemma 3n DSL path with its StableHLO/IREE export harness, and the Qwen / Apertus +decode and tool-calling fixes. + +**The Gemma 3n export actually emits.** Engine 0.53.0 closes SKaiNET#1247 — allocation-free void +tracing, aliased constant extraction, strict StableHLO conversion, and an array-free path for the +≥2 GiB tied embedding — and the harness streams weights through the `BufferResolver` (#396). The +full 30-layer E2B export produces a 15k-line module with zero failure comments and a 4.6 GB +safetensors in under a minute, where it previously OOMed a 46 GB heap. + +**SafeTensors loading rides the engine.** The per-family hand-rolled SafeTensors materialization +(Gemma, the shared decoder loader, Apertus, Gemma 3n) collapses onto the engine's sharded / +single-file `ParametersLoader`s (#398, #400, #401): each family keeps only its slot table and +allowlist, every dtype decision is the engine's, and `Require(BF16)`/`Require(FP16)` keep-native is +accepted on the SafeTensors lane. + +**skainet-decode on Android.** `llm-apps:skainet-decode-core` (the shared `DecodeSession`) and +`llm-apps:skainet-decode-android`, the repository's first Android application: load a pushed GGUF +memory-mapped, refuse before allocating when it doesn't fit, decode on one thread, and report +`GenerationMetrics` with the page-fault and RSS rows only a device can show (#395). + +**Tool calling re-verified against the new engine.** `Gemma4E2BToolCallSmokeTest` is re-enabled — +the real E2B checkpoint now emits parseable `<|tool_call>` markup (#399) — and FunctionGemma and +Qwen3 tool calling were re-run green in the same pass. + +It builds on **0.40.2**, which rounded out the **compiled on-device path** (standalone +DSL → StableHLO → IREE export modules for FunctionGemma and SmolLM2, the generic Android JNI +runtime `llm-runtime/iree-android`), landed the tool-calling epic substrate behind #35 +(chat-template auto-detection, registerable parser strategies, resolution diagnostics), hoisted +the packed-quant packing into `BlockQuantPacking` with Q5_0/Q5_1 kept packed, and brought +SmolLM2 into the tool-calling families. It builds on **0.38.0**, which completed **Moonshine v2 streaming ASR entirely in the SKaiNET NN DSL** (frontend, sliding-window encoder, adapter, KV-cache decoder — no vendor neural binaries, @@ -199,7 +210,7 @@ The recommended way to consume is via the BOM. It pins every published `skainet- ```kotlin dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.40.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) // Versions resolved from the BOM: implementation("sk.ainet.transformers:skainet-transformers-core") diff --git a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc index 65940c65..25da999d 100644 --- a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc +++ b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc @@ -24,7 +24,7 @@ a DSL decoder is future work. [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.40.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) implementation("sk.ainet.transformers:skainet-transformers-inference-moonshine") } ---- diff --git a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc index f2282eb9..aab15ddf 100644 --- a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc @@ -55,7 +55,7 @@ dependencies { // self-registers it on ART at process start — nothing to call. runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu") - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.40.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) implementation("sk.ainet.transformers:skainet-transformers-core") implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-inference-llama") diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc index 4728f657..8075dad0 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc @@ -25,7 +25,7 @@ In your `build.gradle.kts`: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.40.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") @@ -41,7 +41,7 @@ Or in Maven (Maven needs the `-jvm` classifier suffix on platform artifacts): sk.ainet.transformers skainet-transformers-bom - 0.40.2 + 0.53.0 pom import diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc index b73819ea..1e4bbc31 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc @@ -34,7 +34,7 @@ encoder output to the advertised dimensionality. The runtime applies it automati [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.40.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) implementation("sk.ainet.transformers:skainet-transformers-providers") } ---- diff --git a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc index 909f9060..9e2d0b7d 100644 --- a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc +++ b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc @@ -52,7 +52,7 @@ The pieces you need live in three modules: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.40.2")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") diff --git a/gradle.properties b/gradle.properties index 90f2e90a..13e487da 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.transformers -VERSION_NAME=0.40.2 +VERSION_NAME=0.53.0 POM_DESCRIPTION=SKaiNET-transformers diff --git a/llm-inference/gemma/README.md b/llm-inference/gemma/README.md index e8357e91..f3e03d21 100644 --- a/llm-inference/gemma/README.md +++ b/llm-inference/gemma/README.md @@ -4,7 +4,7 @@ Reusable **Gemma** model (incl. the FunctionGemma tool-calling fine-tune) author a portable graph producer with **no runtime/board/Torq code**. Pair it with the runtime module below to decode on-device. -- **Coordinate:** `sk.ainet.transformers:skainet-transformers-inference-gemma:0.40.2` +- **Coordinate:** `sk.ainet.transformers:skainet-transformers-inference-gemma:0.53.0` - **Targets:** `android`, `iosArm64`, `iosSimulatorArm64`, `macosArm64`, `linuxX64`, `linuxArm64` (broadly portable — mobile through server). - **Entry point:** `gemmaNetwork()` / `GemmaNetworkLoader` (loads a GGUF, builds the DSL graph, incl. the @@ -24,8 +24,8 @@ FunctionGemma has a one-liner facade in `…:skainet-transformers-runtime-kgemma ```kotlin dependencies { - implementation("sk.ainet.transformers:skainet-transformers-inference-gemma:0.40.2") - implementation("sk.ainet.transformers:skainet-transformers-runtime-gemma-iree:0.40.2") // on-device decode + implementation("sk.ainet.transformers:skainet-transformers-inference-gemma:0.53.0") + implementation("sk.ainet.transformers:skainet-transformers-runtime-gemma-iree:0.53.0") // on-device decode } ```