diff --git a/CHANGELOG.md b/CHANGELOG.md index cd9d7068..f0f9ebf9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -9,6 +9,53 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [Unreleased] +## [0.54.0] — 2026-09-07 + +Version lock-step with the engine continues: this release ships against **SKaiNET 0.54.0** +(SKEEP-005's `Schedule` API, the `CoroutineSchedule` pool-deadlock fix, `SafeTensorsParametersLoader` +`tensorFilter` parity, and the `ExperimentalMemoryApi` opt-in gate removed). It also closes out the +FunctionGemma/IREE-Android chunked-KV work (#410) and fixes two bugs found on real hardware: a +broken `runtime-kgemma` Maven Central POM (#408) and an Android crash in FunctionGemma tool-call +parsing (#407). + +### Added — chunked prefill and a stateful Android KV session for FunctionGemma (#406, #410) + +- **Position-selected graphs `gemma_at` / `gemma_prefill_at`** (#415): the LM head runs on one + one-hot-selected position instead of every position in `SEQ`, cutting the redecode step's wasted + work (#406). +- **Chunk prefill-with-past graph `gemma_prefill_with_past`** (#417): a fixed 64-token chunk against + the dynamic cache in one call, with per-head chunk masks (a broadcast over heads to a dynamic + shape isn't expressible in static StableHLO). +- **`IreeKvSession` / `IreeKvDecoder`** (#416, #418): the Android-native stateful KV session — + three IREE sessions, device-resident K/V, zero-copy 512-position tail views for the sliding + layers, native RoPE tables + chunk masks, embedding rows read from the archive, snapshot/restore + without copies. Measured on a MagentaTV One (Mali via Vulkan, bf16 archives): a once-per-process + 843-token catalog prefill, then **p50 5.87 s / max 6.25 s per utterance** (restore + one chunk + + 16 decode tokens), down from minutes on the stateless redecode contract. +- **`iree-android` failure reporting** (#404): native failures surface as reported errors instead + of a silent `null`; bare function names are qualified with `module.` automatically. + +### Fixed + +- **FunctionGemma export `ClassCastException` on `BufferHandle.Floats`** (#405, #420): + `FunctionGemmaExportHarness`, `SmolLm2ExportHarness`, and the bake-irpa tests hard-cast every + external constant to `BufferHandle.Owned`; since engine 0.53.0 (SKaiNET#1247) a constant can + arrive as the aliased `BufferHandle.Floats` instead. Both export harnesses now read every handle + through `DefaultBufferResolver`. +- **`runtime-kgemma`'s Maven Central POM depended on an unpublished coordinate** (#408): pulling + in `:llm-runtime:kgemma3n` (never published) leaked + `SKaiNET-transformers.llm-runtime:kgemma3n-jvm:unspecified` into the POM, breaking resolution for + any external consumer. Rather than just changing the dependency's scope, Gemma 3n itself stops + being published (SKaiNET-transformers#377: maturity gate 0/5, hand-rolled runtime that + force-dequantizes the whole model, postponed by decision) — source-only until #377's maturity + gate is met. `runtime-kgemma`'s CLI loses its `--arch gemma3n` variant accordingly; + `skainet-cli` (never published) is unaffected. +- **Android crash in the official FunctionGemma tool-call parser** (#407): + `FunctionGemmaOfficialToolCallParserStrategy.CALL_RE` had an unescaped closing `}` — tolerated by + `java.util.regex` on the JVM, rejected by Android's ICU-backed engine with a + `PatternSyntaxException` at class-init, so every tool-call parse crashed on ART before the first + match. One-character fix; no-op on the JVM. + ## [0.53.0] — 2026-09-02 Version lock-step with the engine is restored: this release ships against **SKaiNET 0.53.0** diff --git a/README.md b/README.md index 50101691..b0972855 100644 --- a/README.md +++ b/README.md @@ -108,32 +108,30 @@ Honest status — see the project-status note at the top of this README. ## Current release -The current release is **0.53.0** (against **SKaiNET 0.53.0**) — version lock-step with the engine -is restored, and the release ships everything accumulated since 0.40.2: BitNet b1.58, the -engine-loader migration (every GGUF weight loader a thin engine wrapper, MAPPED residency by -default), the Gemma 3n DSL path with its StableHLO/IREE export harness, and the Qwen / Apertus -decode and tool-calling fixes. - -**The Gemma 3n export actually emits.** Engine 0.53.0 closes SKaiNET#1247 — allocation-free void -tracing, aliased constant extraction, strict StableHLO conversion, and an array-free path for the -≥2 GiB tied embedding — and the harness streams weights through the `BufferResolver` (#396). The -full 30-layer E2B export produces a 15k-line module with zero failure comments and a 4.6 GB -safetensors in under a minute, where it previously OOMed a 46 GB heap. - -**SafeTensors loading rides the engine.** The per-family hand-rolled SafeTensors materialization -(Gemma, the shared decoder loader, Apertus, Gemma 3n) collapses onto the engine's sharded / -single-file `ParametersLoader`s (#398, #400, #401): each family keeps only its slot table and -allowlist, every dtype decision is the engine's, and `Require(BF16)`/`Require(FP16)` keep-native is -accepted on the SafeTensors lane. - -**skainet-decode on Android.** `llm-apps:skainet-decode-core` (the shared `DecodeSession`) and -`llm-apps:skainet-decode-android`, the repository's first Android application: load a pushed GGUF -memory-mapped, refuse before allocating when it doesn't fit, decode on one thread, and report -`GenerationMetrics` with the page-fault and RSS rows only a device can show (#395). - -**Tool calling re-verified against the new engine.** `Gemma4E2BToolCallSmokeTest` is re-enabled — -the real E2B checkpoint now emits parseable `<|tool_call>` markup (#399) — and FunctionGemma and -Qwen3 tool calling were re-run green in the same pass. +The current release is **0.54.0** (against **SKaiNET 0.54.0**) — version lock-step with the engine +continues, plus two bugs fixed on real hardware and the FunctionGemma/IREE-Android chunked-KV work +closed out. + +**A stateful Android KV session for FunctionGemma.** `IreeKvSession` / `IreeKvDecoder` +(`llm-runtime/iree-android`) prefill the tool catalog once per process, snapshot the KV state, and +per-turn prefill only the new chunk — device-resident K/V, zero-copy tail views for the sliding +layers, embedding rows read straight from the archive. Measured on a MagentaTV One (Mali via +Vulkan): **p50 5.87 s / max 6.25 s per utterance**, down from minutes on the stateless redecode +contract. Position-selected graphs (`gemma_at`/`gemma_prefill_at`) and a chunk prefill-with-past +graph (`gemma_prefill_with_past`, #415, #417) get there without every step re-running the LM head +over the whole sequence. + +**Two bugs found on real hardware, fixed.** `runtime-kgemma`'s published POM depended on an +unpublished `:llm-runtime:kgemma3n` coordinate and failed to resolve for any external consumer — +fixed by unpublishing Gemma 3n itself rather than papering over the dependency (it's maturity-gate +0/5 and postponed per its own tracking issue). The official FunctionGemma tool-call parser crashed +with `PatternSyntaxException` on every Android device (ICU rejects an unescaped `}` that the JVM's +regex engine tolerates) — a one-character fix. + +It builds on **0.53.0**, which restored version lock-step with the engine (SKaiNET 0.53.0's +billion-parameter export fixes and sharded SafeTensors loader), collapsed the per-family +hand-rolled SafeTensors materialization onto the engine's loaders, shipped `skainet-decode` as the +repository's first Android application, and re-verified tool calling against the new engine. It builds on **0.40.2**, which rounded out the **compiled on-device path** (standalone DSL → StableHLO → IREE export modules for FunctionGemma and SmolLM2, the generic Android JNI @@ -210,7 +208,7 @@ The recommended way to consume is via the BOM. It pins every published `skainet- ```kotlin dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.54.0")) // Versions resolved from the BOM: implementation("sk.ainet.transformers:skainet-transformers-core") diff --git a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc index 25da999d..e9669b71 100644 --- a/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc +++ b/docs/modules/ROOT/pages/reference/moonshine-encoder.adoc @@ -24,7 +24,7 @@ a DSL decoder is future work. [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.54.0")) implementation("sk.ainet.transformers:skainet-transformers-inference-moonshine") } ---- diff --git a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc index aab15ddf..9ef9dede 100644 --- a/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-getting-started.adoc @@ -55,7 +55,7 @@ dependencies { // self-registers it on ART at process start — nothing to call. runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu") - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.54.0")) implementation("sk.ainet.transformers:skainet-transformers-core") implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-inference-llama") diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc index 8075dad0..c0d76625 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-java.adoc @@ -25,7 +25,7 @@ In your `build.gradle.kts`: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.54.0")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") @@ -41,7 +41,7 @@ Or in Maven (Maven needs the `-jvm` classifier suffix on platform artifacts): sk.ainet.transformers skainet-transformers-bom - 0.53.0 + 0.54.0 pom import diff --git a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc index 1e4bbc31..afe6d6d1 100644 --- a/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc +++ b/docs/modules/ROOT/pages/tutorials/getting-started-leaf.adoc @@ -34,7 +34,7 @@ encoder output to the advertised dimensionality. The runtime applies it automati [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.54.0")) implementation("sk.ainet.transformers:skainet-transformers-providers") } ---- diff --git a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc index 9e2d0b7d..e0e69dd4 100644 --- a/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc +++ b/docs/modules/ROOT/pages/tutorials/llama3-tool-calling.adoc @@ -52,7 +52,7 @@ The pieces you need live in three modules: [source,kotlin] ---- dependencies { - implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.53.0")) + implementation(platform("sk.ainet.transformers:skainet-transformers-bom:0.54.0")) implementation("sk.ainet.transformers:skainet-transformers-runtime-kllama") implementation("sk.ainet.transformers:skainet-transformers-agent") diff --git a/gradle.properties b/gradle.properties index 13e487da..d50b8380 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.transformers -VERSION_NAME=0.53.0 +VERSION_NAME=0.54.0 POM_DESCRIPTION=SKaiNET-transformers diff --git a/llm-inference/gemma/README.md b/llm-inference/gemma/README.md index f3e03d21..0f14495c 100644 --- a/llm-inference/gemma/README.md +++ b/llm-inference/gemma/README.md @@ -4,7 +4,7 @@ Reusable **Gemma** model (incl. the FunctionGemma tool-calling fine-tune) author a portable graph producer with **no runtime/board/Torq code**. Pair it with the runtime module below to decode on-device. -- **Coordinate:** `sk.ainet.transformers:skainet-transformers-inference-gemma:0.53.0` +- **Coordinate:** `sk.ainet.transformers:skainet-transformers-inference-gemma:0.54.0` - **Targets:** `android`, `iosArm64`, `iosSimulatorArm64`, `macosArm64`, `linuxX64`, `linuxArm64` (broadly portable — mobile through server). - **Entry point:** `gemmaNetwork()` / `GemmaNetworkLoader` (loads a GGUF, builds the DSL graph, incl. the @@ -24,8 +24,8 @@ FunctionGemma has a one-liner facade in `…:skainet-transformers-runtime-kgemma ```kotlin dependencies { - implementation("sk.ainet.transformers:skainet-transformers-inference-gemma:0.53.0") - implementation("sk.ainet.transformers:skainet-transformers-runtime-gemma-iree:0.53.0") // on-device decode + implementation("sk.ainet.transformers:skainet-transformers-inference-gemma:0.54.0") + implementation("sk.ainet.transformers:skainet-transformers-runtime-gemma-iree:0.54.0") // on-device decode } ```