diff --git a/CHANGELOG.md b/CHANGELOG.md index 615f5686..eb049534 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,16 @@ ## [Unreleased] +## [0.52.1] - 2026-09-01 + +Headline: **the self-healing dispatch now heals the ternary path too.** 0.52.0 made +`KernelDispatch` populate itself so no consumer could silently lose its kernels — but the +ternary packs were not in the discovery set, so a BitNet consumer still needed two explicit +install calls or quietly ran ~120× slower. That gap is closed; a consumer on this release loads +`BITNET_B1_58`/`BITNET_PLANES` weights and dispatches to the vendored NeoGPU kernels with zero +bootstrap code. Validated downstream: SKaiNET-transformers' CLI dropped its last explicit kernel +installs against this release and decodes BitNet-2B4T at full speed on discovery alone. + ### Fixed - **Ternary kernel packs join the self-healing dispatch SPI** @@ -16,6 +26,100 @@ Q-series formats. Kotlin/Native still installs explicitly (no `ServiceLoader` there); the ternary tutorial's install table now says which targets are automatic. +## [0.52.0] - 2026-08-31 + +Headline: **the engine stops silently running on the scalar floor.** A downstream Gemma 4 port +was generating garbage at roughly 0.04 tok/s, and the investigation +([#1220](https://github.com/SKaiNET-developers/SKaiNET/issues/1220)) found the cause split across +both repositories — but the engine's share of it was one theme repeated: a fast path that exists, +is compiled in, and never gets used, with nothing saying so. `KernelDispatch` was never populated +in production at all, so every matmul fell back to the decoding reference kernel; dense FP32 +weights in mapped or off-heap storage missed the kernel that serves them and dequantized instead; +and the fallback itself was routed to a no-op trace sink, which is why a ~1000x degradation could +sit in a release undetected. Those are closed, and the dispatcher now installs itself on first use +rather than trusting every entry point to remember. Alongside that, Android's native targets now +run the whole dependency chain, not just its first two modules. + +### Added + +- **`ViewKernelPack` SPI and self-healing dispatch** + ([#1220](https://github.com/SKaiNET-developers/SKaiNET/issues/1220), + [#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): `KernelDispatch` populates + itself on first use via `ensureInstalled()`, backed by a new `ViewKernelPack` service interface + with `installPlatformKernelPacks()` actuals per platform — ServiceLoader-based on JVM and + Android, explicit on Kotlin/Native, which has no ServiceLoader. Applications no longer have to + call an install routine at startup and no longer silently lose every kernel when they forget. + The two backends ship discovery metadata: `FfmRowMajorKernelPackFactory` + (`skainet-backend-native-cpu`) and `JniMappedKernelPackFactory` (`skainet-backend-jni-cpu`). +- **`androidNativeArm32`/`androidNativeArm64` across the downstream chain** + ([#1239](https://github.com/SKaiNET-developers/SKaiNET/pull/1239)): `skainet-io-gguf`, + `skainet-lang-dag`, `skainet-compile-dag`, `skainet-compile-opt`, `skainet-backend-api`, + `skainet-backend-cpu`, plus `skainet-lang-models` and `skainet-compile-json` to close the target + set over test compilations. Only `skainet-io-core` and friends had these targets before, so a + consumer building for an Android device could not resolve the rest of what it needed. + `skainet-io-core`'s 64-bit split source set has no counterpart here: none of these modules has + posix-typed code, so arm32's `Int`-width `ssize_t`/`size_t` does not reach them. +- **Mapped-serving encodings derived from kernel registrations** + ([#1193](https://github.com/SKaiNET-developers/SKaiNET/issues/1193), + [#1215](https://github.com/SKaiNET-developers/SKaiNET/pull/1215)): + `KernelDispatch.mappedServableEncodings()` reports which encodings a `MappedCapableKernel` + actually serves right now, replacing a hand-kept list that could drift from the registry it + described. +- **`gemma4` in the model registries** + ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): `TokenizerFactory` accepts + the architecture and `ModelArchitecture.ggufIdMap` maps `"gemma4"` to `GEMMA`, so a Gemma 4 GGUF + loads through the engine's own routes instead of throwing. +- **Dense FP32 GEMV path in the Panama kernel** + ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): `PanamaVectorMatmulKernel` + gains `gemvRows()` for the m ≤ 8 shapes a decode step actually issues — 16.7x at m=1 over the + general blocked path, which was written for prefill-sized work. + +### Fixed + +- **Dense FP32 weights in mapped or off-heap storage fell back to dequantization** + ([#1218](https://github.com/SKaiNET-developers/SKaiNET/pull/1218)): the kernel that serves them + only recognised `Heap`, so a memory-mapped model — the whole point of mapped staging — took the + slow path. Now served from any storage kind. +- **The reference-kernel fallback was invisible** + ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): `DefaultCpuOps` hardcoded + `NoopTraceSink` at both dispatch sites, so falling back to the decoding reference kernel emitted + nothing. `KernelDispatch` gains a `defaultSink` and warns once, loudly, the first time it + happens. +- **`SpecialTokenSplitter` lost word boundaries when decoding token by token** + ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): it did not override + `decodeToken`, so streaming consumers of any SentencePiece GGUF with special tokens saw spaces + disappear from the output. Also fixes `token_type` parsing, which discarded `UInt`-typed GGUF + metadata and could silently drop a model's special tokens. +- **`ar`/`ranlib` selection for the aarch64 cross build on macOS hosts** + ([#1209](https://github.com/SKaiNET-developers/SKaiNET/pull/1209)): the build picked the host's + Mach-O tools for an ELF target, producing archives the linker rejected. + +### Performance + +- **Small-shape FP32 matmul** ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): + a direct-loop path under `SMALL_FP32_MATMUL_WORK` skips blocking overhead that costs more than it + saves at decode sizes, and `transposedDenseWeight()` caches the transpose instead of rebuilding + it per call. Measured end to end on a downstream Gemma 4 port: ~2.3x on both decode and prefill. + +### Docs + +- Kernel-selection explanation page, covering the two registries and how a weight reaches a kernel + ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)). +- Architecture reference gains its missing building blocks — kernel dispatch, ternary, AOT + conversion ([#1216](https://github.com/SKaiNET-developers/SKaiNET/pull/1216)). +- DARC/SKEEP onboarding, issue taxonomy and an `F1Score` worked example for contributors + ([#1238](https://github.com/SKaiNET-developers/SKaiNET/pull/1238)). +- `GITFLOW.adoc` reconciled with the `main` branch reset, documenting the release sequence actually + used from 0.51.0 onward ([#1213](https://github.com/SKaiNET-developers/SKaiNET/pull/1213)). +- Why the `GROUP_128`/`GROUP_64` native decode kernel was closed + ([#1205](https://github.com/SKaiNET-developers/SKaiNET/issues/1205), + [#1214](https://github.com/SKaiNET-developers/SKaiNET/pull/1214)). + +### Dependencies + +- `github/codeql-action/upload-sarif` 4.37.8 → 4.37.9 + ([#1236](https://github.com/SKaiNET-developers/SKaiNET/pull/1236)). + ## [0.51.0] - 2026-08-29 Headline: **ternary/BitNet weights join the memory-mapped weight story 0.50.0 started for every diff --git a/README.md b/README.md index 2540c6f9..3b2912a5 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.51.0")) + implementation(platform("sk.ainet:skainet-bom:0.52.1")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -308,20 +308,34 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.51.0 - -Ternary/BitNet weights join the memory-mapped weight story 0.50.0 started for every other quant -format: - -- **Off-heap ternary storage** — `BitNetB158TensorData` no longer risks the Android ART heap-cap - OOM; `Storage.copyInto`/`copyFrom` give every storage kind one shared bulk-copy primitive. -- **True zero-copy mmap** for `SEQUENTIAL`-layout (NeoGPU-converted) GGUFs, and a zero-copy - native gemv path for off-heap ternary weights on the JVM/FFM kernel. -- **`I2sAotConverter`** (GGUF → GGUF): convert I2_S tensors ahead of time so a controlled model - pipeline never pays a runtime repack. The IREE-facing counterpart lives in - [SKaiNET-IREE-tools](https://github.com/SKaiNET-developers/SKaiNET-IREE-tools). -- **Correctness fix** — scoped dense-FP32 activations no longer silently fall out of the - quantized matmul chooser (was producing wrong logits under `ScopedExecutionContext`). +## What's New in 0.52.1 + +The self-healing dispatch now heals the ternary path too: + +- **Ternary kernels join the discovery set** — the BitNet b1.58 packs (`BITNET_B1_58` LUT gemv, + fused `BITNET_PLANES` lm_head) are now ServiceLoader-discovered like the Q-series, so a + consumer loads ternary weights and gets the vendored NeoGPU kernels with zero bootstrap code — + previously two explicit install calls stood between a BitNet model and its fast path, and + forgetting them was a silent ~120× slowdown. Validated downstream: SKaiNET-transformers decodes + BitNet-2B4T at full speed on discovery alone. + +Plus the 0.52.0 groundwork — the engine stops silently running on the scalar floor: + +- **Self-healing kernel dispatch** — `KernelDispatch` installs itself on first use through the new + `ViewKernelPack` SPI, so an application that never called an install routine no longer loses + every kernel and falls back to the decoding reference path. When a fallback does happen, it now + says so once, loudly, instead of vanishing into a no-op trace sink. +- **Dense FP32 from any storage kind** — mapped and off-heap weights were dequantizing because the + kernel serving them only recognised `Heap`, which defeated the point of memory-mapped staging. +- **Android native across the chain** — `androidNativeArm32`/`Arm64` now build and publish from the + whole downstream dependency graph, not just `skainet-io-core`, so on-device consumers can + actually resolve what they need. +- **Faster decode-shaped matmul** — a dense FP32 GEMV path for the m ≤ 8 shapes decode issues + (16.7x at m=1), a direct-loop path for small work, and a cached weight transpose. ~2.3x decode + and prefill measured end to end on a downstream Gemma 4 port. +- **Gemma 4 loads through the engine's own routes** — `gemma4` is registered in `TokenizerFactory` + and `ModelArchitecture`, and `SpecialTokenSplitter` no longer drops word boundaries when + decoding token by token. See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release. @@ -347,6 +361,14 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.52.x) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the Gemma 4 engine-gap + arc: self-healing kernel dispatch and the `ViewKernelPack` SPI, dense FP32 for mapped/off-heap + storage, the decode-shaped FP32 kernel work, and Android-native targets across the downstream + chain; in 0.52.1, the ternary packs joining the self-healing SPI — closing the BitNet half of + the same "fast path exists but never gets used" theme + ### Contributors (0.51.0) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — off-heap ternary diff --git a/docs/antora.yml b/docs/antora.yml index b4948094..9e852ff2 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.51.0 + skainet_version: 0.52.1 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/reference/architecture.adoc b/docs/modules/ROOT/pages/reference/architecture.adoc index a99c2af1..a10c3afb 100644 --- a/docs/modules/ROOT/pages/reference/architecture.adoc +++ b/docs/modules/ROOT/pages/reference/architecture.adoc @@ -24,7 +24,7 @@ real models use, not the long tail in PyTorch / NumPy. * *Run untrusted user code.* Kernels are trusted code; security is about not corrupting memory, not about sandboxing. -image::SKaiNET-compiler.svg[Architecture diagram of the SKaiNET compiler pipeline] +image::SKaiNET-compiler.png[Architecture diagram of the SKaiNET compiler pipeline] == 2. Constraints diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index 37f85d44..dd03af5a 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.51.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.52.1`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). diff --git a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc index 0b5d0fab..cd62f273 100644 --- a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc +++ b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc @@ -1,6 +1,6 @@ = AI-NET Operators Reference -Generated from version `0.51.0` on 2026-08-29 +Generated from version `0.52.1` on 2026-09-01 == Operators by Modality diff --git a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc index 2dba6c47..671d4d24 100644 --- a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc @@ -1,7 +1,7 @@ = Operator Coverage Matrix :description: Cross-backend status for every operator function in SKaiNET. -Generated from `operators.json` version `0.51.0` on 2026-08-29. +Generated from `operators.json` version `0.52.1` on 2026-09-01. Rows are `Operator.function` pairs. The `Validated` column shows whether the function's documentation has been DARC-validated by a reviewer (see xref:contributing/darc-workflow.adoc[DARC workflow]). Remaining columns are backends that appear in any function's `statusByBackend` map — a missing entry means the backend makes no claim about the function (treat it as "unknown", not "not supported"). diff --git a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc index 2b49f6c0..12a58fd2 100644 --- a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc @@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules: [source,kotlin] ---- dependencies { - implementation("sk.ainet.core:skainet-lang-core:0.51.0") // tensors, DSL, training - implementation("sk.ainet.core:skainet-backend-cpu:0.51.0") // CPU ops - implementation("sk.ainet.core:skainet-compile-dag:0.51.0") // autograd (training context) - implementation("sk.ainet.core:skainet-data-api:0.51.0") // Dataset / DataBatch - implementation("sk.ainet.core:skainet-data-simple:0.51.0") // embedded Iris - runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.51.0") // NEON kernels (see below) + implementation("sk.ainet.core:skainet-lang-core:0.52.1") // tensors, DSL, training + implementation("sk.ainet.core:skainet-backend-cpu:0.52.1") // CPU ops + implementation("sk.ainet.core:skainet-compile-dag:0.52.1") // autograd (training context) + implementation("sk.ainet.core:skainet-data-api:0.52.1") // Dataset / DataBatch + implementation("sk.ainet.core:skainet-data-simple:0.52.1") // embedded Iris + runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.52.1") // NEON kernels (see below) } ---- diff --git a/gradle.properties b/gradle.properties index 9d44d63c..566d6bfc 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.51.0 +VERSION_NAME=0.52.1 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/