release: 0.52.1 — the self-healing dispatch heals the ternary path (folds in the 0.52.0 release branch)

CHANGELOG.md

@@ -2,6 +2,16 @@
 
 ## [Unreleased]
 
+## [0.52.1] - 2026-09-01
+
+Headline: **the self-healing dispatch now heals the ternary path too.** 0.52.0 made
+`KernelDispatch` populate itself so no consumer could silently lose its kernels — but the
+ternary packs were not in the discovery set, so a BitNet consumer still needed two explicit
+install calls or quietly ran ~120× slower. That gap is closed; a consumer on this release loads
+`BITNET_B1_58`/`BITNET_PLANES` weights and dispatches to the vendored NeoGPU kernels with zero
+bootstrap code. Validated downstream: SKaiNET-transformers' CLI dropped its last explicit kernel
+installs against this release and decodes BitNet-2B4T at full speed on discovery alone.
+
 ### Fixed
 
 - **Ternary kernel packs join the self-healing dispatch SPI**
@@ -16,6 +26,100 @@
   Q-series formats. Kotlin/Native still installs explicitly (no `ServiceLoader` there); the
   ternary tutorial's install table now says which targets are automatic.
 
+## [0.52.0] - 2026-08-31
+
+Headline: **the engine stops silently running on the scalar floor.** A downstream Gemma 4 port
+was generating garbage at roughly 0.04 tok/s, and the investigation
+([#1220](https://github.com/SKaiNET-developers/SKaiNET/issues/1220)) found the cause split across
+both repositories — but the engine's share of it was one theme repeated: a fast path that exists,
+is compiled in, and never gets used, with nothing saying so. `KernelDispatch` was never populated
+in production at all, so every matmul fell back to the decoding reference kernel; dense FP32
+weights in mapped or off-heap storage missed the kernel that serves them and dequantized instead;
+and the fallback itself was routed to a no-op trace sink, which is why a ~1000x degradation could
+sit in a release undetected. Those are closed, and the dispatcher now installs itself on first use
+rather than trusting every entry point to remember. Alongside that, Android's native targets now
+run the whole dependency chain, not just its first two modules.
+
+### Added
+
+- **`ViewKernelPack` SPI and self-healing dispatch**
+  ([#1220](https://github.com/SKaiNET-developers/SKaiNET/issues/1220),
+  [#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): `KernelDispatch` populates
+  itself on first use via `ensureInstalled()`, backed by a new `ViewKernelPack` service interface
+  with `installPlatformKernelPacks()` actuals per platform — ServiceLoader-based on JVM and
+  Android, explicit on Kotlin/Native, which has no ServiceLoader. Applications no longer have to
+  call an install routine at startup and no longer silently lose every kernel when they forget.
+  The two backends ship discovery metadata: `FfmRowMajorKernelPackFactory`
+  (`skainet-backend-native-cpu`) and `JniMappedKernelPackFactory` (`skainet-backend-jni-cpu`).
+- **`androidNativeArm32`/`androidNativeArm64` across the downstream chain**
+  ([#1239](https://github.com/SKaiNET-developers/SKaiNET/pull/1239)): `skainet-io-gguf`,
+  `skainet-lang-dag`, `skainet-compile-dag`, `skainet-compile-opt`, `skainet-backend-api`,
+  `skainet-backend-cpu`, plus `skainet-lang-models` and `skainet-compile-json` to close the target
+  set over test compilations. Only `skainet-io-core` and friends had these targets before, so a
+  consumer building for an Android device could not resolve the rest of what it needed.
+  `skainet-io-core`'s 64-bit split source set has no counterpart here: none of these modules has
+  posix-typed code, so arm32's `Int`-width `ssize_t`/`size_t` does not reach them.
+- **Mapped-serving encodings derived from kernel registrations**
+  ([#1193](https://github.com/SKaiNET-developers/SKaiNET/issues/1193),
+  [#1215](https://github.com/SKaiNET-developers/SKaiNET/pull/1215)):
+  `KernelDispatch.mappedServableEncodings()` reports which encodings a `MappedCapableKernel`
+  actually serves right now, replacing a hand-kept list that could drift from the registry it
+  described.
+- **`gemma4` in the model registries**
+  ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): `TokenizerFactory` accepts
+  the architecture and `ModelArchitecture.ggufIdMap` maps `"gemma4"` to `GEMMA`, so a Gemma 4 GGUF
+  loads through the engine's own routes instead of throwing.
+- **Dense FP32 GEMV path in the Panama kernel**
+  ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): `PanamaVectorMatmulKernel`
+  gains `gemvRows()` for the m ≤ 8 shapes a decode step actually issues — 16.7x at m=1 over the
+  general blocked path, which was written for prefill-sized work.
+
+### Fixed
+
+- **Dense FP32 weights in mapped or off-heap storage fell back to dequantization**
+  ([#1218](https://github.com/SKaiNET-developers/SKaiNET/pull/1218)): the kernel that serves them
+  only recognised `Heap`, so a memory-mapped model — the whole point of mapped staging — took the
+  slow path. Now served from any storage kind.
+- **The reference-kernel fallback was invisible**
+  ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): `DefaultCpuOps` hardcoded
+  `NoopTraceSink` at both dispatch sites, so falling back to the decoding reference kernel emitted
+  nothing. `KernelDispatch` gains a `defaultSink` and warns once, loudly, the first time it
+  happens.
+- **`SpecialTokenSplitter` lost word boundaries when decoding token by token**
+  ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)): it did not override
+  `decodeToken`, so streaming consumers of any SentencePiece GGUF with special tokens saw spaces
+  disappear from the output. Also fixes `token_type` parsing, which discarded `UInt`-typed GGUF
+  metadata and could silently drop a model's special tokens.
+- **`ar`/`ranlib` selection for the aarch64 cross build on macOS hosts**
+  ([#1209](https://github.com/SKaiNET-developers/SKaiNET/pull/1209)): the build picked the host's
+  Mach-O tools for an ELF target, producing archives the linker rejected.
+
+### Performance
+
+- **Small-shape FP32 matmul** ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)):
+  a direct-loop path under `SMALL_FP32_MATMUL_WORK` skips blocking overhead that costs more than it
+  saves at decode sizes, and `transposedDenseWeight()` caches the transpose instead of rebuilding
+  it per call. Measured end to end on a downstream Gemma 4 port: ~2.3x on both decode and prefill.
+
+### Docs
+
+- Kernel-selection explanation page, covering the two registries and how a weight reaches a kernel
+  ([#1221](https://github.com/SKaiNET-developers/SKaiNET/pull/1221)).
+- Architecture reference gains its missing building blocks — kernel dispatch, ternary, AOT
+  conversion ([#1216](https://github.com/SKaiNET-developers/SKaiNET/pull/1216)).
+- DARC/SKEEP onboarding, issue taxonomy and an `F1Score` worked example for contributors
+  ([#1238](https://github.com/SKaiNET-developers/SKaiNET/pull/1238)).
+- `GITFLOW.adoc` reconciled with the `main` branch reset, documenting the release sequence actually
+  used from 0.51.0 onward ([#1213](https://github.com/SKaiNET-developers/SKaiNET/pull/1213)).
+- Why the `GROUP_128`/`GROUP_64` native decode kernel was closed
+  ([#1205](https://github.com/SKaiNET-developers/SKaiNET/issues/1205),
+  [#1214](https://github.com/SKaiNET-developers/SKaiNET/pull/1214)).
+
+### Dependencies
+
+- `github/codeql-action/upload-sarif` 4.37.8 → 4.37.9
+  ([#1236](https://github.com/SKaiNET-developers/SKaiNET/pull/1236)).
+
 ## [0.51.0] - 2026-08-29
 
 Headline: **ternary/BitNet weights join the memory-mapped weight story 0.50.0 started for every

README.md

@@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL):
 ```kotlin
 dependencies {
     // Recommended: import the umbrella BOM and drop versions on the engine modules.
-    implementation(platform("sk.ainet:skainet-bom:0.51.0"))
+    implementation(platform("sk.ainet:skainet-bom:0.52.1"))
 
     implementation("sk.ainet.core:skainet-lang-core")
     implementation("sk.ainet.core:skainet-backend-cpu")
@@ -308,20 +308,34 @@ val withoutLabel = dataPipeline<RawDataset>()
 
 ---
 
-## What's New in 0.51.0
-
-Ternary/BitNet weights join the memory-mapped weight story 0.50.0 started for every other quant
-format:
-
-- **Off-heap ternary storage** — `BitNetB158TensorData` no longer risks the Android ART heap-cap
-  OOM; `Storage.copyInto`/`copyFrom` give every storage kind one shared bulk-copy primitive.
-- **True zero-copy mmap** for `SEQUENTIAL`-layout (NeoGPU-converted) GGUFs, and a zero-copy
-  native gemv path for off-heap ternary weights on the JVM/FFM kernel.
-- **`I2sAotConverter`** (GGUF → GGUF): convert I2_S tensors ahead of time so a controlled model
-  pipeline never pays a runtime repack. The IREE-facing counterpart lives in
-  [SKaiNET-IREE-tools](https://github.com/SKaiNET-developers/SKaiNET-IREE-tools).
-- **Correctness fix** — scoped dense-FP32 activations no longer silently fall out of the
-  quantized matmul chooser (was producing wrong logits under `ScopedExecutionContext`).
+## What's New in 0.52.1
+
+The self-healing dispatch now heals the ternary path too:
+
+- **Ternary kernels join the discovery set** — the BitNet b1.58 packs (`BITNET_B1_58` LUT gemv,
+  fused `BITNET_PLANES` lm_head) are now ServiceLoader-discovered like the Q-series, so a
+  consumer loads ternary weights and gets the vendored NeoGPU kernels with zero bootstrap code —
+  previously two explicit install calls stood between a BitNet model and its fast path, and
+  forgetting them was a silent ~120× slowdown. Validated downstream: SKaiNET-transformers decodes
+  BitNet-2B4T at full speed on discovery alone.
+
+Plus the 0.52.0 groundwork — the engine stops silently running on the scalar floor:
+
+- **Self-healing kernel dispatch** — `KernelDispatch` installs itself on first use through the new
+  `ViewKernelPack` SPI, so an application that never called an install routine no longer loses
+  every kernel and falls back to the decoding reference path. When a fallback does happen, it now
+  says so once, loudly, instead of vanishing into a no-op trace sink.
+- **Dense FP32 from any storage kind** — mapped and off-heap weights were dequantizing because the
+  kernel serving them only recognised `Heap`, which defeated the point of memory-mapped staging.
+- **Android native across the chain** — `androidNativeArm32`/`Arm64` now build and publish from the
+  whole downstream dependency graph, not just `skainet-io-core`, so on-device consumers can
+  actually resolve what they need.
+- **Faster decode-shaped matmul** — a dense FP32 GEMV path for the m ≤ 8 shapes decode issues
+  (16.7x at m=1), a direct-loop path for small work, and a cached weight transpose. ~2.3x decode
+  and prefill measured end to end on a downstream Gemma 4 port.
+- **Gemma 4 loads through the engine's own routes** — `gemma4` is registered in `TokenizerFactory`
+  and `ModelArchitecture`, and `SpecialTokenSplitter` no longer drops word boundaries when
+  decoding token by token.
 
 See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release.
 
@@ -347,6 +361,14 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix:
 
 Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET).
 
+### Contributors (0.52.x)
+
+- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the Gemma 4 engine-gap
+  arc: self-healing kernel dispatch and the `ViewKernelPack` SPI, dense FP32 for mapped/off-heap
+  storage, the decode-shaped FP32 kernel work, and Android-native targets across the downstream
+  chain; in 0.52.1, the ternary packs joining the self-healing SPI — closing the BitNet half of
+  the same "fast path exists but never gets used" theme
+
 ### Contributors (0.51.0)
 
 - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — off-heap ternary

docs/antora.yml

@@ -15,7 +15,7 @@ asciidoc:
     framework_name: SKaiNET
     # Current SKaiNET release — bump once per release; referenced as
     # {skainet_version} in dependency snippets (blocks need subs="attributes+").
-    skainet_version: 0.51.0
+    skainet_version: 0.52.1
     ksp_version: 2.2.21-2.0.5
     dokka_version: 2.1.0
     asciidoctorj_version: 3.0.0

docs/modules/ROOT/pages/reference/architecture.adoc

@@ -24,7 +24,7 @@ real models use, not the long tail in PyTorch / NumPy.
 * *Run untrusted user code.* Kernels are trusted code; security is
 about not corrupting memory, not about sandboxing.
 
-image::SKaiNET-compiler.svg[Architecture diagram of the SKaiNET compiler pipeline]
+image::SKaiNET-compiler.png[Architecture diagram of the SKaiNET compiler pipeline]
 
 == 2. Constraints
 

docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc

@@ -1,7 +1,7 @@
 = Kernel × platform support matrix
 :description: Which compute-kernel provider serves each weight format on each KMP target.
 
-Generated from `kernel-support.json` (version `0.51.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.
+Generated from `kernel-support.json` (version `0.52.1`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.
 
 Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only).
 

docs/modules/ROOT/pages/reference/operators/generated/index.adoc

@@ -1,6 +1,6 @@
 = AI-NET Operators Reference
 
-Generated from version `0.51.0` on 2026-08-29
+Generated from version `0.52.1` on 2026-09-01
 
 == Operators by Modality
 

docs/modules/ROOT/pages/reference/ops-status-matrix.adoc

@@ -1,7 +1,7 @@
 = Operator Coverage Matrix
 :description: Cross-backend status for every operator function in SKaiNET.
 
-Generated from `operators.json` version `0.51.0` on 2026-08-29.
+Generated from `operators.json` version `0.52.1` on 2026-09-01.
 
 Rows are `Operator.function` pairs. The `Validated` column shows whether the function's documentation has been DARC-validated by a reviewer (see xref:contributing/darc-workflow.adoc[DARC workflow]). Remaining columns are backends that appear in any function's `statusByBackend` map — a missing entry means the backend makes no claim about the function (treat it as "unknown", not "not supported").
 

docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc

@@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules:
 [source,kotlin]
 ----
 dependencies {
-    implementation("sk.ainet.core:skainet-lang-core:0.51.0")     // tensors, DSL, training
-    implementation("sk.ainet.core:skainet-backend-cpu:0.51.0")   // CPU ops
-    implementation("sk.ainet.core:skainet-compile-dag:0.51.0")   // autograd (training context)
-    implementation("sk.ainet.core:skainet-data-api:0.51.0")      // Dataset / DataBatch
-    implementation("sk.ainet.core:skainet-data-simple:0.51.0")   // embedded Iris
-    runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.51.0")  // NEON kernels (see below)
+    implementation("sk.ainet.core:skainet-lang-core:0.52.1")     // tensors, DSL, training
+    implementation("sk.ainet.core:skainet-backend-cpu:0.52.1")   // CPU ops
+    implementation("sk.ainet.core:skainet-compile-dag:0.52.1")   // autograd (training context)
+    implementation("sk.ainet.core:skainet-data-api:0.52.1")      // Dataset / DataBatch
+    implementation("sk.ainet.core:skainet-data-simple:0.52.1")   // embedded Iris
+    runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.52.1")  // NEON kernels (see below)
 }
 ----
 

gradle.properties

@@ -1,5 +1,5 @@
 GROUP=sk.ainet.core
-VERSION_NAME=0.51.0
+VERSION_NAME=0.52.1
 POM_DESCRIPTION=SKaiNET
 
 POM_URL=https://github.com/SKaiNET-developers/skainet/