Release 0.51.0

CHANGELOG.md

@@ -2,6 +2,78 @@
 
 ## [Unreleased]
 
+## [0.51.0] - 2026-08-29
+
+Headline: **ternary/BitNet weights join the memory-mapped weight story 0.50.0 started for every
+other quant format.** 0.50.0 shipped mapped staging for every GGML block format but left ternary
+(`BITNET_B1_58`) heap-staging, flagged then as needing the work tracked in
+[#1198](https://github.com/SKaiNET-developers/SKaiNET/issues/1198). That work is done: off-heap
+storage removes the Android ART heap-cap OOM risk a repacked ternary weight used to carry, and a
+`SEQUENTIAL`-layout (NeoGPU-converted) GGUF now gets a true zero-copy mmap load with no repack at
+all. A new AOT converter lets a build that owns its model pipeline pay that repack cost once,
+offline, instead of on every load — its IREE-facing counterpart lives in a new home,
+[SKaiNET-IREE-tools](https://github.com/SKaiNET-developers/SKaiNET-IREE-tools), on the
+architectural grounds that compiled-target-specific conversion belongs beside the compiler
+toolchain it targets, not inside core (see [#1207](https://github.com/SKaiNET-developers/SKaiNET/issues/1207)).
+
+### Added
+
+- **Off-heap storage for packed ternary/quantized weights**
+  ([#1202](https://github.com/SKaiNET-developers/SKaiNET/issues/1202),
+  [#1206](https://github.com/SKaiNET-developers/SKaiNET/pull/1206)): `Storage` gains
+  `copyInto`/`copyFrom` bulk byte primitives implemented for every concrete storage kind (`Heap`,
+  `SegmentStorage`, `MappedFileStorage`, `DirectBufferStorage`, `MappedBufferStorage`,
+  `NativeMallocStorage`, `NativeMappedStorage`). `PackedBlockStorage` gains a `packedStorage`
+  property (default: wraps `packedData` in `Storage.Heap`, so every existing quantized format is
+  unaffected); `packedView` reads from it instead of always re-wrapping the raw array.
+  `BitNetB158TensorData` gains a `Storage`-backed constructor/`fromStorage()` factory — the
+  per-element accessors lazily snapshot off-heap storage only if actually touched, so inference
+  through the GEMV kernels never materializes a heap copy. `StreamingGgufParametersLoader`
+  allocates off-heap for repacked I2_S payloads at or above `PlannerProfile.OFF_HEAP_THRESHOLD`
+  (256 KB) instead of a permanent `ByteArray`.
+- **Zero-copy native gemv for off-heap ternary weights**
+  ([#1202](https://github.com/SKaiNET-developers/SKaiNET/issues/1202),
+  [#1206](https://github.com/SKaiNET-developers/SKaiNET/pull/1206)): `TernaryF32GemvNative` gains
+  `gemvPackedStorage()`, letting the JVM/FFM face hand a `SegmentStorage`'s `MemorySegment`
+  straight to the native downcall — the weight is never copied, not even once, where the
+  previous `gemvPacked` path re-copied the *entire* weight matrix into a fresh arena on every
+  row of every call, independent of storage kind. `NativeTernaryF32ViewKernel` now dispatches on
+  the weight's storage kind instead of only accepting `Storage.Heap`, so an off-heap ternary
+  weight keeps the fast NEON/FFM path instead of silently falling back to the slow reference
+  kernel.
+- **Zero-copy mmap for `SEQUENTIAL`-layout I2_S tensors**
+  ([#1203](https://github.com/SKaiNET-developers/SKaiNET/issues/1203),
+  [#1208](https://github.com/SKaiNET-developers/SKaiNET/pull/1208)): `I2sRepack.toSequentialPayload`
+  no longer copies a `SEQUENTIAL` buffer that's already exactly the target payload (the common
+  NeoGPU-converted case). More significantly, the loader's mmap-eligibility branch — previously
+  keyed on a hardcoded encoding whitelist that didn't include `I2_S` — now maps a `SEQUENTIAL`
+  I2_S tensor directly off the file whenever its trailing bytes are provably the real scale (no
+  companion `<name>_scale` tensor overriding them), giving it the same zero-copy path Q4_K/Q8_0/etc.
+  already had. `GROUP_128`/`GROUP_64` payloads and companion-scored `SEQUENTIAL` files correctly
+  keep repacking.
+- **`I2sAotConverter`: AOT GGUF → GGUF conversion for I2_S**
+  ([#1207](https://github.com/SKaiNET-developers/SKaiNET/issues/1207),
+  [#1210](https://github.com/SKaiNET-developers/SKaiNET/pull/1210)): reads an arbitrary GGUF,
+  repacks I2_S tensors into `SEQUENTIAL`+trailer order ahead of time, drops the now-redundant
+  companion scale tensor, and passes every other tensor and all KV metadata through unchanged —
+  the converted file always takes the new zero-copy mmap path above, with no on-device cost at
+  all. `GgufTensorEntry` (the writer's tensor-entry type) gains a `rawBytes` passthrough mode to
+  support this without going through the element-indexed `TensorFlatten` path.
+- **GGUF I2_S → `.irpa` conversion** (IREE-facing counterpart, in
+  [SKaiNET-IREE-tools#1](https://github.com/SKaiNET-developers/SKaiNET-IREE-tools/pull/1), not
+  this repo): a standalone Python tool producing an IREE parameter archive directly from a
+  GGUF's ternary tensors, for `iree-compile --iree-opt-import-parameters=`.
+
+### Fixed
+
+- **Scoped dense-FP32 activations silently fell out of the quantized matmul chooser**
+  ([#1211](https://github.com/SKaiNET-developers/SKaiNET/pull/1211)): `chooseQuantizedMatmul2D`
+  accepted only `FloatArrayTensorData`/`MemorySegmentBackedData` activations; a
+  `ScopedExecutionContext` forward's slab-backed `StorageFloatTensorData` fell through to
+  `matmulGeneric`, whose per-element `get()` on a `Q8MemorySegmentTensorData` weight returns the
+  raw quantization byte, not the value — silently wrong logits (the #993 class of bug). Any dense
+  activation (`encoding == null`) is now accepted via its own offset-aware `copyToFloatArray()`.
+
 ## [0.50.0] - 2026-08-28
 
 Headline: **model size on Android is now a page-cache question, not a heap question — and decode is

README.md

@@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL):
 ```kotlin
 dependencies {
     // Recommended: import the umbrella BOM and drop versions on the engine modules.
-    implementation(platform("sk.ainet:skainet-bom:0.49.0"))
+    implementation(platform("sk.ainet:skainet-bom:0.51.0"))
 
     implementation("sk.ainet.core:skainet-lang-core")
     implementation("sk.ainet.core:skainet-backend-cpu")
@@ -296,69 +296,22 @@ val withoutLabel = dataPipeline<RawDataset>()
 
 ---
 
-## What's New in 0.49.0
-
-The **SKEEP-jump release**: 0.40.1 → 0.49.0, ~100 merged PRs — the SKEEP-003 memory & storage
-architecture complete, from accepted proposal to shipped system. This is the release downstream
-repositories (SKaiNET-transformers, the IREE conformity pipeline) should build on. It **removes
-every façade the architecture replaced** — see the Breaking-changes section of
-[CHANGELOG.md](CHANGELOG.md) for the migration map (`QuantPolicy`/`StagingPolicy`/`WeightOrientation`
-→ `WeightForm`; `@Place`/`@Weights`/`StorageSpec`/old `MemoryPlanner` → `AllocationResolver`).
-
-- **One storage model** — `Storage` / `Scope` / `Format` / `Layout` / `TensorView`: enforced ownership
-  (use-after-free throws, loudly), scoped lifetimes, `prepack()` as the visible relayout,
-  `materialize()` as the single copy point.
-- **Decisions are resolved, not declared** — `WeightFormResolver` picks a weight's in-memory form from
-  *file × profile × kernels*; `AllocationResolver` picks domain and scope, and `explain()` says why,
-  per tensor, before a byte of payload is read. You always outrank the resolver
-  (per-tensor `weightFormFor` > uniform `weightForm` > resolver).
-- **Flat-memory decode** — `ctx.forwardScope(slabFloats) { … }` recycles one slab per step; creation
-  *and op outputs* draw from it, and the FP32 fast paths + JVM Panama vector kernels are offset-aware,
-  so scoped tensors keep SIMD speed. Steady-state decode allocates zero new slab bytes per step.
-- **"Will it fit?" in seconds, any format** — header-only footprint plans for GGUF, **safetensors and
-  ONNX** (external-data sidecars priced correctly), with `PlannerProfile.EDGE` for embedded devices:
-  `skainet-plan model.onnx --profile edge --budget 2.1G`, exit code 0/1.
-- **The compile lane carries what the runtime decides** — tensor identity, structural encodings
-  (`skainet.tensor_layouts`: block sizes and bit widths as integers, not names) and block order flow
-  into the exported MLIR and `.irpa`; `HloGenerator.generate(target = …)` runs the first layout pass
-  on the production path.
-- **BitNet / ternary compute** — `BITNET_PLANES` multi-plane packing, i2s GGUF import, and the vendored
-  NeoGPU ternary f32 NEON kernel through FFM, JNI and Kotlin/Native, including a fused lm_head kernel.
-- **Docs that cannot rot** — the Android classifier and ternary getting-started tutorials are compiled
-  *and executed* in CI (the Iris training loop asserts held-out accuracy ≥ 0.80), plus the
-  virtual-tensors explanation and SKEEP-003a resolution record.
-
-### Previously, in 0.40.1
-
-- **Correctness hotfix: packed-quant `transpose()` was silently wrong, not crashing.** `ops.matmul(x, ops.transpose(W))` on a packed-quantized weight (Q4_0/Q5_0/Q5_1/Q8_0/Q4_K/Q5_K/Q6_K) with more than one quant block per row produced silently incorrect output — sometimes all-zero — across the scalar, Panama-vector, *and* native (FFM/JNI) kernel tiers, with no exception raised. `transpose()` now performs a real block-grid byte permutation instead of a shape-only relabel; a misaligned packed tensor now throws instead of silently truncating. Closes [#968](https://github.com/SKaiNET-developers/SKaiNET/issues/968). **Upgrading is strongly recommended** for anyone calling `ops.transpose()` on packed-quantized weights.
-
-### Previously, in 0.40.0
-
-- **Android models grow past the ART heap cap.** Off-heap/mmap tensor storage shares the JVM's memory-mapped weight loading with Android — dense F32 tensors serve as zero-heap mapped views, and weight bytes live in OS-paged file-backed pages instead of the managed heap. A 640 MB dense model now loads with **1.4 MB** of heap allocation.
-- **GGUF `DEQUANTIZE_TO_FP32` no longer over-allocates.** A 1.1B Q4_K_M GGUF transiently needed >12 GB heap against a ~4.4 GB dense-FP32 floor. Three compounding allocation sources in the loader and K-quant kernels are fixed, bringing peak live allocation to ~1.05x of the dense FP32 size.
-- **Q5_0/Q5_1 packed matmul reaches the native tier.** New NEON C kernels for both formats are wired into the FFM (JVM), Kotlin/Native, and Android JNI providers — unblocking the packed Q5_1 path for Q5_K_M checkpoints under `NATIVE_OPTIMIZED`.
-- **SKaiNET reaches iOS and macOS natively.** `skainet-backend-native-cpu` now publishes `iosArm64`, `iosSimulatorArm64`, and `macosArm64` Kotlin/Native targets with embedded kernel archives; a single Apple arm64 archive dispatches FEAT_DotProd at runtime, so one build serves A12 through M-series.
-
-### Previously, in 0.39.1
-
-- **Eager CPU ops run primitive FP32 fast paths.** The generic per-element paths (index-array allocations, boxed accessors, dtype dispatch) dominated on-device LLM decode — 83% of end-to-end SmolLM2-135M decode time on a Pixel 8a was non-matmul overhead even with the NEON backend. Hot ops (arithmetic, activations, unary math, softmax/logSoftmax, reductions, concat, reshape) now run flat primitive loops over the dense `FloatArray` buffer, benefiting every non-JVM target — Android, Kotlin/Native, JS/Wasm. `DirectCpuExecutionContext.ops` is also cached instead of rebuilt per access.
-- **README points LLM users to SKaiNET-transformers.** A callout under "Start in 5 minutes" makes clear that LLM inference lives in the SKaiNET-transformers repository — this repo is the engine underneath.
-
-### Previously, in 0.39.0
-
-- **On-device AI on Android — a NEON kernel backend.** New `skainet-backend-jni-cpu` module: the hand-tuned ARM matmul kernels reach Android through a JNI bridge (ART has no `java.lang.foreign`, so the FFM provider can never run there). Two `.so` tiers are built from the same sources and selected at load time from `/proc/cpuinfo` — a baseline `armv8-a` build that runs on every 64-bit core, and an `armv8.2-a+dotprod` build for the `vdotq_s32` Q4_K/Q6_K paths — so a single artifact is safe from Cortex-A53 up. Measured on a Pixel 8a: **~24 tok/s** SmolLM2-135M Q8_0 decode versus ~3.8 scalar (6.4x), clearing the on-device usability bar. The provider auto-registers via `ServiceLoader`; an app just adds the AAR.
-- **Android GGUF loading no longer OOMs.** `createRandomAccessSource` returned `null` on Android, forcing every model load through a full-file heap read that exhausted the ART heap on real devices. It now streams via positional `FileChannel` reads across `skainet-io-gguf` / `-safetensors` / `-onnx`.
-- **Published Kotlin/Native kernel klibs are linkable.** The static kernel archive is now embedded into the cinterop klib, so downstream K/N consumers of `skainet-backend-native-cpu` (`-linuxx64` / `-linuxarm64`, and the path future Apple targets will use) link with no manual setup. A NEON body was also added for the Q4_0 matmul kernel.
-- **Tensor-storage correctness pass.** Fail-fast on unsupported GGUF quant types instead of silently dropping weights; truthful ownership labels and real byte counts in the storage layer; a materializable `FileBacked`/`Aliased` transfer path; and a rank-safe default `copyToFloatArray`.
-
-### Previously, in 0.38.0
-
-- **Streaming KV-cache decode (dynamic dimensions)** — a first-class `Dim` vocabulary makes "dynamic extent" explicit instead of an overloaded `-1`, and the StableHLO emitter renders it as an MLIR `?`. One compiled vmfb now serves every autoregressive decode step with a growing cache, instead of one fixed cache length. Verified end-to-end: the full FunctionGemma `with_past` decode graph and the Moonshine v2 decoder (dynamic self *and* cross caches) self-compile from the DSL to a CPU vmfb — graphs that could not be compiled before. Static graphs are emitted byte-for-byte unchanged.
-- **Narrow-float (BF16 + FP16) weights kept packed** — SafeTensors F16 and GGUF F16/BF16 weights load `KEEP_NATIVE`, two bytes per element at rest instead of widening to FP32, and reach format-specific matmul kernels still packed. Narrow floats are a storage width only: kernels widen to f32 lanes and accumulate in f32.
-- **Both narrow formats now beat the FP32 SGEMM** — BF16 by 1.8–1.9x, FP16 by 1.5–1.7x on a 4096x11008 projection. Getting there took a zero-copy transpose for input-major weights (the per-token transpose previously widened the tensor elementwise, 4.4 s per projection), a native FFM FP16 kernel to match the existing BF16 one, and tiling both kernels so the weight is read once per matmul rather than once per input row.
-- **Allocation-free shape-only tracing** — `VoidTensorOps` propagates shapes through a `ShapeOnlyTensorData` that allocates no backing buffer, so a dynamic extent flows through a whole decode trace instead of throwing on a negative-size allocation.
-
-See [CHANGELOG.md](CHANGELOG.md) for details and the full release history.
+## What's New in 0.51.0
+
+Ternary/BitNet weights join the memory-mapped weight story 0.50.0 started for every other quant
+format:
+
+- **Off-heap ternary storage** — `BitNetB158TensorData` no longer risks the Android ART heap-cap
+  OOM; `Storage.copyInto`/`copyFrom` give every storage kind one shared bulk-copy primitive.
+- **True zero-copy mmap** for `SEQUENTIAL`-layout (NeoGPU-converted) GGUFs, and a zero-copy
+  native gemv path for off-heap ternary weights on the JVM/FFM kernel.
+- **`I2sAotConverter`** (GGUF → GGUF): convert I2_S tensors ahead of time so a controlled model
+  pipeline never pays a runtime repack. The IREE-facing counterpart lives in
+  [SKaiNET-IREE-tools](https://github.com/SKaiNET-developers/SKaiNET-IREE-tools).
+- **Correctness fix** — scoped dense-FP32 activations no longer silently fall out of the
+  quantized matmul chooser (was producing wrong logits under `ScopedExecutionContext`).
+
+See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release.
 
 ---
 
@@ -382,6 +335,12 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix:
 
 Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET).
 
+### Contributors (0.51.0)
+
+- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — off-heap ternary
+  storage, zero-copy mmap for `SEQUENTIAL` I2_S, the AOT GGUF converter and its IREE-facing
+  counterpart in SKaiNET-IREE-tools, and the scoped dense-FP32 activation matmul-chooser fix
+
 ### Contributors (0.49.0)
 
 - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the SKEEP-003 memory & storage architecture end to end: M0/M1/M2 milestones, the weight-form and placement-resolution arcs, scope-recycled execution, multi-format footprint analysis, the compile-lane carriage arc, the BitNet/ternary kernel track, and the release docs

docs/antora.yml

@@ -15,7 +15,7 @@ asciidoc:
     framework_name: SKaiNET
     # Current SKaiNET release — bump once per release; referenced as
     # {skainet_version} in dependency snippets (blocks need subs="attributes+").
-    skainet_version: 0.49.0
+    skainet_version: 0.51.0
     ksp_version: 2.2.21-2.0.5
     dokka_version: 2.1.0
     asciidoctorj_version: 3.0.0

docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc

@@ -1,7 +1,7 @@
 = Kernel × platform support matrix
 :description: Which compute-kernel provider serves each weight format on each KMP target.
 
-Generated from `kernel-support.json` (version `0.49.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.
+Generated from `kernel-support.json` (version `0.51.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh.
 
 Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only).
 

docs/modules/ROOT/pages/reference/operators/generated/index.adoc

@@ -1,6 +1,6 @@
 = AI-NET Operators Reference
 
-Generated from version `0.49.0` on 2026-08-26
+Generated from version `0.51.0` on 2026-08-29
 
 == Operators by Modality
 

docs/modules/ROOT/pages/reference/ops-status-matrix.adoc

@@ -1,7 +1,7 @@
 = Operator Coverage Matrix
 :description: Cross-backend status for every operator function in SKaiNET.
 
-Generated from `operators.json` version `0.49.0` on 2026-08-26.
+Generated from `operators.json` version `0.51.0` on 2026-08-29.
 
 Rows are `Operator.function` pairs. The `Validated` column shows whether the function's documentation has been DARC-validated by a reviewer (see xref:contributing/darc-workflow.adoc[DARC workflow]). Remaining columns are backends that appear in any function's `statusByBackend` map — a missing entry means the backend makes no claim about the function (treat it as "unknown", not "not supported").
 

docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc

@@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules:
 [source,kotlin]
 ----
 dependencies {
-    implementation("sk.ainet.core:skainet-lang-core:0.49.0")     // tensors, DSL, training
-    implementation("sk.ainet.core:skainet-backend-cpu:0.49.0")   // CPU ops
-    implementation("sk.ainet.core:skainet-compile-dag:0.49.0")   // autograd (training context)
-    implementation("sk.ainet.core:skainet-data-api:0.49.0")      // Dataset / DataBatch
-    implementation("sk.ainet.core:skainet-data-simple:0.49.0")   // embedded Iris
-    runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.49.0")  // NEON kernels (see below)
+    implementation("sk.ainet.core:skainet-lang-core:0.51.0")     // tensors, DSL, training
+    implementation("sk.ainet.core:skainet-backend-cpu:0.51.0")   // CPU ops
+    implementation("sk.ainet.core:skainet-compile-dag:0.51.0")   // autograd (training context)
+    implementation("sk.ainet.core:skainet-data-api:0.51.0")      // Dataset / DataBatch
+    implementation("sk.ainet.core:skainet-data-simple:0.51.0")   // embedded Iris
+    runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.51.0")  // NEON kernels (see below)
 }
 ----
 

gradle.properties

@@ -1,5 +1,5 @@
 GROUP=sk.ainet.core
-VERSION_NAME=0.50.0
+VERSION_NAME=0.51.0
 POM_DESCRIPTION=SKaiNET
 
 POM_URL=https://github.com/SKaiNET-developers/skainet/