From 47e8f9a8f47ac07aad3780924a5ec4db7849b4e4 Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Wed, 12 Aug 2026 10:16:43 +0200 Subject: [PATCH] release: prepare 0.40.1 (hotfix: #968) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Bump VERSION_NAME 0.40.0 -> 0.40.1 and update all version-carrying docs for a hotfix release scoped to the single functional delta since the 0.40.0 tag: #969 (fixes #968). - CHANGELOG: new [0.40.1] - 2026-08-12 entry under [Unreleased], headline framing this as a correctness hotfix (silently wrong output, not a crash); ### Fixed covers DefaultCpuOps.transpose() doing a shape-only relabel instead of a real block-grid byte permutation for packed quantized formats (Q4_0/Q5_0/Q5_1/Q8_0/Q4_K/Q5_K/Q6_K) whenever blocksPerInputDim>1, across scalar, Panama-vector, and native kernel tiers; misaligned input now throws instead of silently truncating; credits the ground-truth regression test approach. - README: BOM snippet -> 0.40.1, What's New in 0.40.1 (correctness fix, recommend upgrading), 0.40.0 demoted to "Previously", Contributors (0.40.1). - docs/antora.yml: skainet_version attribute 0.40.0 -> 0.40.1. - kernel-support-matrix.adoc: regenerated via generateKernelMatrix on this base — no kernel/tier changes, version stamp only -> 0.40.1. Branch prep only — no tag created. Co-Authored-By: Claude Fable 5 --- CHANGELOG.md | 32 +++++++++++++++++++ README.md | 12 +++++-- docs/antora.yml | 2 +- .../reference/kernel-support-matrix.adoc | 2 +- gradle.properties | 2 +- 5 files changed, 45 insertions(+), 5 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b639730d8..0a97ae2c1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,38 @@ ## [Unreleased] +## [0.40.1] - 2026-08-12 + +Headline: **correctness hotfix — silently wrong output, not a crash.** `DefaultCpuOps.transpose()` for packed quantized weights (Q4_0/Q5_0/Q5_1/Q8_0/Q4_K/Q5_K/Q6_K) performed a shape-only relabel instead of a real block-grid byte permutation whenever a row spanned more than one quant block (`blocksPerInputDim > 1` — true of virtually every real model). `ops.matmul(x, ops.transpose(W))` fed the packed-quant kernels bytes in the wrong order across all three kernel tiers — scalar, Panama-vector, and native (FFM/JNI) — silently producing wrong numbers, sometimes all-zero output, with no exception raised. Upgrading is strongly recommended for anyone using packed-quantized weights with `ops.transpose()`. + +### Fixed + +- **Packed-quant `transpose()` silently corrupted matmul output** + ([#968](https://github.com/SKaiNET-developers/SKaiNET/issues/968), + [#969](https://github.com/SKaiNET-developers/SKaiNET/pull/969)) + — `DefaultCpuOps.transpose()` swapped the tensor's shape metadata without + physically reordering the underlying `packedData` bytes, on the assumption + that the packed-quant matmul kernels index those bytes block-major + regardless of layout. That assumption only holds when there is a single + quant block per row (`blocksPerInputDim == 1`); for any wider row the + canonical (row-major) and kernel-native block orderings are literal + transposes of the `(outputDim, blocksPerInputDim)` block grid and do not + coincide, so a freshly-loaded weight run through `ops.transpose()` fed the + scalar, Panama-vector, and native (FFM/JNI) kernel tiers alike bytes in the + wrong order — for all seven packed formats (Q4_0/Q5_0/Q5_1/Q8_0/Q4_K/Q5_K/Q6_K). + `transpose()` now performs a real `O(bytes)` block-grid permutation + (`transposePackedBlocks`); a misaligned packed tensor (`inputDim` not a + multiple of the format's block size) now throws `IllegalArgumentException` + instead of silently truncating a partial trailing block. + `DefaultCpuOpsJvm`'s separate, independently-buggy shape-swap-only + interception for `Q4_KTensorData` is removed, falling through to the + shared corrected implementation. Caught by a new ground-truth regression + test (`NativeLazyTransposeGroundTruthReproTest`) that dequants each packed + format's canonical and kernel-native byte layouts independently and checks + both the classic (`transpose` + matmul) and pre-transposed paths against + that ground truth, per format — the kind of test the original "same bytes, + new shape" optimization lacked. + ## [0.40.0] - 2026-08-11 Headline: **big models fit on real devices, and SKaiNET reaches iOS/macOS diff --git a/README.md b/README.md index e22990639..14824c55f 100644 --- a/README.md +++ b/README.md @@ -53,7 +53,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.40.0")) + implementation(platform("sk.ainet:skainet-bom:0.40.1")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -297,7 +297,11 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.40.0 +## What's New in 0.40.1 + +- **Correctness hotfix: packed-quant `transpose()` was silently wrong, not crashing.** `ops.matmul(x, ops.transpose(W))` on a packed-quantized weight (Q4_0/Q5_0/Q5_1/Q8_0/Q4_K/Q5_K/Q6_K) with more than one quant block per row produced silently incorrect output — sometimes all-zero — across the scalar, Panama-vector, *and* native (FFM/JNI) kernel tiers, with no exception raised. `transpose()` now performs a real block-grid byte permutation instead of a shape-only relabel; a misaligned packed tensor now throws instead of silently truncating. Closes [#968](https://github.com/SKaiNET-developers/SKaiNET/issues/968). **Upgrading is strongly recommended** for anyone calling `ops.transpose()` on packed-quantized weights. + +### Previously, in 0.40.0 - **Android models grow past the ART heap cap.** Off-heap/mmap tensor storage shares the JVM's memory-mapped weight loading with Android — dense F32 tensors serve as zero-heap mapped views, and weight bytes live in OS-paged file-backed pages instead of the managed heap. A 640 MB dense model now loads with **1.4 MB** of heap allocation. - **GGUF `DEQUANTIZE_TO_FP32` no longer over-allocates.** A 1.1B Q4_K_M GGUF transiently needed >12 GB heap against a ~4.4 GB dense-FP32 floor. Three compounding allocation sources in the loader and K-quant kernels are fixed, bringing peak live allocation to ~1.05x of the dense FP32 size. @@ -354,6 +358,10 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.40.1) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — packed-quant `transpose()` block-grid correctness fix, all three kernel tiers (#968, #969) + ### Contributors (0.40.0) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — off-heap/mmap tensor storage on Android (#921), GGUF `DEQUANTIZE_TO_FP32` over-allocation fix (#782), native Q5_0/Q5_1 packed matmul kernels (#708), Apple arm64 runtime FEAT_DotProd dispatch (#958), Apple iOS/macOS Kotlin/Native kernel targets (#959) diff --git a/docs/antora.yml b/docs/antora.yml index 4737fc66e..a03272633 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.40.0 + skainet_version: 0.40.1 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index 6709ff4a5..4c4104a19 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.40.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.40.1`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). diff --git a/gradle.properties b/gradle.properties index dce5d82b2..05b1dc8f0 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.40.0 +VERSION_NAME=0.40.1 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/