From 096565a6dd32b992082fae08880fe25b261f648e Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Fri, 24 Apr 2026 23:50:24 +0200 Subject: [PATCH] chore(release): prepare 0.20.0 - CHANGELOG: add 0.20.0 section (Q6_K matmul, Q4_K/Q6_K lazy transpose, SDPA recording + StableHLO lowering, SDPA shape validation, dep bumps). - README: bump Quickstart coordinates to 0.20.0 and refresh "What's New". - gradle.properties: drop -SNAPSHOT and enable RELEASE_SIGNING_ENABLED. Co-Authored-By: Claude Opus 4.7 (1M context) --- CHANGELOG.md | 21 +++++++++++++++++++++ README.md | 19 +++++++++---------- gradle.properties | 4 ++-- 3 files changed, 32 insertions(+), 12 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index b80d1b673..0a25bc130 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,6 +2,27 @@ ## [Unreleased] +## [0.20.0] - 2026-04-24 + +### Added + +#### Quantized matmul (Q4_K / Q6_K on CPU) +- **Q6_K Native Matmul**: New `Q6_KTensorData` / `Q6_KBlockTensorData` in `skainet-lang-core` stores 210-byte ggml Q6_K blocks verbatim (128 `ql` + 64 `qh` + 16 scales + 2 f16 `d`), row-major by default, with a `dequantizeBlock` path matching the `DequantOps` reference line-for-line. `DefaultCpuOpsJvm.chooseQuantizedMatmul` dispatches to a new `JvmQuantizedVectorKernels.matmulQ6_KVec` SIMD kernel (Kotlin Vector API, same `floatSpecies` as the Q4_K / Q8_0 kernels) using a dequant-one-block-to-scratch-then-SIMD-dot pattern. New `TensorEncoding.Q6_K` variant. Unblocks running Gemma 4 E2B Q4_K_M (and any mostly-Q4_K + Q6_K checkpoint) through the DSL path without a ~12 GB FP32 dequant blow-up at load. +- **Q4_K Lazy Shape-Swap Transpose**: `DefaultCpuOpsJvm.transpose(Q4_KTensorData)` now returns a new `Q4_KBlockTensorData` wrapping the *same* packed byte array with swapped shape — mirroring the existing Q4/Q8 MemorySegment lazy-transpose path. `matmulQ4_KVec`'s input-block-major layout produces correct values under the swapped shape without any physical data reordering, so `linearProject(x, W)` can run `matmul(x, transpose(Q4_K_W))` without round-tripping through FP32. Validated at the DSL level by `GemmaDslQ4KTest` in the transformers repo (Δ logits = 4.29e-6 vs the FP32 baseline). +- **Q6_K Lazy Transpose**: Same shape-swap specialization extended to `Q6_KTensorData`, enabling the same DSL path for Q6_K weights. +- **Lazy-Transpose Invariant Tests**: New `QuantizedMemSegMatmulTest` cases pin the two load-bearing properties of the Q4_K and Q6_K transpose specializations — (1) shape is swapped; (2) `packedData` is the SAME byte-array reference, not a copy — so the path cannot silently regress to the generic element-wise transpose (which would `ClassCastException` on packed nibbles). + +#### StableHLO → IREE compilation +- **SDPA Recording + StableHLO Emission**: `scaledDotProductAttention` is now recorded by `RecordingExecution` (was silently delegating without recording, like `conv1d` before #532) and lowered to StableHLO by `NeuralNetOperationsConverter`. The decomposition is `dot_general(Q, K.T)` (batching dims `[0,1]`, contracting dims `[3]×[3]`) → scale → optional mask → softmax (max-subtract-exp-sum-div) → `dot_general(weights, V)` (contracting dims `[3]×[2]`). New `ScaledDotProductAttentionOperation` in `TensorOperations` with output-shape inference (output shape = query shape). New `SdpaHloExportTest` verifies tape → graph → MLIR with `dot_general`; `TapeAttentionPermuteBugTest` pins a regression around raw array permute producing zero constants. `ShapeOperationsConverter.concatenate` input-type annotation fix. (#543) + +### Fixed +- **SDPA Q/K/V Shape Validation**: `scaledDotProductAttention` previously required only rank-4 inputs, so a mismatch in `head_dim` (e.g. Q=512 vs K=256, as seen in real Gemma 4 E2B where mixed-head-dim layers share a KV cache) surfaced as an `ArrayIndexOutOfBoundsException` buried 2000+ lines deep in the dot-product loop. Added `require()` preconditions on matching batch, head count, Q/K head_dim, Q/V head_dim, and K/V `seqKV`, each with a message naming the offending dimensions. New `SDPAShapeValidationTest` (5 cases, `commonTest`) pins the contract. + +### Dependencies +- Kotlin: 2.3.20 → 2.3.21 (including JVM toolchain and `plugin.serialization`). +- Android Gradle Plugin: 9.1.1 → 9.2.0. +- `io.ktor:ktor-client-core`: 3.4.2 → 3.4.3. + ## [0.19.1] - 2026-04-21 ### Fixed diff --git a/README.md b/README.md index 4a0be801a..e5dcfeb6f 100644 --- a/README.md +++ b/README.md @@ -19,8 +19,8 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { - implementation("sk.ainet.core:SKaiNET-lang-core:0.19.0") - implementation("sk.ainet.core:SKaiNET-backend-cpu:0.19.0") + implementation("sk.ainet.core:SKaiNET-lang-core:0.20.0") + implementation("sk.ainet.core:SKaiNET-backend-cpu:0.20.0") } ``` @@ -137,14 +137,13 @@ SKaiNET is a modular ecosystem. While this repository contains the core engine, --- -## What's New in 0.19.0 +## What's New in 0.20.0 -- **Qwen / GPT-2 Byte-Level BPE Tokenizer** — Full GPT-2-style pipeline (byte-to-unicode, pretokenization regex, merge-rank BPE, atomic special-token splitting). Builds from GGUF metadata or HuggingFace `tokenizer.json`; verified against Qwen2.5-0.5B reference token IDs. -- **LLaMA / SentencePiece Tokenizer** — llama.cpp SPM pipeline with whitespace escape, **score-priority** BPE (SPM rule, opposite of GPT-2 merge-rank), and `<0xNN>` byte fallback. Builds from GGUF (`tokenizer.ggml.model == "llama"`) and HuggingFace Unigram `tokenizer.json`. -- **`TokenizerFactory` Per-Architecture Dispatch** — Tokenizer selection is now per-architecture, not per file format. Qwen/GPT-2 → byte-level BPE, LLaMA/Gemma/TinyLlama → SentencePiece, regardless of whether weights come from GGUF or SafeTensors. -- **Byte-Level BPE Fix for Qwen/GPT-2** — Previously these models encoded text into garbage tokens because `GgufModelMetadata` ignored `tokenizer.ggml.merges` entirely, blocking chat mode and tool calling. (#463) -- **LLaMA GGUF Tokenization Fix** — `TokenizerFactory` previously threw `UnsupportedTokenizerException` for LLaMA-family GGUFs; the new SentencePiece path closes that gap. (#464) -- **GGUF UInt Field Fix** — UINT32 fields (e.g. `tokenizer.ggml.bos_token_id`) are Kotlin `UInt` value classes, not subclasses of `Number`, and were silently dropped by `as? Number` casts. Fixed via a `toIntFlexible` helper that handles every signed and unsigned numeric type GGUF can produce. +- **Q6_K Native Matmul** — New `Q6_KTensorData` stores 210-byte ggml blocks verbatim and a Vector-API SIMD kernel (`matmulQ6_KVec`) dispatches from `DefaultCpuOpsJvm.chooseQuantizedMatmul`. Together with the existing Q4_K infra, this unblocks running Gemma 4 E2B Q4_K_M (and any mostly-Q4_K + Q6_K checkpoint) through the DSL path without a ~12 GB FP32 dequant blow-up at load. +- **Q4_K / Q6_K Lazy Shape-Swap Transpose** — `ops.transpose` on `Q4_KTensorData` / `Q6_KTensorData` now returns a new tensor wrapping the *same* packed byte array with swapped shape, matching the existing Q4/Q8 MemorySegment path. `linearProject(x, W)` can run `matmul(x, transpose(W))` on Q4_K/Q6_K weights without round-tripping through FP32 (Δ logits = 4.29e-6 vs FP32 baseline on Gemma). +- **SDPA → StableHLO / IREE** — `scaledDotProductAttention` is now recorded by `RecordingExecution` and lowered to StableHLO as `dot_general(Q, K.T)` → scale → optional mask → softmax → `dot_general(weights, V)`, so attention blocks compile end-to-end through the SKaiNET → StableHLO → IREE path. (#543) +- **SDPA Q/K/V Shape Validation** — Mismatched `head_dim` between Q/K or Q/V (seen in real Gemma 4 E2B with mixed-head-dim layers sharing a KV cache) used to surface as an `ArrayIndexOutOfBoundsException` deep in the dot-product loop; `scaledDotProductAttention` now fails fast with `require()` messages naming the offending dimensions. +- **Toolchain bumps** — Kotlin 2.3.21, AGP 9.2.0, Ktor client 3.4.3. See [CHANGELOG.md](CHANGELOG.md) for the full release history. @@ -153,7 +152,7 @@ See [CHANGELOG.md](CHANGELOG.md) for the full release history. ## Roadmap - **Q1 2026**: Comprehensive documentation ✅ -- **Q2 2026**: TurboQuant KV-cache compression ✅ (shipped in 0.18.0); Qwen/LLaMA tokenizers ✅ (shipped in 0.19.0) +- **Q2 2026**: TurboQuant KV-cache compression ✅ (shipped in 0.18.0); Qwen/LLaMA tokenizers ✅ (shipped in 0.20.0) - **Q3 2026**: Agentic AI enhancements ✅ (tool calling shipped in 0.13.0; ongoing) - **Q4 2026**: Federated learning support for multi-device training diff --git a/gradle.properties b/gradle.properties index 4c9273e4a..b638262ba 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.20.0-SNAPSHOT +VERSION_NAME=0.20.0 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/ @@ -38,4 +38,4 @@ kotlin.mpp.stability.nowarn=true SONATYPE_HOST=DEFAULT SONATYPE_AUTOMATIC_RELEASE=false -RELEASE_SIGNING_ENABLED=false +RELEASE_SIGNING_ENABLED=true