diff --git a/CHANGELOG.md b/CHANGELOG.md index b41e3c46..530b5dc6 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -2,8 +2,32 @@ ## [Unreleased] +## [0.53.0] - 2026-09-02 + +Headline: **the export pipeline emits billion-parameter models.** Tracing a 4.5B-parameter +Gemma 3n E2B through the DSL → tape → StableHLO path could not finish on a 48 GB host +([#1247](https://github.com/SKaiNET-developers/SKaiNET/issues/1247)): shape-only tracing +materialized real zero buffers, constant extraction copied every weight while the originals +stayed live, and when the converter did fail it said so in MLIR comments and exited 0 — so the +first broken node cascaded through a thousand more and still produced a "module". Each of those +is closed, and the last one turned out to hide a fourth: the tied embedding is exactly one byte +larger than a JVM array can hold, which no amount of widening could fix — external constants now +travel as the aliased float array they already are. The same repro that OOMed a 46 GB heap now +exports the full model in under a minute with zero failure comments, and the sharded SafeTensors +loader the transformers families kept re-implementing lives in the engine. + ### Added +- **`ShardedSafeTensorsParametersLoader` — the sharded-index SafeTensors loader** + ([#1246](https://github.com/SKaiNET-developers/SKaiNET/issues/1246), + [#1252](https://github.com/SKaiNET-developers/SKaiNET/pull/1252)): a `ParametersLoader` over + `model.safetensors.index.json`, riding `StreamingShardedSafeTensorsReader.openFromIndex` with the + single-file loader's BF16/FP16 policies (`withPolicy` parity), a fail-fast dtype pre-scan before + any tensor is delivered, and a `tensorFilter` hook so size guards and name allowlists stay + family-side while every dtype decision stays in the engine. The dtype dispatch and dequant + helpers moved into a shared internal `SafeTensorsMaterializer`, so both loaders materialize + identically (parity-tested). Downstream: SKaiNET-transformers' hand-rolled per-family SafeTensors + loaders can collapse onto it. - **`BufferHandle.Floats` — array-free path for ≥2 GiB constants** ([#1247](https://github.com/SKaiNET-developers/SKaiNET/issues/1247)): external FP32 constants now ride the aliased `FloatArray` end-to-end (graph → `ExternalParameterRef` → `.irpa`), never @@ -16,6 +40,14 @@ ### Changed +- **Void tracing allocates nothing** + ([#1247](https://github.com/SKaiNET-developers/SKaiNET/issues/1247), + [#1249](https://github.com/SKaiNET-developers/SKaiNET/pull/1249)): `VoidTensorOps` recorded every + shape-propagation op into a real dense zero buffer, allocated twice and retained by the trace — + ~9 GB of zeros across a 30-layer Gemma 3n E2B trace. Static-shape void results are now lazy + placeholders (readers still see zeros; unread ops allocate nothing), `ShapeOnlyTensorData` is + public for consumers that hand-roll shape-only data, and `matmulWeightTransposed` no longer + constructs the transposed intermediate. A 4096×4096 traced op stack tracks 0 bytes. - **Graph constants alias live weights; packed params fail loudly** ([#1247](https://github.com/SKaiNET-developers/SKaiNET/issues/1247)): `TraceToGraphBuilder` no longer copies every frozen float weight into the graph — the constant's `initial_value` aliases diff --git a/README.md b/README.md index 3842ef59..7f430f92 100644 --- a/README.md +++ b/README.md @@ -51,7 +51,7 @@ Add the core dependencies (Gradle Kotlin DSL): ```kotlin dependencies { // Recommended: import the umbrella BOM and drop versions on the engine modules. - implementation(platform("sk.ainet:skainet-bom:0.52.0")) + implementation(platform("sk.ainet:skainet-bom:0.53.0")) implementation("sk.ainet.core:skainet-lang-core") implementation("sk.ainet.core:skainet-backend-cpu") @@ -308,29 +308,27 @@ val withoutLabel = dataPipeline() --- -## What's New in 0.52.0 - -The engine stops silently running on the scalar floor: - -- **Self-healing kernel dispatch** — `KernelDispatch` installs itself on first use through the new - `ViewKernelPack` SPI, so an application that never called an install routine no longer loses - every kernel and falls back to the decoding reference path. When a fallback does happen, it now - says so once, loudly, instead of vanishing into a no-op trace sink. -- **Ternary/BitNet kernels in the discovery set** — the `BITNET_B1_58` LUT gemv and the fused - `BITNET_PLANES` lm_head packs are ServiceLoader-discovered like the Q-series, so a BitNet - consumer gets the vendored NeoGPU kernels with zero bootstrap code. Validated downstream: - SKaiNET-transformers decodes BitNet-2B4T at full speed on discovery alone. -- **Dense FP32 from any storage kind** — mapped and off-heap weights were dequantizing because the - kernel serving them only recognised `Heap`, which defeated the point of memory-mapped staging. -- **Android native across the chain** — `androidNativeArm32`/`Arm64` now build and publish from the - whole downstream dependency graph, not just `skainet-io-core`, so on-device consumers can - actually resolve what they need. -- **Faster decode-shaped matmul** — a dense FP32 GEMV path for the m ≤ 8 shapes decode issues - (16.7x at m=1), a direct-loop path for small work, and a cached weight transpose. ~2.3x decode - and prefill measured end to end on a downstream Gemma 4 port. -- **Gemma 4 loads through the engine's own routes** — `gemma4` is registered in `TokenizerFactory` - and `ModelArchitecture`, and `SpecialTokenSplitter` no longer drops word boundaries when - decoding token by token. +## What's New in 0.53.0 + +The export pipeline emits billion-parameter models: + +- **Full Gemma 3n E2B export** — the DSL → tape → StableHLO path traced a 4.5B-parameter model + into an OOM at a 46 GB heap; the same repro now exports the whole model in under a minute. + Shape-only tracing no longer materializes zero buffers, graph constants alias the live weights + instead of copying them, and ≥2 GiB constants travel as `BufferHandle.Floats` — an aliased + float array — because the tied embedding is exactly one byte over what a JVM byte array holds. +- **Conversion fails loudly** — `StableHloConverter` is strict by default: an unconvertible node + throws `HloConversionException`, an unresolved operand throws `MissingOperandException`, and a + packed weight reaching constant extraction throws `PackedConstantException` instead of silently + becoming a function argument. `ConversionErrorPolicy.LENIENT` restores the old + comment-and-continue behavior for inspection. +- **Sharded SafeTensors in the engine** — `ShardedSafeTensorsParametersLoader` consumes + `model.safetensors.index.json` with the single-file loader's BF16/FP16 policies, a fail-fast + dtype pre-scan, and a `tensorFilter` hook; the per-family hand-rolled loaders downstream can + collapse onto it. +- **Registry gaps the strictness surfaced** — `clamp` and the camelCase `indexSelect` the tracer + actually emits now lower, and `createBasic` registers the neural-net converter like + `createExtended` does. See [CHANGELOG.md](CHANGELOG.md) for full release notes, including every prior release. @@ -356,6 +354,12 @@ We love contributions! Whether it's a new operator, documentation, or a bug fix: Browse the full codebase documentation on [DeepWiki](https://deepwiki.com/SKaiNET-developers/SKaiNET). +### Contributors (0.53.0) + +- **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the billion-parameter + export arc: allocation-free void tracing, aliased constant extraction, strict StableHLO + conversion, the array-free `BufferHandle.Floats` path, and the sharded SafeTensors loader + ### Contributors (0.52.0) - **Michal Harakal** ([@michalharakal](https://github.com/michalharakal)) — the Gemma 4 engine-gap diff --git a/docs/antora.yml b/docs/antora.yml index 118292ef..ac0e802c 100644 --- a/docs/antora.yml +++ b/docs/antora.yml @@ -15,7 +15,7 @@ asciidoc: framework_name: SKaiNET # Current SKaiNET release — bump once per release; referenced as # {skainet_version} in dependency snippets (blocks need subs="attributes+"). - skainet_version: 0.52.0 + skainet_version: 0.53.0 ksp_version: 2.2.21-2.0.5 dokka_version: 2.1.0 asciidoctorj_version: 3.0.0 diff --git a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc index 1d8cb84a..51bde34d 100644 --- a/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/kernel-support-matrix.adoc @@ -1,7 +1,7 @@ = Kernel × platform support matrix :description: Which compute-kernel provider serves each weight format on each KMP target. -Generated from `kernel-support.json` (version `0.52.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. +Generated from `kernel-support.json` (version `0.53.0`) by `KernelSupportMatrixTest` — registry introspection of the registered `KernelProvider` implementations. Do not edit by hand; run `./gradlew generateKernelMatrix` to refresh. Each cell is the best (highest-priority) provider that serves `Float32 × format` `matmul` on that platform: *native-ffm* (100) → *panama-vector* (50) → *scalar* (0). An empty cell (`—`) means no provider carries a kernel there (the format is dequant-to-FP32 only). diff --git a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc index a7be6e3b..541a83fb 100644 --- a/docs/modules/ROOT/pages/reference/operators/generated/index.adoc +++ b/docs/modules/ROOT/pages/reference/operators/generated/index.adoc @@ -1,6 +1,6 @@ = AI-NET Operators Reference -Generated from version `0.52.0` on 2026-08-31 +Generated from version `0.53.0` on 2026-09-02 == Operators by Modality diff --git a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc index 4c55c0e9..cec66f93 100644 --- a/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc +++ b/docs/modules/ROOT/pages/reference/ops-status-matrix.adoc @@ -1,7 +1,7 @@ = Operator Coverage Matrix :description: Cross-backend status for every operator function in SKaiNET. -Generated from `operators.json` version `0.52.0` on 2026-08-31. +Generated from `operators.json` version `0.53.0` on 2026-09-02. Rows are `Operator.function` pairs. The `Validated` column shows whether the function's documentation has been DARC-validated by a reviewer (see xref:contributing/darc-workflow.adoc[DARC workflow]). Remaining columns are backends that appear in any function's `statusByBackend` map — a missing entry means the backend makes no claim about the function (treat it as "unknown", not "not supported"). diff --git a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc index 74d2cd6d..825ee0b0 100644 --- a/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc +++ b/docs/modules/ROOT/pages/tutorials/android-classifier-getting-started.adoc @@ -19,12 +19,12 @@ An Android project with Kotlin. Add the SKaiNET modules: [source,kotlin] ---- dependencies { - implementation("sk.ainet.core:skainet-lang-core:0.52.0") // tensors, DSL, training - implementation("sk.ainet.core:skainet-backend-cpu:0.52.0") // CPU ops - implementation("sk.ainet.core:skainet-compile-dag:0.52.0") // autograd (training context) - implementation("sk.ainet.core:skainet-data-api:0.52.0") // Dataset / DataBatch - implementation("sk.ainet.core:skainet-data-simple:0.52.0") // embedded Iris - runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.52.0") // NEON kernels (see below) + implementation("sk.ainet.core:skainet-lang-core:0.53.0") // tensors, DSL, training + implementation("sk.ainet.core:skainet-backend-cpu:0.53.0") // CPU ops + implementation("sk.ainet.core:skainet-compile-dag:0.53.0") // autograd (training context) + implementation("sk.ainet.core:skainet-data-api:0.53.0") // Dataset / DataBatch + implementation("sk.ainet.core:skainet-data-simple:0.53.0") // embedded Iris + runtimeOnly("sk.ainet.core:skainet-backend-jni-cpu:0.53.0") // NEON kernels (see below) } ---- diff --git a/gradle.properties b/gradle.properties index be85600a..468c5cf2 100644 --- a/gradle.properties +++ b/gradle.properties @@ -1,5 +1,5 @@ GROUP=sk.ainet.core -VERSION_NAME=0.52.0 +VERSION_NAME=0.53.0 POM_DESCRIPTION=SKaiNET POM_URL=https://github.com/SKaiNET-developers/skainet/