feat(memory): block order belongs to the layout — BlockOrder, a visible prepack adapter, the normative doc (#973.1)

docs/design/memory/packed-weight-layout.md

@@ -0,0 +1,77 @@
+# Packed weight layout — the normative contract
+
+Every packed (block-quantized) weight in SKaiNET is stored in **one of exactly two block orders**,
+and which one it is in is a property of the value, not of the type it happens to have or of the
+module that produced it. This document is that contract; kdocs link here instead of restating it.
+
+Written for [#973](https://github.com/SKaiNET-developers/SKaiNET/issues/973), whose census found
+seven mutually contradicting statements of it inside this repository alone — two of which had
+already produced wrong numbers in production ([#968](https://github.com/SKaiNET-developers/SKaiNET/issues/968),
+[#971](https://github.com/SKaiNET-developers/SKaiNET/issues/971)).
+
+## The two orders
+
+A 2-D weight is logically `[out, in]`. Its blocks tile the **input** dimension, so a row of
+`in` elements is `blocksPerRow = in / blockSize` blocks, and the whole weight is an
+`out × blocksPerRow` grid of blocks. Flattening that grid is where the two orders come from:
+
+| order | flat block index | who produces it | who reads it |
+|---|---|---|---|
+| `BlockOrder.ROW_MAJOR` (canonical) | `o * blocksPerRow + b` | GGUF files, `Q*Quantizer`, `TernaryCodec` | `toFloatArray()`, `get()`, the reference matmul, `bitnet_gemv` |
+| `BlockOrder.INPUT_BLOCK_MAJOR` (kernel feed order) | `b * out + o` | `TensorView.prepack(INPUT_BLOCK_MAJOR)` | the packed matmul kernels — scalar, Panama, native C, JNI |
+
+**They coincide only when `blocksPerRow == 1`.** For any weight wider than one block — that is,
+virtually every real weight — reading one order as the other produces a block-permuted matrix:
+finite, plausible numbers that are simply wrong. Nothing crashes. That is the entire reason this
+document exists.
+
+## Where the order lives
+
+`Layout.blockOrder`. A `TensorView` over packed bytes carries it, and the order is expressed **in
+the strides**, not in a branch: input-block-major is `strides = [1, out]` over the block grid
+instead of `[blocksPerRow, 1]`. Everything else — `narrow`, `transpose`, `get`, `toFloatArray` —
+therefore works unchanged on a view in either order, and a prepacked view still describes the same
+matrix.
+
+`OperandKey`'s `LayoutClass` splits the same way: `BLOCKED_ROW_MAJOR` and `BLOCKED_INPUT_MAJOR`.
+A kernel **declares** which order it reads, in its `KernelKey`. That is what makes the dispatcher
+able to insert a relayout instead of the caller having to know.
+
+## Converting between them
+
+`TensorView.prepack(order, scope, sink)`:
+
+- returns `this` when the order already matches — the only free case;
+- otherwise copies the blocks into `scope` and emits `TraceEvent.AdapterInserted` with the byte
+  count, so the conversion shows up in the trace with its price;
+- the result carries the new order on its layout.
+
+It is a *conversion*, and it is named as one. It is not `transpose`: a true transpose of a
+block-quantized weight would need runs of quantized values along the other axis, i.e.
+requantization. What the engine historically called a "packed transpose" was this conversion
+wearing transpose's name and a swapped shape label that lied about the data
+(see #973, "the deeper semantic problem"); replacing that with a weight-transposing matmul
+primitive is [#1096](https://github.com/SKaiNET-developers/SKaiNET/issues/1096).
+
+## Rules
+
+1. **A file's bytes are `ROW_MAJOR`.** Anything loaded from GGUF, produced by a quantizer, or
+   written by `TernaryCodec` is canonical. A loader never silently relayouts.
+2. **A kernel declares its order in its key.** No kernel may assume; no caller may guess.
+3. **A conversion is visible.** It allocates in a scope and emits an adapter event. A relayout that
+   does not appear in the trace is a bug.
+4. **`get()` and `toFloatArray()` always mean the same thing** in either order — they read through
+   the layout. A view whose decoded content depends on which module produced it is a bug.
+5. **`packedData` byte semantics are public API.** Changing the order a type holds is a
+   minor/major change, never a patch — this is what let a byte-layout change ship as a green-CI
+   hotfix once already.
+
+## Status
+
+`Layout.blockOrder`, the `LayoutClass` split, `prepack` and this document land with
+[#1094](https://github.com/SKaiNET-developers/SKaiNET/issues/1094). The remaining work is tracked
+as sub-issues of #973: bridging the packed SPI kernels through the ordered key
+([#1095](https://github.com/SKaiNET-developers/SKaiNET/issues/1095)), the weight-transposing matmul
+primitive (#1096), engine-owned prepacking and cross-repo contract fixtures
+([#1097](https://github.com/SKaiNET-developers/SKaiNET/issues/1097)), and normalizing weight
+orientation at the load boundary ([#1098](https://github.com/SKaiNET-developers/SKaiNET/issues/1098)).

skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/BitNetGemvKernel.kt

@@ -20,7 +20,7 @@ import sk.ainet.lang.tensor.storage.TensorEncoding
  * exactly as the vector kernels will factor them.
  *
  * Operands: `[rows, k]` activations in [I8Absmax.FORMAT] × `[n, k]` ternary weights in canonical
- * (row-major block) order — the order [TernaryCodec] produces and GGUF stores. Output `[rows, n]`.
+ * ([sk.ainet.lang.memory.BlockOrder.ROW_MAJOR]) block order — the order [TernaryCodec] produces and GGUF stores. Output `[rows, n]`.
  *
  * The weight's codes are read once per call, not once per row: a decode step is one row against
  * the whole matrix, so hoisting it is the difference between O(rows·n·k) decodes and O(n·k).
@@ -108,7 +108,7 @@ public class BitNetGemvKernel(override val key: KernelKey) : ViewKernel {
             op = "matmul",
             operands = listOf(
                 OperandKey.contiguous(I8Absmax.FORMAT),
-                OperandKey(weightFormat, LayoutClass.BLOCKED),
+                OperandKey(weightFormat, LayoutClass.BLOCKED_ROW_MAJOR),
             ),
         )
 

skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/KernelKey.kt

@@ -51,7 +51,10 @@ public data class OperandKey(val format: Format, val layout: LayoutClass) {
         /** Describe [view]: dense-and-gap-free is `CONTIGUOUS`, a packed layout is `BLOCKED`, anything else `STRIDED`. */
         public fun of(view: TensorView): OperandKey {
             val cls = when {
-                view.layout.blocked -> LayoutClass.BLOCKED
+                view.layout.blocked -> when (view.layout.blockOrder) {
+                    sk.ainet.lang.memory.BlockOrder.ROW_MAJOR -> LayoutClass.BLOCKED_ROW_MAJOR
+                    sk.ainet.lang.memory.BlockOrder.INPUT_BLOCK_MAJOR -> LayoutClass.BLOCKED_INPUT_MAJOR
+                }
                 view.isContiguous -> LayoutClass.CONTIGUOUS
                 else -> LayoutClass.STRIDED
             }
@@ -65,12 +68,30 @@ public data class OperandKey(val format: Format, val layout: LayoutClass) {
 
 /**
  * How an operand's bytes are arranged, as far as kernel selection cares: one gap-free run
- * ([CONTIGUOUS]), a strided view over a larger buffer ([STRIDED]), or block-packed ([BLOCKED]).
+ * ([CONTIGUOUS]), a strided view over a larger buffer ([STRIDED]), or block-packed in one of the
+ * two orders that exist ([BLOCKED_ROW_MAJOR], [BLOCKED_INPUT_MAJOR]).
+ *
  * A kernel that declares `CONTIGUOUS` gets a gather adapter inserted for a `STRIDED` operand
- * (§5.1) — the adapter is visible in the trace, never hidden inside a kernel.
+ * (§5.1) — the adapter is visible in the trace, never hidden inside a kernel. The two blocked
+ * classes exist for the same reason: a packed kernel reads its weight in *one* of the two block
+ * orders, and #973 is what happens when that is left implicit. A kernel declares which one it
+ * takes, and the dispatcher relayouts when the operand disagrees.
  */
 @ExperimentalMemoryApi
-public enum class LayoutClass { CONTIGUOUS, STRIDED, BLOCKED }
+public enum class LayoutClass {
+    CONTIGUOUS,
+    STRIDED,
+
+    /** Blocks in file order: `o * blocksPerRow + b` ([sk.ainet.lang.memory.BlockOrder.ROW_MAJOR]). */
+    BLOCKED_ROW_MAJOR,
+
+    /** Blocks in kernel feed order: `b * outputDim + o` ([sk.ainet.lang.memory.BlockOrder.INPUT_BLOCK_MAJOR]). */
+    BLOCKED_INPUT_MAJOR,
+    ;
+
+    /** True for either blocked class — when the question is "packed or not". */
+    public val isBlocked: Boolean get() = this == BLOCKED_ROW_MAJOR || this == BLOCKED_INPUT_MAJOR
+}
 
 /** Thrown when no registered kernel and no adapter chain can serve a key; lists what is registered. */
 @ExperimentalMemoryApi

skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Q5_0MatmulKernel.kt

@@ -26,7 +26,10 @@ package sk.ainet.backend.api.kernel
  * write the `outputDim` floats starting at `output[outputOffset]`.
  *
  * Packed-weight **block-major** row contract: `weight` holds blocks laid
- * out `(blockIdx * outputDim + o) * 22`. Matches `Q5_0BlockTensorData.packedData`.
+ * The weight is **input-block-major** (Q5_0BlockTensorData's bytes are canonical
+ * row-major — a weight reaches this kernel through `TensorView.prepack`, not by
+ * reinterpretation). One contract, written down in
+ * `docs/design/memory/packed-weight-layout.md` (#973).
  *
  * `inputDim` MUST be a multiple of 32 (the Q5_0 block size).
  */

skainet-backends/skainet-backend-api/src/commonMain/kotlin/sk/ainet/backend/api/kernel/Q6KMatmulKernel.kt

@@ -23,7 +23,10 @@ package sk.ainet.backend.api.kernel
  * write the `outputDim` floats starting at `output[outputOffset]`.
  *
  * Packed-weight **block-major** row contract: blocks laid out
- * `(blockIdx * outputDim + o) * 210`. Matches `Q6_KBlockTensorData.packedData`.
+ * The weight is **input-block-major** (Q6_KBlockTensorData's bytes are canonical
+ * row-major — a weight reaches this kernel through `TensorView.prepack`, not by
+ * reinterpretation). One contract, written down in
+ * `docs/design/memory/packed-weight-layout.md` (#973).
  *
  * `inputDim` MUST be a multiple of 256 (the Q6_K super-block size).
  */

skainet-backends/skainet-backend-api/src/commonTest/kotlin/sk/ainet/backend/api/kernel/KernelKeyDispatchTest.kt

@@ -68,8 +68,12 @@ class KernelKeyDispatchTest {
         val key = KernelKey.matmul(a, w)
         assertEquals("matmul", key.op); assertEquals(2, key.operands.size)
         assertEquals(OperandKey(Format.dense(FP32), LayoutClass.CONTIGUOUS), key.operands[0])
-        assertEquals(OperandKey(Format(FP32, TensorEncoding.Q8_0), LayoutClass.BLOCKED), key.operands[1])
-        assertEquals("matmul(Float32/Dense(4B) contiguous × Float32/Q8_0 blocked) @host", key.toString())
+        assertEquals(
+            OperandKey(Format(FP32, TensorEncoding.Q8_0), LayoutClass.BLOCKED_ROW_MAJOR),
+            key.operands[1],
+            "a weight loaded from a file is canonical, and the key says so (#973)",
+        )
+        assertEquals("matmul(Float32/Dense(4B) contiguous × Float32/Q8_0 blocked_row_major) @host", key.toString())
         // a strided operand is a different key — that is the point of keying on layout
         val strided = denseView(Shape(4, 8)) { it.toFloat() }.narrow(1, 0, 4)
         assertEquals(LayoutClass.STRIDED, OperandKey.of(strided).layout)

skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api

@@ -686,6 +686,14 @@ public final class sk/ainet/lang/memory/BlockDecoder$DefaultImpls {
 	public static fun decodeElement (Lsk/ainet/lang/memory/BlockDecoder;Lsk/ainet/lang/memory/Storage;Lsk/ainet/lang/memory/Layout;J)F
 }
 
+public final class sk/ainet/lang/memory/BlockOrder : java/lang/Enum {
+	public static final field INPUT_BLOCK_MAJOR Lsk/ainet/lang/memory/BlockOrder;
+	public static final field ROW_MAJOR Lsk/ainet/lang/memory/BlockOrder;
+	public static fun getEntries ()Lkotlin/enums/EnumEntries;
+	public static fun valueOf (Ljava/lang/String;)Lsk/ainet/lang/memory/BlockOrder;
+	public static fun values ()[Lsk/ainet/lang/memory/BlockOrder;
+}
+
 public final class sk/ainet/lang/memory/BlockSpec {
 	public static final field Companion Lsk/ainet/lang/memory/BlockSpec$Companion;
 	public static final field PER_TENSOR_BLOCK I
@@ -811,11 +819,12 @@ public final class sk/ainet/lang/memory/I8Absmax {
 
 public final class sk/ainet/lang/memory/Layout {
 	public static final field Companion Lsk/ainet/lang/memory/Layout$Companion;
-	public fun <init> (Lsk/ainet/lang/tensor/Shape;[IJIZI)V
-	public synthetic fun <init> (Lsk/ainet/lang/tensor/Shape;[IJIZIILkotlin/jvm/internal/DefaultConstructorMarker;)V
+	public fun <init> (Lsk/ainet/lang/tensor/Shape;[IJIZILsk/ainet/lang/memory/BlockOrder;)V
+	public synthetic fun <init> (Lsk/ainet/lang/tensor/Shape;[IJIZILsk/ainet/lang/memory/BlockOrder;ILkotlin/jvm/internal/DefaultConstructorMarker;)V
 	public final fun byteOffsetOf ([I)J
 	public fun equals (Ljava/lang/Object;)Z
 	public final fun getBlockAxis ()I
+	public final fun getBlockOrder ()Lsk/ainet/lang/memory/BlockOrder;
 	public final fun getBlocked ()Z
 	public final fun getElementBytes ()I
 	public final fun getElementCount ()J
@@ -837,8 +846,8 @@ public final class sk/ainet/lang/memory/Layout {
 }
 
 public final class sk/ainet/lang/memory/Layout$Companion {
-	public final fun blocked (Lsk/ainet/lang/tensor/Shape;IIJ)Lsk/ainet/lang/memory/Layout;
-	public static synthetic fun blocked$default (Lsk/ainet/lang/memory/Layout$Companion;Lsk/ainet/lang/tensor/Shape;IIJILjava/lang/Object;)Lsk/ainet/lang/memory/Layout;
+	public final fun blocked (Lsk/ainet/lang/tensor/Shape;IIJLsk/ainet/lang/memory/BlockOrder;)Lsk/ainet/lang/memory/Layout;
+	public static synthetic fun blocked$default (Lsk/ainet/lang/memory/Layout$Companion;Lsk/ainet/lang/tensor/Shape;IIJLsk/ainet/lang/memory/BlockOrder;ILjava/lang/Object;)Lsk/ainet/lang/memory/Layout;
 	public final fun rowMajor (Lsk/ainet/lang/tensor/Shape;Lsk/ainet/lang/memory/Format;J)Lsk/ainet/lang/memory/Layout;
 	public static synthetic fun rowMajor$default (Lsk/ainet/lang/memory/Layout$Companion;Lsk/ainet/lang/tensor/Shape;Lsk/ainet/lang/memory/Format;JILjava/lang/Object;)Lsk/ainet/lang/memory/Layout;
 	public final fun rowMajorStrides (Lsk/ainet/lang/tensor/Shape;)[I
@@ -1057,6 +1066,14 @@ public final class sk/ainet/lang/memory/ProcessMemorySample {
 	public fun toString ()Ljava/lang/String;
 }
 
+public final class sk/ainet/lang/memory/RelayoutedBlockDecoder : sk/ainet/lang/memory/BlockDecoder {
+	public fun <init> (Lsk/ainet/lang/memory/BlockDecoder;IILsk/ainet/lang/memory/BlockOrder;)V
+	public fun decodeBlock (Lsk/ainet/lang/memory/Storage;J[FI)V
+	public fun decodeElement (Lsk/ainet/lang/memory/Storage;Lsk/ainet/lang/memory/Layout;J)F
+	public fun getBlockSize ()I
+	public fun getBytesPerBlock ()I
+}
+
 public final class sk/ainet/lang/memory/ScalePlacement : java/lang/Enum {
 	public static final field BLOCK_HEAD Lsk/ainet/lang/memory/ScalePlacement;
 	public static final field BLOCK_TAIL Lsk/ainet/lang/memory/ScalePlacement;
@@ -1220,6 +1237,7 @@ public final class sk/ainet/lang/memory/TensorView {
 	public static final field Companion Lsk/ainet/lang/memory/TensorView$Companion;
 	public fun <init> (Lsk/ainet/lang/tensor/Shape;Lsk/ainet/lang/memory/Format;Lsk/ainet/lang/memory/Layout;Lsk/ainet/lang/memory/Storage;Lsk/ainet/lang/tensor/TensorId;Lsk/ainet/lang/memory/BlockDecoder;)V
 	public synthetic fun <init> (Lsk/ainet/lang/tensor/Shape;Lsk/ainet/lang/memory/Format;Lsk/ainet/lang/memory/Layout;Lsk/ainet/lang/memory/Storage;Lsk/ainet/lang/tensor/TensorId;Lsk/ainet/lang/memory/BlockDecoder;ILkotlin/jvm/internal/DefaultConstructorMarker;)V
+	public final fun decoderOrNull ()Lsk/ainet/lang/memory/BlockDecoder;
 	public final fun get ([I)F
 	public final fun getElementCount ()J
 	public final fun getFormat ()Lsk/ainet/lang/memory/Format;
@@ -1232,6 +1250,8 @@ public final class sk/ainet/lang/memory/TensorView {
 	public final fun materialize (Lsk/ainet/lang/memory/Format;Lsk/ainet/lang/memory/Scope;)Lsk/ainet/lang/memory/TensorView;
 	public static synthetic fun materialize$default (Lsk/ainet/lang/memory/TensorView;Lsk/ainet/lang/memory/Format;Lsk/ainet/lang/memory/Scope;ILjava/lang/Object;)Lsk/ainet/lang/memory/TensorView;
 	public final fun narrow (III)Lsk/ainet/lang/memory/TensorView;
+	public final fun prepack (Lsk/ainet/lang/memory/BlockOrder;Lsk/ainet/lang/memory/Scope;Lsk/ainet/lang/memory/trace/TraceSink;)Lsk/ainet/lang/memory/TensorView;
+	public static synthetic fun prepack$default (Lsk/ainet/lang/memory/TensorView;Lsk/ainet/lang/memory/BlockOrder;Lsk/ainet/lang/memory/Scope;Lsk/ainet/lang/memory/trace/TraceSink;ILjava/lang/Object;)Lsk/ainet/lang/memory/TensorView;
 	public final fun set ([IF)V
 	public final fun squeeze (I)Lsk/ainet/lang/memory/TensorView;
 	public final fun step (II)Lsk/ainet/lang/memory/TensorView;