feat(memory): M2 acceptance — process probe, ternary decode harness, BitNet-2B plan, release record (SKEEP-003 M2, S2.10)

docs/design/memory/m2-acceptance.md

@@ -0,0 +1,86 @@
+# SKEEP-003 M1 and M2 acceptance — what is measured, and where
+
+Milestone record for [#1042](https://github.com/SKaiNET-developers/SKaiNET/issues/1042), closing
+the M2 tracker [#1003](https://github.com/SKaiNET-developers/SKaiNET/issues/1003). Every row says
+what was checked, where the number comes from, and — where a criterion is not closed — what is
+missing. A criterion asserted by a test in this repository runs on every commit, on every target
+the test suite covers.
+
+The device numbers below come from an **ARMv8.2 Cortex-A55 reference board**: two cores,
+1.9 GB RAM, `asimddp`, Linux. It is a 2 GB-class device, which is the class M2 targets, but it is
+not Android — it has no ART, so the criteria that are about the *managed heap* cannot be closed
+there.
+
+## M1 — Flat decode
+
+| ID | Criterion | Status | Evidence |
+|---|---|---|---|
+| M1-A1 | Memory flat across decode steps | **met** | `DecodeAcceptanceTest.m1a1…` — forward scope 0 bytes between steps, one distinct per-step value after warm-up. Runs in CI and on the reference board. |
+| M1-A2 | Peak RSS during load ≤ file + KV + slab + 100 MB | open | needs a real GGUF and a load path with a model; belongs to the decode sample in SKaiNET-transformers |
+| M1-A3 | Zero forward-scope allocations per step after warm-up | **met** | `DecodeAcceptanceTest.m1a3…` — 0 FORWARD allocations between steps 4→12 |
+| M1-A4 | #993 / #991 through the registry | **met** | repro tests in #1027, dispatched by `KernelKey`, no special-casing |
+| M1-A5 | Decode tok/s and matmul benchmarks within 3 % | partial | no hot path was modified (argued per PR); a tok/s number needs the real-model sample |
+| M1-A6 | Packed matmul bit-identical for every encoding | **met** | golden parity gate (`scripts/pr-gate.sh --golden`), JVM + Kotlin/Native |
+| M1-A7 | Perfetto trace shape | **met** | `DecodeAcceptanceTest.m1a7…` — one track per scope, kernel spans by `TensorId`, live-bytes counter returning to zero |
+| M1-A8 | Plan vs actual within 10 % | **met** | `DecodeAcceptanceTest.m1a8…` — `PlanVsActual.withinTolerance`, adapter bytes 0 |
+| M1-A9 | develop green, API source-compatible | **met** | every slice passed the full gate; all new API additive and `@ExperimentalMemoryApi` |
+
+## M2 — 1.58-bit on a 2 GB board
+
+| ID | Criterion | Status | Evidence |
+|---|---|---|---|
+| M2-A1 | BitNet-2B decodes, resident ≤ 1.3 GB, RSS flat | **planned + machinery met**, model run open | `BitNet2BPlanTest` computes the checkpoint's resident total from its geometry: **1.19 GB** with a quantized KV cache at ctx 2048, **1.29 GB** with bf16. `M2AcceptanceTest` shows the machinery (ternary weights, int8 adapter, KV ring) keeps memory flat, on CI and on the reference board. Decoding an actual BitNet checkpoint needs the model stack in SKaiNET-transformers. |
+| M2-A2 | NEON `bitnet_gemv` parity 1e-5, ≥ 3× reference on Cortex-A55 | **met** (with a caveat) | parity **exact** (relative error 0 — the arithmetic is integer until the block scale). On the reference board, k=1024 n=256: reference 21.3 ms → NEON 0.091 ms. That fallback number came from a *debug* Kotlin/Native build, so treat the multiple as an upper bound; against compiler-vectorized C the intrinsics buy 1.08–1.24×. |
+| M2-A3 | Page-fault rate per decode step after warm-up ≈ 0 | **met** | `M2AcceptanceTest.m2a3…` — read from `/proc/self` by `MemoryProbe`. On the reference board: **0 major faults** across 12 decode steps, RSS **14 MB before and 14 MB after**, and **0 bytes** of growth at both 4 and 48 steps. |
+| M2-A4 | Ring wrap-around gives identical logits | **met** | `WindowedKvTest` — a ring that wrapped several times and a cache that never wrapped produce bit-identical output over the same window; the test first asserts the window really wrapped |
+| M2-A5 | Android mmap closes #921/#922: Llama-1B Q4_K_M on a 2 GB device | open | the configuration exists (`AndroidGguf.loader()`, `staging = MAPPED`) and the fit check answers before loading, but packed weights still reach the managed heap: the packed matmul SPI takes `ByteArray`s, and a buffer-aware kernel needs the byte-order contract of #973 settled. Also needs an Android device — the reference board has no ART. |
+| M2-A6 | All M1 criteria still pass | **met** | the M1 suite runs unchanged in CI and passed on the reference board alongside the M2 suite (15 tests, all green) |
+
+## What the reference board measured
+
+Running the M1 and M2 suites from the Kotlin/Native binary on the Cortex-A55 board — 15 tests, all
+green — the ternary decode harness reported:
+
+```
+[m2] steps=12  before: rss=14 MB majflt=0 minflt=4813   after: rss=14 MB majflt=0 minflt=5037
+[m2] rss growth: 4 steps → 0 bytes, 48 steps → 0 bytes
+```
+
+Zero major faults: nothing went to disk during steady-state decode. The minor faults are page-cache
+touches, and the resident set is the same after forty-eight steps as after four — which is the
+property M1-A1 states in allocation events and M2-A3 states in the kernel's own numbers, now
+observed on the target class of device rather than inferred.
+
+## The numbers behind M2-A1
+
+Computed by the planner from BitNet-b1.58-2B-4T's geometry — 30 layers, 2560 hidden, 6912 FFN,
+20 heads / 5 KV heads, 128 256 vocab — the same way `skainet-plan` computes them from a GGUF header:
+
+| part | bytes |
+|---|---:|
+| ternary linear weights (TQ2_0, 2.0625 bits/element) | 512 MB |
+| token embedding table (bf16, output head tied) | 657 MB |
+| KV cache @ ctx 2048, bf16 | 150 MB |
+| KV cache @ ctx 2048, TurboQuant-4 | 44 MB |
+| **resident, quantized cache** | **1.19 GB** |
+| **resident, bf16 cache** | **1.29 GB** |
+
+Two things worth saying plainly:
+
+- **The embedding table outweighs the ternary stack.** 657 MB of bf16 embeddings against 512 MB of
+  1.58-bit weights: past a certain point a "2-bit model" is an embedding-table problem. The
+  planner says so before anything is loaded, which is the point of M0.
+- **A 2 GB device does not hold this model.** With the mobile profile's 700 MB reserve, 1.5 GB free
+  leaves 800 MB, and 1.19 GB does not fit in 800 MB however it is staged. `BitNet2BPlanTest`
+  asserts the refusal, and that a 4 GB device does hold it with mapped weights. M2's title is
+  aspirational for *this* checkpoint; the machinery it names is in place.
+
+## What closing M2 still needs
+
+1. **#973** — the packed-quant byte-order contract. Until it is settled, packed weights cannot be
+   handed to a kernel as a mapped view, which is what M2-A5 and the packed half of M2-A1 wait on.
+2. **The decode sample in SKaiNET-transformers** — a real checkpoint, a tokenizer and a generation
+   loop, which this repository deliberately does not have. M1-A2, M1-A5's tok/s and M2-A1's
+   measured run belong there.
+3. **An Android device** for the ART-heap criteria; the reference board answers the RAM and
+   page-fault questions but not the managed-heap ones.

skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt

@@ -0,0 +1,139 @@
+package sk.ainet.exec.harness
+
+import sk.ainet.lang.memory.ExperimentalMemoryApi
+import sk.ainet.lang.memory.I8Absmax
+import sk.ainet.lang.memory.MemoryProbe
+import sk.ainet.lang.memory.ScopeKind
+import sk.ainet.lang.memory.trace.TraceEvent
+import kotlin.test.Test
+import kotlin.test.assertEquals
+import kotlin.test.assertTrue
+
+/**
+ * Milestone M2's memory criteria, asserted on every commit against the ternary decode harness
+ * (#1042): ternary weights, an int8 activation adapter, a KV ring that wraps, and — where the
+ * platform can answer — the process's own resident set and page-fault counters.
+ *
+ * The shape is small so this runs in a browser too; the same assertions hold at a larger shape on
+ * the reference device, where the numbers for the release table were taken. What this cannot
+ * assert is a *model*: BitNet-2B's resident total is the planner's answer (M2-A1) and the decode
+ * sample's measurement, not something a synthetic harness can claim.
+ */
+@OptIn(ExperimentalMemoryApi::class)
+class M2AcceptanceTest {
+
+    private val steps = 12
+
+    @Test
+    fun m2a1_memoryIsFlatAcrossDecodeStepsWithTernaryWeights() {
+        val h = TernaryDecodeHarness()
+        try {
+            h.decode(steps)
+            val live = h.liveBytes()
+            assertEquals(0L, live[ScopeKind.FORWARD] ?: 0L, "the forward scope is empty between steps")
+            assertTrue((live[ScopeKind.MODEL] ?: 0L) > 0, "weights and the KV ring stay resident")
+
+            val resets = h.sink.eventsOf<TraceEvent.ScopeReset>().filter { it.scope == ScopeKind.FORWARD }
+            assertEquals(steps + 1, resets.size, "one reset per step, plus the warm-up step")
+            assertTrue(resets.all { it.liveBytesAfter == 0L }, "every reset returns the slab to zero")
+            val perStep = resets.map { it.liveBytesBefore }.drop(1).distinct()
+            assertEquals(1, perStep.size, "steady-state forward use must be identical every step, saw $perStep")
+        } finally {
+            h.close()
+        }
+    }
+
+    @Test
+    fun m2a1_theTernaryWeightsCostWhatTheEncodingSays() {
+        val h = TernaryDecodeHarness()
+        try {
+            // 2.0625 bits per element: 66 bytes per 256-element block, no more
+            val elements = 4L * 256 * 256      // two projections per layer, two layers
+            assertEquals(elements / 256 * 66, h.weightBytes, "TQ2_0 weights are 66 bytes per 256 elements")
+            val allocated = h.sink.eventsOf<TraceEvent.Allocation>()
+                .filter { it.scope == ScopeKind.MODEL && it.site == "adopted" }
+                .sumOf { it.bytes }
+            assertEquals(h.weightBytes, allocated, "and that is exactly what the model scope reports")
+        } finally {
+            h.close()
+        }
+    }
+
+    @Test
+    fun m2f3_everyStepPaysForOneActivationAdapterAndNothingElse() {
+        val h = TernaryDecodeHarness()
+        try {
+            h.decode(steps)
+            for (step in 2..steps) {
+                val adapters = h.adaptersInStep(step)
+                assertTrue(
+                    adapters.all { it.kind == "requantize-i8-absmax" },
+                    "step $step should only requantize activations, saw ${adapters.map { it.kind }}",
+                )
+                assertEquals(4, adapters.size, "one adapter per ternary matmul (two per layer, two layers)")
+                assertEquals(
+                    I8Absmax.bytesFor(rows = 1, cols = 256), adapters.first().bytes,
+                    "the adapter costs the codes plus one scale — the §5.3 number",
+                )
+            }
+        } finally {
+            h.close()
+        }
+    }
+
+    @Test
+    fun m2a3_steadyStateDecodeDoesNotFaultToDisk() {
+        val h = TernaryDecodeHarness()
+        try {
+            val (before, after) = h.decode(steps)
+            println("[m2] steps=$steps weights=${h.weightBytes} bytes  before: $before  after: $after")
+            val faults = after.majorFaultsSince(before)
+            if (faults == null) {
+                // a browser or Wasm host cannot answer; the structural assertions above still hold
+                assertTrue(before.rssBytes == null, "a platform that knows its RSS should know its faults too")
+                return
+            }
+            assertEquals(0L, faults, "after warm-up a decode step must not go to disk (before=$before after=$after)")
+        } finally {
+            h.close()
+        }
+    }
+
+    @Test
+    fun m2a3_theResidentSetDoesNotGrowWithSteps() {
+        val short = TernaryDecodeHarness()
+        val long = TernaryDecodeHarness()
+        try {
+            val (beforeShort, afterShort) = short.decode(4)
+            val (beforeLong, afterLong) = long.decode(4 * steps)
+            val rssShort = afterShort.rssBytes?.minus(beforeShort.rssBytes ?: 0)
+            val rssLong = afterLong.rssBytes?.minus(beforeLong.rssBytes ?: 0)
+            println("[m2] rss growth: 4 steps → $rssShort bytes, ${4 * steps} steps → $rssLong bytes")
+            if (rssShort == null || rssLong == null) return       // platform cannot answer
+
+            // Twelve times the steps must not mean twelve times the memory: the forward slab is
+            // recycled and the KV ring wraps, so growth is bounded by GC noise, not by step count.
+            val slack = 32L * 1024 * 1024
+            assertTrue(
+                rssLong <= rssShort + slack,
+                "RSS grew with the number of steps: 4 steps → $rssShort bytes, ${4 * steps} steps → $rssLong bytes",
+            )
+        } finally {
+            short.close()
+            long.close()
+        }
+    }
+
+    @Test
+    fun m2a4_theKvRingWrapsWithinTheRun() {
+        val h = TernaryDecodeHarness(ctx = 8)
+        try {
+            h.decode(20)      // more steps than the ring holds
+            assertTrue(h.liveBytes()[ScopeKind.MODEL]!! > 0)
+            // the ring's own parity is asserted in WindowedKvTest; here it just has to keep working
+            assertEquals(0L, h.liveBytes()[ScopeKind.FORWARD] ?: 0L)
+        } finally {
+            h.close()
+        }
+    }
+}

skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/TernaryDecodeHarness.kt

@@ -0,0 +1,145 @@
+package sk.ainet.exec.harness
+
+import sk.ainet.backend.api.kernel.KernelDispatch
+import sk.ainet.backend.api.kernel.TernaryKernelPacks
+import sk.ainet.lang.memory.ExperimentalMemoryApi
+import sk.ainet.lang.memory.Format
+import sk.ainet.lang.memory.ForwardScope
+import sk.ainet.lang.memory.I8Absmax
+import sk.ainet.lang.memory.MemoryProbe
+import sk.ainet.lang.memory.ModelScope
+import sk.ainet.lang.memory.ProcessMemorySample
+import sk.ainet.lang.memory.ScopeKind
+import sk.ainet.lang.memory.Storage
+import sk.ainet.lang.memory.TensorView
+import sk.ainet.lang.memory.TernaryBlockDecoder
+import sk.ainet.lang.memory.TernaryCodec
+import sk.ainet.lang.memory.sample
+import sk.ainet.lang.memory.trace.RecordingTraceSink
+import sk.ainet.lang.memory.trace.TraceEvent
+import sk.ainet.lang.memory.trace.decodeStep
+import sk.ainet.lang.tensor.Shape
+import sk.ainet.lang.tensor.TensorId
+import sk.ainet.lang.tensor.storage.DefaultKvCacheStore
+import sk.ainet.lang.tensor.storage.KvCacheConfig
+import sk.ainet.lang.tensor.storage.TensorEncoding
+import sk.ainet.lang.types.FP32
+
+/**
+ * The M2 shape of the M1 decode harness (#1042): **ternary** weights, an int8 activation adapter,
+ * a sliding-window KV ring, and the process-level counters the milestone is judged on.
+ *
+ * Everything here is the real machinery — `TernaryCodec` weights in a [ModelScope],
+ * `I8Absmax.requantize` into the [ForwardScope] every step, dispatch through [KernelDispatch] onto
+ * `bitnet_gemv`, a KV ring that wraps — so the assertions are about what SKaiNET actually does,
+ * not about a mock. It is still not a model: no tokenizer, no checkpoint, no sampling. The
+ * BitNet-2B numbers belong to the decode sample in SKaiNET-transformers; what belongs here is that
+ * the memory behaviour holds, on every target, including a 2 GB ARM board.
+ */
+@OptIn(ExperimentalMemoryApi::class)
+public class TernaryDecodeHarness(
+    public val layers: Int = 2,
+    public val hidden: Int = 256,
+    public val ctx: Int = 32,
+    public val kvHeads: Int = 2,
+    public val heads: Int = 4,
+) {
+    public val sink: RecordingTraceSink = RecordingTraceSink()
+    private val model = ModelScope(sink, "m2-harness")
+
+    init {
+        require(hidden % 256 == 0) { "the ternary kernel works in TQ2_0 blocks of 256; hidden=$hidden" }
+        KernelDispatch.clearForTesting()
+        TernaryKernelPacks.install(native = null)     // the portable kernel: what a device without the pack runs
+    }
+
+    /** Per-layer ternary weights, TQ2_0, resident in the model scope. */
+    private val weights: List<TensorView> = buildList {
+        for (l in 0 until layers) {
+            add(ternaryWeight(hidden, hidden, TensorId(listOf("model", "layers[$l]", "attn"), "q_proj.weight")))
+            add(ternaryWeight(hidden, hidden, TensorId(listOf("model", "layers[$l]", "mlp"), "up_proj.weight")))
+        }
+    }
+
+    private val kv = DefaultKvCacheStore(
+        KvCacheConfig(numLayers = layers, numHeads = kvHeads, headDim = hidden / heads, maxSeqLen = ctx),
+        model,
+        slidingWindow = true,                          // #1036: the ring, so a long run does not grow
+    )
+
+    private val forward = ForwardScope(slabFloats = 4 * hidden + 64, sink = sink, name = "m2-decode")
+
+    /** Bytes of ternary weight resident in the model scope. */
+    public val weightBytes: Long = weights.sumOf { it.format.physicalBytes(it.elementCount) ?: 0L }
+
+    private fun ternaryWeight(rows: Int, cols: Int, id: TensorId): TensorView {
+        var seed = id.canonical.hashCode()
+        val values = FloatArray(rows * cols) {
+            seed = seed * 1103515245 + 12345
+            ((seed ushr 16) % 3 - 1) * 0.5f
+        }
+        val bytes = TernaryCodec.encode(TensorEncoding.TQ2_0, values)
+        val storage = model.adopt(Storage.Heap.wrap(bytes, mutable = false, origin = id, sink = sink))
+        return TensorView.packed(
+            storage, Shape(rows, cols), TensorEncoding.TQ2_0,
+            TernaryBlockDecoder(TensorEncoding.TQ2_0), id = id,
+        )
+    }
+
+    /** Run [steps] decode steps; returns the process counters before and after the timed region. */
+    public fun decode(steps: Int): Pair<ProcessMemorySample, ProcessMemorySample> {
+        val activation = FloatArray(hidden) { (it % 13) * 0.0625f }
+        val k = FloatArray(kvHeads * (hidden / heads)) { 0.25f }
+        // warm up outside the measured window: first-touch faults are not steady-state behaviour
+        step(0, activation, k)
+        val before = MemoryProbe.sample()
+        for (s in 1..steps) step(s, activation, k)
+        val after = MemoryProbe.sample()
+        return before to after
+    }
+
+    private fun step(step: Int, activation: FloatArray, k: FloatArray) {
+        sink.decodeStep(step) {
+            val slab = forward.allocateFloats(hidden, TensorId(listOf("model"), "hidden", "step=$step"))
+            activation.copyInto(slab.floats!!, slab.arrayOffset)
+            val dense = TensorView.dense(slab, Shape(1, hidden), FP32, TensorId(listOf("model"), "hidden", "step=$step"))
+            for (w in weights) {
+                val out = forward.allocateFloats(w.shape[0], TensorId(listOf("model"), "proj", "step=$step"))
+                val outView = TensorView.dense(out, Shape(1, w.shape[0]), FP32)
+                // the dispatcher requantizes the activation into the forward scope and picks bitnet_gemv
+                KernelDispatch.matmul(dense, w, outView, forward, sink)
+            }
+            for (l in 0 until layers) kv.appendToken(l, k, k)
+            forward.reset()
+        }
+    }
+
+    /** Live bytes per scope as the event stream saw them. */
+    public fun liveBytes(): Map<ScopeKind, Long> {
+        val live = HashMap<ScopeKind, Long>()
+        for (e in sink.events()) when (e) {
+            is TraceEvent.Allocation -> live[e.scope] = (live[e.scope] ?: 0L) + e.bytes
+            is TraceEvent.Free -> live[e.scope] = ((live[e.scope] ?: 0L) - e.bytes).coerceAtLeast(0L)
+            is TraceEvent.ScopeReset -> live[e.scope] = e.liveBytesAfter
+            else -> Unit
+        }
+        return live
+    }
+
+    /** Adapter events recorded during step [step]. */
+    public fun adaptersInStep(step: Int): List<TraceEvent.AdapterInserted> {
+        val out = ArrayList<TraceEvent.AdapterInserted>()
+        var current = 0
+        for (e in sink.events()) {
+            if (e is TraceEvent.PhaseBegin && e.phase == "decode") current = e.step ?: current
+            if (e is TraceEvent.AdapterInserted && current == step) out += e
+        }
+        return out
+    }
+
+    public fun close() {
+        forward.close()
+        model.close()
+        KernelDispatch.clearForTesting()
+    }
+}