From 6a875ff6b815766ffeccb9de76cf137a14f1f808 Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Mon, 24 Aug 2026 17:09:07 +0200 Subject: [PATCH] =?UTF-8?q?feat(memory):=20M2=20acceptance=20=E2=80=94=20p?= =?UTF-8?q?rocess-level=20probe,=20ternary=20decode=20harness,=20BitNet-2B?= =?UTF-8?q?=20plan,=20release=20record?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Closes #1042 (SKEEP-003 M2, S2.10; M2-A1, M2-A3, M2-A6). The milestone's criteria are about what the *operating system* sees, and nothing in the repository could see it. This adds the probe, the harness that exercises the M2 machinery, the arithmetic for the model M2 is named after, and the record of what is and is not closed. - `MemoryProbe` + `ProcessMemorySample`: RSS and major/minor page faults from `/proc/self` on JVM, Android and Kotlin/Native-on-Linux; `null` where the platform cannot answer (a browser has no /proc, and callers print "—" rather than guessing). Emits the `Counters.RSS` and `Counters.PAGE_FAULTS` that #1035 declared and nothing produced. - `TernaryDecodeHarness`: the M1 harness in M2 shape — TQ2_0 weights in a ModelScope, `I8Absmax` requantization into the Forward scope every step, dispatch onto `bitnet_gemv`, a KV ring that wraps. - `M2AcceptanceTest`: memory flat across steps with ternary weights, the weights costing exactly 66 bytes per 256 elements, one activation adapter per matmul and nothing else, zero major faults in steady state, and a resident set that does not grow with the step count. - `BitNet2BPlanTest`: M2-A1's arithmetic from BitNet-b1.58-2B-4T's geometry — 1.19 GB resident with a quantized cache, 1.29 GB with bf16, both under 1.3 GB; the margin is 148 MB against 42 MB, and at ctx 8192 only the quantized cache still fits, which is why the mobile profile quantizes it automatically. - `docs/design/memory/m2-acceptance.md`: the M1 and M2 tables side by side, each row saying met / partial / open and where the number came from. Measured on an ARMv8.2 Cortex-A55 reference board (2 cores, 1.9 GB RAM), running the M1 and M2 suites from the Kotlin/Native binary — 15 tests, all green: [m2] steps=12 before: rss=14 MB majflt=0 after: rss=14 MB majflt=0 [m2] rss growth: 4 steps → 0 bytes, 48 steps → 0 bytes Two findings the record states plainly rather than burying. BitNet-2B's bf16 embedding table (657 MB) outweighs its entire ternary stack (512 MB), so past a point a "2-bit model" is an embedding-table problem. And a 2 GB device does *not* hold this checkpoint: after the mobile profile's 700 MB reserve, 1.5 GB free leaves 800 MB, and 1.19 GB does not fit in 800 MB however it is staged — the planner refuses before loading, and the test asserts both the refusal and that a 4 GB device does hold it. M2-A5 and the measured half of M2-A1 stay open, for reasons named in the record: packed weights still reach the managed heap until #973 settles the byte-order contract, and decoding a real checkpoint needs the model stack in SKaiNET-transformers. Gate: scripts/pr-gate.sh — all legs passed. Co-Authored-By: Claude Fable 5 --- docs/design/memory/m2-acceptance.md | 86 +++++++++ .../sk/ainet/exec/harness/M2AcceptanceTest.kt | 139 ++++++++++++++ .../exec/harness/TernaryDecodeHarness.kt | 145 +++++++++++++++ .../api/jvm/skainet-lang-core.api | 28 +++ .../sk/ainet/lang/memory/MemoryProbe.kt | 61 +++++++ .../sk/ainet/lang/memory/MemoryProbeTest.kt | 68 +++++++ .../lang/memory/plan/BitNet2BPlanTest.kt | 170 ++++++++++++++++++ .../sk/ainet/lang/memory/MemoryProbe.js.kt | 9 + .../lang/memory/MemoryProbe.jvmAndroid.kt | 44 +++++ .../ainet/lang/memory/MemoryProbe.native.kt | 46 +++++ .../ainet/lang/memory/MemoryProbe.wasmJs.kt | 9 + .../ainet/lang/memory/MemoryProbe.wasmWasi.kt | 9 + 12 files changed, 814 insertions(+) create mode 100644 docs/design/memory/m2-acceptance.md create mode 100644 skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt create mode 100644 skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/TernaryDecodeHarness.kt create mode 100644 skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/MemoryProbe.kt create mode 100644 skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/MemoryProbeTest.kt create mode 100644 skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/plan/BitNet2BPlanTest.kt create mode 100644 skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.js.kt create mode 100644 skainet-lang/skainet-lang-core/src/jvmAndroidMain/kotlin/sk/ainet/lang/memory/MemoryProbe.jvmAndroid.kt create mode 100644 skainet-lang/skainet-lang-core/src/nativeMain/kotlin/sk/ainet/lang/memory/MemoryProbe.native.kt create mode 100644 skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmJs.kt create mode 100644 skainet-lang/skainet-lang-core/src/wasmWasiMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmWasi.kt diff --git a/docs/design/memory/m2-acceptance.md b/docs/design/memory/m2-acceptance.md new file mode 100644 index 000000000..6bcf89889 --- /dev/null +++ b/docs/design/memory/m2-acceptance.md @@ -0,0 +1,86 @@ +# SKEEP-003 M1 and M2 acceptance — what is measured, and where + +Milestone record for [#1042](https://github.com/SKaiNET-developers/SKaiNET/issues/1042), closing +the M2 tracker [#1003](https://github.com/SKaiNET-developers/SKaiNET/issues/1003). Every row says +what was checked, where the number comes from, and — where a criterion is not closed — what is +missing. A criterion asserted by a test in this repository runs on every commit, on every target +the test suite covers. + +The device numbers below come from an **ARMv8.2 Cortex-A55 reference board**: two cores, +1.9 GB RAM, `asimddp`, Linux. It is a 2 GB-class device, which is the class M2 targets, but it is +not Android — it has no ART, so the criteria that are about the *managed heap* cannot be closed +there. + +## M1 — Flat decode + +| ID | Criterion | Status | Evidence | +|---|---|---|---| +| M1-A1 | Memory flat across decode steps | **met** | `DecodeAcceptanceTest.m1a1…` — forward scope 0 bytes between steps, one distinct per-step value after warm-up. Runs in CI and on the reference board. | +| M1-A2 | Peak RSS during load ≤ file + KV + slab + 100 MB | open | needs a real GGUF and a load path with a model; belongs to the decode sample in SKaiNET-transformers | +| M1-A3 | Zero forward-scope allocations per step after warm-up | **met** | `DecodeAcceptanceTest.m1a3…` — 0 FORWARD allocations between steps 4→12 | +| M1-A4 | #993 / #991 through the registry | **met** | repro tests in #1027, dispatched by `KernelKey`, no special-casing | +| M1-A5 | Decode tok/s and matmul benchmarks within 3 % | partial | no hot path was modified (argued per PR); a tok/s number needs the real-model sample | +| M1-A6 | Packed matmul bit-identical for every encoding | **met** | golden parity gate (`scripts/pr-gate.sh --golden`), JVM + Kotlin/Native | +| M1-A7 | Perfetto trace shape | **met** | `DecodeAcceptanceTest.m1a7…` — one track per scope, kernel spans by `TensorId`, live-bytes counter returning to zero | +| M1-A8 | Plan vs actual within 10 % | **met** | `DecodeAcceptanceTest.m1a8…` — `PlanVsActual.withinTolerance`, adapter bytes 0 | +| M1-A9 | develop green, API source-compatible | **met** | every slice passed the full gate; all new API additive and `@ExperimentalMemoryApi` | + +## M2 — 1.58-bit on a 2 GB board + +| ID | Criterion | Status | Evidence | +|---|---|---|---| +| M2-A1 | BitNet-2B decodes, resident ≤ 1.3 GB, RSS flat | **planned + machinery met**, model run open | `BitNet2BPlanTest` computes the checkpoint's resident total from its geometry: **1.19 GB** with a quantized KV cache at ctx 2048, **1.29 GB** with bf16. `M2AcceptanceTest` shows the machinery (ternary weights, int8 adapter, KV ring) keeps memory flat, on CI and on the reference board. Decoding an actual BitNet checkpoint needs the model stack in SKaiNET-transformers. | +| M2-A2 | NEON `bitnet_gemv` parity 1e-5, ≥ 3× reference on Cortex-A55 | **met** (with a caveat) | parity **exact** (relative error 0 — the arithmetic is integer until the block scale). On the reference board, k=1024 n=256: reference 21.3 ms → NEON 0.091 ms. That fallback number came from a *debug* Kotlin/Native build, so treat the multiple as an upper bound; against compiler-vectorized C the intrinsics buy 1.08–1.24×. | +| M2-A3 | Page-fault rate per decode step after warm-up ≈ 0 | **met** | `M2AcceptanceTest.m2a3…` — read from `/proc/self` by `MemoryProbe`. On the reference board: **0 major faults** across 12 decode steps, RSS **14 MB before and 14 MB after**, and **0 bytes** of growth at both 4 and 48 steps. | +| M2-A4 | Ring wrap-around gives identical logits | **met** | `WindowedKvTest` — a ring that wrapped several times and a cache that never wrapped produce bit-identical output over the same window; the test first asserts the window really wrapped | +| M2-A5 | Android mmap closes #921/#922: Llama-1B Q4_K_M on a 2 GB device | open | the configuration exists (`AndroidGguf.loader()`, `staging = MAPPED`) and the fit check answers before loading, but packed weights still reach the managed heap: the packed matmul SPI takes `ByteArray`s, and a buffer-aware kernel needs the byte-order contract of #973 settled. Also needs an Android device — the reference board has no ART. | +| M2-A6 | All M1 criteria still pass | **met** | the M1 suite runs unchanged in CI and passed on the reference board alongside the M2 suite (15 tests, all green) | + +## What the reference board measured + +Running the M1 and M2 suites from the Kotlin/Native binary on the Cortex-A55 board — 15 tests, all +green — the ternary decode harness reported: + +``` +[m2] steps=12 before: rss=14 MB majflt=0 minflt=4813 after: rss=14 MB majflt=0 minflt=5037 +[m2] rss growth: 4 steps → 0 bytes, 48 steps → 0 bytes +``` + +Zero major faults: nothing went to disk during steady-state decode. The minor faults are page-cache +touches, and the resident set is the same after forty-eight steps as after four — which is the +property M1-A1 states in allocation events and M2-A3 states in the kernel's own numbers, now +observed on the target class of device rather than inferred. + +## The numbers behind M2-A1 + +Computed by the planner from BitNet-b1.58-2B-4T's geometry — 30 layers, 2560 hidden, 6912 FFN, +20 heads / 5 KV heads, 128 256 vocab — the same way `skainet-plan` computes them from a GGUF header: + +| part | bytes | +|---|---:| +| ternary linear weights (TQ2_0, 2.0625 bits/element) | 512 MB | +| token embedding table (bf16, output head tied) | 657 MB | +| KV cache @ ctx 2048, bf16 | 150 MB | +| KV cache @ ctx 2048, TurboQuant-4 | 44 MB | +| **resident, quantized cache** | **1.19 GB** | +| **resident, bf16 cache** | **1.29 GB** | + +Two things worth saying plainly: + +- **The embedding table outweighs the ternary stack.** 657 MB of bf16 embeddings against 512 MB of + 1.58-bit weights: past a certain point a "2-bit model" is an embedding-table problem. The + planner says so before anything is loaded, which is the point of M0. +- **A 2 GB device does not hold this model.** With the mobile profile's 700 MB reserve, 1.5 GB free + leaves 800 MB, and 1.19 GB does not fit in 800 MB however it is staged. `BitNet2BPlanTest` + asserts the refusal, and that a 4 GB device does hold it with mapped weights. M2's title is + aspirational for *this* checkpoint; the machinery it names is in place. + +## What closing M2 still needs + +1. **#973** — the packed-quant byte-order contract. Until it is settled, packed weights cannot be + handed to a kernel as a mapped view, which is what M2-A5 and the packed half of M2-A1 wait on. +2. **The decode sample in SKaiNET-transformers** — a real checkpoint, a tokenizer and a generation + loop, which this repository deliberately does not have. M1-A2, M1-A5's tok/s and M2-A1's + measured run belong there. +3. **An Android device** for the ART-heap criteria; the reference board answers the RAM and + page-fault questions but not the managed-heap ones. diff --git a/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt b/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt new file mode 100644 index 000000000..f83e09f6b --- /dev/null +++ b/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/M2AcceptanceTest.kt @@ -0,0 +1,139 @@ +package sk.ainet.exec.harness + +import sk.ainet.lang.memory.ExperimentalMemoryApi +import sk.ainet.lang.memory.I8Absmax +import sk.ainet.lang.memory.MemoryProbe +import sk.ainet.lang.memory.ScopeKind +import sk.ainet.lang.memory.trace.TraceEvent +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * Milestone M2's memory criteria, asserted on every commit against the ternary decode harness + * (#1042): ternary weights, an int8 activation adapter, a KV ring that wraps, and — where the + * platform can answer — the process's own resident set and page-fault counters. + * + * The shape is small so this runs in a browser too; the same assertions hold at a larger shape on + * the reference device, where the numbers for the release table were taken. What this cannot + * assert is a *model*: BitNet-2B's resident total is the planner's answer (M2-A1) and the decode + * sample's measurement, not something a synthetic harness can claim. + */ +@OptIn(ExperimentalMemoryApi::class) +class M2AcceptanceTest { + + private val steps = 12 + + @Test + fun m2a1_memoryIsFlatAcrossDecodeStepsWithTernaryWeights() { + val h = TernaryDecodeHarness() + try { + h.decode(steps) + val live = h.liveBytes() + assertEquals(0L, live[ScopeKind.FORWARD] ?: 0L, "the forward scope is empty between steps") + assertTrue((live[ScopeKind.MODEL] ?: 0L) > 0, "weights and the KV ring stay resident") + + val resets = h.sink.eventsOf().filter { it.scope == ScopeKind.FORWARD } + assertEquals(steps + 1, resets.size, "one reset per step, plus the warm-up step") + assertTrue(resets.all { it.liveBytesAfter == 0L }, "every reset returns the slab to zero") + val perStep = resets.map { it.liveBytesBefore }.drop(1).distinct() + assertEquals(1, perStep.size, "steady-state forward use must be identical every step, saw $perStep") + } finally { + h.close() + } + } + + @Test + fun m2a1_theTernaryWeightsCostWhatTheEncodingSays() { + val h = TernaryDecodeHarness() + try { + // 2.0625 bits per element: 66 bytes per 256-element block, no more + val elements = 4L * 256 * 256 // two projections per layer, two layers + assertEquals(elements / 256 * 66, h.weightBytes, "TQ2_0 weights are 66 bytes per 256 elements") + val allocated = h.sink.eventsOf() + .filter { it.scope == ScopeKind.MODEL && it.site == "adopted" } + .sumOf { it.bytes } + assertEquals(h.weightBytes, allocated, "and that is exactly what the model scope reports") + } finally { + h.close() + } + } + + @Test + fun m2f3_everyStepPaysForOneActivationAdapterAndNothingElse() { + val h = TernaryDecodeHarness() + try { + h.decode(steps) + for (step in 2..steps) { + val adapters = h.adaptersInStep(step) + assertTrue( + adapters.all { it.kind == "requantize-i8-absmax" }, + "step $step should only requantize activations, saw ${adapters.map { it.kind }}", + ) + assertEquals(4, adapters.size, "one adapter per ternary matmul (two per layer, two layers)") + assertEquals( + I8Absmax.bytesFor(rows = 1, cols = 256), adapters.first().bytes, + "the adapter costs the codes plus one scale — the §5.3 number", + ) + } + } finally { + h.close() + } + } + + @Test + fun m2a3_steadyStateDecodeDoesNotFaultToDisk() { + val h = TernaryDecodeHarness() + try { + val (before, after) = h.decode(steps) + println("[m2] steps=$steps weights=${h.weightBytes} bytes before: $before after: $after") + val faults = after.majorFaultsSince(before) + if (faults == null) { + // a browser or Wasm host cannot answer; the structural assertions above still hold + assertTrue(before.rssBytes == null, "a platform that knows its RSS should know its faults too") + return + } + assertEquals(0L, faults, "after warm-up a decode step must not go to disk (before=$before after=$after)") + } finally { + h.close() + } + } + + @Test + fun m2a3_theResidentSetDoesNotGrowWithSteps() { + val short = TernaryDecodeHarness() + val long = TernaryDecodeHarness() + try { + val (beforeShort, afterShort) = short.decode(4) + val (beforeLong, afterLong) = long.decode(4 * steps) + val rssShort = afterShort.rssBytes?.minus(beforeShort.rssBytes ?: 0) + val rssLong = afterLong.rssBytes?.minus(beforeLong.rssBytes ?: 0) + println("[m2] rss growth: 4 steps → $rssShort bytes, ${4 * steps} steps → $rssLong bytes") + if (rssShort == null || rssLong == null) return // platform cannot answer + + // Twelve times the steps must not mean twelve times the memory: the forward slab is + // recycled and the KV ring wraps, so growth is bounded by GC noise, not by step count. + val slack = 32L * 1024 * 1024 + assertTrue( + rssLong <= rssShort + slack, + "RSS grew with the number of steps: 4 steps → $rssShort bytes, ${4 * steps} steps → $rssLong bytes", + ) + } finally { + short.close() + long.close() + } + } + + @Test + fun m2a4_theKvRingWrapsWithinTheRun() { + val h = TernaryDecodeHarness(ctx = 8) + try { + h.decode(20) // more steps than the ring holds + assertTrue(h.liveBytes()[ScopeKind.MODEL]!! > 0) + // the ring's own parity is asserted in WindowedKvTest; here it just has to keep working + assertEquals(0L, h.liveBytes()[ScopeKind.FORWARD] ?: 0L) + } finally { + h.close() + } + } +} diff --git a/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/TernaryDecodeHarness.kt b/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/TernaryDecodeHarness.kt new file mode 100644 index 000000000..6ed7796b7 --- /dev/null +++ b/skainet-backends/skainet-backend-cpu/src/commonTest/kotlin/sk/ainet/exec/harness/TernaryDecodeHarness.kt @@ -0,0 +1,145 @@ +package sk.ainet.exec.harness + +import sk.ainet.backend.api.kernel.KernelDispatch +import sk.ainet.backend.api.kernel.TernaryKernelPacks +import sk.ainet.lang.memory.ExperimentalMemoryApi +import sk.ainet.lang.memory.Format +import sk.ainet.lang.memory.ForwardScope +import sk.ainet.lang.memory.I8Absmax +import sk.ainet.lang.memory.MemoryProbe +import sk.ainet.lang.memory.ModelScope +import sk.ainet.lang.memory.ProcessMemorySample +import sk.ainet.lang.memory.ScopeKind +import sk.ainet.lang.memory.Storage +import sk.ainet.lang.memory.TensorView +import sk.ainet.lang.memory.TernaryBlockDecoder +import sk.ainet.lang.memory.TernaryCodec +import sk.ainet.lang.memory.sample +import sk.ainet.lang.memory.trace.RecordingTraceSink +import sk.ainet.lang.memory.trace.TraceEvent +import sk.ainet.lang.memory.trace.decodeStep +import sk.ainet.lang.tensor.Shape +import sk.ainet.lang.tensor.TensorId +import sk.ainet.lang.tensor.storage.DefaultKvCacheStore +import sk.ainet.lang.tensor.storage.KvCacheConfig +import sk.ainet.lang.tensor.storage.TensorEncoding +import sk.ainet.lang.types.FP32 + +/** + * The M2 shape of the M1 decode harness (#1042): **ternary** weights, an int8 activation adapter, + * a sliding-window KV ring, and the process-level counters the milestone is judged on. + * + * Everything here is the real machinery — `TernaryCodec` weights in a [ModelScope], + * `I8Absmax.requantize` into the [ForwardScope] every step, dispatch through [KernelDispatch] onto + * `bitnet_gemv`, a KV ring that wraps — so the assertions are about what SKaiNET actually does, + * not about a mock. It is still not a model: no tokenizer, no checkpoint, no sampling. The + * BitNet-2B numbers belong to the decode sample in SKaiNET-transformers; what belongs here is that + * the memory behaviour holds, on every target, including a 2 GB ARM board. + */ +@OptIn(ExperimentalMemoryApi::class) +public class TernaryDecodeHarness( + public val layers: Int = 2, + public val hidden: Int = 256, + public val ctx: Int = 32, + public val kvHeads: Int = 2, + public val heads: Int = 4, +) { + public val sink: RecordingTraceSink = RecordingTraceSink() + private val model = ModelScope(sink, "m2-harness") + + init { + require(hidden % 256 == 0) { "the ternary kernel works in TQ2_0 blocks of 256; hidden=$hidden" } + KernelDispatch.clearForTesting() + TernaryKernelPacks.install(native = null) // the portable kernel: what a device without the pack runs + } + + /** Per-layer ternary weights, TQ2_0, resident in the model scope. */ + private val weights: List = buildList { + for (l in 0 until layers) { + add(ternaryWeight(hidden, hidden, TensorId(listOf("model", "layers[$l]", "attn"), "q_proj.weight"))) + add(ternaryWeight(hidden, hidden, TensorId(listOf("model", "layers[$l]", "mlp"), "up_proj.weight"))) + } + } + + private val kv = DefaultKvCacheStore( + KvCacheConfig(numLayers = layers, numHeads = kvHeads, headDim = hidden / heads, maxSeqLen = ctx), + model, + slidingWindow = true, // #1036: the ring, so a long run does not grow + ) + + private val forward = ForwardScope(slabFloats = 4 * hidden + 64, sink = sink, name = "m2-decode") + + /** Bytes of ternary weight resident in the model scope. */ + public val weightBytes: Long = weights.sumOf { it.format.physicalBytes(it.elementCount) ?: 0L } + + private fun ternaryWeight(rows: Int, cols: Int, id: TensorId): TensorView { + var seed = id.canonical.hashCode() + val values = FloatArray(rows * cols) { + seed = seed * 1103515245 + 12345 + ((seed ushr 16) % 3 - 1) * 0.5f + } + val bytes = TernaryCodec.encode(TensorEncoding.TQ2_0, values) + val storage = model.adopt(Storage.Heap.wrap(bytes, mutable = false, origin = id, sink = sink)) + return TensorView.packed( + storage, Shape(rows, cols), TensorEncoding.TQ2_0, + TernaryBlockDecoder(TensorEncoding.TQ2_0), id = id, + ) + } + + /** Run [steps] decode steps; returns the process counters before and after the timed region. */ + public fun decode(steps: Int): Pair { + val activation = FloatArray(hidden) { (it % 13) * 0.0625f } + val k = FloatArray(kvHeads * (hidden / heads)) { 0.25f } + // warm up outside the measured window: first-touch faults are not steady-state behaviour + step(0, activation, k) + val before = MemoryProbe.sample() + for (s in 1..steps) step(s, activation, k) + val after = MemoryProbe.sample() + return before to after + } + + private fun step(step: Int, activation: FloatArray, k: FloatArray) { + sink.decodeStep(step) { + val slab = forward.allocateFloats(hidden, TensorId(listOf("model"), "hidden", "step=$step")) + activation.copyInto(slab.floats!!, slab.arrayOffset) + val dense = TensorView.dense(slab, Shape(1, hidden), FP32, TensorId(listOf("model"), "hidden", "step=$step")) + for (w in weights) { + val out = forward.allocateFloats(w.shape[0], TensorId(listOf("model"), "proj", "step=$step")) + val outView = TensorView.dense(out, Shape(1, w.shape[0]), FP32) + // the dispatcher requantizes the activation into the forward scope and picks bitnet_gemv + KernelDispatch.matmul(dense, w, outView, forward, sink) + } + for (l in 0 until layers) kv.appendToken(l, k, k) + forward.reset() + } + } + + /** Live bytes per scope as the event stream saw them. */ + public fun liveBytes(): Map { + val live = HashMap() + for (e in sink.events()) when (e) { + is TraceEvent.Allocation -> live[e.scope] = (live[e.scope] ?: 0L) + e.bytes + is TraceEvent.Free -> live[e.scope] = ((live[e.scope] ?: 0L) - e.bytes).coerceAtLeast(0L) + is TraceEvent.ScopeReset -> live[e.scope] = e.liveBytesAfter + else -> Unit + } + return live + } + + /** Adapter events recorded during step [step]. */ + public fun adaptersInStep(step: Int): List { + val out = ArrayList() + var current = 0 + for (e in sink.events()) { + if (e is TraceEvent.PhaseBegin && e.phase == "decode") current = e.step ?: current + if (e is TraceEvent.AdapterInserted && current == step) out += e + } + return out + } + + public fun close() { + forward.close() + model.close() + KernelDispatch.clearForTesting() + } +} diff --git a/skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api b/skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api index 6248faeb6..02bd67733 100644 --- a/skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api +++ b/skainet-lang/skainet-lang-core/api/jvm/skainet-lang-core.api @@ -938,6 +938,17 @@ public final class sk/ainet/lang/memory/MemoryDebug$Entry { public fun toString ()Ljava/lang/String; } +public final class sk/ainet/lang/memory/MemoryProbe { + public static final field INSTANCE Lsk/ainet/lang/memory/MemoryProbe; + public final fun majorFaults ()Ljava/lang/Long; + public final fun minorFaults ()Ljava/lang/Long; + public final fun rssBytes ()Ljava/lang/Long; +} + +public final class sk/ainet/lang/memory/MemoryProbeKt { + public static final fun sample (Lsk/ainet/lang/memory/MemoryProbe;)Lsk/ainet/lang/memory/ProcessMemorySample; +} + public final class sk/ainet/lang/memory/ModelScope : sk/ainet/lang/memory/Scope { public fun ()V public fun (Lsk/ainet/lang/memory/trace/TraceSink;Ljava/lang/String;)V @@ -1029,6 +1040,23 @@ public final class sk/ainet/lang/memory/PlatformStorageInfo { public fun toString ()Ljava/lang/String; } +public final class sk/ainet/lang/memory/ProcessMemorySample { + public fun (Ljava/lang/Long;Ljava/lang/Long;Ljava/lang/Long;)V + public final fun component1 ()Ljava/lang/Long; + public final fun component2 ()Ljava/lang/Long; + public final fun component3 ()Ljava/lang/Long; + public final fun copy (Ljava/lang/Long;Ljava/lang/Long;Ljava/lang/Long;)Lsk/ainet/lang/memory/ProcessMemorySample; + public static synthetic fun copy$default (Lsk/ainet/lang/memory/ProcessMemorySample;Ljava/lang/Long;Ljava/lang/Long;Ljava/lang/Long;ILjava/lang/Object;)Lsk/ainet/lang/memory/ProcessMemorySample; + public final fun emitTo (Lsk/ainet/lang/memory/trace/TraceSink;)V + public fun equals (Ljava/lang/Object;)Z + public final fun getMajorFaults ()Ljava/lang/Long; + public final fun getMinorFaults ()Ljava/lang/Long; + public final fun getRssBytes ()Ljava/lang/Long; + public fun hashCode ()I + public final fun majorFaultsSince (Lsk/ainet/lang/memory/ProcessMemorySample;)Ljava/lang/Long; + public fun toString ()Ljava/lang/String; +} + public final class sk/ainet/lang/memory/ScalePlacement : java/lang/Enum { public static final field BLOCK_HEAD Lsk/ainet/lang/memory/ScalePlacement; public static final field BLOCK_TAIL Lsk/ainet/lang/memory/ScalePlacement; diff --git a/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/MemoryProbe.kt b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/MemoryProbe.kt new file mode 100644 index 000000000..c7b57f075 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/commonMain/kotlin/sk/ainet/lang/memory/MemoryProbe.kt @@ -0,0 +1,61 @@ +package sk.ainet.lang.memory + +import sk.ainet.lang.memory.trace.Counters +import sk.ainet.lang.memory.trace.TraceSink +import sk.ainet.lang.memory.trace.counter + +/** + * What the *operating system* thinks this process is using (SKEEP-003 §4.9; M2-A1/M2-A3). + * + * The allocation events say what SKaiNET asked for; this says what the process actually holds and + * how often it had to go to disk for it. Both are needed: a plan that matches the allocation + * events and a resident set that keeps growing means the bytes are escaping somewhere the model + * does not see, and mapped weights are only "free" while the page-fault rate stays near zero. + * + * Every value is `null` where the platform cannot answer — a browser has no `/proc`, and neither + * has a Wasm host. Callers report "—", they do not guess. + */ +@ExperimentalMemoryApi +public expect object MemoryProbe { + /** Resident set size in bytes, or `null` when the platform cannot say. */ + public fun rssBytes(): Long? + + /** Major page faults since process start — the ones that went to disk. */ + public fun majorFaults(): Long? + + /** Minor page faults since process start — the ones satisfied from page cache. */ + public fun minorFaults(): Long? +} + +/** One sample of the process-level counters, with the fields the platform could answer. */ +@ExperimentalMemoryApi +public data class ProcessMemorySample( + val rssBytes: Long?, + val majorFaults: Long?, + val minorFaults: Long?, +) { + /** Emit what is known as trace counters, so it lands beside the plan in an exported trace. */ + public fun emitTo(sink: TraceSink) { + if (!sink.isEnabled) return + rssBytes?.let { sink.counter(Counters.RSS, it) } + majorFaults?.let { sink.counter(Counters.PAGE_FAULTS, it, unit = "faults") } + } + + /** Major faults between this sample and an [earlier] one, or `null` if either is unknown. */ + public fun majorFaultsSince(earlier: ProcessMemorySample): Long? { + val now = majorFaults ?: return null + val then = earlier.majorFaults ?: return null + return now - then + } + + override fun toString(): String = buildString { + append("rss=").append(rssBytes?.let { "${it / (1024 * 1024)} MB" } ?: "—") + append(" majflt=").append(majorFaults?.toString() ?: "—") + append(" minflt=").append(minorFaults?.toString() ?: "—") + } +} + +/** Sample all three counters at once. */ +@ExperimentalMemoryApi +public fun MemoryProbe.sample(): ProcessMemorySample = + ProcessMemorySample(rssBytes(), majorFaults(), minorFaults()) diff --git a/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/MemoryProbeTest.kt b/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/MemoryProbeTest.kt new file mode 100644 index 000000000..3f07aa307 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/MemoryProbeTest.kt @@ -0,0 +1,68 @@ +package sk.ainet.lang.memory + +import sk.ainet.lang.memory.trace.Counters +import sk.ainet.lang.memory.trace.RecordingTraceSink +import sk.ainet.lang.memory.trace.TraceEvent +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * #1042 (M2-A1/M2-A3): the process-level counters — what the OS says this process holds, and how + * often it had to fetch a page from disk. + * + * The values are platform-dependent by nature, so the assertions are about the *contract*: a + * platform either answers with something plausible or says `null`, and whatever it answers reaches + * the trace as a counter. The numbers themselves are read on the reference device. + */ +@OptIn(ExperimentalMemoryApi::class) +class MemoryProbeTest { + + @Test + fun aSampleIsEitherPlausibleOrHonestlyAbsent() { + val sample = MemoryProbe.sample() + sample.rssBytes?.let { + assertTrue(it > 0, "a resident set of $it bytes is not plausible") + assertTrue(it < 64L * 1024 * 1024 * 1024, "a resident set of $it bytes is not plausible") + } + sample.majorFaults?.let { assertTrue(it >= 0) } + sample.minorFaults?.let { assertTrue(it >= 0) } + assertTrue(sample.toString().contains("rss="), sample.toString()) + } + + @Test + fun theResidentSetGrowsWhenTheProcessActuallyAllocates() { + val before = MemoryProbe.rssBytes() + if (before == null) return // platform cannot answer; nothing to assert + // touch every page so the allocation is resident, not just reserved + val chunk = ByteArray(32 * 1024 * 1024) + for (i in chunk.indices step 4096) chunk[i] = 1 + val after = MemoryProbe.rssBytes()!! + assertTrue(after >= before, "RSS went backwards: $before → $after (kept ${chunk.size} bytes alive)") + } + + @Test + fun faultCountersOnlyEverMoveForward() { + val first = MemoryProbe.sample() + val chunk = ByteArray(8 * 1024 * 1024) + for (i in chunk.indices step 4096) chunk[i] = 1 + val second = MemoryProbe.sample() + val delta = second.majorFaultsSince(first) + if (delta != null) { + assertTrue(delta >= 0, "major faults went backwards by $delta") + assertEquals(0L, delta, "touching freshly allocated anonymous memory must not fault to disk") + } + assertTrue(chunk.isNotEmpty()) + } + + @Test + fun whatTheProbeKnowsReachesTheTrace() { + val sink = RecordingTraceSink() + val sample = MemoryProbe.sample() + sample.emitTo(sink) + val counters = sink.eventsOf().associate { it.name to it.value } + assertEquals(sample.rssBytes != null, counters.containsKey(Counters.RSS)) + assertEquals(sample.majorFaults != null, counters.containsKey(Counters.PAGE_FAULTS)) + sample.rssBytes?.let { assertEquals(it, counters[Counters.RSS]) } + } +} diff --git a/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/plan/BitNet2BPlanTest.kt b/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/plan/BitNet2BPlanTest.kt new file mode 100644 index 000000000..982d29050 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/commonTest/kotlin/sk/ainet/lang/memory/plan/BitNet2BPlanTest.kt @@ -0,0 +1,170 @@ +package sk.ainet.lang.memory.plan + +import sk.ainet.lang.memory.ExperimentalMemoryApi +import sk.ainet.lang.memory.Format +import sk.ainet.lang.tensor.storage.TensorEncoding +import sk.ainet.lang.types.BF16 +import sk.ainet.lang.types.FP32 +import kotlin.test.Test +import kotlin.test.assertEquals +import kotlin.test.assertTrue + +/** + * M2-A1's arithmetic (#1042): **BitNet-b1.58-2B fits in 1.3 GB resident**, computed from the + * model's geometry the way `skainet-plan` computes it from a GGUF header. + * + * The claim M2-A1 makes is about *planning* — what a 2 B-parameter ternary model needs before a + * byte is read — and that is checkable here, exactly, without a checkpoint. What a real + * BitNet-2B *measures* on a device belongs to the decode sample in SKaiNET-transformers; this + * pins the number the planner will predict when it gets there. + * + * Geometry: 30 layers, 2560 hidden, 6912 FFN, 20 heads / 5 KV heads (GQA), 128 256 vocab — the + * published shape of BitNet-b1.58-2B-4T. Ternary linear weights are TQ2_0 (2.0625 bits/element on + * disk); the embedding table and the output head stay bf16, which is what a b1.58 checkpoint ships + * and, as the numbers below show, is where most of the resident memory actually goes. + */ +@OptIn(ExperimentalMemoryApi::class) +class BitNet2BPlanTest { + + private val mb = 1024L * 1024L + private val layers = 30 + private val hidden = 2560 + private val ffn = 6912 + private val heads = 20 + private val kvHeads = 5 + private val headDim = hidden / heads + private val vocab = 128_256 + + private fun input(ctx: Int, kvMode: KvCacheMode = KvCacheMode.BF16): PlanInput { + val ternary = Format(FP32, TensorEncoding.TQ2_0) + val narrow = Format(BF16, TensorEncoding.Dense(2)) + val weights = ArrayList() + + fun ternaryTensor(name: String, elements: Long) { + weights += PlanTensor(name, null, ternary, elements, ternary.physicalBytes(elements)!!) + } + for (l in 0 until layers) { + // attention: q, k, v, o — GQA narrows k and v + ternaryTensor("blk.$l.attn_q", hidden.toLong() * hidden) + ternaryTensor("blk.$l.attn_k", hidden.toLong() * kvHeads * headDim) + ternaryTensor("blk.$l.attn_v", hidden.toLong() * kvHeads * headDim) + ternaryTensor("blk.$l.attn_output", hidden.toLong() * hidden) + // feed-forward: gate, up, down + ternaryTensor("blk.$l.ffn_gate", hidden.toLong() * ffn) + ternaryTensor("blk.$l.ffn_up", hidden.toLong() * ffn) + ternaryTensor("blk.$l.ffn_down", ffn.toLong() * hidden) + } + // One embedding table: the output head is tied to it, as the released checkpoint has it. + // At 128 256 × 2560 in bf16 that single table is 657 MB — more than the entire ternary + // stack, and the reason a "2-bit model" is not a 500 MB model. + val embeddingElements = vocab.toLong() * hidden + weights += PlanTensor("token_embd", null, narrow, embeddingElements, narrow.physicalBytes(embeddingElements)!!) + + val geometry = ModelGeometry( + layers = layers, heads = heads, kvHeads = kvHeads, headDim = headDim, + embeddingLength = hidden, feedForwardLength = ffn, vocabSize = vocab, + ) + return PlanInput("BitNet-b1.58-2B", "bitnet", weights, geometry, ctx, kvMode = kvMode) + } + + @Test + fun theTernaryWeightsAreWhatTheEncodingPromises() { + val plan = MemoryPlans.plan(input(ctx = 2048)) + val ternaryElements = layers.toLong() * + (2L * hidden * hidden + 2L * hidden * kvHeads * headDim + 3L * hidden * ffn) + val ternaryBytes = ternaryElements / 256 * 66 + val embeddingBytes = vocab.toLong() * hidden * 2 + assertEquals(ternaryBytes + embeddingBytes, plan.weightsBytes) + assertEquals(2.0625, ternaryBytes * 8.0 / ternaryElements, 1e-9, "TQ2_0 is 2.0625 bits per element") + assertTrue( + ternaryBytes in (500 * mb)..(520 * mb), + "2 B ternary parameters cost ~512 MB, got ${MemoryPlans.formatBytes(ternaryBytes)}", + ) + assertTrue( + embeddingBytes > ternaryBytes, + "the bf16 embedding table (${MemoryPlans.formatBytes(embeddingBytes)}) outweighs the ternary stack " + + "(${MemoryPlans.formatBytes(ternaryBytes)}) — that is where a 2-bit model's memory actually goes", + ) + } + + @Test + fun m2a1_theModelIsResidentUnderOnePointThreeGigabytes() { + val limit = (1.3 * 1024 * mb).toLong() + val quantizedKv = MemoryPlans.plan(input(ctx = 2048, kvMode = KvCacheMode.TURBOQUANT_4)) + assertTrue( + quantizedKv.residentBytes <= limit, + "M2-A1: resident (weights + KV) must fit 1.3 GB, was ${MemoryPlans.formatBytes(quantizedKv.residentBytes)}\n" + + quantizedKv.render(), + ) + + // A bf16 cache fits too — but with 42 MB of headroom against the quantized cache's 148 MB, + // and the margin closes as the context grows. That thin margin is why the mobile profile + // quantizes the cache itself rather than leaving it to the caller (#1039). + val bf16Kv = MemoryPlans.plan(input(ctx = 2048)) + assertTrue(bf16Kv.residentBytes <= limit, bf16Kv.render()) + val bf16Margin = limit - bf16Kv.residentBytes + val quantizedMargin = limit - quantizedKv.residentBytes + assertTrue( + quantizedMargin > bf16Margin * 3, + "bf16 margin ${MemoryPlans.formatBytes(bf16Margin)} vs quantized ${MemoryPlans.formatBytes(quantizedMargin)}", + ) + + // and at a longer context the bf16 cache does push it over, while the quantized one does not + val longBf16 = MemoryPlans.plan(input(ctx = 8192)) + val longQuantized = MemoryPlans.plan(input(ctx = 8192, kvMode = KvCacheMode.TURBOQUANT_4)) + assertTrue(longBf16.residentBytes > limit, "ctx 8192 with bf16: ${MemoryPlans.formatBytes(longBf16.residentBytes)}") + assertTrue(longQuantized.residentBytes <= limit, "ctx 8192 quantized: ${longQuantized.render()}") + } + + @Test + fun aTwoGigabyteDeviceCannotHoldThisModelAndTheProfileSaysSoBeforeLoading() { + // 2 GB total, ~1.5 GB free — the reference board's own numbers. After the profile's 700 MB + // reserve for the OS and the rest of the app, 800 MB remain, and a 1.2 GB model does not + // fit in 800 MB however it is staged. The value of the planner is saying that *first*. + val profiled = PlannerProfile.MOBILE_2GB.plan(input(ctx = 2048), availableBytes = 1500 * mb) + assertEquals(false, profiled.fits, profiled.render()) + val failure = kotlin.test.assertFailsWith { profiled.requireFits() } + assertTrue(failure.message!!.contains("mobile-2gb")) + assertTrue(profiled.plan.suggestions().isNotEmpty(), "and it says what would help") + } + + @Test + fun aFourGigabyteDeviceHoldsItWithMappedWeights() { + val profiled = PlannerProfile.MOBILE_2GB.plan(input(ctx = 2048), availableBytes = 3200 * mb) + profiled.requireFits() + val device = DeviceMemory( + totalRamBytes = 4096 * mb, availableRamBytes = 3200 * mb, + heapMaxBytes = 512 * mb, heapUsedBytes = 40 * mb, lowMemoryThresholdBytes = 256 * mb, + ) + // mapped weights: the managed heap carries only the cache, the slab and the headroom + profiled.requireFits(device) + assertTrue(profiled.plan.residentBytes <= (1.3 * 1024 * mb).toLong(), profiled.render()) + } + + @Test + fun m2a1_theSameModelWithHeapStagingDoesNotFitThatDevice() { + // the counterfactual that makes the mapped path worth having (#921/#922) + val profiled = PlannerProfile.DESKTOP.plan(input(ctx = 2048), availableBytes = 3200 * mb) + val phone = DeviceMemory( + totalRamBytes = 4096 * mb, availableRamBytes = 3200 * mb, + heapMaxBytes = 512 * mb, heapUsedBytes = 40 * mb, lowMemoryThresholdBytes = 180 * mb, + ) + val fit = profiled.plan.fitOn(phone, weightsMapped = false) + assertTrue(!fit.fits, "a 1.2 GB model cannot live on a 512 MB heap:\n${fit.render()}") + assertEquals("managed heap", fit.blockingPool) + val mapped = profiled.plan.fitOn(phone, weightsMapped = true) + assertTrue(mapped.heap.fits, "mapped, the heap is no longer the problem — only the RAM budget is") + } + + @Test + fun aLongerContextIsTheFirstThingToGive() { + val short = MemoryPlans.plan(input(ctx = 2048)) + val long = MemoryPlans.plan(input(ctx = 8192)) + assertTrue(long.kvBytes > short.kvBytes) + assertEquals(short.weightsBytes, long.weightsBytes, "context length does not change the weights") + + // and quantizing the cache is what the mobile profile does about it + val quantized = MemoryPlans.plan(input(ctx = 8192, kvMode = KvCacheMode.TURBOQUANT_4)) + assertTrue(quantized.kvBytes < long.kvBytes / 2, "TurboQuant-4 must more than halve a bf16 cache") + } +} diff --git a/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.js.kt b/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.js.kt new file mode 100644 index 000000000..4383c2eb6 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/jsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.js.kt @@ -0,0 +1,9 @@ +package sk.ainet.lang.memory + +/** No process-level view here: a browser or Wasm host does not expose one. */ +@ExperimentalMemoryApi +public actual object MemoryProbe { + public actual fun rssBytes(): Long? = null + public actual fun majorFaults(): Long? = null + public actual fun minorFaults(): Long? = null +} diff --git a/skainet-lang/skainet-lang-core/src/jvmAndroidMain/kotlin/sk/ainet/lang/memory/MemoryProbe.jvmAndroid.kt b/skainet-lang/skainet-lang-core/src/jvmAndroidMain/kotlin/sk/ainet/lang/memory/MemoryProbe.jvmAndroid.kt new file mode 100644 index 000000000..777d5aa4f --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/jvmAndroidMain/kotlin/sk/ainet/lang/memory/MemoryProbe.jvmAndroid.kt @@ -0,0 +1,44 @@ +package sk.ainet.lang.memory + +import java.io.File + +/** + * JVM and Android: `/proc/self` — the kernel's own numbers, not the JVM's. + * + * `Runtime.totalMemory()` describes the managed heap, which is exactly the thing that is *not* + * interesting once weights live in mapped pages. `statm` field 2 is the resident page count and + * `stat` fields 10 and 12 are the minor and major fault counters (see `proc(5)`). + * + * Returns `null` off Linux (macOS, Windows), where these files do not exist. + */ +@ExperimentalMemoryApi +public actual object MemoryProbe { + + private val pageSize: Long = 4096L + + public actual fun rssBytes(): Long? { + val fields = read("/proc/self/statm")?.split(" ") ?: return null + val pages = fields.getOrNull(1)?.trim()?.toLongOrNull() ?: return null + return pages * pageSize + } + + public actual fun majorFaults(): Long? = statField(12) + + public actual fun minorFaults(): Long? = statField(10) + + /** `/proc/self/stat` is 1-indexed in `proc(5)`; the comm field may contain spaces, so split after it. */ + private fun statField(index: Int): Long? { + val stat = read("/proc/self/stat") ?: return null + val afterComm = stat.substringAfterLast(") ") + val fields = afterComm.split(" ") + // field 3 (state) is the first token after comm, so `index` maps to `index - 3` + return fields.getOrNull(index - 3)?.trim()?.toLongOrNull() + } + + private fun read(path: String): String? = try { + val file = File(path) + if (file.canRead()) file.readText().trim() else null + } catch (e: Exception) { + null + } +} diff --git a/skainet-lang/skainet-lang-core/src/nativeMain/kotlin/sk/ainet/lang/memory/MemoryProbe.native.kt b/skainet-lang/skainet-lang-core/src/nativeMain/kotlin/sk/ainet/lang/memory/MemoryProbe.native.kt new file mode 100644 index 000000000..f9ed339c5 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/nativeMain/kotlin/sk/ainet/lang/memory/MemoryProbe.native.kt @@ -0,0 +1,46 @@ +package sk.ainet.lang.memory + +import kotlinx.cinterop.ExperimentalForeignApi +import kotlinx.cinterop.refTo +import kotlinx.cinterop.toKString +import platform.posix.fclose +import platform.posix.fgets +import platform.posix.fopen + +/** + * Kotlin/Native on Linux: the same `/proc/self` files the JVM actual reads. On a platform without + * them (macOS, iOS) `fopen` fails and every value is `null`, which is the honest answer. + */ +@OptIn(ExperimentalForeignApi::class) +@ExperimentalMemoryApi +public actual object MemoryProbe { + + private const val PAGE_SIZE: Long = 4096L + + public actual fun rssBytes(): Long? { + val fields = read("/proc/self/statm")?.split(" ") ?: return null + val pages = fields.getOrNull(1)?.trim()?.toLongOrNull() ?: return null + return pages * PAGE_SIZE + } + + public actual fun majorFaults(): Long? = statField(12) + + public actual fun minorFaults(): Long? = statField(10) + + private fun statField(index: Int): Long? { + val stat = read("/proc/self/stat") ?: return null + val afterComm = stat.substringAfterLast(") ") + return afterComm.split(" ").getOrNull(index - 3)?.trim()?.toLongOrNull() + } + + private fun read(path: String): String? { + val file = fopen(path, "r") ?: return null + try { + val buffer = ByteArray(4096) + val line = fgets(buffer.refTo(0), buffer.size, file) ?: return null + return line.toKString().trim() + } finally { + fclose(file) + } + } +} diff --git a/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmJs.kt b/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmJs.kt new file mode 100644 index 000000000..4383c2eb6 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/wasmJsMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmJs.kt @@ -0,0 +1,9 @@ +package sk.ainet.lang.memory + +/** No process-level view here: a browser or Wasm host does not expose one. */ +@ExperimentalMemoryApi +public actual object MemoryProbe { + public actual fun rssBytes(): Long? = null + public actual fun majorFaults(): Long? = null + public actual fun minorFaults(): Long? = null +} diff --git a/skainet-lang/skainet-lang-core/src/wasmWasiMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmWasi.kt b/skainet-lang/skainet-lang-core/src/wasmWasiMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmWasi.kt new file mode 100644 index 000000000..4383c2eb6 --- /dev/null +++ b/skainet-lang/skainet-lang-core/src/wasmWasiMain/kotlin/sk/ainet/lang/memory/MemoryProbe.wasmWasi.kt @@ -0,0 +1,9 @@ +package sk.ainet.lang.memory + +/** No process-level view here: a browser or Wasm host does not expose one. */ +@ExperimentalMemoryApi +public actual object MemoryProbe { + public actual fun rssBytes(): Long? = null + public actual fun majorFaults(): Long? = null + public actual fun minorFaults(): Long? = null +}