From 62fd1c23d8ab1c088894de1e0a0e6a008f2cc431 Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Sun, 23 Aug 2026 11:11:07 +0200 Subject: [PATCH] =?UTF-8?q?bench(spike):=20Phase-2=20TensorView/Storage=20?= =?UTF-8?q?access-path=20spike=20=E2=80=94=20JMH=20benchmarks,=20flat-RSS?= =?UTF-8?q?=20loop,=20report=20(SKEEP-003=20decision=20#6,=20#1016)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Throw-away model of the proposed Storage (heap FloatArray | off-heap MemorySegment) and TensorView (offset/length over a storage) measured against raw arrays: elementwise add over 1 M floats and gemv 256x1024, each as raw / view-get / view-unwrap-once over heap and off-heap, plus a raw-with-offsets control; and FlatRssSpike, a 3000-step decode-shaped loop with a recycled Arena.ofShared bump slab vs fresh heap arrays. Findings (docs/design/memory/spike-p2-access-path-2026-08-23.md): the view indirection is within budget when unwrapped once per call (gemv +0..4 %); offset-based array indexing, not the view, costs 1.9x on the vectorizable elementwise loop (C2 SuperWord); per-element get() is the slow reference path (+80 % heap, 5.5x segment); off-heap activations need Vector-API-over-segment kernels, so Scope.Forward on the JVM should default to heap slabs; the slab keeps RSS at 62-86 MiB over 3000 steps vs 214 MiB with per-step arrays. Verdict: go for Phase 2 with the unwrap-once rule; the <= 3 % elementwise budget is re-checked on the real vector kernels at S1.4/S1.7. Android half pending the reference device. Also: -PjmhIncludes= to run a subset of the JMH suite, and the runFlatRssSpike JavaExec task. Closes #1016 Co-Authored-By: Claude Fable 5 --- .../memory/spike-p2-access-path-2026-08-23.md | 69 ++++++ .../benchmarks/jvm-cpu-jmh/build.gradle.kts | 13 ++ .../sk/ainet/bench/spike/FlatRssSpike.kt | 91 ++++++++ .../ainet/bench/spike/TensorViewSpikeBench.kt | 197 ++++++++++++++++++ 4 files changed, 370 insertions(+) create mode 100644 docs/design/memory/spike-p2-access-path-2026-08-23.md create mode 100644 skainet-backends/benchmarks/jvm-cpu-jmh/src/jmh/kotlin/sk/ainet/bench/spike/FlatRssSpike.kt create mode 100644 skainet-backends/benchmarks/jvm-cpu-jmh/src/jmh/kotlin/sk/ainet/bench/spike/TensorViewSpikeBench.kt diff --git a/docs/design/memory/spike-p2-access-path-2026-08-23.md b/docs/design/memory/spike-p2-access-path-2026-08-23.md new file mode 100644 index 000000000..49f55774b --- /dev/null +++ b/docs/design/memory/spike-p2-access-path-2026-08-23.md @@ -0,0 +1,69 @@ +# Phase-2 spike — TensorView / Storage access-path cost on the JVM — 2026-08-23 + +SKEEP-003 decision #6 / issue #1016. Before milestone M1 introduces `Storage`, `Scope` and `TensorView`, measure what a view layer between a kernel and its bytes costs, with a throw-away model of the proposed types: `TensorViewSpikeBench` and `FlatRssSpike` in `skainet-backends/benchmarks/jvm-cpu-jmh` (package `sk.ainet.bench.spike`; run with `-PjmhIncludes='spike.*'` and `runFlatRssSpike`). Budget (decision #6): elementwise ≤ 3 %, matmul within noise; above that, the access path is redesigned — unwrap to the raw array / segment once per call — before Phase 2 proper. + +- Commit: `develop` @ 3c7c7d19 + the spike benchmarks · HotSpot JDK 25 (`--enable-preview --add-modules jdk.incubator.vector`) · JMH fork 1 / 3 warmup / 5 iterations · i7-9750H, machine idle. +- Model: `SpikeStorage` sealed (`Heap(FloatArray)` | `OffHeap(MemorySegment)`), `SpikeView(storage, offset, length)` with `get`/`set` dispatching on the storage kind, and the two fast paths a kernel takes once per call: `asHeapArray()` and `segment()`. + +## Elementwise `c = a + b`, 1 M floats (avgt, µs/op — lower is better) + +| Variant | µs/op | vs raw | +|---|---:|---:| +| raw `FloatArray`, zero-based loop (`add_raw`) | 286 ± 16 | +0 % | +| raw `FloatArray`, loop-invariant offsets — **control, no view** (`add_rawOffset`) | 539 ± 24 | +88 % | +| view over heap, `asHeapArray()` once, offset loop (`add_viewHeapUnwrap`) | 567 ± 182 | +98 % | +| view over heap, `get`/`set` per element (`add_viewHeapGet`) | 516 ± 15 | +81 % | +| view over `MemorySegment`, `segment()` once, `getAtIndex` per element (`add_viewOffHeapUnwrap`) | 495 ± 19 | +73 % | +| view over `MemorySegment`, `get`/`set` per element (`add_viewOffHeapGet`) | 1579 ± 3 | +452 % | + +## GEMV 256 × 1024 (avgt, µs/op) + +| Variant | µs/op | vs raw | +|---|---:|---:| +| raw arrays (`gemv_raw`) | 249 ± 1 | +0 % | +| view over heap, unwrap once (`gemv_viewHeapUnwrap`) | 253 ± 1 | +1 % | +| view over heap, `get` per element (`gemv_viewHeapGet`) | 249 ± 1 | +0 % | +| view over `MemorySegment`, unwrap once (`gemv_viewOffHeapUnwrap`) | 257 ± 2 | +3 % | +| view over `MemorySegment`, `get` per element (`gemv_viewOffHeapGet`) | 259 ± 2 | +4 % | + +## Flat-RSS decode loop — `Forward` slab vs heap arrays per step (3 000 steps, `-Xmx256m`) + +``` +FlatRssSpike: 3000 steps, per-step activations = 1 MiB, pid=3758280 +--- Forward slab (Arena.ofShared bump + reset per step) --- + step 1 RSS 62 MiB heapUsed 9 MiB slab allocations so far 193 (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object) + step 50 RSS 65 MiB heapUsed 10 MiB slab allocations so far 9650 (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object) + step 100 RSS 66 MiB heapUsed 10 MiB slab allocations so far 19300 (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object) + step 500 RSS 69 MiB heapUsed 13 MiB slab allocations so far 96500 (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object) + step 1000 RSS 73 MiB heapUsed 17 MiB slab allocations so far 193000 (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object) + step 1500 RSS 77 MiB heapUsed 21 MiB slab allocations so far 289500 (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object) + step 2000 RSS 78 MiB heapUsed 21 MiB slab allocations so far 386000 (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object) + step 3000 RSS 86 MiB heapUsed 29 MiB slab allocations so far 579000 (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object) + sink=8.64284E8 +--- Heap arrays per step (today: GC-backed) --- + step 1 RSS 86 MiB heapUsed 31 MiB heap allocations so far 193 + step 50 RSS 141 MiB heapUsed 82 MiB heap allocations so far 9650 + step 100 RSS 174 MiB heapUsed 71 MiB heap allocations so far 19300 + step 500 RSS 213 MiB heapUsed 123 MiB heap allocations so far 96500 + step 1000 RSS 213 MiB heapUsed 74 MiB heap allocations so far 193000 + step 1500 RSS 214 MiB heapUsed 15 MiB heap allocations so far 289500 + step 2000 RSS 214 MiB heapUsed 111 MiB heap allocations so far 386000 + step 3000 RSS 214 MiB heapUsed 149 MiB heap allocations so far 579000 + sink=8.64284E8 +``` + +## What the numbers say + +1. **The view indirection itself is within budget when a kernel unwraps once per call.** GEMV: heap unwrap +1 %, heap `get` 0 %, off-heap +3–4 % (the dot-product reduction is not auto-vectorized, so per-element overhead is hidden). Elementwise: `add_viewHeapUnwrap` (566 ± 182) ≈ `add_rawOffset` (539 ± 24) — the view adds nothing measurable on top of its control. +2. **What *is* expensive is offset-based indexing, not the view:** `add_rawOffset` — raw arrays, no view, just loop-invariant offsets — is 1.9× slower than the zero-based loop. C2's auto-vectorizer (SuperWord) handles the zero-based `c[i] = a[i] + b[i]` and does not handle the three-offset form here. Consequence for M1: JVM elementwise kernels operating on views must not depend on C2 auto-vectorization; SKaiNET's vector kernels (`JvmVectorKernels`, explicit `FloatVector.fromArray(species, array, offset)`) take offsets explicitly and are unaffected — the ≤ 3 % elementwise budget is to be *measured on those kernels* when the façades land (S1.4 gate re-runs `ElementwiseAdd1MBench`), and scalar fallbacks should special-case `offset == 0`. +3. **Per-element `get()` through a view is the slow path, by design.** +80 % over heap, 5.5× over `MemorySegment` on a vectorizable loop. Rule 4 (decoding `get()`) stays the reference path for correctness; production kernels receive a `TensorView` and call `asHeapArray()` / `segment()` once. +4. **Off-heap activations on the JVM need Vector-API-over-segment kernels.** Scalar `getAtIndex` loops over `MemorySegment` are 1.7× (unwrapped) to 5.5× (through the view) slower than arrays. Weights are already consumed as segments by the Panama quantized kernels (`ByteVector.fromMemorySegment`), so mapped/off-heap **weights** are fine; for **activations**, `Scope.Forward`'s JVM binding should default to a **heap slab** (recycled `FloatArray`s / bump-allocated arrays) and make off-heap opt-in until the elementwise kernels have segment variants. Input for S1.1b / S1.2. +5. **Flat RSS:** with a recycled bump slab the process stays at 62–86 MiB over 3 000 steps with zero slab bytes allocated after warm-up; with fresh heap arrays per step it climbs to 214 MiB and plateaus there (GC heap expanded to its ceiling). The slab run's residual growth (heapUsed 9 → 29 MiB) is the one `MemorySegment` view object `allocate()` creates per activation (193 per step) — the real `TensorView` will be a small object too: cheap, not free; pool or reuse views on the hot loop if minor GCs show up in the M1 trace. + +## Verdict (decision #6) + +**Go for Phase 2**, with the access-path rule written into the M1 slices: kernels take `TensorView`s and unwrap once (`asHeapArray()` / `segment()`); `get()` decodes and is the reference path only; JVM `Forward` scope allocates heap slabs by default; vector kernels keep explicit offsets. The elementwise ≤ 3 % budget is re-checked on the real vector kernels at S1.4 (façades) and S1.7 (dispatch) against `docs/design/memory/baseline-2026-08-22.md`. + +## Open: Android half + +The same measurement on a Cortex-A55-class 2 GB device (direct `ByteBuffer` path) is pending the reference-device choice (PRD §9 open item). The JVM result already sets the rule (unwrap once, heap activations by default); the Android run decides whether direct `ByteBuffer` activations are viable there and is recorded on #1016 when the device is available. diff --git a/skainet-backends/benchmarks/jvm-cpu-jmh/build.gradle.kts b/skainet-backends/benchmarks/jvm-cpu-jmh/build.gradle.kts index 370f78921..d68cf96f7 100644 --- a/skainet-backends/benchmarks/jvm-cpu-jmh/build.gradle.kts +++ b/skainet-backends/benchmarks/jvm-cpu-jmh/build.gradle.kts @@ -13,6 +13,8 @@ jmh { fork.set(1) warmupIterations.set(3) iterations.set(5) + // Run a subset: ./gradlew :skainet-backends:benchmarks:jvm-cpu-jmh:jmh -PjmhIncludes='spike.*' + (project.findProperty("jmhIncludes") as String?)?.let { includes.set(listOf(it)) } //timeOnIteration.set(org.gradle.api.tasks.testing.logging.TestLogEvent.values().size.toLong()) jvmArgs.set(listOf("--enable-preview", "--add-modules", "jdk.incubator.vector")) } @@ -21,3 +23,14 @@ jmh { tasks.withType().configureEach { jvmArgs("--enable-preview", "--add-modules", "jdk.incubator.vector") } + +// SKEEP-003 Phase-2 spike (#1016): flat-RSS decode loop, Forward slab vs heap arrays. +// ./gradlew :skainet-backends:benchmarks:jvm-cpu-jmh:runFlatRssSpike -Pspike.steps=2000 -Pspike.mode=both +tasks.register("runFlatRssSpike") { + group = "benchmark" + description = "SKEEP-003 P2 spike: RSS over decode steps with a recycled Forward slab vs heap arrays" + classpath = sourceSets["jmh"].runtimeClasspath + mainClass.set("sk.ainet.bench.spike.FlatRssSpike") + jvmArgs("--enable-preview", "--add-modules", "jdk.incubator.vector", "-Xmx256m") + args((project.findProperty("spike.steps") as String?) ?: "2000", (project.findProperty("spike.mode") as String?) ?: "both") +} diff --git a/skainet-backends/benchmarks/jvm-cpu-jmh/src/jmh/kotlin/sk/ainet/bench/spike/FlatRssSpike.kt b/skainet-backends/benchmarks/jvm-cpu-jmh/src/jmh/kotlin/sk/ainet/bench/spike/FlatRssSpike.kt new file mode 100644 index 000000000..ca971fbb5 --- /dev/null +++ b/skainet-backends/benchmarks/jvm-cpu-jmh/src/jmh/kotlin/sk/ainet/bench/spike/FlatRssSpike.kt @@ -0,0 +1,91 @@ +package sk.ainet.bench.spike + +import java.io.File +import java.lang.foreign.Arena +import java.lang.foreign.MemorySegment +import java.lang.foreign.ValueLayout + +/** + * SKEEP-003 Phase-2 spike, second half (decision #6, #1016): is decode-step memory flat when + * activations come from a recycled `Forward` slab (`Arena.ofShared` + bump offset + `reset()` + * per step) — versus fresh heap arrays per step (today's behaviour, GC-backed)? + * + * Simulates a 1B-class decode step: per step `LAYERS × K` activation buffers of `HIDDEN` floats + * (plus a `VOCAB` logits row), written and read once. Prints RSS (from /proc/self/status) at + * steps 50, 500, 1000, … and the allocation count. Run: + * + * `./gradlew :skainet-backends:benchmarks:jvm-cpu-jmh:runFlatRssSpike` (see build.gradle.kts). + */ +object FlatRssSpike { + private const val LAYERS = 16 + private const val BUFFERS_PER_LAYER = 12 + private const val HIDDEN = 2048 + private const val VOCAB = 128_256 + + /** The Forward-slab model: one shared arena, bump allocation, reset per step — zero steady-state allocation. */ + class ForwardSlab(bytes: Long) { + private val arena = Arena.ofShared() + private val slab: MemorySegment = arena.allocate(bytes, 64) + private var offset = 0L + var allocations = 0L; private set + fun allocate(bytes: Long): MemorySegment { + val aligned = (bytes + 63) and 63L.inv() + require(offset + aligned <= slab.byteSize()) { "Forward slab exhausted: need $aligned at $offset of ${slab.byteSize()}" } + val s = slab.asSlice(offset, bytes); offset += aligned; allocations++; return s + } + fun reset() { offset = 0L } + fun close() = arena.close() + } + + private fun rssMiB(): Long = File("/proc/self/status").readLines().firstOrNull { it.startsWith("VmRSS:") } + ?.split(Regex("\\s+"))?.getOrNull(1)?.toLongOrNull()?.let { it / 1024 } ?: -1 + + @JvmStatic + fun main(args: Array) { + val steps = args.getOrNull(0)?.toIntOrNull() ?: 2000 + val mode = args.getOrNull(1) ?: "both" + val stepBytes = LAYERS.toLong() * BUFFERS_PER_LAYER * HIDDEN * 4 + VOCAB.toLong() * 4 + println("FlatRssSpike: $steps steps, per-step activations = ${stepBytes / 1024 / 1024} MiB, pid=${ProcessHandle.current().pid()}") + if (mode == "both" || mode == "slab") runSlab(steps, stepBytes) + if (mode == "both" || mode == "heap") runHeap(steps) + } + + private fun checkpoints(steps: Int) = setOf(1, 50, 100, 500, 1000, 1500, 2000, 3000, 5000, steps) + + private fun heapUsedMiB(): Long = (Runtime.getRuntime().totalMemory() - Runtime.getRuntime().freeMemory()) / 1024 / 1024 + + private fun runSlab(steps: Int, stepBytes: Long) { + val slab = ForwardSlab(stepBytes + (1 shl 20)) + val marks = checkpoints(steps) // hoisted: the harness itself must not allocate per step + var sink = 0f + println("--- Forward slab (Arena.ofShared bump + reset per step) ---") + for (step in 1..steps) { + for (l in 0 until LAYERS) for (b in 0 until BUFFERS_PER_LAYER) { + val seg = slab.allocate(HIDDEN * 4L) + seg.setAtIndex(ValueLayout.JAVA_FLOAT, (step % HIDDEN).toLong(), step.toFloat()) + sink += seg.getAtIndex(ValueLayout.JAVA_FLOAT, (step % HIDDEN).toLong()) + } + val logits = slab.allocate(VOCAB * 4L); logits.setAtIndex(ValueLayout.JAVA_FLOAT, 0, sink) + slab.reset() + if (step in marks) println(" step %5d RSS %6d MiB heapUsed %4d MiB slab allocations so far %d (slab bytes allocated per step after warm-up: 0; each allocate() still creates one MemorySegment view object)".format(step, rssMiB(), heapUsedMiB(), slab.allocations)) + } + slab.close() + println(" sink=$sink") + } + + private fun runHeap(steps: Int) { + var sink = 0f + var allocations = 0L + val marks = checkpoints(steps) + println("--- Heap arrays per step (today: GC-backed) ---") + for (step in 1..steps) { + for (l in 0 until LAYERS) for (b in 0 until BUFFERS_PER_LAYER) { + val arr = FloatArray(HIDDEN); allocations++ + arr[step % HIDDEN] = step.toFloat(); sink += arr[step % HIDDEN] + } + val logits = FloatArray(VOCAB); allocations++; logits[0] = sink + if (step in marks) println(" step %5d RSS %6d MiB heapUsed %4d MiB heap allocations so far %d".format(step, rssMiB(), heapUsedMiB(), allocations)) + } + println(" sink=$sink") + } +} diff --git a/skainet-backends/benchmarks/jvm-cpu-jmh/src/jmh/kotlin/sk/ainet/bench/spike/TensorViewSpikeBench.kt b/skainet-backends/benchmarks/jvm-cpu-jmh/src/jmh/kotlin/sk/ainet/bench/spike/TensorViewSpikeBench.kt new file mode 100644 index 000000000..e15abcbdd --- /dev/null +++ b/skainet-backends/benchmarks/jvm-cpu-jmh/src/jmh/kotlin/sk/ainet/bench/spike/TensorViewSpikeBench.kt @@ -0,0 +1,197 @@ +package sk.ainet.bench.spike + +import org.openjdk.jmh.annotations.Benchmark +import org.openjdk.jmh.annotations.BenchmarkMode +import org.openjdk.jmh.annotations.Level +import org.openjdk.jmh.annotations.Mode +import org.openjdk.jmh.annotations.OutputTimeUnit +import org.openjdk.jmh.annotations.Param +import org.openjdk.jmh.annotations.Scope +import org.openjdk.jmh.annotations.Setup +import org.openjdk.jmh.annotations.State +import org.openjdk.jmh.infra.Blackhole +import java.lang.foreign.Arena +import java.lang.foreign.MemorySegment +import java.lang.foreign.ValueLayout +import java.util.concurrent.TimeUnit + +/** + * SKEEP-003 Phase-2 spike (decision #6, issue #1016): what does it cost to put a `TensorView` + * (shape + layout + storage) between a kernel and its bytes? + * + * A throw-away model of the proposed types — `SpikeStorage` (sealed: heap `FloatArray` or off-heap + * `MemorySegment`) and `SpikeView` (offset/length over a storage, element `get`/`set` that + * dispatches on the storage kind, plus the "unwrap once per call" fast path kernels are expected to + * use). Each kernel is measured against its raw-array baseline: + * + * - elementwise `c = a + b` over 1 M floats — budget ≤ 3 % (decision #6); + * - `gemv` 256 × 1024 (dot products, the memory-bound decode shape) — budget: within noise. + * + * Variants: `raw` (FloatArray baseline) · `viewHeapGet` (per-element get/set through the view over + * heap storage) · `viewHeapUnwrap` (view.asHeapArray() once, then raw loop) · + * `viewOffHeapGet` (per-element through MemorySegment) · `viewOffHeapUnwrap` (segment fetched once, + * element access via ValueLayout). + */ +@State(Scope.Benchmark) +@BenchmarkMode(Mode.AverageTime) +@OutputTimeUnit(TimeUnit.MICROSECONDS) +open class TensorViewSpikeBench { + + sealed interface SpikeStorage { + val size: Int + class Heap(val array: FloatArray) : SpikeStorage { override val size: Int get() = array.size } + class OffHeap(val segment: MemorySegment) : SpikeStorage { override val size: Int get() = (segment.byteSize() / 4).toInt() } + } + + /** Shape + layout (offset, contiguous) + storage; never owns bytes. */ + class SpikeView(val storage: SpikeStorage, val offset: Int, val length: Int) { + fun get(i: Int): Float = when (val s = storage) { + is SpikeStorage.Heap -> s.array[offset + i] + is SpikeStorage.OffHeap -> s.segment.getAtIndex(ValueLayout.JAVA_FLOAT, (offset + i).toLong()) + } + fun set(i: Int, v: Float) { when (val s = storage) { + is SpikeStorage.Heap -> s.array[offset + i] = v + is SpikeStorage.OffHeap -> s.segment.setAtIndex(ValueLayout.JAVA_FLOAT, (offset + i).toLong(), v) + } } + /** The fast path a kernel takes once per call: the backing array, or null for off-heap. */ + fun asHeapArray(): FloatArray? = (storage as? SpikeStorage.Heap)?.array + fun segment(): MemorySegment? = (storage as? SpikeStorage.OffHeap)?.segment + } + + @Param("1000000") + var n: Int = 1_000_000 + + private val rows = 256 + private val cols = 1024 + + // raw + private lateinit var a: FloatArray + private lateinit var b: FloatArray + private lateinit var c: FloatArray + private lateinit var w: FloatArray + private lateinit var x: FloatArray + private lateinit var y: FloatArray + // views over heap + private lateinit var va: SpikeView + private lateinit var vb: SpikeView + private lateinit var vc: SpikeView + private lateinit var vw: SpikeView + private lateinit var vx: SpikeView + private lateinit var vy: SpikeView + // views over off-heap + private lateinit var oa: SpikeView + private lateinit var ob: SpikeView + private lateinit var oc: SpikeView + private lateinit var ow: SpikeView + private lateinit var ox: SpikeView + private lateinit var oy: SpikeView + private val arena: Arena = Arena.ofShared() + + @Setup(Level.Trial) + fun setup() { + a = FloatArray(n) { (it % 97) * 0.01f }; b = FloatArray(n) { (it % 89) * 0.02f }; c = FloatArray(n) + w = FloatArray(rows * cols) { (it % 31) * 0.001f }; x = FloatArray(cols) { (it % 17) * 0.05f }; y = FloatArray(rows) + va = SpikeView(SpikeStorage.Heap(a), 0, n); vb = SpikeView(SpikeStorage.Heap(b), 0, n); vc = SpikeView(SpikeStorage.Heap(c), 0, n) + vw = SpikeView(SpikeStorage.Heap(w), 0, rows * cols); vx = SpikeView(SpikeStorage.Heap(x), 0, cols); vy = SpikeView(SpikeStorage.Heap(y), 0, rows) + fun seg(src: FloatArray): MemorySegment { val s = arena.allocate(src.size * 4L, 64); MemorySegment.copy(src, 0, s, ValueLayout.JAVA_FLOAT, 0, src.size); return s } + oa = SpikeView(SpikeStorage.OffHeap(seg(a)), 0, n); ob = SpikeView(SpikeStorage.OffHeap(seg(b)), 0, n); oc = SpikeView(SpikeStorage.OffHeap(seg(c)), 0, n) + ow = SpikeView(SpikeStorage.OffHeap(seg(w)), 0, rows * cols); ox = SpikeView(SpikeStorage.OffHeap(seg(x)), 0, cols); oy = SpikeView(SpikeStorage.OffHeap(seg(y)), 0, rows) + } + + // ---------------- elementwise add, 1 M ---------------- + + @Benchmark fun add_raw(bh: Blackhole) { + val a = a; val b = b; val c = c + for (i in 0 until n) c[i] = a[i] + b[i] + bh.consume(c) + } + + /** Control: raw arrays with loop-invariant offsets (what the unwrap path compiles to) — isolates the offset cost from the view. */ + @Benchmark fun add_rawOffset(bh: Blackhole) { + val a = a; val b = b; val c = c + val oa = va.offset; val ob = vb.offset; val oc = vc.offset + for (i in 0 until n) c[oc + i] = a[oa + i] + b[ob + i] + bh.consume(c) + } + + @Benchmark fun add_viewHeapGet(bh: Blackhole) { + val va = va; val vb = vb; val vc = vc + for (i in 0 until n) vc.set(i, va.get(i) + vb.get(i)) + bh.consume(vc) + } + + @Benchmark fun add_viewHeapUnwrap(bh: Blackhole) { + val a = va.asHeapArray()!!; val b = vb.asHeapArray()!!; val c = vc.asHeapArray()!! + val oa = va.offset; val ob = vb.offset; val oc = vc.offset + for (i in 0 until n) c[oc + i] = a[oa + i] + b[ob + i] + bh.consume(c) + } + + @Benchmark fun add_viewOffHeapGet(bh: Blackhole) { + val va = oa; val vb = ob; val vc = oc + for (i in 0 until n) vc.set(i, va.get(i) + vb.get(i)) + bh.consume(vc) + } + + @Benchmark fun add_viewOffHeapUnwrap(bh: Blackhole) { + val sa = oa.segment()!!; val sb = ob.segment()!!; val sc = oc.segment()!! + for (i in 0 until n) { + val l = i.toLong() + sc.setAtIndex(ValueLayout.JAVA_FLOAT, l, sa.getAtIndex(ValueLayout.JAVA_FLOAT, l) + sb.getAtIndex(ValueLayout.JAVA_FLOAT, l)) + } + bh.consume(sc) + } + + // ---------------- gemv 256 × 1024 ---------------- + + @Benchmark fun gemv_raw(bh: Blackhole) { + val w = w; val x = x; val y = y + for (r in 0 until rows) { + var acc = 0f; val base = r * cols + for (k in 0 until cols) acc += w[base + k] * x[k] + y[r] = acc + } + bh.consume(y) + } + + @Benchmark fun gemv_viewHeapGet(bh: Blackhole) { + val vw = vw; val vx = vx; val vy = vy + for (r in 0 until rows) { + var acc = 0f; val base = r * cols + for (k in 0 until cols) acc += vw.get(base + k) * vx.get(k) + vy.set(r, acc) + } + bh.consume(vy) + } + + @Benchmark fun gemv_viewHeapUnwrap(bh: Blackhole) { + val w = vw.asHeapArray()!!; val x = vx.asHeapArray()!!; val y = vy.asHeapArray()!! + val ow = vw.offset; val ox = vx.offset; val oy = vy.offset + for (r in 0 until rows) { + var acc = 0f; val base = ow + r * cols + for (k in 0 until cols) acc += w[base + k] * x[ox + k] + y[oy + r] = acc + } + bh.consume(y) + } + + @Benchmark fun gemv_viewOffHeapGet(bh: Blackhole) { + val vw = ow; val vx = ox; val vy = oy + for (r in 0 until rows) { + var acc = 0f; val base = r * cols + for (k in 0 until cols) acc += vw.get(base + k) * vx.get(k) + vy.set(r, acc) + } + bh.consume(vy) + } + + @Benchmark fun gemv_viewOffHeapUnwrap(bh: Blackhole) { + val sw = ow.segment()!!; val sx = ox.segment()!!; val sy = oy.segment()!! + for (r in 0 until rows) { + var acc = 0f; val base = (r * cols).toLong() + for (k in 0 until cols) acc += sw.getAtIndex(ValueLayout.JAVA_FLOAT, base + k) * sx.getAtIndex(ValueLayout.JAVA_FLOAT, k.toLong()) + sy.setAtIndex(ValueLayout.JAVA_FLOAT, r.toLong(), acc) + } + bh.consume(sy) + } +}