Skip to content
Closed
38 changes: 38 additions & 0 deletions docs/design/memory/baseline-2026-08-22.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
# Benchmark baseline — 2026-08-22 (SKEEP-003 S0.9)

Reference numbers recorded **before** the memory-architecture migration (M1 `Storage`/`TensorView` façades, registry dispatch), to compare against in later slices (decision #6 budget: matmul within noise, elementwise ≤ 3 %).

- Commit: `5611b918` (develop + golden parity gate)
- Machine: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz · Linux-6.8.0-60-generic-x86_64-with-glibc2.34 · openjdk version "25.0.3" 2026-04-21
- Suite: `./gradlew :skainet-backends:benchmarks:jvm-cpu-jmh:jmh` (JMH, fork 1, 3 warmup, 5 measurement iterations, `--enable-preview --add-modules jdk.incubator.vector`), run with nothing else building on the machine.
- Not included: `:skainet-lang:skainet-lang-core:jvmBenchmark` (kotlinx-benchmark `StorageBenchmarks`/`TensorBenchmarks`/`TurboQuantBenchmarks`) — with the module's default iteration settings a full run takes hours; run it filtered per class when a slice touches the storage hot paths and attach the numbers to that PR.

| Benchmark | Params | Mode | Cnt | Score | ± | Units |
|---|---|---|---|---:|---:|---|
| `add_1M_fp32` <br><sub>ElementwiseAdd1MBench</sub> | vectorEnabled=true | thrpt | 5 | 1.705 | 0.030 | ops/ms |
| `add_1M_fp32` <br><sub>ElementwiseAdd1MBench</sub> | vectorEnabled=false | thrpt | 5 | 1.217 | 0.011 | ops/ms |
| `mean_1M_fp32` <br><sub>Reductions1MBench</sub> | vectorEnabled=true | thrpt | 5 | 1.026 | 0.011 | ops/ms |
| `mean_1M_fp32` <br><sub>Reductions1MBench</sub> | vectorEnabled=false | thrpt | 5 | 1.026 | 0.004 | ops/ms |
| `sum_1M_fp32` <br><sub>Reductions1MBench</sub> | vectorEnabled=true | thrpt | 5 | 1.026 | 0.003 | ops/ms |
| `sum_1M_fp32` <br><sub>Reductions1MBench</sub> | vectorEnabled=false | thrpt | 5 | 1.027 | 0.006 | ops/ms |
| `matmul_fp32_square` <br><sub>KernelMatmulBench</sub> | provider=scalar, size=256 | avgt | 5 | 22.858 | 1.734 | ms/op |
| `matmul_fp32_square` <br><sub>KernelMatmulBench</sub> | provider=scalar, size=512 | avgt | 5 | 207.298 | 0.961 | ms/op |
| `matmul_fp32_square` <br><sub>KernelMatmulBench</sub> | provider=scalar, size=1024 | avgt | 5 | 1685.182 | 88.376 | ms/op |
| `matmul_fp32_square` <br><sub>KernelMatmulBench</sub> | provider=panama, size=256 | avgt | 5 | 1.186 | 0.008 | ms/op |
| `matmul_fp32_square` <br><sub>KernelMatmulBench</sub> | provider=panama, size=512 | avgt | 5 | 9.157 | 0.227 | ms/op |
| `matmul_fp32_square` <br><sub>KernelMatmulBench</sub> | provider=panama, size=1024 | avgt | 5 | 80.878 | 0.374 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=true, size=256, vectorEnabled=true | avgt | 5 | 1.226 | 0.010 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=true, size=256, vectorEnabled=false | avgt | 5 | 21.129 | 1.062 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=true, size=512, vectorEnabled=true | avgt | 5 | 9.216 | 0.094 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=true, size=512, vectorEnabled=false | avgt | 5 | 209.844 | 0.821 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=true, size=1024, vectorEnabled=true | avgt | 5 | 82.036 | 2.492 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=true, size=1024, vectorEnabled=false | avgt | 5 | 1709.814 | 5.381 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=false, size=256, vectorEnabled=true | avgt | 5 | 1.225 | 0.043 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=false, size=256, vectorEnabled=false | avgt | 5 | 20.479 | 0.784 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=false, size=512, vectorEnabled=true | avgt | 5 | 9.371 | 0.162 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=false, size=512, vectorEnabled=false | avgt | 5 | 199.811 | 46.629 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=false, size=1024, vectorEnabled=true | avgt | 5 | 82.912 | 0.778 | ms/op |
| `matmul_fp32_square` <br><sub>MatmulBench</sub> | blasEnabled=false, size=1024, vectorEnabled=false | avgt | 5 | 1757.350 | 126.247 | ms/op |
| `matmul_q4k_panama` <br><sub>QuantizedMatmulBench</sub> | shape=1024-1024 | avgt | 5 | 0.112 | 0.017 | ms/op |
| `matmul_q4k_panama` <br><sub>QuantizedMatmulBench</sub> | shape=4096-1024 | avgt | 5 | 0.336 | 0.003 | ms/op |
| `matmul_q4k_panama` <br><sub>QuantizedMatmulBench</sub> | shape=4096-4096 | avgt | 5 | 1.223 | 0.017 | ms/op |
Loading