From 3c0ee71d285d4de5637d1fbccf154522789a53c2 Mon Sep 17 00:00:00 2001 From: Michal Harakal Date: Sat, 22 Aug 2026 21:55:02 +0200 Subject: [PATCH] docs(memory): JMH benchmark baseline 2026-08-22 for the SKEEP-003 migration (#1005 follow-up) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Reference numbers recorded on develop @ 5611b918 before the M1 Storage / TensorView façade and registry-dispatch slices: the jvm-cpu-jmh suite (MatmulBench, KernelMatmulBench, QuantizedMatmulBench, ElementwiseAdd1MBench, Reductions1MBench; fork 1, 3 warmup, 5 measurement iterations), run with nothing else building on the machine. Later slices compare against this table (decision #6: matmul within noise, elementwise <= 3 %). The kotlinx-benchmark StorageBenchmarks suite is not included: with the module's default iteration settings a full run takes hours; run it filtered per class when a slice touches the storage hot paths. Refs #1005 Co-Authored-By: Claude Fable 5 --- docs/design/memory/baseline-2026-08-22.md | 38 +++++++++++++++++++++++ 1 file changed, 38 insertions(+) create mode 100644 docs/design/memory/baseline-2026-08-22.md diff --git a/docs/design/memory/baseline-2026-08-22.md b/docs/design/memory/baseline-2026-08-22.md new file mode 100644 index 000000000..b3f149ed1 --- /dev/null +++ b/docs/design/memory/baseline-2026-08-22.md @@ -0,0 +1,38 @@ +# Benchmark baseline — 2026-08-22 (SKEEP-003 S0.9) + +Reference numbers recorded **before** the memory-architecture migration (M1 `Storage`/`TensorView` façades, registry dispatch), to compare against in later slices (decision #6 budget: matmul within noise, elementwise ≤ 3 %). + +- Commit: `5611b918` (develop + golden parity gate) +- Machine: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz · Linux-6.8.0-60-generic-x86_64-with-glibc2.34 · openjdk version "25.0.3" 2026-04-21 +- Suite: `./gradlew :skainet-backends:benchmarks:jvm-cpu-jmh:jmh` (JMH, fork 1, 3 warmup, 5 measurement iterations, `--enable-preview --add-modules jdk.incubator.vector`), run with nothing else building on the machine. +- Not included: `:skainet-lang:skainet-lang-core:jvmBenchmark` (kotlinx-benchmark `StorageBenchmarks`/`TensorBenchmarks`/`TurboQuantBenchmarks`) — with the module's default iteration settings a full run takes hours; run it filtered per class when a slice touches the storage hot paths and attach the numbers to that PR. + +| Benchmark | Params | Mode | Cnt | Score | ± | Units | +|---|---|---|---|---:|---:|---| +| `add_1M_fp32`
ElementwiseAdd1MBench | vectorEnabled=true | thrpt | 5 | 1.705 | 0.030 | ops/ms | +| `add_1M_fp32`
ElementwiseAdd1MBench | vectorEnabled=false | thrpt | 5 | 1.217 | 0.011 | ops/ms | +| `mean_1M_fp32`
Reductions1MBench | vectorEnabled=true | thrpt | 5 | 1.026 | 0.011 | ops/ms | +| `mean_1M_fp32`
Reductions1MBench | vectorEnabled=false | thrpt | 5 | 1.026 | 0.004 | ops/ms | +| `sum_1M_fp32`
Reductions1MBench | vectorEnabled=true | thrpt | 5 | 1.026 | 0.003 | ops/ms | +| `sum_1M_fp32`
Reductions1MBench | vectorEnabled=false | thrpt | 5 | 1.027 | 0.006 | ops/ms | +| `matmul_fp32_square`
KernelMatmulBench | provider=scalar, size=256 | avgt | 5 | 22.858 | 1.734 | ms/op | +| `matmul_fp32_square`
KernelMatmulBench | provider=scalar, size=512 | avgt | 5 | 207.298 | 0.961 | ms/op | +| `matmul_fp32_square`
KernelMatmulBench | provider=scalar, size=1024 | avgt | 5 | 1685.182 | 88.376 | ms/op | +| `matmul_fp32_square`
KernelMatmulBench | provider=panama, size=256 | avgt | 5 | 1.186 | 0.008 | ms/op | +| `matmul_fp32_square`
KernelMatmulBench | provider=panama, size=512 | avgt | 5 | 9.157 | 0.227 | ms/op | +| `matmul_fp32_square`
KernelMatmulBench | provider=panama, size=1024 | avgt | 5 | 80.878 | 0.374 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=true, size=256, vectorEnabled=true | avgt | 5 | 1.226 | 0.010 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=true, size=256, vectorEnabled=false | avgt | 5 | 21.129 | 1.062 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=true, size=512, vectorEnabled=true | avgt | 5 | 9.216 | 0.094 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=true, size=512, vectorEnabled=false | avgt | 5 | 209.844 | 0.821 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=true, size=1024, vectorEnabled=true | avgt | 5 | 82.036 | 2.492 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=true, size=1024, vectorEnabled=false | avgt | 5 | 1709.814 | 5.381 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=false, size=256, vectorEnabled=true | avgt | 5 | 1.225 | 0.043 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=false, size=256, vectorEnabled=false | avgt | 5 | 20.479 | 0.784 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=false, size=512, vectorEnabled=true | avgt | 5 | 9.371 | 0.162 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=false, size=512, vectorEnabled=false | avgt | 5 | 199.811 | 46.629 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=false, size=1024, vectorEnabled=true | avgt | 5 | 82.912 | 0.778 | ms/op | +| `matmul_fp32_square`
MatmulBench | blasEnabled=false, size=1024, vectorEnabled=false | avgt | 5 | 1757.350 | 126.247 | ms/op | +| `matmul_q4k_panama`
QuantizedMatmulBench | shape=1024-1024 | avgt | 5 | 0.112 | 0.017 | ms/op | +| `matmul_q4k_panama`
QuantizedMatmulBench | shape=4096-1024 | avgt | 5 | 0.336 | 0.003 | ms/op | +| `matmul_q4k_panama`
QuantizedMatmulBench | shape=4096-4096 | avgt | 5 | 1.223 | 0.017 | ms/op |