Various kernel opts

[Human9000-bit]

• prefetch data into cpu cache aot

... More to come

[Human9000-bit]

/bench

[github-actions]

ONNX CPU benchmark

GitHub-hosted runner signal, advisory only. Base and head are compared within the same architecture job.

arch	case	base p50	head p50	delta
aarch64	gemm_relu_256x512x512	1.18 ms	1.19 ms	+1.4% slower
aarch64	mobilenet_v3_small_224	3.60 ms	3.93 ms	+9.2% slower
aarch64	resnet18_224	55.50 ms	55.20 ms	-0.5% flat
aarch64	small_conv_relu_64	201 us	259 us	+28.9% slower
aarch64	small_dw_pw_64	191 us	176 us	-7.9% faster
aarch64	small_gemm_relu_1x128	7 us	8 us	+14.3% slower
aarch64	small_residual_conv_64	443 us	594 us	+34.1% slower
aarch64	winograd_3x3_20x20_c256	3.41 ms	3.48 ms	+2.1% slower
aarch64	winograd_3x3_80x80_c64	6.23 ms	6.23 ms	+0.1% flat
aarch64	yolo11n_bus_640	84.08 ms	84.63 ms	+0.7% flat
aarch64	yolov8n_bus_640	94.40 ms	93.75 ms	-0.7% flat
x86_64	gemm_relu_256x512x512	1.01 ms	1.00 ms	-1.1% faster
x86_64	mobilenet_v3_small_224	5.89 ms	5.88 ms	-0.2% flat
x86_64	resnet18_224	43.22 ms	43.53 ms	+0.7% flat
x86_64	small_conv_relu_64	411 us	434 us	+5.6% slower
x86_64	small_dw_pw_64	162 us	150 us	-7.4% faster
x86_64	small_gemm_relu_1x128	7 us	13 us	+85.7% slower
x86_64	small_residual_conv_64	452 us	458 us	+1.3% slower
x86_64	winograd_3x3_20x20_c256	5.79 ms	5.67 ms	-2.0% faster
x86_64	winograd_3x3_80x80_c64	7.23 ms	7.15 ms	-1.1% faster
x86_64	yolo11n_bus_640	97.23 ms	97.67 ms	+0.4% flat
x86_64	yolov8n_bus_640	105.89 ms	108.96 ms	+2.9% slower

Head op/kernel summary

arch	case	ops	dispatched kernels
aarch64	gemm_relu_256x512x512	Gemm x1, Relu x1	Gemm via blocked-mr4 x1
aarch64	mobilenet_v3_small_224	Conv x31, HardSigmoid x28, Mul x28, Conv_Relu x10, GlobalAveragePool x10, Conv_Add_fused x5, +3 more	Conv via nhwc-gemm-prepacked/pw-gemm x21, Conv_Relu via nhwc-gemm-prepacked/pw-gemm x9, Conv via dw-nhwc-padded/dw-neon x8, Gemm via row-gemm x2, Conv via nhwc-gemm-prepacked/pw-16x16-direct x1, Conv via nhwc-padded/first-layer-rgb x1, +1 more
aarch64	resnet18_224	Conv_Relu x9, Conv_Add_Relu x8, Conv x3, Flatten x1, Gemm x1, GlobalAveragePool x1, +1 more	Conv_Relu via nhwc-padded/im2col-gemm x9, Conv_Add_Relu via nhwc-padded/im2col-gemm x5, Conv via nhwc-padded/im2col-gemm x3, Conv_Add_Relu via nhwc-gemm-prepacked/row-fma x3, Gemm via row-gemm x1
aarch64	small_conv_relu_64	Conv_Relu x1	Conv_Relu via nhwc-padded/im2col-gemm x1
aarch64	small_dw_pw_64	FusedDwPw x1	-
aarch64	small_gemm_relu_1x128	Gemm x1, Relu x1	Gemm via row-gemm x1
aarch64	small_residual_conv_64	Conv_Add_Relu x1	Conv_Add_Relu via nhwc-padded/im2col-gemm x1
aarch64	winograd_3x3_20x20_c256	Conv_Relu x1	Conv_Relu via nhwc-padded/im2col-gemm x1
aarch64	winograd_3x3_80x80_c64	Conv_Relu x1	Conv_Relu via nhwc-padded/im2col-gemm x1
aarch64	yolo11n_bus_640	Conv x85, Mul x83, Sigmoid x78, Concat x39, Add x22, Reshape x17, +18 more	Conv via nhwc-gemm-prepacked/pw-gemm x44, Conv via nhwc-padded/im2col-gemm x34, Conv via dw-nhwc-padded/dw-neon x6, MatMul via blocked-mr8 x2, Conv via nhwc-padded/first-layer-rgb x1, Conv_Add via dw-nhwc-padded/dw-neon x1
aarch64	yolov8n_bus_640	Conv x64, Mul x61, Sigmoid x58, Concat x33, Add x17, Reshape x14, +15 more	Conv via nhwc-padded/im2col-gemm x38, Conv via nhwc-gemm-prepacked/pw-gemm x25, Conv via nhwc-padded/first-layer-rgb x1
x86_64	gemm_relu_256x512x512	Gemm x1, Relu x1	Gemm via blocked-mr4 x1
x86_64	mobilenet_v3_small_224	Conv x31, HardSigmoid x28, Mul x28, Conv_Relu x10, GlobalAveragePool x10, Conv_Add_fused x5, +3 more	Conv via nhwc-gemm-prepacked/pw-gemm x20, Conv_Relu via nhwc-gemm-prepacked/pw-gemm x9, Conv via dw-nhwc-padded/dw-avx-fma x8, Gemm via row-gemm x2, Conv via nhwc-gemm-prepacked/pw-16x16-direct x1, Conv via nhwc-gemm-prepacked/pw-nx16-direct x1, +2 more
x86_64	resnet18_224	Conv_Relu x9, Conv_Add_Relu x8, Conv x3, Flatten x1, Gemm x1, GlobalAveragePool x1, +1 more	Conv_Relu via nhwc-padded/im2col-gemm x6, Conv_Add_Relu via nhwc-gemm-prepacked/row-fma x3, Conv_Add_Relu via nhwc-padded/winograd-3x3 x3, Conv_Relu via nhwc-padded/winograd-3x3 x3, Conv via nhwc-padded/im2col-gemm x2, Conv_Add_Relu via nhwc-padded/im2col-gemm x2, +2 more
x86_64	small_conv_relu_64	Conv_Relu x1	Conv_Relu via nhwc-padded/winograd-3x3 x1
x86_64	small_dw_pw_64	FusedDwPw x1	-
x86_64	small_gemm_relu_1x128	Gemm x1, Relu x1	Gemm via row-gemm x1
x86_64	small_residual_conv_64	Conv_Add_Relu x1	Conv_Add_Relu via nhwc-padded/winograd-3x3 x1
x86_64	winograd_3x3_20x20_c256	Conv_Relu x1	Conv_Relu via nhwc-padded/winograd-3x3 x1
x86_64	winograd_3x3_80x80_c64	Conv_Relu x1	Conv_Relu via nhwc-padded/winograd-3x3 x1
x86_64	yolo11n_bus_640	Conv x85, Mul x83, Sigmoid x78, Concat x39, Add x22, Reshape x17, +18 more	Conv via nhwc-padded/winograd-3x3 x28, Conv via nhwc-gemm-prepacked/pw-gemm x24, Conv via nhwc-gemm-prepacked/pw-nx16-direct x20, Conv via dw-nhwc-padded/dw-avx-fma x6, Conv via nhwc-padded/im2col-gemm x6, MatMul via blocked-mr4 x2, +2 more
x86_64	yolov8n_bus_640	Conv x64, Mul x61, Sigmoid x58, Concat x33, Add x17, Reshape x14, +15 more	Conv via nhwc-padded/winograd-3x3 x32, Conv via nhwc-gemm-prepacked/pw-gemm x17, Conv via nhwc-gemm-prepacked/pw-nx16-direct x8, Conv via nhwc-padded/im2col-gemm x6, Conv via nhwc-padded/first-layer-rgb x1

Raw JSON artifacts include per-run min/p50/avg/p95/p99, optimized node counts, graph-cost summaries, yscv CPU dispatch report, and per-node runner profile summaries.