Skip to content

Launch @kernel kernels on N-d grids, indexing in 32 bits - #797

Merged
maleadt merged 2 commits into
tb/ki-0.3from
tb/ndlaunch
Sep 30, 2026
Merged

Launch @kernel kernels on N-d grids, indexing in 32 bits#797
maleadt merged 2 commits into
tb/ki-0.3from
tb/ndlaunch

Conversation

@maleadt

@maleadt maleadt commented Sep 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

@index assumes that a kernel was launched on a 1-D grid. To get Cartesian indices, every work-item decomposes its linear group and local id with 64-bit integer divisions: 2(N−1) of them for a dynamically-sized N-d ndrange, and the same again for the bounds check (#470, #396). On GPUs, that is often most of a kernel's index arithmetic.

This PR lets a backend launch a kernel on a grid with the shape of the iteration space, and lets @index compute in 32 bits. With the CUDA.jl side of it (JuliaGPU/CUDA.jl#3304), on an RTX 5080, compared to the KA 0.10 port (JuliaGPU/CUDA.jl#3302), both with the inlining fix from #798:

kernel before after
broadcast 3-D F32 A .* v (256³) 163 µs 80 µs (−51%)
broadcast 3-D Int8 A .+ v 633 µs 296 µs (−53%)
broadcast 4-D F32 A .* v 255 µs 191 µs (−25%)
permutedims 3-D 275 µs 230 µs (−16%)
KA 3-D Cartesian kernel, dynamic ndrange 179 µs 161 µs (−10%)
KA 4-D Cartesian kernel, dynamic ndrange 249 µs 192 µs (−23%)

1-D and memory-bound 2-D kernels don't change. Oceananigans runs 1–3% faster per F32 time step. There are more numbers in the CUDA.jl PR.

How it works

The hidden context argument of a @kernel (CompilerMetadata) gets a launch field that says how the backend launched the kernel, and the index functions specialize on it:

  • NDLaunch{T}: the grid has the shape of the iteration space, so the hardware group and local ids are the Cartesian positions and no divisions are needed. This works for as many dimensions as the hardware grid has (3, according to the KernelInterface limits); larger iteration spaces use a linear launch.
  • LinearLaunch{T}: the 1-D grid used so far.

Either way, @index computes in T: Int32 whenever the padded iteration space fits, Int otherwise. It still returns Ints. Linear indices keep their x-fastest order, so sub-groups still hold consecutive work-items.

KA.select_launch chooses the launch from the iteration space and the backend's limits (KI.max_work_group_size, KI.max_work_group_dims and KI.max_num_groups). When the workgroup size is tuned after compiling, the choice has to hold for every size that tuning can pick, because the launch is part of the context type and thus of the compiled kernel. It does, because tuning distributes work-items the same way select_launch assumes (KA.launch_workgroupsize). An iteration space with more than typemax(Int) work-items is now rejected with an ArgumentError.

Adopting this is optional for a backend: a context without a launch is indexed as before (now with the same code, as a LinearLaunch{Int}). PoCL adopts it here. docs/src/implementations.md describes what a backend's launch has to do; #801 then moves that launch into KA, so that backends don't have to.

Packages that customize the iteration space (a custom partition or expand, like Oceananigans) keep working: only the iteration spaces KA creates itself take the direct path, and the launch is chosen from the iteration space that is actually launched. An Adapt rule for CompilerMetadata has to pass the launch along, though.

The second commit also fixes two bugs in the default indexing path: 0-d ndranges threw a bounds error, and under --check-bounds=yes the index bounds checks crashed PoCL's compiler for kernels with @synchronize (pocl/pocl#2345).

@github-actions

github-actions Bot commented Sep 27, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmark Results

Show table
main 7509cf4... main / 7509cf4...
const/@Const/Float32/262144 0.175 ± 0.018 ms 0.165 ± 0.017 ms 1.07 ± 0.15
const/@Const/Float32/65536 0.0673 ± 0.02 ms 0.0654 ± 0.019 ms 1.03 ± 0.43
const/@Const/Float64/262144 0.271 ± 0.018 ms 0.264 ± 0.015 ms 1.03 ± 0.089
const/@Const/Float64/65536 0.102 ± 0.019 ms 0.107 ± 0.015 ms 0.951 ± 0.22
const/unmarked/Float32/262144 0.266 ± 0.02 ms 0.253 ± 0.016 ms 1.05 ± 0.1
const/unmarked/Float32/65536 0.0951 ± 0.018 ms 0.0882 ± 0.017 ms 1.08 ± 0.29
const/unmarked/Float64/262144 0.464 ± 0.029 ms 0.455 ± 0.026 ms 1.02 ± 0.087
const/unmarked/Float64/65536 0.143 ± 0.017 ms 0.143 ± 0.019 ms 0.999 ± 0.18
launch/3D static workgroup, dynamic ndrange 20.1 ± 19 μs 25.8 ± 25 μs 0.778 ± 1.1
launch/3D static workgroup, static ndrange 18.8 ± 13 μs 18.3 ± 11 μs 1.03 ± 0.95
launch/dynamic workgroup, dynamic ndrange 21.4 ± 18 μs 22.5 ± 22 μs 0.949 ± 1.2
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 22.8 ± 21 μs 21.5 ± 22 μs 1.06 ± 1.5
launch/static workgroup, dynamic ndrange 19.4 ± 15 μs 18.4 ± 18 μs 1.06 ± 1.3
launch/static workgroup, static ndrange 19.1 ± 13 μs 18.5 ± 16 μs 1.03 ± 1.1
partition/dynamic workgroup, dynamic ndrange 0.0454 ± 0.0063 μs 0.0439 ± 0.002 μs 1.03 ± 0.15
partition/static workgroup, dynamic ndrange 0.0535 ± 0.011 μs 0.0549 ± 0.01 μs 0.974 ± 0.26
partition/static workgroup, static ndrange 1.72 ± 0.049 ns 1.13 ± 0.004 ns 1.53 ± 0.044
saxpy/default/Float16/1024 25.3 ± 19 μs 24.1 ± 19 μs 1.05 ± 1.2
saxpy/default/Float16/1048576 0.338 ± 0.026 ms 0.229 ± 0.022 ms 1.48 ± 0.18
saxpy/default/Float16/16384 0.0487 ± 0.022 ms 0.0447 ± 0.022 ms 1.09 ± 0.72
saxpy/default/Float16/2048 25.8 ± 18 μs 24.9 ± 22 μs 1.04 ± 1.1
saxpy/default/Float16/256 21 ± 15 μs 19.9 ± 18 μs 1.05 ± 1.2
saxpy/default/Float16/262144 0.127 ± 0.029 ms 0.102 ± 0.026 ms 1.25 ± 0.43
saxpy/default/Float16/32768 0.0598 ± 0.032 ms 0.0494 ± 0.025 ms 1.21 ± 0.89
saxpy/default/Float16/4096 0.0347 ± 0.025 ms 28.9 ± 23 μs 1.2 ± 1.3
saxpy/default/Float16/512 20.8 ± 13 μs 20.3 ± 15 μs 1.03 ± 0.98
saxpy/default/Float16/64 20.4 ± 13 μs 20.1 ± 15 μs 1.02 ± 0.97
saxpy/default/Float16/65536 0.0612 ± 0.026 ms 0.0598 ± 0.027 ms 1.02 ± 0.64
saxpy/default/Float32/1024 20.5 ± 12 μs 20.5 ± 15 μs 1 ± 0.93
saxpy/default/Float32/1048576 0.295 ± 0.04 ms 0.291 ± 0.032 ms 1.01 ± 0.18
saxpy/default/Float32/16384 0.0463 ± 0.023 ms 0.0451 ± 0.024 ms 1.03 ± 0.75
saxpy/default/Float32/2048 22.3 ± 18 μs 21.5 ± 12 μs 1.04 ± 1
saxpy/default/Float32/256 20.6 ± 13 μs 20.2 ± 16 μs 1.02 ± 1
saxpy/default/Float32/262144 0.126 ± 0.026 ms 0.118 ± 0.025 ms 1.07 ± 0.31
saxpy/default/Float32/32768 0.0527 ± 0.03 ms 0.0454 ± 0.021 ms 1.16 ± 0.86
saxpy/default/Float32/4096 25.9 ± 23 μs 25.8 ± 27 μs 1 ± 1.4
saxpy/default/Float32/512 20.5 ± 8.5 μs 19.9 ± 11 μs 1.03 ± 0.73
saxpy/default/Float32/64 22.6 ± 20 μs 20 ± 16 μs 1.13 ± 1.3
saxpy/default/Float32/65536 0.0626 ± 0.02 ms 0.0546 ± 0.024 ms 1.14 ± 0.62
saxpy/default/Float64/1024 20.8 ± 10 μs 21.6 ± 16 μs 0.963 ± 0.85
saxpy/default/Float64/1048576 0.557 ± 0.064 ms 0.519 ± 0.07 ms 1.07 ± 0.19
saxpy/default/Float64/16384 0.0478 ± 0.023 ms 0.0475 ± 0.025 ms 1.01 ± 0.72
saxpy/default/Float64/2048 22.6 ± 12 μs 22.7 ± 18 μs 0.996 ± 0.93
saxpy/default/Float64/256 20.4 ± 9.9 μs 19.6 ± 14 μs 1.04 ± 0.89
saxpy/default/Float64/262144 0.19 ± 0.027 ms 0.181 ± 0.027 ms 1.05 ± 0.22
saxpy/default/Float64/32768 0.0567 ± 0.024 ms 0.0555 ± 0.025 ms 1.02 ± 0.63
saxpy/default/Float64/4096 0.0396 ± 0.033 ms 0.0405 ± 0.03 ms 0.978 ± 1.1
saxpy/default/Float64/512 20.5 ± 10 μs 20.2 ± 13 μs 1.02 ± 0.84
saxpy/default/Float64/64 20 ± 9.3 μs 20.8 ± 17 μs 0.962 ± 0.92
saxpy/default/Float64/65536 0.0824 ± 0.028 ms 0.0794 ± 0.027 ms 1.04 ± 0.5
saxpy/static workgroup=(1024,)/Float16/1024 26.4 ± 23 μs 23.3 ± 23 μs 1.14 ± 1.5
saxpy/static workgroup=(1024,)/Float16/1048576 0.332 ± 0.025 ms 0.235 ± 0.023 ms 1.42 ± 0.17
saxpy/static workgroup=(1024,)/Float16/16384 0.0468 ± 0.022 ms 0.0465 ± 0.021 ms 1.01 ± 0.65
saxpy/static workgroup=(1024,)/Float16/2048 24.8 ± 16 μs 24.7 ± 17 μs 1 ± 0.95
saxpy/static workgroup=(1024,)/Float16/256 26.7 ± 24 μs 25.6 ± 24 μs 1.04 ± 1.4
saxpy/static workgroup=(1024,)/Float16/262144 0.125 ± 0.027 ms 0.104 ± 0.027 ms 1.2 ± 0.4
saxpy/static workgroup=(1024,)/Float16/32768 0.0527 ± 0.027 ms 0.0501 ± 0.026 ms 1.05 ± 0.77
saxpy/static workgroup=(1024,)/Float16/4096 0.04 ± 0.025 ms 27.1 ± 20 μs 1.48 ± 1.4
saxpy/static workgroup=(1024,)/Float16/512 23.5 ± 24 μs 24 ± 24 μs 0.981 ± 1.4
saxpy/static workgroup=(1024,)/Float16/64 26 ± 24 μs 26.6 ± 24 μs 0.976 ± 1.3
saxpy/static workgroup=(1024,)/Float16/65536 0.0635 ± 0.026 ms 0.0643 ± 0.03 ms 0.987 ± 0.61
saxpy/static workgroup=(1024,)/Float32/1024 19.9 ± 14 μs 21.7 ± 15 μs 0.917 ± 0.89
saxpy/static workgroup=(1024,)/Float32/1048576 0.29 ± 0.037 ms 0.301 ± 0.035 ms 0.964 ± 0.17
saxpy/static workgroup=(1024,)/Float32/16384 0.0437 ± 0.021 ms 0.0456 ± 0.024 ms 0.957 ± 0.69
saxpy/static workgroup=(1024,)/Float32/2048 22.8 ± 17 μs 22.6 ± 14 μs 1.01 ± 0.96
saxpy/static workgroup=(1024,)/Float32/256 21 ± 21 μs 25 ± 24 μs 0.84 ± 1.2
saxpy/static workgroup=(1024,)/Float32/262144 0.125 ± 0.025 ms 0.122 ± 0.024 ms 1.03 ± 0.29
saxpy/static workgroup=(1024,)/Float32/32768 0.0466 ± 0.021 ms 0.045 ± 0.019 ms 1.04 ± 0.65
saxpy/static workgroup=(1024,)/Float32/4096 27.1 ± 23 μs 24.7 ± 25 μs 1.1 ± 1.5
saxpy/static workgroup=(1024,)/Float32/512 22.3 ± 22 μs 21.2 ± 21 μs 1.05 ± 1.5
saxpy/static workgroup=(1024,)/Float32/64 24.3 ± 24 μs 25.4 ± 25 μs 0.956 ± 1.3
saxpy/static workgroup=(1024,)/Float32/65536 0.0601 ± 0.018 ms 0.0554 ± 0.022 ms 1.08 ± 0.54
saxpy/static workgroup=(1024,)/Float64/1024 20.7 ± 13 μs 22 ± 18 μs 0.942 ± 0.96
saxpy/static workgroup=(1024,)/Float64/1048576 0.549 ± 0.065 ms 0.524 ± 0.081 ms 1.05 ± 0.2
saxpy/static workgroup=(1024,)/Float64/16384 0.0446 ± 0.019 ms 0.0444 ± 0.018 ms 1.01 ± 0.59
saxpy/static workgroup=(1024,)/Float64/2048 22.4 ± 15 μs 23.4 ± 20 μs 0.956 ± 1
saxpy/static workgroup=(1024,)/Float64/256 26.8 ± 25 μs 28.5 ± 23 μs 0.94 ± 1.2
saxpy/static workgroup=(1024,)/Float64/262144 0.183 ± 0.027 ms 0.185 ± 0.027 ms 0.986 ± 0.21
saxpy/static workgroup=(1024,)/Float64/32768 0.053 ± 0.019 ms 0.0521 ± 0.021 ms 1.02 ± 0.56
saxpy/static workgroup=(1024,)/Float64/4096 0.0379 ± 0.028 ms 0.0382 ± 0.028 ms 0.992 ± 1
saxpy/static workgroup=(1024,)/Float64/512 24.7 ± 24 μs 26.8 ± 24 μs 0.919 ± 1.2
saxpy/static workgroup=(1024,)/Float64/64 23.5 ± 24 μs 26.1 ± 24 μs 0.898 ± 1.2
saxpy/static workgroup=(1024,)/Float64/65536 0.0763 ± 0.024 ms 0.0757 ± 0.024 ms 1.01 ± 0.45
time_to_load 0.674 ± 0.0044 s 0.637 ± 0.016 s 1.06 ± 0.027
main 7509cf4... main / 7509cf4...
const/@Const/Float32/262144 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
const/@Const/Float32/65536 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
const/@Const/Float64/262144 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
const/@Const/Float64/65536 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
const/unmarked/Float32/262144 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
const/unmarked/Float32/65536 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
const/unmarked/Float64/262144 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
const/unmarked/Float64/65536 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
launch/3D static workgroup, dynamic ndrange 5 allocs: 0.141 kB 7 allocs: 0.203 kB 0.692
launch/3D static workgroup, static ndrange 6 allocs: 0.172 kB 6 allocs: 0.172 kB 1
launch/dynamic workgroup, dynamic ndrange 15 allocs: 0.312 kB 3 allocs: 0.0625 kB 5
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 1 allocs: 16 B 4 allocs: 0.0781 kB 0.2
launch/static workgroup, dynamic ndrange 1 allocs: 16 B 3 allocs: 0.0469 kB 0.333
launch/static workgroup, static ndrange 2 allocs: 0.0469 kB 2 allocs: 0.0469 kB 1
partition/dynamic workgroup, dynamic ndrange 2 allocs: 0.0625 kB 2 allocs: 0.0625 kB 1
partition/static workgroup, dynamic ndrange 2 allocs: 32 B 2 allocs: 32 B 1
partition/static workgroup, static ndrange 0 allocs: 0 B 0 allocs: 0 B
saxpy/default/Float16/1024 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float16/1048576 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float16/16384 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float16/2048 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float16/256 16 allocs: 0.328 kB 4 allocs: 0.0781 kB 4.2
saxpy/default/Float16/262144 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float16/32768 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float16/4096 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float16/512 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float16/64 16 allocs: 0.328 kB 4 allocs: 0.0781 kB 4.2
saxpy/default/Float16/65536 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/1024 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/1048576 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/16384 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/2048 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/256 16 allocs: 0.328 kB 4 allocs: 0.0781 kB 4.2
saxpy/default/Float32/262144 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/32768 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/4096 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/512 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float32/64 16 allocs: 0.328 kB 4 allocs: 0.0781 kB 4.2
saxpy/default/Float32/65536 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/1024 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/1048576 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/16384 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/2048 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/256 16 allocs: 0.328 kB 4 allocs: 0.0781 kB 4.2
saxpy/default/Float64/262144 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/32768 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/4096 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/512 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/default/Float64/64 16 allocs: 0.328 kB 4 allocs: 0.0781 kB 4.2
saxpy/default/Float64/65536 24 allocs: 0.453 kB 7 allocs: 0.125 kB 3.62
saxpy/static workgroup=(1024,)/Float16/1024 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float16/1048576 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float16/16384 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float16/2048 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float16/256 1 allocs: 16 B 4 allocs: 0.0625 kB 0.25
saxpy/static workgroup=(1024,)/Float16/262144 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float16/32768 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float16/4096 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float16/512 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float16/64 1 allocs: 16 B 4 allocs: 0.0625 kB 0.25
saxpy/static workgroup=(1024,)/Float16/65536 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/1024 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/1048576 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/16384 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/2048 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/256 1 allocs: 16 B 4 allocs: 0.0625 kB 0.25
saxpy/static workgroup=(1024,)/Float32/262144 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/32768 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/4096 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/512 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float32/64 1 allocs: 16 B 4 allocs: 0.0625 kB 0.25
saxpy/static workgroup=(1024,)/Float32/65536 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/1024 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/1048576 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/16384 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/2048 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/256 1 allocs: 16 B 4 allocs: 0.0625 kB 0.25
saxpy/static workgroup=(1024,)/Float64/262144 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/32768 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/4096 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/512 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
saxpy/static workgroup=(1024,)/Float64/64 1 allocs: 16 B 4 allocs: 0.0625 kB 0.25
saxpy/static workgroup=(1024,)/Float64/65536 4 allocs: 0.0625 kB 7 allocs: 0.109 kB 0.571
time_to_load 0.2 k allocs: 11.8 kB 0.2 k allocs: 11.8 kB 1

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@maleadt maleadt mentioned this pull request Sep 27, 2026
Open
@maleadt
maleadt force-pushed the tb/ndlaunch branch 4 times, most recently from a9b94b9 to 523886d Compare September 27, 2026 13:59
@maleadt maleadt mentioned this pull request Sep 27, 2026
Closed
vchuravy
vchuravy approved these changes Sep 27, 2026
View reviewed changes
Comment thread docs/src/implementations.md Outdated
Comment thread src/pocl/backend.jl Outdated
Comment thread src/launch.jl Outdated
@vchuravy vchuravy mentioned this pull request Sep 27, 2026
Closed
@christiangnrd

This comment was marked as resolved.

Sign in to view
@maleadt
maleadt force-pushed the tb/ndlaunch branch 2 times, most recently from ff75616 to b7cf4cb Compare September 27, 2026 20:12
@maleadt maleadt mentioned this pull request Sep 27, 2026
Merged
@maleadt
maleadt changed the base branch from main to tb/ki-limits September 27, 2026 20:17
@maleadt
maleadt added this pull request to stack #799 September 27, 2026 20:17
@maleadt

This comment was marked as resolved.

Sign in to view
Base automatically changed from tb/ki-limits to main September 28, 2026 04:41
@maleadt maleadt mentioned this pull request Sep 28, 2026
Merged
@maleadt
maleadt force-pushed the tb/ndlaunch branch 2 times, most recently from f1cccb3 to 5e0f16e Compare September 28, 2026 09:18
@maleadt maleadt mentioned this pull request Sep 28, 2026
Merged
@maleadt
maleadt removed this pull request from stack #799 September 28, 2026 09:19
@maleadt
maleadt changed the base branch from main to tb/ki-0.3 September 28, 2026 09:19
@maleadt
maleadt added this pull request to stack #802 September 28, 2026 09:19
@maleadt
maleadt force-pushed the tb/ndlaunch branch 2 times, most recently from 5b9fb86 to bdf401d Compare September 28, 2026 10:09
@maleadt
maleadt added this pull request to stack #804 September 28, 2026 10:15
@maleadt
maleadt removed this pull request from stack #804 September 29, 2026 20:22
@maleadt
maleadt added this pull request to stack #808 September 29, 2026 20:22
@maleadt
maleadt force-pushed the tb/ndlaunch branch 3 times, most recently from a5a53b9 to 8f123c8 Compare September 30, 2026 08:16
@maleadt
Launch @kernel kernels on N-d grids, indexing in 32 bits
e883ad8 
`@index` assumes that a kernel was launched on a 1-D grid, so every work-item
decomposes its linear group and local id into Cartesian positions, with 64-bit
integer divisions for a dynamically-sized ndrange.

Backends can now record how they launched a kernel in its context, and `@index`
specializes on that:

- `NDLaunch{T}`: the grid has the shape of the iteration space (for as many
  dimensions as the backend's grid has), so the hardware ids are the Cartesian
  positions and no divisions are needed;
- `LinearLaunch{T}`: the 1-D grid used so far.

Either way `@index` computes in `T`, `Int32` whenever the padded iteration
space fits, and still returns `Int`s.

`select_launch` chooses the launch from the iteration space and the backend's
limits. When the workgroup size will be tuned, the choice holds for every
workgroup size tuning can pick, so the context type (and thus the compiled
kernel) doesn't change. Contexts without a launch keep the existing behavior,
so backends opt in; `__validindex` is now generic, dispatching on the launch.
PoCL launches on N-d grids.
@maleadt
Index kernels without a launch configuration like a LinearLaunch{Int}
7509cf4 
The default path decomposed the hardware ids by indexing `blocks(iterspace)` and
`workitems(iterspace)`, which throws for 0-d ndranges (`linear_index` reads
`I.I[0]`), and which under `--check-bounds=yes` puts bounds checks depending on
the local id in front of `@synchronize`. POCL crashes on the resulting barrier
in non-uniform control flow, e.g. for a 3-D ndrange with partial workgroups.
Computing the positions like a `LinearLaunch{Int}` avoids both, and leaves
only one implementation of the index functions.
@maleadt maleadt mentioned this pull request Sep 30, 2026
Open
@maleadt
maleadt merged commit 0366b82 into main Sep 30, 2026
68 of 69 checks passed
@maleadt
maleadt deleted the tb/ndlaunch branch September 30, 2026 11:33
This was referenced Sep 30, 2026
Open
Open
Open
Open
Open
Open
Open
Open
@vchuravy vchuravy mentioned this pull request Oct 3, 2026
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@vchuravy vchuravy vchuravy approved these changes

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@maleadt @christiangnrd @vchuravy