Skip to content

CI: soft-fail back ends that can't be installed, test OpenCL.jl only on Buildkite - #803

Merged
maleadt merged 1 commit into
mainfrom
tb/ci-reverse
Sep 30, 2026
Merged

CI: soft-fail back ends that can't be installed, test OpenCL.jl only on Buildkite#803
maleadt merged 1 commit into
mainfrom
tb/ci-reverse

Conversation

@maleadt

@maleadt maleadt commented Sep 28, 2026

Copy link
Copy Markdown
Member

Two changes to how KA tests the back ends, so that a KA PR that back ends don't support yet (like KernelInterface 0.3, #800) shows back-end failures as soft failures instead of red CI:

  • Like GPUCompiler's downstream tests, a Buildkite back-end step exits with status 3, which is a soft failure, when the back end's branch can't be installed together with this KernelAbstractions. Before, only developing KernelAbstractions itself did that; a back-end branch requiring another KernelInterface failed the build. Test failures still fail it.
  • OpenCL.jl is only tested on Buildkite now. Its PoCL tests (SPIR-V and OpenCL C) run in the existing Buildkite OpenCL step, next to NVIDIA's, instead of in a separate GitHub Actions job that installed the same OpenCL.jl branch again and couldn't soft-fail. [only opencl] and [skip opencl] select both.

@maleadt
maleadt added this pull request to stack #804 September 28, 2026 10:15
@github-actions

github-actions Bot commented Sep 28, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmark Results

Show table
main 1dc3be5... main / 1dc3be5...
const/@Const/Float32/262144 0.364 ± 0.021 ms 0.365 ± 0.02 ms 0.996 ± 0.079
const/@Const/Float32/65536 0.133 ± 0.023 ms 0.138 ± 0.021 ms 0.964 ± 0.22
const/@Const/Float64/262144 0.454 ± 0.027 ms 0.453 ± 0.028 ms 1 ± 0.085
const/@Const/Float64/65536 0.156 ± 0.014 ms 0.157 ± 0.013 ms 0.997 ± 0.12
const/unmarked/Float32/262144 1.29 ± 0.033 ms 1.28 ± 0.03 ms 1 ± 0.034
const/unmarked/Float32/65536 0.362 ± 0.033 ms 0.349 ± 0.022 ms 1.04 ± 0.12
const/unmarked/Float64/262144 1.51 ± 0.067 ms 1.86 ± 0.048 ms 0.812 ± 0.042
const/unmarked/Float64/65536 0.419 ± 0.02 ms 0.419 ± 0.019 ms 1 ± 0.066
launch/3D static workgroup, dynamic ndrange 0.0501 ± 0.033 ms 0.0509 ± 0.032 ms 0.984 ± 0.9
launch/3D static workgroup, static ndrange 0.0514 ± 0.03 ms 0.0506 ± 0.031 ms 1.02 ± 0.86
launch/dynamic workgroup, dynamic ndrange 0.0514 ± 0.031 ms 0.0521 ± 0.029 ms 0.987 ± 0.81
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 0.0428 ± 0.032 ms 0.0458 ± 0.03 ms 0.935 ± 0.93
launch/static workgroup, dynamic ndrange 0.0508 ± 0.03 ms 0.0504 ± 0.03 ms 1.01 ± 0.84
launch/static workgroup, static ndrange 0.0477 ± 0.032 ms 0.0501 ± 0.03 ms 0.953 ± 0.86
partition/dynamic workgroup, dynamic ndrange 0.0615 ± 0.011 μs 0.062 ± 0.01 μs 0.992 ± 0.23
partition/static workgroup, dynamic ndrange 0.0697 ± 0.014 μs 0.0698 ± 0.014 μs 1 ± 0.29
partition/static workgroup, static ndrange 1.74 ± 0.01 ns 1.74 ± 0.001 ns 0.999 ± 0.0058
saxpy/default/Float16/1024 0.0544 ± 0.029 ms 0.0561 ± 0.027 ms 0.969 ± 0.69
saxpy/default/Float16/1048576 0.887 ± 0.021 ms 0.885 ± 0.028 ms 1 ± 0.039
saxpy/default/Float16/16384 0.0519 ± 0.029 ms 0.0517 ± 0.029 ms 1.01 ± 0.81
saxpy/default/Float16/2048 0.0576 ± 0.027 ms 0.0578 ± 0.026 ms 0.996 ± 0.65
saxpy/default/Float16/256 0.044 ± 0.034 ms 0.0599 ± 0.029 ms 0.734 ± 0.67
saxpy/default/Float16/262144 0.271 ± 0.032 ms 0.269 ± 0.032 ms 1.01 ± 0.17
saxpy/default/Float16/32768 0.0659 ± 0.029 ms 0.0661 ± 0.029 ms 0.997 ± 0.62
saxpy/default/Float16/4096 0.0637 ± 0.024 ms 0.0641 ± 0.022 ms 0.993 ± 0.51
saxpy/default/Float16/512 0.0502 ± 0.034 ms 0.0552 ± 0.034 ms 0.909 ± 0.84
saxpy/default/Float16/64 0.054 ± 0.033 ms 0.0558 ± 0.032 ms 0.968 ± 0.8
saxpy/default/Float16/65536 0.093 ± 0.03 ms 0.0932 ± 0.031 ms 0.998 ± 0.46
saxpy/default/Float32/1024 0.0546 ± 0.032 ms 0.0561 ± 0.032 ms 0.972 ± 0.79
saxpy/default/Float32/1048576 0.514 ± 0.03 ms 0.514 ± 0.033 ms 1 ± 0.086
saxpy/default/Float32/16384 0.0518 ± 0.03 ms 0.0577 ± 0.03 ms 0.897 ± 0.69
saxpy/default/Float32/2048 0.0533 ± 0.025 ms 0.0536 ± 0.025 ms 0.995 ± 0.67
saxpy/default/Float32/256 0.0482 ± 0.034 ms 0.0533 ± 0.034 ms 0.904 ± 0.86
saxpy/default/Float32/262144 0.182 ± 0.017 ms 0.18 ± 0.024 ms 1.01 ± 0.16
saxpy/default/Float32/32768 0.0585 ± 0.03 ms 0.0632 ± 0.031 ms 0.924 ± 0.65
saxpy/default/Float32/4096 0.0584 ± 0.027 ms 0.0595 ± 0.023 ms 0.98 ± 0.58
saxpy/default/Float32/512 0.0485 ± 0.033 ms 0.0568 ± 0.03 ms 0.854 ± 0.73
saxpy/default/Float32/64 0.0504 ± 0.033 ms 0.0566 ± 0.03 ms 0.89 ± 0.76
saxpy/default/Float32/65536 0.0934 ± 0.03 ms 0.0924 ± 0.03 ms 1.01 ± 0.47
saxpy/default/Float64/1024 0.0521 ± 0.032 ms 0.0586 ± 0.028 ms 0.888 ± 0.69
saxpy/default/Float64/1048576 0.679 ± 0.11 ms 0.567 ± 0.098 ms 1.2 ± 0.29
saxpy/default/Float64/16384 0.0573 ± 0.03 ms 0.0541 ± 0.03 ms 1.06 ± 0.82
saxpy/default/Float64/2048 0.0543 ± 0.026 ms 0.0541 ± 0.026 ms 1 ± 0.68
saxpy/default/Float64/256 0.0433 ± 0.033 ms 0.048 ± 0.034 ms 0.901 ± 0.94
saxpy/default/Float64/262144 0.186 ± 0.02 ms 0.184 ± 0.018 ms 1.01 ± 0.14
saxpy/default/Float64/32768 0.082 ± 0.031 ms 0.0794 ± 0.031 ms 1.03 ± 0.56
saxpy/default/Float64/4096 0.0587 ± 0.024 ms 0.0581 ± 0.023 ms 1.01 ± 0.57
saxpy/default/Float64/512 0.0512 ± 0.032 ms 0.0531 ± 0.033 ms 0.965 ± 0.85
saxpy/default/Float64/64 0.0527 ± 0.033 ms 0.0485 ± 0.034 ms 1.09 ± 1
saxpy/default/Float64/65536 0.0985 ± 0.028 ms 0.0986 ± 0.028 ms 0.998 ± 0.4
saxpy/static workgroup=(1024,)/Float16/1024 0.0517 ± 0.032 ms 0.0543 ± 0.027 ms 0.953 ± 0.76
saxpy/static workgroup=(1024,)/Float16/1048576 0.878 ± 0.024 ms 0.879 ± 0.029 ms 0.999 ± 0.042
saxpy/static workgroup=(1024,)/Float16/16384 0.0528 ± 0.029 ms 0.0539 ± 0.029 ms 0.98 ± 0.75
saxpy/static workgroup=(1024,)/Float16/2048 0.0545 ± 0.028 ms 0.055 ± 0.028 ms 0.991 ± 0.71
saxpy/static workgroup=(1024,)/Float16/256 0.0512 ± 0.034 ms 0.0566 ± 0.029 ms 0.904 ± 0.76
saxpy/static workgroup=(1024,)/Float16/262144 0.266 ± 0.031 ms 0.265 ± 0.031 ms 1 ± 0.17
saxpy/static workgroup=(1024,)/Float16/32768 0.0687 ± 0.029 ms 0.068 ± 0.029 ms 1.01 ± 0.6
saxpy/static workgroup=(1024,)/Float16/4096 0.0575 ± 0.027 ms 0.0567 ± 0.027 ms 1.01 ± 0.67
saxpy/static workgroup=(1024,)/Float16/512 0.0508 ± 0.034 ms 0.058 ± 0.027 ms 0.876 ± 0.72
saxpy/static workgroup=(1024,)/Float16/64 0.0534 ± 0.031 ms 0.0556 ± 0.03 ms 0.961 ± 0.75
saxpy/static workgroup=(1024,)/Float16/65536 0.0968 ± 0.029 ms 0.102 ± 0.03 ms 0.949 ± 0.4
saxpy/static workgroup=(1024,)/Float32/1024 0.0515 ± 0.032 ms 0.0531 ± 0.032 ms 0.97 ± 0.85
saxpy/static workgroup=(1024,)/Float32/1048576 0.511 ± 0.028 ms 0.515 ± 0.032 ms 0.993 ± 0.082
saxpy/static workgroup=(1024,)/Float32/16384 0.0554 ± 0.027 ms 0.0571 ± 0.027 ms 0.969 ± 0.66
saxpy/static workgroup=(1024,)/Float32/2048 0.0522 ± 0.026 ms 0.053 ± 0.025 ms 0.985 ± 0.67
saxpy/static workgroup=(1024,)/Float32/256 0.0558 ± 0.028 ms 0.056 ± 0.03 ms 0.996 ± 0.73
saxpy/static workgroup=(1024,)/Float32/262144 0.178 ± 0.021 ms 0.179 ± 0.019 ms 0.995 ± 0.16
saxpy/static workgroup=(1024,)/Float32/32768 0.0663 ± 0.029 ms 0.0667 ± 0.029 ms 0.994 ± 0.61
saxpy/static workgroup=(1024,)/Float32/4096 0.0565 ± 0.026 ms 0.0575 ± 0.024 ms 0.984 ± 0.61
saxpy/static workgroup=(1024,)/Float32/512 0.0512 ± 0.034 ms 0.051 ± 0.034 ms 1 ± 0.93
saxpy/static workgroup=(1024,)/Float32/64 0.0568 ± 0.028 ms 0.0552 ± 0.029 ms 1.03 ± 0.74
saxpy/static workgroup=(1024,)/Float32/65536 0.0861 ± 0.028 ms 0.0881 ± 0.028 ms 0.978 ± 0.45
saxpy/static workgroup=(1024,)/Float64/1024 0.0535 ± 0.031 ms 0.0545 ± 0.031 ms 0.981 ± 0.79
saxpy/static workgroup=(1024,)/Float64/1048576 0.551 ± 0.053 ms 0.565 ± 0.098 ms 0.976 ± 0.19
saxpy/static workgroup=(1024,)/Float64/16384 0.0621 ± 0.027 ms 0.0609 ± 0.028 ms 1.02 ± 0.64
saxpy/static workgroup=(1024,)/Float64/2048 0.0525 ± 0.026 ms 0.0522 ± 0.025 ms 1.01 ± 0.69
saxpy/static workgroup=(1024,)/Float64/256 0.0556 ± 0.027 ms 0.0552 ± 0.029 ms 1.01 ± 0.72
saxpy/static workgroup=(1024,)/Float64/262144 0.185 ± 0.022 ms 0.182 ± 0.019 ms 1.02 ± 0.16
saxpy/static workgroup=(1024,)/Float64/32768 0.0755 ± 0.029 ms 0.0762 ± 0.029 ms 0.991 ± 0.54
saxpy/static workgroup=(1024,)/Float64/4096 0.056 ± 0.026 ms 0.0561 ± 0.026 ms 0.998 ± 0.65
saxpy/static workgroup=(1024,)/Float64/512 0.0502 ± 0.033 ms 0.0564 ± 0.029 ms 0.89 ± 0.74
saxpy/static workgroup=(1024,)/Float64/64 0.0549 ± 0.029 ms 0.0557 ± 0.029 ms 0.984 ± 0.73
saxpy/static workgroup=(1024,)/Float64/65536 0.0948 ± 0.027 ms 0.0946 ± 0.027 ms 1 ± 0.4
time_to_load 0.782 ± 0.0029 s 0.784 ± 0.0041 s 0.996 ± 0.0064
main 1dc3be5... main / 1dc3be5...
const/@Const/Float32/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float32/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float64/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/@Const/Float64/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float32/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float32/65536 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float64/262144 1 allocs: 16 B 1 allocs: 16 B 1
const/unmarked/Float64/65536 1 allocs: 16 B 1 allocs: 16 B 1
launch/3D static workgroup, dynamic ndrange 5 allocs: 0.141 kB 5 allocs: 0.141 kB 1
launch/3D static workgroup, static ndrange 6 allocs: 0.172 kB 6 allocs: 0.172 kB 1
launch/dynamic workgroup, dynamic ndrange 15 allocs: 0.312 kB 15 allocs: 0.312 kB 1
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 1 allocs: 16 B 1 allocs: 16 B 1
launch/static workgroup, dynamic ndrange 1 allocs: 16 B 1 allocs: 16 B 1
launch/static workgroup, static ndrange 2 allocs: 0.0469 kB 2 allocs: 0.0469 kB 1
partition/dynamic workgroup, dynamic ndrange 2 allocs: 0.0625 kB 2 allocs: 0.0625 kB 1
partition/static workgroup, dynamic ndrange 2 allocs: 32 B 2 allocs: 32 B 1
partition/static workgroup, static ndrange 0 allocs: 0 B 0 allocs: 0 B
saxpy/default/Float16/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float16/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float16/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float16/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float32/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float32/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float32/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/1024 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/1048576 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/16384 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/2048 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/256 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float64/262144 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/32768 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/4096 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/512 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/default/Float64/64 16 allocs: 0.328 kB 16 allocs: 0.328 kB 1
saxpy/default/Float64/65536 24 allocs: 0.453 kB 24 allocs: 0.453 kB 1
saxpy/static workgroup=(1024,)/Float16/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float16/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float16/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float16/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float32/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float32/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float32/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/1024 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/1048576 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/16384 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/2048 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/256 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float64/262144 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/32768 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/4096 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/512 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
saxpy/static workgroup=(1024,)/Float64/64 1 allocs: 16 B 1 allocs: 16 B 1
saxpy/static workgroup=(1024,)/Float64/65536 4 allocs: 0.0625 kB 4 allocs: 0.0625 kB 1
time_to_load 0.2 k allocs: 11.8 kB 0.2 k allocs: 11.8 kB 1

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@maleadt
CI: test OpenCL.jl only on Buildkite, and soft-fail back ends that ca…
1dc3be5 
…n't be installed

- run OpenCL.jl's KernelAbstractions and KernelInterface tests on PoCL (SPIR-V and
  OpenCL C) next to NVIDIA in the Buildkite OpenCL step, instead of PoCL in a
  separate GitHub Actions job that installed the same branch again;
- like GPUCompiler's downstream tests, exit with status 3 (a soft failure) when a
  back end's branch can't be installed together with this KernelAbstractions,
  e.g. while it doesn't support a new KernelInterface yet; test failures still
  fail the build.
This was referenced Sep 29, 2026
Merged
Merged
@maleadt
maleadt removed this pull request from stack #804 September 29, 2026 20:22
@maleadt
maleadt added this pull request to stack #808 September 29, 2026 20:22
@maleadt

maleadt commented Sep 30, 2026

Copy link
Copy Markdown
Member Author

CI failures unrelated.

@maleadt
maleadt merged commit 7d655fd into main Sep 30, 2026
68 of 77 checks passed
@maleadt
maleadt deleted the tb/ci-reverse branch September 30, 2026 04:57
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@maleadt