Skip to content

KernelInterface - #917

Closed
christiangnrd wants to merge 4 commits into
mainfrom
kainterface
Closed

christiangnrd wants to merge 4 commits into
mainfrom
kainterface

Conversation

@christiangnrd

@christiangnrd christiangnrd commented Aug 16, 2026 •

Copy link
Copy Markdown
Member

Do not merge until KernelInterface has been reviewed and interface fully decided

@github-actions github-actions Bot left a comment •

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Metal Benchmarks

Details
Benchmark suite Current: ebbef29 Previous: 7e34a9c Ratio
array/accumulate/Float32/1d 389084 ns 394042 ns 0.99
array/accumulate/Float32/dims=1 366792 ns 363917 ns 1.01
array/accumulate/Float32/dims=1L 8766375 ns 8768250 ns 1.00
array/accumulate/Float32/dims=2 427833 ns 431958 ns 0.99
array/accumulate/Float32/dims=2L 2537292 ns 2525542 ns 1.00
array/accumulate/Int64/1d 953250 ns 958667 ns 0.99
array/accumulate/Int64/dims=1 935542 ns 939000 ns 1.00
array/accumulate/Int64/dims=1L 9462666 ns 9471125 ns 1.00
array/accumulate/Int64/dims=2 1238375 ns 1232875 ns 1.00
array/accumulate/Int64/dims=2L 6537417 ns 6544791 ns 1.00
array/broadcast 253375 ns 247625 ns 1.02
array/construct 2375 ns 2459 ns 0.97
array/permutedims/2d 484417 ns 483625 ns 1.00
array/permutedims/3d 1054625 ns 1064792 ns 0.99
array/permutedims/4d 1316458 ns 1319125 ns 1.00
array/private/copy 226625 ns 226542 ns 1.00
array/private/copyto!/cpu_to_gpu 215792 ns 215750 ns 1.00
array/private/copyto!/gpu_to_cpu 212292 ns 212875 ns 1.00
array/private/copyto!/gpu_to_gpu 218000 ns 216834 ns 1.01
array/private/iteration/findall/bool 1179875 ns 1197625 ns 0.99
array/private/iteration/findall/int 1371958 ns 1381000 ns 0.99
array/private/iteration/findfirst/bool 1186458 ns 1200958 ns 0.99
array/private/iteration/findfirst/int 1205416 ns 1207000 ns 1.00
array/private/iteration/findmin/1d 1247625 ns 1248791 ns 1.00
array/private/iteration/findmin/2d 1097459 ns 1106166 ns 0.99
array/private/iteration/logical 1853834 ns 1867375 ns 0.99
array/private/iteration/scalar 1377042 ns 1400417 ns 0.98
array/random/rand/Float32 437541 ns 443000 ns 0.99
array/random/rand/Int64 517125 ns 522250 ns 0.99
array/random/rand!/Float32 423167 ns 428750 ns 0.99
array/random/rand!/Int64 456625 ns 455500 ns 1.00
array/random/randn/Float32 412042 ns 411375 ns 1.00
array/random/randn!/Float32 398458 ns 401792 ns 0.99
array/reductions/mapreduce/Float32/1d 469125 ns 437084 ns 1.07
array/reductions/mapreduce/Float32/dims=1 372709 ns 374375 ns 1.00
array/reductions/mapreduce/Float32/dims=1L 642333 ns 635625 ns 1.01
array/reductions/mapreduce/Float32/dims=2 376959 ns 377375 ns 1.00
array/reductions/mapreduce/Float32/dims=2L 1237333 ns 1258167 ns 0.98
array/reductions/mapreduce/Int64/1d 667584 ns 666833 ns 1.00
array/reductions/mapreduce/Int64/dims=1 656500 ns 653500 ns 1.00
array/reductions/mapreduce/Int64/dims=1L 1028542 ns 1028542 ns 1
array/reductions/mapreduce/Int64/dims=2 810292 ns 812833 ns 1.00
array/reductions/mapreduce/Int64/dims=2L 2208792 ns 2209542 ns 1.00
array/reductions/reduce/Float32/1d 480291 ns 463500 ns 1.04
array/reductions/reduce/Float32/dims=1 375834 ns 375500 ns 1.00
array/reductions/reduce/Float32/dims=1L 636792 ns 636667 ns 1.00
array/reductions/reduce/Float32/dims=2 244792 ns 252416 ns 0.97
array/reductions/reduce/Float32/dims=2L 481834 ns 473666 ns 1.02
array/reductions/reduce/Int64/1d 655041 ns 666833 ns 0.98
array/reductions/reduce/Int64/dims=1 649208 ns 655750 ns 0.99
array/reductions/reduce/Int64/dims=1L 1022667 ns 1033792 ns 0.99
array/reductions/reduce/Int64/dims=2 261459 ns 257917 ns 1.01
array/reductions/reduce/Int64/dims=2L 669167 ns 669750 ns 1.00
array/shared/copy 128125 ns 129833 ns 0.99
array/shared/copyto!/cpu_to_gpu 37083 ns 36791 ns 1.01
array/shared/copyto!/gpu_to_cpu 36708 ns 37417 ns 0.98
array/shared/copyto!/gpu_to_gpu 37291 ns 37917 ns 0.98
array/shared/iteration/findall/bool 1179750 ns 1193833 ns 0.99
array/shared/iteration/findall/int 1360750 ns 1377084 ns 0.99
array/shared/iteration/findfirst/bool 1015750 ns 1022750 ns 0.99
array/shared/iteration/findfirst/int 1027042 ns 1031208 ns 1.00
array/shared/iteration/findmin/1d 1095917 ns 1104834 ns 0.99
array/shared/iteration/findmin/2d 1092791 ns 1106167 ns 0.99
array/shared/iteration/logical 1725042 ns 1737750 ns 0.99
array/shared/iteration/scalar 3703.125 ns 3843.75 ns 0.96
array/sorting/1d 2093375 ns 2079916 ns 1.01
array/sorting/2d 8296333 ns 8328250 ns 1.00
integration/byval/reference 1146291 ns 1150959 ns 1.00
integration/byval/slices=1 1144917 ns 1133125 ns 1.01
integration/byval/slices=2 2049750 ns 2067125 ns 0.99
integration/byval/slices=3 6649417 ns 6636875 ns 1.00
integration/metaldevrt 414000 ns 424750 ns 0.97
kernel/indexing 223042 ns 225791 ns 0.99
kernel/indexing_checked 406000 ns 412542 ns 0.98
kernel/launch 5632 ns 6033.4 ns 0.93
kernel/rand 412042 ns 416625 ns 0.99
latency/import 1785535083 ns 1777164625 ns 1.00
latency/precompile 32675415416 ns 32777388583 ns 1.00
latency/ttfp 2255822792 ns 2248797500 ns 1.00
metal/synchronization/context 551.9144385026738 ns 546.3475935828877 ns 1.01
metal/synchronization/stream 343.3727272727273 ns 360.64593301435406 ns 0.95

This comment was automatically generated by workflow using github-action-benchmark.

@christiangnrd
christiangnrd force-pushed the kainterface branch 2 times, most recently from a17deff to 998ec45 Compare August 18, 2026 19:31
@christiangnrd
christiangnrd changed the base branch from main to mtcache August 18, 2026 19:32
Base automatically changed from mtcache to main August 18, 2026 20:47
@christiangnrd
christiangnrd force-pushed the kainterface branch 10 times, most recently from 1599850 to 4ff4748 Compare August 22, 2026 13:31
@codecov

codecov Bot commented Aug 22, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.12346% with 16 lines in your changes missing coverage. Please review.
✅ Project coverage is 86.70%. Comparing base (7e34a9c) to head (ea43a3e).

Files with missing lines Patch % Lines
src/MetalKernelsOld.jl 89.77% 9 Missing ⚠️
src/MetalKernels.jl 89.85% 7 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main     #917      +/-   ##
==========================================
- Coverage   87.01%   86.70%   -0.31%     
==========================================
  Files          92       93       +1     
  Lines        6733     6817      +84     
==========================================
+ Hits         5859     5911      +52     
- Misses        874      906      +32     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@christiangnrd
christiangnrd marked this pull request as draft September 3, 2026 15:30
@christiangnrd
christiangnrd force-pushed the kainterface branch 4 times, most recently from 6dfff48 to 2e329ac Compare September 7, 2026 22:43
@christiangnrd
christiangnrd force-pushed the kainterface branch 2 times, most recently from a00d6cf to fb76fed Compare September 18, 2026 13:51
@christiangnrd
christiangnrd force-pushed the kainterface branch 3 times, most recently from 136f491 to 014a836 Compare September 21, 2026 21:33
@christiangnrd
christiangnrd marked this pull request as ready for review September 22, 2026 01:01
@christiangnrd
christiangnrd removed this pull request from stack #923 September 27, 2026 12:47
@christiangnrd
christiangnrd added this pull request to stack #984 September 27, 2026 12:47
Implement the KernelInterface API: allocation, asynchronous copies,
device selection, launch limits, kernel compilation and launch, and the
device-side indexing, sub-group, local memory, barrier and shuffle
intrinsics.

The back end lives in `Metal.MetalInterface` and isn't public yet;
`KernelInterface.get_backend` returns it for an `MtlArray`. The
KernelAbstractions back end, `MetalBackend`, is unchanged (it moves to
`MetalKernelsOld.jl`). `Metal.versioninfo` also reports the
KernelInterface version.

Co-authored-by: Tim Besard <tim.besard@gmail.com>
- subtype `KI.Backend`, and launch through `KI.launch`, which receives the
  validated launch geometry;
- implement the typed index queries with `% T`, and let KernelInterface
  derive the global ones;
- typed sub-group queries, with `get_sub_group_size` counting the threads
  present in a partial SIMD-group; `supports_subgroups`, and
  `supports_shuffle` for the types `simd_shuffle_down` supports;
- `max_work_group_size(kernel)` is the pipeline's
  `maxTotalThreadsPerThreadgroup`, and `max_num_groups` keeps the grid
  within 32 bits per dimension;
- `sub_group_size` is a guarantee: `kernel_function` checks the pipeline's
  `threadExecutionWidth`;
- `kernel_function` keeps the backend it was given;
- `copyto!` checks lengths; forward `unsafe_free!`;
- drop the `zeros`/`ones` overrides, which are generic now.

KernelInterface 0.3 isn't registered yet, so get it from the
KernelAbstractions.jl branch.
Encode a wait for the event in the task's command buffer instead of
blocking the host thread in waitUntilSignaledValue, so that the work the
task queues next is ordered after the event without stalling the host.
Julia doesn't specialize a method on `args...` that it only passes through, which
made every launch through KernelInterface's generic launch dispatch dynamically
(+1.2 µs and +1 kB per launch on CUDA).
@maleadt

maleadt commented Sep 30, 2026

Copy link
Copy Markdown
Member

#994

@maleadt maleadt closed this Sep 30, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants