Skip to content

Time kernels on the device without synchronizing (prototype) - #837

Draft
vchuravy wants to merge 1 commit into
vc/tracing-v2from
vc/device-timestamps
Draft

Time kernels on the device without synchronizing (prototype)#837
vchuravy wants to merge 1 commit into
vc/tracing-v2from
vc/device-timestamps

Conversation

@vchuravy

@vchuravy vchuravy commented Oct 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Member

Stacked on #836. This is a prototype of what CUDA's @profile gets from CUPTI: timing kernels on the device without forcing synchronization.

Design

  • New optional KernelInterface hooks. Both fallbacks report no support, so existing backends are unaffected.
    • KI.record_timestamp(backend) puts a timestamp on the calling task's queue without blocking. It returns nothing if the backend doesn't support timestamps.
    • KI.elapsed_time(backend, start, stop)::Int64 returns the device nanoseconds between two timestamps, waiting cooperatively if needed.
  • Recording. When a tracer sets records_kernels, launch_kernel records timestamps around the native launch only, so compilation isn't counted as device time. The tracer gets a KernelTimer holding the pair.
    • On backends without timestamps, the launch synchronizes instead and is timed on the host. Such kernels are marked * in the report.
  • @profile (now device = true by default). It keeps the timestamp pairs and turns them into durations once the profiled expression has finished, the way CUPTI buffers activity records. The report has separate host-side and device-side sections.
    • Launches no longer synchronize by default.
    • In trace = true mode, kernels are placed on the host timeline relative to the first kernel on each device, assuming that one started when it was launched. This is approximate.

CUDA implementation

Not part of this PR. It is local branch vc/ka-timestamps of CUDA.jl, on top of ka-0.10:

function KI.record_timestamp(::CUDABackend)
    ev = CuEvent(CUDACore.EVENT_DEFAULT)
    record(ev, stream())
    return ev
end

function KI.elapsed_time(::CUDABackend, start::CuEvent, stop::CuEvent)
    synchronize(stop)   # cooperatively, as `KI.synchronize` does
    return round(Int64, 1.0e9 * CUDACore.elapsed(start, stop))
end

Results (Quadro RTX 4000, 4M-element mul2/add)

Host-side activity:
 Time (%)  Total time  Calls  Avg time  Min time  Max time  Name
    6.8 %      206 µs     10   20.6 µs   13.2 µs   77.3 µs  step
    3.5 %      106 µs     10   10.6 µs   5.88 µs   47.1 µs  mul2
    2.7 %     81.5 µs     10   8.15 µs   6.43 µs   20.6 µs  add

Device-side activity:
 Time (%)  Total time  Calls  Avg time  Min time  Max time  Name
   51.3 %     1.56 ms     10    156 µs    155 µs    157 µs  add
   45.8 %     1.39 ms     10    139 µs    137 µs    151 µs  mul2

  • Device times match CUDA. CUDA.@elapsed gives a median of 138 µs for mul2, and the profiler measures 138 µs. Timing on the host with synchronize = true instead gives 143 µs, which includes sync overhead.

  • Cost per launch, small kernel:

    Per launch
    Untraced 2.95 µs
    @profile device = false 3.45 µs
    @profile 9.4 µs

    The extra ~6 µs is most likely from creating two new CuEvents, with finalizers, on every launch; an event pool would avoid it. Nothing changes outside @profile.

Tasks

Each kernel is attributed to the task that launched it, and in trace = true mode it appears on a per-queue row such as CUDABackend 1, task 2. Kernels running concurrently on different tasks' streams therefore appear side by side, rather than stacked as if nested.

As in #836, @profile only records the tasks of its expression, and an unrelated task doesn't pay for device timestamps. CUDA can measure elapsed time between events on different streams of the same context, so placing kernels from different streams on the timeline works.

Verified on the RTX 4000 with three KA.@spawn tasks, each running its kernels on its own stream:

 97.7 ms   62.3 µs  CUDABackend 1, task 3  mul2
 97.7 ms    100 µs  CUDABackend 1, task 2  mul2
 97.8 ms    113 µs  CUDABackend 1, task 4  mul2

Caveats

  • Not verified against current main. CUDA.jl's ka-0.10 branch still requires LLVM.jl 9, while main requires LLVM.jl 10. The GPU numbers above come from these commits replayed onto 2e027f09, the last commit before the LLVM.jl 10 bump. The test suite here passes on main, where the CPU backend takes the host-timed path.
  • Not yet done:
    • Timestamp hooks for POCL (OpenCL profiling events) and AMDGPU (HIP events).
    • Device time for operations other than kernel launches, such as copies.
    • Pooling events.
    • Updating the docs page for the new defaults.

🤖 Generated with Claude Code

@vchuravy
vchuravy added this pull request to stack #838 October 4, 2026 08:38
@codecov

codecov Bot commented Oct 4, 2026 •
edited
Loading

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.26549% with 11 lines in your changes missing coverage. Please review.
✅ Project coverage is 80.68%. Comparing base (d6c0967) to head (cf18848).

Files with missing lines Patch % Lines
src/profiler.jl 88.73% 8 Missing ⚠️
src/profiling.jl 90.90% 3 Missing ⚠️
Additional details and impacted files 
@@                Coverage Diff                @@
##           vc/tracing-v2     #837      +/-   ##
=================================================
+ Coverage          76.32%   80.68%   +4.36%     
=================================================
  Files                 29       29              
  Lines               2547     2496      -51     
=================================================
+ Hits                1944     2014      +70     
+ Misses               603      482     -121

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@github-actions

github-actions Bot commented Oct 4, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

Benchmark Results

Show table
main cf18848... main / cf18848...
const/@Const/Float32/262144 0.346 ± 0.015 ms 0.344 ± 0.016 ms 1 ± 0.064
const/@Const/Float32/65536 0.115 ± 0.01 ms 0.116 ± 0.011 ms 0.989 ± 0.13
const/@Const/Float64/262144 0.604 ± 0.014 ms 0.606 ± 0.017 ms 0.998 ± 0.036
const/@Const/Float64/65536 0.242 ± 0.012 ms 0.25 ± 0.013 ms 0.966 ± 0.069
const/unmarked/Float32/262144 0.659 ± 0.017 ms 0.658 ± 0.017 ms 1 ± 0.036
const/unmarked/Float32/65536 0.196 ± 0.011 ms 0.197 ± 0.012 ms 0.995 ± 0.082
const/unmarked/Float64/262144 1.24 ± 0.013 ms 1.24 ± 0.015 ms 0.999 ± 0.016
const/unmarked/Float64/65536 0.359 ± 0.016 ms 0.396 ± 0.016 ms 0.908 ± 0.054
launch/3D static workgroup, dynamic ndrange 15 ± 2.8 μs 15.7 ± 29 μs 0.956 ± 1.7
launch/3D static workgroup, static ndrange 21.1 ± 29 μs 14.7 ± 27 μs 1.43 ± 3.3
launch/dynamic workgroup, dynamic ndrange 16.1 ± 4.9 μs 14.5 ± 1 μs 1.11 ± 0.34
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 14.4 ± 1.7 μs 17 ± 13 μs 0.844 ± 0.66
launch/static workgroup, dynamic ndrange 22.9 ± 9.6 μs 24.2 ± 26 μs 0.946 ± 1.1
launch/static workgroup, static ndrange 14.6 ± 2.1 μs 18.6 ± 15 μs 0.784 ± 0.65
partition/dynamic workgroup, dynamic ndrange 0.0528 ± 0.01 μs 0.0531 ± 0.0033 μs 0.993 ± 0.2
partition/static workgroup, dynamic ndrange 0.0556 ± 0.011 μs 0.0578 ± 0.011 μs 0.962 ± 0.25
partition/static workgroup, static ndrange 1.35 ± 0.004 ns 1.35 ± 0.005 ns 0.999 ± 0.0047
saxpy/default/Float16/1024 17.1 ± 1.2 μs 18 ± 26 μs 0.949 ± 1.4
saxpy/default/Float16/1048576 0.484 ± 0.11 ms 0.487 ± 0.031 ms 0.995 ± 0.24
saxpy/default/Float16/16384 0.047 ± 0.01 ms 0.0454 ± 0.0093 ms 1.03 ± 0.31
saxpy/default/Float16/2048 0.0423 ± 0.029 ms 0.0409 ± 0.031 ms 1.04 ± 1.1
saxpy/default/Float16/256 20.2 ± 28 μs 20.9 ± 28 μs 0.966 ± 1.8
saxpy/default/Float16/262144 0.148 ± 0.013 ms 0.154 ± 0.016 ms 0.962 ± 0.13
saxpy/default/Float16/32768 0.0512 ± 0.0068 ms 0.0529 ± 0.0086 ms 0.969 ± 0.2
saxpy/default/Float16/4096 16 ± 4.6 μs 0.0429 ± 0.031 ms 0.372 ± 0.29
saxpy/default/Float16/512 19.2 ± 24 μs 18.4 ± 8.7 μs 1.04 ± 1.4
saxpy/default/Float16/64 21.1 ± 24 μs 24.8 ± 24 μs 0.85 ± 1.3
saxpy/default/Float16/65536 0.0669 ± 0.0094 ms 0.0669 ± 0.0078 ms 1 ± 0.18
saxpy/default/Float32/1024 15.1 ± 1.3 μs 17.4 ± 12 μs 0.865 ± 0.62
saxpy/default/Float32/1048576 0.675 ± 0.079 ms 0.679 ± 0.077 ms 0.993 ± 0.16
saxpy/default/Float32/16384 0.0456 ± 0.03 ms 0.0462 ± 0.011 ms 0.987 ± 0.69
saxpy/default/Float32/2048 18.6 ± 24 μs 18.5 ± 27 μs 1.01 ± 2
saxpy/default/Float32/256 17.2 ± 15 μs 17.8 ± 7.5 μs 0.967 ± 0.93
saxpy/default/Float32/262144 0.211 ± 0.028 ms 0.213 ± 0.022 ms 0.993 ± 0.17
saxpy/default/Float32/32768 0.0561 ± 0.011 ms 0.0558 ± 0.014 ms 1.01 ± 0.31
saxpy/default/Float32/4096 0.043 ± 0.023 ms 0.0435 ± 0.024 ms 0.99 ± 0.75
saxpy/default/Float32/512 18.3 ± 25 μs 20.8 ± 27 μs 0.879 ± 1.7
saxpy/default/Float32/64 21.3 ± 28 μs 18.4 ± 8.9 μs 1.16 ± 1.6
saxpy/default/Float32/65536 0.0744 ± 0.015 ms 0.0771 ± 0.0086 ms 0.966 ± 0.22
saxpy/default/Float64/1024 19.2 ± 25 μs 17.1 ± 9.6 μs 1.12 ± 1.6
saxpy/default/Float64/1048576 1.42 ± 0.15 ms 1.42 ± 0.29 ms 1 ± 0.23
saxpy/default/Float64/16384 0.0537 ± 0.012 ms 0.0572 ± 0.0074 ms 0.939 ± 0.24
saxpy/default/Float64/2048 0.0419 ± 0.028 ms 0.0394 ± 0.028 ms 1.06 ± 1
saxpy/default/Float64/256 16.1 ± 1.4 μs 18 ± 30 μs 0.894 ± 1.5
saxpy/default/Float64/262144 0.356 ± 0.039 ms 0.369 ± 0.063 ms 0.965 ± 0.19
saxpy/default/Float64/32768 0.0752 ± 0.012 ms 0.0771 ± 0.009 ms 0.976 ± 0.19
saxpy/default/Float64/4096 0.0436 ± 0.022 ms 16 ± 5.9 μs 2.72 ± 1.7
saxpy/default/Float64/512 17.4 ± 11 μs 17.8 ± 12 μs 0.978 ± 0.88
saxpy/default/Float64/64 28.7 ± 28 μs 18 ± 24 μs 1.59 ± 2.7
saxpy/default/Float64/65536 0.118 ± 0.015 ms 0.119 ± 0.016 ms 0.991 ± 0.18
saxpy/static workgroup=(1024,)/Float16/1024 0.0412 ± 0.032 ms 18.9 ± 27 μs 2.18 ± 3.5
saxpy/static workgroup=(1024,)/Float16/1048576 0.483 ± 0.1 ms 0.491 ± 0.026 ms 0.984 ± 0.21
saxpy/static workgroup=(1024,)/Float16/16384 0.0462 ± 0.0099 ms 0.0471 ± 0.0095 ms 0.981 ± 0.29
saxpy/static workgroup=(1024,)/Float16/2048 17.3 ± 23 μs 25.8 ± 29 μs 0.671 ± 1.2
saxpy/static workgroup=(1024,)/Float16/256 15.8 ± 3.4 μs 22.2 ± 13 μs 0.714 ± 0.45
saxpy/static workgroup=(1024,)/Float16/262144 0.149 ± 0.013 ms 0.156 ± 0.017 ms 0.954 ± 0.13
saxpy/static workgroup=(1024,)/Float16/32768 0.0519 ± 0.007 ms 0.0547 ± 0.011 ms 0.948 ± 0.22
saxpy/static workgroup=(1024,)/Float16/4096 0.0421 ± 0.029 ms 22.1 ± 31 μs 1.91 ± 3
saxpy/static workgroup=(1024,)/Float16/512 0.0419 ± 0.031 ms 15.8 ± 2.2 μs 2.66 ± 2
saxpy/static workgroup=(1024,)/Float16/64 0.0412 ± 0.031 ms 24.9 ± 23 μs 1.66 ± 2
saxpy/static workgroup=(1024,)/Float16/65536 0.0657 ± 0.0081 ms 0.0667 ± 0.0084 ms 0.985 ± 0.17
saxpy/static workgroup=(1024,)/Float32/1024 0.0408 ± 0.028 ms 18.5 ± 28 μs 2.21 ± 3.6
saxpy/static workgroup=(1024,)/Float32/1048576 0.645 ± 0.064 ms 0.689 ± 0.089 ms 0.936 ± 0.15
saxpy/static workgroup=(1024,)/Float32/16384 0.0457 ± 0.012 ms 0.0471 ± 0.014 ms 0.97 ± 0.39
saxpy/static workgroup=(1024,)/Float32/2048 18 ± 25 μs 16.9 ± 8 μs 1.06 ± 1.6
saxpy/static workgroup=(1024,)/Float32/256 17.2 ± 1.8 μs 17.4 ± 2.1 μs 0.992 ± 0.16
saxpy/static workgroup=(1024,)/Float32/262144 0.221 ± 0.036 ms 0.22 ± 0.028 ms 1 ± 0.21
saxpy/static workgroup=(1024,)/Float32/32768 0.0566 ± 0.005 ms 0.0563 ± 0.011 ms 1.01 ± 0.22
saxpy/static workgroup=(1024,)/Float32/4096 23.4 ± 28 μs 20.7 ± 28 μs 1.13 ± 2
saxpy/static workgroup=(1024,)/Float32/512 16 ± 2.2 μs 18.7 ± 26 μs 0.858 ± 1.2
saxpy/static workgroup=(1024,)/Float32/64 17.5 ± 2.1 μs 26.8 ± 29 μs 0.651 ± 0.71
saxpy/static workgroup=(1024,)/Float32/65536 0.0774 ± 0.0076 ms 0.0769 ± 0.01 ms 1.01 ± 0.17
saxpy/static workgroup=(1024,)/Float64/1024 17.4 ± 2.2 μs 15.4 ± 4.8 μs 1.13 ± 0.38
saxpy/static workgroup=(1024,)/Float64/1048576 1.31 ± 0.21 ms 1.47 ± 0.23 ms 0.895 ± 0.2
saxpy/static workgroup=(1024,)/Float64/16384 0.0548 ± 0.013 ms 0.0581 ± 0.009 ms 0.944 ± 0.27
saxpy/static workgroup=(1024,)/Float64/2048 0.0412 ± 0.029 ms 30.4 ± 29 μs 1.35 ± 1.6
saxpy/static workgroup=(1024,)/Float64/256 17.9 ± 13 μs 17.8 ± 9.2 μs 1.01 ± 0.88
saxpy/static workgroup=(1024,)/Float64/262144 0.346 ± 0.035 ms 0.36 ± 0.038 ms 0.961 ± 0.14
saxpy/static workgroup=(1024,)/Float64/32768 0.0755 ± 0.011 ms 0.0762 ± 0.017 ms 0.991 ± 0.26
saxpy/static workgroup=(1024,)/Float64/4096 0.0418 ± 0.0065 ms 0.0424 ± 0.032 ms 0.986 ± 0.76
saxpy/static workgroup=(1024,)/Float64/512 18.1 ± 8.4 μs 18.7 ± 28 μs 0.965 ± 1.5
saxpy/static workgroup=(1024,)/Float64/64 0.042 ± 0.027 ms 17.2 ± 8.6 μs 2.44 ± 2
saxpy/static workgroup=(1024,)/Float64/65536 0.117 ± 0.014 ms 0.117 ± 0.021 ms 0.997 ± 0.21
time_to_load 0.516 ± 0.0049 s 0.516 ± 0.001 s 0.999 ± 0.0098
main cf18848... main / cf18848...
const/@Const/Float32/262144 9 allocs: 0.203 kB 9 allocs: 0.203 kB 1
const/@Const/Float32/65536 9 allocs: 0.203 kB 9 allocs: 0.203 kB 1
const/@Const/Float64/262144 9 allocs: 0.203 kB 9 allocs: 0.203 kB 1
const/@Const/Float64/65536 9 allocs: 0.203 kB 9 allocs: 0.203 kB 1
const/unmarked/Float32/262144 9 allocs: 0.203 kB 9 allocs: 0.203 kB 1
const/unmarked/Float32/65536 9 allocs: 0.203 kB 9 allocs: 0.203 kB 1
const/unmarked/Float64/262144 9 allocs: 0.203 kB 9 allocs: 0.203 kB 1
const/unmarked/Float64/65536 9 allocs: 0.203 kB 9 allocs: 0.203 kB 1
launch/3D static workgroup, dynamic ndrange 9 allocs: 0.219 kB 9 allocs: 0.219 kB 1
launch/3D static workgroup, static ndrange 9 allocs: 0.219 kB 9 allocs: 0.219 kB 1
launch/dynamic workgroup, dynamic ndrange 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
launch/dynamic workgroup, dynamic ndrange, workgroupsize given 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
launch/static workgroup, dynamic ndrange 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
launch/static workgroup, static ndrange 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
partition/dynamic workgroup, dynamic ndrange 2 allocs: 0.0625 kB 2 allocs: 0.0625 kB 1
partition/static workgroup, dynamic ndrange 2 allocs: 32 B 2 allocs: 32 B 1
partition/static workgroup, static ndrange 0 allocs: 0 B 0 allocs: 0 B
saxpy/default/Float16/1024 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float16/1048576 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/default/Float16/16384 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float16/2048 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float16/256 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/default/Float16/262144 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/default/Float16/32768 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float16/4096 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float16/512 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float16/64 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/default/Float16/65536 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/default/Float32/1024 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float32/1048576 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/default/Float32/16384 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float32/2048 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float32/256 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/default/Float32/262144 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/default/Float32/32768 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float32/4096 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float32/512 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float32/64 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/default/Float32/65536 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float64/1024 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float64/1048576 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/default/Float64/16384 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float64/2048 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float64/256 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/default/Float64/262144 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/default/Float64/32768 8 allocs: 0.141 kB 12 allocs: 0.25 kB 0.562
saxpy/default/Float64/4096 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float64/512 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/default/Float64/64 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/default/Float64/65536 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/static workgroup=(1024,)/Float16/1024 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float16/1048576 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/static workgroup=(1024,)/Float16/16384 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float16/2048 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float16/256 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/static workgroup=(1024,)/Float16/262144 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/static workgroup=(1024,)/Float16/32768 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float16/4096 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float16/512 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float16/64 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/static workgroup=(1024,)/Float16/65536 12 allocs: 0.25 kB 8 allocs: 0.141 kB 1.78
saxpy/static workgroup=(1024,)/Float32/1024 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float32/1048576 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/static workgroup=(1024,)/Float32/16384 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float32/2048 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float32/256 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/static workgroup=(1024,)/Float32/262144 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/static workgroup=(1024,)/Float32/32768 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float32/4096 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float32/512 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float32/64 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/static workgroup=(1024,)/Float32/65536 12 allocs: 0.25 kB 8 allocs: 0.141 kB 1.78
saxpy/static workgroup=(1024,)/Float64/1024 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float64/1048576 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/static workgroup=(1024,)/Float64/16384 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float64/2048 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float64/256 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/static workgroup=(1024,)/Float64/262144 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
saxpy/static workgroup=(1024,)/Float64/32768 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float64/4096 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float64/512 8 allocs: 0.141 kB 8 allocs: 0.141 kB 1
saxpy/static workgroup=(1024,)/Float64/64 5 allocs: 0.0938 kB 5 allocs: 0.0938 kB 1
saxpy/static workgroup=(1024,)/Float64/65536 12 allocs: 0.25 kB 12 allocs: 0.25 kB 1
time_to_load 0.204 k allocs: 11.9 kB 0.2 k allocs: 11.8 kB 1.02

Benchmark Plots

A plot of the benchmark results have been uploaded as an artifact to the workflow run for this PR.
Go to "Actions"->"Benchmark a pull request"->[the most recent run]->"Artifacts" (at the bottom).

@vchuravy
vchuravy force-pushed the vc/device-timestamps branch 5 times, most recently from 3e1a30a to 990d507 Compare October 4, 2026 15:50
@vchuravy
Time kernels on the device without synchronizing (prototype)
cf18848 
Adds optional KernelInterface hooks, `record_timestamp(backend)` and
`elapsed_time(backend, start, stop)`, with a fallback that reports no
support. Tracers that set `records_kernels` get a `KernelTimer` for every
launch, with timestamps recorded around the native enqueue alone, so
compilation isn't counted. On backends without timestamps the launch
synchronizes and is timed on the host instead.

`@profile` uses this by default (`device = true`), resolving the
timestamps once the profiled expression has run, and reports host-side
and device-side activity separately. Launches no longer synchronize by
default.

Assisted-by: Claude Code (Opus 5.5)
@vchuravy
vchuravy force-pushed the vc/device-timestamps branch from 990d507 to cf18848 Compare October 4, 2026 17:39

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@vchuravy