Skip to content

[ingress][torch] Add benchmark functionality to torch.compile backend#193

Merged
tkarna merged 2 commits into
llvm:mainfrom
tkarna:torch-benchmark
Jun 17, 2026
Merged

[ingress][torch] Add benchmark functionality to torch.compile backend#193
tkarna merged 2 commits into
llvm:mainfrom
tkarna:torch-benchmark

Conversation

@tkarna

@tkarna tkarna commented Jun 17, 2026

Copy link
Copy Markdown
Contributor
  • JITFunction now has a benchmark(*args, nwarmup, nruns) function that can be used to time the kernel.
  • MLIRBackend.jit_function holds a reference to the latest compiled JITFunction.
  • Updated xegpu torch_matmul.py example to use this benchmark method.

Basic usage:

model = ... # create torch model
backend = gpu_backend(...)
model.compile(backend=backend)
outputs = model(input_a, input_b)
# accessing backend.jit_function raises an error if a model has not been compiled yet
times = backend.jit_function.benchmark(input_a, input_b, nwarmup=500, nruns=500)
avg_time = times.mean() * 1e6  # microseconds

@tkarna tkarna requested review from adam-smnk and rengolin June 17, 2026 11:22
@tkarna tkarna merged commit 5c44df5 into llvm:main Jun 17, 2026
3 checks passed
@tkarna tkarna deleted the torch-benchmark branch June 17, 2026 12:14
rengolin added a commit to rengolin/lighthouse that referenced this pull request Jun 17, 2026
Follow up from llvm#193, now we can run benchmarks inside torch-compile mode
rengolin added a commit that referenced this pull request Jun 17, 2026
Follow up from #193, now we can run benchmarks inside torch-compile mode
@tkarna

tkarna commented Jun 18, 2026

Copy link
Copy Markdown
Contributor Author

The torch_matmul.py exercises a simple torch model that does not have any parameters (e.g., layer weights). If the model has parameters, those must be prepended to the jit_fn.benchmark call arguments:

times = backend.jit_function.benchmark(*model.parameters(), *inputs, nruns=nruns, nwarmup=nwarmup)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants