AOT-compiling what are normally JIT kernels

[sayakpaul]

Some internal points:

precompiling PTX from triton kernels so users could fetch workload specific binaries without any JIT overhead

The second (AOT-compiling what are normally JIT kernels) is very impactful:

• It's a feature that pip-installing the kernels does not have (unless the developers hacked something in, but there is no generalized method), so using a Hub kernel would provide a large benefit over a pip install.
• JIT-compilation is a real issue for ephemeral inference containers where boot-up time can be really important.