Skip to content

Capture PTX that programs load at run time - #174

Merged
CodingInAVan merged 3 commits into
mainfrom
capture-loaded-ptx
Oct 20, 2026
Merged

CodingInAVan merged 3 commits into
mainfrom
capture-loaded-ptx

Conversation

@CodingInAVan

@CodingInAVan CodingInAVan commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

Why

PTX was read only from the executable's fatbin with cuobjdump -ptx (DictionaryManager::flushPtx). cuda-oxide (Rust) kernels carry their PTX in an .oxart section and hand the text to cuModuleLoadData at run time (cuda-core 0.3.1 load_module_from_image), so cuobjdump finds nothing. Rust Performance Lab sessions in dev show SASS but every kernel's PTX request returns 204, while C++ sessions of the same labs have PTX (sm_89).

What

  • ModuleLoadPtxHandler copies the PTX passed to cuModuleLoadData, cuModuleLoadDataEx, and cuLinkAddData_v2 with CU_JIT_INPUT_PTX. It copies at API exit and only when the call returned CUDA_SUCCESS, so a rejected PTX cannot shadow a fallback with the same kernel names. It runs only where cubins are captured (NeedsCubinCapture).
  • Not captured: cuLibraryLoadData, which returns success before the driver compiles PTX (see Testing) and mostly carries the CUDA runtime's own fatbins. Loads by file name (cuModuleLoad, cuLinkAddFile, cuLibraryLoadFromFile) are not captured either. cuda-oxide, NVRTC users and Numba pass the image to the captured calls.
  • DictionaryManager::enqueueLoadedPtx keeps an image only when it is PTX text: optional whitespace, // lines and /* */ comments, then .version. ELF cubins and fatbins are rejected at their first byte. Reads stop at the NUL or the size limit. Each module is capped at 64 MiB, the pending total at 256 MiB, and an identical module is kept once.
  • The PTX flush emits the queued modules through the existing per-kernel parser, now emitPtxText. A kernel already emitted is not repeated. The one-time cuobjdump dump skips kernels the run-time path already emitted in the session; otherwise its output is unchanged, per-arch copies included.
  • flushDisassembly also drains loaded PTX when no cubins are pending, so a module loaded after the last flush is not lost.
  • reset(), which runs at every session start, now clears the PTX state too: the queued hashes, the emitted kernel names and the executable-dump flag. A later session in the same process gets PTX for the modules it loads, and runs cuobjdump again (before, a second session never ran it).

No backend change is needed: PTX joins kernels and SASS by md5 of the function name, and Rust kernel names in the PTX (layernorm_rows) match their SASS names.

Testing

  • LoadedPtxTest (3 tests):
    • A cuda-oxide-shaped module emits one ptx_disassembly per kernel, with its arch and module header. The same module loaded again adds nothing.
    • After reset(), the same module is emitted again for the new session. This test fails without the fix.
    • Only PTX text is recognized: ELF, fatbin, a cut-off image and an unclosed block comment are rejected, and a block-comment opening is accepted.
  • run_tests.py --cpp-only on Windows, CUDA 13.3: 644 tests passed.
  • Driver repro on an RTX 5060 Laptop GPU, Windows driver 610.62, with CUDA_MODULE_LOADING lazy, eager and unset, each case in its own process:
    • PTX with an unsupported .version fails with CUDA_ERROR_UNSUPPORTED_PTX_VERSION at the call for cuModuleLoadData, cuModuleLoadDataEx and cuLinkAddData, in every mode.
    • cuLibraryLoadData returns CUDA_SUCCESS for the same PTX and fails only at cuLibraryGetKernel / cuLibraryGetModule.
    • Valid PTX loads, resolves and launches through all four APIs.
  • Not yet run on a GPU in dev. Before release, in dev: a Rust lab session shows PTX beside SASS with matching kernel names, a C++ session still does, and the repro gives the same results on a Linux driver.

PTX was read only from the executable's fatbin with cuobjdump. cuda-oxide
(Rust) kernels hand their PTX to cuModuleLoadData at run time, so their
sessions had SASS but no PTX. A driver-API handler now copies PTX text passed
to cuModuleLoadData, cuModuleLoadDataEx, cuLibraryLoadData and cuLinkAddData
(binaries are ignored), and the PTX flush emits its kernels through the same
per-kernel parser. Kernels cuobjdump already emitted are not repeated, and
cuobjdump's own output is unchanged.
- Keep a module's PTX only when its load returned CUDA_SUCCESS (API exit),
  so a rejected PTX cannot shadow a fallback with the same kernel names.
- reset() clears the queued hashes, the emitted kernel names and the
  executable-dump flag, so a later session in the same process gets PTX for
  the modules it loads and runs cuobjdump again.
- The one-time cuobjdump dump skips kernels the run-time path already
  emitted in this session; its own per-arch copies are kept.
- PTX may open with a /* */ comment; one never closed is not PTX.
- Note that loads by file name (cuModuleLoad, cuLinkAddFile,
  cuLibraryLoadFromFile) are not captured.
cuLibraryLoadData returns success before the driver compiles PTX: on driver
610.62 an unsupported .version loads with CUDA_SUCCESS and fails only at
cuLibraryGetKernel / cuLibraryGetModule, with CUDA_MODULE_LOADING lazy, eager
or unset. Its success therefore does not prove the PTX is the code that runs,
and the API mostly carries the CUDA runtime's own fatbins.

cuModuleLoadData, cuModuleLoadDataEx and cuLinkAddData reject the same PTX
with CUDA_ERROR_UNSUPPORTED_PTX_VERSION before they return in all three
modes, so their success is final and they stay captured. cuda-oxide, NVRTC
users and Numba hand PTX to those.
@CodingInAVan
CodingInAVan merged commit d2dd769 into main Oct 20, 2026
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant