Skip to content

Validate atomics for the GCN target, and work around two LLVM atomic bugs - #997

Merged
maleadt merged 4 commits into
mainfrom
tb/gcn-atomics
Oct 7, 2026
Merged

maleadt merged 4 commits into
mainfrom
tb/gcn-atomics

Conversation

@maleadt

@maleadt maleadt commented Oct 6, 2026 •

Copy link
Copy Markdown
Member

Atomics reach the GCN back-end from AMDGPU.jl, UnsafeAtomics and Atomix, Enzyme and Julia's own intrinsics, and some of them don't compile or crash the compiler without pointing at the Julia code. Like #996 does for PTX, this validates them on the IR, and works around two LLVM bugs.

Validation rejects, with an InvalidIRError and a Julia backtrace:

  • atomics wider than 64 bits and under-aligned atomics. Julia's LLVM compiles them to libatomic calls (__atomic_compare_exchange_16), which only fail when the code object is loaded (hipErrorNoBinaryForGpu); the external back-end rejects them without Julia frames;
  • atomics on memory other than flat, global, local, private and buffer fat pointers, which the back-end rejects or aborts on;
  • unknown synchronization scopes. cluster is accepted when the LLVM that generates code knows it (LLVM 22+), which for the external back-end is AMDGPU_LLVM_Backend_jll's version rather than Julia's.

The workarounds expand atomics to compare-exchange loops after optimization, also when the IR isn't validated:

The tests check each rejection and, for the workarounds, the compile failure or crash without them (in a subprocess) and that the cases LLVM handles stay native. The GCN tests pass on Julia 1.10 through 1.13, and the full suite on 1.13 (at d1d5546). The AMDGPU_LLVM_Backend_jll 23.1.3 change (37e1d20) was checked with the GCN tests on Julia 1.13 and nightly (LLVM 22, which exercises the in-process workaround).

Atomics reach the GCN back-end from AMDGPU.jl, UnsafeAtomics and
Atomix, Enzyme and Julia's intrinsics, so check them on the IR like
the PTX target does. Julia's LLVM compiles atomics wider than 64 bits
or under-aligned ones to libatomic calls, which only fail when the
code object is loaded (hipErrorNoBinaryForGpu); the external back-end
rejects them without pointing at the Julia code that performs them.
Atomics on constant, region or buffer resource memory and unknown
synchronization scopes fail in the back-end or abort it.

The cluster scope is accepted when the LLVM that generates code knows
it (LLVM 22 and later), which for the external back-end is the
version of AMDGPU_LLVM_Backend_jll rather than Julia's.
Since LLVM 22, the AMDGPU back-end selects 32-bit `atomicrmw usub_sat`
natively on gfx10.3 and later, but it has no pattern for flat memory,
and on local memory only for gfx12, so it fails with "Cannot select"
(llvm/llvm-project#229442). This breaks UnsafeAtomics' saturating
subtraction on the external back-end (LLVM 23): on local memory for
gfx10.3 and gfx11, and on flat memory for gfx12, or for gfx10.3 and
gfx11 once the operation carries !amdgpu.no.fine.grained.memory.

Expand these operations to compare-exchange loops after optimization,
keyed on the version of the LLVM that generates code. Flat memory is
expanded on every target, where LLVM otherwise chooses between the
instruction and a loop based on scope and metadata. This runs whether
or not the IR is validated.
Before LLVM 21, AMDGPUAtomicOptimizer broadcasts the result of an 8-
or 16-bit atomic read-modify-write on a uniform address with an
illegal readfirstlane (fixed by llvm/llvm-project#128388). With
Julia's own LLVM this aborts the process (LLVM 20, Julia 1.13) or
crashes it (LLVM 18, Julia 1.12), e.g. for an Atomix `+=` on an Int8
array element whose result is used.

The optimizer can only be disabled through a process-wide option, so
expand the operations it handles to compare-exchange loops when their
result is used, without predicting uniformity. This only affects the
in-process back-end; the external one uses LLVM 23.
@codecov

codecov Bot commented Oct 7, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 88.73%. Comparing base (e6ff2e6) to head (d1d5546).
⚠️ Report is 5 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main     #997      +/-   ##
==========================================
+ Coverage   88.62%   88.73%   +0.11%     
==========================================
  Files          30       30              
  Lines        6443     6498      +55     
==========================================
+ Hits         5710     5766      +56     
+ Misses        733      732       -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

AMDGPU_LLVM_Backend_jll 23.1.3 carries the fix for llvm/llvm-project#229442,
so require it and keep expanding 32-bit usub_sat to a compare-exchange loop
only for the in-process back-end. On gfx12 flat memory this now selects the
native flat_atomic_sub_clamp_u32.
@maleadt
maleadt merged commit 32d4838 into main Oct 7, 2026
32 of 33 checks passed
@maleadt
maleadt deleted the tb/gcn-atomics branch October 7, 2026 06:45
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant