Skip to content

Keep the sign in integer powers of negative numbers - #3301

Merged
maleadt merged 2 commits into
JuliaGPU:mainfrom
timesselens:pr/fastmath-pow-negative-base
Sep 28, 2026
Merged

maleadt merged 2 commits into
JuliaGPU:mainfrom
timesselens:pr/fastmath-pow-negative-base

Conversation

@timesselens

@timesselens timesselens commented Sep 26, 2026 •

Copy link
Copy Markdown
Contributor

On the device, @fastmath x^n with x::Float32 or Float16 and an integer n outside -1:3 returned NaN for negative x. Literal exponents were affected too, since @fastmath x^7 goes through pow_fast(x, Val(7)) and then the same method.

k!(y, x, n) = (i = threadIdx().x; @inbounds y[i] = @fastmath x[i]^n; nothing)
x = CuArray(Float32[1.5, -0.5, 3, -2]); y = similar(x)
@cuda threads=4 k!(y, x, 7); Array(y)
# main:    Float32[17.085943, NaN, 2186.9993, NaN]
# this PR: Float32[17.085943, -0.007812505, 2186.9993, -128.0]

The override from #3098 calls __nv_fast_powf(x, Float32(n)), which computes exp2(n * log2(x)). This PR computes the power of abs(x) and copies the sign of x when n is odd, the same way LLVM's AMDGPU back-end expands a fast pown (AMDGPULibCalls.cpp). The parity comes from the integer n, because Float32(n) can round an odd exponent to an even one (16_777_217). With a literal exponent the sign handling compiles to a single copysign.f32, and positive bases give the same results as before.

The precise ^(::Float32/Float64, ::Integer) from #3218 had the same rounding problem: (-1f0)^16_777_217 returned 1. The second commit takes the sign from the integer exponent there as well.

Alternatives considered: __nv_powif (what CUDA C's pow(float, int) uses, and the equivalent of the CPU's llvm.powi) matches the CPU @fastmath result bit for bit and is more accurate (for example 0 against 5 ulp on (-0.5f0)^7, and 153 against 1726 ulp on 1.001f0^5000). But with an exponent that is only known at run time it is 1.6× slower for n = 7 and 5× slower for n = 1000 (RTX 5080), so pow_fast keeps the fast path.

timesselens and others added 2 commits September 28, 2026 18:24
`@fastmath x^n` with an integer `n` outside -1:3 was computed as
`__nv_fast_powf(x, Float32(n))`, that is exp2(n * log2(x)), which is NaN
for x < 0: `(-0.5f0)^7` gave NaN where the CPU gives -0.0078125. This
also affected literal exponents, which reach the same method through
`pow_fast(x, ::Val)`. Float64 goes through `__nv_pow` and was not
affected.

Take the power of `abs(x)` and copy the sign of `x` for odd `n`, which is
how LLVM's AMDGPU back-end expands a fast `pown`. The parity comes from
the integer exponent, so an odd exponent that Float32 rounds to an even
number (16_777_217) keeps its sign. The Float16 method now calls the
Float32 one with the integer exponent, so it gets the same handling.

Found while differentiating WaterLily.jl kernels with Enzyme on a GTX
1080.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
`^(::Float32, ::Integer)` converts the exponent and calls `__nv_powf`,
which handles negative bases, but the conversion can round an odd
exponent to an even one: `(-1f0)^16_777_217` gave 1. The same holds for
Float64 beyond 2^53. Take the sign from the integer exponent, as the fast
variant now does.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
@maleadt
maleadt force-pushed the pr/fastmath-pow-negative-base branch from 8f0df95 to d284892 Compare September 28, 2026 16:27
@maleadt maleadt changed the title Keep the sign in fast integer powers of negative Float32/Float16 numbers Keep the sign in integer powers of negative numbers Sep 28, 2026
@maleadt

maleadt commented Sep 28, 2026

Copy link
Copy Markdown
Member

LGTM, thanks! I also applied this to the non-fastmath ^(::Float32/Float64, ::Integer). And some other minor improvements: switched to copysign, which compiles to a single instruction for a literal exponent, and made the tests compare element by element, since the array-norm ≈ hid a Float16 sign flip.

@maleadt
maleadt merged commit 51513d4 into JuliaGPU:main Sep 28, 2026
1 check was pending
@codecov

codecov Bot commented Sep 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 85.90%. Comparing base (f8606eb) to head (d284892).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #3301   +/-   ##
=======================================
  Coverage   85.89%   85.90%           
=======================================
  Files         187      187           
  Lines       19022    19025    +3     
=======================================
+ Hits        16339    16343    +4     
+ Misses       2683     2682    -1     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants