Skip to content

@Const can lead to wrong results on CPU in Julia 1.11/1.12 #652

Closed

Description

@maximilian-gelbrecht

MWE below @vchuravy

Works in 1.10, but not in 1.11/1.12

"""
Minimal working example to demonstrate the difference between:
1. Using `array[I] +=` in a loop (BROKEN)
2. Using a local accumulator then `array[I] =` (WORKS)

This appears to be a bug in KernelAbstractions.jl or me misunderstanding how it handles
compound assignment operators on array elements within loops.
"""

using KernelAbstractions
using Test

# Version 1: Using += directly on array element (BROKEN)
@kernel function accumulate_broken!(output, input, @Const(n))
    I = @index(Global, Cartesian)
    i = I[1]
    j = I[2]
    
    # This should accumulate input[i, k] for k in j:n
    # But it produces WRONG results!
    for k in j:n
        output[I] += input[i, k]
    end
end

# Version 2: Using local accumulator (WORKS)
@kernel function accumulate_fixed!(output, input, @Const(n))
    I = @index(Global, Cartesian)
    i = I[1]
    j = I[2]
    
    # Use local accumulator
    sum_val = zero(eltype(output))
    for k in j:n
        sum_val += input[i, k]
    end
    output[I] = sum_val
end

# Test function
function test_kernel_bug()
    println("="^70)
    println("Minimal Kernel Bug Demonstration")
    println("="^70)
    
    # Setup test data
    n = 8
    m = 5
    
    input = Float32[i + k for i in 1:m, k in 1:n]
    output_broken = zeros(Float32, m, n)
    output_fixed = zeros(Float32, m, n)
    output_cpu = zeros(Float32, m, n)
    
    println("\nInput matrix ($(m)×$(n)):")
    display(input)
    println("\n")
    
    # CPU reference implementation
    println("Computing CPU reference...")
    for i in 1:m
        for j in 1:n
            for k in j:n
                output_cpu[i, j] += input[i, k]
            end
        end
    end
    
    # Version 1: Broken (using +=)
    println("Running BROKEN kernel (using output[I] += ...)...")
    backend = CPU()
    kernel_broken! = accumulate_broken!(backend)
    kernel_broken!(output_broken, input, n, ndrange=size(output_broken))
    KernelAbstractions.synchronize(backend)
    
    # Version 2: Fixed (using local accumulator)
    println("Running FIXED kernel (using local accumulator)...")
    kernel_fixed! = accumulate_fixed!(backend)
    kernel_fixed!(output_fixed, input, n, ndrange=size(output_fixed))
    KernelAbstractions.synchronize(backend)
    
    # Compare results
    println("\n" * "="^70)
    println("RESULTS")
    println("="^70)
    
    println("\nCPU Reference:")
    display(output_cpu)
    println("\n")
    
    println("\nBROKEN Kernel (output[I] +=):")
    display(output_broken)
    println("\n")
    
    println("\nFIXED Kernel (local accumulator):")
    display(output_fixed)
    println("\n")
    
    # Check if broken version is actually broken
    is_broken_wrong = !(output_broken ≈ output_cpu)
    is_fixed_correct = output_fixed ≈ output_cpu
    
    return (is_broken_wrong, is_fixed_correct)
end

# Run the test
if abspath(PROGRAM_FILE) == @__FILE__
    test_kernel_bug()
end

Activity

  1. giordano commented on Nov 3, 2025

    @giordano
    Collaborator

    @maximilian-gelbrecht can you share the operating system (if Linux, also the distribution) and CPU where you can reproduce this issue?

  2. maximilian-gelbrecht commented on Nov 3, 2025

    @maximilian-gelbrecht
    Author

    MacOS 14.6.1 with M3

  3. milankl commented on Nov 3, 2025

    @milankl

    @simone-silvestri is that ☝🏼 the reason why Oceananigans doesn't run on 1.11 or did you fix this problem already?

  4. giordano commented on Nov 3, 2025

    @giordano
    Collaborator

    Yes, it seems to be the same issue because the offending Ocenanigans example works with #653, but I could reproduce the error in Oceananigans only on Linux and only on some machines, never on macOS, that's why I was curious about where @maximilian-gelbrecht observed the issue.

  5. simone-silvestri commented on Nov 3, 2025

    @simone-silvestri

    Yeah, this is the PR in question: CliMA/Oceananigans.jl#4836.
    Now we have other problems 😅, but they popped up just toda,y so we are hopeful that this week we can get oceananigans on > 1.11

  6. simone-silvestri commented on Nov 3, 2025

    @simone-silvestri

    MacOS always seemed to work for us

  7. giordano commented on Nov 3, 2025

    @giordano
    Collaborator

    Yeah, I didn't have problems with Oceananigans on M1 and M4, I tested on macOS 12.6 and 15.5

  8. vchuravy commented on Nov 3, 2025

    @vchuravy
    Member

    I tagged a KernelAbstraction release, but importantly you can't use @Const. If you annotate a function with @Const it is still possible to hit this bug

  9. maximilian-gelbrecht commented on Nov 4, 2025

    @maximilian-gelbrecht
    Author

    Great that the fix was so quick! @vchuravy Shall I close this issue or do you want to keep it open for related issues?

  10. vchuravy commented on Nov 19, 2025

    @vchuravy
    Member

    It's sadly still broken if the user uses @Const, so let's leave it open.

  11. changed the title [-]Accumulating in Arrays can lead to wrong results on CPU in Julia 1.11/1.12[/-] [+]`@Const` can lead to wrong results on CPU in Julia 1.11/1.12[/+] on Nov 19, 2025
  12. added a commit that references this issue on Jul 15, 2026
  13. stillyslalom commented on Sep 24, 2026

    @stillyslalom

    Likely fixed by JuliaLang/julia#63130 for 1.14.

  14. added
    kernelsWriting and compiling kernels: kernel language, intrinsics, codegen, back-end interface.
    poclThe PoCL-based CPU back-end.
    on Oct 19, 2026
  15. added theissue type on Oct 19, 2026
  16. maleadt commented on Oct 19, 2026

    @maleadt
    Member

    Fixed in 0.10: the CPU back end now compiles kernels through PoCL (#556), where @Const arrays use !invariant.load instead of Julia's @aliasscope, and @kernel no longer emits aliasscope regions at all (#785). Both kernels of the MWE, also with @Const on input instead of n, give correct results on Julia 1.12 and 1.13. The underlying Julia miscompilation is fixed by JuliaLang/julia#63130.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    kernelsWriting and compiling kernels: kernel language, intrinsics, codegen, back-end interface.poclThe PoCL-based CPU back-end.

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions