Repository navigation
Forward strided-view matrix products to the storage-level mul! - #800
Merged
maleadt merged 1 commit intoOct 16, 2026
Merged
Conversation
aalquwayfili
force-pushed
the
forward-strided-views-to-backend
branch
from
October 11, 2026 13:33
230cb46 to
2229926
Compare
aalquwayfili
marked this pull request as ready for review
October 11, 2026 13:33
Contributor
Author
|
I narrowed this. The first version forwarded every product with a view, and I had only tested it with CUDA.jl. Reading the other back-ends, it would have broken them:
Now only plain matrix-matrix products with contiguous columns are forwarded. The rest stays on the generic kernels, as on main. I updated the description. The part about AMDGPU.jl and oneAPI.jl is from reading their code, I have no AMD or Intel GPU. Buildkite has not really run on this PR before: build 2011 finished in 3 seconds because I pushed while it was still a draft. So this push is the first run on the other back-ends. If you want symm and gemv forwarded too, I can add an opt in for back-ends that handle views there. |
maleadt
force-pushed
the
forward-strided-views-to-backend
branch
2 times, most recently
from
October 15, 2026 20:24
6f25d37 to
6f53c2d
Compare
Member
|
LGTM, thanks. Squashed, rebased, and slightly reworked. |
Since JuliaGPU#760, any product with a strided GPU view goes to the generic matmul kernel, which bypasses the vendor libraries back-ends hook up through the storage-level `mul!(C, tA, tB, A, B, α, β)`. For the view(A, 1:4000, :) * B case in JuliaGPU#799 that is about 10x slower than cuBLAS. Forward plain matrix-matrix products (wrapper chars N, T or C) to that method again when every matrix is a valid BLAS operand: unit row stride and a leading dimension of at least the number of rows. The gemm wrappers in the back-ends take stride(X, 2) as the leading dimension without validating it, so row-stepping views would give wrong results and reversed columns invalid arguments. Dimension mismatches also stay on the generic kernel, which reports them, since Metal.jl's native method does not. Symmetric and Hermitian operands and matrix-vector products keep the generic kernels, as AMDGPU.jl's symm!, hemm! and gemv! do not accept views. Fixes JuliaGPU#799. Co-authored-by: Tim Besard <tim.besard@gmail.com>
maleadt
force-pushed
the
forward-strided-views-to-backend
branch
from
October 16, 2026 06:56
6f53c2d to
97ef6e8
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #799.
Since #760, any product involving a strided GPU view goes to GPUArrays' generic matmul kernel. That skips the storage-level
mul!(C, tA, tB, A, B, α, β)that back-ends overload to call their vendor libraries, somul!(C, view(A, 1:4000, :), B)no longer uses cuBLAS. This PR forwards plain matrix-matrix products with views back to that method when every matrix is a valid BLAS operand:N,TorC;stride(X, 2)) of at least its number of rows;Everything else stays on the generic kernels, as on main:
stride(X, 2)asldawithout checking the layout. A row-stepping view gives wrong results there, and reversed columns make cuBLAS reject the call.symm!,hemm!andgemv!only accept denseROCArrays.mul!doesn't check them, so leaving those to the generic kernel keeps theDimensionMismatcherror.The #799 MWE on an RTX 5080 with CUDA.jl 6.4.2 drops from 2.5 ms (generic kernel) to 0.26 ms (cuBLAS). I measured that with the change applied to GPUArrays 11.5.16, since no CUDA.jl release supports GPUArrays 12 yet. In the same setup,
linalg/mul!/strided-viewspasses 69/69 onCuArray. The new reversed-column tests error withCUBLAS_STATUS_INVALID_VALUEwithout the leading-dimension check. On this branch,JLArray/linalgpasses on Julia 1.10, 1.12 and 1.13.