Skip to content

Forward strided-view matrix products to the storage-level mul! - #800

Merged
maleadt merged 1 commit into
JuliaGPU:mainfrom
aalquwayfili:forward-strided-views-to-backend
Oct 16, 2026
Merged

maleadt merged 1 commit into
JuliaGPU:mainfrom
aalquwayfili:forward-strided-views-to-backend

Conversation

@aalquwayfili

@aalquwayfili aalquwayfili commented Sep 30, 2026 •

Copy link
Copy Markdown
Contributor

Fixes #799.

Since #760, any product involving a strided GPU view goes to GPUArrays' generic matmul kernel. That skips the storage-level mul!(C, tA, tB, A, B, α, β) that back-ends overload to call their vendor libraries, so mul!(C, view(A, 1:4000, :), B) no longer uses cuBLAS. This PR forwards plain matrix-matrix products with views back to that method when every matrix is a valid BLAS operand:

  • both wrapper chars are N, T or C;
  • every matrix has a unit row stride and a leading dimension (stride(X, 2)) of at least its number of rows;
  • the dimensions match.

Everything else stays on the generic kernels, as on main:

  • Row-stepping views and reversed columns: the gemm wrappers in AMDGPU.jl and oneAPI.jl pass stride(X, 2) as lda without checking the layout. A row-stepping view gives wrong results there, and reversed columns make cuBLAS reject the call.
  • Symmetric and Hermitian operands, and matrix-vector products: AMDGPU.jl's symm!, hemm! and gemv! only accept dense ROCArrays.
  • Mismatched dimensions: Metal.jl's native mul! doesn't check them, so leaving those to the generic kernel keeps the DimensionMismatch error.

The #799 MWE on an RTX 5080 with CUDA.jl 6.4.2 drops from 2.5 ms (generic kernel) to 0.26 ms (cuBLAS). I measured that with the change applied to GPUArrays 11.5.16, since no CUDA.jl release supports GPUArrays 12 yet. In the same setup, linalg/mul!/strided-views passes 69/69 on CuArray. The new reversed-column tests error with CUBLAS_STATUS_INVALID_VALUE without the leading-dimension check. On this branch, JLArray/linalg passes on Julia 1.10, 1.12 and 1.13.

@aalquwayfili
aalquwayfili force-pushed the forward-strided-views-to-backend branch from 230cb46 to 2229926 Compare October 11, 2026 13:33
@aalquwayfili
aalquwayfili marked this pull request as ready for review October 11, 2026 13:33
@aalquwayfili

Copy link
Copy Markdown
Contributor Author

I narrowed this. The first version forwarded every product with a view, and I had only tested it with CUDA.jl. Reading the other back-ends, it would have broken them:

  • AMDGPU.jl and oneAPI.jl pass any strided view to gemm! with lda = stride(A, 2) and no check on stride(A, 1), so view(A, 1:2:8, :) * B would have given wrong results.
  • AMDGPU.jl's gemv!, symm! and hemm! only take ROCMatrix and ROCVector, so a view there would have been a MethodError.

Now only plain matrix-matrix products with contiguous columns are forwarded. The rest stays on the generic kernels, as on main. I updated the description.

The part about AMDGPU.jl and oneAPI.jl is from reading their code, I have no AMD or Intel GPU. Buildkite has not really run on this PR before: build 2011 finished in 3 seconds because I pushed while it was still a draft. So this push is the first run on the other back-ends.

If you want symm and gemv forwarded too, I can add an opt in for back-ends that handle views there.

@maleadt
maleadt force-pushed the forward-strided-views-to-backend branch 2 times, most recently from 6f25d37 to 6f53c2d Compare October 15, 2026 20:24
@maleadt maleadt changed the title Forward strided-view products to the storage-level mul! methods Forward strided-view matrix products to the storage-level mul! Oct 15, 2026
@maleadt

maleadt commented Oct 15, 2026

Copy link
Copy Markdown
Member

LGTM, thanks. Squashed, rebased, and slightly reworked.

Since JuliaGPU#760, any product with a strided GPU view goes to the generic
matmul kernel, which bypasses the vendor libraries back-ends hook up
through the storage-level `mul!(C, tA, tB, A, B, α, β)`. For the view(A,
1:4000, :) * B case in JuliaGPU#799 that is about 10x slower than cuBLAS.

Forward plain matrix-matrix products (wrapper chars N, T or C) to that
method again when every matrix is a valid BLAS operand: unit row stride
and a leading dimension of at least the number of rows. The gemm
wrappers in the back-ends take stride(X, 2) as the leading dimension
without validating it, so row-stepping views would give wrong results
and reversed columns invalid arguments. Dimension mismatches also stay
on the generic kernel, which reports them, since Metal.jl's native
method does not. Symmetric and Hermitian operands and matrix-vector
products keep the generic kernels, as AMDGPU.jl's symm!, hemm! and gemv!
do not accept views.

Fixes JuliaGPU#799.

Co-authored-by: Tim Besard <tim.besard@gmail.com>
@maleadt
maleadt force-pushed the forward-strided-views-to-backend branch from 6f53c2d to 97ef6e8 Compare October 16, 2026 06:56
@maleadt
maleadt merged commit 85ac86f into JuliaGPU:main Oct 16, 2026
17 of 18 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Strided SubArray views of CuArray no longer reach cuBLAS in mul! since v11.5.11

2 participants