perf: calibrate 4-bit GEMM dispatch for SM103

[heiheiha798]

Summary

• split B300 (SM103) dispatch calibration from the shared SM100/B200 path
• use the fused 4-bit GEMM through M=32 when N tiles exceed 3/4 of an SM wave
• keep M=33 on dequantize + F.linear and leave SM100/B200 behavior unchanged
• add focused SM103 threshold and SM100 regression tests
• add a tracked SM103 benchmark CLI with identity gating, configurable cases, and JSONL output

Closes #1.

Why

The existing major == 10 heuristic was calibrated with B200 data and grouped B300 with it. Direct measurements on a B300 SXM6 AC (148 SMs) show that the fused kernel wins consistently at near-wave occupancy through M=32, including square, wide-K, and wide-N model projections. The larger custom tile selected at M=33 generally loses, so this change keeps that boundary conservative.

Performance

Both paths were measured in the same Slurm allocation with alternating order. The native SM103 densification used 20 warmups and 7 rounds of 100 CUDA-event samples per path. Values below are fp16/NF4/blocksize 64/nested statistics; bf16 showed the same crossover.

After building the desired bitsandbytes CUDA library, the reported native crossover methodology is reproduced by running this command inside a one-B300 Slurm allocation:

PYTHONPATH=. python benchmarking/gemm_4bit_sm103_dispatch.py \
  --grid crossover \
  --warmup 20 \
  --repetitions 100 \
  --rounds 7 \
  --build-label native-sm103 \
  --output results/sm103-native-crossover.jsonl

The preset covers all 162 reported dtype/shape/M cells. --case NAME:N:K:M1,M2,... can be repeated to replace the preset, while --dtype, --blocksize, --quant-type, and --[no-]compress-statistics control focused runs. The CLI refuses to run unless exactly one NVIDIA B300 with compute capability 10.3 is visible. It records the Slurm job, node, existing Git commit ID, software/library identity, complete samples, percentiles, the current candidate decision as dispatch_custom, and custom/fallback correctness in JSONL.

N x K	M	baseline fallback (ms)	selected custom (ms)	speedup
7168 x 7168	16	0.123424	0.115200	7.14%
8192 x 8192	32	0.156192	0.135776	15.04%
8192 x 11008	32	0.205408	0.176896	16.12%
11008 x 4096	32	0.115360	0.100960	14.26%

Native build evidence:

• Slurm jobs: identity 4716, pilot/build 4721, densification 4724
• build: cmake -G Ninja -DCOMPUTE_BACKEND=cuda -DCOMPUTE_CAPABILITY=103 -DCMAKE_BUILD_TYPE=Release ...
• densification: 162 crossover cells, 700 samples per path
• correctness: max absolute custom/fallback difference 0.03125; max relative RMS error 0.00396316

Published-binary-compatible code paths were measured separately using the repository's CUDA 13 x86_64 target list (75;80;86;89;90;100;120), with 20 warmups and 5 rounds of 100 samples per path. The crossover matched the native build:

N x K	M	fallback (ms)	custom (ms)	speedup
7168 x 7168	16	0.123424	0.115168	7.17%
8192 x 8192	32	0.156224	0.133856	16.71%
11008 x 4096	32	0.115360	0.100960	14.26%
14336 x 4096	32	0.135776	0.102720	32.18%

• Slurm job: 4730
• 112 affected/boundary cells, 500 samples per path
• correctness: max absolute difference 0.03125; max relative RMS error 0.00397209

The pilot also found a possible custom win through M=64 for very large vocabulary projections. It is intentionally excluded here because the near-wave M<=32 rule has broader native and compatible coverage and a cleaner boundary.

Tracked-harness smoke on the native SM103 library:

• Slurm job 4752, commit e32fde2f51d9d7f2279db82580b514f67085a06e, B300 SXM6 AC (SM103, 148 SMs)
• command: --case tracked_smoke:8192:8192:31,32,33 --dtype fp16 --warmup 2 --repetitions 5 --rounds 2
• median custom/fallback ratios: M=31 0.8585, M=32 0.8807, M=33 1.2152; dispatch selected custom/custom/fallback respectively
• correctness: max absolute difference 0.001953125; max relative RMS error 9.08e-06

Validation

Slurm job 4772 on B300/SM103 at exact candidate commit dd86a57cd53862e2f21cbbc716bb48695d3a1f89:

• dispatch tests, including exact 3/4-wave boundaries: 14 passed
• existing test_gemm_4bit selection: 200 passed
• functional matmul_4bit: 1,442 passed, 864 skipped
• autograd matmul_4bit: 768 passed

The job used Python 3.12.3, PyTorch 2.13.0+cu130, CUDA 13.0, driver 580.126.09, and a B300 SXM6 AC with 148 SMs. Job 4771 exited at the shell commit gate before Python, GPU initialization, or tests because its wrapper contained an incorrect expected commit ID; it is recorded as a failed setup attempt and is not validation evidence.

Local static checks:

ruff check bitsandbytes/backends/cuda/ops.py tests/test_ops.py
ruff format --check bitsandbytes/backends/cuda/ops.py tests/test_ops.py
pre-commit run --all-files
BNB_TEST_DEVICE=cpu pytest -q tests/test_ops.py -k TestGemm4bitDispatch --tb=short

All ten pre-commit hooks passed, including full-repository Ruff, format, YAML, whitespace, typos, and clang-format checks. The CPU-only collection check used PyTorch 2.13.0+cpu (no CUDA runtime or raw-stream symbol) and cleanly collected then skipped only the 14 CUDA selector cases. No C++ kernels, build targets, public APIs, or non-SM103 dispatch thresholds are changed.