feat(musa): 26 training-hot-path mudnn kernels

[lvyufeng]

Summary

Extends the MUSA backend's mudnn coverage by 26 operators, chosen for the training
hot path. Coverage goes 64 -> 90 kernels. This is a performance change: every one of
these ops previously reached cpu_fallback, which round-trips the data D2H+H2D.

All of it is table-driven through the existing scripts/codegen_mudnn.py -- the two
gates (symbol gate against libmudnn.so, wrapper gate against
csrc/aten/generated/register.inc) and the idempotent conf append are unchanged.
Template count grows with call shape, not with op count, which is what makes the
batch cheap: 26 ops needed 14 new templates.

New ops, by shape:

• forward unary: tan round mish hardswish hardsigmoid isnan isinf
• parameterized unary: leaky_relu elu softplus clamp clamp_min clamp_max
• binary: logical_xor floor_divide
• ternary: addcmul addcdiv where.self
• matmul: addmm baddbmm
• activation backwards: sigmoid_backward tanh_backward silu_backward
gelu_backward threshold_backward leaky_relu_backward

Three things the mudnn headers actively mislead about

Each was pinned with a standalone C++ probe against libmudnn.so v3300 (no torch),
comparing to CPU formulas -- not assumed from the header:

• Unary::CLIP is (alpha=lo, beta=hi) and beta defaults to 0, so clamp_min
  must set beta to +inf explicitly or everything above 0 is clipped away.
• Unary::SOFTPLUS's setters are inverted vs aten: SetAlpha carries aten's
  beta, SetBeta carries aten's threshold. With neither set it returns inf.
• Unary::HARDSIGMOID is clamp(alpha*x + beta, 0, 1) with both defaulting to 0,
  i.e. all-zeros unless configured. aten's is alpha=1/6, beta=0.5.

Binary's 12 *_BW modes return status 0 for either operand order, so only the
numbers reveal the convention: SIGMOID_BW/TANH_BW take (grad, output), while
SILU_BW/GELU_*_BW/LEAKY_RELU_BW/THRESHOLD_BW take (grad, input).

addmm has three branches

MatMul computes d = alpha*A@B + beta*C + gamma*bias, and RunWithBiasAdd has two
forms with different algebra. The 3-arg form reads the output buffer for its beta
term (unusable here -- it would read uninitialized memory), so this uses the 4-arg
form. Which coefficient carries aten's beta then depends on the shape of self:

self	call	aten beta rides on
same shape as out	RunWithBiasAdd(d, a, b, c, {})	beta
1-D, length N	RunWithBiasAdd(d, a, b, d, bias) (c aliases d)	gamma
scalar or [M,1]	plain Run, then out.add_(self, beta)	host-side

MatMul/BatchMatMul also reject any non-contiguous tensor ("MatMulRun only support
contiguous tensor"), 0-strided C included -- hence the .contiguous() calls, and why
a 1-D self routes through the vector bias slot rather than being expanded.

Ops deliberately left on cpu_fallback

Authoritative mode counts from the header are Unary 67 / Binary 38, and they
contain no equivalent for bitwise_and/or/xor/not, gcd, lcm, logical_not,
HARDSWISH_BW, HARDSIGMOID_BW or SOFTPLUS_BW. relu6 and rsqrt_backward have
no wrapper in register.inc (both CompositeImplicitAutograd, so they decompose
anyway). Unregistered ops reach cpu_fallback and stay correct; registering an op
with no kernel behind it would instead fail the dispatcher's "backend not registered"
check -- so anything uncertain stayed out.

Test plan

Environment: torch==2.10.0+cpu, 8x MTT S5000, built with --no-build-isolation.

• Numerics vs CPU, every new op, float32 + float16 + int64 where applicable:
  fp32 maxerr <= 2e-5. The addmm fp16 cases differ by 0.0039, which is 1 ULP at
  magnitude ~5 (2^-8), not a miscompute.
• Backward: all 12 *_BW modes checked against torch.ops.aten.*_backward, plus
  end-to-end autograd through relu/gelu/silu/sigmoid/tanh/leaky_relu/elu/softplus/
  hardsigmoid/mish and linear's x/w/b gradients.
• Suite: pytest tests/ gives 482 passed / 209 skipped / 3 xpassed, against a
  478-passed baseline -- passed only goes up. (The 4 failures are tests/manual/metax
  vendor tests AttributeError-ing on a MUSA box; the 9 errors are the known missing
  transformers. Neither is touched by this change.)
• Performance, 2048x2048 fp32, each kernel vs the fallback it replaces: all 20
  benchmarked ops are faster, from 3.66x (linear) to 41x (addcmul); softplus
  35x, gelu_backward 31x, floor_divide 23x. Nothing regressed, so nothing was
  removed from the table.
• ruff==0.15.12 check + format clean.
• Only MUSA-only files change (backends/musa/, codegen_mudnn.py,
  backends_musa.conf), so cuda/gcu are unaffected by construction. A
  configure-only check for them is not runnable on this box -- it has no CUDA
  toolkit, so cmake stops at CMakeDetermineCUDACompiler.

Regenerating with python scripts/codegen_mudnn.py at this commit reproduces the
checked-in generated files exactly.

🤖 Generated with Claude Code