[cuda] implement max_delta_step output cap on the CUDA tree learner

[maxwbuckley]

Problem: the delta

When training with device_type="cuda", max_delta_step was silently ignored. The leaf-output cap

if (max_delta_step > 0 && std::fabs(ret) > max_delta_step) {
  ret = Common::Sign(ret) * max_delta_step;
}

exists only in the CPU FeatureHistogram::CalculateSplittedLeafOutput (the USE_MAX_OUTPUT path). The CUDA leaf-output device functions had no max_delta_step parameter and no cap.

Measured delta before this PR (400×6 synthetic data, learning_rate=1.0, num_leaves=15, 5 rounds, gpu_use_dp=true; tree-0 leaf-value spread should be ≤ 2*max_delta_step):

objective	max_delta_step	leaf spread CPU	leaf spread CUDA	cap (2·mds)	max pred delta
binary	0.05	0.100	4.150	0.1	17.7
binary	0.1	0.200	4.150	0.2	17.7
binary	0.5	1.000	4.150	1.0	18.9
regression	0.05	0.100	4.017	0.1	2.48
regression	0.1	0.200	4.017	0.2	2.23
regression	0.5	1.000	4.017	1.0	0.66

CPU enforces the cap; CUDA's leaves are unbounded — predictions diverge by up to 17.7 (raw score).

Fix

Implements the cap on the CUDA tree learner, identical to the CPU formula, threaded as a runtime double (0 = inactive) through:

• CUDALeafSplits::CalculateSplittedLeafOutput / GetLeafGain / GetSplitGains (the cap sits between the L1 step and the smoothing step, exactly like CPU; the closed-form gain shortcut is bypassed when the cap is active, matching CPU's !USE_MAX_OUTPUT && !USE_SMOOTHING condition)
• the root-leaf init kernels (cuda_leaf_splits.cu)
• every best-split-finder kernel — numerical, categorical, global-memory, and discretized variants (cuda_best_split_finder.cu)
• the root SetLeafOutput and the refit kernels (cuda_single_gpu_tree_learner.cpp/.cu)

The first commit's Log::Fatal guard is removed: max_delta_step now works on CUDA instead of being rejected.

Result: the delta after the fix

Same measurement, post-fix:

objective	max_delta_step	leaf spread CPU	leaf spread CUDA	cap (2·mds)	max pred delta
binary	0.05	0.100	0.100 ✓	0.1	0.40 †
binary	0.1	0.200	0.200 ✓	0.2	0.40 †
binary	0.5	1.000	1.000 ✓	1.0	1.27 †
regression	0.05	0.100	0.100 ✓	0.1	0.15 †
regression	0.1	0.200	0.200 ✓	0.2	0.18 †
regression	0.5	1.000	1.000 ✓	1.0	4.4e-16 ✓

And in the non-degenerate regime (regression, learning_rate=0.1, 10 rounds, min_data_in_leaf=5):

max_delta_step	max pred delta
1.0	0.0 (bit-identical)
2.0	3.3e-16
5.0	4.4e-16

† Why some configs still differ structurally: with learning_rate=1.0 + min_data_in_leaf=1 + a tiny cap, every leaf saturates at ±max_delta_step, so all candidate split gains collapse onto a plateau of equal values. CPU and CUDA then pick different — but equally optimal — splits from the plateau (ULP-level FP tie-breaking, the known lightgbm-org#6055 family; same root cause PR #13 addresses). The cap itself is enforced identically (spread columns match exactly), and training loss agrees within 0.6% in those configs. This is a property of gain plateaus, not of the max_delta_step implementation.

Tests (in test_dual.py, gated on TASK=cuda)

Test	Cases	What it pins down
test_cuda_max_delta_step_caps_outputs_like_cpu	6	leaf spread ≤ 2·mds on both backends, spreads equal — fails on the old build (CUDA spread 4.15 vs cap 0.1)
test_cuda_max_delta_step_matches_cpu_exactly	3	bit-level prediction parity (atol=1e-10) in non-saturating configs
test_cuda_max_delta_step_loss_matches_cpu_when_saturated	2	cap enforced on every tree + training loss within 2% in the plateau regime

All 11 cases pass with the fix; the cap-enforcement test fails on the unpatched build. Full test_dual.py suite (65 tests) passes.

🤖 Generated with Claude Code

[BelixRogner]

Thank you, Max — and thank you, Claude Code (independently 🙂). Verdict: SOLID, no bug found.

Reviewed against the CPU USE_MAX_OUTPUT cap in CalculateSplittedLeafOutput:

• Cap formula is equivalent, and the Common::Sign(0)=0 vs CUDA ret>=0?+md:-md difference is provably unreachable — the branch is gated on fabs(ret) > max_delta_step > 0, so ret==0 never enters it. (Cosmetically a real Sign() would read more faithfully, but behavior is identical.)
• Cap sits in the correct position: L1 step → cap → smoothing, matching CPU.
• The closed-form gain shortcut is correctly bypassed when the cap is active — the CPU compile-time !USE_MAX_OUTPUT && !USE_SMOOTHING guard is faithfully turned into a runtime !USE_SMOOTHING && max_delta_step <= 0.0 guard.
• Every site is threaded: leaf-output/gain functions, root-init kernels, all four best-split-finder families + their global-memory variants, root SetLeafOutput, and the refit kernels. I grepped every CalculateSplittedLeafOutput/GetLeafGain/GetSplitGains call — no site takes the param without applying it, none missed. min_gain_shift propagates cap-aware, consistent with CPU.
• Default 0 is a true no-op (every application guarded by max_delta_step > 0).

Your gain-plateau scope note is honest: the structural divergence at learning_rate=1.0 is the expected ULP-tie signature, not a cap defect — the spread-exact and bit-parity assertions would break if it were a cap bug.

Non-blocking: no path_smooth>0 + cap, categorical, or lambda_l1>0 + cap test; core cap behavior is well covered.

Blocker is only the required ruff format + git merge master (these five overlap). macOS red = dask socket flake.

P.S. — you threaded max_delta_step through ~9 files and a dozen kernels without missing one, then capped your output at "ran the linter: no". 😄 ruff format has a step cap of exactly one command.