[None][feat] Helix speculative verify groups: fp8 + fp4 MLA and DSpark by reasonsolo · Pull Request #19273 · NVIDIA/TensorRT-LLM

[None][feat] add Helix speculative verify-group support to the MLA kernels

reasonsolo

committed

58758a8

[None][feat] wire the Helix verify-group runtime through the FP4 MLA path

reasonsolo

committed

d5ac4cd

[None][feat] extend Helix verify groups to the CuTe DSL MLA decode backend

reasonsolo

committed

3ff6554

[None][feat] support Helix verify groups on the fp8 CuTe DSL MLA decode

reasonsolo

committed

606cd25

[None][feat] enable DSpark speculative decoding under Helix CP for Kimi K3

reasonsolo

committed

0b83838

[None][fix] let CuTeDSL MLA serve 96 heads once multi-token decode is allowed

reasonsolo

committed

8a8d77e

[None][fix] import mamba_effective_tp_size from the parent package

reasonsolo

committed

16962d9

[None][fix] pack Helix host state in the overlap extend and generation paths

reasonsolo

committed

205ff61

[None][fix] use the global Helix position for the Q1 K-residual RoPE lookup

reasonsolo

committed

b8abb3f

[None][fix] count only generation tokens in the Helix verify-group reject

reasonsolo

committed

5b0f93a

[None][fix] reject a nonpositive zeroKvMaskDivisor before launch

reasonsolo

committed

99f34b0

[None][test] cover the shared Mamba sharding rule

reasonsolo

committed

d6e41e2

[None][fix] arm the per-token Helix metadata for speculative decoding only

reasonsolo

committed

0a74206

[None][fix] forward kv_bounds into the FP8 CuTe DSL MLA decode runner

reasonsolo

committed

57e2ca1

[None][fix] gate the Helix spec arming at a single choke point

reasonsolo

committed

4a2e323

[None][chore] drop the dead num_ctx_tokens parameter from the helix recompute

reasonsolo

committed

b5443ff

[None][fix] reject non-uniform Helix verify groups instead of mis-slicing

reasonsolo

committed

568a315

[None][chore] read _helix_spec_tokens_valid directly where the type is known

reasonsolo

committed

7f44677

[None][chore] use the descriptive mamba_effective_tp_size name at call sites

reasonsolo

committed

f8a7c97

[None][chore] drop the model name from the generic helix x speculation error

reasonsolo

committed

314398b

[None][chore] report when the Helix NCCL reformat outgrows dynamo's cache

reasonsolo

committed

00937dc

[None][doc] state the single-token assumption in _set_helix_rank_fields

reasonsolo

committed

b15c645

[None][doc] state the generation-only index base of the helix kv_lens branch

reasonsolo

committed

19ba373

[None][fix] pair the helix per-sequence buffers with the generation slice

reasonsolo

committed

69fc33d

[None][fix] pass the new kv_bounds argument in the standalone MLA decode runners

reasonsolo

committed

554da98

[None][fix] narrow the H=96 CuteDSL exception to multi-token Helix groups

reasonsolo

committed

c7efa36

[None][chore] give the helix round-robin local length one definition

reasonsolo

committed

23f430b

[None][fix] reject max_concurrency under Kimi K3 helix speculation

reasonsolo

committed

3083a1a

[None][fix] reject draft_len_schedule under Kimi K3 helix speculation

reasonsolo

committed

68a85f1

[None][fix] give the attention metadata stubs the helix fields the checks read

reasonsolo

committed

15abfa2

[None][fix] give the QSA model_config stub the fields the helix manager check reads

reasonsolo

committed

fc706b4

[None][fix] make the helix spec_config stub a concrete non-DSpark mode

reasonsolo

committed

0a5c71a

[None][fix] index the MLA tactic elements past the new kv_bounds cache key

reasonsolo

committed

99b2488

Navigation Menu

[None][feat] Helix speculative verify groups: fp8 + fp4 MLA and DSpark - #19273

[None][feat] Helix speculative verify groups: fp8 + fp4 MLA and DSpark#19273
reasonsolo merged 33 commits into
NVIDIA:mainfrom
reasonsolo:user/lizhiz/mb-helix

Commits on Sep 21, 2026

[None][feat] add Helix speculative verify-group support to the MLA kernels

[None][feat] wire the Helix verify-group runtime through the FP4 MLA path

[None][feat] extend Helix verify groups to the CuTe DSL MLA decode backend

[None][feat] support Helix verify groups on the fp8 CuTe DSL MLA decode

[None][feat] enable DSpark speculative decoding under Helix CP for Kimi K3

[None][fix] let CuTeDSL MLA serve 96 heads once multi-token decode is allowed

[None][fix] import mamba_effective_tp_size from the parent package

[None][fix] pack Helix host state in the overlap extend and generation paths

[None][fix] use the global Helix position for the Q1 K-residual RoPE lookup

[None][fix] count only generation tokens in the Helix verify-group reject

[None][fix] reject a nonpositive zeroKvMaskDivisor before launch

[None][test] cover the shared Mamba sharding rule

[None][fix] arm the per-token Helix metadata for speculative decoding only

[None][fix] forward kv_bounds into the FP8 CuTe DSL MLA decode runner

[None][fix] gate the Helix spec arming at a single choke point

[None][chore] drop the dead num_ctx_tokens parameter from the helix recompute

[None][fix] reject non-uniform Helix verify groups instead of mis-slicing

[None][chore] read _helix_spec_tokens_valid directly where the type is known

[None][chore] use the descriptive mamba_effective_tp_size name at call sites

[None][chore] drop the model name from the generic helix x speculation error

[None][chore] report when the Helix NCCL reformat outgrows dynamo's cache

[None][doc] state the single-token assumption in _set_helix_rank_fields

[None][doc] state the generation-only index base of the helix kv_lens branch

[None][fix] pair the helix per-sequence buffers with the generation slice

[None][fix] pass the new kv_bounds argument in the standalone MLA decode runners

[None][fix] narrow the H=96 CuteDSL exception to multi-token Helix groups

[None][chore] give the helix round-robin local length one definition

[None][fix] reject max_concurrency under Kimi K3 helix speculation

[None][fix] reject draft_len_schedule under Kimi K3 helix speculation

[None][fix] give the attention metadata stubs the helix fields the checks read

[None][fix] give the QSA model_config stub the fields the helix manager check reads

[None][fix] make the helix spec_config stub a concrete non-DSpark mode

[None][fix] index the MLA tactic elements past the new kv_bounds cache key