[None][feat] Add GLM-5.3-Flash (glm5_next) support

[ruocheng-nv]

Description

Adds GLM-5.3-Flash (zai-org/GLM-5.3-Flash, architecture Glm5NextForConditionalGeneration) to the TensorRT-LLM PyTorch backend using the official FP8 checkpoint.

Implements the GLM-5.3-Flash text decoder, native MTP, and vision tower using shared TensorRT-LLM modules. The decoder combines KDA linear attention, sparse MLA with a pool-based indexer, mHC, and MoE.

Shared-module changes:

• KDA (kimi_kda, KDA decode kernels): low-rank output gates, 64 attention heads, and FP32 gate parameters required by GLM-5.3-Flash, while preserving the existing full-rank gate path. The Kimi K3 KDA MTP decode kernel promotes the state-slot index to Int64 so large state offsets do not overflow (outputs bitwise identical for existing models).
• mHC: autotuner buckets cover the configured token range; bucket sets are unchanged for existing models at max_num_tokens <= 8192.
• MLA: the config check allows qk_rope_head_dim == 0, which GLM-5.3-Flash uses; other models are unaffected.
• Executor: routing to a GLM hybrid cache manager (KDA state + sparse-MLA indexer cache), per-layer head dimensions, multimodal engine and checkpoint-loader hooks.
• MoE A2A CI unblock: following @zhangcl's recommendation, this PR cherry-picks the NVIDIA kernel-driver gate ported from [None][refactor] overhaul NVLink one-sided all-to-all #19610, so CFT falls back to fence under CUDA forward compatibility. On these systems, a newer user-mode library can expose CFT entry points even when the kernel driver cannot create logical endpoints; without this gate, the reused MoE A2A tests fail in cuLogicalEndpointCreate and abort MPI independently of GLM validation.

Key validated features: TP4/EP4, CUDA graphs, overlap scheduling, chunked prefill, MTP, attention DP (including MTP), FP8 KV cache, prefix reuse with periodic state snapshots, image/video inputs, and text PD disaggregation with and without MTP. Validated on B200.

Documentation: see docs/source/deployment-guide/deployment-guide-for-glm-5.3-flash-on-trtllm.md (serving configs, benchmarking, and the 1K/1K performance chart). Text-only serving uses the Transformers version installed with TensorRT LLM; image and video inputs require transformers==5.17.0. Shared requirements are unchanged.

Release timing: This PR addresses an explicit customer request for GLM-5.3-Flash support.

Test Coverage

Correctness and accuracy

• Full GSM8K comparisons against HF match within 0.08 percentage points, with and without MTP3, under the matched thinking-mode protocol: 1,319 questions, greedy decoding, and a 4,096-token output budget.
• Separate full GSM8K regression runs using the repository harness pass for attention DP + MTP3, TP4/EP4 + FP8 KV, and attention DP + MTP3 + FP8 KV. Scores are 93.37%, 93.29%, and 93.86%, respectively. These use the 5-shot raw-completion protocol with a 256-token output budget and are not directly comparable to the HF thinking-mode scores above.
• Native HF image/video feature parity tests pass with transformers==5.17.0. Full-model validation also covers MMMU, video serving, snapshot-based prefix reuse, chunked prefill, runtime behavior, and NIXL disaggregated serving.
• Focused unit coverage checks checkpoint routing, layer masks, FP32 gate sharding, encoder-only construction, low-rank KDA projections, sparse-cache layout and dispatch, FP8 cache storage and graph replay, mixed attention-DP/MTP batches, and large KDA state offsets.

CI

Existing unit-test stages are reused (GLM contract tests are added to l0_b200). Full-model coverage adds one TP4/EP4 MTP3 GSM8K case to the existing four-GPU GB300 pre-merge stage (GB300-4_GPUs-PyTorch-1), with CUDA graphs, overlap scheduling, and chunked prefill enabled. Attention DP + MTP3 + FP8 KV and prefix reuse are registered in GB300 multi-GPU post-merge, and MMMU is added to QA. No new Jenkins stages are introduced.

Local resource reference: the MTP3 GSM8K test body took approximately 7.3 minutes on 4x B200, with 8.3 minutes total launcher wall time.

Performance

Measured on 4x B200 with FP8 weights and BF16 KV cache using benchmark_serving: 1,024 input / 1,024 output tokens, random token IDs, seed 0, greedy decoding, ignore_eos, natural MTP acceptance. Each concurrency is warmed up with 2 x C requests, then measured twice with 5 x C requests per run. Output tok/s/user is 1000 / mean TPOT and excludes TTFT; Output tok/s/GPU is aggregate output throughput over the complete request window, including prefill and TTFT, divided by four. The deployment guide chart uses a 2%-tolerant Pareto frontier over the measured TP4 / EP4 and attention DP4 / EP4 configurations (MTP draft length 0, 1, 3): a point is omitted when another point is within 2% of its per-user speed and has higher per-GPU throughput. The table below shows selected frontier points.

C	Frontier configuration	Output tok/s/user	Output tok/s/GPU
1	TP4 / EP4 + MTP3	389	92
4	TP4 / EP4 + MTP3	263	224
16	TP4 / EP4 + MTP3	153	556
64	TP4 / EP4 + MTP1	78	1,150
128	Attention DP4 / EP4 + MTP1	60	1,723
256	Attention DP4 / EP4 + MTP1	46	2,735

Without MTP, TP4 / EP4 reaches 5.57 ms mean TPOT at C=1 (180 tok/s/user) and 6,161 tok/s aggregate output throughput at C=128, so MTP3 improves single-user decode speed by about 2.2x on this synthetic workload. With attention DP, max_batch_size applies per rank (64 per rank for C=256). These measurements cover the tested configurations and concurrency range and do not establish peak throughput or maximum-context performance.

PR Checklist

Please review the following before submitting your PR:

• PR description clearly explains what and why.

• PR follows TRT-LLM Coding Guidelines to the best of your knowledge.

• Test cases are provided for new code paths (see test instructions).

• If PR introduces API changes, an appropriate PR label is added: api-compatible or api-breaking. For api-breaking, include BREAKING in the PR title.

• Any new dependencies have been scanned for license and vulnerabilities.

• CODEOWNERS is updated if ownership changes.

• Documentation is updated as needed.

• Update the architecture diagram if there is a significant design change.

• The assigned reviewers are appropriate for the PR.

• Please check this after reviewing the above items as appropriate for this PR.

[ruocheng-nv]

/bot run --disable-fail-fast

[tensorrt-cicd]

PR_Github #75486 [ run ] triggered by Bot. Commit: 1ec4cfe Link to invocation

[tensorrt-cicd]

PR_Github #75486 [ run ] completed with state SUCCESS. Commit: 1ec4cfe
/LLM/main/L0_MergeRequest_PR pipeline #62219 completed with status: 'SUCCESS'
Pipeline passed with automatic retried tests. Check the rerun report for details.

CI Report

Link to invocation