[Klaud Cold] qwen3.8next-fp8-h200-sglang-agentic-mtp: Qwen3.8-Flash-Next FP8 SGLang AgentX on H200 / H200 上 Qwen3.8-Flash-Next FP8 SGLang AgentX 配方 - #2753
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
…link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline stale-comment finding, I also checked the lmsysorg/sglang:qwen38flashnext image pin flagged as non-standard — the config already has several other floating/non-digest tags (vllm/vllm-openai:kimi-k3, lmsysorg/sglang-staging:dev-cu13-pr-35880, ...-c185066 commit-suffix tags), so this isn't a deviation from repo convention.
Extended reasoning...
One inline finding (stale copy-pasted comment referencing Qwen3.5's golden AL rationale) is already queued for posting. Beyond that, I independently verified the image-pinning candidate that was ruled out in this run: grepping all image: entries in configs/nvidia-master.yaml shows multiple existing recipes already use non-semver, non-date, non-sha256-pinned tags (e.g., vllm/vllm-openai:kimi-k3, lmsysorg/sglang-staging:dev-cu13-pr-35880, and several ghcr.io#semianalysisai/trtllm-deepseek-v4:feat-deepseek_v4-c185066 commit-suffix tags), so the new lmsysorg/sglang:qwen38flashnext tag is consistent with established repo practice rather than an outlier deserving a flag.
| # AgentX pins acceptance to the committed golden AL so submissions are compared | ||
| # on system performance at a fixed acceptance target rather than on draft-head | ||
| # quality (golden_al_distribution/README.md). 3.39 is the Qwen3.5 MTP curve at | ||
| # num_speculative_tokens=3, thinking_on (golden_al_distribution/qwen3.5_mtp.yaml) | ||
| # -- the same value the GB300 Qwen3.5 AgentX srt-slurm recipes pin. | ||
| # SGLANG_SIMULATE_ACC_TOKEN_MODE landed in SGLang v0.5.16, which is why this | ||
| # recipe pins that image rather than the non-MTP agentic sibling's v0.5.12. | ||
| # |
There was a problem hiding this comment.
🟡 Stale copy-pasted comment block still describes the Qwen3.5 golden AL rationale (3.39, golden_al_distribution/qwen3.5_mtp.yaml, GB300 srt-slurm pin, v0.5.16 vs v0.5.12 image reasoning) even though the code three lines below sets SGLANG_SIMULATE_ACC_LEN=3.24 for Qwen3.8next with its own correct rationale comment.
Extended reasoning...
A reviewer or future maintainer reading lines 118-125 sees a rationale for '3.39' tied to Qwen3.5's golden curve and an SGLang version-pin justification that does not apply to this recipe (image is lmsysorg/sglang:qwen38flashnext, not a version tag), creating confusion about why the actual exported value (3.24) and its interim/unverified status differ from the documented golden-AL claim, risking someone 'fixing' the value back to 3.39 or trusting the wrong image-pin justification.
Verification: nit — comment-only inconsistency, no runtime effect. Lines 118-124 are a stale copy-pasted comment block: "3.39 is the Qwen3.5 MTP curve at num_speculative_tokens=3, thinking_on (golden_al_distribution/qwen3.5_mtp.yaml) -- the same value the GB300 Qwen3.5 AgentX srt-slurm recipes pin. SGLANG_SIMULATE_ACC_TOKEN_MODE landed in SGLang v0.5.16, which is why this recipe pins that image rather than…
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33035462357 |
…AgentX on H200 Add the Qwen3.8-Flash-Next AgentX recipe on H200, served by SGLang with native NEXTN MTP. H200 is Hopper, so this arm serves the FP8 checkpoint rather than the NVFP4 one the Blackwell arms use: NVFP4 needs SM100 tensor cores. TP8 keeps the 172.8 GiB checkpoint near 22 GiB per rank, and the attention backend stays flashinfer for sm_90. Throughput runs pin an interim acceptance length of 3.24 at three speculative tokens; eval-only runs keep real target verification. 新增 H200 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 以原生 NEXTN MTP 提供 服务。H200 属 Hopper 架构，无 SM100 张量核心，无法运行 NVFP4，故本分支使用 FP8 权重而非 Blackwell 分支所用的 NVFP4。TP8 下每卡权重约 22 GiB，注意力后端保持 sm_90 适用的 flashinfer。吞吐运行按 3 个投机 token 锁定临时接受长度 3.24，仅评测 运行仍使用真实目标验证。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The recipe was copied from the Qwen3.5 FP8 H200 sibling and inherited TP8/EP1 plus that model's EAGLE speculative algorithm. The SGLang cookbook playground's verified low-latency single-node command for Qwen3.8-Flash-Next on H200 is --tp 4 --ep 4, sharding the 512-expert MoE with expert parallelism rather than sharding attention eight ways, with memory fraction 0.85, chunked prefill 8192, the flashinfer linear-attention prefill and decode backends, bfloat16 Mamba SSM and NEXTN rather than EAGLE. NVFP4 is greyed out for Hopper in the cookbook, confirming FP8 for this arm. Acceptance length moves from the interim 3.24 to the committed golden thinking_on value of 2.32. 本配方原样照搬自 Qwen3.5 FP8 H200 同类配方，带入了 TP8/EP1 与该模型的 EAGLE 投机算法。SGLang cookbook playground 给出的 Qwen3.8-Flash-Next H200 低时延单节点 验证命令为 --tp 4 --ep 4，即以专家并行切分 512 专家 MoE，而非把注意力切成八份； 并使用显存占用 0.85、chunked prefill 8192、flashinfer 线性注意力预填充与解码后端、 bfloat16 Mamba SSM，投机算法为 NEXTN 而非 EAGLE。cookbook 中 Hopper 的 NVFP4 为不可选，印证本分支应使用 FP8。接受长度由临时值 3.24 改为已提交黄金曲线的 thinking_on 取值 2.32。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
b31a482 to
65a1a76
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33036574839 |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 65a1a76. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33036807681 |
The H200 run died at server launch with qwen3.8next_fp8_h200_sglang_mtp.sh: line 184: MAX_RUNNING_REQUESTS: unbound variable The Qwen3.5 H200 template this was derived from leaves --max-running-requests and --cuda-graph-max-bs commented out, so unlike the B200, B300 and MI355X templates it never defines either variable. The rewrite to the cookbook command passes both flags, and the script runs under set -u. Define them the way the other three arms do: two times the AgentX concurrency for the scheduler, and graph capture at the concurrency capped to 64. Keeping --max-running-requests explicit also matters here because NEXTN silently resets it to 48 when it is unset. Audited the serve command of all four arms for other variables that are neither assigned in the script nor supplied by the matrix: none. H200 运行在启动服务时失败：MAX_RUNNING_REQUESTS 未定义。本脚本所参照的 Qwen3.5 H200 模板把 --max-running-requests 与 --cuda-graph-max-bs 注释掉了，因此不同于 B200、B300 与 MI355X 模板，它从未定义这两个变量；而改写为 cookbook 命令后这两个 参数都会传入，且脚本运行在 set -u 下。 现按其余三个分支的做法定义：调度器取 AgentX 并发的两倍，图捕获取并发值并以 64 封顶。此处保持 --max-running-requests 显式设置同样重要，因为 NEXTN 在其未设置时 会静默重置为 48。已审计四个分支服务命令中所有既未在脚本内赋值、也非由矩阵提供的 变量：无遗漏。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33037877255 |
The H200 cells got all the way to CUDA graph capture and died there: AssertionError: initial_state must be float32, got torch.bfloat16 flashinfer/gdn_decode.py:761 in gated_delta_rule_mtp via gdn_backend.py target_verify, during "Capture target verify CUDA graph" With NEXTN on, the GDN linear-attention backend routes verification through flashinfer's gated_delta_rule_mtp, and that kernel requires the SSM state in float32. The cookbook command pairs --mamba-ssm-dtype bfloat16 with NEXTN, but this flashinfer build rejects the combination. Of the two halves, the state dtype is the one that can move: the linear-attention backends are what the cookbook verifies, and dropping speculative decoding would defeat the recipe. H200 各单元一直跑到 CUDA graph 捕获阶段才失败：flashinfer 的 gated_delta_rule_mtp 断言 initial_state 必须为 float32，而实际为 bfloat16。启用 NEXTN 后，GDN 线性注意力后端的验证路径会走到该内核，而它要求 SSM 状态为 float32。cookbook 命令把 --mamba-ssm-dtype bfloat16 与 NEXTN 并列，但当前 flashinfer 构建不接受该组合。两者之中可调整的是状态精度：线性注意力后端正是 cookbook 所验证的部分，而去掉投机解码则会使该配方失去意义。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33038487711 |
|
/reuse-sweep-run |
merge_with_reuse.sh refused PR #2753 with ERROR: new changelog entries must be separated from history by one empty line and appended at the end validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line. Main's perf-changelog.yaml now ends "...pull/2688\n", so the entry needed one blank line ahead of it and did not have one. Historical bytes were already exact; this only inserts the separator. merge_with_reuse.sh 因变更记录格式校验拒绝了 PR #2753：追加条目必须与历史之间空一 行。validate_perf_changelog.py 要求追加部分以 "\n- config-keys:" 开头（除非基础 文件本身以空行结尾），而当前 main 的 perf-changelog.yaml 以 "...pull/2688\n" 结尾， 因此需要在条目前补一个空行。历史字节本就完全一致，此处仅插入该分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
/reuse-sweep-run |
|
/reuse-sweep-run |
Same fix as PR #2753. validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line, and merge_with_reuse.sh could not resolve the perf-changelog conflict without it: ERROR: new changelog entries must be separated from history by one empty line and appended at the end Historical bytes were already exact; this only inserts the separator. 与 PR #2753 相同的修复。validate_perf_changelog.py 要求追加部分以 "\n- config-keys:" 开头（除非基础文件本身以空行结尾），缺少该空行时 merge_with_reuse.sh 无法解决 perf-changelog 冲突。历史字节本就完全一致，此处仅插入 分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ext NVFP4 SGLang AgentX on B300 / B300 上 Qwen3.8-Flash-Next NVFP4 SGLang AgentX 配方 (#2752) * qwen3.8next-fp4-b300-sglang-agentic-mtp: day-zero Qwen3.8-Flash-Next AgentX on B300 Add the Qwen3.8-Flash-Next AgentX recipe on B300, served by SGLang with native NEXTN MTP. Mirrors the B200 arm: RadixArk/Qwen3.8-Flash-Next-NVFP4 at TP4 with modelopt_fp4 quantization, following the Qwen3.5 NVFP4 B300 sibling. Throughput runs pin an interim acceptance length of 3.24 at three speculative tokens; eval-only runs keep real target verification. 新增 B300 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 以原生 NEXTN MTP 提供 服务。与 B200 分支一致：使用 RadixArk/Qwen3.8-Flash-Next-NVFP4，TP4， modelopt_fp4 量化，参照 Qwen3.5 NVFP4 B300 同类配方。吞吐运行按 3 个投机 token 锁定临时接受长度 3.24，仅评测运行仍使用真实目标验证。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fill perf-changelog pr-link for #2752 / 补全 #2752 的 perf-changelog pr-link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Use the cookbook's verified TP1 command and the committed golden AL Same correction as the B200 arm. The SGLang cookbook playground's verified single-node command for Qwen3.8-Flash-Next on B300 is --tp 1: 126 GiB of NVFP4 weights fit on one 288 GB part. It takes the hybrid GDN linear-attention backends rather than a single --attention-backend, reads quantization from the checkpoint, and drops the trtllm_mha / flashinfer_trtllm MoE flags inherited from the Qwen3.5 recipe. --max-running-requests stays explicit, since NEXTN silently resets it to 48 when unset. Acceptance length moves from the interim 3.24 to the committed golden thinking_on value of 2.32. 与 B200 分支相同的修正。SGLang cookbook playground 给出的 Qwen3.8-Flash-Next B300 单节点验证命令为 --tp 1：126 GiB 的 NVFP4 权重可放入单张 288 GB 卡。该命令 改用混合 GDN 线性注意力的独立后端，量化直接从权重读取，并去掉了沿袭自 Qwen3.5 配方的 trtllm_mha 与 flashinfer_trtllm MoE 参数。--max-running-requests 保持显式 设置，因为 NEXTN 在其未设置时会静默重置为 48。接受长度由临时值 3.24 改为已提交 黄金曲线的 thinking_on 取值 2.32。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Use a float32 Mamba SSM state so the NEXTN verify kernel accepts it flashinfer's gated_delta_rule_mtp asserts initial_state.dtype == torch.float32 and aborts CUDA graph capture on the bfloat16 SSM state the cookbook command specifies. Confirmed on the H200 arm, which died at "Capture target verify CUDA graph"; this arm runs the same GDN backend, the same kernel and the same NEXTN, so it would fail identically once it got that far. flashinfer 的 gated_delta_rule_mtp 断言 initial_state 必须为 float32，遇到 cookbook 命令指定的 bfloat16 SSM 状态会在 CUDA graph 捕获阶段中止。该问题已在 H200 分支确认；本分支使用相同的 GDN 后端、相同内核与相同的 NEXTN，运行到同一阶段 必然同样失败。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Keep the bfloat16 SSM state on Blackwell My previous commit copied the H200 float32 fix here, and SGLang refused to start at all: ValueError: --linear-attn-decode-backend flashinfer on SM100+ requires --mamba-ssm-dtype bfloat16, got 'float32' The two architectures want opposite things. On SM100+ the flashinfer linear attention decode backend requires a bfloat16 state and validates it in server_args before launch. On Hopper the same backend's MTP verify kernel asserts a float32 state during CUDA graph capture. So the cookbook's bfloat16 is correct here and the H200 arm is the exception, not the other way round. Reverted, with the asymmetry written down so neither arm gets 'fixed' to match the other again. 上一个提交把 H200 的 float32 改动照搬到本分支，导致 SGLang 直接拒绝启动：SM100+ 上使用 flashinfer 线性注意力解码后端时必须搭配 bfloat16。两种架构的要求正好相反： SM100+ 在 server_args 中校验并要求 bfloat16；而 Hopper 上同一后端的 MTP 验证内核 在 CUDA graph 捕获阶段要求 float32。因此 cookbook 的 bfloat16 在此处是正确的， H200 才是例外。现已回退，并把这一不对称写入注释，避免两侧再被互相“对齐”。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Separate the appended changelog entry from history with a blank line Same fix as PR #2753. validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line, and merge_with_reuse.sh could not resolve the perf-changelog conflict without it: ERROR: new changelog entries must be separated from history by one empty line and appended at the end Historical bytes were already exact; this only inserts the separator. 与 PR #2753 相同的修复。validate_perf_changelog.py 要求追加部分以 "\n- config-keys:" 开头（除非基础文件本身以空行结尾），缺少该空行时 merge_with_reuse.sh 无法解决 perf-changelog 冲突。历史字节本就完全一致，此处仅插入 分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same fix as PRs #2753 and #2752. validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line, and merge_with_reuse.sh refuses the merge without it. Historical bytes were already exact; this only inserts the separator. 与 PR #2753、#2752 相同的修复：追加条目必须与历史之间空一行，否则 merge_with_reuse.sh 会拒绝合并。历史字节本就完全一致，此处仅插入该分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ext FP8 SGLang AgentX on H100 / H100 上 Qwen3.8-Flash-Next FP8 SGLang AgentX 配方 (#2756) * qwen3.8next-fp8-h100-sglang-agentic-mtp: day-zero Qwen3.8-Flash-Next AgentX on H100 Add the Qwen3.8-Flash-Next AgentX recipe on H100, served by SGLang with native NEXTN MTP. H100 is Hopper, so FP8: NVFP4 needs SM100 tensor cores. The SGLang cookbook does not list H100, so this is the H200 arm adjusted for the smaller part rather than a verified command. TP8 with EP8 instead of the cookbook's TP4/EP4, because 172.8 GiB at TP4 is ~43 GiB per rank of an 80 GB card and leaves too little for the 256k-capped agentic traces; TP8 halves that. Memory fraction 0.75 rather than 0.85, matching the Qwen3.5 H100 sibling. The SSM state is float32, as Hopper's flashinfer verify kernel requires and unlike the bfloat16 the Blackwell arms must use. No launcher change: the H100 model-path gate is inside the multinode branch, so the single-node path leaves MODEL_PATH unset and the bench script downloads into the mounted HuggingFace cache. 新增 H100 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 以原生 NEXTN MTP 提供 服务。H100 属 Hopper 架构，故使用 FP8（NVFP4 需要 SM100 张量核心）。 SGLang cookbook 未列出 H100，因此本配方是按较小硬件调整后的 H200 分支，而非官方 验证命令：采用 TP8/EP8 而非 cookbook 的 TP4/EP4——172.8 GiB 在 TP4 下每卡约 43 GiB，对 80 GB 显存而言留给 256k 智能体轨迹的空间过少，TP8 可将其减半；显存 占用取 0.75 而非 0.85，与 Qwen3.5 H100 同类配方一致。SSM 状态为 float32，这是 Hopper 上 flashinfer 验证内核的要求，与 Blackwell 分支必须使用的 bfloat16 相反。 无需改动 launcher：H100 的权重路径分支位于多节点条件内，单节点路径下 MODEL_PATH 保持未设置，基准脚本会下载到已挂载的 HuggingFace 缓存。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fill perf-changelog pr-link for #2756 / 补全 #2756 的 perf-changelog pr-link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Separate the appended changelog entry from history with a blank line Same fix as PRs #2753 and #2752. validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line, and merge_with_reuse.sh refuses the merge without it. Historical bytes were already exact; this only inserts the separator. 与 PR #2753、#2752 相同的修复：追加条目必须与历史之间空一行，否则 merge_with_reuse.sh 会拒绝合并。历史字节本就完全一致，此处仅插入该分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 participant
Summary / 摘要
Qwen3.8-Flash-Next AgentX recipe on H200, served by SGLang with native NEXTN MTP. SGLang is this model's plan-of-record engine per
MODELS.md.H200 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 提供服务，采用原生 NEXTN MTP。按
MODELS.md，SGLang 是该模型的 PoR 引擎。Config key / 配置项
qwen3.8next-fp8-h200-sglang-agentic-mtp—Qwen/Qwen3.8-Flash-Next-FP8,lmsysorg/sglang:qwen38flashnext,cluster:h200-dgxc, TP8/EP1, conc[1, 4, 8, 12, 16], no KV offloading.FP8, not NVFP4 / 使用 FP8 而非 NVFP4
This is the one deviation from "NVFP4 for NVIDIA". H200 is Hopper (sm_90) and has no SM100 tensor cores, so it cannot run the NVFP4 checkpoint — SGLang's
modelopt_fp4path is Blackwell-only. This arm therefore servesQwen/Qwen3.8-Flash-Next-FP8(172.8 GiB), exactly as the existingqwen3.5-fp8-h200-sglang-agentic-mtpsibling serves the Qwen3.5 FP8 checkpoint. The attention backend staysflashinfer;trtllm_mhais Blackwell-only too. Confirmed by the requester: run H200 at FP8, since there is no NVFP4 path on Hopper.这是与「NVIDIA 一律用 NVFP4」的唯一偏离。H200 属 Hopper（sm_90），无 SM100 张量核心，无法运行 NVFP4 权重，SGLang 的
modelopt_fp4路径仅支持 Blackwell。因此本分支使用Qwen/Qwen3.8-Flash-Next-FP8（172.8 GiB），与现有qwen3.5-fp8-h200-sglang-agentic-mtp的做法一致；注意力后端保持flashinfer（trtllm_mha同样仅限 Blackwell）。已确认：H200 以 FP8 运行，因为 Hopper 上没有 NVFP4 路径。Recipe decisions / 配方要点
TP8/EP1. 172.8 GiB of weights is ~22 GiB per rank on H200's 141 GB HBM3e, leaving ample room for the 256k-capped agentic traces. TP8 matches the Qwen3.5 H200 sibling. EP1 keeps the day-zero arm off the DeepEP/a2a path.
Image.
lmsysorg/sglang:qwen38flashnext, the model bring-up tag published 2026-08-26 (verified on Docker Hub).Serve flags are carried from
qwen3.5_fp8_h200_mtp.sh:--quantization fp8,fp8_e4m3KV cache, bf16 Mamba SSM,flashinferattention with allreduce fusion,SGLANG_ENABLE_SPEC_V2=1, EAGLE/NEXTN with 3 steps / eagle-topk 1 / 4 draft tokens. The template's hardcoded--served-model-name "Qwen/Qwen3.5-397B-A17B-FP8"line is dropped so the served name follows$MODEL.No launcher change needed:
runners/launch_h200-dgxc-slurm.shalready resolvesagentic/<prefix>_<precision>_h200_<framework>_mtp.sh.TP8/EP1：H200 141 GB HBM3e 下每卡权重约 22 GiB，为 256k 上限的智能体轨迹留出充足显存；TP8 与 Qwen3.5 H200 同类配方一致，EP1 让首发分支避开 DeepEP/a2a 路径。
镜像：
lmsysorg/sglang:qwen38flashnext（2026-08-26 发布的模型适配标签，已核实）。服务参数沿用
qwen3.5_fp8_h200_mtp.sh；模板中硬编码的--served-model-name "Qwen/Qwen3.5-397B-A17B-FP8"已删除，改由$MODEL决定。无需改动 launcher。
Acceptance length / 接受长度
SGLANG_SIMULATE_ACC_LEN=3.24. Three speculative tokens per verification step is the MTP=3 cell, measured thinking=off in speedbench-al run 33031708148 (MTP 1–6 → AL 1.87 / 2.61 / 3.24 / 3.74 / 4.18 / 4.40). Interim value — thethinking=oncollection is still running andgolden_al_distribution/qwen3.8next_mtp.yamlis not committed yet.SGLANG_SIMULATE_ACC_LEN=3.24，取自 speedbench-al 运行 33031708148 的 thinking=off 实测 MTP=3 值；为临时值，待黄金曲线提交后刷新。Files / 改动文件
benchmarks/single_node/agentic/qwen3.8next_fp8_h200_sglang_mtp.sh(new)configs/nvidia-master.yaml— new entry after the Qwen3.5 FP8 H200 siblingperf-changelog.yaml— appended entryPart of a four-PR set, one per chip: B200 (#2751), B300 (#2752), H200 (this), MI355X.
本 PR 属于按芯片划分的四个 PR 之一：B200（#2751）、B300（#2752）、H200（本 PR）、MI355X。
🤖 Generated with Claude Code
Note
Low Risk
Adds benchmark configuration and a new shell launcher only; no changes to production serving, auth, or data paths.
Overview
Adds a day-zero AgentX benchmark arm for Qwen/Qwen3.8-Flash-Next-FP8 on H200 using SGLang with NEXTN MTP speculative decoding, registered as
qwen3.8next-fp8-h200-sglang-agentic-mtp.The new launcher script follows the existing H100/H200 AgentX replay pattern (256k-capped traces, optional HiCache,
aiperfreplay, multi-tokenizercached_tokens_detailspatch) but targets the Qwen3.8 cookbook setup: TP4 + EP4,mem-fraction-static0.85, flashinfer linear-attention backends, and--mamba-ssm-dtype float32instead of bfloat16 so NEXTN verification can capture CUDA graphs with this flashinfer build. Throughput runs pin simulated acceptance to golden thinking_on AL 2.32 at three speculative tokens;EVAL_ONLYdisables simulation for real eval scoring.configs/nvidia-master.yamlexposes the recipe (lmsysorg/sglang:qwen38flashnext, agentic-coding, conc 1/4/8/12/16, no KV offload in the search space).perf-changelog.yamldocuments the addition and serve-flag rationale (FP8 on Hopper vs NVFP4).Reviewed by Cursor Bugbot for commit 2eccb1a. Bugbot is set up for automated code reviews on this repo. Configure here.