[Klaud Cold] minimaxm3-fp4-b200-vllm-agentic-mtp: MiniMax-M3 NVFP4 B200 AgentX with EAGLE3-GQA MTP and golden-AL synthetic acceptance / 新增 MiniMax-M3 NVFP4 B200 vLLM EAGLE3-GQA MTP AgentX 配方(黄金 AL 合成接受率) - #2511
Conversation
1bde68c to
cb83053
Compare
| - "Second B200 delta: draft staging. The B300 sibling stages the EAGLE3-GQA head under /data/models, which does not exist on b200-dgxc; the launcher bind-mounts only $MODEL_PATH itself, so the draft is staged in the container-local parent directory instead of polluting the shared checkpoint directory. Same approach the deprecated 8k1k B200 MiniMax-M3 MTP recipe used." | ||
| - "Search space mirrors the B300 sibling's TP2 / TP4 / TP4+SimpleCPU arms and truncates each where B200's smaller HBM pool puts the cliff: TP4 GPU-resident conc [1, 2, 5, 8, 10, 12, 15, 20], TP2 GPU-resident conc [1, 2], TP4 SimpleCPU conc [20, 30, 40, 50, 60, 65, 70, 75]. The ~220 GB NVFP4 checkpoint leaves roughly 104 GB of KV at TP2 and 428 GB at TP4 on B200's 180 GB HBM3e, against 298 GB and 817 GB for the same topologies on B300, so TP2 keeps only its low-concurrency corner and the host tier picks up one step earlier at TP4. Same B200/B300 asymmetry the qwen3.5 NVFP4 AgentX MTP pair already publishes." | ||
| - "dram-utilization 0.683 is carried over unchanged from the B300 entry and resolves to the same 1,024 GB engine-level CPU KV budget at TP4: b200-dgxc and b300-nv both report more installed DRAM than MAX_AGENTIC_AVAILABLE_CPU_DRAM_MIB, so both clamp to the same 3 TB AgentX ceiling before the proportional-GPU rule applies." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 The new perf-changelog.yaml entry uses pr-link: TBD, but the merge-prep automation only accepts XXX (or the literal expected PR URL) as a placeholder. Change TBD to XXX so utils/merge_with_reuse.sh's canonicalization step can auto-fill the real PR link at merge time instead of hard-failing.
Extended reasoning...
perf-changelog.yaml line 5483 sets pr-link: TBD for the newly appended minimaxm3-fp4-b200-vllm-agentic-mtp entry. utils/validate_perf_changelog.py defines the accepted placeholders as:
PR_LINK_PLACEHOLDERS = {
"XXX",
"https://github.com/SemiAnalysisAI/InferenceX/pull/XXX",
}TBD is not a member of this set, and validate_added_pr_link() (called from compare_entries()) raises ChangelogValidationError for any appended pr-link that is neither a placeholder nor the exact expected .../pull/<pr_number> URL.
Why PR-time CI doesn't catch it: the CLI entry point main() -> validate_matrix_compatible_change() only runs validate_generated_config() (which shells out to process_changelog.py, and that script never inspects pr-link at all) plus a trailing-newline check. It never calls compare_entries(), so the PR description's claim that validate_perf_changelog.py --base-ref origin/main passes is accurate — but it's validating a different thing than what the merge path validates.
Where it actually breaks: utils/merge_with_reuse.sh unconditionally invokes utils/prepare_perf_changelog_merge.py's canonicalize step during the standard reuse-merge flow (the path used by the /merge-prs and /find-mergeable-claude-prs skills). That script's canonicalize_appended_links() calls compare_entries(base, head, pr_number) with a real PR number, which routes into validate_added_pr_link() — and since TBD is neither a placeholder nor the canonical URL, it raises ChangelogValidationError("new PR entry must use ... or an XXX placeholder; found TBD"). Even if that check were bypassed, canonicalize_appended_links()'s own loop separately raises ChangelogValidationError(f"appended entry {index+1} has unexpected pr-link {link!r}") for the same reason — the automation is specifically built to recognize XXX and rewrite it to the real merged PR URL, and TBD doesn't match that contract.
Concrete walkthrough:
- PR [Klaud Cold] minimaxm3-fp4-b200-vllm-agentic-mtp: MiniMax-M3 NVFP4 B200 AgentX with EAGLE3-GQA MTP and golden-AL synthetic acceptance / 新增 MiniMax-M3 NVFP4 B200 vLLM EAGLE3-GQA MTP AgentX 配方(黄金 AL 合成接受率) #2511 is opened with
pr-link: TBDon the new changelog entry. - CI runs
validate_perf_changelog.py --base-ref origin/main→ passes, since that path never touchespr-link. - PR gets approved and someone runs the standard merge automation (
merge_with_reuse.sh <pr_number>). - The script reaches its canonicalization step and calls
prepare_perf_changelog_merge.py'scanonicalize_appended_links(). compare_entries()detects the new entry'spr-link: TBD, callsvalidate_added_pr_link("TBD", 2511), and since"TBD" not in PR_LINK_PLACEHOLDERSand it doesn't matchhttps://github.com/SemiAnalysisAI/InferenceX/pull/2511, it raises.- The merge hard-fails under
set -euo pipefail, requiring a manual edit topr-linkbefore merge can proceed.
Fix: change pr-link: TBD to pr-link: XXX (matching AGENTS.md's documented convention of pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX for new entries), so the merge automation can auto-canonicalize it to the real PR URL without manual intervention.
This doesn't affect the benchmark recipe's correctness at all — it's purely a changelog metadata issue — but it will concretely block the standard automated merge path for this PR until fixed, which is why I'm not marking it as a nit despite the trivial one-token fix.
| mkdir -p "$MODEL_PATH" | ||
| MODEL_DOWNLOAD_LOCK="${MODEL_PATH%/}.download.lock" | ||
| echo "Checkpoint at $MODEL_PATH is incomplete; acquiring $MODEL_DOWNLOAD_LOCK" | ||
| exec 9>"$MODEL_DOWNLOAD_LOCK" | ||
| flock -w "${MODEL_DOWNLOAD_LOCK_TIMEOUT:-21600}" 9 || { | ||
| echo "Error: timed out waiting for another cell to stage $MODEL_PATH" >&2 | ||
| exit 1 | ||
| } | ||
| if checkpoint_is_complete "$MODEL_PATH"; then | ||
| echo "Another cell staged $MODEL_PATH while we waited" | ||
| else | ||
| hf download "$HF_MODEL_ID" --local-dir "$MODEL_PATH" | ||
| fi | ||
| flock -u 9 | ||
| exec 9>&- | ||
| checkpoint_is_complete "$MODEL_PATH" || { | ||
| echo "Error: $MODEL_PATH is still incomplete after hf download $HF_MODEL_ID." >&2 | ||
| exit 1 | ||
| } |
There was a problem hiding this comment.
🔴 The checkpoint-download flock's lock file is placed at ${MODEL_PATH%/}.download.lock, a sibling of $MODEL_PATH in its parent directory — but launch_b200-dgxc.sh (line 509) bind-mounts only $MODEL_PATH itself into each cell's container, so that parent (and the lock file inside it) is private per-container overlay, not shared storage. On a cold start where the checkpoint isn't pre-staged, every concurrent cell in the sweep gets its own uncontested lock and simultaneously runs hf download against the same shared $MODEL_PATH — exactly the multi-writer race the flock was added to prevent. Fix by placing the lock file inside $MODEL_PATH (e.g. $MODEL_PATH/.download.lock), which is actually bind-mounted and shared across cells.
Extended reasoning...
The bug: In minimaxm3_fp4_b200_mtp.sh (lines 83-101), when the checkpoint at $MODEL_PATH is incomplete, the script does:
MODEL_DOWNLOAD_LOCK="${MODEL_PATH%/}.download.lock"
exec 9>"$MODEL_DOWNLOAD_LOCK"
flock -w "${MODEL_DOWNLOAD_LOCK_TIMEOUT:-21600}" 9 || { ... }For the minimaxm3/fp4 case, MODEL_PATH=/scratch/fsw/models/MiniMax-M3-NVFP4 (launch_b200-dgxc.sh:73), so MODEL_DOWNLOAD_LOCK resolves to /scratch/fsw/models/MiniMax-M3-NVFP4.download.lock — a sibling file in the parent directory /scratch/fsw/models/, not a path inside $MODEL_PATH itself.
Why the lock never crosses containers: launch_b200-dgxc.sh:509 bind-mounts only:
--container-mounts=$GITHUB_WORKSPACE:$CONTAINER_MOUNT_DIR,$MODEL_PATH:$MODEL_PATH,$AIPERF_MMAP_CACHE_HOST_PATH:/aiperf_mmap_cache
The parent directory /scratch/fsw/models/ is never bind-mounted from the host. Every cell in the sweep runs as its own salloc --exclusive/srun allocation with its own container (lines 484-513), so each container's view of /scratch/fsw/models/ above the $MODEL_PATH mount point is its own private overlay filesystem. The PR's own comment on the draft-staging block two paragraphs below confirms this exact model of the parent directory: "the launcher bind-mounts only $MODEL_PATH itself ... so its parent exists solely inside the container overlay — writable, but per-job." That statement applies equally to the lock file, which lives at that same per-job parent path.
Concrete walk-through: Suppose the checkpoint is not pre-staged (a fresh cluster, or a purged cache) and the sweep launches the 8-point TP4 GPU-resident arm (conc = 1, 2, 5, 8, 10, 12, 15, 20) concurrently, each as its own srun/container:
- Cell A (conc=1) checks
checkpoint_is_complete "$MODEL_PATH"→ false. It runsexec 9>/scratch/fsw/models/MiniMax-M3-NVFP4.download.lockinside its own container overlay, creating a private lock file no other container can see. - Cell B (conc=2), running concurrently in a separate container, does the exact same thing at the same path string — but that path resolves to its own private overlay copy, not Cell A's.
flockacquires instantly since there is no real contention. - Both (and cells C through H) proceed to run
hf download nvidia/MiniMax-M3-NVFP4 --local-dir "$MODEL_PATH"at the same time, all writing into the one real, bind-mounted, shared$MODEL_PATHdirectory on/scratch/fsw. - Result: up to 8 concurrent ~220 GB downloads writing shards into the same shared directory — wasted bandwidth, and a real risk that one cell's
checkpoint_is_completecheck sees "all shards present" while a sibling cell is still mid-write on some of those same shard files, loading a truncated/corrupt checkpoint.
This is precisely the multi-writer race the flock block's own comment says it exists to prevent ("serialize: one cell pulls the ~220 GB checkpoint and the rest wait on it rather than racing as writers").
Why nothing else catches this: checkpoint_is_complete only checks for shard presence, not size/hash, so a checkpoint mid-write by another cell can appear complete once all shard filenames exist, even if content is still being written. hf download's internal locking (if any) is scoped to a single process/host and does not span separate containers on separate nodes.
Fix: Put the lock file inside the shared mount, e.g. MODEL_DOWNLOAD_LOCK="$MODEL_PATH/.download.lock" (or ${MODEL_PATH%/}/.download.lock), since $MODEL_PATH itself is the one path actually bind-mounted and shared across all cells. This is a one-line fix and doesn't require any other structural change to the block.
Scope note: In steady state, the checkpoint is normally pre-staged on /scratch/fsw/models (per the launcher's own documented pre-provisioned paths), so checkpoint_is_complete short-circuits and this download+flock block never executes. The bug only bites the cold-start/day-zero staging path — but it fully defeats the safety mechanism the PR explicitly added for exactly that scenario, and the failure mode when triggered is concurrent corruption-prone writers on a real shared 220 GB checkpoint directory.
| hf download "$HF_MODEL_ID" | ||
| export MODEL_PATH="$HF_MODEL_ID" | ||
| fi | ||
|
|
||
| # B200: the B300 sibling stages the draft under /data/models, which does not | ||
| # exist on b200-dgxc. The launcher bind-mounts only $MODEL_PATH itself | ||
| # (--container-mounts=...,$MODEL_PATH:$MODEL_PATH,...), so its parent exists | ||
| # solely inside the container overlay -- writable, but per-job. Stage the draft | ||
| # there rather than inside $MODEL_PATH, which is the shared read-mostly | ||
| # checkpoint directory and must not be polluted. The EAGLE3-GQA head is small | ||
| # next to the target, so the per-job pull is cheap; this is what the | ||
| # now-deprecated 8k1k B200 MiniMax-M3 MTP recipe did as well. | ||
| DRAFT_MODEL_PATH="${DRAFT_MODEL_PATH:-$(dirname "${MODEL_PATH%/}")/${DRAFT_MODEL##*/}}" |
There was a problem hiding this comment.
🟡 In the else-branch where MODEL_PATH is unset (manual runs without a pre-set path), MODEL_PATH is set to the HF repo id nvidia/MiniMax-M3-NVFP4, but DRAFT_MODEL_PATH is still derived via dirname "${MODEL_PATH%/}", which resolves to the bogus relative path nvidia/MiniMax-M3-EAGLE3-GQA instead of a real staging directory. This branch is dead in the actual b200-dgxc sweep since the launcher always exports a real filesystem MODEL_PATH, so it only affects manual/bring-up runs; fix by mirroring the B300 sibling'''s else-branch, which sets DRAFT_MODEL_PATH=$DRAFT_MODEL directly.
Extended reasoning...
The bug: benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.sh has an else-branch (lines ~104-105) that runs when MODEL_PATH is not pre-set:
else
hf download "$HF_MODEL_ID"
export MODEL_PATH="$HF_MODEL_ID"
fiHF_MODEL_ID is a Hugging Face repo id string (nvidia/MiniMax-M3-NVFP4), not a filesystem path. Further down (line 116), DRAFT_MODEL_PATH is computed unconditionally from MODEL_PATH regardless of which branch set it:
DRAFT_MODEL_PATH="${DRAFT_MODEL_PATH:-$(dirname "${MODEL_PATH%/}")/${DRAFT_MODEL##*/}}"Step-by-step proof:
MODEL_PATH="nvidia/MiniMax-M3-NVFP4"(set by the else-branch).${MODEL_PATH%/}strips a trailing slash (none present) → unchanged.dirname "nvidia/MiniMax-M3-NVFP4"→"nvidia".${DRAFT_MODEL##*/}whereDRAFT_MODEL="Inferact/MiniMax-M3-EAGLE3-GQA"→"MiniMax-M3-EAGLE3-GQA".- Result:
DRAFT_MODEL_PATH="nvidia/MiniMax-M3-EAGLE3-GQA"— a nonsensical relative path built from the target model's org (nvidia) rather than any real staging directory.
Why this formula exists at all: it's correct in the if [[ -n "${MODEL_PATH:-}" ]] branch, where MODEL_PATH really is a filesystem path (e.g. /scratch/fsw/models/MiniMax-M3-NVFP4), so dirname yields a legitimate parent directory to stage the draft alongside the target. The bug is that this same formula is reused unconditionally after the if/else, even though the else-branch's MODEL_PATH is a repo id, not a path.
Why the B300 sibling doesn't have this issue: minimaxm3_fp4_b300_mtp.sh's else-branch explicitly sets DRAFT_MODEL_PATH="$DRAFT_MODEL" (the draft's own repo id) directly, never deriving it from MODEL_PATH. The B200 port collapsed this into a single shared line after the if/else, which broke that independence.
Impact: This is dead code in the actual b200-dgxc sweep — runners/launch_b200-dgxc.sh always rewrites MODEL to a real cluster filesystem path before invoking this script, so MODEL_PATH is always set going in, and the else-branch (and this bug) never executes during a real sweep run. It only bites someone running the script manually/for local bring-up without pre-setting MODEL_PATH. Even then it doesn't crash: hf download "$DRAFT_MODEL" --local-dir "$DRAFT_MODEL_PATH" will happily create ./nvidia/MiniMax-M3-EAGLE3-GQA relative to the current working directory and the subsequent vLLM invocation would likely still resolve it as a local path. The practical effect is a confusing, incorrectly-named ./nvidia/ directory left in the CWD rather than a real staging location — messy but not a functional break.
Fix: In the else-branch, set DRAFT_MODEL_PATH="$DRAFT_MODEL" directly (mirroring the B300 sibling), or otherwise make the dirname-based derivation conditional on the if-branch only.
cb83053 to
873b59e
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31141622151 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31141913741 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31141913741 |
…GLE3-GQA MTP recipe Port of the validated B300 sibling minimaxm3-fp4-b300-vllm-agentic-mtp to b200-dgxc. Same image, same Inferact/MiniMax-M3-EAGLE3-GQA draft head at three speculative tokens, same golden AL 2.78, same TP2/TP4 topologies and SimpleCPU offload arm, so the two SKU curves stay directly comparable. Spec-decode only, per the AgentX policy in MODELS.md. This is MiniMax-M3's only active B200 configuration: the Single-turn 8k1k scenario that carried the previous B200 curves was deprecated in #2493. Two B200 deltas: the checkpoint-resolution guard (launch_b200-dgxc.sh rewrites MODEL to a cluster-local path, so the HF repo id is kept in HF_MODEL_ID and the checkpoint is verified against its shard index behind a flock) and draft staging (the B300 /data/models path does not exist on b200-dgxc). Concurrency per arm is truncated where B200's 180 GB HBM3e puts the cliff. 中文:新增 MiniMax-M3 NVFP4 在 B200 上的 AgentX(agentic-coding)配方,使用 vLLM 搭配 Inferact/MiniMax-M3-EAGLE3-GQA 草稿头做投机解码(投机 token 数 3,黄金接受 长度 2.78)。镜像、草稿模型、TP2/TP4 拓扑与 SimpleCPU 卸载分支均与 B300 同源配方 一致,保证两个 SKU 的曲线可直接对比。按 MODELS.md 的 AgentX 策略仅发布投机解码 分支;#2493 停用 8k1k 场景后,这是 MiniMax-M3 在 B200 上唯一的有效配置。B200 的 两处差异为检查点解析保护(启动器会把 MODEL 改写为集群本地路径)与草稿模型的暂存 路径;各分支并发上限按 B200 的 180 GB 显存下调。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
中文:为 #2511 填写 perf-changelog 的 pr-link。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The recipe's memory rationale cited a ~220 GB checkpoint, derived by halving the documented ~444 GB MXFP8 figure. nvidia/MiniMax-M3-NVFP4 is actually 250.1 GB across 88 shards, so the derived KV budgets were ~30 GB optimistic on every arm: TP2 has ~74 GB rather than ~104 GB and TP4 ~398 GB rather than ~428 GB on B200, against ~268 GB and ~787 GB on B300. No search-space change -- the corrected numbers only reinforce the existing truncations (TP2 capped at conc 2, host tier engaged at conc 20, hard stop at 40). 中文:修正配方中 NVFP4 检查点大小的说明。此前按 MXFP8 的 ~444 GB 折半估算为 ~220 GB,实际 nvidia/MiniMax-M3-NVFP4 为 250.1 GB(88 个分片),因此各分支推导 出的 KV 预算均偏乐观约 30 GB。搜索空间不变——修正后的数值只会进一步支持现有的 并发截断。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…1M context Both TP2 cells failed identically at engine init on run 31141913741 while every TP4 cell passed: ValueError: To serve at least one request with the model's max seq len (1048576), 37.62 GiB KV cache is needed, which is larger than the available KV cache memory (31.72 GiB). With the ~250 GB NVFP4 checkpoint resident, two B200s leave a 31.72 GiB KV pool at --gpu-memory-utilization 0.9, and vLLM estimates the achievable max model length at 883,840 -- below the agentic corpus context. This is structural, not a concurrency cliff, so no conc-list would have made the arm run and lowering the grid would only burn engine starts. The B300 sibling keeps its TP2 arm; on 288 GB GPUs the same topology has ~268 GB of KV. Recipe is now TP4-only: 8 GPU-resident points plus 3 SimpleCPU points. 中文:移除 TP2 分支。在 run 31141913741 中两个 TP2 单元均在引擎初始化阶段失败,而 所有 TP4 单元均通过:约 250 GB 的 NVFP4 权重驻留后,两张 B200 在 --gpu-memory-utilization 0.9 下仅剩 31.72 GiB 的 KV 池,而按模型 1,048,576 的 max_model_len 计算,单个请求就需要 37.62 GiB。vLLM 估算 TP2 可支持的最大上下文为 883,840,低于智能体语料的上下文长度,因此这是结构性限制而非并发瓶颈,调整 conc-list 无济于事。B300 同源配方保留 TP2 分支(288 GB 显存下该拓扑约有 268 GB KV)。本配方现为纯 TP4:8 个显存驻留点加 3 个 SimpleCPU 点。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2221c9d to
3e3db68
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31210279071 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=31210279071 |
|
/stage-results 31210279071 |
|
@functionstackx staged run 31210279071: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-07~r31210279071 This run remains available across future @functionstackx 已将运行 31210279071 发布到预发布环境:https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-07~r31210279071 后续的 |
|
/reuse-sweep-run |
3e3db68 to
f70cedc
Compare
# Conflicts: # configs/nvidia-master.yaml # perf-changelog.yaml
f70cedc to
41316a7
Compare
|
/reuse-sweep-run |
Single-node agentic-coding recipe for MiniMax-M3 (NVFP4, MoE) on B200 with vLLM and EAGLE3-GQA speculative decoding — the B200 sibling of the merged B300 recipe from #2328.
This is MiniMax-M3's only active B200 configuration. The Single-turn 8k1k scenario that carried the previous B200 M3 curves (
minimaxm3-fp4-b200-vllm,minimaxm3-fp4-b200-vllm-mtp) was deprecated and archived underconfigs/deprecated/in #2493, leaving B200 with no M3 coverage at all. Shipped spec-decode only, per the AgentX policy inMODELS.md: agentic recipes are run and published with speculative decoding enabled rather than as an STP/MTP A/B, because synthetic acceptance already makes spec-decode results comparable across submissions.Contents
benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.shconfigs/nvidia-master.yaml→minimaxm3-fp4-b200-vllm-agentic-mtp(11 configs, TP4-only)perf-changelog.yamlentryCarried over from B300 unchanged
Everything that determines the shape of the curve is the B300 script verbatim, so the two SKUs are directly comparable:
vllm/vllm-openai:nightly-5e35a6f4f9bbc217c599692157ca985c894373f7Inferact/MiniMax-M3-EAGLE3-GQA, 3 speculative tokens, drafter onFLASH_ATTNgolden_al_distribution/minimaxm3_eagle3_gqa.yaml,minimax-m3.thinking_on[3]use_trtllm_attention=true), FP8 indexer KV,--kv-cache-dtype fp8--block-size 128(mandatory, MSA sparse/index cache),--language-model-only,--enable-prefix-caching,--max-cudagraph-capture-size 512,--max-num-batched-tokens 16384,--stream-interval 20,minimax_m3reasoning + tool-call parsers,thinking_modeenabledSimpleCPUOffloadConnector, lazy modedram-utilizationThe image tag is the one the B300 AgentX MTP sibling runs and the same tag the deprecated
minimaxm3-fp4-b200-vllm8k1k entry was bumped to in #2468, so it is already proven on this model, precision and SKU.Throughput runs pin vLLM synthetic rejection sampling to the golden AL (
rejection_sample_method=synthetic,synthetic_acceptance_length=2.78).EVAL_ONLYruns drop synthetic acceptance and keep real target verification — synthetic acceptance bypasses verification and zeroes the eval score.B200 deltas
Only two blocks differ from the B300 script, both marked
B200:in the source.1. Checkpoint-resolution guard.
runners/launch_b200-dgxc.shresolves the checkpoint to/scratch/fsw/models/MiniMax-M3-NVFP4and then rewritesMODELto that path before invoking the bench script, sohf download "$MODEL"cannot work the way it does onb300-nv, whereMODELstays the HF repo id. The script keeps the repo id inHF_MODEL_ID, verifies the checkpoint by walking every shard named inmodel.safetensors.index.jsonrather than trusting anls -Aemptiness test, and serializes any download behind aflockso the concurrencies of one sweep do not race as writers on the shared path.2. Draft staging. The B300 sibling stages the EAGLE3-GQA head under
/data/models, which does not exist onb200-dgxc. The launcher bind-mounts only$MODEL_PATHitself, so the draft is staged in the container-local parent directory rather than polluting the shared checkpoint directory — the same approach the now-deprecated 8k1k B200 M3 MTP recipe used.--gpu-memory-utilizationstays at 0.9 (the value both the B300 sibling and the deprecated 8k1k B200 NVFP4 MTP recipe ran) and is now overridable viaGPU_MEMORY_UTILIZATIONfor bring-up debugging.Search space
TP4-only. The B300 sibling's TP2 arm is dropped (see below); the two surviving arms are truncated where B200's smaller HBM pool puts the cliff:
The 250 GB NVFP4 checkpoint (88 shards) leaves roughly 398 GB of KV at TP4 on B200's 180 GB HBM3e, against ~787 GB on B300's 288 GB.
Why TP2 was dropped
Both TP2 cells failed identically at engine init on run 31141913741 while every TP4 cell passed:
With the checkpoint resident, two B200s leave a 31.72 GiB KV pool at gmu 0.9, and one request at the model's 1M
max_model_lenneeds 37.62 GiB. This is structural, not a concurrency cliff — the achievable max model length at TP2 is 883,840, below the agentic corpus context — so noconc-listwould have made the arm run. The B300 sibling keeps its TP2 arm; on 288 GB GPUs that topology has ~268 GB of KV. So TP2 keeps only its low-concurrency corner, TP4 samples 5–15 densely because the cliff lands inside that range at roughly half the KV (and retains 20 past it), and the host tier picks up one step earlier at conc 20.The whole recipe stops at concurrency 40. The B300 sibling's SimpleCPU arm runs to 75, but on B200 everything past 40 is the host tier absorbing a working set that never fit in the ~398 GB TP4 pool to begin with — that tail traces offload bandwidth rather than the SKU, and costs one engine start per point to do it. This is the same B200/B300 asymmetry the
qwen3.5NVFP4 AgentX MTP pair already publishes.dram-utilization: 0.683is the B300 value verbatim and resolves to the same 1,024 GB engine-level CPU KV budget at TP4:b200-dgxc(3,095,781 MiB) andb300-nv(2,964,436 MiB) both report more installed DRAM thanMAX_AGENTIC_AVAILABLE_CPU_DRAM_MIB(2,861,022 MiB), so both clamp to the same 3 TB AgentX ceiling before the proportional-GPU rule applies. GPU-resident points receive a zero budget.Validation
generate_sweep_configs.py full-sweep --model-prefix minimaxm3 --runner-type cluster:b200-dgxc→ 11 configs, TP4-only, max conc 40,total-cpu-dram-gb1024 on the SimpleCPU arm and 0 elsewherevalidate_perf_changelog.py --base-ref origin/main→ passespytest utils/matrix_logic/→ 224 passedlaunch_b200-dgxc.sh's logic →benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.shbash -nclean; image tag confirmed present on Docker HubLabeled
full-sweep-enabled.中文说明
MiniMax-M3(NVFP4,混合专家(MOE))在 B200 单节点上的 agentic-coding 配方,使用 vLLM 搭配 EAGLE3-GQA 投机解码,是 #2328 已合并的 B300 配方的 B200 同源版本。
这是 MiniMax-M3 在 B200 上唯一的有效配置:承载此前 B200 M3 曲线的 Single-turn 8k1k 场景已在 #2493 中停用并归档到
configs/deprecated/。按MODELS.md的 AgentX 策略仅发布投机解码分支——合成接受率已使投机解码结果在各提交之间可比,无需再跑非投机解码对照组。与 B300 保持一致的部分
镜像
vllm/vllm-openai:nightly-5e35a6f4f9bbc217c599692157ca985c894373f7、草稿模型Inferact/MiniMax-M3-EAGLE3-GQA(投机 token 数 3,草稿头固定FLASH_ATTN)、黄金接受长度 2.78(golden_al_distribution/minimaxm3_eagle3_gqa.yaml的thinking_on[3])、FlashInfer TRT-LLM 注意力搭配 FP8 indexer KV、--block-size 128、--enable-prefix-caching、SimpleCPU DRAM 卸载以及dram-utilization: 0.683全部沿用 B300 脚本,确保两个 SKU 的曲线可直接对比。该镜像标签正是 B300 同源配方所用、也是 #2468 中minimaxm3-fp4-b200-vllm升级到的标签,已在该模型、精度与 SKU 上得到验证。吞吐量运行将 vLLM 合成拒绝采样固定到黄金接受长度;
EVAL_ONLY运行关闭合成接受、保留真实目标验证(合成接受会跳过验证并使评估得分归零)。B200 的两处差异
runners/launch_b200-dgxc.sh会把检查点解析为集群本地路径并将MODEL改写为该路径,因此hf download "$MODEL"在此运行器上无法工作(B300 上MODEL保持为 HF 仓库 id)。脚本将仓库 id 保存在HF_MODEL_ID中,并通过遍历model.safetensors.index.json中列出的每个分片来校验检查点完整性,而不是依赖ls -A判空;下载过程用flock串行化,避免同一次扫描的多个并发单元同时写入共享路径。/data/models,该路径在b200-dgxc上不存在。启动器只挂载$MODEL_PATH本身,因此草稿改为暂存到容器本地的父目录,而不污染共享检查点目录——与已停用的 8k1k B200 M3 MTP 配方做法一致。--gpu-memory-utilization保持 0.9,并新增GPU_MEMORY_UTILIZATION覆盖入口便于调试。搜索空间
与 B300 相同的三个分支,各自在 B200 显存瓶颈处截断:仅保留 TP4:GPU 驻留并发 1、2、5、8、10、12、15、20;TP4 + SimpleCPU 并发 20、30、40。B300 同源配方的 TP2 分支未予移植——在 run 31141913741 中两个 TP2 单元均在引擎初始化阶段失败(约 250 GB 权重驻留后 KV 池仅 31.72 GiB,而按 1,048,576 的 max_model_len 单个请求需 37.62 GiB;vLLM 估算 TP2 可支持上下文为 883,840,低于语料长度),属结构性限制而非并发瓶颈。整个配方的并发上限为 40:B300 同源配方的 SimpleCPU 分支跑到 75,但在 B200 上超过 40 的部分只是主机侧在承接一个本来就装不进 ~398 GB TP4 显存池的工作集,该尾部反映的是卸载带宽而非 SKU 本身,且每个点都要多启动一次引擎。250 GB 的 NVFP4 权重在 B200 的 180 GB 显存上,TP4 约剩 398 GB 用作 KV,而 B300 相同拓扑约为 787 GB;因此 TP4 在 5–15 区间加密采样,主机侧缓存提前一档在并发 20 接入。这与
qwen3.5NVFP4 AgentX MTP 配方对已有的 B200/B300 差异保持一致。dram-utilization: 0.683沿用 B300 数值,在 TP4 上解析出相同的 1,024 GB 引擎级 CPU KV 预算:b200-dgxc与b300-nv的实装内存均超过MAX_AGENTIC_AVAILABLE_CPU_DRAM_MIB,两者都会先被收敛到相同的 3 TB AgentX 上限,再套用按 GPU 比例分配的规则。验证
配置生成 11 个配置(仅 TP4)、最大并发 40,且 CPU KV 预算符合预期;
validate_perf_changelog.py通过;pytest utils/matrix_logic/224 项全部通过;启动器脚本路由解析到minimaxm3_fp4_b200_mtp.sh;bash -n无语法错误;镜像标签已在 Docker Hub 上确认存在。已添加full-sweep-enabled标签。