Skip to content

[Klaud Cold] minimaxm3-fp4-b200-vllm-agentic-mtp: MiniMax-M3 NVFP4 B200 AgentX with EAGLE3-GQA MTP and golden-AL synthetic acceptance / 新增 MiniMax-M3 NVFP4 B200 vLLM EAGLE3-GQA MTP AgentX 配方(黄金 AL 合成接受率) - #2511

Merged
functionstackx merged 6 commits into
mainfrom
feat/minimaxm3-fp4-b200-vllm-agentic-mtp
Aug 8, 2026

Conversation

@functionstackx

@functionstackx functionstackx commented Aug 6, 2026

Copy link
Copy Markdown
Collaborator

Single-node agentic-coding recipe for MiniMax-M3 (NVFP4, MoE) on B200 with vLLM and EAGLE3-GQA speculative decoding — the B200 sibling of the merged B300 recipe from #2328.

This is MiniMax-M3's only active B200 configuration. The Single-turn 8k1k scenario that carried the previous B200 M3 curves (minimaxm3-fp4-b200-vllm, minimaxm3-fp4-b200-vllm-mtp) was deprecated and archived under configs/deprecated/ in #2493, leaving B200 with no M3 coverage at all. Shipped spec-decode only, per the AgentX policy in MODELS.md: agentic recipes are run and published with speculative decoding enabled rather than as an STP/MTP A/B, because synthetic acceptance already makes spec-decode results comparable across submissions.

Contents

  • benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.sh
  • configs/nvidia-master.yamlminimaxm3-fp4-b200-vllm-agentic-mtp (11 configs, TP4-only)
  • perf-changelog.yaml entry

Carried over from B300 unchanged

Everything that determines the shape of the curve is the B300 script verbatim, so the two SKUs are directly comparable:

value
Image vllm/vllm-openai:nightly-5e35a6f4f9bbc217c599692157ca985c894373f7
Draft model Inferact/MiniMax-M3-EAGLE3-GQA, 3 speculative tokens, drafter on FLASH_ATTN
Golden AL 2.78 — golden_al_distribution/minimaxm3_eagle3_gqa.yaml, minimax-m3.thinking_on[3]
Attention FlashInfer TRT-LLM (use_trtllm_attention=true), FP8 indexer KV, --kv-cache-dtype fp8
Other serve flags --block-size 128 (mandatory, MSA sparse/index cache), --language-model-only, --enable-prefix-caching, --max-cudagraph-capture-size 512, --max-num-batched-tokens 16384, --stream-interval 20, minimax_m3 reasoning + tool-call parsers, thinking_mode enabled
DRAM offload vLLM SimpleCPUOffloadConnector, lazy mode
dram-utilization 0.683

The image tag is the one the B300 AgentX MTP sibling runs and the same tag the deprecated minimaxm3-fp4-b200-vllm 8k1k entry was bumped to in #2468, so it is already proven on this model, precision and SKU.

Throughput runs pin vLLM synthetic rejection sampling to the golden AL (rejection_sample_method=synthetic, synthetic_acceptance_length=2.78). EVAL_ONLY runs drop synthetic acceptance and keep real target verification — synthetic acceptance bypasses verification and zeroes the eval score.

B200 deltas

Only two blocks differ from the B300 script, both marked B200: in the source.

1. Checkpoint-resolution guard. runners/launch_b200-dgxc.sh resolves the checkpoint to /scratch/fsw/models/MiniMax-M3-NVFP4 and then rewrites MODEL to that path before invoking the bench script, so hf download "$MODEL" cannot work the way it does on b300-nv, where MODEL stays the HF repo id. The script keeps the repo id in HF_MODEL_ID, verifies the checkpoint by walking every shard named in model.safetensors.index.json rather than trusting an ls -A emptiness test, and serializes any download behind a flock so the concurrencies of one sweep do not race as writers on the shared path.

2. Draft staging. The B300 sibling stages the EAGLE3-GQA head under /data/models, which does not exist on b200-dgxc. The launcher bind-mounts only $MODEL_PATH itself, so the draft is staged in the container-local parent directory rather than polluting the shared checkpoint directory — the same approach the now-deprecated 8k1k B200 M3 MTP recipe used.

--gpu-memory-utilization stays at 0.9 (the value both the B300 sibling and the deprecated 8k1k B200 NVFP4 MTP recipe ran) and is now overridable via GPU_MEMORY_UTILIZATION for bring-up debugging.

Search space

TP4-only. The B300 sibling's TP2 arm is dropped (see below); the two surviving arms are truncated where B200's smaller HBM pool puts the cliff:

TP KV conc B300 sibling
4 GPU-resident 1, 2, 5, 8, 10, 12, 15, 20 1, 2, 5, 10, 15, 20
4 SimpleCPU DRAM 20, 30, 40 30, 40, 50, 60, 65, 70, 75

The 250 GB NVFP4 checkpoint (88 shards) leaves roughly 398 GB of KV at TP4 on B200's 180 GB HBM3e, against ~787 GB on B300's 288 GB.

Why TP2 was dropped

Both TP2 cells failed identically at engine init on run 31141913741 while every TP4 cell passed:

ValueError: To serve at least one request with the model's max seq len (1048576),
37.62 GiB KV cache is needed, which is larger than the available KV cache memory
(31.72 GiB). Based on the available memory, the estimated maximum model length is 883840.

With the checkpoint resident, two B200s leave a 31.72 GiB KV pool at gmu 0.9, and one request at the model's 1M max_model_len needs 37.62 GiB. This is structural, not a concurrency cliff — the achievable max model length at TP2 is 883,840, below the agentic corpus context — so no conc-list would have made the arm run. The B300 sibling keeps its TP2 arm; on 288 GB GPUs that topology has ~268 GB of KV. So TP2 keeps only its low-concurrency corner, TP4 samples 5–15 densely because the cliff lands inside that range at roughly half the KV (and retains 20 past it), and the host tier picks up one step earlier at conc 20.

The whole recipe stops at concurrency 40. The B300 sibling's SimpleCPU arm runs to 75, but on B200 everything past 40 is the host tier absorbing a working set that never fit in the ~398 GB TP4 pool to begin with — that tail traces offload bandwidth rather than the SKU, and costs one engine start per point to do it. This is the same B200/B300 asymmetry the qwen3.5 NVFP4 AgentX MTP pair already publishes.

dram-utilization: 0.683 is the B300 value verbatim and resolves to the same 1,024 GB engine-level CPU KV budget at TP4: b200-dgxc (3,095,781 MiB) and b300-nv (2,964,436 MiB) both report more installed DRAM than MAX_AGENTIC_AVAILABLE_CPU_DRAM_MIB (2,861,022 MiB), so both clamp to the same 3 TB AgentX ceiling before the proportional-GPU rule applies. GPU-resident points receive a zero budget.

Validation

  • generate_sweep_configs.py full-sweep --model-prefix minimaxm3 --runner-type cluster:b200-dgxc → 11 configs, TP4-only, max conc 40, total-cpu-dram-gb 1024 on the SimpleCPU arm and 0 elsewhere
  • validate_perf_changelog.py --base-ref origin/main → passes
  • pytest utils/matrix_logic/ → 224 passed
  • Launcher script resolution replayed against launch_b200-dgxc.sh's logic → benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.sh
  • bash -n clean; image tag confirmed present on Docker Hub

Labeled full-sweep-enabled.


中文说明

MiniMax-M3(NVFP4,混合专家(MOE))在 B200 单节点上的 agentic-coding 配方,使用 vLLM 搭配 EAGLE3-GQA 投机解码,是 #2328 已合并的 B300 配方的 B200 同源版本。

这是 MiniMax-M3 在 B200 上唯一的有效配置:承载此前 B200 M3 曲线的 Single-turn 8k1k 场景已在 #2493 中停用并归档到 configs/deprecated/。按 MODELS.md 的 AgentX 策略仅发布投机解码分支——合成接受率已使投机解码结果在各提交之间可比,无需再跑非投机解码对照组。

与 B300 保持一致的部分

镜像 vllm/vllm-openai:nightly-5e35a6f4f9bbc217c599692157ca985c894373f7、草稿模型 Inferact/MiniMax-M3-EAGLE3-GQA(投机 token 数 3,草稿头固定 FLASH_ATTN)、黄金接受长度 2.78(golden_al_distribution/minimaxm3_eagle3_gqa.yamlthinking_on[3])、FlashInfer TRT-LLM 注意力搭配 FP8 indexer KV、--block-size 128--enable-prefix-caching、SimpleCPU DRAM 卸载以及 dram-utilization: 0.683 全部沿用 B300 脚本,确保两个 SKU 的曲线可直接对比。该镜像标签正是 B300 同源配方所用、也是 #2468minimaxm3-fp4-b200-vllm 升级到的标签,已在该模型、精度与 SKU 上得到验证。

吞吐量运行将 vLLM 合成拒绝采样固定到黄金接受长度;EVAL_ONLY 运行关闭合成接受、保留真实目标验证(合成接受会跳过验证并使评估得分归零)。

B200 的两处差异

  1. 检查点解析保护runners/launch_b200-dgxc.sh 会把检查点解析为集群本地路径并将 MODEL 改写为该路径,因此 hf download "$MODEL" 在此运行器上无法工作(B300 上 MODEL 保持为 HF 仓库 id)。脚本将仓库 id 保存在 HF_MODEL_ID 中,并通过遍历 model.safetensors.index.json 中列出的每个分片来校验检查点完整性,而不是依赖 ls -A 判空;下载过程用 flock 串行化,避免同一次扫描的多个并发单元同时写入共享路径。
  2. 草稿模型暂存路径:B300 同源配方将草稿头放在 /data/models,该路径在 b200-dgxc 上不存在。启动器只挂载 $MODEL_PATH 本身,因此草稿改为暂存到容器本地的父目录,而不污染共享检查点目录——与已停用的 8k1k B200 M3 MTP 配方做法一致。

--gpu-memory-utilization 保持 0.9,并新增 GPU_MEMORY_UTILIZATION 覆盖入口便于调试。

搜索空间

与 B300 相同的三个分支,各自在 B200 显存瓶颈处截断:仅保留 TP4:GPU 驻留并发 1、2、5、8、10、12、15、20;TP4 + SimpleCPU 并发 20、30、40。B300 同源配方的 TP2 分支未予移植——在 run 31141913741 中两个 TP2 单元均在引擎初始化阶段失败(约 250 GB 权重驻留后 KV 池仅 31.72 GiB,而按 1,048,576 的 max_model_len 单个请求需 37.62 GiB;vLLM 估算 TP2 可支持上下文为 883,840,低于语料长度),属结构性限制而非并发瓶颈。整个配方的并发上限为 40:B300 同源配方的 SimpleCPU 分支跑到 75,但在 B200 上超过 40 的部分只是主机侧在承接一个本来就装不进 ~398 GB TP4 显存池的工作集,该尾部反映的是卸载带宽而非 SKU 本身,且每个点都要多启动一次引擎。250 GB 的 NVFP4 权重在 B200 的 180 GB 显存上,TP4 约剩 398 GB 用作 KV,而 B300 相同拓扑约为 787 GB;因此 TP4 在 5–15 区间加密采样,主机侧缓存提前一档在并发 20 接入。这与 qwen3.5 NVFP4 AgentX MTP 配方对已有的 B200/B300 差异保持一致。

dram-utilization: 0.683 沿用 B300 数值,在 TP4 上解析出相同的 1,024 GB 引擎级 CPU KV 预算:b200-dgxcb300-nv 的实装内存均超过 MAX_AGENTIC_AVAILABLE_CPU_DRAM_MIB,两者都会先被收敛到相同的 3 TB AgentX 上限,再套用按 GPU 比例分配的规则。

验证

配置生成 11 个配置(仅 TP4)、最大并发 40,且 CPU KV 预算符合预期;validate_perf_changelog.py 通过;pytest utils/matrix_logic/ 224 项全部通过;启动器脚本路由解析到 minimaxm3_fp4_b200_mtp.shbash -n 无语法错误;镜像标签已在 Docker Hub 上确认存在。已添加 full-sweep-enabled 标签。

@functionstackx
functionstackx requested a review from a team August 6, 2026 19:40
@functionstackx
functionstackx force-pushed the feat/minimaxm3-fp4-b200-vllm-agentic-mtp branch 2 times, most recently from 1bde68c to cb83053 Compare August 6, 2026 19:51
Comment thread perf-changelog.yaml Outdated
- "Second B200 delta: draft staging. The B300 sibling stages the EAGLE3-GQA head under /data/models, which does not exist on b200-dgxc; the launcher bind-mounts only $MODEL_PATH itself, so the draft is staged in the container-local parent directory instead of polluting the shared checkpoint directory. Same approach the deprecated 8k1k B200 MiniMax-M3 MTP recipe used."
- "Search space mirrors the B300 sibling's TP2 / TP4 / TP4+SimpleCPU arms and truncates each where B200's smaller HBM pool puts the cliff: TP4 GPU-resident conc [1, 2, 5, 8, 10, 12, 15, 20], TP2 GPU-resident conc [1, 2], TP4 SimpleCPU conc [20, 30, 40, 50, 60, 65, 70, 75]. The ~220 GB NVFP4 checkpoint leaves roughly 104 GB of KV at TP2 and 428 GB at TP4 on B200's 180 GB HBM3e, against 298 GB and 817 GB for the same topologies on B300, so TP2 keeps only its low-concurrency corner and the host tier picks up one step earlier at TP4. Same B200/B300 asymmetry the qwen3.5 NVFP4 AgentX MTP pair already publishes."
- "dram-utilization 0.683 is carried over unchanged from the B300 entry and resolves to the same 1,024 GB engine-level CPU KV budget at TP4: b200-dgxc and b300-nv both report more installed DRAM than MAX_AGENTIC_AVAILABLE_CPU_DRAM_MIB, so both clamp to the same 3 TB AgentX ceiling before the proportional-GPU rule applies."
pr-link: TBD

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The new perf-changelog.yaml entry uses pr-link: TBD, but the merge-prep automation only accepts XXX (or the literal expected PR URL) as a placeholder. Change TBD to XXX so utils/merge_with_reuse.sh's canonicalization step can auto-fill the real PR link at merge time instead of hard-failing.

Extended reasoning...

perf-changelog.yaml line 5483 sets pr-link: TBD for the newly appended minimaxm3-fp4-b200-vllm-agentic-mtp entry. utils/validate_perf_changelog.py defines the accepted placeholders as:

PR_LINK_PLACEHOLDERS = {
    "XXX",
    "https://github.com/SemiAnalysisAI/InferenceX/pull/XXX",
}

TBD is not a member of this set, and validate_added_pr_link() (called from compare_entries()) raises ChangelogValidationError for any appended pr-link that is neither a placeholder nor the exact expected .../pull/<pr_number> URL.

Why PR-time CI doesn't catch it: the CLI entry point main() -> validate_matrix_compatible_change() only runs validate_generated_config() (which shells out to process_changelog.py, and that script never inspects pr-link at all) plus a trailing-newline check. It never calls compare_entries(), so the PR description's claim that validate_perf_changelog.py --base-ref origin/main passes is accurate — but it's validating a different thing than what the merge path validates.

Where it actually breaks: utils/merge_with_reuse.sh unconditionally invokes utils/prepare_perf_changelog_merge.py's canonicalize step during the standard reuse-merge flow (the path used by the /merge-prs and /find-mergeable-claude-prs skills). That script's canonicalize_appended_links() calls compare_entries(base, head, pr_number) with a real PR number, which routes into validate_added_pr_link() — and since TBD is neither a placeholder nor the canonical URL, it raises ChangelogValidationError("new PR entry must use ... or an XXX placeholder; found TBD"). Even if that check were bypassed, canonicalize_appended_links()'s own loop separately raises ChangelogValidationError(f"appended entry {index+1} has unexpected pr-link {link!r}") for the same reason — the automation is specifically built to recognize XXX and rewrite it to the real merged PR URL, and TBD doesn't match that contract.

Concrete walkthrough:

  1. PR [Klaud Cold] minimaxm3-fp4-b200-vllm-agentic-mtp: MiniMax-M3 NVFP4 B200 AgentX with EAGLE3-GQA MTP and golden-AL synthetic acceptance / 新增 MiniMax-M3 NVFP4 B200 vLLM EAGLE3-GQA MTP AgentX 配方(黄金 AL 合成接受率) #2511 is opened with pr-link: TBD on the new changelog entry.
  2. CI runs validate_perf_changelog.py --base-ref origin/main → passes, since that path never touches pr-link.
  3. PR gets approved and someone runs the standard merge automation (merge_with_reuse.sh <pr_number>).
  4. The script reaches its canonicalization step and calls prepare_perf_changelog_merge.py's canonicalize_appended_links().
  5. compare_entries() detects the new entry's pr-link: TBD, calls validate_added_pr_link("TBD", 2511), and since "TBD" not in PR_LINK_PLACEHOLDERS and it doesn't match https://github.com/SemiAnalysisAI/InferenceX/pull/2511, it raises.
  6. The merge hard-fails under set -euo pipefail, requiring a manual edit to pr-link before merge can proceed.

Fix: change pr-link: TBD to pr-link: XXX (matching AGENTS.md's documented convention of pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX for new entries), so the merge automation can auto-canonicalize it to the real PR URL without manual intervention.

This doesn't affect the benchmark recipe's correctness at all — it's purely a changelog metadata issue — but it will concretely block the standard automated merge path for this PR until fixed, which is why I'm not marking it as a nit despite the trivial one-token fix.

Comment on lines +83 to +101
mkdir -p "$MODEL_PATH"
MODEL_DOWNLOAD_LOCK="${MODEL_PATH%/}.download.lock"
echo "Checkpoint at $MODEL_PATH is incomplete; acquiring $MODEL_DOWNLOAD_LOCK"
exec 9>"$MODEL_DOWNLOAD_LOCK"
flock -w "${MODEL_DOWNLOAD_LOCK_TIMEOUT:-21600}" 9 || {
echo "Error: timed out waiting for another cell to stage $MODEL_PATH" >&2
exit 1
}
if checkpoint_is_complete "$MODEL_PATH"; then
echo "Another cell staged $MODEL_PATH while we waited"
else
hf download "$HF_MODEL_ID" --local-dir "$MODEL_PATH"
fi
flock -u 9
exec 9>&-
checkpoint_is_complete "$MODEL_PATH" || {
echo "Error: $MODEL_PATH is still incomplete after hf download $HF_MODEL_ID." >&2
exit 1
}

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔴 The checkpoint-download flock's lock file is placed at ${MODEL_PATH%/}.download.lock, a sibling of $MODEL_PATH in its parent directory — but launch_b200-dgxc.sh (line 509) bind-mounts only $MODEL_PATH itself into each cell's container, so that parent (and the lock file inside it) is private per-container overlay, not shared storage. On a cold start where the checkpoint isn't pre-staged, every concurrent cell in the sweep gets its own uncontested lock and simultaneously runs hf download against the same shared $MODEL_PATH — exactly the multi-writer race the flock was added to prevent. Fix by placing the lock file inside $MODEL_PATH (e.g. $MODEL_PATH/.download.lock), which is actually bind-mounted and shared across cells.

Extended reasoning...

The bug: In minimaxm3_fp4_b200_mtp.sh (lines 83-101), when the checkpoint at $MODEL_PATH is incomplete, the script does:

MODEL_DOWNLOAD_LOCK="${MODEL_PATH%/}.download.lock"
exec 9>"$MODEL_DOWNLOAD_LOCK"
flock -w "${MODEL_DOWNLOAD_LOCK_TIMEOUT:-21600}" 9 || { ... }

For the minimaxm3/fp4 case, MODEL_PATH=/scratch/fsw/models/MiniMax-M3-NVFP4 (launch_b200-dgxc.sh:73), so MODEL_DOWNLOAD_LOCK resolves to /scratch/fsw/models/MiniMax-M3-NVFP4.download.lock — a sibling file in the parent directory /scratch/fsw/models/, not a path inside $MODEL_PATH itself.

Why the lock never crosses containers: launch_b200-dgxc.sh:509 bind-mounts only:

--container-mounts=$GITHUB_WORKSPACE:$CONTAINER_MOUNT_DIR,$MODEL_PATH:$MODEL_PATH,$AIPERF_MMAP_CACHE_HOST_PATH:/aiperf_mmap_cache

The parent directory /scratch/fsw/models/ is never bind-mounted from the host. Every cell in the sweep runs as its own salloc --exclusive/srun allocation with its own container (lines 484-513), so each container's view of /scratch/fsw/models/ above the $MODEL_PATH mount point is its own private overlay filesystem. The PR's own comment on the draft-staging block two paragraphs below confirms this exact model of the parent directory: "the launcher bind-mounts only $MODEL_PATH itself ... so its parent exists solely inside the container overlay — writable, but per-job." That statement applies equally to the lock file, which lives at that same per-job parent path.

Concrete walk-through: Suppose the checkpoint is not pre-staged (a fresh cluster, or a purged cache) and the sweep launches the 8-point TP4 GPU-resident arm (conc = 1, 2, 5, 8, 10, 12, 15, 20) concurrently, each as its own srun/container:

  1. Cell A (conc=1) checks checkpoint_is_complete "$MODEL_PATH" → false. It runs exec 9>/scratch/fsw/models/MiniMax-M3-NVFP4.download.lock inside its own container overlay, creating a private lock file no other container can see.
  2. Cell B (conc=2), running concurrently in a separate container, does the exact same thing at the same path string — but that path resolves to its own private overlay copy, not Cell A's. flock acquires instantly since there is no real contention.
  3. Both (and cells C through H) proceed to run hf download nvidia/MiniMax-M3-NVFP4 --local-dir "$MODEL_PATH" at the same time, all writing into the one real, bind-mounted, shared $MODEL_PATH directory on /scratch/fsw.
  4. Result: up to 8 concurrent ~220 GB downloads writing shards into the same shared directory — wasted bandwidth, and a real risk that one cell's checkpoint_is_complete check sees "all shards present" while a sibling cell is still mid-write on some of those same shard files, loading a truncated/corrupt checkpoint.

This is precisely the multi-writer race the flock block's own comment says it exists to prevent ("serialize: one cell pulls the ~220 GB checkpoint and the rest wait on it rather than racing as writers").

Why nothing else catches this: checkpoint_is_complete only checks for shard presence, not size/hash, so a checkpoint mid-write by another cell can appear complete once all shard filenames exist, even if content is still being written. hf download's internal locking (if any) is scoped to a single process/host and does not span separate containers on separate nodes.

Fix: Put the lock file inside the shared mount, e.g. MODEL_DOWNLOAD_LOCK="$MODEL_PATH/.download.lock" (or ${MODEL_PATH%/}/.download.lock), since $MODEL_PATH itself is the one path actually bind-mounted and shared across all cells. This is a one-line fix and doesn't require any other structural change to the block.

Scope note: In steady state, the checkpoint is normally pre-staged on /scratch/fsw/models (per the launcher's own documented pre-provisioned paths), so checkpoint_is_complete short-circuits and this download+flock block never executes. The bug only bites the cold-start/day-zero staging path — but it fully defeats the safety mechanism the PR explicitly added for exactly that scenario, and the failure mode when triggered is concurrent corruption-prone writers on a real shared 220 GB checkpoint directory.

Comment on lines +104 to +116
hf download "$HF_MODEL_ID"
export MODEL_PATH="$HF_MODEL_ID"
fi

# B200: the B300 sibling stages the draft under /data/models, which does not
# exist on b200-dgxc. The launcher bind-mounts only $MODEL_PATH itself
# (--container-mounts=...,$MODEL_PATH:$MODEL_PATH,...), so its parent exists
# solely inside the container overlay -- writable, but per-job. Stage the draft
# there rather than inside $MODEL_PATH, which is the shared read-mostly
# checkpoint directory and must not be polluted. The EAGLE3-GQA head is small
# next to the target, so the per-job pull is cheap; this is what the
# now-deprecated 8k1k B200 MiniMax-M3 MTP recipe did as well.
DRAFT_MODEL_PATH="${DRAFT_MODEL_PATH:-$(dirname "${MODEL_PATH%/}")/${DRAFT_MODEL##*/}}"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 In the else-branch where MODEL_PATH is unset (manual runs without a pre-set path), MODEL_PATH is set to the HF repo id nvidia/MiniMax-M3-NVFP4, but DRAFT_MODEL_PATH is still derived via dirname "${MODEL_PATH%/}", which resolves to the bogus relative path nvidia/MiniMax-M3-EAGLE3-GQA instead of a real staging directory. This branch is dead in the actual b200-dgxc sweep since the launcher always exports a real filesystem MODEL_PATH, so it only affects manual/bring-up runs; fix by mirroring the B300 sibling'''s else-branch, which sets DRAFT_MODEL_PATH=$DRAFT_MODEL directly.

Extended reasoning...

The bug: benchmarks/single_node/agentic/minimaxm3_fp4_b200_mtp.sh has an else-branch (lines ~104-105) that runs when MODEL_PATH is not pre-set:

else
    hf download "$HF_MODEL_ID"
    export MODEL_PATH="$HF_MODEL_ID"
fi

HF_MODEL_ID is a Hugging Face repo id string (nvidia/MiniMax-M3-NVFP4), not a filesystem path. Further down (line 116), DRAFT_MODEL_PATH is computed unconditionally from MODEL_PATH regardless of which branch set it:

DRAFT_MODEL_PATH="${DRAFT_MODEL_PATH:-$(dirname "${MODEL_PATH%/}")/${DRAFT_MODEL##*/}}"

Step-by-step proof:

  1. MODEL_PATH = "nvidia/MiniMax-M3-NVFP4" (set by the else-branch).
  2. ${MODEL_PATH%/} strips a trailing slash (none present) → unchanged.
  3. dirname "nvidia/MiniMax-M3-NVFP4""nvidia".
  4. ${DRAFT_MODEL##*/} where DRAFT_MODEL="Inferact/MiniMax-M3-EAGLE3-GQA""MiniMax-M3-EAGLE3-GQA".
  5. Result: DRAFT_MODEL_PATH="nvidia/MiniMax-M3-EAGLE3-GQA" — a nonsensical relative path built from the target model's org (nvidia) rather than any real staging directory.

Why this formula exists at all: it's correct in the if [[ -n "${MODEL_PATH:-}" ]] branch, where MODEL_PATH really is a filesystem path (e.g. /scratch/fsw/models/MiniMax-M3-NVFP4), so dirname yields a legitimate parent directory to stage the draft alongside the target. The bug is that this same formula is reused unconditionally after the if/else, even though the else-branch's MODEL_PATH is a repo id, not a path.

Why the B300 sibling doesn't have this issue: minimaxm3_fp4_b300_mtp.sh's else-branch explicitly sets DRAFT_MODEL_PATH="$DRAFT_MODEL" (the draft's own repo id) directly, never deriving it from MODEL_PATH. The B200 port collapsed this into a single shared line after the if/else, which broke that independence.

Impact: This is dead code in the actual b200-dgxc sweep — runners/launch_b200-dgxc.sh always rewrites MODEL to a real cluster filesystem path before invoking this script, so MODEL_PATH is always set going in, and the else-branch (and this bug) never executes during a real sweep run. It only bites someone running the script manually/for local bring-up without pre-setting MODEL_PATH. Even then it doesn't crash: hf download "$DRAFT_MODEL" --local-dir "$DRAFT_MODEL_PATH" will happily create ./nvidia/MiniMax-M3-EAGLE3-GQA relative to the current working directory and the subsequent vLLM invocation would likely still resolve it as a local path. The practical effect is a confusing, incorrectly-named ./nvidia/ directory left in the CWD rather than a real staging location — messy but not a functional break.

Fix: In the else-branch, set DRAFT_MODEL_PATH="$DRAFT_MODEL" directly (mirroring the B300 sibling), or otherwise make the dirname-based derivation conditional on the if-branch only.

@functionstackx
functionstackx force-pushed the feat/minimaxm3-fp4-b200-vllm-agentic-mtp branch from cb83053 to 873b59e Compare August 7, 2026 02:33
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

1 similar comment
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

functionstackx and others added 4 commits August 7, 2026 15:09
…GLE3-GQA MTP recipe

Port of the validated B300 sibling minimaxm3-fp4-b300-vllm-agentic-mtp to
b200-dgxc. Same image, same Inferact/MiniMax-M3-EAGLE3-GQA draft head at three
speculative tokens, same golden AL 2.78, same TP2/TP4 topologies and SimpleCPU
offload arm, so the two SKU curves stay directly comparable.

Spec-decode only, per the AgentX policy in MODELS.md. This is MiniMax-M3's only
active B200 configuration: the Single-turn 8k1k scenario that carried the
previous B200 curves was deprecated in #2493.

Two B200 deltas: the checkpoint-resolution guard (launch_b200-dgxc.sh rewrites
MODEL to a cluster-local path, so the HF repo id is kept in HF_MODEL_ID and the
checkpoint is verified against its shard index behind a flock) and draft staging
(the B300 /data/models path does not exist on b200-dgxc). Concurrency per arm is
truncated where B200's 180 GB HBM3e puts the cliff.

中文:新增 MiniMax-M3 NVFP4 在 B200 上的 AgentX(agentic-coding)配方,使用 vLLM
搭配 Inferact/MiniMax-M3-EAGLE3-GQA 草稿头做投机解码(投机 token 数 3,黄金接受
长度 2.78)。镜像、草稿模型、TP2/TP4 拓扑与 SimpleCPU 卸载分支均与 B300 同源配方
一致,保证两个 SKU 的曲线可直接对比。按 MODELS.md 的 AgentX 策略仅发布投机解码
分支;#2493 停用 8k1k 场景后,这是 MiniMax-M3 在 B200 上唯一的有效配置。B200 的
两处差异为检查点解析保护(启动器会把 MODEL 改写为集群本地路径)与草稿模型的暂存
路径;各分支并发上限按 B200 的 180 GB 显存下调。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
中文:为 #2511 填写 perf-changelog 的 pr-link。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The recipe's memory rationale cited a ~220 GB checkpoint, derived by halving
the documented ~444 GB MXFP8 figure. nvidia/MiniMax-M3-NVFP4 is actually
250.1 GB across 88 shards, so the derived KV budgets were ~30 GB optimistic on
every arm: TP2 has ~74 GB rather than ~104 GB and TP4 ~398 GB rather than
~428 GB on B200, against ~268 GB and ~787 GB on B300.

No search-space change -- the corrected numbers only reinforce the existing
truncations (TP2 capped at conc 2, host tier engaged at conc 20, hard stop
at 40).

中文:修正配方中 NVFP4 检查点大小的说明。此前按 MXFP8 的 ~444 GB 折半估算为
~220 GB,实际 nvidia/MiniMax-M3-NVFP4 为 250.1 GB(88 个分片),因此各分支推导
出的 KV 预算均偏乐观约 30 GB。搜索空间不变——修正后的数值只会进一步支持现有的
并发截断。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…1M context

Both TP2 cells failed identically at engine init on run 31141913741 while every
TP4 cell passed:

  ValueError: To serve at least one request with the model's max seq len
  (1048576), 37.62 GiB KV cache is needed, which is larger than the available
  KV cache memory (31.72 GiB).

With the ~250 GB NVFP4 checkpoint resident, two B200s leave a 31.72 GiB KV pool
at --gpu-memory-utilization 0.9, and vLLM estimates the achievable max model
length at 883,840 -- below the agentic corpus context. This is structural, not
a concurrency cliff, so no conc-list would have made the arm run and lowering
the grid would only burn engine starts. The B300 sibling keeps its TP2 arm; on
288 GB GPUs the same topology has ~268 GB of KV.

Recipe is now TP4-only: 8 GPU-resident points plus 3 SimpleCPU points.

中文:移除 TP2 分支。在 run 31141913741 中两个 TP2 单元均在引擎初始化阶段失败,而
所有 TP4 单元均通过:约 250 GB 的 NVFP4 权重驻留后,两张 B200 在
--gpu-memory-utilization 0.9 下仅剩 31.72 GiB 的 KV 池,而按模型 1,048,576 的
max_model_len 计算,单个请求就需要 37.62 GiB。vLLM 估算 TP2 可支持的最大上下文为
883,840,低于智能体语料的上下文长度,因此这是结构性限制而非并发瓶颈,调整
conc-list 无济于事。B300 同源配方保留 TP2 分支(288 GB 显存下该拓扑约有 268 GB
KV)。本配方现为纯 TP4:8 个显存驻留点加 3 个 SimpleCPU 点。

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@functionstackx
functionstackx force-pushed the feat/minimaxm3-fp4-b200-vllm-agentic-mtp branch from 2221c9d to 3e3db68 Compare August 7, 2026 19:10
@github-actions

github-actions Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

1 similar comment
@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

@functionstackx

Copy link
Copy Markdown
Collaborator Author

/stage-results 31210279071

@github-actions

github-actions Bot commented Aug 8, 2026

Copy link
Copy Markdown
Contributor

@functionstackx staged run 31210279071: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-07~r31210279071

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

@functionstackx 已将运行 31210279071 发布到预发布环境:https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-07~r31210279071

后续的 /stage-results 请求不会移除此运行;再次发布相同的运行 ID 会更新其预发布数据。预发布工作流

@functionstackx

Copy link
Copy Markdown
Collaborator Author

/reuse-sweep-run

@functionstackx
functionstackx force-pushed the feat/minimaxm3-fp4-b200-vllm-agentic-mtp branch from 3e3db68 to f70cedc Compare August 8, 2026 01:21
# Conflicts:
#	configs/nvidia-master.yaml
#	perf-changelog.yaml
@functionstackx
functionstackx force-pushed the feat/minimaxm3-fp4-b200-vllm-agentic-mtp branch from f70cedc to 41316a7 Compare August 8, 2026 01:22
@functionstackx

Copy link
Copy Markdown
Collaborator Author

/reuse-sweep-run

@functionstackx
functionstackx merged commit 75908a8 into main Aug 8, 2026
26 checks passed
@functionstackx
functionstackx deleted the feat/minimaxm3-fp4-b200-vllm-agentic-mtp branch August 8, 2026 01:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

Development

Successfully merging this pull request may close these issues.

1 participant