[contrib] Add MiMo-V2.5-Pro (Xiaomi, 384 experts MoE, FP8 on Trn2) - #150

Open

whn09 wants to merge 48 commits into

aws-neuron:mainfrom

whn09:contrib/MiMo-V2.5-Pro

Commits on Apr 27, 2026

[contrib] Add MiMo-V2.5-Pro initial port (copied from MiMo-V2-Pro, renamed)

whn09
and
claude
committed
82d7ead
[contrib] MiMo-V2.5-Pro: apply review findings from HF reference diff

whn09
and
claude
committed
e4a13c1

Commits on Apr 28, 2026

[contrib] MiMo-V2.5-Pro: document current WIP status in README

whn09
and
claude
committed
78b8e0a
[contrib] MiMo-V2.5-Pro: correct FP8 root-cause framing in Status

whn09
and
claude
committed
e990f76
[contrib] MiMo-V2.5-Pro: parallel preprocess + NVMe mount docs

whn09
and
claude
committed
f0d9c0b
[contrib] MiMo-V2.5-Pro: set AWS Llama-405B FP8 env vars in smoke scripts

whn09
and
claude
committed
b455a1e
[contrib] MiMo-V2.5-Pro: fix tokenizer padding_side='left' for decoder-only LM

whn09
and
claude
committed
9d29bbf
[contrib] MiMo-V2.5-Pro: skip FP8 quant on q/k/v_proj (attention path)

whn09
and
claude
committed
2a4c9ff
[contrib] MiMo-V2.5-Pro: try use_torch_block_wise + restore FP8 q/k/v

whn09
and
claude
committed
2b8b577

Commits on Apr 29, 2026

[contrib] MiMo-V2.5-Pro: wire up vLLM serving and record FP8 perf

whn09
and
claude
committed
b3a8487
[contrib] MiMo-V2.5-Pro: fix physical NC count (128, not 32)

whn09
and
claude
committed
bbb1e1f
[contrib] MiMo-V2.5-Pro: split vLLM launcher into start/bench/sanity trio

whn09
and
claude
committed
d0eb413
[contrib] MiMo-V2.5-Pro: swap sanity_check default prompt to self-intro

whn09
and
claude
committed
2a181c1
[contrib] MiMo-V2.5-Pro: BF16-attn recipe restores coherent output

whn09
and
claude
committed
82d5797
[contrib] MiMo-V2.5-Pro: fold BF16 attn into preprocess; drop repatch

whn09
and
claude
committed
cf60b91
[contrib] MiMo-V2.5-Pro: standardize on pytorch_inference_vllm_0_16 venv

whn09
and
claude
committed
62fea53
[contrib] MiMo-V2.5-Pro: trim Status, reframe perf numbers as historical

whn09
and
claude
committed
e9ae094
[contrib] MiMo-V2.5-Pro: fix maintainer name typo
whn09
committed
d1ea946
[contrib] MiMo-V2.5-Pro: align start_vllm_server.sh with BF16-attn ckpt

whn09
and
claude
committed
ad81f3f
[contrib] MiMo-V2.5-Pro: fit sanity + bench within seq_len=256

whn09
and
claude
committed
5a297cb
[contrib] MiMo-V2.5-Pro: sanity_check.sh — short system prompt, /v1/chat

whn09
and
claude
committed
965a947
[contrib] MiMo-V2.5-Pro: default vLLM to smoke NEFF (workaround)

whn09
and
claude
committed
935510a
Revert "[contrib] MiMo-V2.5-Pro: default vLLM to smoke NEFF (workaround)"
whn09
committed
6da7188

Commits on Apr 30, 2026

[contrib] MiMo-V2.5-Pro: bump default seq_len 256 -> 512; document vLLM bug

whn09
and
claude
committed
af27106

Commits on Jul 21, 2026

Merge branch 'aws-neuron:main' into contrib/MiMo-V2.5-Pro
whn09
authored
be39ff8
[contrib] MiMo-V2.5-Pro: fix README (HF size ~1TB not 290GB, remove non-existent FP8->BF16 fallback, record verified toolchain versions, fix NEURON_COMPILED_ARTIFACTS default path, re-verify smoke …

whn09
and
claude
committed
c1e3a12
[contrib] MiMo-V2.5-Pro: fix vLLM-neuron patch so serving actually loads

whn09
and
claude
committed
f5c7d44
[contrib] MiMo-V2.5-Pro: add smoke-path throughput benchmark + prefill/decode profiling data

whn09
and
claude
committed
9a89c6c
[contrib] MiMo-V2.5-Pro: fix smoke scripts default COMPILED_PATH to models/compiled/

whn09
and
claude
committed
37b3511
[contrib] MiMo-V2.5-Pro: switch vllm-neuron to release-0.5.3

whn09
and
claude
committed
7647724
[contrib] MiMo-V2.5-Pro: fix vLLM bench to seq512 + reusable artifacts path

whn09
and
claude
committed
5262e2d
[contrib] MiMo-V2.5-Pro: add H100 GPU baseline + generalize run_bench_single.sh

whn09
and
claude
committed
a4c762a
[contrib] MiMo-V2.5-Pro: add SGLang H100 baseline + EFA Docker images, 3-way comparison

whn09
and
claude
committed
f5e9e9b
[contrib] MiMo-V2.5-Pro: complete H100 benchmark matrix (c=1/16/48, EFA vs socket)

whn09
and
claude
committed
775fd51
[contrib] MiMo-V2.5-Pro: add SGLang socket c=1/c=16 (complete EFA-vs-socket matrix)

whn09
and
claude
committed
7f5e694
[contrib] MiMo-V2.5-Pro: restore main README (was clobbered by h100 README in 7f5e694)

whn09
and
claude
committed
eec7111
[contrib] MiMo-V2.5-Pro: add vLLM DP+EP config, best vLLM multi-node 270 tok/s

whn09
and
claude
committed
0b7a22a
[contrib] MiMo-V2.5-Pro: add vLLM DP+EP c=1/c=16 (complete matrix)

whn09
and
claude
committed
a2fea21
[contrib] MiMo-V2.5-Pro: fair H100 rerun -- cache off + chunked prefill on both stacks

whn09
and
claude
committed
6f2c14d
[contrib] MiMo-V2.5-Pro: fix decode sliding-window mask to use per-batch-slot position

whn09
and
claude
committed
3d8b987
Fix batched-decode garble: per-row causal upper bound on prior KV

whn09
and
claude
committed
429ad08

Commits on Jul 22, 2026

h100/Pro: add 2-node long-context @ c=48 exploration (up to 32K input)

whn09
and
claude
committed
c42e5ef

Commits on Jul 23, 2026

h100/Pro: add 2-node 4K/128 bench at c=1/16/48 (Trn2 4K align)

whn09
and
claude
committed
7e2ad31
Port data-parallel + context-parallel attention infra to MiMo-V2.5-Pro

whn09
and
claude
committed
4a183f4
docs: note Pro long-context infra and the per-core HBM limit

whn09
and
claude
committed
06ae0d1

Commits on Jul 29, 2026

docs: pin neuronx-cc to 2.25.3371 -- 2.26.6360 miscompiles this model

whn09
and
claude
committed
9e729db

Commits on Jul 30, 2026

Pro: opt-in SWA KV cache truncation, and the measured context ceiling

whn09
and
claude
committed
8b2915d
Pro: document the S3 compiled-artifact backups and shard reuse

whn09
and
claude
committed
4404c60