port MI355X DeepSeek V4 ATOM disagg to srt-slurm

[cquil11]

Summary

Ports the active dsv4-fp4-mi355x-atom-disagg 8k/1k submission from the configuration-specific AMD launcher to the native ATOM and AToMesh implementation stacked in SemiAnalysisAI/srt-slurm#7.

Scope

• preserve the original 1P1D TP8 points: c4, c8, c16, c32, c64, c128
• preserve the original 2P1D DPA+TP8 points: c256, c512, c768, c1024, c2048
• run the unchanged InferenceX utils/bench_serving/benchmark_serving.py through the custom benchmark contract
• use PR7's native backend: atom and frontend: atomesh orchestration
• use rocm/atom-dev:nightly_202608251555 at ATOM commit 4e0848dcdf94f3d4be0c1e3ce51db99bc93f8b8c, including its bundled AToMesh and Mooncake implementation
• use the hardware-proven Mooncake TCP path initially; RDMA validation follows separately
• remove the exact legacy wrapper and obsolete deprecated 1k/1k entry that depended on it
• do not depend on Infera or srt-slurm PR3

Dependency structure

• srt-slurm AMD runtime: feat(runtime): add AMD accelerator support srt-slurm#1
• stacked native ATOM/AToMesh implementation: feat(atom): add native ATOM and AToMesh support srt-slurm#7
• this InferenceX PR is based directly on feat(amd): orchestrate AMD clusters with srt-slurm #2542, which owns the shared MI355X launcher, cluster profile, and legacy cleanup

Local validation

• both recipes schema-load against exact srt-slurm PR7 head 76e7d76961b2dcb27cb05c1e9e0910ceb75104ec
• generated InferenceX matrix contains exactly two topology jobs with exact node demand: 1P1D=2 nodes and 2P1D=3 nodes
• 277 focused InferenceX matrix, changelog, and node-demand tests pass
• YAML parsing, shell syntax, and git diff --check pass
• direct image audit confirms native atom.entrypoints.openai_server, Mooncake, and /usr/local/bin/atomesh are bundled in the selected image

Hardware validation

WIP. The PR remains draft until exact-head full-sweep validation completes. Workflow, Slurm, endpoint, and artifact evidence will be added here during validation.

---

Note

Low Risk
YAML-only benchmark and matrix wiring with no InferenceX serving code changes; misconfiguration could waste multi-node GPU time but does not touch auth or data paths.

Overview
Moves the DeepSeek-V4-Pro FP4 MI355X disaggregated ATOM 8k/1k lane from the legacy AMD launcher onto srt-slurm with native AToMesh routing and Mooncake TCP KV transfer (RDMA deferred until a newer image is validated).

Adds two Slurm recipes: 1P1D TP8 (one prefill + one decode node) and 2P1D DPA+TP8 (two prefill workers with DP attention/TBO plus decode). Each recipe runs a custom benchmark shell that loops the original concurrency grid—1P1D at 4–128, 2P1D at 256–2048—via unchanged benchmark_serving.py, archives runtime logs, and pins rocm/atom-dev:nightly_202608251555 / ATOM+AToMesh commits.

Registers dsv4-fp4-mi355x-atom-disagg in amd-master.yaml with one matrix row per topology (conc-list: [1]) so the recipe-owned loops are not duplicated per concurrency point. Documents the change in perf-changelog.yaml and ties orchestration to srt-slurm PR #7.

^{Reviewed by Cursor Bugbot for commit 2e73c89. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[cquil11]

Live 2P/1D DPA validation update:\n\n- Slurm 39443 completed all 64 model shards and CUDA-graph capture on both prefills and the decode.\n- Startup then failed deterministically in every logical worker because each ATOM data-parallel EngineCore tried to bind the same KV-event ZMQ endpoint. One DP rank bound successfully; the others raised Address already in use from Infera BlockManager hook initialization, so the worker could not register.\n- This is an Infera multi-DP KV-event publisher bug, not Slurm, MI355X hardware, model loading, Mooncake, or the srt-slurm topology.\n- The upstream fix is open as AMD-AGI/Infera#121. It gives every DP rank a local PUB ingress and relays all rank streams through the one endpoint advertised for the logical worker, retaining complete KV-aware cache visibility.\n- Focused engine/router validation is green: 50 tests passed, including two simultaneous EngineCore publishers through one advertised endpoint; Ruff, pre-commit, and diff checks passed.\n- The current exact-head workflow was not cancelled. Its 1P/1D arm is still running naturally as Slurm 39448. I will pin the reviewed fix and launch a fresh exact-head validation only after this attempt closes.

[claude]

Claude finished @cquil11's task in 4m 33s —— View job

---

Review complete

• Gather context (diff vs agent/srt-slurm-amd-integration, read changed files)
• Review the two new srt-slurm recipe YAMLs
• Review configs/amd-master.yaml changes (image accessibility, config key format)
• Validate perf-changelog.yaml entry placement and content
• Post review feedback

Summary

Found 1 blocking issue and 1 minor issue (posted as inline comments):

1. 🔴 srt-slurm pin discrepancy (perf-changelog.yaml:6571): the changelog claims PR7 is pinned at 10a58d2e…, but the effective pin is SRT_SLURM_COMMIT=c87d7b34… in runners/launch_mi355x-amds-srt.sh:8, which this PR does not touch (and the PR body cites a third SHA, 76e7d76…, as the validated head). The new recipes require PR7's backend: atom / frontend: atomesh / mooncake_protocol support, so if the launcher pin predates PR7, both topology jobs fail at srtctl apply. Bump the launcher pin to the validated PR7 head, or correct the changelog if c87d7b34 really includes ATOM support.
2. 🟡 Stray comment (configs/amd-master.yaml:913): dangling duplicated # 1P1D TP8 copy-paste leftover at end of the new block.

Everything else checked out: the changelog entry is correctly appended at the end of the file and both master-config and changelog were updated together; dsv4 is an established config-key prefix in this repo; rocm/atom-dev:nightly_202608251555 is a public Docker Hub image; the conc-list: [1] + recipe-owned concurrency loop follows the parent PR's established srt-slurm pattern; the result contract (/results/${SLURM_JOB_ID}/fixed-seq/*-c<N>.json) matches the launcher's collection/parsing logic; all benchmark_serving.py flags used (--backend openai, --num-warmups, --percentile-metrics, --trust-remote-code) exist; and the launcher's DP-attn mutation hook (which only knows sglang_config) is safely skipped for these entries since ep: 1.

Note: the inferencemax-repos MCP server failed to connect this session, and a cross-repo gh api lookup of the srt-slurm SHAs required approval, so I could not externally confirm what c87d7b34 contains — the pin finding is based on the in-repo discrepancy.

[claude]

🔴 BLOCKING: This changelog entry claims srt-slurm PR #7 is pinned at 10a58d2ebb4d756c423424049c736b909b72e14f, but nothing in this PR (or the repo) actually pins that commit. The shared launcher runners/launch_mi355x-amds-srt.sh:8 still has SRT_SLURM_COMMIT="c87d7b34b009be920896126013ad6dc74c5a99d5", which was set by the parent PR (#2542) for the SGLang-native lanes, and this PR does not modify the launcher.

Why it matters: The two new recipes depend on srt-slurm PR7 features (backend: type: atom, frontend: type: atomesh, connector: mooncake, mooncake_protocol, atom_config). If c87d7b34 predates PR7's native ATOM/AToMesh support, srtctl apply will fail schema validation for both topology jobs and the sweep can never pass. Note the PR body also cites a third SHA (76e7d76961…) as the validated PR7 head, so the intended pin is ambiguous — right now three different commits are referenced and only c87d7b34 is effective.

Fix: Bump SRT_SLURM_COMMIT in runners/launch_mi355x-amds-srt.sh to the exact PR7 head you validated against (and make the changelog text match that SHA), or if c87d7b34 genuinely contains the ATOM/AToMesh support, correct this changelog line to the real pin. Fix this →

[claude]

🟡 WARNING: Copy-paste leftover — this trailing # 1P1D TP8 comment duplicates the label of the block above and has no entry after it.

Suggested change

# 1P1D TP8

[claude]

Looks good — straightforward config port (new ATOM disagg benchmark recipes + matching amd-master.yaml matrix entry and changelog record). The reported dangling # 1P1D TP8 comment is a harmless cosmetic nit and doesn't need to block this.

Extended reasoning...

Overview

This PR adds two new srt-slurm benchmark recipe YAMLs (disagg-1p1d-tp8-8k1k.yaml and disagg-2p1d-dpa-tp8-8k1k.yaml) for DeepSeek-V4-Pro FP4 ATOM disaggregated serving on MI355X, a corresponding dsv4-fp4-mi355x-atom-disagg entry in configs/amd-master.yaml, and a perf-changelog.yaml record documenting the port from the legacy AMD launcher to native srt-slurm/AToMesh orchestration. It is purely additive config/data — no application logic, scripts, or CI behavior is touched.

Security risks

None identified. The changes are declarative YAML (image tags, resource counts, benchmark shell commands run inside an already-trusted Slurm/container pipeline). No secrets, credentials, or injection-relevant surfaces are introduced.

Level of scrutiny

Low. This is a mechanical, self-contained config addition with no ambiguity in intent, matching an existing pattern already used elsewhere in amd-master.yaml (e.g., the preceding Qwen3.5 srt-slurm migration comment block). The only finding surfaced — a stray duplicate # 1P1D TP8 comment at the end of the search-space list with no entry following it — is purely cosmetic and does not affect parsing or benchmark behavior.

Other factors

A separate candidate issue (prefill max-num-seqs: 256 fixed while the benchmark sweep goes to max-concurrency 2048 in the 2P1D config) was investigated by the earlier bug-hunting pass and ruled out, consistent with decode being the component sized to the full concurrency sweep in a disaggregated P/D topology. No third-party CHANGES_REQUESTED or unresolved objections are present in the conversation timeline, and the single commit in scope matches the diff reviewed here.

[claude]

🟡 nit: dangling duplicate # 1P1D TP8 comment after the last search-space entry, with no row following it.

Extended reasoning...

A future editor reading the file sees a label comment with nothing under it, suggesting a third row was meant to be added but was left out (or the label was copy-pasted and forgotten), making the intent of the trailing comment ambiguous when they extend this matrix.

Verification: nit. configs/amd-master.yaml line 913 is # 1P1D TP8, the final line of the file added by this diff. It follows the second search-space row's last entry (- "DECODE_NODES=1" at line 912) with no YAML mapping under it, and duplicates the # 1P1D TP8 label already present at line 891 for that same row. It is a dangling/orphaned comment. Cosmetic only — YAML ignores comments, so no parsing or…