Skip to content

[Feat] DeepEP M2N: flexible Attention/FFN placement across pods - #51

Open
ronenkat wants to merge 6 commits into
mainfrom
m2n-flexible-placement
Open

ronenkat wants to merge 6 commits into
mainfrom
m2n-flexible-placement

Conversation

@ronenkat

@ronenkat ronenkat commented Nov 2, 2026

Copy link
Copy Markdown

The M2N connector accepted only placements whose NVLink domains held FFN-first, contiguous blocks of the role world, so pods that each hold FFN and Attention ranks (2 x 4A1F) were rejected.

The handshake now plans a transport order from the ranks' NVLink domains: domain by domain, FFN ranks first in each. When the FFN ranks fill the first domains the order is the role world and nothing changes. When every domain holds the same number of FFN ranks, the lanes use a gloo group in transport order and afdEP-comm's per-domain mode (ffn_ranks_per_domain). The role world still drives the handshake, model checks, control plane and expert ownership.

  • layout.py: RankPlacement, TransportLayout, plan_transport_layout
  • connector.py: transport layout in HandshakeResult, transport group, ffn_ranks_per_domain on each lane, network plan in transport order
  • pin afdEP-comm to the ffn-domains-hybrid head (fdff967)
  • recipe: Qwen3-235B-A22B-FP8 2 x 4A1F multi-pod recipe
  • E2E: Qwen3 multi-pod 1A1F,1A1F M2N eager/graph/DBO cases
  • docs and deploy-afd-k8s skill: placement rule, POD=AFD_

PLEASE FILL IN THE PR DESCRIPTION AND MAKE SURE THE CHECKLIST ITEMS HAVE BEEN CONSIDERED.

Purpose

Issue

  • Related issue(s): #
  • Closing keyword, only if fully resolved: Closes #

Scope

  • In scope:
  • Out of scope:

Implementation Notes

Test Plan

Test Result

Docs Impact

  • Files updated:
  • If none, reason:

Essential PR Checklist
  • Purpose is clear and linked to public context when possible.
  • Scope is bounded.
  • Compatibility with vLLM v0.32.0 and, for NPU changes, the pinned vLLM-Ascend source is considered, as applicable.
  • No changes are made to the vLLM source checkout.
  • Plugin-owned classes or explicit dotted class paths are preferred over monkey patches.
  • Any compat shim or monkey patch is isolated, idempotent, version-guarded, documented, and tested.
  • Imports remain CPU-safe; CUDA-heavy work is delayed or GPU-gated.
  • Validation evidence is included, including skipped GPU tests when applicable.
  • Documentation impact is stated.

ronenkat and others added 6 commits November 2, 2026 15:35
The M2N connector accepted only placements whose NVLink domains held
FFN-first, contiguous blocks of the role world, so pods that each hold
FFN and Attention ranks (2 x 4A1F) were rejected.

The handshake now plans a transport order from the ranks' NVLink
domains: domain by domain, FFN ranks first in each. When the FFN ranks
fill the first domains the order is the role world and nothing changes.
When every domain holds the same number of FFN ranks, the lanes use a
gloo group in transport order and afdEP-comm's per-domain mode
(ffn_ranks_per_domain). The role world still drives the handshake,
model checks, control plane and expert ownership.

- layout.py: RankPlacement, TransportLayout, plan_transport_layout
- connector.py: transport layout in HandshakeResult, transport group,
  ffn_ranks_per_domain on each lane, network plan in transport order
- pin afdEP-comm to the ffn-domains-hybrid head (fdff967)
- recipe: Qwen3-235B-A22B-FP8 2 x 4A1F multi-pod recipe
- E2E: Qwen3 multi-pod 1A1F,1A1F M2N eager/graph/DBO cases
- docs and deploy-afd-k8s skill: placement rule, POD=AFD_<index>

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: ronenkat <16743404+ronenkat@users.noreply.github.com>
The plugin's INFO lines are not printed under vLLM's logging setup, so the
docs should not point users at the transport log line. A placement the
connector cannot carry still fails at startup with the pods' shapes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: ronenkat <16743404+ronenkat@users.noreply.github.com>
Signed-off-by: ronenkat <16743404+ronenkat@users.noreply.github.com>
afdEP-comm PR #27 (FFN ranks on every node) merged and shipped as tag
v1.1.0 (741b9aa). Its source tree matches the PR head fdff967 that this
branch pinned, so the per-domain transport is unchanged. Pin the tag in
pyproject.toml; uv.lock keeps the resolved commit.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: ronenkat <16743404+ronenkat@users.noreply.github.com>
Resolve the deploy-afd-k8s conflicts from the references/ -> resources/
rename and the single-recipe-parser rework: keep the POD=AFD_<index>
deploy example before the new step 4f heading, and move the AFD_<index>
GPU_COUNT rule from deploy.md step 2 into resolve-recipe.md, since those
recipes build CUDA_VISIBLE_DEVICES with seq rather than literal lists.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Signed-off-by: ronenkat <16743404+ronenkat@users.noreply.github.com>
@github-actions github-actions Bot added documentation Improvements or additions to documentation NVIDIA labels Nov 9, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

documentation Improvements or additions to documentation NVIDIA

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant