Skip to content

Nccl translator#50

Open
Amos-Rai-KEYS wants to merge 12 commits into
mainfrom
nccl-translator
Open

Nccl translator#50
Amos-Rai-KEYS wants to merge 12 commits into
mainfrom
nccl-translator

Conversation

@Amos-Rai-KEYS

@Amos-Rai-KEYS Amos-Rai-KEYS commented Apr 21, 2026

Copy link
Copy Markdown
Collaborator

Add NCCL topology translator

Summary

This branch adds a new NCCL translator that parses NCCL-generated topology XML into the infragraph device/infrastructure model, alongside annotations capturing per-GPU and per-NIC metadata. It also generalizes the translator framework to support multiple tools and adds a --device_name option.

What's new

NCCL translator (src/infragraph/translators/nccl_translator.py — 604 lines)

  • NcclParser — parses NCCL XML topology into a Device/Infrastructure graph: CPUs and CPU fabric links, PCI bridge hierarchy, GPUs/XPUs, NICs, and NVLink/NVSwitch peer connections.
  • NcclHelper.generate_nccl_topology() — generates a fresh topology XML at runtime by loading libnccl.so/libcudart.so and calling ncclCommInitAll (used when no --input file is supplied).
  • get_annotations() — produces an InfraGraphService annotated with per-GPU metadata (rank, busid, device model, CPU affinity) and per-NIC metadata.
  • run_nccl_parser() — writes both the translated device file and an annotated_infragraph.json (infrastructure + annotations).

Translator framework changes

  • translator_handler.py — adds nccl to supported translators and threads a device_name argument through; requires --device_name for nccl.
  • __main__.py — new --device_name CLI option; default output renamed dev.yamldevice.yaml.

Tests & notebooks

  • test_nccl_translator.py — validates component counts, CPU/NVSwitch parsing, edges/links.
  • dgx_a100_nccl_topo.xml — DGX A100 mock fixture.
  • nccl_translate.py — example notebook.

@Amos-Rai-KEYS Amos-Rai-KEYS self-assigned this Apr 21, 2026
@Amos-Rai-KEYS Amos-Rai-KEYS marked this pull request as ready for review June 11, 2026 13:15
@ajbalogh ajbalogh self-requested a review June 16, 2026 14:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant