Skip to content

[codex] Sync H100 runner capacity with Slurm / 同步 H100 运行器与 Slurm 容量 - #2739

Draft
cquil11 wants to merge 1 commit into
mainfrom
codex/sync-h100-runner-capacity
Draft

[codex] Sync H100 runner capacity with Slurm / 同步 H100 运行器与 Slurm 容量#2739
cquil11 wants to merge 1 commit into
mainfrom
codex/sync-h100-runner-capacity

Conversation

@cquil11

@cquil11 cquil11 commented Aug 26, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Remove h100-dgxc-slurm_13, _14, and _15 from the generic h100 and cluster:h100-dgxc runner inventories.
  • Preserve _16 through _19, which carry the specialized multinode and disaggregated-serving labels.
  • Match the checked-in inventory to the live H100 Slurm partition and GitHub Actions runner pool: 17 nodes and 17 runners.

The three idle runner registrations were also removed from the live GitHub Actions runner registry before opening this PR.

Why

The provider permanently reduced the original 20-node H100 allocation. The live Slurm partition now exposes 17 nodes, while GitHub still advertised 20 runner capacity tokens, allowing the weighted scheduler to over-admit by three nodes.

Validation

  • Verified hpc-gpu-1 reports 17 Slurm nodes.
  • Verified GitHub reports 17 online, idle cluster:h100-dgxc runners after the sync.
  • python -m pytest utils/matrix_logic/test_validation.py utils/matrix_logic/test_generate_sweep_configs.py -q — 232 passed
  • git diff --check

中文说明

  • 从通用 h100cluster:h100-dgxc 运行器清单中移除 h100-dgxc-slurm_13_14_15
  • 保留带有多节点和分离式推理专用标签的 _16_19
  • 使仓库中的运行器清单与实时 H100 Slurm 分区和 GitHub Actions 运行器池保持一致:17 个节点、17 个运行器。

创建本 PR 前,已从 GitHub Actions 实时运行器注册表中删除这三个空闲运行器。

原因

云服务商永久缩减了原有的 20 节点 H100 配额。实时 Slurm 分区目前提供 17 个节点,但 GitHub 仍发布 20 个运行器容量令牌,导致加权调度器可能超额准入三个节点。

验证

  • 已确认 hpc-gpu-1 报告 17 个 Slurm 节点。
  • 同步后已确认 GitHub 报告 17 个在线且空闲的 cluster:h100-dgxc 运行器。
  • python -m pytest utils/matrix_logic/test_validation.py utils/matrix_logic/test_generate_sweep_configs.py -q — 232 项测试通过
  • git diff --check

Remove three generic H100 runner entries after the provider permanently reduced the cluster and the live Slurm partition settled at 17 nodes. Preserve the specialized multinode and disaggregated runners.

中文:云服务商永久缩减 H100 集群后,实时 Slurm 分区现为 17 个节点,因此移除三个通用 H100 运行器条目,并保留多节点和分离式推理专用运行器。
@cquil11

cquil11 commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

Closing because the project scope is now limited to porting active, nondeprecated AMD multi-node configurations to srt-slurm. This PR is outside that scope.

@cquil11

cquil11 commented Aug 27, 2026

Copy link
Copy Markdown
Collaborator Author

Reopening: this PR predates the recent out-of-scope configuration-porting detour and was closed by mistake during an overly broad cleanup. Its prior state is being restored.

@cquil11 cquil11 reopened this Aug 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Development

Successfully merging this pull request may close these issues.

1 participant