[issue2] GPU 内通信方式最佳通信区间阈值拟合器(One-Shot/P2P/NVLS) - #28
Open
shsaihdsaiudh wants to merge 1 commit into
Open
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
关联 Issue
#2
提交内容
src/code/issue2/threshold_model_2026.py:把 One-Shot / P2P Two-Shot / Multimem(NVLS) 三类机内 AllReduce 的标定数据拟合成"按消息大小自动选择算法"的切换阈值模型。输入为标定 CSV:
处理流程:
--max-sm-occupancy-pct:在 SM 占用预算约束下选最快项,而不是无约束选最快项。这一项对应 issue 中"NVLS 规约在交换芯片上完成、SM 几乎空闲"的取舍——当 AllReduce 需要与计算 overlap 时,纯时延最优未必是整体最优。注意:若某个消息区间没有任何候选满足 SM 预算,该点会回退为无约束选最快(可能超出预算),以保全区间完整性。验证
python -m pytest src/code/issue2/test_threshold_model_2026.py -q # 2 passed python src/code/issue2/threshold_model_2026.py --synthetic-demo python src/code/issue2/threshold_model_2026.py --synthetic-demo --max-sm-occupancy-pct 10合成数据可复现出预期的三段结构(小消息 One-Shot → 中消息 P2P Two-Shot → 大消息 Multimem/NVLS)。加上
--max-sm-occupancy-pct 10后,由于合成数据中仅 NVLS 的 SM 占用(4%)满足预算,三个区间塌缩为单一 NVLS 区间(全消息范围选 NVLS),体现"约束改变选择"的效果。结果写入src/code/issue2/results-2026/。当前边界
本 PR 提交的是阈值模型与拟合器本身,即验收要求 2 中"切换阈值模型(含决策边界)"的可运行实现。
results-2026/synthetic_measurements.csv是合成标定数据,文件内已显式标注,仅用于验证拟合逻辑,不构成任何性能结论。验收要求 1(三类实现在各 message size 下的实测通信时间与 SM/Warp 占用报告)和验收要求 2 的收益验证,需要在含 NVSwitch 的单机多卡环境上用目标实现重新采集后回填。提交者当前为单卡环境,无 NVSwitch,无法采集真实数据。工具的输入格式已按上述 CSV 固定,采集到真机数据后可直接替换
--calibration重跑。