[https://nvbugs/6793964][test] Unwaive test_row_linear[2-unbalanced] - #19569
Conversation
The cudaIpcMemHandle_t compat shim added in NVIDIA#19184 fixed the root cause. Signed-off-by: Lori Ren <lorir@nvidia.com>
|
/bot run |
|
Navigate logical layers of code changes, visualize relationships, and explore their blast radius. No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Repository: NVIDIA/TensorRT-LLM/.coderabbit.yaml Review profile: CHILL Plan: Enterprise Run ID: 📒 Files selected for processing (1)
💤 Files with no reviewable changes (1)
Included review availability: Your plan provides up to 12 included reviews per hour; 11 remain after this review. WalkthroughThe pull request removes one DGX H100 skip waiver and its associated NVBug reference for ChangesTest waiver update
Priority: ⬇️ Low Estimated code review effort: 1 (Trivial) | ~2 minutes Change: Other Suggested reviewers: Merge Risk: ⚪ Minimal · up to The re-enabled test no longer retains the identified H100 IPC compatibility deadlock trigger. No actionable merge-blocking risk remains. 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
|
PR_Github #75144 [ run ] triggered by Bot. Commit: |
|
Automatically added "ci: full pre-merge approved" because this PR has satisfied the required GitHub review approvals. Unresolved review conversations and other required checks remain independent merge requirements. |
|
PR_Github #75144 [ run ] completed with state
|
|
/bot run |
|
PR_Github #75190 [ run ] triggered by Bot. Commit: |
|
PR_Github #75190 [ run ] completed with state |
Dev Engineer Review
The change removes the DGX H100 waiver and NVBug reference for
test_row_linear[2-unbalanced]. No source-code changes are present. The compatibility fix must be available on the target branch before removing the waiver.QA Engineer Review
tests/integration/test_lists/waives.txtwas modified. Thetest_row_linear[2-unbalanced]waiver was removed. The author reports that bothtest_row_linearparametrizations pass on 2x H100. Coverage verdict:sufficient.Per-File QA Perspective
tests/integration/test_lists/waives.txt: Removes the DGX H100 skip entry fortest_row_linear[2-unbalanced]and its NVBug reference. This test-list-only change does not require atest-db/orqa/entry review.Description
test_row_linear[2-unbalanced]was waived on DGX_H100 after it hit the pytest timeout inpost-merge and was reported as
Test terminated unexpectedly. The cause was acuda-bindings drift:
cudaIpcMemHandle_tstopped exposing.reserved, soIpcMemoryraised on one rank while the other was already waiting in the allgather, deadlocking the
MPI pool until the stage was killed.
#19184 already fixed that on main with a
.reserved/getPtr()compat shim in_ipc_utils.py. This PR removes the waive that outlived it.Test Coverage
tests/unittest/_torch/multi_gpu/test_linear.py::test_row_linear— both parametrizationspass on 2x H100. The bug names only
[2-unbalanced], but that is an artifact of which casewas in flight when the pool deadlocked, so
[2-balanced]was verified as well.PR Checklist
Please review the following before submitting your PR:
api-compatibleorapi-breaking. Forapi-breaking, includeBREAKINGin the PR title.