Bump AITER to 4fa508ef - #31
Draft
lauri9 wants to merge 5 commits into
Draft
Conversation
Track upstream AITER at the commit validated in the aiter-bump test branch. Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerate against 4fa508ef: drop stale Union import hunk, use Optional[...] annotations for infer_schema, and retarget hunks to the new file layout. Co-authored-by: Cursor <cursoragent@cursor.com>
Nested _validate_cu in aiter/ops/mha.py breaks torch.compile when annotated with PEP 604 syntax. Apply aiter_mha_pep604_dynamo.patch in Dockerfile.ci; remove it after PyTorch is bumped above 2.9. Co-authored-by: Cursor <cursoragent@cursor.com>
lauri9
marked this pull request as ready for review
August 27, 2026 08:57
lauri9
marked this pull request as draft
August 28, 2026 10:17
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bump AITER to 4fa508ef and refresh CI patches for PyTorch 2.9 compatibility.
Closes #979
Background
Part of plan to remove triton attention.
Upstream AITER is bumped forward to
4fa508ef, which brings in the MHAv4 API and changes file layout and type annotations.On the current PyTorch 2.9 stack, AITER's nested
_validate_cuhelper inaiter/ops/mha.pyuses PEP 604 (X | None) syntax that causes a Dynamo graph break undertorch.compile. This is resolved in the PR with a temporary patch.Goals
torch.compilecompatibility for AITER MHA on PyTorch 2.9 with temporary patchTasks
AITER_COMMITto4fa508ef2935110ff99adf2743ea93807dbd9c67indocker/Dockerfile.ci.patches/aiter_fp8_sparse_asm_kernel.patchfor the new AITER layout (retargeted hunks, updatedmutates_argstyping).patches/aiter_mha_pep604_dynamo.patchand apply it in the CI Dockerfile to replace PEP 604 annotations withOptional[...]in_validate_cu.Tests
torch.compilepaths that exercise AITER MHA on PyTorch 2.9.Performance on MI350X - no significant changes
Performance on MI300X - no significant changes
Other
docker/Dockerfile.ci.