Skip to content

Bump xDiT stack - #32

Open
lauri9 wants to merge 18 commits into
mainfrom
feature/1001-bump-xdit
Open

Bump xDiT stack#32
lauri9 wants to merge 18 commits into
mainfrom
feature/1001-bump-xdit

Conversation

@lauri9

@lauri9 lauri9 commented Aug 31, 2026

Copy link
Copy Markdown
Contributor

Bump xDiT, transformers, diffusers, and DistVAE.

Closes #1001

Background

Bump xDiT to support MHAv4 dense and sparse backends.

Diffusers bump is needed to support new LTX2.5 API in xDiT. Transformers bump is needed to support the new diffusers version. DistVAE is bumped to meet xDiT requirements (now need DistVAE>=0.1.0).

Goals

  • Bump xDiT stack, adding support for MHAv4 attention kernels

Tasks

  • Bump SW stack
  • Verify performance and visual quality

Tests

CI run here.

Visually all outputs look good. Performance was tested compared to last AITER bump.

MI350X performance

model xDiT bump AITER 4ce24d1 Ratio
flux.usp 0.78 0.84 108.2%
flux.single_gpu 2.35 2.38 101.4%
flux.usp_2k 2.17 2.22 102.3%
flux.single_gpu_2k 11.88 12.01 101.0%
flux2.default 4.39 4.44 101.1%
flux2.quantgemm.gfx950 2.36 2.44 103.3%
flux2.quantgemm_sageattn.gfx950 2.49 2.57 103.3%
flux2.ti2i_2k 12.38 12.57 101.5%
flux2.t2i_1k 2.84 2.87 101.0%
flux2.t2i_2k 10.55 10.74 101.8%
flux2.t-multi-i2i_1k 5.87 5.95 101.4%
flux2.t-multi-i2i_2k 14.83 15.05 101.4%
flux2_klein.default 0.48 0.48 100.2%
flux_kontext.default 1.45 1.50 103.3%
hunyuanvideo.default 104.76 104.82 100.1%
hunyuanvideo.fp8attn 63.68 65.16 102.3%
hunyuanvideo.sageattn.gfx950 87.72 87.52 99.8%
hunyuanvideo_1_5.default 115.92 115.67 99.8%
ltx2_3.default 15.97 16.06 100.5%
qwen_image.default 4.75 4.81 101.3%
qwen_image_edit.default 14.81 15.07 101.8%
stablediffusion_3_5.default 0.82 0.81 99.3%
wan2_1.default 74.03 73.15 98.8%
wan2_2.default 75.89 75.21 99.1%
wan2_2.quantgemm.gfx950 67.31 67.25 99.9%
wan2_2.quantgemm_fp8attn.gfx950 39.09 40.50 103.6%
wan2_2.quantgemm_sageattn.gfx950 52.99 53.04 100.1%
wan2_2.quantgemm_spargeattn.gfx950 30.48 30.59 100.4%
z_image.default 2.56 2.64 103.2%

MI300X performance. These configs were rerun manually due to transient issues on CI: flux.usp, flux.single_gpu, flux2.quantgemm.gfx942, flux2.quantgemm_sageattn.gfx942, hunyuanvideo.default

model xDiT bump AITER 4ce24d1 Ratio
flux.usp 1.21 1.28 105.7%
flux.single_gpu 3.78 3.79 100.4%
flux.usp_2k 3.82 3.78 99.1%
flux.single_gpu_2k 20.09 20.10 100.0%
flux2.default 7.70 7.69 99.9%
flux2.quantgemm.gfx942 5.36 5.52 103.1%
flux2.quantgemm_sageattn.gfx942 5.63 5.73 101.8%
flux2.ti2i_2k 22.53 22.58 100.2%
flux2.t2i_1k 5.04 5.05 100.2%
flux2.t2i_2k 18.51 18.59 100.4%
flux2.t-multi-i2i_1k 9.86 10.00 101.4%
flux2.t-multi-i2i_2k 25.35 25.64 101.1%
flux2_klein.default 0.78 0.79 101.4%
flux_kontext.default 2.40 2.45 102.2%
hunyuanvideo.default 178.01 176.50 99.2%
hunyuanvideo.fp8attn 118.22 129.46 109.5%
hunyuanvideo.sageattn.gfx942 147.90 148.18 100.2%
hunyuanvideo_1_5.default 219.06 219.14 100.0%
ltx2_3.default 27.31 27.07 99.1%
qwen_image.default 8.03 8.19 101.9%
qwen_image_edit.default 22.07 22.72 103.0%
stablediffusion_3_5.default 1.26 1.25 99.3%
wan2_1.default 133.41 133.72 100.2%
wan2_2.default 133.20 133.32 100.1%
wan2_2.quantgemm.gfx942 120.52 121.03 100.4%
wan2_2.quantgemm_fp8attn.gfx942 76.97 88.04 114.4%
wan2_2.quantgemm_sageattn.gfx942 99.30 99.53 100.2%
wan2_2.quantgemm_spargeattn.gfx942 73.01 73.81 101.1%
z_image.default 4.37 4.53 103.7%

Other

n/a

lauri9 and others added 18 commits August 25, 2026 12:54
Track upstream AITER at the commit validated in the aiter-bump test branch.

Co-authored-by: Cursor <cursoragent@cursor.com>
Regenerate against 4fa508ef: drop stale Union import hunk, use Optional[...]
annotations for infer_schema, and retarget hunks to the new file layout.

Co-authored-by: Cursor <cursoragent@cursor.com>
Nested _validate_cu in aiter/ops/mha.py breaks torch.compile when annotated
with PEP 604 syntax. Apply aiter_mha_pep604_dynamo.patch in Dockerfile.ci;
remove it after PyTorch is bumped above 2.9.

Co-authored-by: Cursor <cursoragent@cursor.com>
commit 27ada84
Author: Mikko Lauri <mikko.lauri@amd.com>
Date:   Tue Aug 25 12:54:44 2026 +0000

    Work around PyTorch 2.9 Dynamo PEP 604 graph break in AITER MHA.

    Nested _validate_cu in aiter/ops/mha.py breaks torch.compile when annotated
    with PEP 604 syntax. Apply aiter_mha_pep604_dynamo.patch in Dockerfile.ci;
    remove it after PyTorch is bumped above 2.9.

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 2208972
Author: Mikko Lauri <mikko.lauri@amd.com>
Date:   Tue Aug 25 12:54:44 2026 +0000

    Refresh sparse FP8 ASM patch for bumped AITER.

    Regenerate against 4fa508ef: drop stale Union import hunk, use Optional[...]
    annotations for infer_schema, and retarget hunks to the new file layout.

    Co-authored-by: Cursor <cursoragent@cursor.com>

commit 95a1f02
Author: Mikko Lauri <mikko.lauri@amd.com>
Date:   Tue Aug 25 12:54:44 2026 +0000

    Bump AITER_COMMIT to 4fa508ef for CI builds.

    Track upstream AITER at the commit validated in the aiter-bump test branch.

    Co-authored-by: Cursor <cursoragent@cursor.com>
@lauri9
lauri9 marked this pull request as ready for review September 2, 2026 07:44
@lauri9
lauri9 requested review from a team as code owners September 2, 2026 07:44
@lauri9 lauri9 changed the title Feature/1001 bump xdit Bump xDiT stack Sep 2, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant