Skip to content

Matrix-Game-3: --fa_version 0 (SDPA) is not implemented; attention() asserts a flash-attn install instead of falling back #86

Description

@stayinalive181

Matrix-Game-3/wan/modules/attention.py no longer has the SDPA fallback that the original Wan attention module ships. The non-FA3 branch of flash_attention() is:

else:
    assert FLASH_ATTN_2_AVAILABLE or FLASH_ATTN_3_AVAILABLE

Two consequences:

  • Without a flash-attn install, every attention call dies on that assertion, so the model cannot run at all on a machine where flash-attn does not build (the CLI's --fa_version 0 suggests SDPA is meant to be selectable).
  • With flash-attn 2 installed, --fa_version 0 still runs the flash-attn 2 kernel: version=0 is neither 3 nor handled anywhere, so it falls through to the same branch. --fa_version 3 with only FA2 installed does degrade with a warning, which is the configuration that worked for us on an H100.

Suggested fix: restore the torch.nn.functional.scaled_dot_product_attention path for version == 0 (and as the fallback when neither flash-attn build is importable), with the warning the upstream Wan module prints. Happy to open a PR if that is the intended behaviour.

Related, still open as of today: the multi-GPU path passes fa_version= to attention() whose parameter is named version (#80, fix in #61), so --ulysses_size > 1 raises TypeError on a clean checkout.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions