Skip to content

How hard is it to add DFlash / DFlash 2 to transformers, and is it even worth it? #162

Description

@ilperev

Just curious. I have an almost ready PR for DSpark there myself (was doing it to speed up a model on ZeroGPU Space), but then I realized that there is essentially no full official DFlash support there, and the Qwen 3.5 4B model with DFlash doesn't run out of the box—you need to rewrite parts of the library and implement the missing pieces.

Why didn't you add it there initially when you introduced DFlash? Is it just not worth the effort? Personally, my motivation was running it on ZeroGPU Space, but once I got llama-cpp-python running there, I realized, honestly, why even bother?

Still, from a project perspective, having at least some support in Transformers just for the sake of the docs on Hugging Face seems to make sense... I guess. I don't know. In my view, it's worth having support there simply because Transformers is the foundational baseline for all architectures in general. After all, it's not primarily designed for high-performance inference anyway, but rather serves as the central hub uniting all model architectures in a single library.

That is mainly why I opened this issue—I'm genuinely curious to hear your thoughts on this, coming from people whose expertise in this field is leagues ahead of mine.

By the way, I'll soon be updating my Spaces to feature your drafters, and I will definitely link back to your project!

Thank you for what you do and for doing it for the community! Also thank you for immediately adding support to vLLM, SGLang, and llama.cpp, and for publishing DFlash2 for Qwen 3.8 27B and Muse Glimmer 30B right away.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions