Skip to content

Add LLaDA 2.0 (inclusionAI) model support #2

Description

@iOptimizeThings

Background

LLaDA 2.0 (by inclusionAI, Ant Group) is the next generation of LLaDA, published after the gsai-ml LLaDA-1.5 release supported in v0.1. Despite the shared "LLaDA" name, LLaDA 2.0 uses a block-diffusion architecture (related to BD3-LMs) rather than the pure masked-diffusion contract of LLaDA-8B/1.5.

This is not a drop-in addition. Block diffusion changes:

  • Forward pass shape (block-wise attention masks)
  • The denoising loop (per-block iteration is structural, not a config flag like block_length)
  • Quality gates (different reference implementation to bit-match against)

Acceptance criteria

  • Vendor the inclusionAI reference loop into reference/
  • Add dlmserve/models/llada2.py loader
  • Implement block-diffusion denoise path in dlmserve/denoise_loop.py (or new file)
  • Reference-match gate at temperature=0 (token-exact vs the vendored reference)
  • All 4 LocalLeap quality gates pass (or document scope reduction)
  • LocalLeap calibration run for LLaDA 2.0 (scripts/calibrate_local_leap.py --model inclusionAI/LLaDA-2.0-...)
  • README + perf_log updates

References

Metadata

Metadata

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions