feat: Support ZeRO-2 based on DistributedOptimizer#110

Open

Chamberlain0w0 wants to merge 7 commits into

Commits on Apr 20, 2026

Commits on May 13, 2026

fix: fix distopt behavior on gradient accumulation cases
Chamberlain0w0
committed

Commits on May 14, 2026

fix: fix some descriptions in comments
Chamberlain0w0
committed

Commits on May 20, 2026

fix: resolve comments
Chamberlain0w0
committed

Commits on May 21, 2026

fix: refactor zero-2 AccumulateGrad bypasser
Chamberlain0w0
committed