Skip to content
#

zero-optimizer

Star

Here is 1 public repository matching this topic...

xiyiji / cs336-assignment2-systems

Star 0

Stanford CS336 Assignment 2 (Systems), implemented from scratch: FlashAttention-2 in PyTorch + Triton (fwd + two-pass bwd), DDP variants, ZeRO-1 optimizer sharding, FSDP with mixed precision, profiling/benchmark harness, full writeup

cuda pytorch transformer triton cs336 performance-benchmarking gpu-kernels distributed-training mixed-precision distributed-data-parallel fsdp llm-training flash-attention activation-checkpointing zero-optimizer
  • Updated Sep 14, 2026
  • Python

Add this topic to your repo

To associate your repository with the zero-optimizer topic, visit your repo's landing page and select "manage topics."

Learn more