Advanced Mathematical Optimization & Deep Learning Optimizers from scratch. Covers KKT duality, L-BFGS, proximal methods (ADMM, FISTA), stochastic algorithms (SVRG, Lion), and cutting-edge deep learning optimizers like K-FAC, Shampoo, Sophia, SAM, and Muon. Bridging strict convex calculus with large-scale Transformer training.
python admm convex-optimization distributed-optimization l-bfgs stochastic-optimization mathematical-optimization deep-learning-optimizers proximal-gradient transformer-training
-
Updated
May 27, 2026 - Jupyter Notebook