deep-learning-theory

A curated list of papers of interesting empirical study and insight on deep learning. Continually updating...

codebase for "A Theory of the Inductive Bias and Generalization of Kernel Regression and Wide Neural Networks"

Code for "Information-Theoretic Local Minima Characterization and Regularization"

PyTorch implementation for "Unveiling Induction Heads: Provable Training Dynamics and Feature Learning in Transformers", NeurIPS 2024

From-scratch NumPy implementations of core deep learning architectures (DNN, CNN, RNN). A journey into the first principles of AI.

Code to reproduce the paper "Deconstructing the Goldilocks Zone of Neural Network Initialization"

Resolving the Black Box Interpretability Problem by Proving the Mathematical Necessity of J-space via Hessian Rank Deficiency in Lean 4.

Proof 5: Why neural networks prefer low frequencies — a ReLU frequency tradeoff theorem. 3/3 theorems empirically verified.

Technical portfolio for Stanford CS229 (Summer 2020). First-principles approach to ML: Math-heavy derivations, NumPy-from-scratch implementations, and research project.

Proof 4: What neural networks cannot see — the Penrose aperiodicity barrier. 3/3 theorems empirically verified.

Labs from the Master MVA "Fondements théoriques du Deep Learning" class

LIFE 框架：基于动态同构的自指耗散智能模型 LIFE Framework: Life Is Full Effective — A Dynamic Isomorphic Dissipative Intelligence

Proof 7: Why the Cantor function (Devil's Staircase) is structurally hard for ReLU networks. All theorems verified.

Representation Group Flow — MSc Mathematics thesis (NISER, 2023): an effective statistical-field-theory of a deep MLP at initialization; how representations flow with depth.