THU Machine Learning Sample Code Repository

清华大学机器学习课程 —— 样本代码仓库

简介

本仓库基于周志华《机器学习》(西瓜书) + Bishop PRML + Murphy PML 体系构建，提供 24 个可运行 Python 实现，涵盖 24 章、4 大知识领域。从经典 ML 到深度学习再到 LLM，全方位覆盖。适合：

• 清华机器学习课程的学习者
• 面试/科研/竞赛的系统性 ML 知识参考
• scikit-learn + NumPy 的动手实践

仓库结构 (4 部 24 章)

ml/
├── Hello.py                              # 仓库入口
│
╔══ Part I: 基础篇 (01-06) ═══════════════════════════════════════════╗
║                                                                      ║
├── 01_introduction/        ML概论 + Bias-Variance + 过拟合/欠拟合
├── 02_linear_regression/   OLS / Ridge(L2) / Lasso(L1) / 梯度下降
├── 03_logistic_regression/ Sigmoid / Softmax / 决策边界 / Numpy实现
├── 04_model_selection/     K-Fold CV / AUC-ROC / F1 / Confusion Matrix
├── 05_feature_engineering/ 标准化 / OneHot / 多项式 / SelectKBest
├── 06_probability/         朴素贝叶斯 / MLE vs MAP / GaussianNB 实现
║                                                                      ║
╚══════════════════════════════════════════════════════════════════════╝

╔══ Part II: 经典算法 (07-12) ════════════════════════════════════════╗
║                                                                      ║
├── 07_svm/                 SVM核技巧 / RBF / 支持向量 / 软间隔
├── 08_decision_trees/      ID3/C4.5/CART / Gini/Entropy / 剪枝可视化
├── 09_ensemble/            RF / AdaBoost / GBDT / Bagging vs Boosting
├── 10_knn_clustering/      K-Means vs DBSCAN / K-Means++ / 噪声检测
├── 11_dimensionality/      PCA(线性) vs t-SNE(非线性) / 可视化对比
├── 12_anomaly/             IsolationForest / LOF / OneClassSVM
║                                                                      ║
╚══════════════════════════════════════════════════════════════════════╝

╔══ Part III: 深度学习 (13-18) ════════════════════════════════════════╗
║                                                                      ║
├── 13_neural_basics/       MLP + 反向传播 (纯NumPy实现!)
├── 14_optimization/        SGD/Momentum/RMSprop/Adam + 学习率调度
├── 15_cnn/                 Conv2D手工演示 + LeNet/AlexNet/ResNet对比
├── 16_rnn_lstm/            LSTM三门机制 / GRU / 梯度消失解决
├── 17_transformer/         Self-Attention (纯NumPy) / Multi-Head
├── 18_transfer_learning/   微调策略 / LoRA低秩分解 (参数效率)
║                                                                      ║
╚══════════════════════════════════════════════════════════════════════╝

╔══ Part IV: 前沿主题 (19-24) ════════════════════════════════════════╗
║                                                                      ║
├── 19_generative/          GAN/VAE/Diffusion 三大生成模型对比
├── 20_reinforcement/       Q-Learning / Policy Gradient / Actor-Critic
├── 21_graph_learning/      GCN消息传递 / GraphSAGE / GAT注意力
├── 22_mlops/               ML生命周期 / Drift / CI/CD / 工具栈
├── 23_fairness/            公平性指标 / SHAP / LIME / 可解释性
├── 24_llm/                 GPT/ScalingLaws/RLHF/RAG/COT/Prompting
║                                                                      ║
╚══════════════════════════════════════════════════════════════════════╝

快速开始

# 经典ML
python3 02_linear_regression/linear_regression.py
python3 07_svm/svm.py

# 深度学习
python3 13_neural_basics/mlp_backprop.py      # 反向传播 NumPy!
python3 17_transformer/transformer.py         # Self-Attention NumPy!

# 前沿
python3 24_llm/llm_concepts.py

各章核心速查

Part	章节	核心	关键概念
I	概论	Bias-Variance, 过拟合	Train/Val/Test split
I	线性回归	OLS, L1/L2 正则化	Normal Equation, SGD
I	逻辑回归	Sigmoid, Softmax	Cross-Entropy, 决策边界
I	模型选择	K-Fold CV, AUC-ROC	Precision/Recall/F1
I	特征工程	StandardScaler, OneHot	Polynomial, SelectKBest
I	概率	Naive Bayes, MLE/MAP	GaussianNB 实现
II	SVM	核技巧, 最大间隔	Support Vectors, RBF
II	决策树	ID3/C4.5/CART	Gini, Entropy, Pruning
II	集成	RF, AdaBoost, GBDT	Bagging vs Boosting
II	KNN/聚类	K-Means, DBSCAN	Elbow Method, ε-neighborhood
II	降维	PCA(线性), t-SNE	Explained Variance
II	异常检测	IsolationForest, LOF	Contamination, Novelty
III	神经网络	MLP + Backprop (NumPy)	ReLU, Softmax, Chain Rule
III	优化器	SGD/Momentum/Adam	β₁/β₂, Cosine Annealing
III	CNN	Conv2D, Pooling	ResNet残差, VGG, Inception
III	RNN/LSTM	LSTM三门前向	Forget/Input/Output Gate
III	Transformer	Self-Attention (NumPy)	QKV, Multi-Head, Positional
III	迁移学习	Fine-Tune, LoRA	Low-Rank Adaptation
IV	生成模型	GAN/VAE/Diffusion	Mode Collapse, DDPM
IV	强化学习	Q-Learning, Policy Gradient	Bellman Eq, ε-greedy
IV	图学习	GCN/GAT 消息传递	Adjacency Normalization
IV	MLOps	Drift, CI/CD, Serving	Data/Concept Drift
IV	公平性	Demographic Parity, SHAP	Equal Opportunity
IV	LLM	GPT/RLHF/RAG/COT	Scaling Laws, DPO, Prompt

参考资料

• 西瓜书: 周志华,《机器学习》
• PRML: Bishop, Pattern Recognition and Machine Learning
• PML: Murphy, Probabilistic Machine Learning
• D2L: Zhang et al., Dive into Deep Learning (d2l.ai)

构建说明

本仓库由以下 AI 协作完成：

• 代码架构与实现: Claude (Anthropic)
• 推理引擎: DeepSeek V4 Pro (1M 上下文)

许可

MIT License