| Network | First Usage |
|---|---|
| LeNet-5 | Hand written digit classification |
| AlexNet | ImageNet Dataset |
| VGG-16 | ImageNet Dataset |
LeNet-5 is a very simple network - By modern standards -. It only has 7 layers;
- among which there are 3 convolutional layers (C1, C3 and C5)
- 2 sub-sampling (pooling) layers (S2 and S4)
- 1 fully connected layer (F6)
- Output layer
- Too similar to LeNet-5
- It has more filters per layer
- It uses ReLU instead of tanh
- SGD with momentum
- Uses dropout instead of regularaization
- It is painfully slow to train (It has 138 million parameters 🙄)