1. LeNet

两层卷积+两层池化+两层全连接

卷积就是让每一层shape不断压缩变小【高宽减少】，通道数增多，把特征信息放到不同的通道里面。每一个通道认为是一个模式。然后再做全连接的输入。

2. 代码

import torch
from torch import nn
from d2l import torch as d2l

class Reshape(torch.nn.Module):
    def forward(self, x):
        return x.view(-1, 1, 28, 28)

# 去掉了原始模型最后一层的高斯激活
net = nn.Sequential(Reshape(), 
    nn.Conv2d(1, 6, kernel_size=5, padding=2), nn.Sigmoid(), 
    nn.AvgPool2d(kernel_size=2, stride=2), 
    nn.Conv2d(6, 16, kernel_size=5), nn.Sigmoid(), 
    nn.AvgPool2d(kernel_size=2, stride=2), 
    nn.Flatten(), 
    nn.Linear(16*5*5, 120), nn.Sigmoid(), 
    nn.Linear(120, 84), nn.Sigmoid(), 
    nn.Linear(84, 10))

X = torch.rand(size=(1, 1, 28, 28), dtype=torch.float32)
for layer in net:
    X = layer(X)
    print(layer.__class__.__name__,'output shape: \t',X.shape)

# 第一个卷积层使用2个像素的填充，来补偿卷积核5*5导致的特征减少 输出的特征维度是没有减少的

Conv2d output shape: 	 torch.Size([1, 6, 28, 28])
Sigmoid output shape: 	 torch.Size([1, 6, 28, 28])
AvgPool2d output shape: 	 torch.Size([1, 6, 14, 14])
Conv2d output shape: 	 torch.Size([1, 16, 10, 10])
Sigmoid output shape: 	 torch.Size([1, 16, 10, 10])
AvgPool2d output shape: 	 torch.Size([1, 16, 5, 5])
Flatten output shape: 	 torch.Size([1, 400])
Linear output shape: 	 torch.Size([1, 120])
Sigmoid output shape: 	 torch.Size([1, 120])
Linear output shape: 	 torch.Size([1, 84])
Sigmoid output shape: 	 torch.Size([1, 84])
Linear output shape: 	 torch.Size([1, 10])
.....
loss 0.470, train acc 0.823, test acc 0.817
35740.9 examples/sec on cuda:0

3. QA

cnn explainer 看到每一个通道学习到的是什么东西。

https://poloclub.github.io/cnn-explainer/