Pytorch 实践（刘二大人）8：多分类问题 - PyTorch - 机器学习

数学思路

数据表征

将$28 \times 28$像素的灰度图像展开为784维向量，$x ∈ ℝ^{784}$，像素值归一化至[0, 1]区间[^1]。标签采用one-hot编码，构建目标函数$y ∈ {0,1}^{10}$[^2]

前向传播

网络维度变化$784\Longrightarrow 512\Longrightarrow 256\Longrightarrow 128\Longrightarrow 64\Longrightarrow 10$

第一层：$z_1=W_1x+b_1,a_1=ReLU(z_1)=max(0,z_1)$[^3]
第二层：$z_2=W_2a_1+b_2,a_2=ReLU(z_2)$
第三层：$z_3=W_3a_2+b_3,a_3=ReLU(z_3)$
第四层：$z_4=W_4a_3+b_4,a_4=ReLU(z_4)$
输出层：$z_5=W_5a_4+b_5,\hat{y} = \text{softmax}(z_5) = \frac{e^{z_5}}{\sum_{i=1}^{10} e^{z_5^{(i)}}}$

反向传播

$$
\frac{\partial L}{\partial W_1} = \underbrace{\frac{\partial L}{\partial \hat{y}} \frac{\partial \hat{y}}{\partial z_5}}_{\delta_5} \cdot \frac{\partial z_5}{\partial a_4} \cdot \frac{\partial a_4}{\partial z_4} \cdot \frac{\partial z_4}{\partial a_3} \cdot \frac{\partial a_3}{\partial z_3} \cdot \frac{\partial z_3}{\partial a_2} \cdot \frac{\partial a_2}{\partial z_2} \cdot \frac{\partial z_2}{\partial a_1} \cdot \frac{\partial a_1}{\partial z_1} \cdot \frac{\partial z_1}{\partial W_1}
$$

参数优化

对全部参数${W_i, b_i}^{5}{i=1}$，使用带动量的SGD更新：
$$
\begin{aligned}
v{W_i} &:= \gamma v_{W_i} + \eta \frac{\partial L}{\partial W_i} \
W_i &:= W_i - v_{W_i}
\end{aligned}
$$

评估指标

$$
Accuracy = \frac{(Σ_{n=1}^N I(argmax(ŷ^{(n)}) = argmax(y^{(n)})))}{N}
$$

代码思路

库导入与数据预处理

import torch
from torchvision import transforms, datasets
from torch.utils.data import DataLoader
import torch.nn.functional as F
import torch.optim as optim

batch_size = 64
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])

ToTensor()将PIL图像转换为张量，Normalize进行归一化
batch_size=64：每批处理64张图片

数据加载

train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
test_dataset = datasets.MNIST(root='../dataset/mnist', train=False, download=True, transform=transform)
test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size)

shuffle=True表示训练数据打乱，以增强泛化性

神经网络模型

class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.l1 = torch.nn.Linear(784, 512)
        self.l2 = torch.nn.Linear(512, 256)
        self.l3 = torch.nn.Linear(256, 128)
        self.l4 = torch.nn.Linear(128, 64)
        self.l5 = torch.nn.Linear(64, 10)

    def forward(self, x):
        x = x.view(-1, 784)  
        x = F.relu(self.l1(x))
        x = F.relu(self.l2(x))
        x = F.relu(self.l3(x))
        x = F.relu(self.l4(x))
        return self.l5(x)

结构：5层全连接网络（784→512→256→128→64→10），逐步压缩特征。
x = x.view(-1, 784)表示将张量转换为一维向量。-1表示自动推导维度

损失函数与优化器

1 2	criterion = torch.nn.CrossEntropyLoss() optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)

CrossEntroplLoss用于定义交叉熵损失函数，适用于多分类任务，其中每个样本的标签是单个类别

训练函数

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0):
        inputs, target = data
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if batch_idx % 300 == 299:
            print(f'[{epoch+1}, {batch_idx+1}] loss: {running_loss/300:.3f}')
            running_loss = 0.0

清空梯度：optimizer.zero_grad()
前向传播：outputs = model(inputs)
计算损失：loss = criterion(outputs, target)
反向传播：loss.backward()
参数更新：optimizer.step()

测试函数

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0):
        inputs, target = data
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if batch_idx % 300 == 299:
            print(f'[{epoch+1}, {batch_idx+1}] loss: {running_loss/300:.3f}')
            running_loss = 0.0

评估模式：torch.no_grad()禁用梯度计算，节省内存
预测计算：取输出最大值的索引作为预测类别，统计正确率

代码实现

import torch
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader
import torch.nn.functional as F
import torch.optim as optim

batch_size = 64
transform = transforms.Compose([
    transforms.ToTensor(),
    transforms.Normalize((0.1307,), (0.3081,))
])

train_dataset = datasets.MNIST(root='../dataset/mnist/', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size)
test_dataset = datasets.MNIST(root='../dataset/mnist', train=False, download=True,transform=transform)
test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size)

class Net(torch.nn.Module):
    def __init__(self):
        super(Net, self).__init__()
        self.l1 = torch.nn.Linear(784, 512)
        self.l2 = torch.nn.Linear(512, 256)
        self.l3 = torch.nn.Linear(256, 128)
        self.l4 = torch.nn.Linear(128, 64)
        self.l5 = torch.nn.Linear(64, 10)

    def forward(self, x):
        x = x.view(-1, 784)
        x = F.relu(self.l1(x))
        x = F.relu(self.l2(x))
        x = F.relu(self.l3(x))
        x = F.relu(self.l4(x))
        return self.l5(x)

model = Net()

criterion = torch.nn.CrossEntropyLoss()
optimizer = optim.SGD(model.parameters(), lr=0.01, momentum=0.5)

def train(epoch):
    running_loss = 0.0
    for batch_idx, data in enumerate(train_loader, 0):
        inputs, target = data
        optimizer.zero_grad()
        outputs = model(inputs)
        loss = criterion(outputs, target)
        loss.backward()
        optimizer.step()

        running_loss += loss.item()
        if batch_idx % 300 == 299:
            print('[%d, %5d] loss: %.3f' % (epoch+1, batch_idx + 1, running_loss / 300))
            running_loss = 0.0

def test():
    correct = 0
    total = 0
    with torch.no_grad():
        for data in test_loader:
            images, labels = data
            outputs = model(images)
            _, predicted = torch.max(outputs.data, dim=1)
            total += labels.size(0)
            correct += (predicted == labels).sum().item()
    print('accuracy on test set: %d %% ' % (100 * correct / total))

if __name__ == '__main__':
    for epoch in range(10):
        train(epoch)
        test()

[^1]: MNIST图像是灰度图，每个像素值用8位无符号整数，取值范围[0, 255]。通过线性缩放将像素值映射到[0, 1]区间：$x_{归一化}=\frac{x_{原始}}{255}$
[^2]: One-Hot编码是一种将类别标签（如数字0-9）转换为二进制向量的方法。在MNIST中，假设某张图片的数字是3，One-Hot编码后：[0, 0, 0, 1, 0, 0, 0, 0, 0, 0]。y ∈ {0,1}¹⁰ 表示这是一个长度为10的向量，每个元素只能是0或1，且有且仅有一个1。
[^3]: ReLU函数：$f(x)=max(0, x)$

机器学习代码实践