深度学习每周学习总结J6（ResNeXt-50 算法实战与解析 - 猴痘识别）

news2024/11/24 9:35:03
  • 🍨 本文为🔗365天深度学习训练营 中的学习记录博客
  • 🍖 原作者：K同学啊 | 接辅导、项目定制

目录

      • 0. 总结
        • ResNeXt基本介绍
      • 1. 设置GPU
      • 2. 导入数据及处理部分
      • 3. 划分数据集
      • 4. 模型构建部分
      • 5. 设置超参数：定义损失函数，学习率，以及根据学习率定义优化器等
      • 6. 训练函数
      • 7. 测试函数
      • 8. 正式训练
      • 9. 结果可视化
      • 10. 模型的保存
      • 11.使用训练好的模型进行预测

0. 总结

数据导入及处理部分：本次数据导入没有使用torchvision自带的数据集，需要将原始数据进行处理包括数据导入，查看数据分类情况，定义transforms，进行数据类型转换等操作。

划分数据集：划定训练集测试集后，再使用torch.utils.data中的DataLoader()分别加载上一步处理好的训练及测试数据，查看批处理维度.

模型构建部分：ResNeXt-50

设置超参数：在这之前需要定义损失函数，学习率（动态学习率），以及根据学习率定义优化器（例如SGD随机梯度下降），用来在训练中更新参数，最小化损失函数。

定义训练函数：函数的传入的参数有四个，分别是设置好的DataLoader(),定义好的模型，损失函数，优化器。函数内部初始化损失准确率为0，接着开始循环，使用DataLoader()获取一个批次的数据，对这个批次的数据带入模型得到预测值，然后使用损失函数计算得到损失值。接下来就是进行反向传播以及使用优化器优化参数，梯度清零放在反向传播之前或者是使用优化器优化之后都是可以的，一般是默认放在反向传播之前。

定义测试函数：函数传入的参数相比训练函数少了优化器，只需传入设置好的DataLoader(),定义好的模型，损失函数。此外除了处理批次数据时无需再设置梯度清零、返向传播以及优化器优化参数，其余部分均和训练函数保持一致。

训练过程：定义训练次数，有几次就使用整个数据集进行几次训练，初始化四个空list分别存储每次训练及测试的准确率及损失。使用model.train()开启训练模式，调用训练函数得到准确率及损失。使用model.eval()将模型设置为评估模式，调用测试函数得到准确率及损失。接着就是将得到的训练及测试的准确率及损失存储到相应list中并合并打印出来，得到每一次整体训练后的准确率及损失。

结果可视化

模型的保存，调取及使用。在PyTorch中，通常使用 torch.save(model.state_dict(), ‘model.pth’) 保存模型的参数，使用 model.load_state_dict(torch.load(‘model.pth’)) 加载参数。

需要改进优化的地方：确保模型和数据的一致性，都存到GPU或者CPU;注意numclasses不要直接用默认的1000，需要根据实际数据集改进；实例化模型也要注意numclasses这个参数；此外注意测试模型需要用（3,224,224）3表示通道数，这和tensorflow定义的顺序是不用的（224,224,3），做代码转换时需要注意。

关于调优（十分重要）：本次将测试集准确率提升到了94.87%（随机种子设置为42）
1：使用多卡不一定比单卡效果好，需要继续调优
2：本次微调参数主要调整了两点一是初始学习率从1e-4 增大为了3e-4;其次是原来图片预处理只加入了随机水平翻转，本次加入了小角度的随机翻转，随机缩放剪裁，光照变化等，发现有更好的效果。测试集准确率有了很大的提升。从训练后的准确率图像也可以看到，训练准确率和测试准确率很接近甚至能够超过。之前没有做这个改进之前，都是训练准确率远大于测试准确率。

关键代码示例：

import torchvision.transforms as transforms
 
# 定义猴痘识别的 transforms
train_transforms = transforms.Compose([
    transforms.Resize([224, 224]),            # 统一图片尺寸
    transforms.RandomHorizontalFlip(p=0.5),  # 随机水平翻转
    transforms.RandomRotation(degrees=15),   # 小角度随机旋转
    transforms.RandomResizedCrop(size=224, scale=(0.8, 1.2)),  # 随机缩放裁剪
    transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.1),  # 光照变化
    transforms.ToTensor(),                   # 转换为 Tensor 格式
    transforms.Normalize(                    # 标准化
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]
    )
])
ResNeXt基本介绍

ResNeXt是一个基于卷积神经网络（CNN）的深度学习模型，最早由Facebook AI Research（FAIR）团队在2017年提出。它是ResNet（残差网络）的一个变种，通过引入"cardinality"（基数）这一概念，进一步提升了模型的性能。

  • ResNeXt的关键创新

  1. Cardinality（基数）：

    • 传统的卷积神经网络通常通过增加层数或者每层的通道数（宽度）来提升模型的表现，但这可能导致计算和内存开销的大幅增加。ResNeXt通过引入"cardinality"（基数）的概念，指的是每个残差块中并行的路径数量。通过增加并行路径的数量，ResNeXt能够在不显著增加计算量的情况下提升网络的表达能力。
    • 具体来说，ResNeXt在每个残差块中使用了多个分支，每个分支都是相同的网络结构。通过调整分支的数量（即基数）来提高网络的表达能力。

  2. 分组卷积（Group Convolution）：

    • ResNeXt使用了分组卷积，这使得计算量更加高效。分组卷积通过将输入通道分为若干组进行卷积操作，减少了计算量和内存开销。

  3. 结构设计：

    • 在ResNeXt中，残差块的结构是通过多路径结构来增强模型的表现。每个路径相当于一个独立的卷积操作，最终将它们的输出进行合并。这种方法与传统的单路径ResNet不同。
  • 与传统神经网络的对比

  1. 传统CNN（例如AlexNet、VGG等）：

    • 传统的CNN网络通过加深网络层数或增加每一层的神经元来增强网络的表达能力，但这种做法面临梯度消失、过拟合等问题。因此，随着层数的增加，传统CNN的训练变得越来越困难。

  2. ResNet与ResNeXt的优势：

    • ResNet通过残差连接解决了深度神经网络训练时的梯度消失问题，使得网络可以很深而不容易退化。ResNeXt继承了ResNet的优点，但通过引入“基数”来进一步提升性能。相比于简单地增加网络深度或宽度，ResNeXt能够更高效地利用网络容量。
    • ResNeXt通过分支结构使得每个残差块更具表达能力，相较于传统网络和单路径的ResNet，ResNeXt在相同的计算量下通常能够得到更好的效果。

下图是ResNet（左）与ResNeXt（右）block的差异。在ResNet中，输入的具有256个通道的特征经过1×1卷积压缩4倍到64个通道，之后3×3的卷积核用于处理特征，经1×1卷积扩大通道数与原特征残差连接后输出。ResNeXt也是相同的处理策略，但在ResNeXt中，输入的具有256个通道的特征被分为32个组，每组被压缩64倍到4个通道后进行处理。32个组相加后与原特征残差连接后输出。这里cardinatity指的是一个block中所具有的相同分支的数目。

import torch
import torch.nn as nn
import torchvision
from torchvision import datasets,transforms
from torch.utils.data import DataLoader
import torchvision.models as models
import torch.nn.functional as F
from collections import OrderedDict 


import os,PIL,pathlib
import matplotlib.pyplot as plt
import warnings

warnings.filterwarnings('ignore') # 忽略警告信息

plt.rcParams['font.sans-serif'] = ['SimHei'] # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False   # 用来正常显示负号
plt.rcParams['figure.dpi'] = 100 # 分辨率

1. 设置GPU

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
device

device(type='cuda')

2. 导入数据及处理部分

# 获取数据分布情况
path_dir = './data/mpox_recognize/'
path_dir = pathlib.Path(path_dir)

paths = list(path_dir.glob('*'))
# classNames = [str(path).split("\\")[-1] for path in paths] # ['Bananaquit', 'Black Skimmer', 'Black Throated Bushtiti', 'Cockatoo']
classNames = [path.parts[-1] for path in paths]
classNames

['Monkeypox', 'Others']

# 定义transforms 并处理数据
# train_transforms = transforms.Compose([
#     transforms.Resize([224,224]),      # 将输入图片resize成统一尺寸
#     transforms.RandomHorizontalFlip(), # 随机水平翻转
#     transforms.ToTensor(),             # 将PIL Image 或 numpy.ndarray 装换为tensor,并归一化到[0,1]之间
#     transforms.Normalize(              # 标准化处理 --> 转换为标准正太分布（高斯分布），使模型更容易收敛
#         mean = [0.485,0.456,0.406],    # 其中 mean=[0.485,0.456,0.406]与std=[0.229,0.224,0.225] 从数据集中随机抽样计算得到的。
#         std = [0.229,0.224,0.225]
#     )
# ])

# 定义猴痘识别的 transforms 并处理数据
train_transforms = transforms.Compose([
    transforms.Resize([224, 224]),            # 统一图片尺寸
    transforms.RandomHorizontalFlip(p=0.5),  # 随机水平翻转
    transforms.RandomRotation(degrees=15),   # 小角度随机旋转
    transforms.RandomResizedCrop(size=224, scale=(0.8, 1.2)),  # 随机缩放裁剪
    transforms.ColorJitter(brightness=0.2, contrast=0.2, saturation=0.1),  # 光照变化
    transforms.ToTensor(),                   # 转换为 Tensor 格式
    transforms.Normalize(                    # 标准化
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]
    )
])

test_transforms = transforms.Compose([
    transforms.Resize([224,224]),
    transforms.ToTensor(),
    transforms.Normalize(
        mean = [0.485,0.456,0.406],
        std = [0.229,0.224,0.225]
    )
])
total_data = datasets.ImageFolder('./data/mpox_recognize/',transform = train_transforms)
total_data

Dataset ImageFolder
    Number of datapoints: 2142
    Root location: ./data/mpox_recognize/
    StandardTransform
Transform: Compose(
               Resize(size=[224, 224], interpolation=bilinear, max_size=None, antialias=True)
               RandomHorizontalFlip(p=0.5)
               RandomRotation(degrees=[-15.0, 15.0], interpolation=nearest, expand=False, fill=0)
               RandomResizedCrop(size=(224, 224), scale=(0.8, 1.2), ratio=(0.75, 1.3333), interpolation=bilinear, antialias=True)
               ColorJitter(brightness=(0.8, 1.2), contrast=(0.8, 1.2), saturation=(0.9, 1.1), hue=None)
               ToTensor()
               Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])
           )

3. 划分数据集

# 设置随机种子
torch.manual_seed(42)

# 划分数据集
train_size = int(len(total_data) * 0.8)
test_size = len(total_data) - train_size

train_dataset,test_dataset = torch.utils.data.random_split(total_data,[train_size,test_size])
train_dataset,test_dataset

(<torch.utils.data.dataset.Subset at 0x7c9ba5755670>,
 <torch.utils.data.dataset.Subset at 0x7c9ba5755790>)

# 定义DataLoader用于数据集的加载

batch_size = 32 # 如使用多显卡，请确保 batch_size 是显卡数量的倍数。

train_dl = torch.utils.data.DataLoader(
    train_dataset,
    batch_size = batch_size,
    shuffle = True,
    num_workers = 1
)
test_dl = torch.utils.data.DataLoader(
    test_dataset,
    batch_size = batch_size,
    shuffle = True,
    num_workers = 1
)


# 观察数据维度
for X,y in test_dl:
    print("Shape of X [N,C,H,W]: ",X.shape)
    print("Shape of y: ", y.shape,y.dtype)
    break


Shape of X [N,C,H,W]:  torch.Size([32, 3, 224, 224])
Shape of y:  torch.Size([32]) torch.int64

4. 模型构建部分

import torch
import torch.nn as nn
import torch.nn.functional as F


# 定义分组卷积模块
class GroupedConvBlock(nn.Module):
    def __init__(self, in_channels, groups, g_channels, stride):
        super(GroupedConvBlock, self).__init__()
        self.groups = groups
        self.group_convs = nn.ModuleList([
            nn.Conv2d(g_channels, g_channels, kernel_size=3, stride=stride, padding=1, bias=False)
            for _ in range(groups)
        ])
        self.bn = nn.BatchNorm2d(in_channels)
        self.relu = nn.ReLU(inplace=True)

    def forward(self, x):
        # 分组数据
        split_x = torch.split(x, x.size(1) // self.groups, dim=1)
        group_out = [conv(g) for g, conv in zip(split_x, self.group_convs)]
        # 合并数据
        x = torch.cat(group_out, dim=1)
        x = self.bn(x)
        x = self.relu(x)
        return x


# 定义残差模块
class ResNeXtBlock(nn.Module):
    def __init__(self, in_channels, filters, groups=32, stride=1, conv_shortcut=True):
        super(ResNeXtBlock, self).__init__()
        self.conv_shortcut = conv_shortcut
        self.groups = groups
        self.g_channels = filters // groups

        # Shortcut分支
        if conv_shortcut:
            self.shortcut = nn.Sequential(
                nn.Conv2d(in_channels, filters * 2, kernel_size=1, stride=stride, bias=False),
                nn.BatchNorm2d(filters * 2),
            )
        else:
            self.shortcut = nn.Identity()

        # 主分支
        self.conv1 = nn.Sequential(
            nn.Conv2d(in_channels, filters, kernel_size=1, stride=1, bias=False),
            nn.BatchNorm2d(filters),
            nn.ReLU(inplace=True)
        )
        self.grouped_conv = GroupedConvBlock(filters, groups, self.g_channels, stride)
        self.conv3 = nn.Sequential(
            nn.Conv2d(filters, filters * 2, kernel_size=1, stride=1, bias=False),
            nn.BatchNorm2d(filters * 2),
        )
        self.relu = nn.ReLU(inplace=True)

    def forward(self, x):
        shortcut = self.shortcut(x)
        x = self.conv1(x)
        x = self.grouped_conv(x)
        x = self.conv3(x)
        x += shortcut
        x = self.relu(x)
        return x


# 定义 ResNeXt-50 模型
class ResNeXt50(nn.Module):
    def __init__(self, num_classes=1000):
        super(ResNeXt50, self).__init__()
        self.stem = nn.Sequential(
            nn.Conv2d(3, 64, kernel_size=7, stride=2, padding=3, bias=False),
            nn.BatchNorm2d(64),
            nn.ReLU(inplace=True),
            nn.MaxPool2d(kernel_size=3, stride=2, padding=1)
        )

        # 堆叠ResNeXt模块
        self.layer1 = self._make_layer(64, 128, 3, stride=1)
        self.layer2 = self._make_layer(256, 256, 4, stride=2)
        self.layer3 = self._make_layer(512, 512, 6, stride=2)
        self.layer4 = self._make_layer(1024, 1024, 3, stride=2)

        # 全局平均池化和分类层
        self.global_avg_pool = nn.AdaptiveAvgPool2d(1)
        self.fc = nn.Linear(2048, num_classes)

    def _make_layer(self, in_channels, filters, blocks, stride):
        layers = [ResNeXtBlock(in_channels, filters, stride=stride)]
        for _ in range(1, blocks):
            layers.append(ResNeXtBlock(filters * 2, filters, stride=1))
        return nn.Sequential(*layers)

    def forward(self, x):
        x = self.stem(x)
        x = self.layer1(x)
        x = self.layer2(x)
        x = self.layer3(x)
        x = self.layer4(x)
        x = self.global_avg_pool(x)
        x = torch.flatten(x, 1)
        x = self.fc(x)
        return x

model = ResNeXt50(num_classes=len(classNames)).to(device)
model

ResNeXt50(
  (stem): Sequential(
    (0): Conv2d(3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False)
    (1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
    (2): ReLU(inplace=True)
    (3): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
  )
  (layer1): Sequential(
    (0): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(64, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(4, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (1): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(4, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (2): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(256, 128, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(4, 4, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
  )
  (layer2): Sequential(
    (0): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(8, 8, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (1): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (2): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (3): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(8, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
  )
  (layer3): Sequential(
    (0): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (1): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (2): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (3): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (4): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (5): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 512, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(16, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(512, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
  )
  (layer4): Sequential(
    (0): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(1024, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(32, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (1): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(2048, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
    (2): ResNeXtBlock(
      (shortcut): Sequential(
        (0): Conv2d(2048, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (conv1): Sequential(
        (0): Conv2d(2048, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (2): ReLU(inplace=True)
      )
      (grouped_conv): GroupedConvBlock(
        (group_convs): ModuleList(
          (0-31): 32 x Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
        )
        (bn): BatchNorm2d(1024, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
        (relu): ReLU(inplace=True)
      )
      (conv3): Sequential(
        (0): Conv2d(1024, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False)
        (1): BatchNorm2d(2048, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
      )
      (relu): ReLU(inplace=True)
    )
  )
  (global_avg_pool): AdaptiveAvgPool2d(output_size=1)
  (fc): Linear(in_features=2048, out_features=2, bias=True)
)

# 查看模型详情
import torchsummary as summary
summary.summary(model,(3,224,224))

----------------------------------------------------------------
        Layer (type)               Output Shape         Param #
================================================================
            Conv2d-1         [-1, 64, 112, 112]           9,408
       BatchNorm2d-2         [-1, 64, 112, 112]             128
              ReLU-3         [-1, 64, 112, 112]               0
         MaxPool2d-4           [-1, 64, 56, 56]               0
            Conv2d-5          [-1, 256, 56, 56]          16,384
       BatchNorm2d-6          [-1, 256, 56, 56]             512
            Conv2d-7          [-1, 128, 56, 56]           8,192
       BatchNorm2d-8          [-1, 128, 56, 56]             256
              ReLU-9          [-1, 128, 56, 56]               0
           Conv2d-10            [-1, 4, 56, 56]             144
           Conv2d-11            [-1, 4, 56, 56]             144
           Conv2d-12            [-1, 4, 56, 56]             144
           Conv2d-13            [-1, 4, 56, 56]             144
           Conv2d-14            [-1, 4, 56, 56]             144
           Conv2d-15            [-1, 4, 56, 56]             144
           Conv2d-16            [-1, 4, 56, 56]             144
           Conv2d-17            [-1, 4, 56, 56]             144
           Conv2d-18            [-1, 4, 56, 56]             144
           Conv2d-19            [-1, 4, 56, 56]             144
           Conv2d-20            [-1, 4, 56, 56]             144
           Conv2d-21            [-1, 4, 56, 56]             144
           Conv2d-22            [-1, 4, 56, 56]             144
           Conv2d-23            [-1, 4, 56, 56]             144
           Conv2d-24            [-1, 4, 56, 56]             144
           Conv2d-25            [-1, 4, 56, 56]             144
           Conv2d-26            [-1, 4, 56, 56]             144
           Conv2d-27            [-1, 4, 56, 56]             144
           Conv2d-28            [-1, 4, 56, 56]             144
           Conv2d-29            [-1, 4, 56, 56]             144
           Conv2d-30            [-1, 4, 56, 56]             144
           Conv2d-31            [-1, 4, 56, 56]             144
           Conv2d-32            [-1, 4, 56, 56]             144
           Conv2d-33            [-1, 4, 56, 56]             144
           Conv2d-34            [-1, 4, 56, 56]             144
           Conv2d-35            [-1, 4, 56, 56]             144
           Conv2d-36            [-1, 4, 56, 56]             144
           Conv2d-37            [-1, 4, 56, 56]             144
           Conv2d-38            [-1, 4, 56, 56]             144
           Conv2d-39            [-1, 4, 56, 56]             144
           Conv2d-40            [-1, 4, 56, 56]             144
           Conv2d-41            [-1, 4, 56, 56]             144
      BatchNorm2d-42          [-1, 128, 56, 56]             256
             ReLU-43          [-1, 128, 56, 56]               0
 GroupedConvBlock-44          [-1, 128, 56, 56]               0
           Conv2d-45          [-1, 256, 56, 56]          32,768
      BatchNorm2d-46          [-1, 256, 56, 56]             512
             ReLU-47          [-1, 256, 56, 56]               0
     ResNeXtBlock-48          [-1, 256, 56, 56]               0
           Conv2d-49          [-1, 256, 56, 56]          65,536
      BatchNorm2d-50          [-1, 256, 56, 56]             512
           Conv2d-51          [-1, 128, 56, 56]          32,768
      BatchNorm2d-52          [-1, 128, 56, 56]             256
             ReLU-53          [-1, 128, 56, 56]               0
           Conv2d-54            [-1, 4, 56, 56]             144
           Conv2d-55            [-1, 4, 56, 56]             144
           Conv2d-56            [-1, 4, 56, 56]             144
           Conv2d-57            [-1, 4, 56, 56]             144
           Conv2d-58            [-1, 4, 56, 56]             144
           Conv2d-59            [-1, 4, 56, 56]             144
           Conv2d-60            [-1, 4, 56, 56]             144
           Conv2d-61            [-1, 4, 56, 56]             144
           Conv2d-62            [-1, 4, 56, 56]             144
           Conv2d-63            [-1, 4, 56, 56]             144
           Conv2d-64            [-1, 4, 56, 56]             144
           Conv2d-65            [-1, 4, 56, 56]             144
           Conv2d-66            [-1, 4, 56, 56]             144
           Conv2d-67            [-1, 4, 56, 56]             144
           Conv2d-68            [-1, 4, 56, 56]             144
           Conv2d-69            [-1, 4, 56, 56]             144
           Conv2d-70            [-1, 4, 56, 56]             144
           Conv2d-71            [-1, 4, 56, 56]             144
           Conv2d-72            [-1, 4, 56, 56]             144
           Conv2d-73            [-1, 4, 56, 56]             144
           Conv2d-74            [-1, 4, 56, 56]             144
           Conv2d-75            [-1, 4, 56, 56]             144
           Conv2d-76            [-1, 4, 56, 56]             144
           Conv2d-77            [-1, 4, 56, 56]             144
           Conv2d-78            [-1, 4, 56, 56]             144
           Conv2d-79            [-1, 4, 56, 56]             144
           Conv2d-80            [-1, 4, 56, 56]             144
           Conv2d-81            [-1, 4, 56, 56]             144
           Conv2d-82            [-1, 4, 56, 56]             144
           Conv2d-83            [-1, 4, 56, 56]             144
           Conv2d-84            [-1, 4, 56, 56]             144
           Conv2d-85            [-1, 4, 56, 56]             144
      BatchNorm2d-86          [-1, 128, 56, 56]             256
             ReLU-87          [-1, 128, 56, 56]               0
 GroupedConvBlock-88          [-1, 128, 56, 56]               0
           Conv2d-89          [-1, 256, 56, 56]          32,768
      BatchNorm2d-90          [-1, 256, 56, 56]             512
             ReLU-91          [-1, 256, 56, 56]               0
     ResNeXtBlock-92          [-1, 256, 56, 56]               0
           Conv2d-93          [-1, 256, 56, 56]          65,536
      BatchNorm2d-94          [-1, 256, 56, 56]             512
           Conv2d-95          [-1, 128, 56, 56]          32,768
      BatchNorm2d-96          [-1, 128, 56, 56]             256
             ReLU-97          [-1, 128, 56, 56]               0
           Conv2d-98            [-1, 4, 56, 56]             144
           Conv2d-99            [-1, 4, 56, 56]             144
          Conv2d-100            [-1, 4, 56, 56]             144
          Conv2d-101            [-1, 4, 56, 56]             144
          Conv2d-102            [-1, 4, 56, 56]             144
          Conv2d-103            [-1, 4, 56, 56]             144
          Conv2d-104            [-1, 4, 56, 56]             144
          Conv2d-105            [-1, 4, 56, 56]             144
          Conv2d-106            [-1, 4, 56, 56]             144
          Conv2d-107            [-1, 4, 56, 56]             144
          Conv2d-108            [-1, 4, 56, 56]             144
          Conv2d-109            [-1, 4, 56, 56]             144
          Conv2d-110            [-1, 4, 56, 56]             144
          Conv2d-111            [-1, 4, 56, 56]             144
          Conv2d-112            [-1, 4, 56, 56]             144
          Conv2d-113            [-1, 4, 56, 56]             144
          Conv2d-114            [-1, 4, 56, 56]             144
          Conv2d-115            [-1, 4, 56, 56]             144
          Conv2d-116            [-1, 4, 56, 56]             144
          Conv2d-117            [-1, 4, 56, 56]             144
          Conv2d-118            [-1, 4, 56, 56]             144
          Conv2d-119            [-1, 4, 56, 56]             144
          Conv2d-120            [-1, 4, 56, 56]             144
          Conv2d-121            [-1, 4, 56, 56]             144
          Conv2d-122            [-1, 4, 56, 56]             144
          Conv2d-123            [-1, 4, 56, 56]             144
          Conv2d-124            [-1, 4, 56, 56]             144
          Conv2d-125            [-1, 4, 56, 56]             144
          Conv2d-126            [-1, 4, 56, 56]             144
          Conv2d-127            [-1, 4, 56, 56]             144
          Conv2d-128            [-1, 4, 56, 56]             144
          Conv2d-129            [-1, 4, 56, 56]             144
     BatchNorm2d-130          [-1, 128, 56, 56]             256
            ReLU-131          [-1, 128, 56, 56]               0
GroupedConvBlock-132          [-1, 128, 56, 56]               0
          Conv2d-133          [-1, 256, 56, 56]          32,768
     BatchNorm2d-134          [-1, 256, 56, 56]             512
            ReLU-135          [-1, 256, 56, 56]               0
    ResNeXtBlock-136          [-1, 256, 56, 56]               0
          Conv2d-137          [-1, 512, 28, 28]         131,072
     BatchNorm2d-138          [-1, 512, 28, 28]           1,024
          Conv2d-139          [-1, 256, 56, 56]          65,536
     BatchNorm2d-140          [-1, 256, 56, 56]             512
            ReLU-141          [-1, 256, 56, 56]               0
          Conv2d-142            [-1, 8, 28, 28]             576
          Conv2d-143            [-1, 8, 28, 28]             576
          Conv2d-144            [-1, 8, 28, 28]             576
          Conv2d-145            [-1, 8, 28, 28]             576
          Conv2d-146            [-1, 8, 28, 28]             576
          Conv2d-147            [-1, 8, 28, 28]             576
          Conv2d-148            [-1, 8, 28, 28]             576
          Conv2d-149            [-1, 8, 28, 28]             576
          Conv2d-150            [-1, 8, 28, 28]             576
          Conv2d-151            [-1, 8, 28, 28]             576
          Conv2d-152            [-1, 8, 28, 28]             576
          Conv2d-153            [-1, 8, 28, 28]             576
          Conv2d-154            [-1, 8, 28, 28]             576
          Conv2d-155            [-1, 8, 28, 28]             576
          Conv2d-156            [-1, 8, 28, 28]             576
          Conv2d-157            [-1, 8, 28, 28]             576
          Conv2d-158            [-1, 8, 28, 28]             576
          Conv2d-159            [-1, 8, 28, 28]             576
          Conv2d-160            [-1, 8, 28, 28]             576
          Conv2d-161            [-1, 8, 28, 28]             576
          Conv2d-162            [-1, 8, 28, 28]             576
          Conv2d-163            [-1, 8, 28, 28]             576
          Conv2d-164            [-1, 8, 28, 28]             576
          Conv2d-165            [-1, 8, 28, 28]             576
          Conv2d-166            [-1, 8, 28, 28]             576
          Conv2d-167            [-1, 8, 28, 28]             576
          Conv2d-168            [-1, 8, 28, 28]             576
          Conv2d-169            [-1, 8, 28, 28]             576
          Conv2d-170            [-1, 8, 28, 28]             576
          Conv2d-171            [-1, 8, 28, 28]             576
          Conv2d-172            [-1, 8, 28, 28]             576
          Conv2d-173            [-1, 8, 28, 28]             576
     BatchNorm2d-174          [-1, 256, 28, 28]             512
            ReLU-175          [-1, 256, 28, 28]               0
GroupedConvBlock-176          [-1, 256, 28, 28]               0
          Conv2d-177          [-1, 512, 28, 28]         131,072
     BatchNorm2d-178          [-1, 512, 28, 28]           1,024
            ReLU-179          [-1, 512, 28, 28]               0
    ResNeXtBlock-180          [-1, 512, 28, 28]               0
          Conv2d-181          [-1, 512, 28, 28]         262,144
     BatchNorm2d-182          [-1, 512, 28, 28]           1,024
          Conv2d-183          [-1, 256, 28, 28]         131,072
     BatchNorm2d-184          [-1, 256, 28, 28]             512
            ReLU-185          [-1, 256, 28, 28]               0
          Conv2d-186            [-1, 8, 28, 28]             576
          Conv2d-187            [-1, 8, 28, 28]             576
          Conv2d-188            [-1, 8, 28, 28]             576
          Conv2d-189            [-1, 8, 28, 28]             576
          Conv2d-190            [-1, 8, 28, 28]             576
          Conv2d-191            [-1, 8, 28, 28]             576
          Conv2d-192            [-1, 8, 28, 28]             576
          Conv2d-193            [-1, 8, 28, 28]             576
          Conv2d-194            [-1, 8, 28, 28]             576
          Conv2d-195            [-1, 8, 28, 28]             576
          Conv2d-196            [-1, 8, 28, 28]             576
          Conv2d-197            [-1, 8, 28, 28]             576
          Conv2d-198            [-1, 8, 28, 28]             576
          Conv2d-199            [-1, 8, 28, 28]             576
          Conv2d-200            [-1, 8, 28, 28]             576
          Conv2d-201            [-1, 8, 28, 28]             576
          Conv2d-202            [-1, 8, 28, 28]             576
          Conv2d-203            [-1, 8, 28, 28]             576
          Conv2d-204            [-1, 8, 28, 28]             576
          Conv2d-205            [-1, 8, 28, 28]             576
          Conv2d-206            [-1, 8, 28, 28]             576
          Conv2d-207            [-1, 8, 28, 28]             576
          Conv2d-208            [-1, 8, 28, 28]             576
          Conv2d-209            [-1, 8, 28, 28]             576
          Conv2d-210            [-1, 8, 28, 28]             576
          Conv2d-211            [-1, 8, 28, 28]             576
          Conv2d-212            [-1, 8, 28, 28]             576
          Conv2d-213            [-1, 8, 28, 28]             576
          Conv2d-214            [-1, 8, 28, 28]             576
          Conv2d-215            [-1, 8, 28, 28]             576
          Conv2d-216            [-1, 8, 28, 28]             576
          Conv2d-217            [-1, 8, 28, 28]             576
     BatchNorm2d-218          [-1, 256, 28, 28]             512
            ReLU-219          [-1, 256, 28, 28]               0
GroupedConvBlock-220          [-1, 256, 28, 28]               0
          Conv2d-221          [-1, 512, 28, 28]         131,072
     BatchNorm2d-222          [-1, 512, 28, 28]           1,024
            ReLU-223          [-1, 512, 28, 28]               0
    ResNeXtBlock-224          [-1, 512, 28, 28]               0
          Conv2d-225          [-1, 512, 28, 28]         262,144
     BatchNorm2d-226          [-1, 512, 28, 28]           1,024
          Conv2d-227          [-1, 256, 28, 28]         131,072
     BatchNorm2d-228          [-1, 256, 28, 28]             512
            ReLU-229          [-1, 256, 28, 28]               0
          Conv2d-230            [-1, 8, 28, 28]             576
          Conv2d-231            [-1, 8, 28, 28]             576
          Conv2d-232            [-1, 8, 28, 28]             576
          Conv2d-233            [-1, 8, 28, 28]             576
          Conv2d-234            [-1, 8, 28, 28]             576
          Conv2d-235            [-1, 8, 28, 28]             576
          Conv2d-236            [-1, 8, 28, 28]             576
          Conv2d-237            [-1, 8, 28, 28]             576
          Conv2d-238            [-1, 8, 28, 28]             576
          Conv2d-239            [-1, 8, 28, 28]             576
          Conv2d-240            [-1, 8, 28, 28]             576
          Conv2d-241            [-1, 8, 28, 28]             576
          Conv2d-242            [-1, 8, 28, 28]             576
          Conv2d-243            [-1, 8, 28, 28]             576
          Conv2d-244            [-1, 8, 28, 28]             576
          Conv2d-245            [-1, 8, 28, 28]             576
          Conv2d-246            [-1, 8, 28, 28]             576
          Conv2d-247            [-1, 8, 28, 28]             576
          Conv2d-248            [-1, 8, 28, 28]             576
          Conv2d-249            [-1, 8, 28, 28]             576
          Conv2d-250            [-1, 8, 28, 28]             576
          Conv2d-251            [-1, 8, 28, 28]             576
          Conv2d-252            [-1, 8, 28, 28]             576
          Conv2d-253            [-1, 8, 28, 28]             576
          Conv2d-254            [-1, 8, 28, 28]             576
          Conv2d-255            [-1, 8, 28, 28]             576
          Conv2d-256            [-1, 8, 28, 28]             576
          Conv2d-257            [-1, 8, 28, 28]             576
          Conv2d-258            [-1, 8, 28, 28]             576
          Conv2d-259            [-1, 8, 28, 28]             576
          Conv2d-260            [-1, 8, 28, 28]             576
          Conv2d-261            [-1, 8, 28, 28]             576
     BatchNorm2d-262          [-1, 256, 28, 28]             512
            ReLU-263          [-1, 256, 28, 28]               0
GroupedConvBlock-264          [-1, 256, 28, 28]               0
          Conv2d-265          [-1, 512, 28, 28]         131,072
     BatchNorm2d-266          [-1, 512, 28, 28]           1,024
            ReLU-267          [-1, 512, 28, 28]               0
    ResNeXtBlock-268          [-1, 512, 28, 28]               0
          Conv2d-269          [-1, 512, 28, 28]         262,144
     BatchNorm2d-270          [-1, 512, 28, 28]           1,024
          Conv2d-271          [-1, 256, 28, 28]         131,072
     BatchNorm2d-272          [-1, 256, 28, 28]             512
            ReLU-273          [-1, 256, 28, 28]               0
          Conv2d-274            [-1, 8, 28, 28]             576
          Conv2d-275            [-1, 8, 28, 28]             576
          Conv2d-276            [-1, 8, 28, 28]             576
          Conv2d-277            [-1, 8, 28, 28]             576
          Conv2d-278            [-1, 8, 28, 28]             576
          Conv2d-279            [-1, 8, 28, 28]             576
          Conv2d-280            [-1, 8, 28, 28]             576
          Conv2d-281            [-1, 8, 28, 28]             576
          Conv2d-282            [-1, 8, 28, 28]             576
          Conv2d-283            [-1, 8, 28, 28]             576
          Conv2d-284            [-1, 8, 28, 28]             576
          Conv2d-285            [-1, 8, 28, 28]             576
          Conv2d-286            [-1, 8, 28, 28]             576
          Conv2d-287            [-1, 8, 28, 28]             576
          Conv2d-288            [-1, 8, 28, 28]             576
          Conv2d-289            [-1, 8, 28, 28]             576
          Conv2d-290            [-1, 8, 28, 28]             576
          Conv2d-291            [-1, 8, 28, 28]             576
          Conv2d-292            [-1, 8, 28, 28]             576
          Conv2d-293            [-1, 8, 28, 28]             576
          Conv2d-294            [-1, 8, 28, 28]             576
          Conv2d-295            [-1, 8, 28, 28]             576
          Conv2d-296            [-1, 8, 28, 28]             576
          Conv2d-297            [-1, 8, 28, 28]             576
          Conv2d-298            [-1, 8, 28, 28]             576
          Conv2d-299            [-1, 8, 28, 28]             576
          Conv2d-300            [-1, 8, 28, 28]             576
          Conv2d-301            [-1, 8, 28, 28]             576
          Conv2d-302            [-1, 8, 28, 28]             576
          Conv2d-303            [-1, 8, 28, 28]             576
          Conv2d-304            [-1, 8, 28, 28]             576
          Conv2d-305            [-1, 8, 28, 28]             576
     BatchNorm2d-306          [-1, 256, 28, 28]             512
            ReLU-307          [-1, 256, 28, 28]               0
GroupedConvBlock-308          [-1, 256, 28, 28]               0
          Conv2d-309          [-1, 512, 28, 28]         131,072
     BatchNorm2d-310          [-1, 512, 28, 28]           1,024
            ReLU-311          [-1, 512, 28, 28]               0
    ResNeXtBlock-312          [-1, 512, 28, 28]               0
          Conv2d-313         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-314         [-1, 1024, 14, 14]           2,048
          Conv2d-315          [-1, 512, 28, 28]         262,144
     BatchNorm2d-316          [-1, 512, 28, 28]           1,024
            ReLU-317          [-1, 512, 28, 28]               0
          Conv2d-318           [-1, 16, 14, 14]           2,304
          Conv2d-319           [-1, 16, 14, 14]           2,304
          Conv2d-320           [-1, 16, 14, 14]           2,304
          Conv2d-321           [-1, 16, 14, 14]           2,304
          Conv2d-322           [-1, 16, 14, 14]           2,304
          Conv2d-323           [-1, 16, 14, 14]           2,304
          Conv2d-324           [-1, 16, 14, 14]           2,304
          Conv2d-325           [-1, 16, 14, 14]           2,304
          Conv2d-326           [-1, 16, 14, 14]           2,304
          Conv2d-327           [-1, 16, 14, 14]           2,304
          Conv2d-328           [-1, 16, 14, 14]           2,304
          Conv2d-329           [-1, 16, 14, 14]           2,304
          Conv2d-330           [-1, 16, 14, 14]           2,304
          Conv2d-331           [-1, 16, 14, 14]           2,304
          Conv2d-332           [-1, 16, 14, 14]           2,304
          Conv2d-333           [-1, 16, 14, 14]           2,304
          Conv2d-334           [-1, 16, 14, 14]           2,304
          Conv2d-335           [-1, 16, 14, 14]           2,304
          Conv2d-336           [-1, 16, 14, 14]           2,304
          Conv2d-337           [-1, 16, 14, 14]           2,304
          Conv2d-338           [-1, 16, 14, 14]           2,304
          Conv2d-339           [-1, 16, 14, 14]           2,304
          Conv2d-340           [-1, 16, 14, 14]           2,304
          Conv2d-341           [-1, 16, 14, 14]           2,304
          Conv2d-342           [-1, 16, 14, 14]           2,304
          Conv2d-343           [-1, 16, 14, 14]           2,304
          Conv2d-344           [-1, 16, 14, 14]           2,304
          Conv2d-345           [-1, 16, 14, 14]           2,304
          Conv2d-346           [-1, 16, 14, 14]           2,304
          Conv2d-347           [-1, 16, 14, 14]           2,304
          Conv2d-348           [-1, 16, 14, 14]           2,304
          Conv2d-349           [-1, 16, 14, 14]           2,304
     BatchNorm2d-350          [-1, 512, 14, 14]           1,024
            ReLU-351          [-1, 512, 14, 14]               0
GroupedConvBlock-352          [-1, 512, 14, 14]               0
          Conv2d-353         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-354         [-1, 1024, 14, 14]           2,048
            ReLU-355         [-1, 1024, 14, 14]               0
    ResNeXtBlock-356         [-1, 1024, 14, 14]               0
          Conv2d-357         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-358         [-1, 1024, 14, 14]           2,048
          Conv2d-359          [-1, 512, 14, 14]         524,288
     BatchNorm2d-360          [-1, 512, 14, 14]           1,024
            ReLU-361          [-1, 512, 14, 14]               0
          Conv2d-362           [-1, 16, 14, 14]           2,304
          Conv2d-363           [-1, 16, 14, 14]           2,304
          Conv2d-364           [-1, 16, 14, 14]           2,304
          Conv2d-365           [-1, 16, 14, 14]           2,304
          Conv2d-366           [-1, 16, 14, 14]           2,304
          Conv2d-367           [-1, 16, 14, 14]           2,304
          Conv2d-368           [-1, 16, 14, 14]           2,304
          Conv2d-369           [-1, 16, 14, 14]           2,304
          Conv2d-370           [-1, 16, 14, 14]           2,304
          Conv2d-371           [-1, 16, 14, 14]           2,304
          Conv2d-372           [-1, 16, 14, 14]           2,304
          Conv2d-373           [-1, 16, 14, 14]           2,304
          Conv2d-374           [-1, 16, 14, 14]           2,304
          Conv2d-375           [-1, 16, 14, 14]           2,304
          Conv2d-376           [-1, 16, 14, 14]           2,304
          Conv2d-377           [-1, 16, 14, 14]           2,304
          Conv2d-378           [-1, 16, 14, 14]           2,304
          Conv2d-379           [-1, 16, 14, 14]           2,304
          Conv2d-380           [-1, 16, 14, 14]           2,304
          Conv2d-381           [-1, 16, 14, 14]           2,304
          Conv2d-382           [-1, 16, 14, 14]           2,304
          Conv2d-383           [-1, 16, 14, 14]           2,304
          Conv2d-384           [-1, 16, 14, 14]           2,304
          Conv2d-385           [-1, 16, 14, 14]           2,304
          Conv2d-386           [-1, 16, 14, 14]           2,304
          Conv2d-387           [-1, 16, 14, 14]           2,304
          Conv2d-388           [-1, 16, 14, 14]           2,304
          Conv2d-389           [-1, 16, 14, 14]           2,304
          Conv2d-390           [-1, 16, 14, 14]           2,304
          Conv2d-391           [-1, 16, 14, 14]           2,304
          Conv2d-392           [-1, 16, 14, 14]           2,304
          Conv2d-393           [-1, 16, 14, 14]           2,304
     BatchNorm2d-394          [-1, 512, 14, 14]           1,024
            ReLU-395          [-1, 512, 14, 14]               0
GroupedConvBlock-396          [-1, 512, 14, 14]               0
          Conv2d-397         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-398         [-1, 1024, 14, 14]           2,048
            ReLU-399         [-1, 1024, 14, 14]               0
    ResNeXtBlock-400         [-1, 1024, 14, 14]               0
          Conv2d-401         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-402         [-1, 1024, 14, 14]           2,048
          Conv2d-403          [-1, 512, 14, 14]         524,288
     BatchNorm2d-404          [-1, 512, 14, 14]           1,024
            ReLU-405          [-1, 512, 14, 14]               0
          Conv2d-406           [-1, 16, 14, 14]           2,304
          Conv2d-407           [-1, 16, 14, 14]           2,304
          Conv2d-408           [-1, 16, 14, 14]           2,304
          Conv2d-409           [-1, 16, 14, 14]           2,304
          Conv2d-410           [-1, 16, 14, 14]           2,304
          Conv2d-411           [-1, 16, 14, 14]           2,304
          Conv2d-412           [-1, 16, 14, 14]           2,304
          Conv2d-413           [-1, 16, 14, 14]           2,304
          Conv2d-414           [-1, 16, 14, 14]           2,304
          Conv2d-415           [-1, 16, 14, 14]           2,304
          Conv2d-416           [-1, 16, 14, 14]           2,304
          Conv2d-417           [-1, 16, 14, 14]           2,304
          Conv2d-418           [-1, 16, 14, 14]           2,304
          Conv2d-419           [-1, 16, 14, 14]           2,304
          Conv2d-420           [-1, 16, 14, 14]           2,304
          Conv2d-421           [-1, 16, 14, 14]           2,304
          Conv2d-422           [-1, 16, 14, 14]           2,304
          Conv2d-423           [-1, 16, 14, 14]           2,304
          Conv2d-424           [-1, 16, 14, 14]           2,304
          Conv2d-425           [-1, 16, 14, 14]           2,304
          Conv2d-426           [-1, 16, 14, 14]           2,304
          Conv2d-427           [-1, 16, 14, 14]           2,304
          Conv2d-428           [-1, 16, 14, 14]           2,304
          Conv2d-429           [-1, 16, 14, 14]           2,304
          Conv2d-430           [-1, 16, 14, 14]           2,304
          Conv2d-431           [-1, 16, 14, 14]           2,304
          Conv2d-432           [-1, 16, 14, 14]           2,304
          Conv2d-433           [-1, 16, 14, 14]           2,304
          Conv2d-434           [-1, 16, 14, 14]           2,304
          Conv2d-435           [-1, 16, 14, 14]           2,304
          Conv2d-436           [-1, 16, 14, 14]           2,304
          Conv2d-437           [-1, 16, 14, 14]           2,304
     BatchNorm2d-438          [-1, 512, 14, 14]           1,024
            ReLU-439          [-1, 512, 14, 14]               0
GroupedConvBlock-440          [-1, 512, 14, 14]               0
          Conv2d-441         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-442         [-1, 1024, 14, 14]           2,048
            ReLU-443         [-1, 1024, 14, 14]               0
    ResNeXtBlock-444         [-1, 1024, 14, 14]               0
          Conv2d-445         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-446         [-1, 1024, 14, 14]           2,048
          Conv2d-447          [-1, 512, 14, 14]         524,288
     BatchNorm2d-448          [-1, 512, 14, 14]           1,024
            ReLU-449          [-1, 512, 14, 14]               0
          Conv2d-450           [-1, 16, 14, 14]           2,304
          Conv2d-451           [-1, 16, 14, 14]           2,304
          Conv2d-452           [-1, 16, 14, 14]           2,304
          Conv2d-453           [-1, 16, 14, 14]           2,304
          Conv2d-454           [-1, 16, 14, 14]           2,304
          Conv2d-455           [-1, 16, 14, 14]           2,304
          Conv2d-456           [-1, 16, 14, 14]           2,304
          Conv2d-457           [-1, 16, 14, 14]           2,304
          Conv2d-458           [-1, 16, 14, 14]           2,304
          Conv2d-459           [-1, 16, 14, 14]           2,304
          Conv2d-460           [-1, 16, 14, 14]           2,304
          Conv2d-461           [-1, 16, 14, 14]           2,304
          Conv2d-462           [-1, 16, 14, 14]           2,304
          Conv2d-463           [-1, 16, 14, 14]           2,304
          Conv2d-464           [-1, 16, 14, 14]           2,304
          Conv2d-465           [-1, 16, 14, 14]           2,304
          Conv2d-466           [-1, 16, 14, 14]           2,304
          Conv2d-467           [-1, 16, 14, 14]           2,304
          Conv2d-468           [-1, 16, 14, 14]           2,304
          Conv2d-469           [-1, 16, 14, 14]           2,304
          Conv2d-470           [-1, 16, 14, 14]           2,304
          Conv2d-471           [-1, 16, 14, 14]           2,304
          Conv2d-472           [-1, 16, 14, 14]           2,304
          Conv2d-473           [-1, 16, 14, 14]           2,304
          Conv2d-474           [-1, 16, 14, 14]           2,304
          Conv2d-475           [-1, 16, 14, 14]           2,304
          Conv2d-476           [-1, 16, 14, 14]           2,304
          Conv2d-477           [-1, 16, 14, 14]           2,304
          Conv2d-478           [-1, 16, 14, 14]           2,304
          Conv2d-479           [-1, 16, 14, 14]           2,304
          Conv2d-480           [-1, 16, 14, 14]           2,304
          Conv2d-481           [-1, 16, 14, 14]           2,304
     BatchNorm2d-482          [-1, 512, 14, 14]           1,024
            ReLU-483          [-1, 512, 14, 14]               0
GroupedConvBlock-484          [-1, 512, 14, 14]               0
          Conv2d-485         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-486         [-1, 1024, 14, 14]           2,048
            ReLU-487         [-1, 1024, 14, 14]               0
    ResNeXtBlock-488         [-1, 1024, 14, 14]               0
          Conv2d-489         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-490         [-1, 1024, 14, 14]           2,048
          Conv2d-491          [-1, 512, 14, 14]         524,288
     BatchNorm2d-492          [-1, 512, 14, 14]           1,024
            ReLU-493          [-1, 512, 14, 14]               0
          Conv2d-494           [-1, 16, 14, 14]           2,304
          Conv2d-495           [-1, 16, 14, 14]           2,304
          Conv2d-496           [-1, 16, 14, 14]           2,304
          Conv2d-497           [-1, 16, 14, 14]           2,304
          Conv2d-498           [-1, 16, 14, 14]           2,304
          Conv2d-499           [-1, 16, 14, 14]           2,304
          Conv2d-500           [-1, 16, 14, 14]           2,304
          Conv2d-501           [-1, 16, 14, 14]           2,304
          Conv2d-502           [-1, 16, 14, 14]           2,304
          Conv2d-503           [-1, 16, 14, 14]           2,304
          Conv2d-504           [-1, 16, 14, 14]           2,304
          Conv2d-505           [-1, 16, 14, 14]           2,304
          Conv2d-506           [-1, 16, 14, 14]           2,304
          Conv2d-507           [-1, 16, 14, 14]           2,304
          Conv2d-508           [-1, 16, 14, 14]           2,304
          Conv2d-509           [-1, 16, 14, 14]           2,304
          Conv2d-510           [-1, 16, 14, 14]           2,304
          Conv2d-511           [-1, 16, 14, 14]           2,304
          Conv2d-512           [-1, 16, 14, 14]           2,304
          Conv2d-513           [-1, 16, 14, 14]           2,304
          Conv2d-514           [-1, 16, 14, 14]           2,304
          Conv2d-515           [-1, 16, 14, 14]           2,304
          Conv2d-516           [-1, 16, 14, 14]           2,304
          Conv2d-517           [-1, 16, 14, 14]           2,304
          Conv2d-518           [-1, 16, 14, 14]           2,304
          Conv2d-519           [-1, 16, 14, 14]           2,304
          Conv2d-520           [-1, 16, 14, 14]           2,304
          Conv2d-521           [-1, 16, 14, 14]           2,304
          Conv2d-522           [-1, 16, 14, 14]           2,304
          Conv2d-523           [-1, 16, 14, 14]           2,304
          Conv2d-524           [-1, 16, 14, 14]           2,304
          Conv2d-525           [-1, 16, 14, 14]           2,304
     BatchNorm2d-526          [-1, 512, 14, 14]           1,024
            ReLU-527          [-1, 512, 14, 14]               0
GroupedConvBlock-528          [-1, 512, 14, 14]               0
          Conv2d-529         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-530         [-1, 1024, 14, 14]           2,048
            ReLU-531         [-1, 1024, 14, 14]               0
    ResNeXtBlock-532         [-1, 1024, 14, 14]               0
          Conv2d-533         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-534         [-1, 1024, 14, 14]           2,048
          Conv2d-535          [-1, 512, 14, 14]         524,288
     BatchNorm2d-536          [-1, 512, 14, 14]           1,024
            ReLU-537          [-1, 512, 14, 14]               0
          Conv2d-538           [-1, 16, 14, 14]           2,304
          Conv2d-539           [-1, 16, 14, 14]           2,304
          Conv2d-540           [-1, 16, 14, 14]           2,304
          Conv2d-541           [-1, 16, 14, 14]           2,304
          Conv2d-542           [-1, 16, 14, 14]           2,304
          Conv2d-543           [-1, 16, 14, 14]           2,304
          Conv2d-544           [-1, 16, 14, 14]           2,304
          Conv2d-545           [-1, 16, 14, 14]           2,304
          Conv2d-546           [-1, 16, 14, 14]           2,304
          Conv2d-547           [-1, 16, 14, 14]           2,304
          Conv2d-548           [-1, 16, 14, 14]           2,304
          Conv2d-549           [-1, 16, 14, 14]           2,304
          Conv2d-550           [-1, 16, 14, 14]           2,304
          Conv2d-551           [-1, 16, 14, 14]           2,304
          Conv2d-552           [-1, 16, 14, 14]           2,304
          Conv2d-553           [-1, 16, 14, 14]           2,304
          Conv2d-554           [-1, 16, 14, 14]           2,304
          Conv2d-555           [-1, 16, 14, 14]           2,304
          Conv2d-556           [-1, 16, 14, 14]           2,304
          Conv2d-557           [-1, 16, 14, 14]           2,304
          Conv2d-558           [-1, 16, 14, 14]           2,304
          Conv2d-559           [-1, 16, 14, 14]           2,304
          Conv2d-560           [-1, 16, 14, 14]           2,304
          Conv2d-561           [-1, 16, 14, 14]           2,304
          Conv2d-562           [-1, 16, 14, 14]           2,304
          Conv2d-563           [-1, 16, 14, 14]           2,304
          Conv2d-564           [-1, 16, 14, 14]           2,304
          Conv2d-565           [-1, 16, 14, 14]           2,304
          Conv2d-566           [-1, 16, 14, 14]           2,304
          Conv2d-567           [-1, 16, 14, 14]           2,304
          Conv2d-568           [-1, 16, 14, 14]           2,304
          Conv2d-569           [-1, 16, 14, 14]           2,304
     BatchNorm2d-570          [-1, 512, 14, 14]           1,024
            ReLU-571          [-1, 512, 14, 14]               0
GroupedConvBlock-572          [-1, 512, 14, 14]               0
          Conv2d-573         [-1, 1024, 14, 14]         524,288
     BatchNorm2d-574         [-1, 1024, 14, 14]           2,048
            ReLU-575         [-1, 1024, 14, 14]               0
    ResNeXtBlock-576         [-1, 1024, 14, 14]               0
          Conv2d-577           [-1, 2048, 7, 7]       2,097,152
     BatchNorm2d-578           [-1, 2048, 7, 7]           4,096
          Conv2d-579         [-1, 1024, 14, 14]       1,048,576
     BatchNorm2d-580         [-1, 1024, 14, 14]           2,048
            ReLU-581         [-1, 1024, 14, 14]               0
          Conv2d-582             [-1, 32, 7, 7]           9,216
          Conv2d-583             [-1, 32, 7, 7]           9,216
          Conv2d-584             [-1, 32, 7, 7]           9,216
          Conv2d-585             [-1, 32, 7, 7]           9,216
          Conv2d-586             [-1, 32, 7, 7]           9,216
          Conv2d-587             [-1, 32, 7, 7]           9,216
          Conv2d-588             [-1, 32, 7, 7]           9,216
          Conv2d-589             [-1, 32, 7, 7]           9,216
          Conv2d-590             [-1, 32, 7, 7]           9,216
          Conv2d-591             [-1, 32, 7, 7]           9,216
          Conv2d-592             [-1, 32, 7, 7]           9,216
          Conv2d-593             [-1, 32, 7, 7]           9,216
          Conv2d-594             [-1, 32, 7, 7]           9,216
          Conv2d-595             [-1, 32, 7, 7]           9,216
          Conv2d-596             [-1, 32, 7, 7]           9,216
          Conv2d-597             [-1, 32, 7, 7]           9,216
          Conv2d-598             [-1, 32, 7, 7]           9,216
          Conv2d-599             [-1, 32, 7, 7]           9,216
          Conv2d-600             [-1, 32, 7, 7]           9,216
          Conv2d-601             [-1, 32, 7, 7]           9,216
          Conv2d-602             [-1, 32, 7, 7]           9,216
          Conv2d-603             [-1, 32, 7, 7]           9,216
          Conv2d-604             [-1, 32, 7, 7]           9,216
          Conv2d-605             [-1, 32, 7, 7]           9,216
          Conv2d-606             [-1, 32, 7, 7]           9,216
          Conv2d-607             [-1, 32, 7, 7]           9,216
          Conv2d-608             [-1, 32, 7, 7]           9,216
          Conv2d-609             [-1, 32, 7, 7]           9,216
          Conv2d-610             [-1, 32, 7, 7]           9,216
          Conv2d-611             [-1, 32, 7, 7]           9,216
          Conv2d-612             [-1, 32, 7, 7]           9,216
          Conv2d-613             [-1, 32, 7, 7]           9,216
     BatchNorm2d-614           [-1, 1024, 7, 7]           2,048
            ReLU-615           [-1, 1024, 7, 7]               0
GroupedConvBlock-616           [-1, 1024, 7, 7]               0
          Conv2d-617           [-1, 2048, 7, 7]       2,097,152
     BatchNorm2d-618           [-1, 2048, 7, 7]           4,096
            ReLU-619           [-1, 2048, 7, 7]               0
    ResNeXtBlock-620           [-1, 2048, 7, 7]               0
          Conv2d-621           [-1, 2048, 7, 7]       4,194,304
     BatchNorm2d-622           [-1, 2048, 7, 7]           4,096
          Conv2d-623           [-1, 1024, 7, 7]       2,097,152
     BatchNorm2d-624           [-1, 1024, 7, 7]           2,048
            ReLU-625           [-1, 1024, 7, 7]               0
          Conv2d-626             [-1, 32, 7, 7]           9,216
          Conv2d-627             [-1, 32, 7, 7]           9,216
          Conv2d-628             [-1, 32, 7, 7]           9,216
          Conv2d-629             [-1, 32, 7, 7]           9,216
          Conv2d-630             [-1, 32, 7, 7]           9,216
          Conv2d-631             [-1, 32, 7, 7]           9,216
          Conv2d-632             [-1, 32, 7, 7]           9,216
          Conv2d-633             [-1, 32, 7, 7]           9,216
          Conv2d-634             [-1, 32, 7, 7]           9,216
          Conv2d-635             [-1, 32, 7, 7]           9,216
          Conv2d-636             [-1, 32, 7, 7]           9,216
          Conv2d-637             [-1, 32, 7, 7]           9,216
          Conv2d-638             [-1, 32, 7, 7]           9,216
          Conv2d-639             [-1, 32, 7, 7]           9,216
          Conv2d-640             [-1, 32, 7, 7]           9,216
          Conv2d-641             [-1, 32, 7, 7]           9,216
          Conv2d-642             [-1, 32, 7, 7]           9,216
          Conv2d-643             [-1, 32, 7, 7]           9,216
          Conv2d-644             [-1, 32, 7, 7]           9,216
          Conv2d-645             [-1, 32, 7, 7]           9,216
          Conv2d-646             [-1, 32, 7, 7]           9,216
          Conv2d-647             [-1, 32, 7, 7]           9,216
          Conv2d-648             [-1, 32, 7, 7]           9,216
          Conv2d-649             [-1, 32, 7, 7]           9,216
          Conv2d-650             [-1, 32, 7, 7]           9,216
          Conv2d-651             [-1, 32, 7, 7]           9,216
          Conv2d-652             [-1, 32, 7, 7]           9,216
          Conv2d-653             [-1, 32, 7, 7]           9,216
          Conv2d-654             [-1, 32, 7, 7]           9,216
          Conv2d-655             [-1, 32, 7, 7]           9,216
          Conv2d-656             [-1, 32, 7, 7]           9,216
          Conv2d-657             [-1, 32, 7, 7]           9,216
     BatchNorm2d-658           [-1, 1024, 7, 7]           2,048
            ReLU-659           [-1, 1024, 7, 7]               0
GroupedConvBlock-660           [-1, 1024, 7, 7]               0
          Conv2d-661           [-1, 2048, 7, 7]       2,097,152
     BatchNorm2d-662           [-1, 2048, 7, 7]           4,096
            ReLU-663           [-1, 2048, 7, 7]               0
    ResNeXtBlock-664           [-1, 2048, 7, 7]               0
          Conv2d-665           [-1, 2048, 7, 7]       4,194,304
     BatchNorm2d-666           [-1, 2048, 7, 7]           4,096
          Conv2d-667           [-1, 1024, 7, 7]       2,097,152
     BatchNorm2d-668           [-1, 1024, 7, 7]           2,048
            ReLU-669           [-1, 1024, 7, 7]               0
          Conv2d-670             [-1, 32, 7, 7]           9,216
          Conv2d-671             [-1, 32, 7, 7]           9,216
          Conv2d-672             [-1, 32, 7, 7]           9,216
          Conv2d-673             [-1, 32, 7, 7]           9,216
          Conv2d-674             [-1, 32, 7, 7]           9,216
          Conv2d-675             [-1, 32, 7, 7]           9,216
          Conv2d-676             [-1, 32, 7, 7]           9,216
          Conv2d-677             [-1, 32, 7, 7]           9,216
          Conv2d-678             [-1, 32, 7, 7]           9,216
          Conv2d-679             [-1, 32, 7, 7]           9,216
          Conv2d-680             [-1, 32, 7, 7]           9,216
          Conv2d-681             [-1, 32, 7, 7]           9,216
          Conv2d-682             [-1, 32, 7, 7]           9,216
          Conv2d-683             [-1, 32, 7, 7]           9,216
          Conv2d-684             [-1, 32, 7, 7]           9,216
          Conv2d-685             [-1, 32, 7, 7]           9,216
          Conv2d-686             [-1, 32, 7, 7]           9,216
          Conv2d-687             [-1, 32, 7, 7]           9,216
          Conv2d-688             [-1, 32, 7, 7]           9,216
          Conv2d-689             [-1, 32, 7, 7]           9,216
          Conv2d-690             [-1, 32, 7, 7]           9,216
          Conv2d-691             [-1, 32, 7, 7]           9,216
          Conv2d-692             [-1, 32, 7, 7]           9,216
          Conv2d-693             [-1, 32, 7, 7]           9,216
          Conv2d-694             [-1, 32, 7, 7]           9,216
          Conv2d-695             [-1, 32, 7, 7]           9,216
          Conv2d-696             [-1, 32, 7, 7]           9,216
          Conv2d-697             [-1, 32, 7, 7]           9,216
          Conv2d-698             [-1, 32, 7, 7]           9,216
          Conv2d-699             [-1, 32, 7, 7]           9,216
          Conv2d-700             [-1, 32, 7, 7]           9,216
          Conv2d-701             [-1, 32, 7, 7]           9,216
     BatchNorm2d-702           [-1, 1024, 7, 7]           2,048
            ReLU-703           [-1, 1024, 7, 7]               0
GroupedConvBlock-704           [-1, 1024, 7, 7]               0
          Conv2d-705           [-1, 2048, 7, 7]       2,097,152
     BatchNorm2d-706           [-1, 2048, 7, 7]           4,096
            ReLU-707           [-1, 2048, 7, 7]               0
    ResNeXtBlock-708           [-1, 2048, 7, 7]               0
AdaptiveAvgPool2d-709           [-1, 2048, 1, 1]               0
          Linear-710                    [-1, 2]           4,098
================================================================
Total params: 37,555,522
Trainable params: 37,555,522
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.57
Forward/backward pass size (MB): 444.08
Params size (MB): 143.26
Estimated Total Size (MB): 587.92
----------------------------------------------------------------

5. 设置超参数：定义损失函数，学习率，以及根据学习率定义优化器等

# loss_fn = nn.CrossEntropyLoss() # 创建损失函数

# learn_rate = 1e-3 # 初始学习率
# def adjust_learning_rate(optimizer,epoch,start_lr):
#     # 每两个epoch 衰减到原来的0.98
#     lr = start_lr * (0.92 ** (epoch//2))
#     for param_group in optimizer.param_groups:
#         param_group['lr'] = lr
        
# optimizer = torch.optim.Adam(model.parameters(),lr=learn_rate)


# 调用官方接口示例
loss_fn = nn.CrossEntropyLoss()

# learn_rate = 1e-4  
learn_rate = 3e-4
lambda1 = lambda epoch:(0.92**(epoch//2))

optimizer = torch.optim.Adam(model.parameters(),lr = learn_rate)
scheduler = torch.optim.lr_scheduler.LambdaLR(optimizer,lr_lambda=lambda1) # 选定调整方法

6. 训练函数

# 训练函数
def train(dataloader,model,loss_fn,optimizer):
    size = len(dataloader.dataset) # 训练集大小
    num_batches = len(dataloader) # 批次数目
    
    train_loss,train_acc = 0,0
    
    for X,y in dataloader:
        X,y = X.to(device),y.to(device)
        
        # 计算预测误差
        pred = model(X)
        loss = loss_fn(pred,y)
        
        # 反向传播
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        
        # 记录acc与loss
        train_acc += (pred.argmax(1)==y).type(torch.float).sum().item()
        train_loss += loss.item()
        
    train_acc /= size
    train_loss /= num_batches
    
    return train_acc,train_loss

7. 测试函数

# 测试函数
def test(dataloader,model,loss_fn):
    size = len(dataloader.dataset)
    num_batches = len(dataloader)
    
    test_acc,test_loss = 0,0
    
    with torch.no_grad():
        for X,y in dataloader:
            X,y = X.to(device),y.to(device)
            
            # 计算loss
            pred = model(X)
            loss = loss_fn(pred,y)
            
            test_acc += (pred.argmax(1)==y).type(torch.float).sum().item()
            test_loss += loss.item()
            
    test_acc /= size
    test_loss /= num_batches
    
    return test_acc,test_loss

8. 正式训练

import copy

epochs = 60

train_acc = []
train_loss = []
test_acc = []
test_loss = []

best_acc = 0.0

# 检查 GPU 可用性并打印设备信息
if torch.cuda.is_available():
    for i in range(torch.cuda.device_count()):
        print(f"GPU {i}: {torch.cuda.get_device_name(i)}")
        print(f"Initial Memory Allocated: {torch.cuda.memory_allocated(i)/1024**2:.2f} MB")
        print(f"Initial Memory Cached: {torch.cuda.memory_reserved(i)/1024**2:.2f} MB")
else:
    print("No GPU available. Using CPU.")

# 多显卡设置 当前使用的是使用 PyTorch 自带的 DataParallel，后续如有需要可以设置为DistributedDataParallel，这是更加高效的方式
# 且多卡不一定比单卡效果就好，需要调整优化
# if torch.cuda.device_count() > 1:
#     print(f"Using {torch.cuda.device_count()} GPUs")
#     model = nn.DataParallel(model)
# model = model.to('cuda')

for epoch in range(epochs):
    
    # 更新学习率——使用自定义学习率时使用
    # adjust_learning_rate(optimizer,epoch,learn_rate)
    
    model.train()
    epoch_train_acc,epoch_train_loss = train(train_dl,model,loss_fn,optimizer)
    scheduler.step() # 更新学习率——调用官方动态学习率时使用
    
    model.eval()
    epoch_test_acc,epoch_test_loss = test(test_dl,model,loss_fn)
    
    # 保存最佳模型到 best_model
    if epoch_test_acc > best_acc:
        best_acc = epoch_test_acc
        best_model = copy.deepcopy(model)
    
    train_acc.append(epoch_train_acc)
    train_loss.append(epoch_train_loss)
    test_acc.append(epoch_test_acc)
    test_loss.append(epoch_test_loss)
    
    # 获取当前学习率
    lr = optimizer.state_dict()['param_groups'][0]['lr']
    
    template = ('Epoch:{:2d},Train_acc:{:.1f}%,Train_loss:{:.3f},Test_acc:{:.1f}%,Test_loss:{:.3f},Lr:{:.2E}')
    print(template.format(epoch+1,epoch_train_acc*100,epoch_train_loss,epoch_test_acc*100,epoch_test_loss,lr))

    # 实时监控 GPU 状态
    if torch.cuda.is_available():
        for i in range(torch.cuda.device_count()):
            print(f"GPU {i} Usage:")
            print(f"  Memory Allocated: {torch.cuda.memory_allocated(i)/1024**2:.2f} MB")
            print(f"  Memory Cached: {torch.cuda.memory_reserved(i)/1024**2:.2f} MB")
            print(f"  Max Memory Allocated: {torch.cuda.max_memory_allocated(i)/1024**2:.2f} MB")
            print(f"  Max Memory Cached: {torch.cuda.max_memory_reserved(i)/1024**2:.2f} MB")

print('Done','best_acc: ',best_acc)

GPU 0: NVIDIA GeForce RTX 4090
Initial Memory Allocated: 151.84 MB
Initial Memory Cached: 422.00 MB
GPU 1: NVIDIA GeForce RTX 4090
Initial Memory Allocated: 0.00 MB
Initial Memory Cached: 0.00 MB
GPU 2: NVIDIA GeForce RTX 4090
Initial Memory Allocated: 0.00 MB
Initial Memory Cached: 0.00 MB
GPU 3: NVIDIA GeForce RTX 4090
Initial Memory Allocated: 0.00 MB
Initial Memory Cached: 0.00 MB
Epoch: 1,Train_acc:56.1%,Train_loss:0.737,Test_acc:51.5%,Test_loss:0.845,Lr:3.00E-04
GPU 0 Usage:
  Memory Allocated: 737.77 MB
  Memory Cached: 5010.00 MB
  Max Memory Allocated: 4545.57 MB
  Max Memory Cached: 5010.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 2,Train_acc:61.8%,Train_loss:0.668,Test_acc:66.9%,Test_loss:0.655,Lr:2.76E-04
GPU 0 Usage:
  Memory Allocated: 739.44 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 3,Train_acc:63.4%,Train_loss:0.650,Test_acc:60.8%,Test_loss:0.658,Lr:2.76E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 4,Train_acc:65.7%,Train_loss:0.637,Test_acc:61.3%,Test_loss:0.668,Lr:2.54E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 5,Train_acc:66.8%,Train_loss:0.633,Test_acc:65.0%,Test_loss:0.618,Lr:2.54E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 6,Train_acc:66.6%,Train_loss:0.615,Test_acc:62.7%,Test_loss:0.629,Lr:2.34E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 7,Train_acc:66.7%,Train_loss:0.608,Test_acc:64.6%,Test_loss:0.611,Lr:2.34E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 8,Train_acc:68.7%,Train_loss:0.592,Test_acc:67.1%,Test_loss:0.598,Lr:2.15E-04
GPU 0 Usage:
  Memory Allocated: 735.13 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch: 9,Train_acc:69.0%,Train_loss:0.598,Test_acc:67.1%,Test_loss:0.579,Lr:2.15E-04
GPU 0 Usage:
  Memory Allocated: 738.74 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.41 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:10,Train_acc:68.7%,Train_loss:0.575,Test_acc:66.7%,Test_loss:0.561,Lr:1.98E-04
GPU 0 Usage:
  Memory Allocated: 738.24 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:11,Train_acc:70.5%,Train_loss:0.571,Test_acc:72.7%,Test_loss:0.559,Lr:1.98E-04
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:12,Train_acc:71.7%,Train_loss:0.565,Test_acc:68.8%,Test_loss:0.558,Lr:1.82E-04
GPU 0 Usage:
  Memory Allocated: 737.04 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:13,Train_acc:71.2%,Train_loss:0.570,Test_acc:73.4%,Test_loss:0.532,Lr:1.82E-04
GPU 0 Usage:
  Memory Allocated: 738.40 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:14,Train_acc:72.4%,Train_loss:0.553,Test_acc:70.2%,Test_loss:0.544,Lr:1.67E-04
GPU 0 Usage:
  Memory Allocated: 738.67 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:15,Train_acc:71.5%,Train_loss:0.554,Test_acc:70.2%,Test_loss:0.551,Lr:1.67E-04
GPU 0 Usage:
  Memory Allocated: 733.94 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:16,Train_acc:72.8%,Train_loss:0.551,Test_acc:69.5%,Test_loss:0.557,Lr:1.54E-04
GPU 0 Usage:
  Memory Allocated: 735.85 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:17,Train_acc:72.8%,Train_loss:0.528,Test_acc:70.6%,Test_loss:0.573,Lr:1.54E-04
GPU 0 Usage:
  Memory Allocated: 735.85 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:18,Train_acc:72.6%,Train_loss:0.546,Test_acc:70.6%,Test_loss:0.550,Lr:1.42E-04
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:19,Train_acc:74.0%,Train_loss:0.523,Test_acc:74.1%,Test_loss:0.544,Lr:1.42E-04
GPU 0 Usage:
  Memory Allocated: 737.51 MB
  Memory Cached: 5024.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5024.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:20,Train_acc:75.7%,Train_loss:0.494,Test_acc:73.4%,Test_loss:0.532,Lr:1.30E-04
GPU 0 Usage:
  Memory Allocated: 737.07 MB
  Memory Cached: 5122.00 MB
  Max Memory Allocated: 4690.54 MB
  Max Memory Cached: 5122.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:21,Train_acc:77.6%,Train_loss:0.469,Test_acc:74.4%,Test_loss:0.506,Lr:1.30E-04
GPU 0 Usage:
  Memory Allocated: 738.05 MB
  Memory Cached: 5122.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5122.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:22,Train_acc:76.6%,Train_loss:0.488,Test_acc:76.7%,Test_loss:0.478,Lr:1.20E-04
GPU 0 Usage:
  Memory Allocated: 736.08 MB
  Memory Cached: 5122.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5122.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:23,Train_acc:77.1%,Train_loss:0.485,Test_acc:72.3%,Test_loss:0.497,Lr:1.20E-04
GPU 0 Usage:
  Memory Allocated: 736.83 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:24,Train_acc:79.5%,Train_loss:0.454,Test_acc:76.0%,Test_loss:0.519,Lr:1.10E-04
GPU 0 Usage:
  Memory Allocated: 736.83 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:25,Train_acc:77.3%,Train_loss:0.476,Test_acc:76.9%,Test_loss:0.466,Lr:1.10E-04
GPU 0 Usage:
  Memory Allocated: 736.83 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:26,Train_acc:80.7%,Train_loss:0.436,Test_acc:79.7%,Test_loss:0.452,Lr:1.01E-04
GPU 0 Usage:
  Memory Allocated: 736.58 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:27,Train_acc:79.0%,Train_loss:0.429,Test_acc:78.8%,Test_loss:0.429,Lr:1.01E-04
GPU 0 Usage:
  Memory Allocated: 737.29 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:28,Train_acc:78.9%,Train_loss:0.433,Test_acc:79.3%,Test_loss:0.405,Lr:9.34E-05
GPU 0 Usage:
  Memory Allocated: 737.48 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:29,Train_acc:82.4%,Train_loss:0.389,Test_acc:85.5%,Test_loss:0.368,Lr:9.34E-05
GPU 0 Usage:
  Memory Allocated: 737.78 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.07 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:30,Train_acc:82.8%,Train_loss:0.388,Test_acc:81.6%,Test_loss:0.409,Lr:8.59E-05
GPU 0 Usage:
  Memory Allocated: 738.97 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.16 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:31,Train_acc:84.6%,Train_loss:0.361,Test_acc:83.2%,Test_loss:0.408,Lr:8.59E-05
GPU 0 Usage:
  Memory Allocated: 739.15 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.16 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:32,Train_acc:85.7%,Train_loss:0.336,Test_acc:84.8%,Test_loss:0.379,Lr:7.90E-05
GPU 0 Usage:
  Memory Allocated: 739.40 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.39 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:33,Train_acc:86.9%,Train_loss:0.306,Test_acc:86.9%,Test_loss:0.340,Lr:7.90E-05
GPU 0 Usage:
  Memory Allocated: 736.35 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:34,Train_acc:86.8%,Train_loss:0.311,Test_acc:88.1%,Test_loss:0.329,Lr:7.27E-05
GPU 0 Usage:
  Memory Allocated: 738.80 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:35,Train_acc:87.4%,Train_loss:0.312,Test_acc:82.3%,Test_loss:0.394,Lr:7.27E-05
GPU 0 Usage:
  Memory Allocated: 738.49 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:36,Train_acc:87.0%,Train_loss:0.313,Test_acc:87.2%,Test_loss:0.318,Lr:6.69E-05
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:37,Train_acc:88.6%,Train_loss:0.280,Test_acc:88.6%,Test_loss:0.286,Lr:6.69E-05
GPU 0 Usage:
  Memory Allocated: 738.21 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:38,Train_acc:88.8%,Train_loss:0.270,Test_acc:86.9%,Test_loss:0.321,Lr:6.15E-05
GPU 0 Usage:
  Memory Allocated: 736.86 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:39,Train_acc:88.6%,Train_loss:0.283,Test_acc:83.9%,Test_loss:0.338,Lr:6.15E-05
GPU 0 Usage:
  Memory Allocated: 736.86 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:40,Train_acc:90.0%,Train_loss:0.249,Test_acc:89.0%,Test_loss:0.249,Lr:5.66E-05
GPU 0 Usage:
  Memory Allocated: 736.86 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:41,Train_acc:91.0%,Train_loss:0.226,Test_acc:91.8%,Test_loss:0.211,Lr:5.66E-05
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:42,Train_acc:90.1%,Train_loss:0.242,Test_acc:91.1%,Test_loss:0.233,Lr:5.21E-05
GPU 0 Usage:
  Memory Allocated: 736.80 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:43,Train_acc:92.1%,Train_loss:0.196,Test_acc:89.5%,Test_loss:0.245,Lr:5.21E-05
GPU 0 Usage:
  Memory Allocated: 736.57 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:44,Train_acc:93.0%,Train_loss:0.198,Test_acc:90.0%,Test_loss:0.232,Lr:4.79E-05
GPU 0 Usage:
  Memory Allocated: 738.29 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:45,Train_acc:92.6%,Train_loss:0.195,Test_acc:92.3%,Test_loss:0.227,Lr:4.79E-05
GPU 0 Usage:
  Memory Allocated: 738.55 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:46,Train_acc:93.0%,Train_loss:0.184,Test_acc:91.4%,Test_loss:0.263,Lr:4.41E-05
GPU 0 Usage:
  Memory Allocated: 736.11 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:47,Train_acc:93.5%,Train_loss:0.164,Test_acc:93.0%,Test_loss:0.186,Lr:4.41E-05
GPU 0 Usage:
  Memory Allocated: 736.08 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:48,Train_acc:93.9%,Train_loss:0.163,Test_acc:91.8%,Test_loss:0.220,Lr:4.06E-05
GPU 0 Usage:
  Memory Allocated: 737.04 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:49,Train_acc:94.2%,Train_loss:0.163,Test_acc:93.2%,Test_loss:0.223,Lr:4.06E-05
GPU 0 Usage:
  Memory Allocated: 737.07 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:50,Train_acc:93.8%,Train_loss:0.161,Test_acc:92.5%,Test_loss:0.203,Lr:3.73E-05
GPU 0 Usage:
  Memory Allocated: 736.50 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:51,Train_acc:92.3%,Train_loss:0.174,Test_acc:92.8%,Test_loss:0.178,Lr:3.73E-05
GPU 0 Usage:
  Memory Allocated: 735.31 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:52,Train_acc:95.1%,Train_loss:0.134,Test_acc:92.3%,Test_loss:0.191,Lr:3.43E-05
GPU 0 Usage:
  Memory Allocated: 736.88 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:53,Train_acc:95.2%,Train_loss:0.148,Test_acc:93.7%,Test_loss:0.164,Lr:3.43E-05
GPU 0 Usage:
  Memory Allocated: 737.10 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:54,Train_acc:95.0%,Train_loss:0.119,Test_acc:93.0%,Test_loss:0.180,Lr:3.16E-05
GPU 0 Usage:
  Memory Allocated: 737.07 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:55,Train_acc:94.2%,Train_loss:0.143,Test_acc:91.8%,Test_loss:0.197,Lr:3.16E-05
GPU 0 Usage:
  Memory Allocated: 737.07 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:56,Train_acc:96.1%,Train_loss:0.111,Test_acc:93.0%,Test_loss:0.183,Lr:2.91E-05
GPU 0 Usage:
  Memory Allocated: 737.33 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:57,Train_acc:96.1%,Train_loss:0.102,Test_acc:94.9%,Test_loss:0.170,Lr:2.91E-05
GPU 0 Usage:
  Memory Allocated: 738.05 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:58,Train_acc:95.2%,Train_loss:0.120,Test_acc:93.5%,Test_loss:0.201,Lr:2.67E-05
GPU 0 Usage:
  Memory Allocated: 739.19 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:59,Train_acc:95.3%,Train_loss:0.136,Test_acc:94.9%,Test_loss:0.157,Lr:2.67E-05
GPU 0 Usage:
  Memory Allocated: 738.25 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Epoch:60,Train_acc:95.5%,Train_loss:0.106,Test_acc:93.5%,Test_loss:0.167,Lr:2.46E-05
GPU 0 Usage:
  Memory Allocated: 739.19 MB
  Memory Cached: 5124.00 MB
  Max Memory Allocated: 4691.79 MB
  Max Memory Cached: 5124.00 MB
GPU 1 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 2 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
GPU 3 Usage:
  Memory Allocated: 0.00 MB
  Memory Cached: 0.00 MB
  Max Memory Allocated: 0.00 MB
  Max Memory Cached: 0.00 MB
Done best_acc:  0.9487179487179487

9. 结果可视化

epochs_range = range(epochs)

plt.figure(figsize = (12,3))

plt.subplot(1,2,1)
plt.plot(epochs_range,train_acc,label = 'Training Accuracy')
plt.plot(epochs_range,test_acc,label = 'Test Accuracy')
plt.legend(loc = 'lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1,2,2)
plt.plot(epochs_range,train_loss,label = 'Test Accuracy')
plt.plot(epochs_range,test_loss,label = 'Test Loss')
plt.legend(loc = 'lower right')
plt.title('Training and validation Loss')
plt.show()



findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei
findfont: Generic family 'sans-serif' not found because none of the following families were found: SimHei

在这里插入图片描述

​

10. 模型的保存

# 自定义模型保存
# 状态字典保存
torch.save(model.state_dict(),'./模型参数/J6_ResNeXt50_model_state_dict2.pth') # 仅保存状态字典

# 定义模型用来加载参数
best_model = ResNeXt50(num_classes=len(classNames)).to(device)

best_model.load_state_dict(torch.load('./模型参数/J6_ResNeXt50_model_state_dict2.pth')) # 加载状态字典到模型


<All keys matched successfully>

11.使用训练好的模型进行预测

# 指定路径图片预测
from PIL import Image
import torchvision.transforms as transforms

classes = list(total_data.class_to_idx) # classes = list(total_data.class_to_idx)

def predict_one_image(image_path,model,transform,classes):
    
    test_img = Image.open(image_path).convert('RGB')
    # plt.imshow(test_img) # 展示待预测的图片
    
    test_img = transform(test_img)
    img = test_img.to(device).unsqueeze(0)
    
    model.eval()
    output = model(img)
    print(output) # 观察模型预测结果的输出数据
    
    _,pred = torch.max(output,1)
    pred_class = classes[pred]
    print(f'预测结果是:{pred_class}')


# 预测训练集中的某张照片
predict_one_image(image_path='./data/mpox_recognize/Monkeypox/M01_01_04.jpg',
                 model = model,
                 transform = test_transforms,
                 classes = classes
                 )


tensor([[ 2.6236, -2.9544]], device='cuda:0', grad_fn=<AddmmBackward0>)
预测结果是:Monkeypox

classes

['Monkeypox', 'Others']

本文来自互联网用户投稿，该文观点仅代表作者本人，不代表本站立场。本站仅提供信息存储空间服务，不拥有所有权，不承担相关法律责任。如若转载，请注明出处：http://www.coloradmin.cn/o/2246585.html

如若内容造成侵权/违法违规/事实不符，请联系多彩编程网进行投诉反馈，一经查实，立即删除！

相关文章

Transformer架构笔记

Transformer架构笔记

Attention is All You Need. 3.Model Architecture 3.1 整体架构如图 3.2 Encoder与Decoder Encoder&#xff1a;由 N 6 N6 N6个相同的Block/Layer堆叠而成。每个Block有两个子层sub-layer&#xff1a;多头注意力和MLP&#xff08;FFN&#xff0c;前馈神经网络&#xff09;&…
阅读更多...
【大数据学习 | Spark-Core】spark-shell开发

【大数据学习 | Spark-Core】spark-shell开发

spark的代码分为两种 本地代码在driver端直接解析执行没有后续 集群代码&#xff0c;会在driver端进行解析&#xff0c;然后让多个机器进行集群形式的执行计算 spark-shell --master spark://nn1:7077 --executor-cores 2 --executor-memory 2G sc.textFile("/home/ha…
阅读更多...
增量预训练（Pretrain）样本拼接篇

增量预训练（Pretrain）样本拼接篇

增量预训练&#xff08;Pretrain&#xff09;样本拼接篇 一、Pretrain阶段&#xff0c;为什么需要拼接拼接&#xff1f; 为了提高pretrain效率、拓展LLM最大长度&#xff0c;随机将若干条短文本进行拼接是pretrain阶段常见手段。 二、有哪些拼接方式&#xff1f; 拼接方式一…
阅读更多...
【AI最前线】DP双像素sensor相关的AI算法全集：深度估计、图像去模糊去雨去雾恢复、图像重建、自动对焦

【AI最前线】DP双像素sensor相关的AI算法全集：深度估计、图像去模糊去雨去雾恢复、图像重建、自动对焦

Dual Pixel 简介 双像素是成像系统的感光元器件中单帧同时生成的图像&#xff1a;通过双像素可以实现&#xff1a;深度估计、图像去模糊去雨去雾恢复、图像重建 成像原理来源如上&#xff0c;也有遮罩等方式的pd生成&#xff0c;如图双像素视图可以看到光圈的不同一半&#x…
阅读更多...
从零开始-VitePress 构建个人博客上传GitHub自动构建访问

从零开始-VitePress 构建个人博客上传GitHub自动构建访问

从零开始-VitePress 构建个人博客上传GitHub自动构建访问 序言 VitePress 官网&#xff1a;VitePress 中文版 1. 什么是 VitePress VitePress 是一个静态站点生成器 (SSG)&#xff0c;专为构建快速、以内容为中心的站点而设计。简而言之&#xff0c;VitePress 获取用 Markdown…
阅读更多...
使用uniapp编写APP的文件上传

使用uniapp编写APP的文件上传

使用uniapp插件文件选择、文件上传组件&#xff08;图片&#xff0c;视频&#xff0c;文件等&#xff09; - DCloud 插件市场 实用效果&#xff1a; 缺陷是只能一个一个单独上传
阅读更多...
【51单片机】红外遥控

【51单片机】红外遥控

学习使用的开发板&#xff1a;STC89C52RC/LE52RC 编程软件&#xff1a;Keil5 烧录软件&#xff1a;stc-isp 开发板实图&#xff1a; 文章目录 红外遥控硬件电路 NEC协议编码编程实例LCD1602显示Data红外遥控控制扇叶转速 红外遥控 红外遥控是利用红外光进行通信的设备&#…
阅读更多...
【解决】Unity TMPro字体中文显示错误/不全问题

【解决】Unity TMPro字体中文显示错误/不全问题

问题描述&#xff1a;字体变成方块 原因&#xff1a;字体资源所承载的长度有限 1.找一个中文字体放入Assets中 2.选中字体创建为TMPro 字体资源 3.选中创建好的字体资源&#xff08;蓝色的大F&#xff09; 在右边的属性中找到Atlas Width h和 Atlas Heigth,修改的大一点&…
阅读更多...
深度学习：GPT-1的MindSpore实践

深度学习：GPT-1的MindSpore实践

GPT-1简介 GPT-1&#xff08;Generative Pre-trained Transformer&#xff09;是2018年由Open AI提出的一个结合预训练和微调的用于解决文本理解和文本生成任务的模型。它的基础是Transformer架构&#xff0c;具有如下创新点&#xff1a; NLP领域的迁移学习&#xff1a;通过最…
阅读更多...
CKA认证 | Day2 K8s内部监控与日志

CKA认证 | Day2 K8s内部监控与日志

第三章 Kubernetes监控与日志 1、查看集群资源状态 在 Kubernetes 集群中&#xff0c;查看集群资源状态和组件状态是非常重要的操作。以下是一些常用的命令和解释&#xff0c;帮助你更好地管理和监控 Kubernetes 集群。 1.1 查看master组件状态 Kubernetes 的 Master 组件包…
阅读更多...
概念解读｜K8s/容器云/裸金属/云原生...这些都有什么区别？

概念解读｜K8s/容器云/裸金属/云原生...这些都有什么区别？

随着容器技术的日渐成熟&#xff0c;不少企业用户都对应用系统开展了容器化改造。而在容器基础架构层面&#xff0c;很多运维人员都更熟悉虚拟化环境&#xff0c;对“容器圈”的各种概念容易混淆&#xff1a;容器就是 Kubernetes 吗&#xff1f;容器云又是什么&#xff1f;容器…
阅读更多...
JDBC编程---Java

JDBC编程---Java

目录 一、数据库编程的前置 二、Java的数据库编程----JDBC 1.概念 2.JDBC编程的优点 三.导入MySQL驱动包 四、JDBC编程的实战 1.创造数据源&#xff0c;并设置数据库所在的位置&#xff0c;三条固定写法 2.建立和数据库服务器之间的连接&#xff0c;连接好了后&#xff…
阅读更多...
移动充储机器人“小奥”的多场景应用（上）

移动充储机器人“小奥”的多场景应用（上）

在当前现代化城市交通体系中&#xff0c;移动充储机器人“小奥”发挥着至关重要的作用。该机器人不仅是一个简单的设备&#xff0c;而是一个集成了高科技的移动充电站&#xff0c;为新能源汽车提供了一种前所未有的便捷充电解决方案。该机器人配备了先进的电池管理系统&#xf…
阅读更多...
element dialog会隐藏body scroll 导致tab抖动 解决方案如下

element dialog会隐藏body scroll 导致tab抖动 解决方案如下

element dialog会隐藏body scroll 导致tab抖动 解决方案如下 在dialog标签添加 :lockScroll"false"搞定
阅读更多...
Android 功耗分析（底层篇）

Android 功耗分析（底层篇）

最近在网上发现关于功耗分析系列的文章很少&#xff0c;介绍详细的更少&#xff0c;于是便想记录总结一下功耗分析的相关知识&#xff0c;有不对的地方希望大家多指出&#xff0c;互相学习。本系列分为底层篇和上层篇。 大概从基础知识&#xff0c;测试手法&#xff0c;以及案例…
阅读更多...
Bugku CTF_Web——my-first-sqli

Bugku CTF_Web——my-first-sqli

Bugku CTF_Web——my-first-sqli 进入靶场 随便输一个看看 点login没有任何回显 方法一&#xff1a; 上bp抓包 放到repeter测试 试试万能密码&#xff08;靶机过期了重新开了个靶机&#xff09; admin or 11--shellmates{SQLi_goeS_BrrRrRR}方法二&#xff1a; 拿包直接梭…
阅读更多...
BUUCTF—Reverse—easyre（1）

BUUCTF—Reverse—easyre（1）

非常简单的逆向 拿到exe文件先查下信息&#xff0c;是一个64位程序&#xff0c;没有加壳&#xff08;壳是对代码的加密&#xff0c;起混淆保护的作用&#xff0c;一般用来阻止逆向&#xff09;。 然后拖进IDA(64位)进行反汇编 打开以后就可以看到flag flag{this_Is_a_EaSyRe}
阅读更多...
全面击破工程级复杂缓存难题

全面击破工程级复杂缓存难题

目录 一、走进业务中的缓存 &#xff08;一&#xff09;本地缓存 &#xff08;二&#xff09;分布式缓存 二、缓存更新模式分析 &#xff08;一&#xff09;Cache Aside Pattern&#xff08;旁路缓存模式&#xff09; 读操作流程 写操作流程 流程问题思考 问题1&#…
阅读更多...
React基础知识一

React基础知识一

写的东西太多了&#xff0c;照成csdn文档编辑器都开始卡顿了&#xff0c;所以分篇写。 1.安装React 需要安装下面三个包。 react:react核心包 react-dom:渲染需要用到的核心包 babel:将jsx语法转换成React代码的工具。&#xff08;没使用jsx可以不装&#xff09;1.1 在html中…
阅读更多...
Vue3中使用:deep修改element-plus的样式无效怎么办？

Vue3中使用:deep修改element-plus的样式无效怎么办？

前言&#xff1a;当我们用 vue3 :deep() 处理 elementui 中 el-dialog_body和el-dislog__header 的时候样式一直无法生效&#xff0c;遇到这种情况怎么办&#xff1f; 解决办法&#xff1a; 1.直接在 dialog 上面增加class 我试过&#xff0c;也不起作用&#xff0c;最后用这种…
阅读更多...
推荐文章
最新文章

Copyright @ 2022~2023