ShuffleNet图像分类

当前案例不支持在GPU设备上静态图模式运行，其他模式运行皆支持。

ShuffleNet网络介绍

ShuffleNetV1是旷视科技提出的一种计算高效的CNN模型，和MobileNet, SqueezeNet等一样主要应用在移动端，所以模型的设计目标就是利用有限的计算资源来达到最好的模型精度。ShuffleNetV1的设计核心是引入了两种操作：Pointwise Group Convolution和Channel Shuffle，这在保持精度的同时大大降低了模型的计算量。因此，ShuffleNetV1和MobileNet类似，都是通过设计更高效的网络结构来实现模型的压缩和加速。

了解ShuffleNet更多详细内容，详见论文ShuffleNet。

如下图所示，ShuffleNet在保持不低的准确率的前提下，将参数量几乎降低到了最小，因此其运算速度较快，单位参数量对模型准确率的贡献非常高。

图片来源：Bianco S, Cadene R, Celona L, et al. Benchmark analysis of representative deep neural network architectures[J]. IEEE access, 2018, 6: 64270-64277.

模型架构

ShuffleNet最显著的特点在于对不同通道进行重排来解决Group Convolution带来的弊端。通过对ResNet的Bottleneck单元进行改进，在较小的计算量的情况下达到了较高的准确率。

Pointwise Group Convolution

Group Convolution（分组卷积）原理如下图所示，相比于普通的卷积操作，分组卷积的情况下，每一组的卷积核大小为in_channels/g*k*k，一共有g组，所有组共有(in_channels/g*k*k)*out_channels个参数，是正常卷积参数的1/g。分组卷积中，每个卷积核只处理输入特征图的一部分通道，其优点在于参数量会有所降低，但输出通道数仍等于卷积核的数量。

图片来源：Huang G, Liu S, Van der Maaten L, et al. Condensenet: An efficient densenet using learned group convolutions[C]//Proceedings of the IEEE conference on computer vision and pattern recognition. 2018: 2752-2761.

Depthwise Convolution（深度可分离卷积）将组数g分为和输入通道相等的in_channels，然后对每一个in_channels做卷积操作，每个卷积核只处理一个通道，记卷积核大小为1*k*k，则卷积核参数量为：in_channels*k*k，得到的feature maps通道数与输入通道数相等；

Pointwise Group Convolution（逐点分组卷积）在分组卷积的基础上，令每一组的卷积核大小为 $1\times 1$，卷积核参数量为(in_channels/g*1*1)*out_channels。

%%capture captured_output
# 实验环境已经预装了mindspore==2.2.14，如需更换mindspore版本，可更改下面mindspore的版本号
# !pip uninstall mindspore -y
# !pip install -i https://pypi.mirrors.ustc.edu.cn/simple mindspore==2.2.14

# 查看当前 mindspore 版本
!pip show mindspore

Name: mindspore
Version: 2.2.14
Summary: MindSpore is a new open source deep learning training/inference framework that could be used for mobile, edge and cloud scenarios.
Home-page: https://www.mindspore.cn
Author: The MindSpore Authors
Author-email: contact@mindspore.cn
License: Apache 2.0
Location: /home/nginx/miniconda/envs/jupyter/lib/python3.9/site-packages
Requires: asttokens, astunparse, numpy, packaging, pillow, protobuf, psutil, scipy
Required-by: 

from mindspore import nn
import mindspore.ops as ops
from mindspore import Tensor

class GroupConv(nn.Cell):
    def __init__(self, in_channels, out_channels, kernel_size,
                 stride, pad_mode="pad", pad=0, groups=1, has_bias=False):
        super(GroupConv, self).__init__()
        self.groups = groups
        self.convs = nn.CellList()
        for _ in range(groups):
            self.convs.append(nn.Conv2d(in_channels // groups, out_channels // groups,
                                        kernel_size=kernel_size, stride=stride, has_bias=has_bias,
                                        padding=pad, pad_mode=pad_mode, group=1, weight_init='xavier_uniform'))

    def construct(self, x):
        features = ops.split(x, split_size_or_sections=int(len(x[0]) // self.groups), axis=1)
        outputs = ()
        for i in range(self.groups):
            outputs = outputs + (self.convs[i](features[i].astype("float32")),)
        out = ops.cat(outputs, axis=1)
        return out

Channel Shuffle

Group Convolution的弊端在于不同组别的通道无法进行信息交流，堆积GConv层后一个问题是不同组之间的特征图是不通信的，这就好像分成了g个互不相干的道路，每一个人各走各的，这可能会降低网络的特征提取能力。这也是Xception，MobileNet等网络采用密集的1x1卷积（Dense Pointwise Convolution）的原因。

为了解决不同组别通道“近亲繁殖”的问题，ShuffleNet优化了大量密集的1x1卷积（在使用的情况下计算量占用率达到了惊人的93.4%），引入Channel Shuffle机制（通道重排）。这项操作直观上表现为将不同分组通道均匀分散重组，使网络在下一层能处理不同组别通道的信息。

如下图所示，对于g组，每组有n个通道的特征图，首先reshape成g行n列的矩阵，再将矩阵转置成n行g列，最后进行flatten操作，得到新的排列。这些操作都是可微分可导的且计算简单，在解决了信息交互的同时符合了ShuffleNet轻量级网络设计的轻量特征。

为了阅读方便，将Channel Shuffle的代码实现放在下方ShuffleNet模块的代码中。

ShuffleNet模块

如下图所示，ShuffleNet对ResNet中的Bottleneck结构进行由(a)到(b), ©的更改：

1. 将开始和最后的$1\times 1$卷积模块（降维、升维）改成Point Wise Group Convolution；

2. 为了进行不同通道的信息交流，再降维之后进行Channel Shuffle；

3. 降采样模块中，$3\times 3$ Depth Wise Convolution的步长设置为2，长宽降为原来的一般，因此shortcut中采用步长为2的$3\times 3$平均池化，并把相加改成拼接。

class ShuffleV1Block(nn.Cell):
    def __init__(self, inp, oup, group, first_group, mid_channels, ksize, stride):
        super(ShuffleV1Block, self).__init__()
        self.stride = stride
        pad = ksize // 2
        self.group = group
        if stride == 2:
            outputs = oup - inp
        else:
            outputs = oup
        self.relu = nn.ReLU()
        branch_main_1 = [
            GroupConv(in_channels=inp, out_channels=mid_channels,
                      kernel_size=1, stride=1, pad_mode="pad", pad=0,
                      groups=1 if first_group else group),
            nn.BatchNorm2d(mid_channels),
            nn.ReLU(),
        ]
        branch_main_2 = [
            nn.Conv2d(mid_channels, mid_channels, kernel_size=ksize, stride=stride,
                      pad_mode='pad', padding=pad, group=mid_channels,
                      weight_init='xavier_uniform', has_bias=False),
            nn.BatchNorm2d(mid_channels),
            GroupConv(in_channels=mid_channels, out_channels=outputs,
                      kernel_size=1, stride=1, pad_mode="pad", pad=0,
                      groups=group),
            nn.BatchNorm2d(outputs),
        ]
        self.branch_main_1 = nn.SequentialCell(branch_main_1)
        self.branch_main_2 = nn.SequentialCell(branch_main_2)
        if stride == 2:
            self.branch_proj = nn.AvgPool2d(kernel_size=3, stride=2, pad_mode='same')

    def construct(self, old_x):
        left = old_x
        right = old_x
        out = old_x
        right = self.branch_main_1(right)
        if self.group > 1:
            right = self.channel_shuffle(right)
        right = self.branch_main_2(right)
        if self.stride == 1:
            out = self.relu(left + right)
        elif self.stride == 2:
            left = self.branch_proj(left)
            out = ops.cat((left, right), 1)
            out = self.relu(out)
        return out

    def channel_shuffle(self, x):
        batchsize, num_channels, height, width = ops.shape(x)
        group_channels = num_channels // self.group
        x = ops.reshape(x, (batchsize, group_channels, self.group, height, width))
        x = ops.transpose(x, (0, 2, 1, 3, 4))
        x = ops.reshape(x, (batchsize, num_channels, height, width))
        return x

构建ShuffleNet网络

ShuffleNet网络结构如下图所示，以输入图像$224\times 224$，组数3（g = 3）为例，首先通过数量24，卷积核大小为$3\times 3$，stride为2的卷积层，输出特征图大小为$112\times 112$，channel为24；然后通过stride为2的最大池化层，输出特征图大小为$56\times 56$，channel数不变；再堆叠3个ShuffleNet模块（Stage2, Stage3, Stage4），三个模块分别重复4次、8次、4次，其中每个模块开始先经过一次下采样模块（上图©），使特征图长宽减半，channel翻倍（Stage2的下采样模块除外，将channel数从24变为240）；随后经过全局平均池化，输出大小为$1\times 1\times 960$，再经过全连接层和softmax，得到分类概率。

class ShuffleNetV1(nn.Cell):
    def __init__(self, n_class=1000, model_size='2.0x', group=3):
        super(ShuffleNetV1, self).__init__()
        print('model size is ', model_size)
        self.stage_repeats = [4, 8, 4]
        self.model_size = model_size
        if group == 3:
            if model_size == '0.5x':
                self.stage_out_channels = [-1, 12, 120, 240, 480]
            elif model_size == '1.0x':
                self.stage_out_channels = [-1, 24, 240, 480, 960]
            elif model_size == '1.5x':
                self.stage_out_channels = [-1, 24, 360, 720, 1440]
            elif model_size == '2.0x':
                self.stage_out_channels = [-1, 48, 480, 960, 1920]
            else:
                raise NotImplementedError
        elif group == 8:
            if model_size == '0.5x':
                self.stage_out_channels = [-1, 16, 192, 384, 768]
            elif model_size == '1.0x':
                self.stage_out_channels = [-1, 24, 384, 768, 1536]
            elif model_size == '1.5x':
                self.stage_out_channels = [-1, 24, 576, 1152, 2304]
            elif model_size == '2.0x':
                self.stage_out_channels = [-1, 48, 768, 1536, 3072]
            else:
                raise NotImplementedError
        input_channel = self.stage_out_channels[1]
        self.first_conv = nn.SequentialCell(
            nn.Conv2d(3, input_channel, 3, 2, 'pad', 1, weight_init='xavier_uniform', has_bias=False),
            nn.BatchNorm2d(input_channel),
            nn.ReLU(),
        )
        self.maxpool = nn.MaxPool2d(kernel_size=3, stride=2, pad_mode='same')
        features = []
        for idxstage in range(len(self.stage_repeats)):
            numrepeat = self.stage_repeats[idxstage]
            output_channel = self.stage_out_channels[idxstage + 2]
            for i in range(numrepeat):
                stride = 2 if i == 0 else 1
                first_group = idxstage == 0 and i == 0
                features.append(ShuffleV1Block(input_channel, output_channel,
                                               group=group, first_group=first_group,
                                               mid_channels=output_channel // 4, ksize=3, stride=stride))
                input_channel = output_channel
        self.features = nn.SequentialCell(features)
        self.globalpool = nn.AvgPool2d(7)
        self.classifier = nn.Dense(self.stage_out_channels[-1], n_class)

    def construct(self, x):
        x = self.first_conv(x)
        x = self.maxpool(x)
        x = self.features(x)
        x = self.globalpool(x)
        x = ops.reshape(x, (-1, self.stage_out_channels[-1]))
        x = self.classifier(x)
        return x

模型训练和评估

采用CIFAR-10数据集对ShuffleNet进行预训练。

训练集准备与加载

采用CIFAR-10数据集对ShuffleNet进行预训练。CIFAR-10共有60000张32*32的彩色图像，均匀地分为10个类别，其中50000张图片作为训练集，10000图片作为测试集。如下示例使用mindspore.dataset.Cifar10Dataset接口下载并加载CIFAR-10的训练集。目前仅支持二进制版本（CIFAR-10 binary version）。

from download import download

url = "https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/cifar-10-binary.tar.gz"

download(url, "./dataset", kind="tar.gz", replace=True)

Downloading data from https://mindspore-website.obs.cn-north-4.myhuaweicloud.com/notebook/datasets/cifar-10-binary.tar.gz (162.2 MB)

file_sizes: 100%|█████████████████████████████| 170M/170M [00:01<00:00, 147MB/s]
Extracting tar.gz file...
Successfully downloaded / unzipped to ./dataset





'./dataset'

import mindspore as ms
from mindspore.dataset import Cifar10Dataset
from mindspore.dataset import vision, transforms

def get_dataset(train_dataset_path, batch_size, usage):
    image_trans = []
    if usage == "train":
        image_trans = [
            vision.RandomCrop((32, 32), (4, 4, 4, 4)),
            vision.RandomHorizontalFlip(prob=0.5),
            vision.Resize((224, 224)),
            vision.Rescale(1.0 / 255.0, 0.0),
            vision.Normalize([0.4914, 0.4822, 0.4465], [0.2023, 0.1994, 0.2010]),
            vision.HWC2CHW()
        ]
    elif usage == "test":
        image_trans = [
            vision.Resize((224, 224)),
            vision.Rescale(1.0 / 255.0, 0.0),
            vision.Normalize([0.4914, 0.4822, 0.4465], [0.2023, 0.1994, 0.2010]),
            vision.HWC2CHW()
        ]
    label_trans = transforms.TypeCast(ms.int32)
    dataset = Cifar10Dataset(train_dataset_path, usage=usage, shuffle=True)
    dataset = dataset.map(image_trans, 'image')
    dataset = dataset.map(label_trans, 'label')
    dataset = dataset.batch(batch_size, drop_remainder=True)
    return dataset

dataset = get_dataset("./dataset/cifar-10-batches-bin", 128, "train")
batches_per_epoch = dataset.get_dataset_size()

模型训练

本节用随机初始化的参数做预训练。首先调用ShuffleNetV1定义网络，参数量选择"2.0x"，并定义损失函数为交叉熵损失，学习率经过4轮的warmup后采用余弦退火，优化器采用Momentum。最后用train.model中的Model接口将模型、损失函数、优化器封装在model中，并用model.train()对网络进行训练。将ModelCheckpoint、CheckpointConfig、TimeMonitor和LossMonitor传入回调函数中，将会打印训练的轮数、损失和时间，并将ckpt文件保存在当前目录下。

import time
import mindspore
import numpy as np
from mindspore import Tensor, nn
from mindspore.train import ModelCheckpoint, CheckpointConfig, TimeMonitor, LossMonitor, Model, Top1CategoricalAccuracy, Top5CategoricalAccuracy

def train():
    mindspore.set_context(mode=mindspore.PYNATIVE_MODE, device_target="Ascend")
    net = ShuffleNetV1(model_size="2.0x", n_class=10)
    loss = nn.CrossEntropyLoss(weight=None, reduction='mean', label_smoothing=0.1)
    min_lr = 0.0005
    base_lr = 0.05
    lr_scheduler = mindspore.nn.cosine_decay_lr(min_lr,
                                                base_lr,
                                                batches_per_epoch*250,
                                                batches_per_epoch,
                                                decay_epoch=250)
    lr = Tensor(lr_scheduler[-1])
    optimizer = nn.Momentum(params=net.trainable_params(), learning_rate=lr, momentum=0.9, weight_decay=0.00004, loss_scale=1024)
    loss_scale_manager = ms.amp.FixedLossScaleManager(1024, drop_overflow_update=False)
    model = Model(net, loss_fn=loss, optimizer=optimizer, amp_level="O3", loss_scale_manager=loss_scale_manager)
    callback = [TimeMonitor(), LossMonitor()]
    save_ckpt_path = "./"
    config_ckpt = CheckpointConfig(save_checkpoint_steps=batches_per_epoch, keep_checkpoint_max=5)
    ckpt_callback = ModelCheckpoint("shufflenetv1", directory=save_ckpt_path, config=config_ckpt)
    callback += [ckpt_callback]

    print("============== Starting Training ==============")
    start_time = time.time()
    # 由于时间原因，epoch = 5，可根据需求进行调整
    model.train(5, dataset, callbacks=callback)
    use_time = time.time() - start_time
    hour = str(int(use_time // 60 // 60))
    minute = str(int(use_time // 60 % 60))
    second = str(int(use_time % 60))
    print("total time:" + hour + "h " + minute + "m " + second + "s")
    print("============== Train Success ==============")

if __name__ == '__main__':
    train()

model size is  2.0x
============== Starting Training ==============
epoch: 1 step: 1, loss is 2.7894816398620605
epoch: 1 step: 2, loss is 2.6364171504974365
epoch: 1 step: 3, loss is 2.324631690979004
epoch: 1 step: 4, loss is 2.3651773929595947
epoch: 1 step: 5, loss is 2.3691365718841553
epoch: 1 step: 6, loss is 2.6283822059631348
epoch: 1 step: 7, loss is 2.6781303882598877
epoch: 1 step: 8, loss is 2.4975063800811768
epoch: 1 step: 9, loss is 2.3142592906951904
epoch: 1 step: 10, loss is 2.331934928894043
epoch: 1 step: 11, loss is 2.3772664070129395
epoch: 1 step: 12, loss is 2.359488010406494
epoch: 1 step: 13, loss is 2.4175662994384766
epoch: 1 step: 14, loss is 2.3472161293029785
epoch: 1 step: 15, loss is 2.294532537460327
epoch: 1 step: 16, loss is 2.336198091506958
epoch: 1 step: 17, loss is 2.339566946029663
epoch: 1 step: 18, loss is 2.291743040084839
epoch: 1 step: 19, loss is 2.312931537628174
epoch: 1 step: 20, loss is 2.2701454162597656
epoch: 1 step: 21, loss is 2.2745139598846436
epoch: 1 step: 22, loss is 2.166996479034424
epoch: 1 step: 23, loss is 2.2239151000976562
epoch: 1 step: 24, loss is 2.282738208770752
epoch: 1 step: 25, loss is 2.346029758453369
epoch: 1 step: 26, loss is 2.242628574371338
epoch: 1 step: 27, loss is 2.250455856323242
epoch: 1 step: 28, loss is 2.16402006149292
epoch: 1 step: 29, loss is 2.2349720001220703
epoch: 1 step: 30, loss is 2.1936607360839844
epoch: 1 step: 31, loss is 2.279332160949707
epoch: 1 step: 32, loss is 2.2144205570220947
epoch: 1 step: 33, loss is 2.231557607650757
epoch: 1 step: 34, loss is 2.1735870838165283
epoch: 1 step: 35, loss is 2.117922306060791
epoch: 1 step: 36, loss is 2.1843276023864746
epoch: 1 step: 37, loss is 2.2099449634552
epoch: 1 step: 38, loss is 2.126649856567383
epoch: 1 step: 39, loss is 2.1400492191314697
epoch: 1 step: 40, loss is 2.1635758876800537
epoch: 1 step: 41, loss is 2.108804225921631
epoch: 1 step: 42, loss is 2.1741466522216797
epoch: 1 step: 43, loss is 2.1443824768066406
epoch: 1 step: 44, loss is 2.11466646194458
epoch: 1 step: 45, loss is 2.1021575927734375
epoch: 1 step: 46, loss is 2.215764045715332
epoch: 1 step: 47, loss is 2.095478057861328
epoch: 1 step: 48, loss is 2.1447417736053467
epoch: 1 step: 49, loss is 2.1359975337982178
epoch: 1 step: 50, loss is 2.1233294010162354
epoch: 1 step: 51, loss is 2.0877249240875244
epoch: 1 step: 52, loss is 2.09409236907959
epoch: 1 step: 53, loss is 2.0857648849487305
epoch: 1 step: 54, loss is 2.1872661113739014
epoch: 1 step: 55, loss is 2.062849998474121
epoch: 1 step: 56, loss is 2.0739450454711914
epoch: 1 step: 57, loss is 2.0996291637420654
epoch: 1 step: 58, loss is 2.1183764934539795
epoch: 1 step: 59, loss is 2.059744358062744
epoch: 1 step: 60, loss is 2.241192579269409
epoch: 1 step: 61, loss is 2.1036524772644043
epoch: 1 step: 62, loss is 2.0720231533050537
epoch: 1 step: 63, loss is 2.0292201042175293
epoch: 1 step: 64, loss is 2.186490058898926
epoch: 1 step: 65, loss is 2.0250093936920166
epoch: 1 step: 66, loss is 2.0556015968322754
epoch: 1 step: 67, loss is 2.099801540374756
epoch: 1 step: 68, loss is 2.0900049209594727
epoch: 1 step: 69, loss is 2.0917611122131348
epoch: 1 step: 70, loss is 2.0622942447662354
epoch: 1 step: 71, loss is 2.159672260284424
epoch: 1 step: 72, loss is 2.0002942085266113
epoch: 1 step: 73, loss is 2.1159677505493164
epoch: 1 step: 74, loss is 2.0264692306518555
epoch: 1 step: 75, loss is 2.1245148181915283
epoch: 1 step: 76, loss is 2.09993839263916
epoch: 1 step: 77, loss is 2.068373441696167
epoch: 1 step: 78, loss is 2.045447826385498
epoch: 1 step: 79, loss is 2.0015697479248047
epoch: 1 step: 80, loss is 2.123058319091797
epoch: 1 step: 81, loss is 2.1312191486358643
epoch: 1 step: 82, loss is 1.9868097305297852
epoch: 1 step: 83, loss is 2.013669729232788
epoch: 1 step: 84, loss is 2.0548126697540283
epoch: 1 step: 85, loss is 1.9909785985946655
epoch: 1 step: 86, loss is 1.9981229305267334
epoch: 1 step: 87, loss is 2.0516743659973145
epoch: 1 step: 88, loss is 2.0890421867370605
epoch: 1 step: 89, loss is 1.9980154037475586
epoch: 1 step: 90, loss is 2.0141515731811523
epoch: 1 step: 91, loss is 2.151697874069214
epoch: 1 step: 92, loss is 2.1343061923980713
epoch: 1 step: 93, loss is 2.033583641052246
epoch: 1 step: 94, loss is 1.9642503261566162
epoch: 1 step: 95, loss is 2.00840163230896
epoch: 1 step: 96, loss is 2.0554041862487793
epoch: 1 step: 97, loss is 2.208998203277588
epoch: 1 step: 98, loss is 2.055987596511841
epoch: 1 step: 99, loss is 2.0073509216308594
epoch: 1 step: 100, loss is 2.0468127727508545
epoch: 1 step: 101, loss is 1.9658652544021606
epoch: 1 step: 102, loss is 2.0595295429229736
epoch: 1 step: 103, loss is 2.0754847526550293
epoch: 1 step: 104, loss is 1.939456582069397
epoch: 1 step: 105, loss is 2.023190975189209
epoch: 1 step: 106, loss is 2.006678819656372
epoch: 1 step: 107, loss is 1.9567711353302002
epoch: 1 step: 108, loss is 2.083509683609009
epoch: 1 step: 109, loss is 2.10636830329895
epoch: 1 step: 110, loss is 2.0519893169403076
epoch: 1 step: 111, loss is 1.9812997579574585
epoch: 1 step: 112, loss is 2.0779731273651123
epoch: 1 step: 113, loss is 2.030236005783081
epoch: 1 step: 114, loss is 2.0478675365448
epoch: 1 step: 115, loss is 2.063617706298828
epoch: 1 step: 116, loss is 2.0169436931610107
epoch: 1 step: 117, loss is 2.0534791946411133
epoch: 1 step: 118, loss is 2.0846495628356934
epoch: 1 step: 119, loss is 2.084014892578125
epoch: 1 step: 120, loss is 2.0072219371795654
epoch: 1 step: 121, loss is 1.9768632650375366
epoch: 1 step: 122, loss is 1.9183000326156616
epoch: 1 step: 123, loss is 1.9620945453643799
epoch: 1 step: 124, loss is 1.9968945980072021
epoch: 1 step: 125, loss is 2.049635171890259
epoch: 1 step: 126, loss is 2.0318548679351807
epoch: 1 step: 127, loss is 2.011958599090576
epoch: 1 step: 128, loss is 1.9974169731140137
epoch: 1 step: 129, loss is 1.9214017391204834
epoch: 1 step: 130, loss is 1.9999300241470337
epoch: 1 step: 131, loss is 2.03224515914917
epoch: 1 step: 132, loss is 1.9330816268920898
epoch: 1 step: 133, loss is 2.043043613433838
epoch: 1 step: 134, loss is 1.8980891704559326
epoch: 1 step: 135, loss is 1.9312021732330322
epoch: 1 step: 136, loss is 1.9890092611312866
epoch: 1 step: 137, loss is 1.8607290983200073
epoch: 1 step: 138, loss is 1.968400001525879
epoch: 1 step: 139, loss is 2.0289511680603027
epoch: 1 step: 140, loss is 1.9724494218826294
epoch: 1 step: 141, loss is 2.0490927696228027
epoch: 1 step: 142, loss is 1.9316545724868774
epoch: 1 step: 143, loss is 1.9586957693099976
epoch: 1 step: 144, loss is 1.952976942062378
epoch: 1 step: 145, loss is 1.930903673171997
epoch: 1 step: 146, loss is 1.936833143234253
epoch: 1 step: 147, loss is 1.980993628501892
epoch: 1 step: 148, loss is 1.879756212234497
epoch: 1 step: 149, loss is 2.0265610218048096
epoch: 1 step: 150, loss is 1.8828983306884766
epoch: 1 step: 151, loss is 1.9725685119628906
epoch: 1 step: 152, loss is 1.9439165592193604
epoch: 1 step: 153, loss is 1.8751428127288818
epoch: 1 step: 154, loss is 2.1073951721191406
epoch: 1 step: 155, loss is 2.0055952072143555
epoch: 1 step: 156, loss is 1.917148470878601
epoch: 1 step: 157, loss is 1.9495508670806885
epoch: 1 step: 158, loss is 1.9720405340194702
epoch: 1 step: 159, loss is 1.9717621803283691
epoch: 1 step: 160, loss is 2.0548882484436035
epoch: 1 step: 161, loss is 1.9924187660217285
epoch: 1 step: 162, loss is 2.0912423133850098
epoch: 1 step: 163, loss is 1.8856409788131714
epoch: 1 step: 164, loss is 1.9217889308929443
epoch: 1 step: 165, loss is 1.9482144117355347
epoch: 1 step: 166, loss is 2.0016369819641113
epoch: 1 step: 167, loss is 1.8985767364501953
epoch: 1 step: 168, loss is 1.9907989501953125
epoch: 1 step: 169, loss is 1.9516243934631348
epoch: 1 step: 170, loss is 1.9886001348495483
epoch: 1 step: 171, loss is 1.967390537261963
epoch: 1 step: 172, loss is 2.0299999713897705
epoch: 1 step: 173, loss is 1.9969732761383057
epoch: 1 step: 174, loss is 2.10209059715271
epoch: 1 step: 175, loss is 1.9025769233703613
epoch: 1 step: 176, loss is 1.992766261100769
epoch: 1 step: 177, loss is 1.973197340965271
epoch: 1 step: 178, loss is 1.9390497207641602
epoch: 1 step: 179, loss is 1.9551801681518555
epoch: 1 step: 180, loss is 1.9586472511291504
epoch: 1 step: 181, loss is 2.048654556274414
epoch: 1 step: 182, loss is 1.9481562376022339
epoch: 1 step: 183, loss is 1.86690092086792
epoch: 1 step: 184, loss is 2.179234743118286
epoch: 1 step: 185, loss is 1.9455556869506836
epoch: 1 step: 186, loss is 1.93514883518219
epoch: 1 step: 187, loss is 1.9482758045196533
epoch: 1 step: 188, loss is 2.045107364654541
epoch: 1 step: 189, loss is 1.9180610179901123
epoch: 1 step: 190, loss is 1.924575686454773
epoch: 1 step: 191, loss is 1.9538277387619019
epoch: 1 step: 192, loss is 2.053974151611328
epoch: 1 step: 193, loss is 1.8412437438964844
epoch: 1 step: 194, loss is 1.9784162044525146
epoch: 1 step: 195, loss is 2.0053610801696777
epoch: 1 step: 196, loss is 2.0205881595611572
epoch: 1 step: 197, loss is 1.862954020500183
epoch: 1 step: 198, loss is 1.9521949291229248
epoch: 1 step: 199, loss is 1.9148310422897339
epoch: 1 step: 200, loss is 1.9086284637451172
epoch: 1 step: 201, loss is 1.90412437915802
epoch: 1 step: 202, loss is 1.9184082746505737
epoch: 1 step: 203, loss is 1.9259778261184692
epoch: 1 step: 204, loss is 1.938686490058899
epoch: 1 step: 205, loss is 1.9550892114639282
epoch: 1 step: 206, loss is 1.9217603206634521
epoch: 1 step: 207, loss is 1.9987366199493408
epoch: 1 step: 208, loss is 1.8766148090362549
epoch: 1 step: 209, loss is 1.8825844526290894
epoch: 1 step: 210, loss is 2.003648281097412
epoch: 1 step: 211, loss is 1.848933219909668
epoch: 1 step: 212, loss is 1.8375006914138794
epoch: 1 step: 213, loss is 1.9735071659088135
epoch: 1 step: 214, loss is 1.8929799795150757
epoch: 1 step: 215, loss is 1.8706114292144775
epoch: 1 step: 216, loss is 1.8975063562393188
epoch: 1 step: 217, loss is 1.9572205543518066
epoch: 1 step: 218, loss is 1.942191243171692
epoch: 1 step: 219, loss is 1.9094858169555664
epoch: 1 step: 220, loss is 1.962411880493164
epoch: 1 step: 221, loss is 1.9891170263290405
epoch: 1 step: 222, loss is 1.8248779773712158
epoch: 1 step: 223, loss is 1.858994722366333
epoch: 1 step: 224, loss is 1.849124550819397
epoch: 1 step: 225, loss is 1.9346970319747925
epoch: 1 step: 226, loss is 1.826183557510376
epoch: 1 step: 227, loss is 1.9363635778427124
epoch: 1 step: 228, loss is 1.8561036586761475
epoch: 1 step: 229, loss is 1.9091678857803345
epoch: 1 step: 230, loss is 2.0114054679870605
epoch: 1 step: 231, loss is 1.9826502799987793
epoch: 1 step: 232, loss is 1.93659245967865
epoch: 1 step: 233, loss is 1.8998479843139648
epoch: 1 step: 234, loss is 1.9760888814926147
epoch: 1 step: 235, loss is 1.9100254774093628
epoch: 1 step: 236, loss is 2.006518840789795
epoch: 1 step: 237, loss is 1.9446240663528442
epoch: 1 step: 238, loss is 1.943949580192566
epoch: 1 step: 239, loss is 1.9257456064224243
epoch: 1 step: 240, loss is 1.9459772109985352
epoch: 1 step: 241, loss is 1.8472379446029663
epoch: 1 step: 242, loss is 1.8523591756820679
epoch: 1 step: 243, loss is 1.8806179761886597
epoch: 1 step: 244, loss is 1.9071741104125977
epoch: 1 step: 245, loss is 1.8732938766479492
epoch: 1 step: 246, loss is 1.8631165027618408
epoch: 1 step: 247, loss is 1.79851233959198
epoch: 1 step: 248, loss is 1.9632022380828857
epoch: 1 step: 249, loss is 1.895284652709961
epoch: 1 step: 250, loss is 1.8205516338348389
epoch: 1 step: 251, loss is 1.8841073513031006
epoch: 1 step: 252, loss is 1.992820382118225
epoch: 1 step: 253, loss is 1.882340669631958
epoch: 1 step: 254, loss is 1.9065485000610352
epoch: 1 step: 255, loss is 1.986517310142517
epoch: 1 step: 256, loss is 1.968342661857605
epoch: 1 step: 257, loss is 1.9735517501831055
epoch: 1 step: 258, loss is 1.8764246702194214
epoch: 1 step: 259, loss is 1.9203993082046509
epoch: 1 step: 260, loss is 1.9105170965194702
epoch: 1 step: 261, loss is 1.914762258529663
epoch: 1 step: 262, loss is 1.9228638410568237
epoch: 1 step: 263, loss is 1.8883980512619019
epoch: 1 step: 264, loss is 1.9406763315200806
epoch: 1 step: 265, loss is 1.9819233417510986
epoch: 1 step: 266, loss is 1.8144193887710571
epoch: 1 step: 267, loss is 1.9350824356079102
epoch: 1 step: 268, loss is 1.9042448997497559
epoch: 1 step: 269, loss is 1.8953170776367188
epoch: 1 step: 270, loss is 1.9287267923355103
epoch: 1 step: 271, loss is 1.9446814060211182
epoch: 1 step: 272, loss is 1.9106758832931519
epoch: 1 step: 273, loss is 1.9108705520629883
epoch: 1 step: 274, loss is 1.9561392068862915
epoch: 1 step: 275, loss is 1.8204010725021362
epoch: 1 step: 276, loss is 1.8781782388687134
epoch: 1 step: 277, loss is 1.8495337963104248
epoch: 1 step: 278, loss is 1.9355489015579224
epoch: 1 step: 279, loss is 1.9059829711914062
epoch: 1 step: 280, loss is 1.8078060150146484
epoch: 1 step: 281, loss is 1.8691047430038452
epoch: 1 step: 282, loss is 1.915346384048462
epoch: 1 step: 283, loss is 1.900369644165039
epoch: 1 step: 284, loss is 1.9481935501098633
epoch: 1 step: 285, loss is 1.8649606704711914
epoch: 1 step: 286, loss is 1.7809797525405884
epoch: 1 step: 287, loss is 1.828477144241333
epoch: 1 step: 288, loss is 1.8330905437469482
epoch: 1 step: 289, loss is 1.9231735467910767
epoch: 1 step: 290, loss is 1.8781447410583496
epoch: 1 step: 291, loss is 1.901713252067566
epoch: 1 step: 292, loss is 1.866323709487915
epoch: 1 step: 293, loss is 1.8305366039276123
epoch: 1 step: 294, loss is 1.9059938192367554
epoch: 1 step: 295, loss is 1.8912019729614258
epoch: 1 step: 296, loss is 1.8775914907455444
epoch: 1 step: 297, loss is 1.8697059154510498
epoch: 1 step: 298, loss is 1.7604823112487793
epoch: 1 step: 299, loss is 1.8278822898864746
epoch: 1 step: 300, loss is 1.8313566446304321
epoch: 1 step: 301, loss is 1.8744159936904907
epoch: 1 step: 302, loss is 1.8884289264678955
epoch: 1 step: 303, loss is 1.8936772346496582
epoch: 1 step: 304, loss is 1.884610652923584
epoch: 1 step: 305, loss is 1.7419513463974
epoch: 1 step: 306, loss is 1.8269364833831787
epoch: 1 step: 307, loss is 2.0380630493164062
epoch: 1 step: 308, loss is 1.7166903018951416
epoch: 1 step: 309, loss is 1.947814702987671
epoch: 1 step: 310, loss is 1.9079951047897339
epoch: 1 step: 311, loss is 1.9711205959320068
epoch: 1 step: 312, loss is 1.8932350873947144
epoch: 1 step: 313, loss is 1.953242301940918
epoch: 1 step: 314, loss is 1.9016802310943604
epoch: 1 step: 315, loss is 1.8064768314361572
epoch: 1 step: 316, loss is 2.02823543548584
epoch: 1 step: 317, loss is 1.9029710292816162
epoch: 1 step: 318, loss is 1.8800058364868164
epoch: 1 step: 319, loss is 1.956101417541504
epoch: 1 step: 320, loss is 1.8630518913269043
epoch: 1 step: 321, loss is 1.8017828464508057
epoch: 1 step: 322, loss is 1.8763279914855957
epoch: 1 step: 323, loss is 1.937215805053711
epoch: 1 step: 324, loss is 1.8784191608428955
epoch: 1 step: 325, loss is 1.910617470741272
epoch: 1 step: 326, loss is 1.9025781154632568
epoch: 1 step: 327, loss is 1.862633228302002
epoch: 1 step: 328, loss is 1.8413982391357422
epoch: 1 step: 329, loss is 1.7867395877838135
epoch: 1 step: 330, loss is 1.8464076519012451
epoch: 1 step: 331, loss is 1.8866784572601318
epoch: 1 step: 332, loss is 1.8521113395690918
epoch: 1 step: 333, loss is 1.897275686264038
epoch: 1 step: 334, loss is 1.892689824104309
epoch: 1 step: 335, loss is 1.8184235095977783
epoch: 1 step: 336, loss is 1.9274173974990845
epoch: 1 step: 337, loss is 1.8472647666931152
epoch: 1 step: 338, loss is 1.7978050708770752
epoch: 1 step: 339, loss is 1.8442535400390625
epoch: 1 step: 340, loss is 1.9170185327529907
epoch: 1 step: 341, loss is 1.9741616249084473
epoch: 1 step: 342, loss is 1.8357890844345093
epoch: 1 step: 343, loss is 1.8689477443695068
epoch: 1 step: 344, loss is 1.7794508934020996
epoch: 1 step: 345, loss is 1.9481403827667236
epoch: 1 step: 346, loss is 1.8743113279342651
epoch: 1 step: 347, loss is 1.9020572900772095
epoch: 1 step: 348, loss is 1.888008952140808
epoch: 1 step: 349, loss is 1.8339241743087769
epoch: 1 step: 350, loss is 1.8952176570892334
epoch: 1 step: 351, loss is 1.8237746953964233
epoch: 1 step: 352, loss is 1.827472448348999
epoch: 1 step: 353, loss is 1.7785557508468628
epoch: 1 step: 354, loss is 1.882948637008667
epoch: 1 step: 355, loss is 1.9324722290039062
epoch: 1 step: 356, loss is 1.8654732704162598
epoch: 1 step: 357, loss is 1.932706594467163
epoch: 1 step: 358, loss is 1.7482854127883911
epoch: 1 step: 359, loss is 1.7987877130508423
epoch: 1 step: 360, loss is 1.8794881105422974
epoch: 1 step: 361, loss is 1.9126574993133545
epoch: 1 step: 362, loss is 1.9920289516448975
epoch: 1 step: 363, loss is 1.8034894466400146
epoch: 1 step: 364, loss is 1.7955713272094727
epoch: 1 step: 365, loss is 1.8651440143585205
epoch: 1 step: 366, loss is 1.9597488641738892
epoch: 1 step: 367, loss is 1.8829628229141235
epoch: 1 step: 368, loss is 1.8391997814178467
epoch: 1 step: 369, loss is 1.939731478691101
epoch: 1 step: 370, loss is 1.8295228481292725
epoch: 1 step: 371, loss is 1.9174630641937256
epoch: 1 step: 372, loss is 1.9182922840118408
epoch: 1 step: 373, loss is 1.808217167854309
epoch: 1 step: 374, loss is 1.9264957904815674
epoch: 1 step: 375, loss is 1.7429237365722656
epoch: 1 step: 376, loss is 1.831039309501648
epoch: 1 step: 377, loss is 1.802515983581543
epoch: 1 step: 378, loss is 1.8956317901611328
epoch: 1 step: 379, loss is 1.9048898220062256
epoch: 1 step: 380, loss is 1.828395128250122
epoch: 1 step: 381, loss is 1.7988104820251465
epoch: 1 step: 382, loss is 1.8183889389038086
epoch: 1 step: 383, loss is 1.7738791704177856
epoch: 1 step: 384, loss is 1.9464951753616333
epoch: 1 step: 385, loss is 1.869751214981079
epoch: 1 step: 386, loss is 1.9628428220748901
epoch: 1 step: 387, loss is 1.9485324621200562
epoch: 1 step: 388, loss is 1.9883339405059814
epoch: 1 step: 389, loss is 1.8718982934951782
epoch: 1 step: 390, loss is 1.8623334169387817
Train epoch time: 559462.711 ms, per step time: 1434.520 ms
epoch: 2 step: 1, loss is 1.8421223163604736
epoch: 2 step: 2, loss is 2.008204698562622
epoch: 2 step: 3, loss is 1.9049326181411743
epoch: 2 step: 4, loss is 1.777031660079956
epoch: 2 step: 5, loss is 1.7815625667572021
epoch: 2 step: 6, loss is 1.911461353302002
epoch: 2 step: 7, loss is 1.7180767059326172
epoch: 2 step: 8, loss is 1.8715935945510864
epoch: 2 step: 9, loss is 1.903152585029602
epoch: 2 step: 10, loss is 1.8762656450271606
epoch: 2 step: 11, loss is 1.9111047983169556
epoch: 2 step: 12, loss is 1.81650972366333
epoch: 2 step: 13, loss is 1.8876910209655762
epoch: 2 step: 14, loss is 1.8165912628173828
epoch: 2 step: 15, loss is 1.7198039293289185
epoch: 2 step: 16, loss is 1.8527566194534302
epoch: 2 step: 17, loss is 1.8105731010437012
epoch: 2 step: 18, loss is 1.9975801706314087
epoch: 2 step: 19, loss is 1.7746944427490234
epoch: 2 step: 20, loss is 1.7947584390640259
epoch: 2 step: 21, loss is 1.7725882530212402
epoch: 2 step: 22, loss is 1.8505934476852417
epoch: 2 step: 23, loss is 1.8280901908874512
epoch: 2 step: 24, loss is 1.7068424224853516
epoch: 2 step: 25, loss is 1.8458136320114136
epoch: 2 step: 26, loss is 1.8540759086608887
epoch: 2 step: 27, loss is 1.8286898136138916
epoch: 2 step: 28, loss is 1.873181939125061
epoch: 2 step: 29, loss is 1.7826969623565674
epoch: 2 step: 30, loss is 1.902527928352356
epoch: 2 step: 31, loss is 1.8190808296203613
epoch: 2 step: 32, loss is 1.857375144958496
epoch: 2 step: 33, loss is 1.9656012058258057
epoch: 2 step: 34, loss is 1.7600538730621338
epoch: 2 step: 35, loss is 1.9101976156234741
epoch: 2 step: 36, loss is 1.8036359548568726
epoch: 2 step: 37, loss is 1.8577580451965332
epoch: 2 step: 38, loss is 1.871122121810913
epoch: 2 step: 39, loss is 1.9181914329528809
epoch: 2 step: 40, loss is 1.8528406620025635
epoch: 2 step: 41, loss is 1.8752368688583374
epoch: 2 step: 42, loss is 1.8753892183303833
epoch: 2 step: 43, loss is 1.798025369644165
epoch: 2 step: 44, loss is 1.8533127307891846
epoch: 2 step: 45, loss is 1.857120156288147
epoch: 2 step: 46, loss is 1.9277753829956055
epoch: 2 step: 47, loss is 1.8209806680679321
epoch: 2 step: 48, loss is 1.7573074102401733
epoch: 2 step: 49, loss is 1.8116599321365356
epoch: 2 step: 50, loss is 1.805864930152893
epoch: 2 step: 51, loss is 1.8290568590164185
epoch: 2 step: 52, loss is 1.866703748703003
epoch: 2 step: 53, loss is 1.7827622890472412
epoch: 2 step: 54, loss is 1.7083626985549927
epoch: 2 step: 55, loss is 1.8030028343200684
epoch: 2 step: 56, loss is 1.8389008045196533
epoch: 2 step: 57, loss is 1.7535532712936401
epoch: 2 step: 58, loss is 1.7774786949157715
epoch: 2 step: 59, loss is 1.8505266904830933
epoch: 2 step: 60, loss is 1.8493306636810303
epoch: 2 step: 61, loss is 1.7140759229660034
epoch: 2 step: 62, loss is 1.782009243965149
epoch: 2 step: 63, loss is 1.8617455959320068
epoch: 2 step: 64, loss is 1.8823177814483643
epoch: 2 step: 65, loss is 1.816063642501831
epoch: 2 step: 66, loss is 1.9122086763381958
epoch: 2 step: 67, loss is 1.8919082880020142
epoch: 2 step: 68, loss is 1.7285436391830444
epoch: 2 step: 69, loss is 1.8286607265472412
epoch: 2 step: 70, loss is 1.6725664138793945
epoch: 2 step: 71, loss is 1.8424677848815918
epoch: 2 step: 72, loss is 1.9370710849761963
epoch: 2 step: 73, loss is 1.663698434829712
epoch: 2 step: 74, loss is 1.7469241619110107
epoch: 2 step: 75, loss is 1.829254388809204
epoch: 2 step: 76, loss is 1.7615092992782593
epoch: 2 step: 77, loss is 1.8518040180206299
epoch: 2 step: 78, loss is 1.8412467241287231
epoch: 2 step: 79, loss is 1.7873300313949585
epoch: 2 step: 80, loss is 1.7855713367462158
epoch: 2 step: 81, loss is 1.8129326105117798
epoch: 2 step: 82, loss is 1.9314043521881104
epoch: 2 step: 83, loss is 1.7392871379852295
epoch: 2 step: 84, loss is 1.8102706670761108
epoch: 2 step: 85, loss is 1.8198387622833252
epoch: 2 step: 86, loss is 1.8012317419052124
epoch: 2 step: 87, loss is 1.8196797370910645
epoch: 2 step: 88, loss is 1.7679115533828735
epoch: 2 step: 89, loss is 1.7403090000152588
epoch: 2 step: 90, loss is 1.7996976375579834
epoch: 2 step: 91, loss is 1.795515537261963
epoch: 2 step: 92, loss is 1.7741966247558594
epoch: 2 step: 93, loss is 1.8286159038543701
epoch: 2 step: 94, loss is 1.7991762161254883
epoch: 2 step: 95, loss is 1.8904273509979248
epoch: 2 step: 96, loss is 1.7591018676757812
epoch: 2 step: 97, loss is 1.832050085067749
epoch: 2 step: 98, loss is 1.7773542404174805
epoch: 2 step: 99, loss is 1.7545019388198853
epoch: 2 step: 100, loss is 2.0094566345214844
epoch: 2 step: 101, loss is 1.8504834175109863
epoch: 2 step: 102, loss is 1.7474133968353271
epoch: 2 step: 103, loss is 1.7965986728668213
epoch: 2 step: 104, loss is 1.8871524333953857
epoch: 2 step: 105, loss is 1.8443228006362915
epoch: 2 step: 106, loss is 1.7581956386566162
epoch: 2 step: 107, loss is 1.953933835029602
epoch: 2 step: 108, loss is 1.8555583953857422
epoch: 2 step: 109, loss is 1.7896144390106201
epoch: 2 step: 110, loss is 1.9161076545715332
epoch: 2 step: 111, loss is 1.8588619232177734
epoch: 2 step: 112, loss is 1.7940638065338135
epoch: 2 step: 113, loss is 1.814422369003296
epoch: 2 step: 114, loss is 1.724690556526184
epoch: 2 step: 115, loss is 1.883571743965149
epoch: 2 step: 116, loss is 1.8918029069900513
epoch: 2 step: 117, loss is 1.8505539894104004
epoch: 2 step: 118, loss is 1.7429423332214355
epoch: 2 step: 119, loss is 1.8842040300369263
epoch: 2 step: 120, loss is 1.843445062637329
epoch: 2 step: 121, loss is 1.7528703212738037
epoch: 2 step: 122, loss is 1.8474390506744385
epoch: 2 step: 123, loss is 1.821091651916504
epoch: 2 step: 124, loss is 1.8275134563446045
epoch: 2 step: 125, loss is 1.7811741828918457
epoch: 2 step: 126, loss is 1.9219458103179932
epoch: 2 step: 127, loss is 1.8165922164916992
epoch: 2 step: 128, loss is 1.852981686592102
epoch: 2 step: 129, loss is 1.8071051836013794
epoch: 2 step: 130, loss is 1.7604596614837646
epoch: 2 step: 131, loss is 1.8261357545852661
epoch: 2 step: 132, loss is 1.8493021726608276
epoch: 2 step: 133, loss is 1.6813331842422485
epoch: 2 step: 134, loss is 1.8292278051376343
epoch: 2 step: 135, loss is 1.8004331588745117
epoch: 2 step: 136, loss is 1.854839563369751
epoch: 2 step: 137, loss is 1.7656934261322021
epoch: 2 step: 138, loss is 1.773084044456482
epoch: 2 step: 139, loss is 1.834776520729065
epoch: 2 step: 140, loss is 1.776870608329773
epoch: 2 step: 141, loss is 1.7940059900283813
epoch: 2 step: 142, loss is 1.798375129699707
epoch: 2 step: 143, loss is 1.8700058460235596
epoch: 2 step: 144, loss is 1.8540282249450684
epoch: 2 step: 145, loss is 1.842124581336975
epoch: 2 step: 146, loss is 1.8464956283569336
epoch: 2 step: 147, loss is 1.792346477508545
epoch: 2 step: 148, loss is 1.9074304103851318
epoch: 2 step: 149, loss is 1.8775198459625244
epoch: 2 step: 150, loss is 1.9219142198562622
epoch: 2 step: 151, loss is 1.771644949913025
epoch: 2 step: 152, loss is 1.832801342010498
epoch: 2 step: 153, loss is 1.806237816810608
epoch: 2 step: 154, loss is 1.8779716491699219
epoch: 2 step: 155, loss is 1.839617371559143
epoch: 2 step: 156, loss is 1.7294511795043945
epoch: 2 step: 157, loss is 1.885543942451477
epoch: 2 step: 158, loss is 1.7962372303009033
epoch: 2 step: 159, loss is 1.8337970972061157
epoch: 2 step: 160, loss is 1.6701412200927734
epoch: 2 step: 161, loss is 1.912917137145996
epoch: 2 step: 162, loss is 1.7153537273406982
epoch: 2 step: 163, loss is 1.8875322341918945
epoch: 2 step: 164, loss is 1.8108023405075073
epoch: 2 step: 165, loss is 1.8565192222595215
epoch: 2 step: 166, loss is 1.7654728889465332
epoch: 2 step: 167, loss is 1.819212794303894
epoch: 2 step: 168, loss is 1.7317806482315063
epoch: 2 step: 169, loss is 1.8797231912612915
epoch: 2 step: 170, loss is 1.8096650838851929
epoch: 2 step: 171, loss is 1.6969131231307983
epoch: 2 step: 172, loss is 1.782994031906128
epoch: 2 step: 173, loss is 1.774623990058899
epoch: 2 step: 174, loss is 1.918830394744873
epoch: 2 step: 175, loss is 1.8504277467727661
epoch: 2 step: 176, loss is 1.8835976123809814
epoch: 2 step: 177, loss is 1.7272708415985107
epoch: 2 step: 178, loss is 1.7170946598052979
epoch: 2 step: 179, loss is 1.7464662790298462
epoch: 2 step: 180, loss is 1.7861502170562744
epoch: 2 step: 181, loss is 1.7293448448181152
epoch: 2 step: 182, loss is 1.9008872509002686
epoch: 2 step: 183, loss is 1.746420979499817
epoch: 2 step: 184, loss is 1.834214448928833
epoch: 2 step: 185, loss is 1.7694340944290161
epoch: 2 step: 186, loss is 1.8437459468841553
epoch: 2 step: 187, loss is 1.8219029903411865
epoch: 2 step: 188, loss is 1.8351640701293945
epoch: 2 step: 189, loss is 1.782862663269043
epoch: 2 step: 190, loss is 1.85964035987854
epoch: 2 step: 191, loss is 1.8355700969696045
epoch: 2 step: 192, loss is 1.7351152896881104
epoch: 2 step: 193, loss is 1.7778615951538086
epoch: 2 step: 194, loss is 1.752778172492981
epoch: 2 step: 195, loss is 1.9030801057815552
epoch: 2 step: 196, loss is 1.8030743598937988
epoch: 2 step: 197, loss is 1.8280426263809204
epoch: 2 step: 198, loss is 1.77146577835083
epoch: 2 step: 199, loss is 1.7237201929092407
epoch: 2 step: 200, loss is 1.7591524124145508
epoch: 2 step: 201, loss is 1.727362871170044
epoch: 2 step: 202, loss is 1.8194643259048462
epoch: 2 step: 203, loss is 1.7553768157958984
epoch: 2 step: 204, loss is 1.8844705820083618
epoch: 2 step: 205, loss is 1.8579905033111572
epoch: 2 step: 206, loss is 1.8068547248840332
epoch: 2 step: 207, loss is 1.8319528102874756
epoch: 2 step: 208, loss is 1.7797279357910156
epoch: 2 step: 209, loss is 1.8424855470657349
epoch: 2 step: 210, loss is 1.7278661727905273
epoch: 2 step: 211, loss is 1.8495941162109375
epoch: 2 step: 212, loss is 1.7812793254852295
epoch: 2 step: 213, loss is 1.7848618030548096
epoch: 2 step: 214, loss is 1.766847848892212
epoch: 2 step: 215, loss is 1.6953535079956055
epoch: 2 step: 216, loss is 1.795626163482666
epoch: 2 step: 217, loss is 1.7648258209228516
epoch: 2 step: 218, loss is 1.7274357080459595
epoch: 2 step: 219, loss is 1.7118563652038574
epoch: 2 step: 220, loss is 1.8098434209823608
epoch: 2 step: 221, loss is 1.756893277168274
epoch: 2 step: 222, loss is 1.6547437906265259
epoch: 2 step: 223, loss is 1.752312421798706
epoch: 2 step: 224, loss is 1.7653849124908447
epoch: 2 step: 225, loss is 1.8978943824768066
epoch: 2 step: 226, loss is 1.7481962442398071
epoch: 2 step: 227, loss is 1.7389085292816162
epoch: 2 step: 228, loss is 1.6564011573791504
epoch: 2 step: 229, loss is 1.8569375276565552
epoch: 2 step: 230, loss is 1.8110934495925903
epoch: 2 step: 231, loss is 1.732248067855835
epoch: 2 step: 232, loss is 1.8179664611816406
epoch: 2 step: 233, loss is 1.734519362449646
epoch: 2 step: 234, loss is 1.7419822216033936
epoch: 2 step: 235, loss is 1.7818042039871216
epoch: 2 step: 236, loss is 1.7742573022842407
epoch: 2 step: 237, loss is 1.81847083568573
epoch: 2 step: 238, loss is 1.7013851404190063
epoch: 2 step: 239, loss is 1.7850755453109741
epoch: 2 step: 240, loss is 1.8341127634048462
epoch: 2 step: 241, loss is 1.7395367622375488
epoch: 2 step: 242, loss is 1.7220826148986816
epoch: 2 step: 243, loss is 1.796720027923584
epoch: 2 step: 244, loss is 1.758297324180603
epoch: 2 step: 245, loss is 1.7271897792816162
epoch: 2 step: 246, loss is 1.7630000114440918
epoch: 2 step: 247, loss is 1.7617006301879883
epoch: 2 step: 248, loss is 1.8541369438171387
epoch: 2 step: 249, loss is 1.845254898071289
epoch: 2 step: 250, loss is 1.8449022769927979
epoch: 2 step: 251, loss is 1.807826280593872
epoch: 2 step: 252, loss is 1.7920422554016113
epoch: 2 step: 253, loss is 1.8201287984848022
epoch: 2 step: 254, loss is 1.6775565147399902
epoch: 2 step: 255, loss is 1.6942179203033447
epoch: 2 step: 256, loss is 1.7701902389526367
epoch: 2 step: 257, loss is 1.9252827167510986
epoch: 2 step: 258, loss is 1.6032153367996216
epoch: 2 step: 259, loss is 1.7851173877716064
epoch: 2 step: 260, loss is 1.7023965120315552
epoch: 2 step: 261, loss is 1.730016827583313
epoch: 2 step: 262, loss is 1.8869608640670776
epoch: 2 step: 263, loss is 1.7207187414169312
epoch: 2 step: 264, loss is 1.7716741561889648
epoch: 2 step: 265, loss is 1.6237492561340332
epoch: 2 step: 266, loss is 1.6883153915405273
epoch: 2 step: 267, loss is 1.8093061447143555
epoch: 2 step: 268, loss is 1.764934778213501
epoch: 2 step: 269, loss is 1.9322458505630493
epoch: 2 step: 270, loss is 1.8510810136795044
epoch: 2 step: 271, loss is 1.758577585220337
epoch: 2 step: 272, loss is 1.7630869150161743
epoch: 2 step: 273, loss is 1.829620361328125
epoch: 2 step: 274, loss is 1.8193608522415161
epoch: 2 step: 275, loss is 1.8208897113800049
epoch: 2 step: 276, loss is 1.8448278903961182
epoch: 2 step: 277, loss is 1.8102002143859863
epoch: 2 step: 278, loss is 1.766446828842163
epoch: 2 step: 279, loss is 1.7395894527435303
epoch: 2 step: 280, loss is 1.9170942306518555
epoch: 2 step: 281, loss is 1.7587265968322754
epoch: 2 step: 282, loss is 1.7678029537200928
epoch: 2 step: 283, loss is 1.7849278450012207
epoch: 2 step: 284, loss is 1.8514670133590698
epoch: 2 step: 285, loss is 1.7459402084350586
epoch: 2 step: 286, loss is 1.7573826313018799
epoch: 2 step: 287, loss is 1.7697988748550415
epoch: 2 step: 288, loss is 1.8922491073608398
epoch: 2 step: 289, loss is 1.7866861820220947
epoch: 2 step: 290, loss is 1.7756916284561157
epoch: 2 step: 291, loss is 1.6631534099578857
epoch: 2 step: 292, loss is 1.792649745941162
epoch: 2 step: 293, loss is 1.8969694375991821
epoch: 2 step: 294, loss is 1.7774689197540283
epoch: 2 step: 295, loss is 1.774440050125122
epoch: 2 step: 296, loss is 1.9102789163589478
epoch: 2 step: 297, loss is 1.74527108669281
epoch: 2 step: 298, loss is 1.7999274730682373
epoch: 2 step: 299, loss is 1.8518040180206299
epoch: 2 step: 300, loss is 1.785047173500061
epoch: 2 step: 301, loss is 1.7671623229980469
epoch: 2 step: 302, loss is 1.7771998643875122
epoch: 2 step: 303, loss is 1.8071446418762207
epoch: 2 step: 304, loss is 1.745837926864624
epoch: 2 step: 305, loss is 1.7255862951278687
epoch: 2 step: 306, loss is 1.8061946630477905
epoch: 2 step: 307, loss is 1.782383918762207
epoch: 2 step: 308, loss is 1.733461618423462
epoch: 2 step: 309, loss is 1.7178516387939453
epoch: 2 step: 310, loss is 1.78062105178833
epoch: 2 step: 311, loss is 1.7163830995559692
epoch: 2 step: 312, loss is 1.7259130477905273
epoch: 2 step: 313, loss is 1.7228072881698608
epoch: 2 step: 314, loss is 1.757554531097412
epoch: 2 step: 315, loss is 1.7561191320419312
epoch: 2 step: 316, loss is 1.8935904502868652
epoch: 2 step: 317, loss is 1.8784563541412354
epoch: 2 step: 318, loss is 1.737724781036377
epoch: 2 step: 319, loss is 1.6668999195098877
epoch: 2 step: 320, loss is 1.709008812904358
epoch: 2 step: 321, loss is 1.7713356018066406
epoch: 2 step: 322, loss is 1.7318329811096191
epoch: 2 step: 323, loss is 1.8835827112197876
epoch: 2 step: 324, loss is 1.6565098762512207
epoch: 2 step: 325, loss is 1.831712245941162
epoch: 2 step: 326, loss is 1.6410236358642578
epoch: 2 step: 327, loss is 1.828751564025879
epoch: 2 step: 328, loss is 1.7768685817718506
epoch: 2 step: 329, loss is 1.7439451217651367
epoch: 2 step: 330, loss is 1.8685579299926758
epoch: 2 step: 331, loss is 1.7460261583328247
epoch: 2 step: 332, loss is 1.6164201498031616
epoch: 2 step: 333, loss is 1.7188907861709595
epoch: 2 step: 334, loss is 1.6516056060791016
epoch: 2 step: 335, loss is 1.7986702919006348
epoch: 2 step: 336, loss is 1.6903605461120605
epoch: 2 step: 337, loss is 1.8839318752288818
epoch: 2 step: 338, loss is 1.6306712627410889
epoch: 2 step: 339, loss is 1.7975196838378906
epoch: 2 step: 340, loss is 1.7304942607879639
epoch: 2 step: 341, loss is 1.738153100013733
epoch: 2 step: 342, loss is 1.7707561254501343
epoch: 2 step: 343, loss is 1.8245712518692017
epoch: 2 step: 344, loss is 1.808816909790039
epoch: 2 step: 345, loss is 1.8415197134017944
epoch: 2 step: 346, loss is 1.6351916790008545
epoch: 2 step: 347, loss is 1.8274368047714233
epoch: 2 step: 348, loss is 1.72337007522583
epoch: 2 step: 349, loss is 1.7387821674346924
epoch: 2 step: 350, loss is 1.6980061531066895
epoch: 2 step: 351, loss is 1.7490226030349731
epoch: 2 step: 352, loss is 1.7374050617218018
epoch: 2 step: 353, loss is 1.7092061042785645
epoch: 2 step: 354, loss is 1.825583577156067
epoch: 2 step: 355, loss is 1.6807430982589722
epoch: 2 step: 356, loss is 1.6132962703704834
epoch: 2 step: 357, loss is 1.7440011501312256
epoch: 2 step: 358, loss is 1.7009410858154297
epoch: 2 step: 359, loss is 1.7133073806762695
epoch: 2 step: 360, loss is 1.6671321392059326
epoch: 2 step: 361, loss is 1.715044617652893
epoch: 2 step: 362, loss is 1.7273805141448975
epoch: 2 step: 363, loss is 1.6790227890014648
epoch: 2 step: 364, loss is 1.8265048265457153
epoch: 2 step: 365, loss is 1.733555793762207
epoch: 2 step: 366, loss is 1.7168662548065186
epoch: 2 step: 367, loss is 1.6425013542175293
epoch: 2 step: 368, loss is 1.6609174013137817
epoch: 2 step: 369, loss is 1.7377312183380127
epoch: 2 step: 370, loss is 1.6808569431304932
epoch: 2 step: 371, loss is 1.6284534931182861
epoch: 2 step: 372, loss is 1.685543179512024
epoch: 2 step: 373, loss is 1.7767103910446167
epoch: 2 step: 374, loss is 1.7128162384033203
epoch: 2 step: 375, loss is 1.7695434093475342
epoch: 2 step: 376, loss is 1.7186996936798096
epoch: 2 step: 377, loss is 1.8189595937728882
epoch: 2 step: 378, loss is 1.8003448247909546
epoch: 2 step: 379, loss is 1.808626651763916
epoch: 2 step: 380, loss is 1.6892523765563965
epoch: 2 step: 381, loss is 1.7160866260528564
epoch: 2 step: 382, loss is 1.766904592514038
epoch: 2 step: 383, loss is 1.725840449333191
epoch: 2 step: 384, loss is 1.651390790939331
epoch: 2 step: 385, loss is 1.8010120391845703
epoch: 2 step: 386, loss is 1.7001240253448486
epoch: 2 step: 387, loss is 1.6718862056732178
epoch: 2 step: 388, loss is 1.7309114933013916
epoch: 2 step: 389, loss is 1.7690832614898682
epoch: 2 step: 390, loss is 1.705165147781372
Train epoch time: 162454.846 ms, per step time: 416.551 ms
epoch: 3 step: 1, loss is 1.738095998764038
epoch: 3 step: 2, loss is 1.6026263236999512
epoch: 3 step: 3, loss is 1.746885061264038
epoch: 3 step: 4, loss is 1.7208846807479858
epoch: 3 step: 5, loss is 1.6913771629333496
epoch: 3 step: 6, loss is 1.584436058998108
epoch: 3 step: 7, loss is 1.7271636724472046
epoch: 3 step: 8, loss is 1.770618200302124
epoch: 3 step: 9, loss is 1.69355046749115
epoch: 3 step: 10, loss is 1.6623616218566895
epoch: 3 step: 11, loss is 1.848590612411499
epoch: 3 step: 12, loss is 1.6417250633239746
epoch: 3 step: 13, loss is 1.700435996055603
epoch: 3 step: 14, loss is 1.6419398784637451
epoch: 3 step: 15, loss is 1.7566601037979126
epoch: 3 step: 16, loss is 1.7484650611877441
epoch: 3 step: 17, loss is 1.8431851863861084
epoch: 3 step: 18, loss is 1.7223873138427734
epoch: 3 step: 19, loss is 1.7862238883972168
epoch: 3 step: 20, loss is 1.6526002883911133
epoch: 3 step: 21, loss is 1.7336801290512085
epoch: 3 step: 22, loss is 1.7310236692428589
epoch: 3 step: 23, loss is 1.7317748069763184
epoch: 3 step: 24, loss is 1.676335096359253
epoch: 3 step: 25, loss is 1.6377067565917969
epoch: 3 step: 26, loss is 1.6067259311676025
epoch: 3 step: 27, loss is 1.8220208883285522
epoch: 3 step: 28, loss is 1.7839244604110718
epoch: 3 step: 29, loss is 1.7967554330825806
epoch: 3 step: 30, loss is 1.7714719772338867
epoch: 3 step: 31, loss is 1.744256615638733
epoch: 3 step: 32, loss is 1.646437644958496
epoch: 3 step: 33, loss is 1.5809506177902222
epoch: 3 step: 34, loss is 1.6778538227081299
epoch: 3 step: 35, loss is 1.836702585220337
epoch: 3 step: 36, loss is 1.8118674755096436
epoch: 3 step: 37, loss is 1.7432634830474854
epoch: 3 step: 38, loss is 1.758095622062683
epoch: 3 step: 39, loss is 1.7142406702041626
epoch: 3 step: 40, loss is 1.7229909896850586
epoch: 3 step: 41, loss is 1.6695988178253174
epoch: 3 step: 42, loss is 1.6950929164886475
epoch: 3 step: 43, loss is 1.7278995513916016
epoch: 3 step: 44, loss is 1.6640204191207886
epoch: 3 step: 45, loss is 1.7064404487609863
epoch: 3 step: 46, loss is 1.617141842842102
epoch: 3 step: 47, loss is 1.8440532684326172
epoch: 3 step: 48, loss is 1.6273651123046875
epoch: 3 step: 49, loss is 1.6873735189437866
epoch: 3 step: 50, loss is 1.6436452865600586
epoch: 3 step: 51, loss is 1.8179457187652588
epoch: 3 step: 52, loss is 1.659593105316162
epoch: 3 step: 53, loss is 1.8510398864746094
epoch: 3 step: 54, loss is 1.7512688636779785
epoch: 3 step: 55, loss is 1.6312587261199951
epoch: 3 step: 56, loss is 1.6792843341827393
epoch: 3 step: 57, loss is 1.7587475776672363
epoch: 3 step: 58, loss is 1.7010849714279175
epoch: 3 step: 59, loss is 1.7978241443634033
epoch: 3 step: 60, loss is 1.7860329151153564
epoch: 3 step: 61, loss is 1.7901135683059692
epoch: 3 step: 62, loss is 1.8012070655822754
epoch: 3 step: 63, loss is 1.6998785734176636
epoch: 3 step: 64, loss is 1.740113377571106
epoch: 3 step: 65, loss is 1.6761378049850464
epoch: 3 step: 66, loss is 1.8187235593795776
epoch: 3 step: 67, loss is 1.7870328426361084
epoch: 3 step: 68, loss is 1.7076168060302734
epoch: 3 step: 69, loss is 1.6348966360092163
epoch: 3 step: 70, loss is 1.59658682346344
epoch: 3 step: 71, loss is 1.6829596757888794
epoch: 3 step: 72, loss is 1.5759072303771973
epoch: 3 step: 73, loss is 1.6601812839508057
epoch: 3 step: 74, loss is 1.7609620094299316
epoch: 3 step: 75, loss is 1.7049903869628906
epoch: 3 step: 76, loss is 1.707920789718628
epoch: 3 step: 77, loss is 1.6902549266815186
epoch: 3 step: 78, loss is 1.6045417785644531
epoch: 3 step: 79, loss is 1.7730436325073242
epoch: 3 step: 80, loss is 1.5947760343551636
epoch: 3 step: 81, loss is 1.7298614978790283
epoch: 3 step: 82, loss is 1.7010252475738525
epoch: 3 step: 83, loss is 1.7518906593322754
epoch: 3 step: 84, loss is 1.6458005905151367
epoch: 3 step: 85, loss is 1.5622551441192627
epoch: 3 step: 86, loss is 1.6571518182754517
epoch: 3 step: 87, loss is 1.6532336473464966
epoch: 3 step: 88, loss is 1.7147893905639648
epoch: 3 step: 89, loss is 1.712319254875183
epoch: 3 step: 90, loss is 1.644642949104309
epoch: 3 step: 91, loss is 1.672706961631775
epoch: 3 step: 92, loss is 1.6807613372802734
epoch: 3 step: 93, loss is 1.7447028160095215
epoch: 3 step: 94, loss is 1.686726450920105
epoch: 3 step: 95, loss is 1.7170896530151367
epoch: 3 step: 96, loss is 1.7263619899749756
epoch: 3 step: 97, loss is 1.7746891975402832
epoch: 3 step: 98, loss is 1.7600597143173218
epoch: 3 step: 99, loss is 1.6943597793579102
epoch: 3 step: 100, loss is 1.6414724588394165
epoch: 3 step: 101, loss is 1.6134674549102783
epoch: 3 step: 102, loss is 1.6615732908248901
epoch: 3 step: 103, loss is 1.7061666250228882
epoch: 3 step: 104, loss is 1.725416898727417
epoch: 3 step: 105, loss is 1.7154662609100342
epoch: 3 step: 106, loss is 1.6252951622009277
epoch: 3 step: 107, loss is 1.6739877462387085
epoch: 3 step: 108, loss is 1.7048745155334473
epoch: 3 step: 109, loss is 1.9821857213974
epoch: 3 step: 110, loss is 1.7400168180465698
epoch: 3 step: 111, loss is 1.763228178024292
epoch: 3 step: 112, loss is 1.6733592748641968
epoch: 3 step: 113, loss is 1.7924097776412964
epoch: 3 step: 114, loss is 1.8139257431030273
epoch: 3 step: 115, loss is 1.8236865997314453
epoch: 3 step: 116, loss is 1.71184241771698
epoch: 3 step: 117, loss is 1.7690776586532593
epoch: 3 step: 118, loss is 1.7495472431182861
epoch: 3 step: 119, loss is 1.7343249320983887
epoch: 3 step: 120, loss is 1.8288263082504272
epoch: 3 step: 121, loss is 1.8411216735839844
epoch: 3 step: 122, loss is 1.840151071548462
epoch: 3 step: 123, loss is 1.6688729524612427
epoch: 3 step: 124, loss is 1.758663296699524
epoch: 3 step: 125, loss is 1.7347440719604492
epoch: 3 step: 126, loss is 1.7782771587371826
epoch: 3 step: 127, loss is 1.7728439569473267
epoch: 3 step: 128, loss is 1.7996851205825806
epoch: 3 step: 129, loss is 1.6101486682891846
epoch: 3 step: 130, loss is 1.7656822204589844
epoch: 3 step: 131, loss is 1.647795557975769
epoch: 3 step: 132, loss is 1.6030256748199463
epoch: 3 step: 133, loss is 1.5469036102294922
epoch: 3 step: 134, loss is 1.739704966545105
epoch: 3 step: 135, loss is 1.680915117263794
epoch: 3 step: 136, loss is 1.6589123010635376
epoch: 3 step: 137, loss is 1.6818794012069702
epoch: 3 step: 138, loss is 1.7845594882965088
epoch: 3 step: 139, loss is 1.7386850118637085
epoch: 3 step: 140, loss is 1.596389889717102
epoch: 3 step: 141, loss is 1.6886796951293945
epoch: 3 step: 142, loss is 1.680829644203186
epoch: 3 step: 143, loss is 1.704711675643921
epoch: 3 step: 144, loss is 1.767851710319519
epoch: 3 step: 145, loss is 1.6414270401000977
epoch: 3 step: 146, loss is 1.7168920040130615
epoch: 3 step: 147, loss is 1.744308590888977
epoch: 3 step: 148, loss is 1.6562162637710571
epoch: 3 step: 149, loss is 1.621140718460083
epoch: 3 step: 150, loss is 1.7402429580688477
epoch: 3 step: 151, loss is 1.6760350465774536
epoch: 3 step: 152, loss is 1.7514678239822388
epoch: 3 step: 153, loss is 1.70414137840271
epoch: 3 step: 154, loss is 1.7646489143371582
epoch: 3 step: 155, loss is 1.6762090921401978
epoch: 3 step: 156, loss is 1.6464800834655762
epoch: 3 step: 157, loss is 1.6799201965332031
epoch: 3 step: 158, loss is 1.6853820085525513
epoch: 3 step: 159, loss is 1.8027880191802979
epoch: 3 step: 160, loss is 1.647753119468689
epoch: 3 step: 161, loss is 1.6420209407806396
epoch: 3 step: 162, loss is 1.6990537643432617
epoch: 3 step: 163, loss is 1.8327109813690186
epoch: 3 step: 164, loss is 1.849912405014038
epoch: 3 step: 165, loss is 1.7018146514892578
epoch: 3 step: 166, loss is 1.749741554260254
epoch: 3 step: 167, loss is 1.6981201171875
epoch: 3 step: 168, loss is 1.7325869798660278
epoch: 3 step: 169, loss is 1.777476191520691
epoch: 3 step: 170, loss is 1.6686164140701294
epoch: 3 step: 171, loss is 1.838793396949768
epoch: 3 step: 172, loss is 1.7133294343948364
epoch: 3 step: 173, loss is 1.6628972291946411
epoch: 3 step: 174, loss is 1.7172584533691406
epoch: 3 step: 175, loss is 1.6457676887512207
epoch: 3 step: 176, loss is 1.6863107681274414
epoch: 3 step: 177, loss is 1.6602391004562378
epoch: 3 step: 179, loss is 1.601531744003296
epoch: 3 step: 180, loss is 1.6575745344161987
epoch: 3 step: 181, loss is 1.7344472408294678
epoch: 3 step: 182, loss is 1.724416732788086
epoch: 3 step: 183, loss is 1.6952766180038452
epoch: 3 step: 184, loss is 1.647831678390503
epoch: 3 step: 185, loss is 1.7156965732574463
epoch: 3 step: 186, loss is 1.6974105834960938
epoch: 3 step: 187, loss is 1.7163755893707275
epoch: 3 step: 188, loss is 1.711667776107788
epoch: 3 step: 189, loss is 1.641359567642212
epoch: 3 step: 190, loss is 1.7294281721115112
epoch: 3 step: 191, loss is 1.639695167541504
epoch: 3 step: 192, loss is 1.8193395137786865
epoch: 3 step: 193, loss is 1.7793635129928589
epoch: 3 step: 194, loss is 1.650315523147583
epoch: 3 step: 195, loss is 1.7062029838562012
epoch: 3 step: 196, loss is 1.7141425609588623
epoch: 3 step: 197, loss is 1.5884010791778564
epoch: 3 step: 198, loss is 1.665992259979248
epoch: 3 step: 199, loss is 1.6755346059799194
epoch: 3 step: 200, loss is 1.635117530822754
epoch: 3 step: 201, loss is 1.8569953441619873
epoch: 3 step: 202, loss is 1.5871598720550537
epoch: 3 step: 203, loss is 1.6321712732315063
epoch: 3 step: 204, loss is 1.665245771408081
epoch: 3 step: 205, loss is 1.7415229082107544
epoch: 3 step: 206, loss is 1.612705111503601
epoch: 3 step: 207, loss is 1.6181538105010986
epoch: 3 step: 208, loss is 1.7484092712402344
epoch: 3 step: 209, loss is 1.5976970195770264
epoch: 3 step: 210, loss is 1.6238635778427124
epoch: 3 step: 211, loss is 1.6373547315597534
epoch: 3 step: 212, loss is 1.7454135417938232
epoch: 3 step: 213, loss is 1.7687944173812866
epoch: 3 step: 214, loss is 1.7772769927978516
epoch: 3 step: 215, loss is 1.8171579837799072
epoch: 3 step: 216, loss is 1.7133119106292725
epoch: 3 step: 217, loss is 1.8200221061706543
epoch: 3 step: 218, loss is 1.7421140670776367
epoch: 3 step: 219, loss is 1.7156667709350586
epoch: 3 step: 220, loss is 1.8118493556976318
epoch: 3 step: 221, loss is 1.738703727722168
epoch: 3 step: 222, loss is 1.704439401626587
epoch: 3 step: 223, loss is 1.7708739042282104
epoch: 3 step: 224, loss is 1.7138631343841553
epoch: 3 step: 225, loss is 1.5884580612182617
epoch: 3 step: 226, loss is 1.7187530994415283
epoch: 3 step: 227, loss is 1.7776721715927124
epoch: 3 step: 228, loss is 1.6584092378616333
epoch: 3 step: 229, loss is 1.6539154052734375
epoch: 3 step: 230, loss is 1.6891844272613525
epoch: 3 step: 231, loss is 1.6833837032318115
epoch: 3 step: 232, loss is 1.86319899559021
epoch: 3 step: 233, loss is 1.699632167816162
epoch: 3 step: 234, loss is 1.6552497148513794
epoch: 3 step: 235, loss is 1.653317928314209
epoch: 3 step: 236, loss is 1.7271509170532227
epoch: 3 step: 237, loss is 1.6361708641052246
epoch: 3 step: 238, loss is 1.6549464464187622
epoch: 3 step: 239, loss is 1.6872869729995728
epoch: 3 step: 240, loss is 1.7309738397598267
epoch: 3 step: 241, loss is 1.6400158405303955
epoch: 3 step: 242, loss is 1.673208236694336
epoch: 3 step: 243, loss is 1.5568699836730957
epoch: 3 step: 244, loss is 1.7471016645431519
epoch: 3 step: 245, loss is 1.695371150970459
epoch: 3 step: 246, loss is 1.6040992736816406
epoch: 3 step: 247, loss is 1.6593358516693115
epoch: 3 step: 248, loss is 1.587714433670044
epoch: 3 step: 249, loss is 1.580317497253418
epoch: 3 step: 250, loss is 1.711094617843628
epoch: 3 step: 251, loss is 1.7993535995483398
epoch: 3 step: 252, loss is 1.628434658050537
epoch: 3 step: 253, loss is 1.6860698461532593
epoch: 3 step: 254, loss is 1.6765542030334473
epoch: 3 step: 255, loss is 1.72957444190979
epoch: 3 step: 256, loss is 1.700048565864563
epoch: 3 step: 257, loss is 1.6036372184753418
epoch: 3 step: 258, loss is 1.7142815589904785
epoch: 3 step: 259, loss is 1.6210840940475464
epoch: 3 step: 260, loss is 1.7811002731323242
epoch: 3 step: 261, loss is 1.7559787034988403
epoch: 3 step: 262, loss is 1.7547873258590698
epoch: 3 step: 263, loss is 1.6555132865905762
epoch: 3 step: 264, loss is 1.593596339225769
epoch: 3 step: 265, loss is 1.6256158351898193
epoch: 3 step: 266, loss is 1.7168755531311035
epoch: 3 step: 267, loss is 1.722985029220581
epoch: 3 step: 268, loss is 1.756300687789917
epoch: 3 step: 269, loss is 1.7009131908416748
epoch: 3 step: 270, loss is 1.646615743637085
epoch: 3 step: 271, loss is 1.6153324842453003
epoch: 3 step: 272, loss is 1.7144765853881836
epoch: 3 step: 273, loss is 1.6784484386444092
epoch: 3 step: 274, loss is 1.737475037574768
epoch: 3 step: 275, loss is 1.649951457977295
epoch: 3 step: 276, loss is 1.75540030002594
epoch: 3 step: 277, loss is 1.7791203260421753
epoch: 3 step: 278, loss is 1.5753741264343262
epoch: 3 step: 279, loss is 1.6886436939239502
epoch: 3 step: 280, loss is 1.6846976280212402
epoch: 3 step: 281, loss is 1.6107654571533203
epoch: 3 step: 282, loss is 1.699042797088623
epoch: 3 step: 283, loss is 1.6061090230941772
epoch: 3 step: 284, loss is 1.6857482194900513
epoch: 3 step: 285, loss is 1.608324408531189
epoch: 3 step: 286, loss is 1.8041796684265137
epoch: 3 step: 287, loss is 1.587498426437378
epoch: 3 step: 288, loss is 1.6834648847579956
epoch: 3 step: 289, loss is 1.6691396236419678
epoch: 3 step: 290, loss is 1.5802768468856812
epoch: 3 step: 291, loss is 1.6977041959762573
epoch: 3 step: 292, loss is 1.523406744003296
epoch: 3 step: 293, loss is 1.636469841003418
epoch: 3 step: 294, loss is 1.6379210948944092
epoch: 3 step: 295, loss is 1.5465953350067139
epoch: 3 step: 296, loss is 1.6533712148666382
epoch: 3 step: 297, loss is 1.7809641361236572
epoch: 3 step: 298, loss is 1.6230101585388184
epoch: 3 step: 299, loss is 1.6952733993530273
epoch: 3 step: 300, loss is 1.7738311290740967
epoch: 3 step: 301, loss is 1.6317906379699707
epoch: 3 step: 302, loss is 1.556731104850769
epoch: 3 step: 303, loss is 1.6239100694656372
epoch: 3 step: 304, loss is 1.6711719036102295
epoch: 3 step: 305, loss is 1.6111335754394531
epoch: 3 step: 306, loss is 1.5570077896118164
epoch: 3 step: 307, loss is 1.7158706188201904
epoch: 3 step: 308, loss is 1.7499061822891235
epoch: 3 step: 309, loss is 1.7530311346054077
epoch: 3 step: 310, loss is 1.6949537992477417
epoch: 3 step: 311, loss is 1.7187061309814453
epoch: 3 step: 312, loss is 1.6041895151138306
epoch: 3 step: 313, loss is 1.6203274726867676
epoch: 3 step: 314, loss is 1.565751552581787
epoch: 3 step: 315, loss is 1.7410955429077148
epoch: 3 step: 316, loss is 1.6605002880096436
epoch: 3 step: 317, loss is 1.5867042541503906
epoch: 3 step: 318, loss is 1.7341188192367554
epoch: 3 step: 319, loss is 1.7926568984985352
epoch: 3 step: 320, loss is 1.7438678741455078
epoch: 3 step: 321, loss is 1.6300685405731201
epoch: 3 step: 322, loss is 1.6619653701782227
epoch: 3 step: 323, loss is 1.6554431915283203
epoch: 3 step: 324, loss is 1.6898455619812012
epoch: 3 step: 325, loss is 1.6405136585235596
epoch: 3 step: 326, loss is 1.735649824142456
epoch: 3 step: 327, loss is 1.6633617877960205
epoch: 3 step: 328, loss is 1.645141839981079
epoch: 3 step: 329, loss is 1.7677428722381592
epoch: 3 step: 330, loss is 1.6011266708374023
epoch: 3 step: 331, loss is 1.5936017036437988
epoch: 3 step: 332, loss is 1.7715740203857422
epoch: 3 step: 333, loss is 1.657118797302246
epoch: 3 step: 334, loss is 1.6588845252990723
epoch: 3 step: 335, loss is 1.7063355445861816
epoch: 3 step: 336, loss is 1.645137906074524
epoch: 3 step: 337, loss is 1.7760907411575317
epoch: 3 step: 338, loss is 1.714897871017456
epoch: 3 step: 339, loss is 1.5961012840270996
epoch: 3 step: 340, loss is 1.5920907258987427
epoch: 3 step: 341, loss is 1.6564875841140747
epoch: 3 step: 342, loss is 1.672626256942749
epoch: 3 step: 343, loss is 1.819559931755066
epoch: 3 step: 344, loss is 1.6594359874725342
epoch: 3 step: 345, loss is 1.6318385601043701
epoch: 3 step: 346, loss is 1.6468276977539062
epoch: 3 step: 347, loss is 1.5624014139175415
epoch: 3 step: 348, loss is 1.7218772172927856
epoch: 3 step: 349, loss is 1.7989740371704102
epoch: 3 step: 350, loss is 1.5650298595428467
epoch: 3 step: 351, loss is 1.718530535697937
epoch: 3 step: 352, loss is 1.6715630292892456
epoch: 3 step: 353, loss is 1.6873823404312134
epoch: 3 step: 354, loss is 1.523467779159546
epoch: 3 step: 355, loss is 1.773552656173706
epoch: 3 step: 356, loss is 1.7193902730941772
epoch: 3 step: 357, loss is 1.5947933197021484
epoch: 3 step: 358, loss is 1.675398588180542
epoch: 3 step: 359, loss is 1.5624616146087646
epoch: 3 step: 360, loss is 1.643164873123169
epoch: 3 step: 361, loss is 1.7025907039642334
epoch: 3 step: 362, loss is 1.670424461364746
epoch: 3 step: 363, loss is 1.5820398330688477
epoch: 3 step: 364, loss is 1.587092638015747
epoch: 3 step: 365, loss is 1.651233434677124
epoch: 3 step: 366, loss is 1.6764097213745117
epoch: 3 step: 367, loss is 1.7567121982574463
epoch: 3 step: 368, loss is 1.647390365600586
epoch: 3 step: 369, loss is 1.6315557956695557
epoch: 3 step: 370, loss is 1.5873017311096191
epoch: 3 step: 371, loss is 1.592962622642517
epoch: 3 step: 372, loss is 1.5444129705429077
epoch: 3 step: 373, loss is 1.60199773311615
epoch: 3 step: 374, loss is 1.5700989961624146
epoch: 3 step: 375, loss is 1.7058610916137695
epoch: 3 step: 376, loss is 1.637495994567871
epoch: 3 step: 377, loss is 1.5588295459747314
epoch: 3 step: 378, loss is 1.6284081935882568
epoch: 3 step: 379, loss is 1.6830811500549316
epoch: 3 step: 380, loss is 1.498645305633545
epoch: 3 step: 381, loss is 1.69361412525177
epoch: 3 step: 382, loss is 1.7078032493591309
epoch: 3 step: 383, loss is 1.6305315494537354
epoch: 3 step: 384, loss is 1.7470357418060303
epoch: 3 step: 385, loss is 1.7281711101531982
epoch: 3 step: 386, loss is 1.5832252502441406
epoch: 3 step: 387, loss is 1.7049098014831543
epoch: 3 step: 388, loss is 1.5404547452926636
epoch: 3 step: 389, loss is 1.6139895915985107
epoch: 3 step: 390, loss is 1.6153689622879028
Train epoch time: 162764.853 ms, per step time: 417.346 ms
epoch: 4 step: 1, loss is 1.707460641860962
epoch: 4 step: 2, loss is 1.6592729091644287
epoch: 4 step: 3, loss is 1.64607572555542
epoch: 4 step: 4, loss is 1.5657050609588623
epoch: 4 step: 5, loss is 1.607945203781128
epoch: 4 step: 6, loss is 1.6521191596984863
epoch: 4 step: 7, loss is 1.7257267236709595
epoch: 4 step: 8, loss is 1.6123642921447754
epoch: 4 step: 9, loss is 1.521589994430542
epoch: 4 step: 10, loss is 1.6218183040618896
epoch: 4 step: 11, loss is 1.5593960285186768
epoch: 4 step: 12, loss is 1.585852861404419
epoch: 4 step: 13, loss is 1.7191182374954224
epoch: 4 step: 14, loss is 1.8057851791381836
epoch: 4 step: 15, loss is 1.6523622274398804
epoch: 4 step: 16, loss is 1.5024642944335938
epoch: 4 step: 17, loss is 1.5328121185302734
epoch: 4 step: 18, loss is 1.7459771633148193
epoch: 4 step: 19, loss is 1.6062031984329224
epoch: 4 step: 20, loss is 1.6203831434249878
epoch: 4 step: 21, loss is 1.709827184677124
epoch: 4 step: 22, loss is 1.6938754320144653
epoch: 4 step: 23, loss is 1.5854672193527222
epoch: 4 step: 24, loss is 1.6498934030532837
epoch: 4 step: 25, loss is 1.7277477979660034
epoch: 4 step: 26, loss is 1.5943411588668823
epoch: 4 step: 27, loss is 1.5103482007980347
epoch: 4 step: 28, loss is 1.5357271432876587
epoch: 4 step: 29, loss is 1.5766271352767944
epoch: 4 step: 30, loss is 1.7682290077209473
epoch: 4 step: 31, loss is 1.7368242740631104
epoch: 4 step: 32, loss is 1.6021848917007446
epoch: 4 step: 33, loss is 1.6342494487762451
epoch: 4 step: 34, loss is 1.5612132549285889
epoch: 4 step: 35, loss is 1.6568405628204346
epoch: 4 step: 36, loss is 1.6905531883239746
epoch: 4 step: 37, loss is 1.6610316038131714
epoch: 4 step: 38, loss is 1.653122901916504
epoch: 4 step: 39, loss is 1.6279942989349365
epoch: 4 step: 40, loss is 1.646303653717041
epoch: 4 step: 41, loss is 1.562235713005066
epoch: 4 step: 42, loss is 1.650357723236084
epoch: 4 step: 43, loss is 1.6317180395126343
epoch: 4 step: 44, loss is 1.6193413734436035
epoch: 4 step: 45, loss is 1.6614395380020142
epoch: 4 step: 46, loss is 1.6185563802719116
epoch: 4 step: 47, loss is 1.6294162273406982
epoch: 4 step: 48, loss is 1.792521357536316
epoch: 4 step: 49, loss is 1.7450298070907593
epoch: 4 step: 50, loss is 1.6184372901916504
epoch: 4 step: 51, loss is 1.7121673822402954
epoch: 4 step: 52, loss is 1.5947602987289429
epoch: 4 step: 53, loss is 1.6096045970916748
epoch: 4 step: 54, loss is 1.7267335653305054
epoch: 4 step: 55, loss is 1.6439011096954346
epoch: 4 step: 56, loss is 1.7128099203109741
epoch: 4 step: 57, loss is 1.6868995428085327
epoch: 4 step: 58, loss is 1.5768803358078003
epoch: 4 step: 59, loss is 1.4636682271957397
epoch: 4 step: 60, loss is 1.7803531885147095
epoch: 4 step: 61, loss is 1.6169931888580322
epoch: 4 step: 62, loss is 1.4855008125305176
epoch: 4 step: 63, loss is 1.4968881607055664
epoch: 4 step: 64, loss is 1.7545281648635864
epoch: 4 step: 65, loss is 1.7459375858306885
epoch: 4 step: 66, loss is 1.806571364402771
epoch: 4 step: 67, loss is 1.6289211511611938
epoch: 4 step: 68, loss is 1.6332566738128662
epoch: 4 step: 69, loss is 1.5650227069854736
epoch: 4 step: 70, loss is 1.60227370262146
epoch: 4 step: 71, loss is 1.6477112770080566
epoch: 4 step: 72, loss is 1.7204809188842773
epoch: 4 step: 73, loss is 1.5354769229888916
epoch: 4 step: 74, loss is 1.646813988685608
epoch: 4 step: 75, loss is 1.6883156299591064
epoch: 4 step: 76, loss is 1.53816819190979
epoch: 4 step: 77, loss is 1.6474199295043945
epoch: 4 step: 78, loss is 1.5403070449829102
epoch: 4 step: 79, loss is 1.684320330619812
epoch: 4 step: 80, loss is 1.5934255123138428
epoch: 4 step: 81, loss is 1.6041245460510254
epoch: 4 step: 82, loss is 1.6736280918121338
epoch: 4 step: 83, loss is 1.6636868715286255
epoch: 4 step: 84, loss is 1.7006218433380127
epoch: 4 step: 85, loss is 1.6146340370178223
epoch: 4 step: 86, loss is 1.5695500373840332
epoch: 4 step: 87, loss is 1.6297898292541504
epoch: 4 step: 88, loss is 1.599469542503357
epoch: 4 step: 89, loss is 1.5652307271957397
epoch: 4 step: 90, loss is 1.5440871715545654
epoch: 4 step: 91, loss is 1.6689234972000122
epoch: 4 step: 92, loss is 1.5320390462875366
epoch: 4 step: 93, loss is 1.6042300462722778
epoch: 4 step: 94, loss is 1.6973090171813965
epoch: 4 step: 95, loss is 1.5756487846374512
epoch: 4 step: 96, loss is 1.581540822982788
epoch: 4 step: 97, loss is 1.72627592086792
epoch: 4 step: 98, loss is 1.5987930297851562
epoch: 4 step: 99, loss is 1.562492847442627
epoch: 4 step: 100, loss is 1.5926721096038818
epoch: 4 step: 101, loss is 1.5996012687683105
epoch: 4 step: 102, loss is 1.6786695718765259
epoch: 4 step: 103, loss is 1.656341791152954
epoch: 4 step: 104, loss is 1.6191314458847046
epoch: 4 step: 105, loss is 1.6860291957855225
epoch: 4 step: 106, loss is 1.557091236114502
epoch: 4 step: 107, loss is 1.5620195865631104
epoch: 4 step: 108, loss is 1.6631333827972412
epoch: 4 step: 109, loss is 1.683093547821045
epoch: 4 step: 110, loss is 1.7273263931274414
epoch: 4 step: 111, loss is 1.5910454988479614
epoch: 4 step: 112, loss is 1.7791370153427124
epoch: 4 step: 113, loss is 1.563529133796692
epoch: 4 step: 114, loss is 1.5959031581878662
epoch: 4 step: 115, loss is 1.6265878677368164
epoch: 4 step: 116, loss is 1.5800020694732666
epoch: 4 step: 117, loss is 1.6125527620315552
epoch: 4 step: 118, loss is 1.7451469898223877
epoch: 4 step: 119, loss is 1.50901460647583
epoch: 4 step: 120, loss is 1.536970615386963
epoch: 4 step: 121, loss is 1.6209925413131714
epoch: 4 step: 122, loss is 1.5971770286560059
epoch: 4 step: 123, loss is 1.592236042022705
epoch: 4 step: 124, loss is 1.6269769668579102
epoch: 4 step: 125, loss is 1.559425950050354
epoch: 4 step: 126, loss is 1.5998713970184326
epoch: 4 step: 127, loss is 1.6915485858917236
epoch: 4 step: 128, loss is 1.5555686950683594
epoch: 4 step: 129, loss is 1.5827350616455078
epoch: 4 step: 130, loss is 1.4913427829742432
epoch: 4 step: 131, loss is 1.5756770372390747
epoch: 4 step: 132, loss is 1.5569010972976685
epoch: 4 step: 133, loss is 1.584458827972412
epoch: 4 step: 134, loss is 1.663841724395752
epoch: 4 step: 135, loss is 1.7493020296096802
epoch: 4 step: 136, loss is 1.7049176692962646
epoch: 4 step: 137, loss is 1.6486986875534058
epoch: 4 step: 138, loss is 1.7245224714279175
epoch: 4 step: 139, loss is 1.582345962524414
epoch: 4 step: 140, loss is 1.645918607711792
epoch: 4 step: 141, loss is 1.5924559831619263
epoch: 4 step: 142, loss is 1.5649745464324951
epoch: 4 step: 143, loss is 1.550689935684204
epoch: 4 step: 144, loss is 1.6572786569595337
epoch: 4 step: 145, loss is 1.643326759338379
epoch: 4 step: 146, loss is 1.6667256355285645
epoch: 4 step: 147, loss is 1.477785587310791
epoch: 4 step: 148, loss is 1.5880389213562012
epoch: 4 step: 149, loss is 1.543471336364746
epoch: 4 step: 150, loss is 1.5208349227905273
epoch: 4 step: 151, loss is 1.5891993045806885
epoch: 4 step: 152, loss is 1.577231764793396
epoch: 4 step: 153, loss is 1.6796469688415527
epoch: 4 step: 154, loss is 1.5086443424224854
epoch: 4 step: 155, loss is 1.5957432985305786
epoch: 4 step: 156, loss is 1.721947431564331
epoch: 4 step: 157, loss is 1.610168218612671
epoch: 4 step: 158, loss is 1.6663365364074707
epoch: 4 step: 159, loss is 1.5474095344543457
epoch: 4 step: 160, loss is 1.5565019845962524
epoch: 4 step: 161, loss is 1.545720100402832
epoch: 4 step: 162, loss is 1.6591174602508545
epoch: 4 step: 163, loss is 1.4846267700195312
epoch: 4 step: 164, loss is 1.6763046979904175
epoch: 4 step: 165, loss is 1.5621000528335571
epoch: 4 step: 166, loss is 1.577483057975769
epoch: 4 step: 167, loss is 1.7710967063903809
epoch: 4 step: 168, loss is 1.6637777090072632
epoch: 4 step: 169, loss is 1.7688608169555664
epoch: 4 step: 170, loss is 1.6116036176681519
epoch: 4 step: 171, loss is 1.6401437520980835
epoch: 4 step: 172, loss is 1.5993880033493042
epoch: 4 step: 173, loss is 1.6015052795410156
epoch: 4 step: 174, loss is 1.6603703498840332
epoch: 4 step: 175, loss is 1.7026708126068115
epoch: 4 step: 176, loss is 1.6939964294433594
epoch: 4 step: 177, loss is 1.5786694288253784
epoch: 4 step: 178, loss is 1.630711317062378
epoch: 4 step: 179, loss is 1.6390923261642456
epoch: 4 step: 180, loss is 1.518524408340454
epoch: 4 step: 181, loss is 1.7105324268341064
epoch: 4 step: 182, loss is 1.6739760637283325
epoch: 4 step: 183, loss is 1.7299777269363403
epoch: 4 step: 184, loss is 1.5618617534637451
epoch: 4 step: 185, loss is 1.608803629875183
epoch: 4 step: 186, loss is 1.5548045635223389
epoch: 4 step: 187, loss is 1.5411171913146973
epoch: 4 step: 188, loss is 1.5560667514801025
epoch: 4 step: 189, loss is 1.6400456428527832
epoch: 4 step: 190, loss is 1.609632134437561
epoch: 4 step: 191, loss is 1.6996359825134277
epoch: 4 step: 192, loss is 1.6475542783737183
epoch: 4 step: 193, loss is 1.5726449489593506
epoch: 4 step: 194, loss is 1.6745482683181763
epoch: 4 step: 195, loss is 1.7156413793563843
epoch: 4 step: 196, loss is 1.6187055110931396
epoch: 4 step: 197, loss is 1.6842145919799805
epoch: 4 step: 198, loss is 1.560021996498108
epoch: 4 step: 199, loss is 1.6388092041015625
epoch: 4 step: 200, loss is 1.5611517429351807
epoch: 4 step: 201, loss is 1.640512466430664
epoch: 4 step: 202, loss is 1.574932336807251
epoch: 4 step: 203, loss is 1.6864540576934814
epoch: 4 step: 204, loss is 1.5897661447525024
epoch: 4 step: 205, loss is 1.639681100845337
epoch: 4 step: 206, loss is 1.6151649951934814
epoch: 4 step: 207, loss is 1.701015830039978
epoch: 4 step: 208, loss is 1.5985267162322998
epoch: 4 step: 209, loss is 1.6025310754776
epoch: 4 step: 210, loss is 1.5729660987854004
epoch: 4 step: 211, loss is 1.5577808618545532
epoch: 4 step: 212, loss is 1.6719563007354736
epoch: 4 step: 213, loss is 1.7513537406921387
epoch: 4 step: 214, loss is 1.6192559003829956
epoch: 4 step: 215, loss is 1.5251822471618652
epoch: 4 step: 216, loss is 1.6622040271759033
epoch: 4 step: 217, loss is 1.677733302116394
epoch: 4 step: 218, loss is 1.7466681003570557
epoch: 4 step: 219, loss is 1.7833832502365112
epoch: 4 step: 220, loss is 1.5311203002929688
epoch: 4 step: 221, loss is 1.5956809520721436
epoch: 4 step: 222, loss is 1.7169291973114014
epoch: 4 step: 223, loss is 1.5117846727371216
epoch: 4 step: 224, loss is 1.660632610321045
epoch: 4 step: 225, loss is 1.6017080545425415
epoch: 4 step: 226, loss is 1.5508091449737549
epoch: 4 step: 227, loss is 1.609438180923462
epoch: 4 step: 228, loss is 1.5371376276016235
epoch: 4 step: 229, loss is 1.4220298528671265
epoch: 4 step: 230, loss is 1.5922303199768066
epoch: 4 step: 231, loss is 1.7141749858856201
epoch: 4 step: 232, loss is 1.5899684429168701
epoch: 4 step: 233, loss is 1.5493285655975342
epoch: 4 step: 234, loss is 1.520312786102295
epoch: 4 step: 235, loss is 1.6780012845993042
epoch: 4 step: 236, loss is 1.6316182613372803
epoch: 4 step: 237, loss is 1.5333560705184937
epoch: 4 step: 238, loss is 1.5699260234832764
epoch: 4 step: 239, loss is 1.5636818408966064
epoch: 4 step: 240, loss is 1.6082319021224976
epoch: 4 step: 241, loss is 1.6599045991897583
epoch: 4 step: 242, loss is 1.6866505146026611
epoch: 4 step: 243, loss is 1.7223758697509766
epoch: 4 step: 244, loss is 1.5355587005615234
epoch: 4 step: 245, loss is 1.5321441888809204
epoch: 4 step: 246, loss is 1.6730215549468994
epoch: 4 step: 247, loss is 1.536695957183838
epoch: 4 step: 248, loss is 1.6998531818389893
epoch: 4 step: 249, loss is 1.5864908695220947
epoch: 4 step: 250, loss is 1.5906410217285156
epoch: 4 step: 251, loss is 1.680201768875122
epoch: 4 step: 252, loss is 1.6065518856048584
epoch: 4 step: 253, loss is 1.6167361736297607
epoch: 4 step: 254, loss is 1.5980623960494995
epoch: 4 step: 255, loss is 1.7381150722503662
epoch: 4 step: 256, loss is 1.5989296436309814
epoch: 4 step: 257, loss is 1.7228833436965942
epoch: 4 step: 258, loss is 1.7968862056732178
epoch: 4 step: 259, loss is 1.640668511390686
epoch: 4 step: 260, loss is 1.752938151359558
epoch: 4 step: 261, loss is 1.5901141166687012
epoch: 4 step: 262, loss is 1.5666574239730835
epoch: 4 step: 263, loss is 1.7112113237380981
epoch: 4 step: 264, loss is 1.5614955425262451
epoch: 4 step: 265, loss is 1.8075993061065674
epoch: 4 step: 266, loss is 1.6912283897399902
epoch: 4 step: 267, loss is 1.7033109664916992
epoch: 4 step: 268, loss is 1.6171555519104004
epoch: 4 step: 269, loss is 1.6306182146072388
epoch: 4 step: 270, loss is 1.6542105674743652
epoch: 4 step: 271, loss is 1.7862052917480469
epoch: 4 step: 272, loss is 1.6153104305267334
epoch: 4 step: 273, loss is 1.5412614345550537
epoch: 4 step: 274, loss is 1.6377443075180054
epoch: 4 step: 275, loss is 1.593024492263794
epoch: 4 step: 276, loss is 1.691611409187317
epoch: 4 step: 277, loss is 1.6097544431686401
epoch: 4 step: 278, loss is 1.5170634984970093
epoch: 4 step: 279, loss is 1.6219221353530884
epoch: 4 step: 280, loss is 1.796911358833313
epoch: 4 step: 281, loss is 1.6067214012145996
epoch: 4 step: 282, loss is 1.5826404094696045
epoch: 4 step: 283, loss is 1.6567620038986206
epoch: 4 step: 284, loss is 1.648327112197876
epoch: 4 step: 285, loss is 1.5465154647827148
epoch: 4 step: 286, loss is 1.5585488080978394
epoch: 4 step: 287, loss is 1.5965622663497925
epoch: 4 step: 288, loss is 1.4978725910186768
epoch: 4 step: 289, loss is 1.6691133975982666
epoch: 4 step: 290, loss is 1.546517252922058
epoch: 4 step: 291, loss is 1.6596770286560059
epoch: 4 step: 292, loss is 1.6295409202575684
epoch: 4 step: 293, loss is 1.708398699760437
epoch: 4 step: 294, loss is 1.5746595859527588
epoch: 4 step: 295, loss is 1.7255644798278809
epoch: 4 step: 296, loss is 1.6387100219726562
epoch: 4 step: 297, loss is 1.6164095401763916
epoch: 4 step: 298, loss is 1.7466027736663818
epoch: 4 step: 299, loss is 1.6549053192138672
epoch: 4 step: 300, loss is 1.5439045429229736
epoch: 4 step: 301, loss is 1.5386300086975098
epoch: 4 step: 302, loss is 1.547088623046875
epoch: 4 step: 303, loss is 1.7093331813812256
epoch: 4 step: 304, loss is 1.5399609804153442
epoch: 4 step: 305, loss is 1.5834934711456299
epoch: 4 step: 306, loss is 1.6823924779891968
epoch: 4 step: 307, loss is 1.6417996883392334
epoch: 4 step: 308, loss is 1.7023022174835205
epoch: 4 step: 309, loss is 1.5595462322235107
epoch: 4 step: 310, loss is 1.639577865600586
epoch: 4 step: 311, loss is 1.7540440559387207
epoch: 4 step: 312, loss is 1.6507065296173096
epoch: 4 step: 313, loss is 1.6445530652999878
epoch: 4 step: 314, loss is 1.5656191110610962
epoch: 4 step: 315, loss is 1.533369541168213
epoch: 4 step: 316, loss is 1.5350271463394165
epoch: 4 step: 317, loss is 1.5182313919067383
epoch: 4 step: 318, loss is 1.4672918319702148
epoch: 4 step: 319, loss is 1.7378803491592407
epoch: 4 step: 320, loss is 1.585449457168579
epoch: 4 step: 321, loss is 1.5696024894714355
epoch: 4 step: 322, loss is 1.576257586479187
epoch: 4 step: 323, loss is 1.5583630800247192
epoch: 4 step: 324, loss is 1.681649923324585
epoch: 4 step: 325, loss is 1.6656967401504517
epoch: 4 step: 326, loss is 1.5865881443023682
epoch: 4 step: 327, loss is 1.5953400135040283
epoch: 4 step: 328, loss is 1.579019546508789
epoch: 4 step: 329, loss is 1.5257303714752197
epoch: 4 step: 330, loss is 1.5868277549743652
epoch: 4 step: 331, loss is 1.6662447452545166
epoch: 4 step: 332, loss is 1.7174075841903687
epoch: 4 step: 333, loss is 1.5932276248931885
epoch: 4 step: 334, loss is 1.5189863443374634
epoch: 4 step: 335, loss is 1.624251365661621
epoch: 4 step: 336, loss is 1.824230670928955
epoch: 4 step: 337, loss is 1.6288148164749146
epoch: 4 step: 338, loss is 1.6846733093261719
epoch: 4 step: 339, loss is 1.5728745460510254
epoch: 4 step: 340, loss is 1.7626162767410278
epoch: 4 step: 341, loss is 1.5695326328277588
epoch: 4 step: 342, loss is 1.5923177003860474
epoch: 4 step: 343, loss is 1.7262592315673828
epoch: 4 step: 344, loss is 1.7128808498382568
epoch: 4 step: 345, loss is 1.6526490449905396
epoch: 4 step: 346, loss is 1.6303529739379883
epoch: 4 step: 347, loss is 1.6546664237976074
epoch: 4 step: 348, loss is 1.6155762672424316
epoch: 4 step: 349, loss is 1.4786903858184814
epoch: 4 step: 350, loss is 1.6805033683776855
epoch: 4 step: 351, loss is 1.5352566242218018
epoch: 4 step: 352, loss is 1.6188104152679443
epoch: 4 step: 353, loss is 1.5360960960388184
epoch: 4 step: 354, loss is 1.6598856449127197
epoch: 4 step: 355, loss is 1.7540043592453003
epoch: 4 step: 356, loss is 1.5454235076904297
epoch: 4 step: 357, loss is 1.5914056301116943
epoch: 4 step: 358, loss is 1.55709969997406
epoch: 4 step: 359, loss is 1.5878585577011108
epoch: 4 step: 360, loss is 1.5572054386138916
epoch: 4 step: 361, loss is 1.5515426397323608
epoch: 4 step: 362, loss is 1.5145514011383057
epoch: 4 step: 363, loss is 1.7833693027496338
epoch: 4 step: 364, loss is 1.637089490890503
epoch: 4 step: 365, loss is 1.448955774307251
epoch: 4 step: 366, loss is 1.54132878780365
epoch: 4 step: 367, loss is 1.6479949951171875
epoch: 4 step: 368, loss is 1.5792829990386963
epoch: 4 step: 369, loss is 1.5844485759735107
epoch: 4 step: 370, loss is 1.6869173049926758
epoch: 4 step: 371, loss is 1.5646523237228394
epoch: 4 step: 372, loss is 1.637107014656067
epoch: 4 step: 373, loss is 1.4887968301773071
epoch: 4 step: 374, loss is 1.6649634838104248
epoch: 4 step: 375, loss is 1.6566877365112305
epoch: 4 step: 376, loss is 1.7221848964691162
epoch: 4 step: 377, loss is 1.5914655923843384
epoch: 4 step: 378, loss is 1.5245280265808105
epoch: 4 step: 379, loss is 1.5608842372894287
epoch: 4 step: 380, loss is 1.6235350370407104
epoch: 4 step: 381, loss is 1.5598446130752563
epoch: 4 step: 382, loss is 1.4868063926696777
epoch: 4 step: 383, loss is 1.7083985805511475
epoch: 4 step: 384, loss is 1.6136480569839478
epoch: 4 step: 385, loss is 1.5036463737487793
epoch: 4 step: 386, loss is 1.662131667137146
epoch: 4 step: 387, loss is 1.6027405261993408
epoch: 4 step: 388, loss is 1.6517276763916016
epoch: 4 step: 389, loss is 1.7624176740646362
epoch: 4 step: 390, loss is 1.5283300876617432
Train epoch time: 158928.539 ms, per step time: 407.509 ms
epoch: 5 step: 1, loss is 1.6968410015106201
epoch: 5 step: 2, loss is 1.6659231185913086
epoch: 5 step: 3, loss is 1.6322542428970337
epoch: 5 step: 4, loss is 1.562246561050415
epoch: 5 step: 5, loss is 1.6001538038253784
epoch: 5 step: 6, loss is 1.704636812210083
epoch: 5 step: 7, loss is 1.524699330329895
epoch: 5 step: 8, loss is 1.5007227659225464
epoch: 5 step: 9, loss is 1.5927250385284424
epoch: 5 step: 10, loss is 1.6594178676605225
epoch: 5 step: 11, loss is 1.4122264385223389
epoch: 5 step: 12, loss is 1.5685546398162842
epoch: 5 step: 13, loss is 1.5577306747436523
epoch: 5 step: 14, loss is 1.6467230319976807
epoch: 5 step: 15, loss is 1.5789284706115723
epoch: 5 step: 16, loss is 1.5310673713684082
epoch: 5 step: 17, loss is 1.571962833404541
epoch: 5 step: 18, loss is 1.6007356643676758
epoch: 5 step: 19, loss is 1.777547836303711
epoch: 5 step: 20, loss is 1.721032738685608
epoch: 5 step: 21, loss is 1.6644401550292969
epoch: 5 step: 22, loss is 1.5138282775878906
epoch: 5 step: 23, loss is 1.4393078088760376
epoch: 5 step: 24, loss is 1.4975173473358154
epoch: 5 step: 25, loss is 1.6252928972244263
epoch: 5 step: 26, loss is 1.624309778213501
epoch: 5 step: 27, loss is 1.5337384939193726
epoch: 5 step: 28, loss is 1.5230265855789185
epoch: 5 step: 29, loss is 1.5963045358657837
epoch: 5 step: 30, loss is 1.5572736263275146
epoch: 5 step: 31, loss is 1.6821726560592651
epoch: 5 step: 32, loss is 1.5029535293579102
epoch: 5 step: 33, loss is 1.5549367666244507
epoch: 5 step: 34, loss is 1.514319896697998
epoch: 5 step: 35, loss is 1.646799921989441
epoch: 5 step: 36, loss is 1.6222974061965942
epoch: 5 step: 37, loss is 1.5975584983825684
epoch: 5 step: 38, loss is 1.6550614833831787
epoch: 5 step: 39, loss is 1.5024807453155518
epoch: 5 step: 40, loss is 1.5903030633926392
epoch: 5 step: 41, loss is 1.4569761753082275
epoch: 5 step: 42, loss is 1.4985620975494385
epoch: 5 step: 43, loss is 1.6266438961029053
epoch: 5 step: 44, loss is 1.627959966659546
epoch: 5 step: 45, loss is 1.557843565940857
epoch: 5 step: 46, loss is 1.5048809051513672
epoch: 5 step: 47, loss is 1.6361091136932373
epoch: 5 step: 48, loss is 1.601837158203125
epoch: 5 step: 49, loss is 1.686318278312683
epoch: 5 step: 50, loss is 1.6092239618301392
epoch: 5 step: 51, loss is 1.7070386409759521
epoch: 5 step: 52, loss is 1.6534405946731567
epoch: 5 step: 53, loss is 1.6964154243469238
epoch: 5 step: 54, loss is 1.591006875038147
epoch: 5 step: 55, loss is 1.5855302810668945
epoch: 5 step: 56, loss is 1.7110592126846313
epoch: 5 step: 57, loss is 1.6818829774856567
epoch: 5 step: 58, loss is 1.6415116786956787
epoch: 5 step: 59, loss is 1.6080925464630127
epoch: 5 step: 60, loss is 1.547296404838562
epoch: 5 step: 61, loss is 1.490729808807373
epoch: 5 step: 62, loss is 1.7193775177001953
epoch: 5 step: 63, loss is 1.4722144603729248
epoch: 5 step: 64, loss is 1.530238389968872
epoch: 5 step: 65, loss is 1.692598581314087
epoch: 5 step: 66, loss is 1.6009944677352905
epoch: 5 step: 67, loss is 1.650260090827942
epoch: 5 step: 68, loss is 1.591707706451416
epoch: 5 step: 69, loss is 1.504097819328308
epoch: 5 step: 70, loss is 1.609652042388916
epoch: 5 step: 71, loss is 1.6373108625411987
epoch: 5 step: 72, loss is 1.5407304763793945
epoch: 5 step: 73, loss is 1.7436730861663818
epoch: 5 step: 74, loss is 1.56667959690094
epoch: 5 step: 75, loss is 1.635094165802002
epoch: 5 step: 76, loss is 1.6081299781799316
epoch: 5 step: 77, loss is 1.5378100872039795
epoch: 5 step: 78, loss is 1.6670833826065063
epoch: 5 step: 79, loss is 1.5488393306732178
epoch: 5 step: 80, loss is 1.572231411933899
epoch: 5 step: 81, loss is 1.5126221179962158
epoch: 5 step: 82, loss is 1.6492631435394287
epoch: 5 step: 83, loss is 1.5773487091064453
epoch: 5 step: 84, loss is 1.5184754133224487
epoch: 5 step: 85, loss is 1.6492950916290283
epoch: 5 step: 86, loss is 1.3569157123565674
epoch: 5 step: 87, loss is 1.563852071762085
epoch: 5 step: 88, loss is 1.575610637664795
epoch: 5 step: 89, loss is 1.6332216262817383
epoch: 5 step: 90, loss is 1.6355527639389038
epoch: 5 step: 91, loss is 1.5853290557861328
epoch: 5 step: 92, loss is 1.4569050073623657
epoch: 5 step: 93, loss is 1.5537524223327637
epoch: 5 step: 94, loss is 1.4168031215667725
epoch: 5 step: 95, loss is 1.4792532920837402
epoch: 5 step: 96, loss is 1.626610517501831
epoch: 5 step: 97, loss is 1.464279055595398
epoch: 5 step: 98, loss is 1.5759751796722412
epoch: 5 step: 99, loss is 1.6344813108444214
epoch: 5 step: 100, loss is 1.5865296125411987
epoch: 5 step: 101, loss is 1.5722415447235107
epoch: 5 step: 102, loss is 1.5866460800170898
epoch: 5 step: 103, loss is 1.4397867918014526
epoch: 5 step: 104, loss is 1.6298198699951172
epoch: 5 step: 105, loss is 1.6406216621398926
epoch: 5 step: 106, loss is 1.615403652191162
epoch: 5 step: 107, loss is 1.6129870414733887
epoch: 5 step: 108, loss is 1.6771572828292847
epoch: 5 step: 109, loss is 1.6278462409973145
epoch: 5 step: 110, loss is 1.5112825632095337
epoch: 5 step: 111, loss is 1.501893401145935
epoch: 5 step: 112, loss is 1.5857763290405273
epoch: 5 step: 113, loss is 1.5120599269866943
epoch: 5 step: 114, loss is 1.5478036403656006
epoch: 5 step: 115, loss is 1.5000666379928589
epoch: 5 step: 116, loss is 1.6942318677902222
epoch: 5 step: 117, loss is 1.6470630168914795
epoch: 5 step: 118, loss is 1.628605842590332
epoch: 5 step: 119, loss is 1.593959927558899
epoch: 5 step: 120, loss is 1.5264394283294678
epoch: 5 step: 121, loss is 1.5901384353637695
epoch: 5 step: 122, loss is 1.6464496850967407
epoch: 5 step: 123, loss is 1.5332677364349365
epoch: 5 step: 124, loss is 1.5954304933547974
epoch: 5 step: 125, loss is 1.5491483211517334
epoch: 5 step: 126, loss is 1.7041174173355103
epoch: 5 step: 127, loss is 1.6616690158843994
epoch: 5 step: 128, loss is 1.4599387645721436
epoch: 5 step: 129, loss is 1.718249797821045
epoch: 5 step: 130, loss is 1.5128003358840942
epoch: 5 step: 131, loss is 1.4864637851715088
epoch: 5 step: 132, loss is 1.6722159385681152
epoch: 5 step: 133, loss is 1.5668091773986816
epoch: 5 step: 134, loss is 1.605202078819275
epoch: 5 step: 135, loss is 1.5731450319290161
epoch: 5 step: 136, loss is 1.667793869972229
epoch: 5 step: 137, loss is 1.523836374282837
epoch: 5 step: 138, loss is 1.6455575227737427
epoch: 5 step: 139, loss is 1.6336060762405396
epoch: 5 step: 140, loss is 1.5330235958099365
epoch: 5 step: 141, loss is 1.5052452087402344
epoch: 5 step: 142, loss is 1.592769980430603
epoch: 5 step: 143, loss is 1.5359638929367065
epoch: 5 step: 144, loss is 1.673586368560791
epoch: 5 step: 145, loss is 1.5016387701034546
epoch: 5 step: 146, loss is 1.5353078842163086
epoch: 5 step: 147, loss is 1.6564146280288696
epoch: 5 step: 148, loss is 1.6455034017562866
epoch: 5 step: 149, loss is 1.5082948207855225
epoch: 5 step: 150, loss is 1.648041009902954
epoch: 5 step: 151, loss is 1.5716785192489624
epoch: 5 step: 152, loss is 1.4713255167007446
epoch: 5 step: 153, loss is 1.694359540939331
epoch: 5 step: 154, loss is 1.7939517498016357
epoch: 5 step: 155, loss is 1.654297113418579
epoch: 5 step: 156, loss is 1.5738584995269775
epoch: 5 step: 157, loss is 1.4971880912780762
epoch: 5 step: 158, loss is 1.5677478313446045
epoch: 5 step: 159, loss is 1.7518084049224854
epoch: 5 step: 160, loss is 1.5800302028656006
epoch: 5 step: 161, loss is 1.528383493423462
epoch: 5 step: 162, loss is 1.5152841806411743
epoch: 5 step: 163, loss is 1.6984838247299194
epoch: 5 step: 164, loss is 1.6901171207427979
epoch: 5 step: 165, loss is 1.4601478576660156
epoch: 5 step: 166, loss is 1.7045527696609497
epoch: 5 step: 167, loss is 1.557382583618164
epoch: 5 step: 168, loss is 1.5122120380401611
epoch: 5 step: 169, loss is 1.6460380554199219
epoch: 5 step: 170, loss is 1.7393983602523804
epoch: 5 step: 171, loss is 1.4965667724609375
epoch: 5 step: 172, loss is 1.5170384645462036
epoch: 5 step: 173, loss is 1.650547742843628
epoch: 5 step: 174, loss is 1.7205686569213867
epoch: 5 step: 175, loss is 1.5804343223571777
epoch: 5 step: 176, loss is 1.6631768941879272
epoch: 5 step: 177, loss is 1.689775824546814
epoch: 5 step: 178, loss is 1.5447447299957275
epoch: 5 step: 179, loss is 1.5804797410964966
epoch: 5 step: 180, loss is 1.5648460388183594
epoch: 5 step: 181, loss is 1.519169569015503
epoch: 5 step: 182, loss is 1.5897284746170044
epoch: 5 step: 183, loss is 1.6237672567367554
epoch: 5 step: 184, loss is 1.6113545894622803
epoch: 5 step: 185, loss is 1.5218414068222046
epoch: 5 step: 186, loss is 1.5092628002166748
epoch: 5 step: 187, loss is 1.5974726676940918
epoch: 5 step: 188, loss is 1.437012791633606
epoch: 5 step: 189, loss is 1.657314419746399
epoch: 5 step: 190, loss is 1.5997402667999268
epoch: 5 step: 191, loss is 1.7388230562210083
epoch: 5 step: 192, loss is 1.4074933528900146
epoch: 5 step: 193, loss is 1.5641143321990967
epoch: 5 step: 194, loss is 1.6026058197021484
epoch: 5 step: 195, loss is 1.6233866214752197
epoch: 5 step: 196, loss is 1.4921730756759644
epoch: 5 step: 197, loss is 1.5862095355987549
epoch: 5 step: 198, loss is 1.6000502109527588
epoch: 5 step: 199, loss is 1.611676573753357
epoch: 5 step: 200, loss is 1.5682951211929321
epoch: 5 step: 201, loss is 1.6305357217788696
epoch: 5 step: 202, loss is 1.5610345602035522
epoch: 5 step: 203, loss is 1.5682110786437988
epoch: 5 step: 204, loss is 1.4949156045913696
epoch: 5 step: 205, loss is 1.6070489883422852
epoch: 5 step: 206, loss is 1.551421046257019
epoch: 5 step: 207, loss is 1.5265458822250366
epoch: 5 step: 208, loss is 1.592637300491333
epoch: 5 step: 209, loss is 1.5738508701324463
epoch: 5 step: 210, loss is 1.583228349685669
epoch: 5 step: 211, loss is 1.517490267753601
epoch: 5 step: 212, loss is 1.6815751791000366
epoch: 5 step: 213, loss is 1.5768711566925049
epoch: 5 step: 214, loss is 1.6347519159317017
epoch: 5 step: 215, loss is 1.531420350074768
epoch: 5 step: 216, loss is 1.4506142139434814
epoch: 5 step: 217, loss is 1.5179405212402344
epoch: 5 step: 218, loss is 1.6334121227264404
epoch: 5 step: 219, loss is 1.4738596677780151
epoch: 5 step: 220, loss is 1.6142504215240479
epoch: 5 step: 221, loss is 1.5994846820831299
epoch: 5 step: 222, loss is 1.5052489042282104
epoch: 5 step: 223, loss is 1.629508376121521
epoch: 5 step: 224, loss is 1.8033719062805176
epoch: 5 step: 225, loss is 1.5596901178359985
epoch: 5 step: 226, loss is 1.5065029859542847
epoch: 5 step: 227, loss is 1.4694762229919434
epoch: 5 step: 228, loss is 1.4944794178009033
epoch: 5 step: 229, loss is 1.5694572925567627
epoch: 5 step: 230, loss is 1.6390984058380127
epoch: 5 step: 231, loss is 1.6585972309112549
epoch: 5 step: 232, loss is 1.5577807426452637
epoch: 5 step: 233, loss is 1.6133297681808472
epoch: 5 step: 234, loss is 1.6371749639511108
epoch: 5 step: 235, loss is 1.5685757398605347
epoch: 5 step: 236, loss is 1.5548603534698486
epoch: 5 step: 237, loss is 1.5454955101013184
epoch: 5 step: 238, loss is 1.6038284301757812
epoch: 5 step: 239, loss is 1.5180238485336304
epoch: 5 step: 240, loss is 1.5903669595718384
epoch: 5 step: 241, loss is 1.606217861175537
epoch: 5 step: 242, loss is 1.576747179031372
epoch: 5 step: 243, loss is 1.6467961072921753
epoch: 5 step: 244, loss is 1.5293443202972412
epoch: 5 step: 245, loss is 1.5193610191345215
epoch: 5 step: 246, loss is 1.538038969039917
epoch: 5 step: 247, loss is 1.6445249319076538
epoch: 5 step: 248, loss is 1.5168802738189697
epoch: 5 step: 249, loss is 1.4910800457000732
epoch: 5 step: 250, loss is 1.503363013267517
epoch: 5 step: 251, loss is 1.5377938747406006
epoch: 5 step: 252, loss is 1.5161374807357788
epoch: 5 step: 253, loss is 1.5503935813903809
epoch: 5 step: 254, loss is 1.5772137641906738
epoch: 5 step: 255, loss is 1.5763084888458252
epoch: 5 step: 256, loss is 1.5561844110488892
epoch: 5 step: 257, loss is 1.4956920146942139
epoch: 5 step: 258, loss is 1.6509121656417847
epoch: 5 step: 259, loss is 1.566279411315918
epoch: 5 step: 260, loss is 1.5843578577041626
epoch: 5 step: 261, loss is 1.5762419700622559
epoch: 5 step: 262, loss is 1.548414707183838
epoch: 5 step: 263, loss is 1.5279574394226074
epoch: 5 step: 264, loss is 1.3839995861053467
epoch: 5 step: 265, loss is 1.6058135032653809
epoch: 5 step: 266, loss is 1.5328731536865234
epoch: 5 step: 267, loss is 1.5000840425491333
epoch: 5 step: 268, loss is 1.596129059791565
epoch: 5 step: 269, loss is 1.6348944902420044
epoch: 5 step: 270, loss is 1.5765900611877441
epoch: 5 step: 271, loss is 1.5770118236541748
epoch: 5 step: 272, loss is 1.6356513500213623
epoch: 5 step: 273, loss is 1.5493303537368774
epoch: 5 step: 274, loss is 1.551661491394043
epoch: 5 step: 275, loss is 1.6044243574142456
epoch: 5 step: 276, loss is 1.699059247970581
epoch: 5 step: 277, loss is 1.602952241897583
epoch: 5 step: 278, loss is 1.5778919458389282
epoch: 5 step: 279, loss is 1.581085205078125
epoch: 5 step: 280, loss is 1.637789011001587
epoch: 5 step: 281, loss is 1.557830810546875
epoch: 5 step: 282, loss is 1.679073691368103
epoch: 5 step: 283, loss is 1.5902385711669922
epoch: 5 step: 284, loss is 1.5076617002487183
epoch: 5 step: 285, loss is 1.4554983377456665
epoch: 5 step: 286, loss is 1.580453872680664
epoch: 5 step: 287, loss is 1.5890051126480103
epoch: 5 step: 288, loss is 1.548875331878662
epoch: 5 step: 289, loss is 1.4803483486175537
epoch: 5 step: 290, loss is 1.5194365978240967
epoch: 5 step: 291, loss is 1.5418837070465088
epoch: 5 step: 292, loss is 1.5891587734222412
epoch: 5 step: 293, loss is 1.561320185661316
epoch: 5 step: 294, loss is 1.6979957818984985
epoch: 5 step: 295, loss is 1.7171822786331177
epoch: 5 step: 296, loss is 1.4649369716644287
epoch: 5 step: 297, loss is 1.5368256568908691
epoch: 5 step: 298, loss is 1.5674761533737183
epoch: 5 step: 299, loss is 1.5720322132110596
epoch: 5 step: 300, loss is 1.4978396892547607
epoch: 5 step: 301, loss is 1.5296461582183838
epoch: 5 step: 302, loss is 1.5150933265686035
epoch: 5 step: 303, loss is 1.5767050981521606
epoch: 5 step: 304, loss is 1.6172314882278442
epoch: 5 step: 305, loss is 1.670998454093933
epoch: 5 step: 306, loss is 1.5195037126541138
epoch: 5 step: 307, loss is 1.5792880058288574
epoch: 5 step: 308, loss is 1.5197694301605225
epoch: 5 step: 309, loss is 1.579887866973877
epoch: 5 step: 310, loss is 1.547187328338623
epoch: 5 step: 311, loss is 1.5418907403945923
epoch: 5 step: 312, loss is 1.5750237703323364
epoch: 5 step: 313, loss is 1.563706874847412
epoch: 5 step: 314, loss is 1.5549464225769043
epoch: 5 step: 315, loss is 1.5995084047317505
epoch: 5 step: 316, loss is 1.4370160102844238
epoch: 5 step: 317, loss is 1.5404930114746094
epoch: 5 step: 318, loss is 1.6087052822113037
epoch: 5 step: 319, loss is 1.6200518608093262
epoch: 5 step: 320, loss is 1.6411569118499756
epoch: 5 step: 321, loss is 1.463921070098877
epoch: 5 step: 322, loss is 1.5251611471176147
epoch: 5 step: 323, loss is 1.5315260887145996
epoch: 5 step: 324, loss is 1.5517654418945312
epoch: 5 step: 325, loss is 1.5414708852767944
epoch: 5 step: 326, loss is 1.6294476985931396
epoch: 5 step: 327, loss is 1.496427059173584
epoch: 5 step: 328, loss is 1.5134437084197998
epoch: 5 step: 329, loss is 1.4705740213394165
epoch: 5 step: 330, loss is 1.6028826236724854
epoch: 5 step: 331, loss is 1.6761093139648438
epoch: 5 step: 332, loss is 1.6033540964126587
epoch: 5 step: 333, loss is 1.5463560819625854
epoch: 5 step: 334, loss is 1.493402123451233
epoch: 5 step: 335, loss is 1.696936845779419
epoch: 5 step: 336, loss is 1.564387559890747
epoch: 5 step: 337, loss is 1.5496392250061035
epoch: 5 step: 338, loss is 1.6393741369247437
epoch: 5 step: 339, loss is 1.5797892808914185
epoch: 5 step: 340, loss is 1.6611074209213257
epoch: 5 step: 341, loss is 1.2772939205169678
epoch: 5 step: 342, loss is 1.5420176982879639
epoch: 5 step: 343, loss is 1.717203140258789
epoch: 5 step: 344, loss is 1.6533790826797485
epoch: 5 step: 345, loss is 1.5980162620544434
epoch: 5 step: 346, loss is 1.6362414360046387
epoch: 5 step: 347, loss is 1.4364676475524902
epoch: 5 step: 348, loss is 1.5918983221054077
epoch: 5 step: 349, loss is 1.625906229019165
epoch: 5 step: 350, loss is 1.4642462730407715
epoch: 5 step: 351, loss is 1.5123826265335083
epoch: 5 step: 352, loss is 1.7221839427947998
epoch: 5 step: 353, loss is 1.4509187936782837
epoch: 5 step: 354, loss is 1.6407177448272705
epoch: 5 step: 355, loss is 1.5839444398880005
epoch: 5 step: 356, loss is 1.4749165773391724
epoch: 5 step: 357, loss is 1.6423494815826416
epoch: 5 step: 358, loss is 1.5610545873641968
epoch: 5 step: 359, loss is 1.491794466972351
epoch: 5 step: 360, loss is 1.5223362445831299
epoch: 5 step: 361, loss is 1.475632667541504
epoch: 5 step: 362, loss is 1.5109925270080566
epoch: 5 step: 363, loss is 1.451634407043457
epoch: 5 step: 364, loss is 1.4902803897857666
epoch: 5 step: 365, loss is 1.492379903793335
epoch: 5 step: 366, loss is 1.5167975425720215
epoch: 5 step: 367, loss is 1.4670071601867676
epoch: 5 step: 368, loss is 1.5422234535217285
epoch: 5 step: 369, loss is 1.6394360065460205
epoch: 5 step: 370, loss is 1.6121578216552734
epoch: 5 step: 371, loss is 1.6610655784606934
epoch: 5 step: 372, loss is 1.525736927986145
epoch: 5 step: 373, loss is 1.5749610662460327
epoch: 5 step: 374, loss is 1.5841219425201416
epoch: 5 step: 375, loss is 1.4800596237182617
epoch: 5 step: 376, loss is 1.617612361907959
epoch: 5 step: 377, loss is 1.6333978176116943
epoch: 5 step: 378, loss is 1.6120028495788574
epoch: 5 step: 379, loss is 1.6427056789398193
epoch: 5 step: 380, loss is 1.5131359100341797
epoch: 5 step: 381, loss is 1.4777302742004395
epoch: 5 step: 382, loss is 1.6485257148742676
epoch: 5 step: 383, loss is 1.493799090385437
epoch: 5 step: 384, loss is 1.513806939125061
epoch: 5 step: 385, loss is 1.4659197330474854
epoch: 5 step: 386, loss is 1.5628206729888916
epoch: 5 step: 387, loss is 1.7294472455978394
epoch: 5 step: 388, loss is 1.5499292612075806
epoch: 5 step: 389, loss is 1.5456959009170532
epoch: 5 step: 390, loss is 1.5201120376586914
Train epoch time: 155649.256 ms, per step time: 399.101 ms
total time:0h 19m 59s
============== Train Success ==============

训练好的模型保存在当前目录的shufflenetv1-5_390.ckpt中，用作评估。

模型评估

在CIFAR-10的测试集上对模型进行评估。

设置好评估模型的路径后加载数据集，并设置Top 1, Top 5的评估标准，最后用model.eval()接口对模型进行评估。

from mindspore import load_checkpoint, load_param_into_net

def test():
    mindspore.set_context(mode=mindspore.GRAPH_MODE, device_target="Ascend")
    dataset = get_dataset("./dataset/cifar-10-batches-bin", 128, "test")
    net = ShuffleNetV1(model_size="2.0x", n_class=10)
    param_dict = load_checkpoint("shufflenetv1-5_390.ckpt")
    load_param_into_net(net, param_dict)
    net.set_train(False)
    loss = nn.CrossEntropyLoss(weight=None, reduction='mean', label_smoothing=0.1)
    eval_metrics = {'Loss': nn.Loss(), 'Top_1_Acc': Top1CategoricalAccuracy(),
                    'Top_5_Acc': Top5CategoricalAccuracy()}
    model = Model(net, loss_fn=loss, metrics=eval_metrics)
    start_time = time.time()
    res = model.eval(dataset, dataset_sink_mode=False)
    use_time = time.time() - start_time
    hour = str(int(use_time // 60 // 60))
    minute = str(int(use_time // 60 % 60))
    second = str(int(use_time % 60))
    log = "result:" + str(res) + ", ckpt:'" + "./shufflenetv1-5_390.ckpt" \
        + "', time: " + hour + "h " + minute + "m " + second + "s"
    print(log)
    filename = './eval_log.txt'
    with open(filename, 'a') as file_object:
        file_object.write(log + '\n')

if __name__ == '__main__':
    test()

model size is  2.0x


[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.237.531 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.238.059 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.240.297 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.243.557 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.245.444 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.246.612 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.248.503 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.249.973 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.250.794 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.252.283 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.253.806 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.255.633 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.257.154 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.260.748 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.261.175 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.265.496 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.265.927 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.266.753 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.269.662 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.269.747 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.271.991 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.273.120 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.275.692 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.277.167 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.280.115 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.281.581 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.282.423 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.286.378 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.287.936 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.288.394 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.289.888 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:01:00.292.085 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]


result:{'Loss': 1.543383138302045, 'Top_1_Acc': 0.522636217948718, 'Top_5_Acc': 0.9408052884615384}, ckpt:'./shufflenetv1-5_390.ckpt', time: 0h 1m 47s

模型预测

在CIFAR-10的测试集上对模型进行预测，并将预测结果可视化。

import mindspore
import matplotlib.pyplot as plt
import mindspore.dataset as ds

net = ShuffleNetV1(model_size="2.0x", n_class=10)
show_lst = []
param_dict = load_checkpoint("shufflenetv1-5_390.ckpt")
load_param_into_net(net, param_dict)
model = Model(net)
dataset_predict = ds.Cifar10Dataset(dataset_dir="./dataset/cifar-10-batches-bin", shuffle=False, usage="train")
dataset_show = ds.Cifar10Dataset(dataset_dir="./dataset/cifar-10-batches-bin", shuffle=False, usage="train")
dataset_show = dataset_show.batch(16)
show_images_lst = next(dataset_show.create_dict_iterator())["image"].asnumpy()
image_trans = [
    vision.RandomCrop((32, 32), (4, 4, 4, 4)),
    vision.RandomHorizontalFlip(prob=0.5),
    vision.Resize((224, 224)),
    vision.Rescale(1.0 / 255.0, 0.0),
    vision.Normalize([0.4914, 0.4822, 0.4465], [0.2023, 0.1994, 0.2010]),
    vision.HWC2CHW()
        ]
dataset_predict = dataset_predict.map(image_trans, 'image')
dataset_predict = dataset_predict.batch(16)
class_dict = {0:"airplane", 1:"automobile", 2:"bird", 3:"cat", 4:"deer", 5:"dog", 6:"frog", 7:"horse", 8:"ship", 9:"truck"}
# 推理效果展示(上方为预测的结果，下方为推理效果图片)
plt.figure(figsize=(16, 5))
predict_data = next(dataset_predict.create_dict_iterator())
output = model.predict(ms.Tensor(predict_data['image']))
pred = np.argmax(output.asnumpy(), axis=1)
index = 0
for image in show_images_lst:
    plt.subplot(2, 8, index+1)
    plt.title('{}'.format(class_dict[pred[index]]))
    index += 1
    plt.imshow(image)
    plt.axis("off")
plt.show()

model size is  2.0x


[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.565.926 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/1681751341.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.566.042 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/1681751341.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.569.294 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.569.741 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.570.861 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/778396864.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.571.318 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.574.545 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.576.434 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.577.589 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.579.468 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.580.939 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.581.757 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.583.240 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.584.781 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.586.607 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.588.150 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.590.662 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/778396864.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.591.112 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.591.546 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.595.863 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.596.289 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.597.114 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.600.019 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.600.101 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.602.302 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.603.441 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.604.543 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/778396864.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.605.351 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.606.809 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.609.729 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.611.193 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.612.016 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.615.974 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.617.482 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.617.928 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.619.434 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]
[ERROR] CORE(34481,ffffb9059930,python):2024-07-05-13:02:51.621.596 [mindspore/core/utils/file_utils.cc:253] GetRealPath] Get realpath failed, path[/tmp/ipykernel_34481/3162391481.py]