pytorch 图像数据集管理

news2025/2/26 11:37:04

目录

1.数据集的管理说明

2.数据集Dataset类说明

3.图像分类常用的类 ImageFolder


1.数据集的管理说明

        pytorch使用Dataset来管理训练和测试数据集,前文说过 

torchvision.datasets.MNIST

        这些 torchvision.datasets里面的数据集都是继承Dataset而来,对Datasetd 管理使用DataLoader我们使用的的时候,只需要把Dataset类放在DataLoader这个容器里面,在训练的时候 for循环从DataLoader容器里面取出批次的数据,对模型进行训练。

2.数据集Dataset类说明

        我们可以继承Dataset类,对训练和测试数据进行管理,继承Dataset示例:

import torch
from torch.utils.data import Dataset
from torchvision import datasets
from torch.utils.data import DataLoader
from torchvision import transforms
import os
import cv2
#继承from torch.utils.data import Dataset
class CDataSet(Dataset):
    def __init__(self,path):
        self.path = path
        self.list = os.listdir(path)
        self.len = len(self.list)
        self.name = ['cloudy','rain','shine','sunrise']
        self.trans = transforms.ToTensor()
    def __len__(self):
        return self.len
    def __getitem__(self, item):
        self.imgpath = os.path.join(self.path,self.list[item])
        print(self.imgpath)
        img = cv2.imread(self.imgpath)
        img = cv2.cvtColor(img,cv2.COLOR_BGR2RGB)
        img = cv2.resize(img,(100,100))
        img = self.trans(img)
        for i,n in enumerate(self.name):
            if n in self.imgpath:
                label = i+1
                break
        return img,label

ds = CDataSet(r'E:\test\pythonProject\dataset\cloudy')
dl = DataLoader(ds,batch_size=16,shuffle=True)
print(len(ds))
print(len(dl))
print(type(ds))
print(type(dl))
print(next(iter(dl)))


'''
D:\anaconda3\python.exe E:\test\pythonProject\test.py 
300
19
<class '__main__.CDataSet'>
<class 'torch.utils.data.dataloader.DataLoader'>
E:\test\pythonProject\dataset\cloudy\cloudy294.jpg
E:\test\pythonProject\dataset\cloudy\cloudy156.jpg
E:\test\pythonProject\dataset\cloudy\cloudy149.jpg
E:\test\pythonProject\dataset\cloudy\cloudy148.jpg
E:\test\pythonProject\dataset\cloudy\cloudy3.jpg
E:\test\pythonProject\dataset\cloudy\cloudy106.jpg
E:\test\pythonProject\dataset\cloudy\cloudy137.jpg
E:\test\pythonProject\dataset\cloudy\cloudy276.jpg
E:\test\pythonProject\dataset\cloudy\cloudy147.jpg
E:\test\pythonProject\dataset\cloudy\cloudy8.jpg
E:\test\pythonProject\dataset\cloudy\cloudy164.jpg
E:\test\pythonProject\dataset\cloudy\cloudy293.jpg
E:\test\pythonProject\dataset\cloudy\cloudy116.jpg
E:\test\pythonProject\dataset\cloudy\cloudy56.jpg
E:\test\pythonProject\dataset\cloudy\cloudy187.jpg
E:\test\pythonProject\dataset\cloudy\cloudy177.jpg
[tensor([[[[0.2235, 0.2471, 0.3569,  ..., 0.1490, 0.1373, 0.1373],
          [0.2902, 0.4039, 0.4078,  ..., 0.1529, 0.1373, 0.1294],
          [0.3294, 0.4941, 0.4000,  ..., 0.1529, 0.1333, 0.1137],
          ...,
          [0.0118, 0.0118, 0.0118,  ..., 0.0078, 0.0078, 0.0078],
          [0.0118, 0.0118, 0.0118,  ..., 0.0039, 0.0039, 0.0039],
          [0.0118, 0.0118, 0.0118,  ..., 0.0039, 0.0039, 0.0039]],

         [[0.2196, 0.2471, 0.3608,  ..., 0.1725, 0.1608, 0.1608],
          [0.2824, 0.3961, 0.4118,  ..., 0.1765, 0.1608, 0.1529],
          [0.3216, 0.4863, 0.4039,  ..., 0.1765, 0.1569, 0.1373],
          ...,
          [0.0235, 0.0235, 0.0235,  ..., 0.0078, 0.0078, 0.0078],
          [0.0235, 0.0235, 0.0235,  ..., 0.0078, 0.0078, 0.0078],
          [0.0235, 0.0235, 0.0235,  ..., 0.0157, 0.0196, 0.0157]],

         [[0.3098, 0.3412, 0.4510,  ..., 0.2196, 0.2078, 0.2078],
          [0.3686, 0.4824, 0.4980,  ..., 0.2235, 0.2078, 0.2000],
          [0.4078, 0.5725, 0.4863,  ..., 0.2235, 0.2039, 0.1843],
          ...,
          [0.0000, 0.0000, 0.0000,  ..., 0.0157, 0.0157, 0.0157],
          [0.0000, 0.0000, 0.0000,  ..., 0.0157, 0.0157, 0.0157],
          [0.0000, 0.0000, 0.0000,  ..., 0.0078, 0.0039, 0.0078]]],


        [[[0.7059, 0.6902, 0.6824,  ..., 0.5961, 0.6000, 0.6118],
          [0.6980, 0.6824, 0.6745,  ..., 0.6039, 0.6078, 0.6196],
          [0.6863, 0.6706, 0.6588,  ..., 0.6196, 0.6235, 0.6353],
          ...,
          [0.2706, 0.2941, 0.2706,  ..., 0.2745, 0.2745, 0.2706],
          [0.2745, 0.2745, 0.2667,  ..., 0.2784, 0.2902, 0.2745],
          [0.2784, 0.2706, 0.2784,  ..., 0.2824, 0.3020, 0.2784]],

         [[0.7176, 0.7020, 0.6941,  ..., 0.6235, 0.6275, 0.6392],
          [0.7098, 0.6941, 0.6863,  ..., 0.6314, 0.6353, 0.6471],
          [0.6941, 0.6863, 0.6706,  ..., 0.6471, 0.6510, 0.6627],
          ...,
          [0.2784, 0.3020, 0.2824,  ..., 0.2824, 0.2824, 0.2784],
          [0.2824, 0.2824, 0.2745,  ..., 0.2863, 0.2980, 0.2824],
          [0.2863, 0.2784, 0.2863,  ..., 0.2902, 0.3098, 0.2824]],

         [[0.7412, 0.7294, 0.7176,  ..., 0.6471, 0.6510, 0.6627],
          [0.7373, 0.7216, 0.7137,  ..., 0.6549, 0.6588, 0.6706],
          [0.7255, 0.7098, 0.6980,  ..., 0.6706, 0.6745, 0.6863],
          ...,
          [0.1961, 0.2196, 0.2000,  ..., 0.2000, 0.2000, 0.1961],
          [0.2000, 0.2000, 0.1922,  ..., 0.2039, 0.2157, 0.2000],
          [0.2039, 0.1961, 0.2039,  ..., 0.2078, 0.2275, 0.2039]]],


        [[[0.3176, 0.3255, 0.3294,  ..., 0.5529, 0.5255, 0.4824],
          [0.3098, 0.3176, 0.3216,  ..., 0.5608, 0.5255, 0.4824],
          [0.3059, 0.3098, 0.3098,  ..., 0.5686, 0.4941, 0.4588],
          ...,
          [0.4510, 0.4549, 0.3176,  ..., 0.2627, 0.3059, 0.3333],
          [0.3843, 0.4980, 0.4000,  ..., 0.3804, 0.4235, 0.3804],
          [0.4549, 0.6353, 0.7333,  ..., 0.4902, 0.5882, 0.6627]],

         [[0.3333, 0.3373, 0.3412,  ..., 0.5961, 0.5765, 0.5333],
          [0.3255, 0.3333, 0.3373,  ..., 0.6039, 0.5686, 0.5333],
          [0.3216, 0.3255, 0.3255,  ..., 0.6157, 0.5412, 0.5098],
          ...,
          [0.4275, 0.4275, 0.3255,  ..., 0.2627, 0.2902, 0.3176],
          [0.3804, 0.4510, 0.3961,  ..., 0.3529, 0.3843, 0.3529],
          [0.4275, 0.5333, 0.6039,  ..., 0.4353, 0.5098, 0.5569]],

         [[0.3804, 0.3961, 0.4000,  ..., 0.6667, 0.6431, 0.6000],
          [0.3725, 0.3804, 0.3843,  ..., 0.6745, 0.6392, 0.6000],
          [0.3686, 0.3725, 0.3725,  ..., 0.6784, 0.6118, 0.5843],
          ...,
          [0.3843, 0.3843, 0.3255,  ..., 0.2353, 0.2549, 0.2706],
          [0.3412, 0.3882, 0.3725,  ..., 0.2902, 0.3098, 0.2863],
          [0.3804, 0.4039, 0.4275,  ..., 0.3294, 0.3333, 0.3529]]],


        ...,


        [[[0.5843, 0.6000, 0.6471,  ..., 0.3294, 0.3255, 0.3333],
          [0.5412, 0.5529, 0.6627,  ..., 0.3373, 0.3333, 0.3373],
          [0.5137, 0.5098, 0.6235,  ..., 0.3451, 0.3451, 0.3412],
          ...,
          [0.2980, 0.1098, 0.0824,  ..., 0.0000, 0.0000, 0.0000],
          [0.0078, 0.0000, 0.0039,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],

         [[0.5843, 0.6000, 0.6471,  ..., 0.3294, 0.3255, 0.3333],
          [0.5412, 0.5529, 0.6627,  ..., 0.3373, 0.3333, 0.3373],
          [0.5137, 0.5098, 0.6235,  ..., 0.3451, 0.3451, 0.3412],
          ...,
          [0.2980, 0.1098, 0.0824,  ..., 0.0000, 0.0000, 0.0000],
          [0.0078, 0.0000, 0.0039,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]],

         [[0.5843, 0.6000, 0.6471,  ..., 0.3294, 0.3255, 0.3333],
          [0.5412, 0.5529, 0.6627,  ..., 0.3373, 0.3333, 0.3373],
          [0.5137, 0.5098, 0.6235,  ..., 0.3451, 0.3451, 0.3412],
          ...,
          [0.2980, 0.1098, 0.0824,  ..., 0.0000, 0.0000, 0.0000],
          [0.0078, 0.0000, 0.0039,  ..., 0.0000, 0.0000, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0000, 0.0000, 0.0000]]],


        [[[0.5608, 0.5843, 0.6196,  ..., 0.4431, 0.4314, 0.4275],
          [0.5529, 0.5725, 0.6039,  ..., 0.4510, 0.4392, 0.4392],
          [0.5569, 0.5647, 0.5922,  ..., 0.4588, 0.4510, 0.4549],
          ...,
          [0.1020, 0.0784, 0.0627,  ..., 0.1255, 0.1373, 0.1216],
          [0.0431, 0.0627, 0.0510,  ..., 0.0902, 0.1176, 0.1294],
          [0.0902, 0.1059, 0.0588,  ..., 0.0902, 0.0941, 0.1020]],

         [[0.6275, 0.6510, 0.6863,  ..., 0.5020, 0.4902, 0.4863],
          [0.6235, 0.6392, 0.6706,  ..., 0.5098, 0.4980, 0.4980],
          [0.6196, 0.6314, 0.6588,  ..., 0.5176, 0.5098, 0.5098],
          ...,
          [0.1373, 0.1176, 0.0980,  ..., 0.1569, 0.1725, 0.1569],
          [0.0784, 0.0941, 0.0863,  ..., 0.1255, 0.1529, 0.1647],
          [0.1255, 0.1412, 0.0941,  ..., 0.1255, 0.1294, 0.1373]],

         [[0.6039, 0.6275, 0.6627,  ..., 0.4824, 0.4706, 0.4667],
          [0.5961, 0.6157, 0.6471,  ..., 0.4902, 0.4784, 0.4784],
          [0.5961, 0.6078, 0.6353,  ..., 0.4980, 0.4902, 0.4941],
          ...,
          [0.1255, 0.1020, 0.0863,  ..., 0.1451, 0.1608, 0.1451],
          [0.0667, 0.0863, 0.0745,  ..., 0.1137, 0.1412, 0.1529],
          [0.1137, 0.1294, 0.0824,  ..., 0.1137, 0.1176, 0.1255]]],


        [[[0.1922, 0.1882, 0.1843,  ..., 0.1608, 0.1647, 0.1686],
          [0.1961, 0.1922, 0.1882,  ..., 0.1686, 0.1686, 0.1725],
          [0.2000, 0.2000, 0.1961,  ..., 0.1804, 0.1804, 0.1843],
          ...,
          [0.3686, 0.3882, 0.3961,  ..., 0.3098, 0.3098, 0.3098],
          [0.3765, 0.3882, 0.3882,  ..., 0.2980, 0.2980, 0.2980],
          [0.3725, 0.3804, 0.3804,  ..., 0.2941, 0.2941, 0.2941]],

         [[0.1922, 0.1882, 0.1843,  ..., 0.1608, 0.1647, 0.1686],
          [0.1961, 0.1922, 0.1882,  ..., 0.1686, 0.1686, 0.1725],
          [0.2000, 0.2000, 0.1961,  ..., 0.1804, 0.1804, 0.1843],
          ...,
          [0.3686, 0.3882, 0.3961,  ..., 0.3098, 0.3098, 0.3098],
          [0.3765, 0.3882, 0.3882,  ..., 0.2980, 0.2980, 0.2980],
          [0.3725, 0.3804, 0.3804,  ..., 0.2941, 0.2941, 0.2941]],

         [[0.1922, 0.1882, 0.1843,  ..., 0.1608, 0.1647, 0.1686],
          [0.1961, 0.1922, 0.1882,  ..., 0.1686, 0.1686, 0.1725],
          [0.2000, 0.2000, 0.1961,  ..., 0.1804, 0.1804, 0.1843],
          ...,
          [0.3686, 0.3882, 0.3961,  ..., 0.3098, 0.3098, 0.3098],
          [0.3765, 0.3882, 0.3882,  ..., 0.2980, 0.2980, 0.2980],
          [0.3725, 0.3804, 0.3804,  ..., 0.2941, 0.2941, 0.2941]]]]), tensor([1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1])]

进程已结束,退出代码为 0

'''

这里用到的文件夹如图:

注意:这里主要写 

def __init__(self,path):
def __len__(self):
def __getitem__(self, item):

这三个函数

3.图像分类常用的类 ImageFolder

        ImageFolder 使用示例:

        首先整理图像分类分别放在不同的文件夹里面:

然后直接使用 ImageFolder 装载 dataset 文件夹,就会自动分类图片形成数据集可以直接使用:

import torch
from torch.utils.data import Dataset
from torchvision import datasets
from torch.utils.data import DataLoader
from torchvision import transforms


trans = transforms.Compose([transforms.Resize((96,96)),transforms.ToTensor()])
ds = datasets.ImageFolder("./dataset",transform=trans)

test_ds,train_ds = torch.utils.data.random_split(ds,[len(ds)//5,len(ds)-len(ds)//5])#注意这里需要整除因为这里使用整数
dl = DataLoader(train_ds,batch_size=16,shuffle=True)

print(ds.classes)
print(ds.class_to_idx)
print(len(test_ds))
print(len(train_ds))
print(next(iter(dl)))


'''
D:\anaconda3\python.exe E:\test\pythonProject\test.py 
['cloudy', 'rain', 'shine', 'sunrise']
{'cloudy': 0, 'rain': 1, 'shine': 2, 'sunrise': 3}
225
900
[tensor([[[[0.0980, 0.0745, 0.0706,  ..., 0.4431, 0.4314, 0.4157],
          [0.0627, 0.0667, 0.0706,  ..., 0.4941, 0.4510, 0.4510],
          [0.1529, 0.1451, 0.1412,  ..., 0.3882, 0.4275, 0.4510],
          ...,
          [0.1176, 0.1176, 0.1176,  ..., 0.1333, 0.1255, 0.1608],
          [0.1137, 0.1137, 0.1137,  ..., 0.1373, 0.1569, 0.2039],
          [0.1098, 0.1098, 0.1098,  ..., 0.1294, 0.1961, 0.2824]],

         [[0.2745, 0.2314, 0.2118,  ..., 0.3843, 0.3725, 0.3569],
          [0.1922, 0.1765, 0.1686,  ..., 0.4353, 0.3922, 0.3922],
          [0.2275, 0.2000, 0.1843,  ..., 0.3294, 0.3725, 0.3961],
          ...,
          [0.0353, 0.0353, 0.0353,  ..., 0.0784, 0.0667, 0.1059],
          [0.0314, 0.0314, 0.0314,  ..., 0.0784, 0.0824, 0.1216],
          [0.0275, 0.0275, 0.0275,  ..., 0.0745, 0.1137, 0.1725]],

         [[0.4471, 0.4118, 0.3961,  ..., 0.3647, 0.3529, 0.3373],
          [0.3490, 0.3373, 0.3333,  ..., 0.4235, 0.3804, 0.3765],
          [0.3529, 0.3333, 0.3255,  ..., 0.3216, 0.3608, 0.3882],
          ...,
          [0.0235, 0.0235, 0.0235,  ..., 0.0431, 0.0353, 0.0549],
          [0.0196, 0.0196, 0.0196,  ..., 0.0471, 0.0392, 0.0392],
          [0.0157, 0.0157, 0.0157,  ..., 0.0353, 0.0549, 0.0706]]],


        [[[0.0941, 0.0941, 0.0196,  ..., 0.1490, 0.1961, 0.1490],
          [0.1059, 0.1137, 0.0471,  ..., 0.1529, 0.1412, 0.1176],
          [0.0745, 0.1255, 0.1059,  ..., 0.1569, 0.1373, 0.1176],
          ...,
          [0.2196, 0.2549, 0.3059,  ..., 0.4000, 0.3922, 0.3765],
          [0.2118, 0.2471, 0.3020,  ..., 0.3804, 0.3686, 0.3608],
          [0.1922, 0.2235, 0.2784,  ..., 0.3882, 0.3843, 0.3725]],

         [[0.2000, 0.1725, 0.0431,  ..., 0.1686, 0.2196, 0.1569],
          [0.2196, 0.2039, 0.0706,  ..., 0.1765, 0.1647, 0.1373],
          [0.2000, 0.2275, 0.1373,  ..., 0.1804, 0.1608, 0.1412],
          ...,
          [0.2157, 0.2510, 0.3059,  ..., 0.3804, 0.3686, 0.3647],
          [0.2118, 0.2471, 0.3020,  ..., 0.3686, 0.3529, 0.3569],
          [0.1922, 0.2235, 0.2784,  ..., 0.3843, 0.3804, 0.3686]],

         [[0.1961, 0.1765, 0.0627,  ..., 0.1725, 0.2196, 0.1647],
          [0.2118, 0.2039, 0.0941,  ..., 0.1804, 0.1647, 0.1451],
          [0.1882, 0.2235, 0.1569,  ..., 0.1843, 0.1608, 0.1608],
          ...,
          [0.1961, 0.2314, 0.2980,  ..., 0.3804, 0.3686, 0.3608],
          [0.1961, 0.2314, 0.2941,  ..., 0.3647, 0.3529, 0.3490],
          [0.1843, 0.2118, 0.2706,  ..., 0.3765, 0.3725, 0.3608]]],


        [[[0.7804, 0.7804, 0.7804,  ..., 0.6627, 0.6588, 0.6549],
          [0.7765, 0.7765, 0.7765,  ..., 0.6588, 0.6549, 0.6510],
          [0.7725, 0.7725, 0.7725,  ..., 0.6471, 0.6431, 0.6431],
          ...,
          [0.1216, 0.1333, 0.1490,  ..., 0.1647, 0.1647, 0.1608],
          [0.1216, 0.1255, 0.1451,  ..., 0.1725, 0.1725, 0.1765],
          [0.1176, 0.1255, 0.1451,  ..., 0.1686, 0.1569, 0.1451]],

         [[0.7843, 0.7843, 0.7843,  ..., 0.6667, 0.6627, 0.6588],
          [0.7804, 0.7804, 0.7804,  ..., 0.6627, 0.6588, 0.6549],
          [0.7765, 0.7765, 0.7765,  ..., 0.6510, 0.6471, 0.6471],
          ...,
          [0.1608, 0.1490, 0.1373,  ..., 0.1686, 0.1686, 0.1647],
          [0.1569, 0.1451, 0.1294,  ..., 0.1765, 0.1765, 0.1804],
          [0.1569, 0.1412, 0.1294,  ..., 0.1725, 0.1608, 0.1490]],

         [[0.8039, 0.8039, 0.8039,  ..., 0.6863, 0.6824, 0.6784],
          [0.8000, 0.8000, 0.8000,  ..., 0.6824, 0.6784, 0.6745],
          [0.7961, 0.7961, 0.7961,  ..., 0.6706, 0.6667, 0.6667],
          ...,
          [0.0706, 0.0667, 0.0745,  ..., 0.1059, 0.1059, 0.1020],
          [0.0745, 0.0667, 0.0745,  ..., 0.1137, 0.1137, 0.1176],
          [0.0745, 0.0706, 0.0745,  ..., 0.1098, 0.0980, 0.0863]]],


        ...,


        [[[0.0275, 0.1059, 0.2157,  ..., 0.0196, 0.0196, 0.0196],
          [0.0235, 0.1020, 0.1765,  ..., 0.0235, 0.0235, 0.0196],
          [0.0196, 0.0902, 0.1255,  ..., 0.0314, 0.0314, 0.0275],
          ...,
          [0.0784, 0.1059, 0.1255,  ..., 0.1294, 0.1020, 0.0745],
          [0.0745, 0.0863, 0.1020,  ..., 0.0627, 0.0588, 0.0431],
          [0.0588, 0.0667, 0.0824,  ..., 0.0667, 0.0627, 0.0353]],

         [[0.0275, 0.1059, 0.2157,  ..., 0.0157, 0.0157, 0.0157],
          [0.0235, 0.1020, 0.1765,  ..., 0.0235, 0.0235, 0.0196],
          [0.0196, 0.0902, 0.1255,  ..., 0.0314, 0.0314, 0.0275],
          ...,
          [0.0588, 0.0863, 0.1059,  ..., 0.1059, 0.0824, 0.0549],
          [0.0549, 0.0667, 0.0824,  ..., 0.0471, 0.0431, 0.0275],
          [0.0392, 0.0471, 0.0627,  ..., 0.0588, 0.0510, 0.0275]],

         [[0.0275, 0.1059, 0.2157,  ..., 0.0275, 0.0275, 0.0235],
          [0.0235, 0.1020, 0.1765,  ..., 0.0314, 0.0314, 0.0275],
          [0.0196, 0.0902, 0.1255,  ..., 0.0392, 0.0392, 0.0353],
          ...,
          [0.0471, 0.0745, 0.0941,  ..., 0.1059, 0.0824, 0.0549],
          [0.0431, 0.0549, 0.0706,  ..., 0.0431, 0.0392, 0.0235],
          [0.0275, 0.0353, 0.0510,  ..., 0.0510, 0.0471, 0.0235]]],


        [[[0.1412, 0.1412, 0.1412,  ..., 0.1647, 0.1686, 0.1765],
          [0.1451, 0.1373, 0.1333,  ..., 0.1647, 0.1686, 0.1765],
          [0.1490, 0.1412, 0.1373,  ..., 0.1725, 0.1765, 0.1843],
          ...,
          [0.0039, 0.0039, 0.0039,  ..., 0.0118, 0.0078, 0.0078],
          [0.0039, 0.0039, 0.0039,  ..., 0.0078, 0.0039, 0.0039],
          [0.0039, 0.0039, 0.0039,  ..., 0.0078, 0.0039, 0.0039]],

         [[0.2118, 0.2078, 0.2078,  ..., 0.2353, 0.2353, 0.2353],
          [0.2157, 0.2118, 0.2078,  ..., 0.2392, 0.2392, 0.2431],
          [0.2196, 0.2157, 0.2118,  ..., 0.2431, 0.2431, 0.2431],
          ...,
          [0.0039, 0.0039, 0.0039,  ..., 0.0118, 0.0078, 0.0078],
          [0.0039, 0.0039, 0.0039,  ..., 0.0078, 0.0039, 0.0039],
          [0.0039, 0.0039, 0.0039,  ..., 0.0078, 0.0039, 0.0039]],

         [[0.3137, 0.3137, 0.3216,  ..., 0.3373, 0.3373, 0.3255],
          [0.3176, 0.3137, 0.3216,  ..., 0.3412, 0.3412, 0.3412],
          [0.3137, 0.3176, 0.3294,  ..., 0.3451, 0.3451, 0.3451],
          ...,
          [0.0039, 0.0039, 0.0039,  ..., 0.0118, 0.0078, 0.0078],
          [0.0039, 0.0039, 0.0039,  ..., 0.0078, 0.0039, 0.0039],
          [0.0039, 0.0039, 0.0039,  ..., 0.0078, 0.0039, 0.0039]]],


        [[[0.0157, 0.0157, 0.0157,  ..., 0.0980, 0.0941, 0.0824],
          [0.0196, 0.0196, 0.0196,  ..., 0.0980, 0.0941, 0.0824],
          [0.0235, 0.0235, 0.0235,  ..., 0.0980, 0.0941, 0.0824],
          ...,
          [0.0078, 0.0078, 0.0039,  ..., 0.0157, 0.0196, 0.0196],
          [0.0039, 0.0039, 0.0039,  ..., 0.0157, 0.0118, 0.0039],
          [0.0000, 0.0000, 0.0000,  ..., 0.0157, 0.0078, 0.0000]],

         [[0.0510, 0.0510, 0.0510,  ..., 0.1294, 0.1255, 0.1333],
          [0.0549, 0.0549, 0.0549,  ..., 0.1294, 0.1255, 0.1333],
          [0.0588, 0.0588, 0.0588,  ..., 0.1294, 0.1255, 0.1333],
          ...,
          [0.0078, 0.0078, 0.0039,  ..., 0.0118, 0.0157, 0.0157],
          [0.0039, 0.0039, 0.0039,  ..., 0.0118, 0.0078, 0.0000],
          [0.0000, 0.0000, 0.0000,  ..., 0.0118, 0.0039, 0.0000]],

         [[0.1647, 0.1647, 0.1647,  ..., 0.2824, 0.2784, 0.2706],
          [0.1686, 0.1686, 0.1686,  ..., 0.2824, 0.2784, 0.2706],
          [0.1725, 0.1725, 0.1725,  ..., 0.2824, 0.2784, 0.2706],
          ...,
          [0.0157, 0.0157, 0.0118,  ..., 0.0353, 0.0392, 0.0392],
          [0.0118, 0.0118, 0.0118,  ..., 0.0353, 0.0314, 0.0235],
          [0.0078, 0.0078, 0.0078,  ..., 0.0353, 0.0275, 0.0196]]]]), tensor([3, 1, 0, 3, 3, 2, 1, 0, 0, 0, 2, 3, 0, 0, 3, 3])]

进程已结束,退出代码为 0

'''

注意:这里使用函数

train_ds,test_ds = torch.utils.data.random_split(ds,[len(ds)//5,len(ds)-len(ds)//5])#注意这里需要整除,因为这里需要使用整数。

        把数据集分为了训练和测试数据集,从Dataset继承的类都可以用这个分类,记住DatasetDataLoader这个基础类是在torch里面,而关于图片的处理类基本都在torchvision 里面,比如图片的转换到tensor,图片放大缩小功能。

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.coloradmin.cn/o/1477325.html

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈,一经查实,立即删除!

相关文章

【QT+QGIS跨平台编译】之五十六:【QGIS_CORE跨平台编译】—【qgsmeshcalclexer.cpp生成】

文章目录 一、Flex二、生成来源三、构建过程一、Flex Flex (fast lexical analyser generator) 是 Lex 的另一个替代品。它经常和自由软件 Bison 语法分析器生成器 一起使用。Flex 最初由 Vern Paxson 于 1987 年用 C 语言写成。 “flex 是一个生成扫描器的工具,能够识别文本中…

【Python笔记-设计模式】状态模式

一、说明 状态模式是一种行为设计模式&#xff0c;用于解决对象在不同状态下具有不同行为 (一) 解决问题 在对象行为根据对象状态而改变时&#xff0c;规避使用大量的条件语句来判断对象的状态&#xff0c;提高系统可维护性 (二) 使用场景 当对象的行为取决于其状态&#…

vue 部署后修改配置文件(接口IP)

近期&#xff0c;有一个项目&#xff0c;运维在部署的时候&#xff0c;接口ip还没有确定&#xff0c;而且ip后面的路径一直有变动&#xff0c;导致我这里一天打包至少四五次才行&#xff0c;很麻烦&#xff0c;然后看了下有没有打包后修改配置文件修改接口ip的方法&#xff0c;…

kubectl使用及源码阅读

目录 概述实践样例yaml 中的必须字段 kubectl 代码原理kubectl 命令行设置pprof 抓取火焰图kubectl 中的 cobra 七大分组命令kubectl createcreateCmd中的builder模式createCmd中的visitor访问者模式外层VisitorFunc分析 结束 概述 k8s 版本 v1.24.16 kubectl的职责 1.主要的…

深入理解网络通信基本原理和tcp/ip协议

深入理解网络通信基本原理和tcp/ip协议 一、计算机网络体系1&#xff0c;计算机网络体系结构2&#xff0c;网络中数据传输2.1&#xff0c;浏览器中输入一个url的执行流程2.2&#xff0c;数据在网络中是的传输流程 3&#xff0c;三次握手和四次挥手3.1&#xff0c;三次握手3.1.1…

代理IP安全问题:在国外使用代理IP是否安全

目录 前言 一、国外使用代理IP的安全风险 1. 数据泄露 2. 恶意软件 3. 网络攻击 4. 法律风险 二、保护国外使用代理IP的安全方法 1. 选择可信的代理服务器 2. 使用加密协议 3. 定期更新系统和软件 4. 注意网络安全意识 三、案例分析 总结 前言 在互联网时代&…

xilinix 的硬件资源分布

从官方手册UG475中可以看出&#xff0c;下图中的V690T系列的i芯片&#xff0c;其具有的bank数量&#xff0c;已经上下半区的bufg对应的bank关系&#xff0c;实际在开发过程中&#xff0c;可能面临局部资源集中度过高&#xff0c;导致bufg的数量不够的情况&#xff0c;bufg的位置…

Docker数据集与自定义镜像:构建高效容器的关键要素

目录 博客前言 一.数据卷 1.数据卷介绍 2.实战 宿主机和容器共享目录 容器和容器之间共享目录 二.自定义镜像 1.自定义镜像介绍 2.实战 2.1自定义centos&#xff0c;具备vim及ifconfig作用 构建镜像 通过镜像运行一个容器进行测试 2.2自定义tomact&#xff08;文件为…

人工智能_Centos7.9中CPU安装ChatGLM3-6B大模型_安装使用_010---人工智能工作笔记0105

从一个空的虚拟机开始安装: https://www.modelscope.cn/models/ZhipuAI/chatglm3-6b/files 可以看到这里有很多的数据文件,那么这里 这里点击模型文件就可以下载,这个就是chatglm3-6B的文件,需要点击每个文件,然后点击右边的下载,把文件都下载下来

Python3中真真假假True、False、None等含义详解

在Python中&#xff0c;不仅仅和类C一样的真假类似&#xff0c;比如1代表真&#xff0c;0代表假。Python中的真假有着更加广阔的含义范围&#xff0c;Python会把所有的空数据结构视为假&#xff0c;比如 [] (空列表)、 {} &#xff08;空集合&#xff09;、 &#xff08;空字符…

初探2b blender

总结 按照youtube 教程 做了个雏形 心得 从正面, 侧面 视图整体上调整外轮廓流线型趋向, 比如正面看这个发型像个鸡蛋的外轮廓头发重要的是丝滑, 流畅 集束 层次 交错

Flutter开发进阶之Flutter Web加载速度优化

Flutter开发进阶之Flutter Web加载速度优化 通常使用Flutter开发的web加载速度会比较慢&#xff0c;原因是Flutter web需要加载的资源处于国外&#xff0c;以下是据此所做的相应优化。 一、FlutterWeb打包 flutter build web --web-renderer canvaskit使用新命令打包 flut…

面试经典150题——插入区间

"The future belongs to those who believe in the beauty of their dreams." - Eleanor Roosevelt 1. 题目描述 2. 题目分析与解析 2.1 思路一 解决这个问题的思路是基于区间排序和合并的经典算法。这个问题的关键在于如何处理新区间与现有区间的关系&#xff0c…

【C++入门】C++关键字 | 命名空间 | C++的输入输出

目录 0.C与C 1.C的关键字 2.命名空间 2.1域 2.2C中命名冲突问题 2.3命名空间定义 2.4命名空间使用 2.5命令空间的展开&头文件的展开 3.C的输入&输出 3.1cout&cin 3.1<<流插入运算符 3.2>>流提取运算符 0.C与C C是在C的基础之上&#xff…

九州金榜|家庭教育中孩子厌学的原因有哪些?

孩子的成长往往会受到家庭教育&#xff0c;社会环境&#xff0c;学校教育等因素影响&#xff0c;孩子厌学受家庭教育影响是一个重要因素&#xff0c;都是哪些因素会让孩子产生厌学情绪呢&#xff1f;下面九州金榜家庭教育分析一下因为家庭教育因素而让孩子产生厌学的原因&#…

Redis学习------实战篇----2024/02/28

1.集群的session共享问题 2.基于Redis实现共享session登录 //4.保存验证码到redisstringRedisTemplate.opsForValue().set(LOGIN_CODE_KEYphone,code,LOGIN_CODE_TTL, TimeUnit.MINUTES);RedisTemplate RedisTemplate使用的是JdkSerializationRedisSerializer存入数据&#xff…

不看后悔的2024年腾讯云服务器购买优惠活动汇总

腾讯云服务器多少钱一年&#xff1f;62元一年起&#xff0c;2核2G3M配置&#xff0c;腾讯云2核4G5M轻量应用服务器218元一年、756元3年&#xff0c;4核16G12M服务器32元1个月、312元一年&#xff0c;8核32G22M服务器115元1个月、345元3个月&#xff0c;腾讯云服务器网txyfwq.co…

二叉树(C/C++)

本篇将较为详细的介绍二叉树的相关知识&#xff0c;以及二叉树的实现。对于二叉树的相关知识&#xff0c;本篇介绍了其概念、特殊的二叉树、性质还有存储结构。 接着对于实现二叉树的每个函数都有其思路讲解&#xff0c;主要的函数分为&#xff1a;遍历&#xff1a;前中后序遍历…

redis-Redis主从,哨兵和集群模式

一&#xff0c;Redis的主从复制 ​ 主机数据更新后根据配置和策略&#xff0c; 自动同步到备机的master/slaver机制&#xff0c;Master以写为主&#xff0c;Slave以读为主。这样做的好处是读写分离&#xff0c;性能扩展&#xff0c;容灾快速恢复。 1.1 环境搭建 如果你的redi…

【HDFS】Decommision(退役) EC数据节点剩最后几个块卡住的问题

一、背景 近期操作退役EC集群的节点。在退役的过程中,遇到了一些问题。特此总结一下。 本文描述的问题现象是: 每一批次退役10个节点,完全退役成功后开始操作下一批。 但是,中间有一批次有2台节点的Under Replicated Blocks一直是1,不往下降。 处于Decommissioning状态卡…