YOLOv5-C3模块实现

news2024/11/26 6:19:30
  • 🍨 本文为🔗365天深度学习训练营 中的学习记录博客

  • 🍦 参考文章地址: 365天深度学习训练营-第P8周:YOLOv5-C3模块实现

  • 🍖 作者:K同学啊

一、前期准备

1.设置GPU

import torch
from torch import nn
import torchvision
from torchvision import transforms,datasets,models
import matplotlib.pyplot as plt
import os,PIL,pathlib
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
device

device(type='cuda')

2.导入数据

data_dir = './weather_photos/'
data_dir = pathlib.Path(data_dir)

data_paths = list(data_dir.glob('*'))
classNames = [str(path).split('\\')[1] for path in data_paths]
classNames

['cloudy', 'rain', 'shine', 'sunrise']

train_transforms = transforms.Compose([
    transforms.Resize([224,224]),# resize输入图片
    transforms.ToTensor(), # 将PIL Image或numpy.ndarray转换成tensor
    transforms.Normalize(
        mean = [0.485, 0.456, 0.406],
        std = [0.229,0.224,0.225]) # 从数据集中随机抽样计算得到
])

test_transforms = transforms.Compose([
    transforms.Resize([224,224]),# resize输入图片
    transforms.ToTensor(), # 将PIL Image或numpy.ndarray转换成tensor
    transforms.Normalize(
        mean = [0.485, 0.456, 0.406],
        std = [0.229,0.224,0.225]) # 从数据集中随机抽样计算得到
])

total_data = datasets.ImageFolder(data_dir,transform=train_transforms)
total_data

Dataset ImageFolder

Number of datapoints: 1125

Root location: weather_photos

StandardTransform

Transform: Compose(

Resize(size=[224, 224], interpolation=PIL.Image.BILINEAR)

ToTensor()

Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])

)

total_data.class_to_idx

{'cloudy': 0, 'rain': 1, 'shine': 2, 'sunrise': 3}

3.划分数据集

train_size = int(0.8*len(total_data))
test_size = len(total_data) - train_size
train_dataset, test_dataset = torch.utils.data.random_split(total_data,[train_size,test_size])
train_dataset,test_dataset

(<torch.utils.data.dataset.Subset at 0x1e42b97f4f0>,

<torch.utils.data.dataset.Subset at 0x1e42b196a30>)

batch_size = 4
train_dl = torch.utils.data.DataLoader(train_dataset,
                                       batch_size=batch_size,
                                       shuffle=True,
                                       num_workers=1)
test_dl = torch.utils.data.DataLoader(test_dataset,
                                       batch_size=batch_size,
                                       shuffle=True,
                                       num_workers=1)
for X,y in test_dl:
    print('Shape of X [N, C, H, W]:', X.shape)
    print('Shape of y:', y.shape)
    break

Shape of X [N, C, H, W]: torch.Size([4, 3, 224, 224])

Shape of y: torch.Size([4])

二、搭建包含C3模块的模型

1.搭建模型

import torch.nn.functional as F

def autopad(k, p=None):  # kernel, padding
    # Pad to 'same'
    if p is None:
        p = k // 2 if isinstance(k, int) else [x // 2 for x in k]  # auto-pad
    return p

class Conv(nn.Module):
    # Standard convolution
    def __init__(self, c1, c2, k=1, s=1, p=None, g=1, act=True):  # ch_in, ch_out, kernel, stride, padding, groups
        super().__init__()
        self.conv = nn.Conv2d(c1, c2, k, s, autopad(k, p), groups=g, bias=False)
        self.bn = nn.BatchNorm2d(c2)
        self.act = nn.SiLU() if act is True else (act if isinstance(act, nn.Module) else nn.Identity())

    def forward(self, x):
        return self.act(self.bn(self.conv(x)))

class Bottleneck(nn.Module):
    # Standard bottleneck
    def __init__(self, c1, c2, shortcut=True, g=1, e=0.5):  # ch_in, ch_out, shortcut, groups, expansion
        super().__init__()
        c_ = int(c2 * e)  # hidden channels
        self.cv1 = Conv(c1, c_, 1, 1)
        self.cv2 = Conv(c_, c2, 3, 1, g=g)
        self.add = shortcut and c1 == c2

    def forward(self, x):
        return x + self.cv2(self.cv1(x)) if self.add else self.cv2(self.cv1(x))

class C3(nn.Module):
    # CSP Bottleneck with 3 convolutions
    def __init__(self, c1, c2, n=1, shortcut=True, g=1, e=0.5):  # ch_in, ch_out, number, shortcut, groups, expansion
        super().__init__()
        c_ = int(c2 * e)  # hidden channels
        self.cv1 = Conv(c1, c_, 1, 1)
        self.cv2 = Conv(c1, c_, 1, 1)
        self.cv3 = Conv(2 * c_, c2, 1)  # act=FReLU(c2)
        self.m = nn.Sequential(*(Bottleneck(c_, c_, shortcut, g, e=1.0) for _ in range(n)))

    def forward(self, x):
        return self.cv3(torch.cat((self.m(self.cv1(x)), self.cv2(x)), dim=1))

class model_K(nn.Module):
    def __init__(self):
        super(model_K, self).__init__()
        
        # 卷积模块
        self.Conv = Conv(3, 32, 3, 2) 
        
        # C3模块1
        self.C3_1 = C3(32, 64, 3, 2)
        
        # 全连接网络层,用于分类
        self.classifier = nn.Sequential(
            nn.Linear(in_features=802816, out_features=100),
            nn.ReLU(),
            nn.Linear(in_features=100, out_features=4)
        )
        
    def forward(self, x):
        x = self.Conv(x)
        x = self.C3_1(x)
        x = torch.flatten(x, start_dim=1)
        x = self.classifier(x)

        return x

device = "cuda" if torch.cuda.is_available() else "cpu"
print("Using {} device".format(device))
    
model = model_K().to(device)
model

Using cuda device

Out[9]:

model_K(

(Conv): Conv(

(conv): Conv2d(3, 32, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

(C3_1): C3(

(cv1): Conv(

(conv): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

(cv2): Conv(

(conv): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

(cv3): Conv(

(conv): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

(m): Sequential(

(0): Bottleneck(

(cv1): Conv(

(conv): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

(cv2): Conv(

(conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

)

(1): Bottleneck(

(cv1): Conv(

(conv): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

(cv2): Conv(

(conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

)

(2): Bottleneck(

(cv1): Conv(

(conv): Conv2d(32, 32, kernel_size=(1, 1), stride=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

(cv2): Conv(

(conv): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)

(bn): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)

(act): SiLU()

)

)

)

)

(classifier): Sequential(

(0): Linear(in_features=802816, out_features=100, bias=True)

(1): ReLU()

(2): Linear(in_features=100, out_features=4, bias=True)

)

)

2.查看模型详情

import torchsummary as summary
summary.summary(model,(3,224,224))

0

Conv2d-9 [-1, 32, 112, 112] 1,024

BatchNorm2d-10 [-1, 32, 112, 112] 64

SiLU-11 [-1, 32, 112, 112] 0

Conv-12 [-1, 32, 112, 112] 0

Conv2d-13 [-1, 32, 112, 112] 9,216

BatchNorm2d-14 [-1, 32, 112, 112] 64

SiLU-15 [-1, 32, 112, 112] 0

Conv-16 [-1, 32, 112, 112] 0

Bottleneck-17 [-1, 32, 112, 112] 0

Conv2d-18 [-1, 32, 112, 112] 1,024

BatchNorm2d-19 [-1, 32, 112, 112] 64

SiLU-20 [-1, 32, 112, 112] 0

Conv-21 [-1, 32, 112, 112] 0

Conv2d-22 [-1, 32, 112, 112] 9,216

BatchNorm2d-23 [-1, 32, 112, 112] 64

SiLU-24 [-1, 32, 112, 112] 0

Conv-25 [-1, 32, 112, 112] 0

Bottleneck-26 [-1, 32, 112, 112] 0

Conv2d-27 [-1, 32, 112, 112] 1,024

BatchNorm2d-28 [-1, 32, 112, 112] 64

SiLU-29 [-1, 32, 112, 112] 0

Conv-30 [-1, 32, 112, 112] 0

Conv2d-31 [-1, 32, 112, 112] 9,216

BatchNorm2d-32 [-1, 32, 112, 112] 64

SiLU-33 [-1, 32, 112, 112] 0

Conv-34 [-1, 32, 112, 112] 0

Bottleneck-35 [-1, 32, 112, 112] 0

Conv2d-36 [-1, 32, 112, 112] 1,024

BatchNorm2d-37 [-1, 32, 112, 112] 64

SiLU-38 [-1, 32, 112, 112] 0

Conv-39 [-1, 32, 112, 112] 0

Conv2d-40 [-1, 64, 112, 112] 4,096

BatchNorm2d-41 [-1, 64, 112, 112] 128

SiLU-42 [-1, 64, 112, 112] 0

Conv-43 [-1, 64, 112, 112] 0

C3-44 [-1, 64, 112, 112] 0

Linear-45 [-1, 100] 80,281,700

ReLU-46 [-1, 100] 0

Linear-47 [-1, 4] 404

================================================================

Total params: 80,320,536

Trainable params: 80,320,536

Non-trainable params: 0

----------------------------------------------------------------

Input size (MB): 0.57

Forward/backward pass size (MB): 150.06

Params size (MB): 306.40

Estimated Total Size (MB): 457.04

----------------------------------------------------------------

三、训练模型

1.编写训练函数

# 训练循环
def train(dataloader, model, loss_fn, optimizer):
    size = len(dataloader.dataset)  # 训练集的大小,一共900张图片
    num_batches = len(dataloader)   # 批次数目,29(900/32)

    train_loss, train_acc = 0, 0  # 初始化训练损失和正确率
    
    for X, y in dataloader:  # 获取图片及其标签
        X, y = X.to(device), y.to(device)
        
        # 计算预测误差
        pred = model(X)          # 网络输出
        loss = loss_fn(pred, y)  # 计算网络输出和真实值之间的差距,targets为真实值,计算二者差值即为损失
        
        # 反向传播
        optimizer.zero_grad()  # grad属性归零
        loss.backward()        # 反向传播
        optimizer.step()       # 每一步自动更新
        
        # 记录acc与loss
        train_acc  += (pred.argmax(1) == y).type(torch.float).sum().item()
        train_loss += loss.item()
            
    train_acc  /= size
    train_loss /= num_batches

    return train_acc, train_loss

2.编写测试函数

def test (dataloader, model, loss_fn):
    size        = len(dataloader.dataset)  # 测试集的大小,一共10000张图片
    num_batches = len(dataloader)          # 批次数目,8(255/32=8,向上取整)
    test_loss, test_acc = 0, 0
    
    # 当不进行训练时,停止梯度更新,节省计算内存消耗
    with torch.no_grad():
        for imgs, target in dataloader:
            imgs, target = imgs.to(device), target.to(device)
            
            # 计算loss
            target_pred = model(imgs)
            loss        = loss_fn(target_pred, target)
            
            test_loss += loss.item()
            test_acc  += (target_pred.argmax(1) == target).type(torch.float).sum().item()

    test_acc  /= size
    test_loss /= num_batches

    return test_acc, test_loss

3、正式训练

import copy

optimizer = torch.optim.Adam(model.parameters(),lr=1e-4)
# scheduler = torch.optim.lr_scheduler.StepLR(optimizer, step_size=5, gamma=0.92)#学习率每5个epoch衰减成原来的1/10
loss_fn = nn.CrossEntropyLoss()

epochs     = 20

train_loss = []
train_acc  = []
test_loss  = []
test_acc   = []

best_acc = 0

for epoch in range(epochs):
    model.train()
    epoch_train_acc, epoch_train_loss = train(train_dl, model, loss_fn, optimizer)
    
    # scheduler.step()#学习率衰减
    
    model.eval()
    epoch_test_acc, epoch_test_loss = test(test_dl, model, loss_fn)
    
    # 保存最优模型
    if epoch_test_acc > best_acc:
        best_acc = epoch_train_acc
        best_model = copy.deepcopy(model)
        
    train_acc.append(epoch_train_acc)
    train_loss.append(epoch_train_loss)
    test_acc.append(epoch_test_acc)
    test_loss.append(epoch_test_loss)
    
    # 获取当前学习率
    lr = optimizer.state_dict()['param_groups'][0]['lr']
    
    template = ('Epoch:{:2d}, Train_acc:{:.1f}%, Train_loss:{:.3f}, Test_acc:{:.1f}%,Test_loss:{:.3f},Lr:{:.2E}')
    print(template.format(epoch+1, epoch_train_acc*100, epoch_train_loss, epoch_test_acc*100, epoch_test_loss, lr))

# 保存最佳模型
PATH = './best_model.pth'
torch.save(model.state_dict(),PATH)
   
print('Done')
print('best_acc:',best_acc)

...

Epoch:18, Train_acc:99.1%, Train_loss:0.043, Test_acc:84.9%,Test_loss:1.605,Lr:1.00E-04

Epoch:19, Train_acc:99.8%, Train_loss:0.009, Test_acc:89.8%,Test_loss:1.085,Lr:1.00E-04

Epoch:20, Train_acc:99.4%, Train_loss:0.014, Test_acc:89.3%,Test_loss:1.053,Lr:1.00E-04

Done

best_acc: 0.9666666666666667

四、结果可视化

1.Loss与Accuracy图

import matplotlib.pyplot as plt
#隐藏警告
import warnings
warnings.filterwarnings("ignore")               #忽略警告信息
plt.rcParams['font.sans-serif']    = ['SimHei'] # 用来正常显示中文标签
plt.rcParams['axes.unicode_minus'] = False      # 用来正常显示负号
plt.rcParams['figure.dpi']         = 100        #分辨率

epochs_range = range(epochs)

plt.figure(figsize=(12, 3))
plt.subplot(1, 2, 1)

plt.plot(epochs_range, train_acc, label='Training Accuracy')
plt.plot(epochs_range, test_acc, label='Test Accuracy')
plt.legend(loc='lower right')
plt.title('Training and Validation Accuracy')

plt.subplot(1, 2, 2)
plt.plot(epochs_range, train_loss, label='Training Loss')
plt.plot(epochs_range, test_loss, label='Test Loss')
plt.legend(loc='upper right')
plt.title('Training and Validation Loss')
plt.show()

2.模型评估

best_model.eval()
epoch_test_acc,epoch_test_loss = test(test_dl,best_model,loss_fn)
epoch_test_acc,epoch_test_loss

(0.8844444444444445, 1.0431718131294474)

# 查看是否与我们最高准确率一致
epoch_test_acc

0.8844444444444445

本文来自互联网用户投稿,该文观点仅代表作者本人,不代表本站立场。本站仅提供信息存储空间服务,不拥有所有权,不承担相关法律责任。如若转载,请注明出处:http://www.coloradmin.cn/o/143627.html

如若内容造成侵权/违法违规/事实不符,请联系多彩编程网进行投诉反馈,一经查实,立即删除!

相关文章

数据库,计算机网络、操作系统刷题笔记26

数据库&#xff0c;计算机网络、操作系统刷题笔记26 2022找工作是学历、能力和运气的超强结合体&#xff0c;遇到寒冬&#xff0c;大厂不招人&#xff0c;可能很多算法学生都得去找开发&#xff0c;测开 测开的话&#xff0c;你就得学数据库&#xff0c;sql&#xff0c;oracle…

消息队列如何保证消息幂等性消费

1 介绍 我们实际系统中有很多操作&#xff0c;不管你执行多少次&#xff0c;都应该产生一样的效果或返回一样的结果。 例如&#xff1a; 前端页面重复提交选中的数据&#xff0c;服务端只产生对应这个数据的一个反应结果&#xff0c;只保存一次数据。我们发起一笔付款请求&am…

裸露土堆智能识别检测系统 yolo

裸露土堆智能识别检测系统基于pythonyolo计算机视觉深度学习技术&#xff0c;对现场画面中土堆裸露情况进行实时分析检测&#xff0c;若发现画面中的土堆有超过40%部分裸露&#xff0c;则判定为裸露进行抓拍预警。我们选择当下YOLO最新的卷积神经网络YOLOv5来进行裸露土堆识别检…

商用密码安全性评估

商用密码应用安全性评估&#xff08;简称“密评”&#xff09;指在采用商用密码技术、产品和服务集成建设的网络和信息系统中&#xff0c;对其密码应用的合规性、正确性和有效性等进行评估。01办理依据 GM/T0054-2018《信息系统密码应用基本要求》 《信息系统密码测评要求&…

Linux内核内存分配函数kmalloc、kzalloc和vmalloc

在内核环境中&#xff0c;常用的内存分配函数主要有kmalloc、kzalloc和vmalloc这三个。既然这三函数都能在内核申请空间&#xff0c;那么这三个函数有什么区别呢&#xff1f;如何选用呢&#xff1f; kmalloc 首先是kmalloc&#xff0c;其函数原型为 // /include/linux/slab.…

acwing基础课——质数

由数据范围反推算法复杂度以及算法内容 - AcWing 常用代码模板4——数学知识 - AcWing 基本思想&#xff1a; 首先&#xff0c;我们给出质数的定义&#xff0c;指在大于1的自然数中&#xff0c;除了1和该数自身外&#xff0c;无法被其他自然数整除的数。这里考虑三个问题&…

笔记-鼠标悬浮展示图标

鼠标悬浮展示图标 .primaryLink {color: primary-color-dark;}.primaryLink:hover {cursor: pointer;color: link-hover-color-dark;}.itemAction {display: none; }.itemMenu:hover .itemAction {display: block; }

【数据结构进阶】并查集

并查集 正如它的名字一样&#xff0c;并查集&#xff08;Union-Find&#xff09;就是用来对集合进行 合并&#xff08;Union&#xff09; 与 查询&#xff08;Find&#xff09; 操作的一种数据结构。 合并 就是将两个不相交的集合合并成一个集合。 查询 就是查询两个元素是否属…

链表常见OJ题汇总(持续更新)

目录前言一、移除链表中的元素&#xff08;多指针法&#xff09;二、反转链表&#xff08;多指针法&头插法&#xff09;三、链表的中间结点&#xff08;算数法和双指针法&#xff09;四、链表中的第K个结点&#xff08;算数法&双指针法&#xff09;五、合并两个有序链表…

vue 父子组件设置 scoped, 如何导致滚动条失效的

vue父组件的页面结构 // 调用子组件 <process-time-line :nodeArr"nodeArr"></process-time-line> 父组件的样式 <style lang"scss" scoped> ::-webkit-scrollbar {width: 0px;height: 0px;} </style>子组件的页面结构 <div …

学习C语言笔记:字符串和格式化输入/输出

学习内容&#xff1a;1.函数——strlen()&#xff1b;2.关键字——const&#xff1b;3.字符串&#xff1b;4..如何创建、存储字符串&#xff1b;5.如何使用strlen()函数获取字符串的长度&#xff1b;6.用C预处理器指令#define和ANSIC的const修饰符创建符号常量。与程序交互和使…

《Linux运维实战:Centos7.6基于docker-compose一键离线部署redis6.2.8之哨兵集群》

一、部署背景 由于业务系统的特殊性&#xff0c;我们需要面向不通的客户安装我们的业务系统&#xff0c;而作为基础组件中的redis针对不同的客户环境需要多次部署哨兵集群&#xff0c;作为一个运维工程师&#xff0c;提升工作效率也是工作中的重要一环。所以我觉得有必要针对re…

(Java高级教程)第三章Java网络编程-第一节3:网络编程必备网络知识3之IP地址、端口号

文章目录一&#xff1a;网络传输基本流程&#xff08;1&#xff09;数据包&#xff08;2&#xff09;网络传输的基本流程&#xff08;3&#xff09;具体处理过程A&#xff1a;发送数据B&#xff1a;路由转发C&#xff1a;接受数据二&#xff1a;网络中的地址&#xff08;1&…

Elasticsearch-使用入门

_cat /_cat/nodes&#xff1a;查看所有节点 接口&#xff1a;GET http://192.168.177.134:9200/_cat/nodes /_cat/health&#xff1a;查看ES健康状况 接口&#xff1a;GET http://192.168.177.134:9200/_cat/health /_cat/master&#xff1a;查看主节点信息 接口&#xff1a;G…

【Azure 架构师学习笔记】-Azure Logic Apps(3)-演示1

本文属于【Azure 架构师学习笔记】系列。 本文属于【Azure Logic Apps】系列。 接上文【Azure 架构师学习笔记】-Azure Logic Apps&#xff08;2&#xff09;-组件介绍 前言 前面两篇文章大概介绍了一些理论知识&#xff0c;但是为用而学才是最重要的&#xff0c;所以接下来做…

word排版时如何保证每张图片大小一样?

问题描述 为了保证文档的美观性&#xff0c;在对图片进行排版时&#xff0c;最好保证图片的大小一致&#xff0c;尤其是多张图片组成一张大图时。 如果一张张图片调整大小&#xff0c;那真的是毫无技术含量的耗时工作。 解决方案 在这提出一种借助表格的解决办法。比如有4张…

Parasoft发布最广泛的MISRA规则覆盖-C/C++test最新版本正式上线!

作为拥有30多年自动化软件测试经验的全球领导者Parasoft宣布发布Parasoft C/Ctest的最新2022.2版本&#xff0c;支持MISRA C:2012修正案3和MISRA C 202x的草案版本。Parasoft针对C和C软件开发的统一、完全集成的测试解决方案的最新版本&#xff0c;帮助团队实现自动化静态分析和…

【java入门系列三】java基础-控制结构

学习记录&#x1f914;分支控制if-elseswitch分支接收字符for循环控制while循环do-while打印金字塔break终止-可以用label&#xff1a;表明continue与break类似return循环中表示直接退出方法(函数)&#xff0c;主方法直接结束字符串比较trick讨论总结谢谢点赞交流&#xff01;(…

外观模式

外观模式 1.外观模式介绍 1.外观模式&#xff08;Facade&#xff09;&#xff0c;也叫“过程模式&#xff1a;外观模式为子系统中的一组接口提供一个一致的界面&#xff0c;此模式定义了一个高层接口&#xff0c;这个接口使得这一子系统更加容易使用 2.外观模式通过定义一个一…

Linux(06)之获取内核代码

Linux(06)之获取内核代码 Author&#xff1a;OnceDay Date&#xff1a;2023年1月5日 漫漫长路&#xff0c;有人对你微笑过嘛… 参考文档&#xff1a; 《Linux内核设计和实现》 1.概述 linux内核的基本架构如下&#xff1a; 所以每个处理器运行的地方只有以下可能&#xf…