PatchCore：工业异常检测中的全面召回

前言
相关介绍
- - PatchCore的工作原理：
  - 优点：
  - 缺点：
实验环境
项目地址
- Linux
- Windows
项目结构
具体用法
- 准备数据
- 进行训练
- 进行测试
常见问题
- ModuleNotFoundError: No module named 'patchcore'
- - 解决方法
- OMP: Error
- - 解决方法
- AttributeError: 'MVTecDataset' object has no attribute 'transform_std'
- - 解决方法
- 保存结果图时总是会将原图覆盖掉
参考文献

在这里插入图片描述

前言

由于本人水平有限，难免出现错漏，敬请批评改正。
更多精彩内容，可点击进入Python日常小操作专栏、OpenCV-Python小应用专栏、YOLO系列专栏、自然语言处理专栏或我的个人主页查看
基于DETR的人脸伪装检测
YOLOv7训练自己的数据集（口罩检测）
YOLOv8训练自己的数据集（足球检测）
YOLOv10训练自己的数据集（交通标志检测）
YOLOv5：TensorRT加速YOLOv5模型推理
YOLOv5：IoU、GIoU、DIoU、CIoU、EIoU
玩转Jetson Nano（五）：TensorRT加速YOLOv5目标检测
YOLOv5：添加SE、CBAM、CoordAtt、ECA注意力机制
YOLOv5：yolov5s.yaml配置文件解读、增加小目标检测层
Python将COCO格式实例分割数据集转换为YOLO格式实例分割数据集
YOLOv5：使用7.0版本训练自己的实例分割模型（车辆、行人、路标、车道线等实例分割）
使用Kaggle GPU资源免费体验Stable Diffusion开源项目

实验环境

click>= 8.0.3
# cudatoolkit>= 10.2
faiss-cpu
#faiss-gpu
matplotlib>= 3.5.0
pillow>= 8.4.0
pretrainedmodels>= 0.7.4
torch>= 1.10.0
scikit-image>= 0.18.3
scikit-learn>= 1.0.1
scipy>= 1.7.1
torchvision>= 0.11.1
tqdm>= 4.62.3

项目地址

PatchCore 源代码地址：https://github.com/amazon-science/patchcore-inspection.git

Linux

git clone https://github.com/amazon-science/patchcore-inspection.git

cd patchcore-inspection
pip install -r requirements.txt

Windows

请到https://github.com/amazon-science/patchcore-inspection.git网站下载源代码zip压缩包。

cd patchcore-inspection
pip install -r requirements.txt

项目结构

patchcore-inspection-main
├─bin
│  ├─load_and_evaluate_patchcore.py
│  └─run_patchcore.py
├─checkpoints
├─images
├─models
│  ├─IM224_Ensemble_L2-3_P001_D1024-384_PS-3_AN-1
│  ├─IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1
│  ├─IM320_Ensemble_L2-3_P001_D1024-384_PS-3_AN-1
│  ├─IM320_Ensemble_L2-3_P001_D1024-384_PS-5_AN-5
│  ├─IM320_WR50_L2-3_P001_D1024-1024_PS-3_AN-1
│  └─IM320_WR50_L2-3_P001_D1024-1024_PS-5_AN-3
├─mvtec
│  ├─bottle
│  │  ├─ground_truth
│  │  │  ├─broken_large
│  │  │  ├─broken_small
│  │  │  └─contamination
│  │  ├─test
│  │  │  ├─broken_large
│  │  │  ├─broken_small
│  │  │  ├─contamination
│  │  │  └─good
│  │  └─train
│  │      └─good
├─src
│  └─patchcore
│      ├─datasets
│      │  └─__pycache__
│      ├─networks
│      └─__pycache__
└─test

具体用法

准备数据

本文以MVTEC公开数据集中的bottle数据集进行示例。

├─mvtec
│  ├─bottle
│  │  ├─ground_truth
│  │  │  ├─broken_large
│  │  │  ├─broken_small
│  │  │  └─contamination
│  │  ├─test
│  │  │  ├─broken_large
│  │  │  ├─broken_small
│  │  │  ├─contamination
│  │  │  └─good
│  │  └─train
│  │      └─good

在这里插入图片描述

进行训练

#　训练
python bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model --save_segmentation_images --log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results model patch_core -b wideresnet50 -le layer2 -le layer3 --pretrain_embed_dimension 1024 --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 --subdatasets "bottle" mvtec patchcore-inspection-main\mvtec

patchcore-inspection-main>python bin/run_patchcore.py --gpu 0 --seed 0 --save_patchcore_model --save_segmentation_images --log_group IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0 --log_project MVTecAD_Results model patch_core -b wideresnet50 -le layer2 -le layer3 --pretrain_embed_dimension 1024 --target_embed_dimension 1024 --anomaly_scorer_num_nn 1 --patchsize 3 sampler -p 0.1 approx_greedy_coreset dataset --resize 256 --imagesize 224 --subdatasets "bottle" mvtec 
INFO:__main__:Training models (1/1)

Computing support features...:   3%|█▍                                                 | 3/105 [00:21<09:38,  5.67s/it
Computing support features...:   5%|██▍                                                | 5/105 [00:21<04:39,  2.80s/i
Computing support features...:   7%|███▍                                               | 7/105 [00:22<02:41,  1.65s/
Computing support features...:   9%|████▎                                              | 9/105 [00:22<01:41,  1.06s
Computing support features...:  10%|█████▏                                            | 11/105 [00:22<01:07,  1.40
Computing support features...:  12%|█
Subsampling...:   3%|█▌                                                          | 421/16385 [00:00<00:14, 1095.01it/sSubsampling...:   3%|█▉                                                          | 538/16385 [00:00<00:14, 1120.04it/sSubsampling...:   4%|██▍                                                         | 652/16385 [00:00<00:13, 1124.00it/Subsampling...:   5%|██▊                                                         | 767/16385 [00:00<00:13, 1129.79it/Subsampling...:   5%|███▏                                                        | 882/16385 [00:00<00:13, 1132.67itSubsampling...:   6%|███▋                                                        | 999/16385 [00:00<00:13, 1144.19itSubsampling...:   7%|████                                                       | 1115/16385 [00:01<00:13, 1147.80itSubsampling...:   8%|████▍                                                      | 1230/16385 [00:01<00:13, 1145.92iSubsampling...:   8%|████▊                                                      | 1345/16385 [00:01<00:13, 1141.07iSubsampling...:   9%|█████▎                                                     | 1460/16385 [00:01<00:13, 1137.73Subsampling...:  10%|█████▋                                                     | 1577/16385 [00:01<00:12, 1144.88Subsampling...:  10%|██████                                                     | 1692/16385 [00:01<00:12, 1140.60Subsampling...:  11%|██████▌                                                    | 1807/16385 [00:01<00:12, 1137.4Subsampling...:  12%|██████▉

INFO:__main__:Embedding test data with models (1/1)
Inferring...:   2%|█▋                                                                   | 1/42 [00:21<14:41, 21.49s/itInferring...:   5%|███▎                                                                 | 2/42 [00:21<06:05,  9.13s/Inferring...:   7%|████▉                                                                | 3/42 [00:22<03:22,  5.18sInferring...:  10%|██████▌                                                              | 4/42 [00:22<02:05,  3.3Inferring...:  12%|████████▏                                                            | 5/42 [00:23<01:23,  2Inferring...:  14%|█████████▊                                                           | 6/42 [00:23<00:58,  Inferring...:  17%|███████████▌                                                         | 7/42 [00:24<00:42,Inferring...:  19%|█████████████▏                                                       | 8/42 [00:24<00:3Inferring...:  21%|██████████████▊                                                      | 9/42 [00:25<00:Inferring...:  24%|█

Generating Segmentation Images...:   2%|█▏                                              | 2/83 [00:00<00:29,  2.76it/sGenerating Segmentation Images...:   4%|█▋                                              | 3/83 [00:01<00:28,  2.82it/sGenerating Segmentation Images...:   5%|██▎                                             | 4/83 [00:01<00:30,  2.56it/Generating Segmentation Images...:   6%|██▉                                             | 5/83 [00:01<00:28,  2.69it/Generating Segmentation Images...:   7%|███▍                                            | 6/83 [00:02<00:27,  2.76itGenerating Segmentation Images...:   8%|████                                            | 7/83 [00:02<00:27,  2.78itGenerating Segmentation Images...:  10%|████▋                                           | 8/83 [00:02<00:26,  2.85iGenerating Segmentation Images...:  11%|█████▏                                          | 9/83 [00:03<00:25,  2.89Generating Segmentation Images...:  12%|█████▋                           

INFO:__main__:Computing evaluation metrics.
INFO:__main__:instance_auroc: 1.000
INFO:__main__:full_pixel_auroc: 0.985
INFO:__main__:anomaly_pixel_auroc: 0.980
INFO:patchcore.patchcore:Saving PatchCore data.
INFO:__main__:

-----

INFO:patchcore.utils:instance_auroc: 1.000
INFO:patchcore.utils:full_pixel_auroc: 0.985
INFO:patchcore.utils:anomaly_pixel_auroc: 0.980

进行测试

＃测试
python bin/load_and_evaluate_patchcore.py --gpu 0 --seed 0 --save_segmentation_images "evaluateAnswer/bottle" patch_core_loader -p "patchcore-inspection-main\model\MVTecAD_Results\IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0_2\models\mvtec_bottle" dataset --resize 256 --imagesize 224 -d "bottle" mvtec "patchcore-inspection-main\mvtec"

patchcore-inspection-main>python bin/load_and_evaluate_patchcore.py --gpu 0 --seed 0 --save_segmentation_images "evaluateAnswer/bottle" patch_core_loader -p "D:\tfx\mytest\patchcore-inspection-main\model\MVTecAD_Results\IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0_2\models\mvtec_bottle" dataset --resize 256 --imagesize 224 -d "bottle" mvtec "D:\tfx\mytest\patchcore-inspection-main\mvtec"
INFO:__main__:Command line arguments: bin/load_and_evaluate_patchcore.py --gpu 0 --seed 0 --save_segmentation_images evaluateAnswer/bottle patch_core_loader -p D:\tfx\mytest\patchcore-inspection-main\model\MVTecAD_Results\IM224_WR50_L2-3_P01_D1024-1024_PS-3_AN-1_S0_2\models\mvtec_bottle dataset --resize 256 --imagesize 224 -d bottle mvtec 
INFO:__main__:Embedding test data with models (1/1)
Inferring...:   2%|█▋                                                                   | 2/83 [00:21<12:15,  9.08s/itInferring...:   4%|██▍                                                                  | 3/83 [00:22<06:43,  5.05s/iInferring...:   5%|███▎       
Generating Segmentation Images...:   2%|█▏                                              | 2/83 [00:00<00:33,  2.38it/sGenerating Segmentation Images...:   4%|█▋                                              | 3/83 [00:01<00:30,  2.64it/s
Generating Segmentation Images...:   5%|██▎             
Generating Segmentation Images...: 100%|████████████████████████████████████████INFO:__main__:Computing evaluation metrics.
INFO:__main__:instance_auroc: 1.000
INFO:__main__:full_pixel_auroc: 0.985
INFO:__main__:anomaly_pixel_auroc: 0.980
INFO:__main__:

-----

INFO:patchcore.utils:instance_auroc: 1.000
INFO:patchcore.utils:full_pixel_auroc: 0.985
INFO:patchcore.utils:anomaly_pixel_auroc: 0.980

在这里插入图片描述

常见问题

ModuleNotFoundError: No module named ‘patchcore’

Traceback (most recent call last):
  File "run_patchcore.py", line 10, in <module>
    import patchcore.backbones
ModuleNotFoundError: No module named 'patchcore'

解决方法

在运行命令前，先设置环境变量：

# linux
export PYTHONPATH=src

# windows
set PYTHONPATH=src

OMP: Error

OMP: Error #15: Initializing libomp140.x86_64.dll, but found libiomp5md.dll already initialized.
OMP: Hint This means that multiple copies of the OpenMP runtime have been linked into the program. That is dangerous, since it can degrade performance or cause incorrect results. The best thing to do is to ensure that only a single OpenMP runtime is linked into the process, e.g. by avoiding static linking of the OpenMP runtime in any library. As an unsafe, unsupported, undocumented workaround you can set the environment variable KMP_DUPLICATE_LIB_OK=TRUE to allow the program to continue to execute, but that may cause crashes or silently produce incorrect results. For more information, please see http://openmp.llvm.org/

解决方法

在运行命令前，先设置环境变量：

# linux
export KMP_DUPLICATE_LIB_OK=True

# windows
set KMP_DUPLICATE_LIB_OK=True

AttributeError: ‘MVTecDataset’ object has no attribute ‘transform_std’

  File "bin/run_patchcore.py", line 439, in <module>
    main()
  File "D:\Users\Main\anaconda3\envs\pc\lib\site-packages\click\core.py", line 1157, in __call__
    return self.main(*args, **kwargs)
  File "D:\Users\Main\anaconda3\envs\pc\lib\site-packages\click\core.py", line 1078, in main
    rv = self.invoke(ctx)
  File "D:\Users\Main\anaconda3\envs\pc\lib\site-packages\click\core.py", line 1720, in invoke
    return _process_result(rv)
  File "D:\Users\Main\anaconda3\envs\pc\lib\site-packages\click\core.py", line 1657, in _process_result
    value = ctx.invoke(self._result_callback, value, **ctx.params)
  File "D:\Users\Main\anaconda3\envs\pc\lib\site-packages\click\core.py", line 783, in invoke
    return __callback(*args, **kwargs)
  File "bin/run_patchcore.py", line 170, in run
    patchcore.utils.plot_segmentation_images(
  File "D:\tfx\mytest\patchcore-inspection-main\src\patchcore\utils.py", line 51, in plot_segmentation_images
    image = image_transform(image)
  File "bin/run_patchcore.py", line 153, in image_transform
    dataloaders["testing"].dataset.transform_std
AttributeError: 'MVTecDataset' object has no attribute 'transform_std'

解决方法

因为主干是基于imagenet数据集训练的，所以均值和方差都是imagenet，因此可以在patchcore/datasets/mvtec.py中，在类class MVTecDataset 的def init 的函数中加入下面两行
参考issues80：
in patchcore/datasets/mvtec.py in class MVTecDataset in def init add this rows:

self.transform_std = IMAGENET_STD
self.transform_mean = IMAGENET_MEAN

在这里插入图片描述

保存结果图时总是会将原图覆盖掉

src/patchcore/utils.py代码存在bug，需要进行了修改，如下图所示。

# 解决：保存结果图时总是会将原图覆盖掉
folder_name = os.path.basename(os.path.dirname(image_path))
save_path = os.path.join(savefolder, folder_name)
if not os.path.exists(save_path):
    os.makedirs(save_path)
savename = os.path.join(save_path, os.path.basename(image_path))

在这里插入图片描述

参考文献

[1] PatchCore 源代码地址：https://github.com/amazon-science/patchcore-inspection.git
[2] PatchCore 论文地址：https://arxiv.org/abs/2106.08265
[3] https://link.springer.com/content/pdf/10.1007/s11263-020-01400-4.pdf
[4] https://www.mvtec.com/company/research/datasets/mvtec-ad/
[5] https://blog.csdn.net/xiao3_tai/article/details/136504813
[6] https://blog.csdn.net/m0_63828250/article/details/137158892
[7] https://blog.csdn.net/WD_SS/article/details/139673512

由于本人水平有限，难免出现错漏，敬请批评改正。
更多精彩内容，可点击进入Python日常小操作专栏、OpenCV-Python小应用专栏、YOLO系列专栏、自然语言处理专栏或我的个人主页查看
基于DETR的人脸伪装检测
YOLOv7训练自己的数据集（口罩检测）
YOLOv8训练自己的数据集（足球检测）
YOLOv10训练自己的数据集（交通标志检测）
YOLOv5：TensorRT加速YOLOv5模型推理
YOLOv5：IoU、GIoU、DIoU、CIoU、EIoU
玩转Jetson Nano（五）：TensorRT加速YOLOv5目标检测
YOLOv5：添加SE、CBAM、CoordAtt、ECA注意力机制
YOLOv5：yolov5s.yaml配置文件解读、增加小目标检测层
Python将COCO格式实例分割数据集转换为YOLO格式实例分割数据集
YOLOv5：使用7.0版本训练自己的实例分割模型（车辆、行人、路标、车道线等实例分割）
使用Kaggle GPU资源免费体验Stable Diffusion开源项目