Halcon深度学习分类模型

1.Halcon20之后深度学习支持CPU训练模型，没有money买显卡的小伙伴有福了。但是缺点也很明显，就是训练速度超级慢，推理效果也没有GPU好，不过学习用足够。
2.分类模型是Halcon深度学习最简单的模型，可以用在物品分类，缺陷检测等项目。
3.图像预处理和训练代码
*分类网络
dev_update_off ()
dev_close_window ()
WindowWidth := 800
WindowHeight := 600
dev_open_window_fit_size (0, 0, WindowWidth, WindowHeight, -1, -1, WindowHandle)
set_display_font (WindowHandle, 16, ‘mono’, ‘true’, ‘false’)
*训练原图路径
RawImageBaseFolder :=‘D:/训练图/’+ [‘U’,‘SR’,‘MR’,‘BR’,‘C’,‘D’,‘NG’]
*预处理数据存储路径
ExampleDataDir := ‘D:/classify_pill_defects_data’

Dataset directory basename for any outputs written by preprocess_dl_dataset.
DataDirectoryBaseName := ExampleDataDir + ‘/dldataset_pill’

** Set parameters ***

LabelSource for reading in the dataset.
LabelSource := ‘last_folder’
Percentages for splitting the dataset.
TrainingPercent := 70
ValidationPercent := 15
Image dimensions the images are rescaled to during preprocessing.
ImageWidth := 300
ImageHeight := 300
ImageNumChannels := 3
Further parameters for image preprocessing.
NormalizationType := ‘none’
DomainHandling := ‘full_domain’
In order to get a reproducible split we set a random seed.
This means that re-running the script results in the same split of DLDataset.
SeedRand := 42

** Read the labeled data and split it into train, validation and test ***

Set the random seed.
set_system (‘seed_rand’, SeedRand)
Read the dataset with the procedure read_dl_dataset_classification.
Alternatively, you can read a DLDataset dictionary
as created by e.g., the MVTec Deep Learning Tool using read_dict().
read_dl_dataset_classification (RawImageBaseFolder, LabelSource, DLDataset)
Generate the split.
split_dl_dataset (DLDataset, TrainingPercent, ValidationPercent, [])

** Preprocess the dataset ***

Create the output directory if it does not exist yet.
file_exists (ExampleDataDir, FileExists)
if (not FileExists)
make_dir (ExampleDataDir)
endif
Create preprocess parameters.
create_dl_preprocess_param (‘classification’, ImageWidth, ImageHeight, ImageNumChannels, -127, 128, NormalizationType, DomainHandling, [], [], [], [], DLPreprocessParam)
Dataset directory for any outputs written by preprocess_dl_dataset.
DataDirectory := DataDirectoryBaseName + ‘_’ + ImageWidth + ‘x’ + ImageHeight
Preprocess the dataset. This might take a few seconds.
create_dict (GenParam)
set_dict_tuple (GenParam, ‘overwrite_files’, true)
preprocess_dl_dataset (DLDataset, DataDirectory, DLPreprocessParam, GenParam, DLDatasetFileName)
Store preprocess params separately in order to use it e.g. during inference.
PreprocessParamFileBaseName := DataDirectory + ‘/dl_preprocess_param.hdict’
write_dict (DLPreprocessParam, PreprocessParamFileBaseName, [], [])

** Preview the preprocessed dataset ***

Before moving on to training, it is recommended to check the preprocessed dataset.
Display the DLSamples for 10 randomly selected train images.
get_dict_tuple (DLDataset, ‘samples’, DatasetSamples)
find_dl_samples (DatasetSamples, ‘split’, ‘train’, ‘match’, SampleIndices)
tuple_shuffle (SampleIndices, ShuffledIndices)
read_dl_samples (DLDataset, ShuffledIndices[0:9], DLSampleBatchDisplay)

create_dict (WindowHandleDict)
for Index := 0 to |DLSampleBatchDisplay| - 1 by 1
* Loop over samples in DLSampleBatchDisplay.
dev_display_dl_data (DLSampleBatchDisplay[Index], [], DLDataset, ‘classification_ground_truth’, [], WindowHandleDict)
Text := ‘Press Run (F5) to continue’
dev_disp_text (Text, ‘window’, ‘bottom’, ‘right’, ‘black’, [], [])
stop ()
endfor
*

Close windows that have been used for visualization.
dev_close_window_dict (WindowHandleDict)

*检测电脑是否有GPU，如果无GPU则使用CPU训练
query_available_dl_devices ([‘runtime’,‘runtime’], [‘gpu’,‘cpu’], DLDeviceHandles)
if (|DLDeviceHandles| == 0)
throw (‘No supported device found to continue this example.’)
endif

Due to the filter used in query_available_dl_devices, the first device is a GPU, if available.
DLDevice := DLDeviceHandles[0]
get_dl_device_param (DLDevice, ‘type’, DLDeviceType)
if (DLDeviceType == ‘cpu’)
- The number of used threads may have an impact
- on the training duration.
  NumThreadsTraining := 4
  set_system (‘thread_num’, NumThreadsTraining)
  endif

** Set input and output paths ***

File path of the initialized model.
ModelFileName := ‘pretrained_dl_classifier_compact.hdl’
File path of the preprocessed DLDataset.
Note: Adapt DataDirectory after preprocessing with another image size.
DataDirectory := ExampleDataDir + ‘/dldataset_pill_300x300’
DLDatasetFileName := DataDirectory + ‘/dl_dataset.hdict’
DLPreprocessParamFileName := DataDirectory + ‘/dl_preprocess_param.hdict’
Output path of the best evaluated model.
BestModelBaseName := ExampleDataDir + ‘/best_dl_model_classification’
Output path for the final trained model.
FinalModelBaseName := ExampleDataDir + ‘/final_dl_model_classification’

** Set basic parameters ***

The following parameters need to be adapted frequently.
Model parameters.
Batch size. In case this example is run on a GPU,
you can set BatchSize to ‘maximum’ and it will be
determined automatically.
BatchSize := 64
Initial learning rate.
InitialLearningRate := 0.001
Momentum should be high if batch size is small.
Momentum := 0.9
Parameters used by train_dl_model.
Number of epochs to train the model.
NumEpochs := 16
Evaluation interval (in epochs) to calculate evaluation measures on the validation split.
EvaluationIntervalEpochs := 1
Change the learning rate in the following epochs, e.g. [4, 8, 12].
Set it to [] if the learning rate should not be changed.
ChangeLearningRateEpochs := [4,8,12]
Change the learning rate to the following values, e.g. InitialLearningRate * [0.1, 0.01, 0.001].
The tuple has to be of the same length as ChangeLearningRateEpochs.
ChangeLearningRateValues := InitialLearningRate * [0.1,0.01,0.001]

** Set advanced parameters ***

The following parameters might need to be changed in rare cases.
Model parameter.
Set the weight prior.
WeightPrior := 0.0005
Parameters used by train_dl_model.
Control whether training progress is displayed (true/false).
EnableDisplay := true
Set a random seed for training.
RandomSeed := 42
set_system (‘seed_rand’, RandomSeed)
In order to obtain nearly deterministic training results on the same GPU
(system, driver, cuda-version) you could specify “cudnn_deterministic” as
“true”. Note, that this could slow down training a bit.
set_system (‘cudnn_deterministic’, ‘true’)
Set generic parameters of create_dl_train_param.
Please see the documentation of create_dl_train_param for an overview on all available parameters.
GenParamName := []
GenParamValue := []
Augmentation parameters.
If samples should be augmented during training, create the dict required by augment_dl_samples.
Here, we set the augmentation percentage and method.
create_dict (AugmentationParam)
Percentage of samples to be augmented.
set_dict_tuple (AugmentationParam, ‘augmentation_percentage’, 50)
Mirror images along row and column.
set_dict_tuple (AugmentationParam, ‘mirror’, ‘rc’)
GenParamName := [GenParamName,‘augment’]
GenParamValue := [GenParamValue,AugmentationParam]
Change strategies.
It is possible to change model parameters during training.
Here, we change the learning rate if specified above.
if (|ChangeLearningRateEpochs| > 0)
create_dict (ChangeStrategy)
- Specify the model parameter to be changed, here the learning rate.
  set_dict_tuple (ChangeStrategy, ‘model_param’, ‘learning_rate’)
- Start the parameter value at ‘initial_value’.
  set_dict_tuple (ChangeStrategy, ‘initial_value’, InitialLearningRate)
- Reduce the learning rate in the following epochs.
  set_dict_tuple (ChangeStrategy, ‘epochs’, ChangeLearningRateEpochs)
- Reduce the learning rate to the following values.
  set_dict_tuple (ChangeStrategy, ‘values’, ChangeLearningRateValues)
- Collect all change strategies as input.
  GenParamName := [GenParamName,‘change’]
  GenParamValue := [GenParamValue,ChangeStrategy]
  endif
Serialization strategies.
There are several options for saving intermediate models to disk (see create_dl_train_param).
Here, we save the best and the final model to the paths set above.
create_dict (SerializationStrategy)
set_dict_tuple (SerializationStrategy, ‘type’, ‘best’)
set_dict_tuple (SerializationStrategy, ‘basename’, BestModelBaseName)
GenParamName := [GenParamName,‘serialize’]
GenParamValue := [GenParamValue,SerializationStrategy]
create_dict (SerializationStrategy)
set_dict_tuple (SerializationStrategy, ‘type’, ‘final’)
set_dict_tuple (SerializationStrategy, ‘basename’, FinalModelBaseName)
GenParamName := [GenParamName,‘serialize’]
GenParamValue := [GenParamValue,SerializationStrategy]
Display parameters.
In this example, 20% of the training split are selected to display the
evaluation measure for the reduced training split during the training. A lower percentage
helps to speed up the evaluation/training. If the evaluation measure for the training split
shall not be displayed, set this value to 0 (default).
SelectedPercentageTrainSamples := 20
Set the x-axis argument of the training plots.
XAxisLabel := ‘epochs’
create_dict (DisplayParam)
set_dict_tuple (DisplayParam, ‘selected_percentage_train_samples’, SelectedPercentageTrainSamples)
set_dict_tuple (DisplayParam, ‘x_axis_label’, XAxisLabel)
GenParamName := [GenParamName,‘display’]
GenParamValue := [GenParamValue,DisplayParam]

** Read initial model and dataset ***

Check if all necessary files exist.
check_data_availability (ExampleDataDir, DLDatasetFileName, DLPreprocessParamFileName)
Read in the model that was initialized during preprocessing.
read_dl_model (ModelFileName, DLModelHandle)
Read in the preprocessed DLDataset file.
read_dict (DLDatasetFileName, [], [], DLDataset)

** Set model parameters ***

Set model hyper-parameters as specified in the settings above.
set_dl_model_param (DLModelHandle, ‘learning_rate’, InitialLearningRate)
set_dl_model_param (DLModelHandle, ‘momentum’, Momentum)
Set the class names for the model.
get_dict_tuple (DLDataset, ‘class_names’, ClassNames)
set_dl_model_param (DLModelHandle, ‘class_names’, ClassNames)
Get image dimensions from preprocess parameters and set them for the model.
read_dict (DLPreprocessParamFileName, [], [], DLPreprocessParam)
get_dict_tuple (DLPreprocessParam, ‘image_width’, ImageWidth)
get_dict_tuple (DLPreprocessParam, ‘image_height’, ImageHeight)
get_dict_tuple (DLPreprocessParam, ‘image_num_channels’, ImageNumChannels)
set_dl_model_param (DLModelHandle, ‘image_dimensions’, [ImageWidth,ImageHeight,ImageNumChannels])
if (BatchSize == ‘maximum’ and DLDeviceType == ‘gpu’)
set_dl_model_param_max_gpu_batch_size (DLModelHandle, 100)
else
set_dl_model_param (DLModelHandle, ‘batch_size’, BatchSize)
endif
When the batch size is determined, set the device.
set_dl_model_param (DLModelHandle, ‘device’, DLDevice)
if (|WeightPrior| > 0)
set_dl_model_param (DLModelHandle, ‘weight_prior’, WeightPrior)
endif
Set class weights to counteract unbalanced training data. In this example
we choose the default values, since the classes are evenly distributed in the dataset.
tuple_gen_const (|ClassNames|, 1.0, ClassWeights)
set_dl_model_param (DLModelHandle, ‘class_weights’, ClassWeights)

** Train the model ***

Create training parameters.
create_dl_train_param (DLModelHandle, NumEpochs, EvaluationIntervalEpochs, EnableDisplay, RandomSeed, GenParamName, GenParamValue, TrainParam)
Start the training by calling the training operator
train_dl_model_batch () within the following procedure.
train_dl_model (DLDataset, DLModelHandle, TrainParam, 0, TrainResults, TrainInfos, EvaluationInfos)
Stop after the training has finished, before closing the windows.
dev_disp_text (‘Press Run (F5) to continue’, ‘window’, ‘bottom’, ‘right’, ‘black’, [], [])
stop ()
Close training windows.
dev_close_window ()
4.推理代码
dev_update_off ()
dev_close_window ()
WindowWidth := 800
WindowHeight := 600
dev_open_window_fit_size (0, 0, WindowWidth, WindowHeight, -1, -1, WindowHandle)
set_display_font (WindowHandle, 16, ‘mono’, ‘true’, ‘false’)
** INFERENCE **

*检测电脑是否有GPU，如果无GPU则使用CPU推理
query_available_dl_devices ([‘runtime’,‘runtime’], [‘gpu’,‘cpu’], DLDeviceHandles)
if (|DLDeviceHandles| == 0)
throw (‘No supported device found to continue this example.’)
endif

Due to the filter used in query_available_dl_devices, the first device is a GPU, if available.
DLDevice := DLDeviceHandles[0]

*总路径
ExampleDataDir := ‘D:/classify_pill_defects_data’

Dataset directory basename for any outputs written by preprocess_dl_dataset.
DataDirectoryBaseName := ExampleDataDir + ‘/dldataset_pill’
File name of the dict containing parameters used for preprocessing.
Note: Adapt DataDirectory after preprocessing with another image size.
DataDirectory := ExampleDataDir + ‘/dldataset_pill_300x300’
PreprocessParamFileName := DataDirectory + ‘/dl_preprocess_param.hdict’
File name of the finetuned object detection model.
RetrainedModelFileName := ExampleDataDir + ‘/best_dl_model_classification.hdl’
Batch Size used during inference.
BatchSizeInference := 1

** Inference ***

Check if all necessary files exist.
check_data_availability (ExampleDataDir, PreprocessParamFileName, RetrainedModelFileName, false)
Read in the retrained model.
read_dl_model (RetrainedModelFileName, DLModelHandle)
Set the batch size.
set_dl_model_param (DLModelHandle, ‘batch_size’, BatchSizeInference)
Initialize the model for inference.
set_dl_model_param (DLModelHandle, ‘device’, DLDevice)
Get the class names and IDs from the model.
get_dl_model_param (DLModelHandle, ‘class_names’, ClassNames)
get_dl_model_param (DLModelHandle, ‘class_ids’, ClassIDs)
Get the parameters used for preprocessing.
read_dict (PreprocessParamFileName, [], [], DLPreprocessParam)
Create window dictionary for displaying results.
create_dict (WindowHandleDict)
Create dictionary with dataset parameters necessary for displaying.
create_dict (DLDataInfo)
set_dict_tuple (DLDataInfo, ‘class_names’, ClassNames)
set_dict_tuple (DLDataInfo, ‘class_ids’, ClassIDs)
Set generic parameters for visualization.
create_dict (GenParam)
set_dict_tuple (GenParam, ‘scale_windows’, 1.1)

list_files (‘E:/NG’, [‘files’,‘follow_links’], ImageFiles)
tuple_regexp_select (ImageFiles, [‘\.(tif|tiff|gif|bmp|jpg|jpeg|jp2|png|pcx|pgm|ppm|pbm|xwd|ima|hobj)$’,‘ignore_case’], ImageFiles)
for Index := 0 to |ImageFiles| - 1 by 1
read_image (ImageBatch, ImageFiles[Index])
gen_dl_samples_from_images (ImageBatch, DLSampleBatch)
preprocess_dl_samples (DLSampleBatch, DLPreprocessParam)
apply_dl_model (DLModelHandle, DLSampleBatch, [], DLResultBatch)
DLSample := DLSampleBatch[0]
DLResult := DLResultBatch[0]
*获取识别结果参数：分类的结果，批处理中图像的索引，通用参数的名称，通用参数的值
get_dict_tuple (DLResult, ‘classification_class_ids’, ClassificationClassID)
get_dict_tuple (DLResult, ‘classification_class_names’, ClassificationClassName)
get_dict_tuple (DLResult, ‘classification_confidences’, ClassificationClassConfidence)
dev_display (ImageBatch)
Text := ‘预测类为: ’ + ClassificationClassName[0] + ’ 置信度：’+ClassificationClassConfidence[0]
dev_disp_text (Text, ‘window’, ‘top’, ‘left’, ‘red’, ‘box’, ‘false’)
stop ()
endfor
dev_close_window_dict (WindowHandleDict)