1.configs

还是以yolox为例，配置有一项evaluation。用于配置评估是用什么评价指标评估。

https://github.com/open-mmlab/mmdetection/blob/master/configs/yolox/yolox_s_8x8_300e_coco.py#L151

max_epochs = 300
num_last_epochs = 15
interval = 10

evaluation = dict(
    save_best='auto',
    # The evaluation interval is 'interval' when running epoch is
    # less than ‘max_epochs - num_last_epochs’.
    # The evaluation interval is 1 when running epoch is greater than
    # or equal to ‘max_epochs - num_last_epochs’.
    interval=interval,
    dynamic_intervals=[(max_epochs - num_last_epochs, 1)],
    metric='bbox')

其中这几个参数：

interval：指每多少epoch进行一次评测；

dynamic_intervals：指当运行epoch大于或等于“max_epochs-num_last_epochs”时，评估间隔为1。

metric：表示使用什么作为评价指标。

2.具体实现

主体函数，一般是写在自定义数据集的解析dataset类中的。有的会实现在基类中，还是以yolox这个配置文件中对应的数据集为例CocoDataset：

https://github.com/open-mmlab/mmdetection/blob/master/mmdet/datasets/coco.py#L592

    def evaluate(self,
                 results,
                 metric='bbox',
                 logger=None,
                 jsonfile_prefix=None,
                 classwise=False,
                 proposal_nums=(100, 300, 1000),
                 iou_thrs=None,
                 metric_items=None):
        """Evaluation in COCO protocol.
        Args:
            results (list[list | tuple]): Testing results of the dataset.
            metric (str | list[str]): Metrics to be evaluated. Options are
                'bbox', 'segm', 'proposal', 'proposal_fast'.
            logger (logging.Logger | str | None): Logger used for printing
                related information during evaluation. Default: None.
            jsonfile_prefix (str | None): The prefix of json files. It includes
                the file path and the prefix of filename, e.g., "a/b/prefix".
                If not specified, a temp file will be created. Default: None.
            classwise (bool): Whether to evaluating the AP for each class.
            proposal_nums (Sequence[int]): Proposal number used for evaluating
                recalls, such as recall@100, recall@1000.
                Default: (100, 300, 1000).
            iou_thrs (Sequence[float], optional): IoU threshold used for
                evaluating recalls/mAPs. If set to a list, the average of all
                IoUs will also be computed. If not specified, [0.50, 0.55,
                0.60, 0.65, 0.70, 0.75, 0.80, 0.85, 0.90, 0.95] will be used.
                Default: None.
            metric_items (list[str] | str, optional): Metric items that will
                be returned. If not specified, ``['AR@100', 'AR@300',
                'AR@1000', 'AR_s@1000', 'AR_m@1000', 'AR_l@1000' ]`` will be
                used when ``metric=='proposal'``, ``['mAP', 'mAP_50', 'mAP_75',
                'mAP_s', 'mAP_m', 'mAP_l']`` will be used when
                ``metric=='bbox' or metric=='segm'``.
        Returns:
            dict[str, float]: COCO style evaluation metric.
        """

        metrics = metric if isinstance(metric, list) else [metric]
        allowed_metrics = ['bbox', 'segm', 'proposal', 'proposal_fast']
        for metric in metrics:
            if metric not in allowed_metrics:
                raise KeyError(f'metric {metric} is not supported')

        coco_gt = self.coco
        self.cat_ids = coco_gt.get_cat_ids(cat_names=self.CLASSES)

        result_files, tmp_dir = self.format_results(results, jsonfile_prefix)
        eval_results = self.evaluate_det_segm(results, result_files, coco_gt,
                                              metrics, logger, classwise,
                                              proposal_nums, iou_thrs,
                                              metric_items)

        if tmp_dir is not None:
            tmp_dir.cleanup()
        return eval_results

发现是调用 self.evaluate_det_segm这个函数进行评测，发现其调用的是pycocotools进行测评的。

其实除此之外，我们还可以在这里写相应的评测指标：

3.调用

3.1 train.py

主要是在训练时，在评估间隔处进行评测，知道模型好坏。然后可以存性能指标较好的模型，用于后续的应用，也方便进行优化。

训练过程怎么调用evaluation的呢？

（1）训练先调用tools/train.py

https://github.com/open-mmlab/mmdetection/blob/master/tools/train.py#L233

train_detector(
        model,
        datasets,
        cfg,
        distributed=distributed,
        validate=(not args.no_validate),
        timestamp=timestamp,
        meta=meta)

（2）该函数会调用mmdet/apis/train.py

https://github.com/open-mmlab/mmdetection/blob/master/mmdet/apis/train.py#L246

from mmcv.runner import (DistSamplerSeedHook, EpochBasedRunner,
                         Fp16OptimizerHook, OptimizerHook, build_runner,
                         get_dist_info)

runner = build_runner(
        cfg.runner,
        default_args=dict(
            model=model,
            optimizer=optimizer,
            work_dir=cfg.work_dir,
            logger=logger,
            meta=meta))

runner.run(data_loaders, cfg.workflow)

（3）看配置文件

https://github.com/open-mmlab/mmdetection/blob/master/configs/_base_/schedules/schedule_1x.py

runner = dict(type='EpochBasedRunner', max_epochs=12)

使用的是EpochBasedRunner这个Runner。

（4）通过builder_runner构建

mmcv/builder.py at master · open-mmlab/mmcv · GitHub

def build_runner(cfg: dict, default_args: Optional[dict] = None):
    runner_cfg = copy.deepcopy(cfg)
    constructor_type = runner_cfg.pop('constructor',
                                      'DefaultRunnerConstructor')
    runner_constructor = build_runner_constructor(
        dict(
            type=constructor_type,
            runner_cfg=runner_cfg,
            default_args=default_args))
    runner = runner_constructor()
    return runner

（5）调用

mmcv/epoch_based_runner.py at master · open-mmlab/mmcv · GitHub

-----待添加