1.RepLoss 设计思想

物体遮挡问题可以分为类内遮挡和类间遮挡两种情况。类间遮挡产生于扎堆的同类物体，也被称为密集遮挡（crowd occlusion）。
原文连接：https://arxiv.org/abs/1711.07752

密集遮挡的主要影响表现在显著增加了行人定位的难度。比如，当目标行人 T 被行人 B 遮挡之时，由于两者外观特征相似，检测器很可能无法进行定位。从而本应该框定 T 的边界框转而框定 B，导致定位不准确。更糟糕的是，由于非极大值抑制（non-maximum suppression/NMS）需要进一步处理主要的检测结果，从 T 移走的边界框可能会被 B 的预测框抑制，进而造成 T 漏检。即，人群遮挡使得检测器对 NMS 阈值很敏感：较高的阈值会带来更多的误检（false positives），较低的阈值则造成更多的漏检（missed detection）。这会让大多数实例分割框架失效，因为它们也需要精确的检测结果。因此，如何精确地定位人群之中的每个行人是检测器最为关键的问题之一。

2.RepLoss 主要工作

Reulsion loss完整公式如下：

其中 L_Attr 是吸引项，需要预测框靠近其指定目标；L_RepGT 和 L_RepBox 是排斥项，分别需要预测框远离周遭其他的 groundtruth 物体和其他指定目标不同的预测框。系数 α 和 β 充当权重以平衡辅助损失。
第一部分为预测框与真实目标框所产生的损失值（attraction term）
第二部分为预测框与相邻真实目标框所产生的损失值（repulsion term(RepGT)）
第三部分为预测框与相邻不是预测同一真实目标的预测框所产生的损失值（repulsion Box(RepBox))
通过两个相关系数alpha和beta来平衡两部分repulsion损失值。

2.1 吸引项

本文沿用 Smooth_L1 构造吸引项。给定一个 proposal P ∈ P_+，把具有极大值 IoU 的 groundtruth box 作为其指定目标：G^P_Attr = arg max_G∈G IoU(G,P)。B^P 是回归自 proposal P 的预测框。由此吸引损失可计算为：

2.2 排斥项（RepGT）

RepGT 损失旨在使 proposal 受到相邻的非目标 groundtruth 物体的排斥。给定一个 proposal P ∈ P_+，它的排斥 groundtruth 物体被定义为除了其指定目标之外带有最大 IoU 区域的 groundtruth 物体。受 IoU 损失的启发，RepGT 损失被计算以惩罚 B^P 和 G^P_Rep 之间的重叠（由 IoG 定义）。IoG(B, G) ∈ [0, 1]，从而 RepGT 损失可写为：

其中 Smooth_ln 是一个在区间 (0, 1) 连续可微分的平滑 ln 函数，σ ∈ [0, 1) 是调节 RepLoss 对异常值的敏感度的平滑参数。由此可见，proposal 越倾向于与非目标 groundtruth 物体重叠，RepGT 损失对边界框回归器的惩罚就越大，从而有效防止边界框移向相邻的非目标物体。

2.3 排斥项（RepBox）

NMS 是绝大多数检测框架中不可或缺的后处理步骤，为降低检测器对 NMS 的敏感度，作者接着提出 RepBox 损失，意在排斥来自不同指定目标的 proposal。RepBox 损失可计算为：

从上式可以看到，为最小化 RepBox 损失，指定目标不同的两个预测框之间的 IoU 区域需要较小。这意味着 RepBox 损失可以降低 NMS 之后不同回归目标的边界框合并为一的概率，使得检测器在密集场景中更鲁棒。

2.4 总结

Repulsion损失函数由三个部分构成，
第一部分主要作用：预测目标框吸引IOU最大的真实目标框，使得预测更加准确；
第二部分主要作用：远离除IOU最大值之外的最大的IOU目标框
第三部分主要作用：预测框之间互相远离;

3. yolov5+Repulsion

在yolov5中使用Repulsion损失函数解决密集且遮挡的物体，亲测可用

3.1 rep_loss.py

在utils下面创建rep_loss.py文件，rep_loss.py内容如下：

import math
import torch
import torch.nn as nn
import numpy as np


class RepLoss(nn.Module):
    def __init__(self, alpha=1, beta=1, sigma=0.5):
        super(RepLoss, self).__init__()
        self.alpha = alpha
        self.beta = beta
        self.sigma = sigma
        self.eps = 1e-7

    def forward(self, gt_boxes, pre_boxes):
        box_iou = self.bbox_iou(gt_boxes, pre_boxes)
        proposal_overlaps = self.bbox_iou(pre_boxes, pre_boxes, xywh=False)
        max_attr, max_attr_index = box_iou.max(dim=0)
        GT_attr = gt_boxes[max_attr_index]
        box_iou[max_attr_index, range(pre_boxes.shape[0])] = 0
        # 判断是否有多个预测目标，若只有一个预测目标和真实目标匹配，则第二项不开启
        if not box_iou.sum == 0:
            max_rep, max_rep_index = box_iou.max(dim=0)
            GT_rep = gt_boxes[max_rep_index]
            rep_loss = self.Attr(pre_boxes, GT_attr, max_attr) + \
                        self.alpha * self.RepGT(pre_boxes, GT_rep, max_attr) + \
                        self.beta * self.RepBox(proposal_overlaps)
        else:
            rep_loss = self.Attr(GT_attr, pre_boxes, max_attr)  + self.beta*self.RepBox(proposal_overlaps)
        return rep_loss

    def Attr(self, gt_boxes, pre_boxes, max_iou):
        Attr_loss = 0
        for index, (gt_box, pre_box) in enumerate(zip(gt_boxes, pre_boxes)):
            # if max_iou[index] > self.sigma:
            Attr_loss += self.SmoothL1(gt_box, pre_box)
        Attr_loss = Attr_loss.sum() / len(gt_boxes)
        return Attr_loss

    def RepGT(self, gt_boxes, pre_boxes, max_iou):
        RepGT_loss = 0
        count = 0
        for index, (gt_box, pre_box) in enumerate(zip(gt_boxes, pre_boxes)):
            # if max_iou[index] > self.sigma:
            count += 1
            IOG = self.RepGT_iog(gt_box, pre_box)
            if IOG > self.sigma:
                RepGT_loss += ((IOG - self.sigma) / ((1 - self.sigma) - math.log(1 - self.sigma))).sum()
            else:
                RepGT_loss += -(1 - IOG).clamp(min=self.eps).log().sum()
        RepGT_loss = RepGT_loss.sum() / count
        return RepGT_loss

    def RepBox(self, overlaps):
        RepBox_loss = 0
        overlap_loss =0
        count = 0
        #result = overlaps.triu(1)
        for i in range(0, overlaps.shape[0]):
            for j in range(1 + i, overlaps.shape[0]):
                count += 1
                if overlaps[i][j] > self.sigma:
                    RepBox_loss += ((overlaps[i][j] - self.sigma) / ((1 - self.sigma) - math.log(1 - self.sigma))).sum()
                else:
                    RepBox_loss += -(1 - overlaps[i][j]).clamp(min=self.eps).log().sum()
        RepBox_loss = RepBox_loss / count
        return RepBox_loss

    def SmoothL1(self, pred, target, beta=1.0):
        diff = torch.abs(pred - target)
        cond = torch.lt(diff, beta)
        loss = torch.where(cond, 0.5 * diff ** 2 / beta, diff - 0.5 * beta)
        return loss

    def RepGT_iog(self, box1, box2, List=True):
        if List:  # transform from xywh to xyxy
            b1_x1, b1_y1, b1_x2, b1_y2 = box1[0], box1[1], box1[2], box1[3]
            b2_x1, b2_y1, b2_x2, b2_y2 = box2[0], box2[1], box2[2], box2[3]
        else:  # x1, y1, x2, y2 = box1
            b1_x1, b1_y1, b1_x2, b1_y2 = box1[:, 0], box1[:, 1], box1[:, 2], box1[:, 3]
            b2_x1, b2_y1, b2_x2, b2_y2 = box2[:, 0], box2[:, 1], box2[:, 2], box2[:, 3]

        # Intersection area
        inter = (torch.min(b1_x2, b2_x2) - torch.max(b1_x1, b2_x1)).clamp(0) * \
                (torch.min(b1_y2, b2_y2) - torch.max(b1_y1, b2_y1)).clamp(0)
        # g_area Area
        g_area = torch.abs(b2_x2 - b2_x1) * torch.abs(b2_y2 - b2_y1)
        # IoU
        iog = inter / g_area
        return iog

    def bbox_iou(self, bboxes1, bboxes2, xywh=True, eps=1e-7):
        if xywh:
            # chunk(4, -1)表示在最后一个维度上切块
            (x1, y1, w1, h1), (x2, y2, w2, h2) = bboxes1.chunk(4, -1), bboxes2.chunk(4, -1)
            bboxes1[:, 0:1], bboxes1[:, 1:2], bboxes1[:, 2:3], bboxes1[:,
                                                               3:4] = x1 - w1 / 2, y1 - h1 / 2, x1 + w1 / 2, y1 + h1 / 2
            bboxes2[:, 0:1], bboxes2[:, 1:2], bboxes2[:, 2:3], bboxes2[:,
                                                               3:4] = x2 - w2 / 2, y2 - h2 / 2, x2 + w2 / 2, y2 + h2 / 2

        lt = torch.max(bboxes1[:, None, :2], bboxes2[:, :2])  # [rows, cols, 2]
        rb = torch.min(bboxes1[:, None, 2:], bboxes2[:, 2:])  # [rows, cols, 2]

        wh = (rb - lt + 1).clamp(min=0)  # [rows, cols, 2]
        overlap = wh[:, :, 0] * wh[:, :, 1]
        area1 = (bboxes1[:, 2] - bboxes1[:, 0] + 1) * (
                bboxes1[:, 3] - bboxes1[:, 1] + 1)

        area2 = (bboxes2[:, 2] - bboxes2[:, 0] + 1) * (
                bboxes2[:, 3] - bboxes2[:, 1] + 1)
        ious = overlap / (area1[:, None] + area2 - overlap).clamp(min=eps)  # 会产生大于1的值？？？

        return ious.clamp(min=eps, max=1)

3.2 loss.py

需要在loss.py中修改代码，调用rep_loss.py中的Repulsion损失函数，找到class ComputeLoss修改如下：

class ComputeLoss:
    sort_obj_iou = False

    # Compute losses
    def __init__(self, model, autobalance=False):
        device = next(model.parameters()).device  # get model device
        h = model.hyp  # hyperparameters

        # Define criteria
        BCEcls = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([h['cls_pw']], device=device))
        BCEobj = nn.BCEWithLogitsLoss(pos_weight=torch.tensor([h['obj_pw']], device=device))

        # Class label smoothing https://arxiv.org/pdf/1902.04103.pdf eqn 3
        self.cp, self.cn = smooth_BCE(eps=h.get('label_smoothing', 0.0))  # positive, negative BCE targets

        # Focal loss
        g = h['fl_gamma']  # focal loss gamma
        if g > 0:
            BCEcls, BCEobj = FocalLoss(BCEcls, g), FocalLoss(BCEobj, g)

        m = de_parallel(model).model[-1]  # Detect() module
        self.balance = {3: [4.0, 1.0, 0.4]}.get(m.nl, [4.0, 1.0, 0.25, 0.06, 0.02])  # P3-P7
        self.ssi = list(m.stride).index(16) if autobalance else 0  # stride 16 index
        self.BCEcls, self.BCEobj, self.gr, self.hyp, self.autobalance = BCEcls, BCEobj, 1.0, h, autobalance
        self.na = m.na  # number of anchors
        self.nc = m.nc  # number of classes
        self.nl = m.nl  # number of layers
        self.anchors = m.anchors
        self.device = device
        self.rep_loss = RepLoss(alpha=0.5, beta=0.5, sigma=0.5)

    def __call__(self, p, targets):  # predictions, targets
        lcls = torch.zeros(1, device=self.device)  # class loss
        lbox = torch.zeros(1, device=self.device)  # box loss
        lobj = torch.zeros(1, device=self.device)  # object loss
        tcls, tbox, indices, anchors = self.build_targets(p, targets)  # targets

        pre_boxes = []
        bs = p[0].shape[0]
        if self.hyp['Reploss']:
            lrep = torch.zeros(1, device=self.device)
            gt_boxes = targets

        # Losses
        for i, pi in enumerate(p):  # layer index, layer predictions
            b, a, gj, gi = indices[i]  # image, anchor, gridy, gridx
            tobj = torch.zeros(pi.shape[:4], dtype=pi.dtype, device=self.device)  # target obj

            n = b.shape[0]  # number of targets
            if n:
                # pxy, pwh, _, pcls = pi[b, a, gj, gi].tensor_split((2, 4, 5), dim=1)  # faster, requires torch 1.8.0
                # 将pi的内部信息划分为cx,cy,wh,_,cls
                pxy, pwh, _, pcls = pi[b, a, gj, gi].split((2, 2, 1, self.nc), 1)  # target-subset of predictions

                # Regression
                pxy = pxy.sigmoid() * 2 - 0.5
                pwh = (pwh.sigmoid() * 2) ** 2 * anchors[i]
                pbox = torch.cat((pxy, pwh), 1)  # predicted box
                iou = bbox_iou(pbox, tbox[i], EIoU=True).squeeze()  # iou(prediction, target)
                lbox += (1.0 - iou).mean()  # iou loss

                if self.hyp['Reploss']:
                    pre_box = torch.cat(
                        (b[:, None],
                         (pbox[:, 0] + gi)[:, None] / pi.shape[3], (pbox[:, 1] + gj)[:, None] / pi.shape[2],
                         (pbox[:, 2])[:, None] / pi.shape[3], (pbox[:, 3])[:, None] / pi.shape[2]), 1)
                    pre_boxes.append(pre_box)
                

                # Objectness
                iou = iou.detach().clamp(0).type(tobj.dtype)
                if self.sort_obj_iou:
                    j = iou.argsort()
                    b, a, gj, gi, iou = b[j], a[j], gj[j], gi[j], iou[j]
                if self.gr < 1:
                    iou = (1.0 - self.gr) + self.gr * iou
                tobj[b, a, gj, gi] = iou  # iou ratio

                # Classification
                if self.nc > 1:  # cls loss (only if multiple classes)
                    t = torch.full_like(pcls, self.cn, device=self.device)  # targets
                    t[range(n), tcls[i]] = self.cp
                    lcls += self.BCEcls(pcls, t)  # BCE

                # Append targets to text file
                # with open('targets.txt', 'a') as file:
                #     [file.write('%11.5g ' * 4 % tuple(x) + '\n') for x in torch.cat((txy[i], twh[i]), 1)]

            obji = self.BCEobj(pi[..., 4], tobj)
            lobj += obji * self.balance[i]  # obj loss
            if self.autobalance:
                self.balance[i] = self.balance[i] * 0.9999 + 0.0001 / obji.detach().item()

        if self.hyp['Reploss'] and len(pre_boxes) != 0:
            pre_boxes = torch.cat(pre_boxes)
            for i in range(bs):
                gt_index = gt_boxes[:, 0] == i
                gt_batch_boxes = gt_boxes[gt_index, 2:]
                pre_index = pre_boxes[:, 0] == i
                prd_batch_boxes = pre_boxes[pre_index, 1:]
                if len(prd_batch_boxes) != 0 and len(prd_batch_boxes) != 0:
                    lrep += self.rep_loss(gt_batch_boxes, prd_batch_boxes)
            lrep = lrep / bs
            lrep *= self.hyp['rep']
        if self.autobalance:
            self.balance = [x / self.balance[self.ssi] for x in self.balance]
        lbox *= self.hyp['box']
        lobj *= self.hyp['obj']
        lcls *= self.hyp['cls']
        bs = tobj.shape[0]  # batch size

        if self.hyp['Reploss']:
            return (lbox + lobj + lcls + lrep) * bs, torch.cat((lbox, lobj, lcls, lrep)).detach()
        else:
            return (lbox + lobj + lcls) * bs, torch.cat((lbox, lobj, lcls)).detach()

    def build_targets(self, p, targets):
        # Build targets for compute_loss(), input targets(image,class,x,y,w,h)
        na, nt = self.na, targets.shape[0]  # number of anchors, targets
        tcls, tbox, indices, anch = [], [], [], []
        gain = torch.ones(7, device=self.device)  # normalized to gridspace gain
        ai = torch.arange(na, device=self.device).float().view(na, 1).repeat(1, nt)  # same as .repeat_interleave(nt)
        targets = torch.cat((targets.repeat(na, 1, 1), ai[..., None]), 2)  # append anchor indices

        g = 0.5  # bias
        off = torch.tensor(
            [
                [0, 0],
                [1, 0],
                [0, 1],
                [-1, 0],
                [0, -1],  # j,k,l,m
                # [1, 1], [1, -1], [-1, 1], [-1, -1],  # jk,jm,lk,lm
            ],
            device=self.device).float() * g  # offsets

        for i in range(self.nl):
            anchors, shape = self.anchors[i], p[i].shape
            gain[2:6] = torch.tensor(shape)[[3, 2, 3, 2]]  # xyxy gain

            # Match targets to anchors
            t = targets * gain  # shape(3,n,7)
            if nt:
                # Matches
                r = t[..., 4:6] / anchors[:, None]  # wh ratio
                j = torch.max(r, 1 / r).max(2)[0] < self.hyp['anchor_t']  # compare
                # j = wh_iou(anchors, t[:, 4:6]) > model.hyp['iou_t']  # iou(3,n)=wh_iou(anchors(3,2), gwh(n,2))
                t = t[j]  # filter

                # Offsets
                gxy = t[:, 2:4]  # grid xy
                gxi = gain[[2, 3]] - gxy  # inverse
                j, k = ((gxy % 1 < g) & (gxy > 1)).T
                l, m = ((gxi % 1 < g) & (gxi > 1)).T
                j = torch.stack((torch.ones_like(j), j, k, l, m))
                t = t.repeat((5, 1, 1))[j]
                offsets = (torch.zeros_like(gxy)[None] + off[:, None])[j]
            else:
                t = targets[0]
                offsets = 0

            # Define
            bc, gxy, gwh, a = t.chunk(4, 1)  # (image, class), grid xy, grid wh, anchors
            a, (b, c) = a.long().view(-1), bc.long().T  # anchors, image, class
            gij = (gxy - offsets).long()
            gi, gj = gij.T  # grid indices

            # Append
            indices.append((b, a, gj.clamp_(0, shape[2] - 1), gi.clamp_(0, shape[3] - 1)))  # image, anchor, grid
            tbox.append(torch.cat((gxy - gij, gwh), 1))  # box
            anch.append(anchors[a])  # anchors
            tcls.append(c)  # class

        return tcls, tbox, indices, anch

3.3 hyp.scratch.yaml

在配置文件中引入超参：

Reploss: True# True开启Reploss，False关闭Reploss
rep: 2 # Reploss权重

4. 总结

上述代码经测试可以正常使用，本人在测试时只开启前两项吸引项+排斥项（RepGT），训练速度非常慢，需要一直计算IOU。
没有使用第三项排斥项（RepBox）原因：当物体非常密集时，所有预测框之间非常接近，若开启第三项则所有的预测框互相远离，会导致预测结果与真是目标框位置偏差，并且第三项计算量很大。