目标检测中的框合并策略nms及其改进

发表于 更新于
检测算法可能对同一目标产生多次检测的结果，这就需要使用某些算法对检测框去重。

检测算法可能对同一目标产生多次检测的结果，这就需要使用某些算法对检测框去重。常用的两种算法是NMS和Soft-NMS。

NMS

NMS（non maximum suppression，非极大值抑制）算法思想很简单，按照分类概率排序，概率最高的框作为候选框，其它所有与它的IOU高于一个阈值（这是人工指定的超参）的框其概率被置为0。然后在剩余的框里寻找概率第二大的框，其它所有与它的IOU高于一个阈值（这是人工指定的超参）的框其概率被置为0。依次类推。
最终所有的框相互之间的IOU都是小于超参阈值的，或者概率被置为0了。剩下的所有概率非0的框就是最终的检测框。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
import numpy as np
 
def nms(dets, thresh):
    x1 = dets[:, 0]                  # pred bbox top_x
    y1 = dets[:, 1]                  # pred bbox top_y
    x2 = dets[:, 2]                  # pred bbox bottom_x
    y2 = dets[:, 3]                  # pred bbox bottom_y
    scores = dets[:, 4]              # pred bbox cls score
 
    areas = (x2 - x1 + 1) * (y2 - y1 + 1)   # pred bbox areas
    order = scores.argsort()[::-1]          # 对pred bbox按置信度做降序排序
 
    keep = []
    while order.size > 0:
        i = order[0]      # top-1 score pred bbox
        keep.append(i)
        xx1 = np.maximum(x1[i], x1[order[1:]])
        yy1 = np.maximum(y1[i], y1[order[1:]])
        xx2 = np.minimum(x2[i], x2[order[1:]])
        yy2 = np.minimum(y2[i], y2[order[1:]])
 
        w = np.maximum(0.0, xx2 - xx1 + 1)
        h = np.maximum(0.0, yy2 - yy1 + 1)
        inter = w * h
        IoU = inter / (areas[i] + areas[order[1:]] - inter)  # IoU计算
 
        inds = np.where(IoU <= thresh)[0]     # 保留非冗余bbox
        order = order[inds + 1]
 
    return keep    # 最终NMS结果返回

if __name__ == '__main__':

    dets = np.array([[100,120,170,200,0.98],
                     [20,40,80,90,0.94],
                     [20,38,82,88,0.99],
                     [200,380,282,488,0.9],
                     [19,38,75,91, 0.98]])

    print(nms(dets, 0.5))

Soft-NMS

NMS算法是略显粗暴，因为NMS直接将删除所有IoU大于阈值的框。soft-NMS吸取了NMS的教训，在算法执行过程中不是简单的对IoU大于阈值的检测框删除，而是降低得分。算法流程同NMS相同，但是对原置信度得分使用函数运算，目标是降低置信度得分。
可以说NMS是Soft-NMS特殊形式，当得分重置函数采用二值化函数时，Soft-NMS和NMS是相同的。Soft-NMS算法是一种更加通用的非最大抑制算法。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
import numpy as np
 
def soft_nms(dets, method, thresh=0.001, Nt=0.1, sigma=0.5):
    x1 = dets[:, 0]
    y1 = dets[:, 1]
    x2 = dets[:, 2]
    y2 = dets[:, 3]
    scores = dets[:, 4]

    areas = (y2 - y1 + 1.) * (x2 - x1 + 1.) 
    orders = scores.argsort()[::-1]
    keep = []
    
    while orders.size > 0:
        i = orders[0]
        keep.append(i)
        for j in orders[1:]:
            xx1 = np.maximum(x1[i], x1[j])
            yy1 = np.maximum(y1[i], y1[j])
            xx2 = np.minimum(x2[i], x2[j])
            yy2 = np.minimum(y2[i], y2[j])

            w = np.maximum( xx2 - xx1 + 1., 0. )
            h = np.maximum( yy2 - yy1 + 1., 0. )
            inter = w * h
            IoU = inter / (areas[i] + areas[j] - inter)

            if method == 1:  # linear
                if IoU > Nt:
                    weight = 1 - IoU
                else:
                    weight = 1 
            elif method == 2:  # gaussian
                weight = np.exp(-(IoU * IoU)/sigma)
            else:  # original NMS
                if IoU > Nt:
                    weight = 0
                else:
                    weight = 1
            scores[j] = weight * scores[j]

            if scores[j] < thresh:  
                orders = np.delete(orders, np.where(orders == j))

        orders = np.delete(orders, 0)

    return keep

if __name__ == '__main__':

    dets = np.array([[100,120,170,200,0.98],
                     [20,40,80,90,0.94],
                     [20,38,82,88,0.99],
                     [200,380,282,488,0.9],
                     [19,38,75,91, 0.98]])

    print(soft_nms(dets, 2))