IoU and Non-Max Suppression

The problem it solves

A detector's region proposal stage doesn't stop at one box per object — it scores hundreds of candidate boxes and keeps each one above a confidence floor. It for a single real object often means a cluster of overlapping, near-duplicate boxes. Intersection over Union (IoU) is the standard way to measure how much two boxes overlap: the area they share, divided by the total area either one covers. IoU = 1 means identical boxes. IoU = 0 means no overlap at all.

Non-Max Suppression (NMS) uses IoU to collapse each cluster into a single detection. Sort proposals by confidence, keep the most confident one, discard each left box whose IoU with it exceeds a threshold. And repeat with the next survivor. The threshold is the just knob: too low and nearby but distinct objects get merged. Too high and duplicate boxes on the same object all survive. An IoU threshold of 0.5 is the traditional default for test benchmarks like PASCAL VOC, while COCO uses multiple thresholds from 0.5 to 0.95 in steps of 0.05 to reward tighter localisation.

This page walks through the IoU formula with a worked numeric case, builds NMS step by step. And lets you drag boxes in the visualisation to watch IoU update in real time. The code blocks compute IoU from coordinates, run NMS on a set of proposals. And print which boxes survive at other thresholds. It also covers Soft-NMS, which decays confidence scores instead of hard-deleting boxes, reducing the risk of suppressing a valid detection in a crowded scene.

Non-Max Suppression

NMS turns that overlap score into a cleanup rule. Sort each proposal by confidence, descending. Take the top one, keep it. And discard each left box whose IoU with it exceeds a threshold — those are treated as duplicate detections of the same object. Move to the next surviving box by confidence and repeat, until nothing is left to process. The threshold is the just knob: too low and boxes on genuinely other but nearby objects get merged into one. Too high and duplicate boxes on the same object all survive.

Measuring overlap with one count

IoU checks two boxes by dividing the area they share by the area they cover together:

IoU = intersection / union

Worked through. Box A spans (0, 0) to (100, 100). Box B spans (50, 50) to (150, 150).

• Intersection: from (50, 50) to (100, 100) — 50 × 50 = 2,500.
• Union: 10,000 + 10,000 − 2,500 = 17,500.
• IoU = 2,500 / 17,500 = 0.14.

Reading the scale:

Two properties make it the standard measure. It is scale-invariant — a 20-pixel error on a small object scores the same as a proportionally close error on a large one — and it is bounded. So it can be thresholded and averaged.

def iou(a, b):
    x1, y1 = max(a[0], b[0]), max(a[1], b[1])
    x2, y2 = min(a[2], b[2]), min(a[3], b[3])
    inter = max(0, x2 - x1) * max(0, y2 - y1)
    area_a = (a[2] - a[0]) * (a[3] - a[1])
    area_b = (b[2] - b[0]) * (b[3] - b[1])
    return inter / (area_a + area_b - inter)

The two max(0, ...) calls are what handle boxes that do not overlap. Without them a negative width times a negative height gives a positive area and a nonsensical result.

Non-max suppression, step by step

A detector fires at many nearby positions. So one object gives a cluster of boxes. NMS reduces each cluster to one.

1. Discard each box below a confidence threshold.
2. Sort the rest by confidence, highest first.
3. Take the top box and keep it.
4. Remove each left box whose IoU with it exceeds the NMS threshold.
5. Repeat from step 3 with what is left.

Two thresholds control the outcome, and they do other jobs. The confidence threshold decides how sure the net must be before a box is considered at all — raise it for fewer false positives, less it for better recall. The NMS IoU threshold decides how much overlap counts as a duplicate. 0.45–0.5 is typical.

The failure mode is genuinely overlapping objects: two people standing close together give boxes with high IoU. And standard NMS deletes one of them. Soft-NMS addresses this by reducing the confidence of overlapping boxes rather than deleting them outright. So a strong second detection can still survive.

Note also that NMS is applied per class in most implementations — a dog box and a person box overlapping heavily are not duplicates of each other.

Scoring box overlap, then removing the duplicates

Intersection over union is four max/min steps. Non-max suppression is a sort and a loop. Both are written out here, along with the two cases where each one behaves badly.

import numpy as np

def iou(a, b):
    # boxes are (x1, y1, x2, y2)
    ix1, iy1 = max(a[0], b[0]), max(a[1], b[1])
    ix2, iy2 = min(a[2], b[2]), min(a[3], b[3])
    iw, ih = max(0.0, ix2 - ix1), max(0.0, iy2 - iy1)
    inter = iw * ih
    area_a = (a[2] - a[0]) * (a[3] - a[1])
    area_b = (b[2] - b[0]) * (b[3] - b[1])
    return inter / (area_a + area_b - inter)

A = (10.0, 10.0, 50.0, 50.0)
print("IOU, step by step. box A = %s, area %.0f" % (A, (A[2] - A[0]) * (A[3] - A[1])))
print()
print("%28s %10s %10s %10s %8s" % ("box B", "intersect", "union", "IoU", "verdict"))
for B, label in (((10, 10, 50, 50), "identical"),
                 ((20, 20, 60, 60), "shifted"),
                 ((30, 30, 70, 70), "corner touch"),
                 ((10, 10, 30, 30), "B inside A"),
                 ((60, 60, 90, 90), "no overlap")):
    B = tuple(float(v) for v in B)
    ix1, iy1 = max(A[0], B[0]), max(A[1], B[1])
    ix2, iy2 = min(A[2], B[2]), min(A[3], B[3])
    inter = max(0.0, ix2 - ix1) * max(0.0, iy2 - iy1)
    union = (A[2] - A[0]) * (A[3] - A[1]) + (B[2] - B[0]) * (B[3] - B[1]) - inter
    v = iou(A, B)
    print("%28s %10.0f %10.0f %10.4f %8s"
          % (str(tuple(int(x) for x in B)), inter, union, v,
             "match" if v >= 0.5 else "reject"))
print()
print("   the max/min pair is the whole trick: the intersection's left edge")
print("   is the RIGHTMOST of the two left edges, and its right edge is the")
print("   LEFTMOST of the two right edges. if that gives a negative width,")
print("   the boxes do not overlap and the clamp to 0 handles it.")

Next, slide two boxes of equal size apart to see how much overlap the 0.5 threshold really needs.

import numpy as np

def iou(a, b):
    ix = max(0., min(a[2],b[2])-max(a[0],b[0])) * max(0., min(a[3],b[3])-max(a[1],b[1]))
    return ix / ((a[2]-a[0])*(a[3]-a[1]) + (b[2]-b[0])*(b[3]-b[1]) - ix)

A = (10.0, 10.0, 50.0, 50.0)
print("WHAT 0.5 ACTUALLY LOOKS LIKE. two boxes of equal size, slid apart:")
print("%14s %10s %s" % ("offset", "IoU", ""))
for off in (0, 5, 10, 13, 20, 30, 40):
    B = (10.0 + off, 10.0, 50.0 + off, 50.0)
    v = iou(A, B)
    print("%14d %10.4f %s" % (off, v, "#" * int(v * 40)))
print("   a 40-pixel box shifted 13 pixels -- a third of its width -- still")
print("   scores about 0.5. the standard threshold is more forgiving than")
print("   it sounds, which is why COCO reports mAP averaged over 0.5 to")
print("   0.95 rather than at 0.5 alone.")

With IoU set, set up a cluster of seven overlapping proposals to test non-max suppression.

import numpy as np

boxes = np.array([
    [12, 12, 52, 52], [14, 15, 54, 55], [11, 9, 49, 47],   # three on object 1
    [80, 80, 130, 130], [82, 78, 128, 132],                # two on object 2
    [200, 40, 240, 80],                                    # one on object 3
    [36, 20, 76, 60],                                      # object 4, overlapping 1
], float)
scores = np.array([0.92, 0.88, 0.75, 0.95, 0.71, 0.66, 0.80])
names = ["A", "B", "C", "D", "E", "F", "G"]

print("NON-MAX SUPPRESSION. seven detections. objects 1, 2 and 3 are well")
print("separated; a fourth object G genuinely overlaps object 1:")
print("%6s %28s %8s" % ("box", "coords", "score"))
for n, b, s in zip(names, boxes, scores):
    print("%6s %28s %8.2f" % (n, str(tuple(int(x) for x in b)), s))

Now apply the NMS algorithm at threshold 0.5 and trace which boxes survive.

import numpy as np

def iou(a, b):
    ix = max(0., min(a[2],b[2])-max(a[0],b[0])) * max(0., min(a[3],b[3])-max(a[1],b[1]))
    return ix / ((a[2]-a[0])*(a[3]-a[1]) + (b[2]-b[0])*(b[3]-b[1]) - ix)

boxes = np.array([[12,12,52,52],[14,15,54,55],[11,9,49,47],
                  [80,80,130,130],[82,78,128,132],
                  [200,40,240,80],[36,20,76,60]], float)
scores = np.array([0.92, 0.88, 0.75, 0.95, 0.71, 0.66, 0.80])
names = ["A", "B", "C", "D", "E", "F", "G"]
def nms(boxes, scores, thresh):
    order = list(np.argsort(-scores))
    keep, log = [], []
    while order:
        i = order.pop(0)
        keep.append(i)
        survivors = []
        for j in order:
            v = iou(boxes[i], boxes[j])
            if v > thresh:
                log.append((names[i], names[j], v, "suppressed"))
            else:
                log.append((names[i], names[j], v, "kept"))
                survivors.append(j)
        order = survivors
    return keep, log

keep, log = nms(boxes, scores, 0.5)
print("   the algorithm: take the highest-scoring box, suppress everything")
print("   that overlaps it too much, repeat with what is left.")
print()
print("%10s %10s %10s %14s" % ("winner", "compared", "IoU", "outcome"))
for w, c, v, out in log:
    print("%10s %10s %10.4f %14s" % (w, c, v, out))
print()
print("   kept: %s" % ", ".join(names[i] for i in keep))
print("   IoU between A (object 1) and G (object 4): %.4f" % iou(boxes[0], boxes[6]))
print("   they are different objects that happen to overlap, and whether")
print("   NMS keeps both depends entirely on where the threshold sits.")

Sweeping the threshold shows how strongly it controls the outcome.

import numpy as np

def iou(a, b):
    ix = max(0., min(a[2],b[2])-max(a[0],b[0])) * max(0., min(a[3],b[3])-max(a[1],b[1]))
    return ix / ((a[2]-a[0])*(a[3]-a[1]) + (b[2]-b[0])*(b[3]-b[1]) - ix)

boxes = np.array([[12,12,52,52],[14,15,54,55],[11,9,49,47],
                  [80,80,130,130],[82,78,128,132],
                  [200,40,240,80],[36,20,76,60]], float)
scores = np.array([0.92, 0.88, 0.75, 0.95, 0.71, 0.66, 0.80])
names = ["A", "B", "C", "D", "E", "F", "G"]

def nms(boxes, scores, thresh):
    order = list(np.argsort(-scores))
    keep = []
    while order:
        i = order.pop(0)
        keep.append(i)
        order = [j for j in order if iou(boxes[i], boxes[j]) <= thresh]
    return keep

print("THE THRESHOLD IS A REAL DIAL:")
print("%14s %30s %10s" % ("IoU threshold", "kept", "count"))
for t in (0.05, 0.2, 0.3, 0.5, 0.7, 0.95):
    k = nms(boxes, scores, t)
    print("%14.2f %30s %10d" % (t, ", ".join(names[i] for i in k), len(k)))
print("   at a low threshold G is deleted -- a real object lost because it")
print("   overlapped a more confident one. at a high threshold the")
print("   duplicates B, C and E survive. no single value is right for both.")

NMS breaks down when objects genuinely overlap, and Soft-NMS offers a workaround.

import numpy as np

def iou(a, b):
    ix = max(0., min(a[2],b[2])-max(a[0],b[0])) * max(0., min(a[3],b[3])-max(a[1],b[1]))
    return ix / ((a[2]-a[0])*(a[3]-a[1]) + (b[2]-b[0])*(b[3]-b[1]) - ix)

def nms(boxes, scores, thresh):
    order = sorted(range(len(scores)), key=lambda i: -scores[i])
    keep = []
    while order:
        i = order.pop(0)
        keep.append(i)
        order = [j for j in order if iou(boxes[i], boxes[j]) <= thresh]
    return keep

print("WHERE IT FAILS. two people standing close together:")
crowd = np.array([[10, 10, 40, 70], [18, 10, 48, 70]], float)
crowd_scores = np.array([0.9, 0.85])
v = iou(crowd[0], crowd[1])
print("   two genuinely separate objects, IoU %.4f" % v)
for t in (0.3, 0.5, 0.7):
    k = nms(crowd, crowd_scores, t)
    print("      threshold %.1f -> keeps %d box(es)%s"
          % (t, len(k), "  <- a real person deleted" if len(k) < 2 else ""))
print("   NMS cannot tell 'a duplicate of one object' from 'two objects")
print("   that overlap'. it only sees the IoU.")
print()
print("   SOFT-NMS is the usual repair: instead of deleting an overlapping")
print("   box, multiply its score down and let it compete again:")
for v_ in (0.3, 0.5, 0.7, 0.9):
    print("      IoU %.1f -> score x %.3f (gaussian, sigma 0.5)"
          % (v_, np.exp(-v_ ** 2 / 0.5)))
print("   a heavily overlapping box is nearly removed; a marginal one keeps")
print("   most of its score and survives if it was confident enough.")

Try it yourself

1. Read the stage at the default threshold. Box A (confidence 0.95) is kept. Box D, a split object with no real overlap with A, is also kept. Boxes B and C overlap A too heavily and are suppressed as duplicates.
2. Less the threshold toward 0.1. Nothing changes here — B and C now fail a much looser bar. So tightening it more has no effect on this particular layout.
3. Raise the threshold past roughly 0.59. Box B's overlap with A no longer exceeds the (now looser) suppression bar. So B is kept as a second detection of what is really the same object.
4. Raise it past roughly 0.81. Now C survives too — at this threshold NMS considers all four boxes other enough to keep, still though A, B and C clearly describe one dog.

The short of it

IoU is a pure geometry math — it knows nothing about confidence or class. NMS is the policy layer that uses it: keep the most confident box, discard anything that overlaps it past a threshold, repeat. Set the threshold too low and it merges distinct nearby objects into one detection. Set it too high and duplicate boxes on the same object all survive. Each object detector that outputs boxes runs some form of this after its raw proposals come out.

IoU as a loss function

Since IoU is what test measures, training right on it is appealing — and it took a few iterations to make it work.

Plain IoU has a fatal gradient problem: when two boxes do not overlap at all, IoU is 0 for each possible position. So there is no gradient telling the box which way to move. Three refinements fix it:

• GIoU adds a cost based on the smallest box enclosing both. So non-overlapping boxes still get a useful gradient.
• DIoU adds a term for the distance between the box centres, which converges faster.
• CIoU adds an aspect-ratio term as well. And is the default regression loss in a few modern detectors.

The option, still widely used, is smooth L1 on the coordinates — simple and stable. But it optimises a proxy rather than the metric. And it treats a 10-pixel error the same on a small object as on a large one.

Where IoU appears beyond detection

• Giving anchors in training. An anchor box is treated as a positive case if its IoU with a ground-truth box exceeds a threshold. And as background below a less one. Anchors in between are ignored.
• Segmentation. The same ratio computed over pixel masks rather than boxes is the Jaccard index. And mean IoU is the standard segmentation metric.
• Tracking. Frame-to-frame association is often IoU-based — a detection is matched to the track whose last box it overlaps most.
• Test at a few thresholds. COCO's primary metric averages mAP over IoU thresholds from 0.5 to 0.95. It is what makes it reward tight boxes rather than approximate ones.

Common mistakes

• Forgetting to clamp negative overlaps, giving a positive intersection for disjoint boxes.
• Mixing box formats. Applying corner-format code to centre-format boxes gives plausible-looking nonsense.
• Running NMS across classes when it should be per class, deleting legitimate detections of other objects.
• An NMS threshold that is too low, which removes genuine nearby objects. Or too high, which leaves duplicates.
• Tuning the confidence threshold on the test set rather than on validation data.
• Assuming NMS is cheap. With tens of thousands of candidate boxes it can dominate inference time. Batched GPU implementations exist for this reason.

Questions people ask

What IoU threshold means "correct"? 0.5 by the Pascal VOC convention. COCO averages from 0.5 to 0.95 in steps of 0.05. It is stricter and now more standard.

Can IoU exceed 1? No — the intersection can never exceed the union.

Is NMS still needed? For most detectors, yes. DETR-style set-prediction nets avoid it by design, which is one of their main attractions.

How do I handle heavily overlapping objects? Soft-NMS, a more NMS threshold. Or an design that predicts a set right.

Should I train with an IoU-based loss? CIoU or DIoU generally beats smooth L1 on localisation quality. And both are readily on hand in modern detection libraries.

Why is my mAP low despite good-looking boxes? Often the strict IoU thresholds: boxes that look right to a human may score 0.6. It passes at 0.5 and fails at 0.75.

Recap in one screen

• IoU is overlap divided by union — scale-invariant, bounded between 0 and 1.
• Clamp negative overlaps to zero. Or disjoint boxes report a false positive area.
• NMS keeps the highest-confidence box and removes overlapping duplicates, per class.
• Two thresholds matter: confidence (how sure) and NMS IoU (how much overlap is a duplicate).
• GIoU, DIoU and CIoU make IoU usable as a training loss by supplying gradients when boxes do not overlap.