Object Detection with Bounding Boxes
Detection is two problems stapled together: what is it, and where exactly is it. This is the "where" half — a regression problem with IoU sitting at the center of both its loss and its grading.
Predicted Box Offset
offsets from ground truth — all sliders to 0 means a perfect prediction
Predicted vs Ground Truth
—Loss as IoU Improves
Metrics
Object Detection with Bounding Boxes: A Practical Guide
Classification tells you what. A regression head tells you where.
Quick Context
An object detector's output head does two jobs per candidate region: classify what's there, and regress four numbers describing where it is precisely — typically a box center, width and height, or the offsets needed to nudge a fixed anchor box onto the real object. The classification half is an ordinary softmax problem. The localization half is what this module is about, and IoU sits at the center of it twice: once inside the training loss, and again in how the model gets graded afterward.
IoU as loss, IoU as grade
A natural localization loss is simply 1 − IoU between the predicted box and the ground truth box: 0 when they coincide exactly, approaching 1 as they stop overlapping at all. At evaluation time, the same IoU number gets a different job — a detection is usually scored as a true positive only if its IoU with the matching ground truth box clears a threshold, conventionally 0.5 under the PASCAL VOC convention. A box can be visibly "close" and still count as a complete miss if it doesn't clear that bar.
Interactive Exploration Guide
- Read the starting position. The predicted box is offset and undersized relative to the ground truth — overlap is low, loss is close to 1, and it does not count as a match.
- Drag dx and dy toward 0. IoU climbs as the boxes line up spatially, and the loss falls in lockstep — this is the gradient signal a real localization head would be trained on.
- Now close dw and dh too. Position alone isn't enough — a box in the right place but the wrong size still loses IoU on both sides.
- Watch the Match readout as IoU crosses 0.5. It flips from a miss to a match at exactly that boundary, even though the loss was already improving smoothly well before the flip.
Key Takeaway
1 − IoU is a smooth, differentiable signal that a network can actually be trained against. Whether a detection counts as correct at evaluation time is not smooth at all — it's a hard threshold on that same IoU number. A model can make genuine, loss-reducing progress on a box for a long stretch before that progress ever shows up as a correct detection in your metrics.