Watch what the slider destroys
Drag the threshold from 0 to 255 in the visualisation above and three things happen in order.
At very low values everything is foreground: the whole frame is white, because every pixel is at least as bright as almost nothing. As the threshold rises, the darkest regions drop out first — the dark rectangle goes early. Near the middle, the bright disc and the light triangle are still foreground while the background has gone, and the mask actually corresponds to the objects. Push higher still and the objects themselves start to disappear, brightest last.
The readout shows the percentage of pixels currently classified as foreground. That number moving smoothly while the *picture* changes abruptly is the thing worth noticing: there is no single moment where the mask becomes "correct". Correctness is defined by what you wanted, not by the image.
Otsu's method
Choosing the threshold by hand does not scale to a thousand images, so the standard automatic choice is Otsu's method. Switch the control above from Fixed to Otsu and the slider becomes irrelevant.
Otsu treats the problem statistically. Every candidate threshold splits the pixels into two groups, background and foreground. For each split you can compute how far apart the two group means are, weighted by how many pixels each group contains — the *between-class variance*. Otsu tries all 256 candidates and keeps the one that maximises it.
The intuition is that a good threshold produces two groups that are each internally consistent and clearly different from one another. A bad threshold slices through the middle of a group, and the two halves end up with similar means.
Otsu works well when the histogram is genuinely bimodal — two humps with a valley between them, one hump for the object and one for the background. It works badly when it is not: on an image that is 95% background with a small object, or one where illumination smears the two humps together, Otsu confidently returns a number that separates nothing.
Where a global threshold breaks
The assumption underneath everything above is that one number works for the whole image. It usually does not.
Photograph a page of text with a lamp on one side. The paper on the lit side is brighter than the *ink* on the dark side. No single threshold can call the lit paper background and the dark paper background while also calling the dark ink foreground — the values overlap, and overlapping values cannot be separated by a single cut.
The fix is adaptive thresholding: compute a local threshold for each pixel from the mean or the Gaussian-weighted mean of its neighbourhood, then compare the pixel to that. The lit side gets a high local threshold and the dark side a low one, and both are read correctly. The cost is a parameter — the neighbourhood size — which has to be larger than the strokes you want to keep and smaller than the illumination changes you want to remove.
What comes next
A thresholded mask is rarely clean. It has holes where a highlight fell inside an object, and speckles where noise crossed the threshold in the background. The standard follow-up is a morphological open to remove the speckles and a close to fill the holes, which is exactly what [erosion and dilation](erosion_and_dilation.html) are for.
Where it goes wrong
Thresholding a colour image by brightness. A saturated red and a mid-grey can have identical luminance. If colour is what distinguishes your object, threshold a colour channel — usually [hue](colour_spaces_rgb_hsv.html) — not the greyscale.
Trusting Otsu on a unimodal histogram. Check the histogram first. If there is one hump, there is no valley, and Otsu is returning the midpoint of a single distribution.
Thresholding before denoising. A salt-and-pepper pixel is by definition at an extreme value, so it survives any threshold. A median filter first costs almost nothing and removes them entirely.