One number turns a photograph into a silhouette. Choosing it well is the whole difficulty.
Overview
Turning grey into black and white
A greyscale pixel holds one of 256 values. Thresholding replaces it with one of two: if the value is at least *t*, output white. Or else output black.
output(x, y) = 255 if input(x, y) >= t else 0
There's nothing more to the op. Its interest lies fully in the choice of *t*, and in what the result is then used for. After thresholding, an image is no longer a picture. It's a mask, a statement about which pixels belong to a thing and which don't. Counting objects, measuring areas, finding contours and reading printed text all begin here.
Thresholding
This module needs JavaScript: the
images are computed in the page rather than downloaded.
Worth knowing
Thresholding maps every pixel to one of two values by comparing it against a single number.
It throws away almost all the information in the image. That is the point: what remains is shape.
Otsu's method picks the threshold that best separates the pixels into two groups, by maximising the variance between them.
A single global threshold fails the moment lighting is uneven. Adaptive thresholding computes a different value per region.
Thresholding
The simplest segmentation there's, and the one that fails in the most instructive ways.
Watch what the slider destroys
Drag the threshold from 0 to 255 in the visualisation above and three things happen in order.
At very low values all is foreground: the whole frame is white, because each pixel is at least as bright as almost nothing. As the threshold rises, the darkest regions drop out first. The dark rectangle goes early. Near the middle, the bright disc and the light triangle are still foreground while the background has gone, and the mask really matches to the objects. Push higher still and the objects themselves start to disappear, brightest last.
The readout shows the percentage of pixels currently classified as foreground. That number moving smoothly while the *picture* changes abruptly is the thing worth noticing: there's no single moment where the mask turns "correct". Correctness is defined by what you wanted, not by the image.
Otsu's method
Choosing the threshold by hand doesn't scale to a thousand images, so the standard auto choice is Otsu's method. Switch the control above from Fixed to Otsu and the slider turns irrelevant.
Otsu treats the problem statistically. each pick threshold splits the pixels into two groups, background and foreground. For each split you can find how far apart the two group means are, weighted by how many pixels each group holds — the *between-class variance*. Otsu tries all 256 picks and keeps the one that raises it.
The intuition is that a good threshold makes two groups that are each inside steady and clearly different from one another. A bad threshold slices through the middle of a group, and the two halves end up with similar means.
Otsu works well when the histogram is truly bimodal — two humps with a valley between them, one hump for the object and one for the background. It works badly when it isn't: on an image that is 95% background with a small object, or one where illumination smears the two humps together, Otsu confidently returns a number that separates nothing.
Where a global threshold breaks
The assumption underneath all above is that one number works for the whole image. It often doesn't.
Photo a page of text with a lamp on one side. The paper on the lit side is brighter than the *ink* on the dark side. No single threshold can call the lit paper background and the dark paper background while also calling the dark ink foreground — the values share, and shared values can't be separated by a single cut.
The fix is adaptive thresholding: find a local threshold for each pixel from the mean or the Gaussian-weighted mean of its zone, then compare the pixel to that. The lit side gets a high local threshold and the dark side a low one, and both are read right. The cost is a param — the zone size — which has to be larger than the strokes you want to keep and smaller than the illumination changes you want to drop.
What comes next
A thresholded mask is rarely clean. It has holes where a highlight fell inside an object, and speckles where noise crossed the threshold in the background. The standard follow-up is a morphological open to drop the speckles and a close to fill the holes. This Is just what erosion and dilation are for.
What Otsu maximises, worked
Otsu picks the threshold by trying all 256 and keeping the one whose two groups are most separated — the max *between-class variance*, w_bg · w_fg · (mean_bg − mean_fg)². On a tiny bimodal set of pixel values {1, 1, 2, 2, 8, 8, 9, 9}:
threshold
groups
between-class variance
2
{1,1} vs {2,2,8,8,9,9}
5.33
4
{1,1,2,2} vs {8,8,9,9}
12.25
6
{1,1,2,2} vs {8,8,9,9}
12.25
The max, 12.25, is reached by any threshold sitting in the empty valley between the two humps (here 3 through 7 all split the data the same way), which is just where a threshold belongs. A threshold at 2 slices through the low hump and the two groups' means end up closer. So The variance is lower. This is why Otsu works on a truly *bimodal* histogram and fails on a unimodal one: with a single hump there is no valley, and Otsu confidently returns the midpoint of one spread, separating nothing. And it's global. One threshold for the whole image — so uneven lighting. Here Lit background outshines shadowed ink, defeats it even so of the number chosen. This Is what adaptive thresholding fixes with a local threshold per region.
Where it goes wrong with Thresholding
Thresholding a colour image by brightness. A saturated red and a mid-grey can have the same luminance. If colour is what sets apart your object, threshold a colour channel — often hue — not the greyscale.
Trusting Otsu on a unimodal histogram. Check the histogram first. If there is one hump, there's no valley, and Otsu is returning the midpoint of a single spread.
Thresholding before denoising. A salt-and-pepper pixel is by meaning at an extreme value, so it lives any threshold. A median filter first costs almost nothing and drops them fully.
Questions people ask about Thresholding
What does thresholding do? It turns a greyscale image into a binary one by setting pixels above a cut-off to white and the rest to black.
How does Otsu's method choose the threshold? It tries every value and picks the one that maximises the variance between the two resulting classes. It works best when the histogram has two clear peaks.
When does a global threshold fail? When lighting is uneven, so the background brightness changes across the image. Adaptive thresholding computes a separate threshold for each local neighbourhood.
What is adaptive thresholding? Each pixel is compared with the mean or Gaussian-weighted mean of its neighbourhood minus a constant. It copes with shadows and gradients.
Should I blur before thresholding? Usually, yes. Noise pushes pixels across the threshold at random, so a light blur gives cleaner regions.
What do I do about specks left after thresholding? Clean the mask with morphological opening to remove specks and closing to fill small holes.
Turning grey into black and white, and choosing where
A threshold splits pixels into two classes. Otsu's method picks the split automatically, adaptive thresholding picks a different one per region. The case where a global threshold fails is easy to make.
example_01.pyNumPy
import numpy as np
rng = np.random.default_rng(0)
SHADE = " .:-=+*#%@"
def show(a, label):
print(" %s" % label)
lo, hi = a.min(), a.max()
for row in a:
print(" " + "".join(SHADE[int(np.clip((v - lo) / max(hi - lo, 1e-9), 0, 1) * 9)]
for v in row))
H, W = 12, 24
page = np.full((H, W), 200.0)
text = [(2, 3), (2, 4), (2, 5), (3, 3), (4, 3), (4, 4), (4, 5),
(6, 12), (6, 13), (7, 12), (8, 12), (8, 13),
(2, 18), (3, 18), (4, 18), (4, 19), (4, 20),
(8, 4), (8, 5), (9, 4), (9, 5)]
for r, c in text:
page[r, c] = 40.0
page = page * np.linspace(1.0, 0.13, W)[None, :]
page = np.clip(page + rng.normal(0, 3, page.shape), 0, 255)
show(page, "input: dark marks on a page, lit unevenly from the left")
print(" note the right-hand side is genuinely darker -- the page there is")
print(" dimmer than the INK on the left.")
print(" brightest page pixel on the right : %.0f" % page[:, -6:].max())
print(" darkest ink pixel on the left : %.0f"
% min(page[r, c] for r, c in text if c < 8))
print(" no single number can separate ink from page across the whole")
print(" image, and that is the whole problem.")
print()
hist, edges = np.histogram(page, bins=16, range=(0, 256))
print("THE HISTOGRAM, which is what a threshold is chosen from:")
for i, count in enumerate(hist):
if count:
print(" %3.0f-%3.0f %5d %s"
% (edges[i], edges[i + 1], count, "#" * int(40 * count / hist.max())))
Output
The next block implements Otsu's method, which picks the threshold automatically by maximising the between-class variance, then compares global and adaptive thresholding on the unevenly lit page.
example_02.pyNumPy
import numpy as np
rng = np.random.default_rng(0)
H, W = 12, 24
page = np.full((H, W), 200.0)
text = [(2,3),(2,4),(2,5),(3,3),(4,3),(4,4),(4,5),(6,12),(6,13),(7,12),
(8,12),(8,13),(2,18),(3,18),(4,18),(4,19),(4,20),(8,4),(8,5),(9,4),(9,5)]
for r, c in text:
page[r, c] = 40.0
page = page * np.linspace(1.0, 0.13, W)[None, :]
page = np.clip(page + rng.normal(0, 3, page.shape), 0, 255)
def otsu(a, bins=64):
hist_, edges_ = np.histogram(a, bins=bins, range=(0, 256))
total = hist_.sum()
centres = (edges_[:-1] + edges_[1:]) / 2
best_t, best_var = 0.0, -1.0
for i in range(1, bins):
w0 = hist_[:i].sum() / total
w1 = 1 - w0
if w0 == 0 or w1 == 0:
continue
m0 = (hist_[:i] * centres[:i]).sum() / hist_[:i].sum()
m1 = (hist_[i:] * centres[i:]).sum() / hist_[i:].sum()
var = w0 * w1 * (m0 - m1) ** 2
if var > best_var:
best_var, best_t = var, centres[i]
return best_t
t_otsu = otsu(page)
print("OTSU'S METHOD searches every threshold and keeps the one that")
print("maximises the variance BETWEEN the two groups -- equivalently, the")
print("one that minimises the variance within them:")
print(" chosen threshold: %.1f" % t_otsu)
Output
The next block applies global and adaptive thresholding side by side, then measures each against the known ink positions.
example_03.pyNumPy
import numpy as np
rng = np.random.default_rng(0)
H, W = 12, 24
page = np.full((H, W), 200.0)
text = [(2,3),(2,4),(2,5),(3,3),(4,3),(4,4),(4,5),(6,12),(6,13),(7,12),
(8,12),(8,13),(2,18),(3,18),(4,18),(4,19),(4,20),(8,4),(8,5),(9,4),(9,5)]
for r, c in text:
page[r, c] = 40.0
page = page * np.linspace(1.0, 0.13, W)[None, :]
page = np.clip(page + rng.normal(0, 3, page.shape), 0, 255)
SHADE = " .:-=+*#%@"
def show(a, label):
print(" %s" % label)
lo, hi = a.min(), a.max()
for row in a:
print(" " + "".join(SHADE[int(np.clip((v-lo)/max(hi-lo,1e-9),0,1)*9)] for v in row))
def apply_global(a, t):
return (a < t).astype(float)
def adaptive(a, block=7, offset=8.0):
p = np.pad(a, block // 2, mode="edge")
out = np.zeros_like(a)
for i in range(a.shape[0]):
for j in range(a.shape[1]):
local = p[i:i + block, j:j + block].mean()
out[i, j] = 1.0 if a[i, j] < local - offset else 0.0
return out
truth = np.zeros((H, W))
for r, c in text:
truth[r, c] = 1.0
for name, out in [("global, threshold 100", apply_global(page, 100.0)),
("global, Otsu 36", apply_global(page, 36.0)),
("adaptive, 7x7 local mean", adaptive(page))]:
show(out, name)
tp, fp = int((out * truth).sum()), int((out * (1 - truth)).sum())
print(" found %d of %d marks, with %d false positives" % (tp, int(truth.sum()), fp))
print()
Output
The next block sweeps through several global thresholds to show the precision-recall trade-off, then demonstrates how adaptive thresholding sidesteps it entirely.
example_04.pyNumPy
import numpy as np
rng = np.random.default_rng(0)
H, W = 12, 24
page = np.full((H, W), 200.0)
text = [(2,3),(2,4),(2,5),(3,3),(4,3),(4,4),(4,5),(6,12),(6,13),(7,12),
(8,12),(8,13),(2,18),(3,18),(4,18),(4,19),(4,20),(8,4),(8,5),(9,4),(9,5)]
for r, c in text:
page[r, c] = 40.0
page = page * np.linspace(1.0, 0.13, W)[None, :]
page = np.clip(page + rng.normal(0, 3, page.shape), 0, 255)
truth = np.zeros((H, W))
for r, c in text:
truth[r, c] = 1.0
def apply_global(a, t):
return (a < t).astype(float)
def adaptive(a, block=7, offset=8.0):
p = np.pad(a, block // 2, mode="edge")
out = np.zeros_like(a)
for i in range(a.shape[0]):
for j in range(a.shape[1]):
out[i, j] = 1.0 if a[i, j] < p[i:i+block, j:j+block].mean() - offset else 0.0
return out
print("SWEEP OF GLOBAL THRESHOLDS:")
print("%16s %10s %10s %10s" % ("threshold", "found", "missed", "false pos"))
for t in (15, 22, 30, 60, 100, 36):
out = apply_global(page, t)
print("%16.0f %10d %10d %10d"
% (t, int((out*truth).sum()), int(((1-out)*truth).sum()), int((out*(1-truth)).sum())))
print(" ink and page brightness ranges overlap, and no threshold fixes that.")
print()
print("ADAPTIVE compares each pixel to its own neighbourhood:")
for block in (3, 7, 15):
o = adaptive(page, block)
print(" block %2d: found %2d, false positives %2d"
% (block, int((o * truth).sum()), int((o * (1 - truth)).sum())))
print(" a global statistic fails whenever the thing you measure varies across")
print(" the image -- also why normalisation layers exist.")
Output
Check yourself
0 of 3
Answer without scrolling back up.
What does Otsu's method maximise when choosing a threshold?
It tries every candidate threshold and keeps the one whose two resulting groups have the most separated means, weighted by group size.
Why does a single global threshold fail on a page photographed under uneven lighting?
Once background values on one side exceed foreground values on the other, no single cut separates them. Adaptive thresholding computes a local threshold per neighbourhood instead.
Why should a median filter usually run before thresholding a noisy image?
Impulse noise is at 0 or 255 by definition, so it lands on one side of every threshold. Removing it first is cheap and leaves the mask clean.
Cheat sheet
Thresholding
Turn a greyscale image into a binary mask with global and adaptive thresholds, and see where uniform lighting stops being enough.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.