Slide a patch over the image and score every position. The oldest way to find something, and the reason better ways exist.
Overview
The method
Take a patch — the template — and place it over each possible spot in the image. At each spot, score how well the template agrees with what is underneath. The result is a response map with one score per spot, and the highest score is where the template was found.
The template here is cut from the image itself, using the spot controls, so a perfect match is ensured to exist. The grey square marks where it came from and the orange square marks where it was found. Move the controls and they track each other, with the link in the readout staying at 1.000.
Template Matching
This module needs JavaScript: the
images are computed in the page rather than downloaded.
Worth knowing
The template is cut from the image itself, so a perfect match exists and you can see whether the method finds it.
Normalised cross-correlation subtracts the mean and divides by the standard deviation on both sides, so it survives lighting changes.
The output is a response map: one score per position. Its brightest point is the match.
It is not invariant to rotation or scale. Turn the object ten degrees and the correlation collapses.
Template Matching
The most direct way to find a thing in an image. A clear demo of why the field moved on.
Scoring: why not just subtract?
The obvious score is the sum of squared differences. It works, and it breaks the moment lighting changes: brighten the image by 20 and each difference grows, even where the shapes match just right.
Normed cross-link fixes this by standardising both sides before checking — subtract the mean, divide by the standard deviation, then take the dot product:
Subtracting the means drops any constant brightness offset. Cutting by the standard deviations drops any constant contrast scaling. What is left measures pattern rather than pixel values, and the result is bounded between −1 and 1, which makes scores close across spots and images.
That is why NCC is the standard choice. Why the response map here is readable as an image: each value is on the same scale.
Reading the response map
The bright region around the true match isn't a single pixel. It's a blob, because a template shifted by one pixel still shares almost fully and still scores well.
That blob is why picking the max isn't enough . a few nearby spots all score highly, and returning all of them means returning the same spotting a few times. The standard fix is the same one spotting uses: suppress all within a radius of a stronger response, which is just non-max suppression.
Make the template small — drag the size control down — and the map turns noisy, with strong responses in places that merely happen to have a similar local pattern. A small template doesn't hold enough structure to be distinctive.
What it cannot do
The limitations are severe and they're the reason the field developed options.
Rotation. A template rotated by ten degrees no longer aligns, and the link falls sharply. Handling rotation means searching over angles as well as spots.
Scale. The same problem in another dim. Searching over scales too means an image pyramid and a much larger search.
Deformation. A face isn't a rigid patch. Any object that bends, or that varies between instances, can't be shown by one template.
Cost. each spot ×. each scale ×. each angle, each needing a full patch check, is costly, though link via the FFT helps a great deal in the translation-only case.
Where it is still the right tool
Template matching hasn't disappeared, because when its assumptions hold it's exact, needs no training data, and is trivially explainable.
Industrial inspection with a fixed camera and fixed part orientation. Finding a known icon on a screen for UI automation. Aligning scanned docs against a form. Matching a fixed logo. In all of these, spot is the only unknown, which is just the case the method solves.
Where the assumptions fail, the successors are keypoint matching — Harris corners, SIFT, ORB — which find distinctive points and match those, giving rotation and scale steadiness for free, and learned finders, which handle deformation and appearance variation as well.
Why normalisation is the whole trick
The reason NCC beats raw subtraction is one line of math. Take a template of pixel values and a patch that is the same pattern brightened by 20 — the kind of lighting change that breaks sum-of-squared-differences fully, since each difference grows. Normed cross-link subtracts each side's mean and cuts by its standard deviation before checking:
comparison
score
template vs itself
1.000
template vs (itself + 20 brightness)
1.000
template vs a different pattern
well below 1
Subtracting the means drops any constant brightness offset. Cutting by the standard deviations drops any contrast scaling. What remains measures *pattern*, bounded in −1 to 1 so scores are close across spots. That steadiness is why the response map is readable as an image and why the peak stays at 1.000 under a lighting change that would swamp SSD. What norming can't fix is the geometry: a template rotated ten degrees or scaled up no longer aligns, and the link merges — which is the wall that sent the field toward keypoint finders that live rotation and scale.
Reading the response map is its own lesson: the bright region around the true match is a blob, not a point. A template shifted one pixel still shares almost fully and still scores well. So taking the argmax returns the same spotting a few times, and the fix is the same non-max suppression that object spotting uses. Where template matching lives is just the case its assumptions fit: a fixed camera and fixed orientation, where spot is the only unknown — industrial inspection, finding an icon for UI automation, aligning a scanned form. When the object can rotate, scale or deform, it's the wrong method rather than one that needs tuning, and the successors are keypoint matchers like SIFT and ORB that get rotation and scale steadiness for free.
Where it goes wrong with Template Matching
Sum of squared differences under changing light. Use NCC.
Taking the argmax without suppression. The peak is a blob, not a point.
A template that is too small. Not distinctive. The map fills with false peaks.
Expecting steadiness it doesn't have. If the object can rotate or change size, this is the wrong method, not a method that needs tuning.
Questions people ask about Template Matching
How does template matching work? It slides the template across the image and scores the similarity at every position. The best score marks the most likely location.
Which matching method should I use? Normalised correlation coefficient, TM_CCOEFF_NORMED, is a good default. It is tolerant of brightness and contrast changes, unlike plain squared difference.
Does it handle rotation or scale changes? No. It only finds the template at the same size and orientation, so you would need to search over rotated and scaled copies.
How do I find multiple matches? Threshold the response map and take the local maxima, with non-maximum suppression so one object does not give a cluster of hits.
Why is it slow on large images? It does a comparison at every pixel position over the whole template area. FFT-based correlation or an image pyramid speeds it up considerably.
When should I use feature matching or a detector instead? When the object varies in scale, rotation, viewpoint or lighting. Template matching suits fixed, known appearances such as UI icons or printed marks.
Sliding a patch, and the correlation that lies
Template matching is the simplest possible finder: slide a patch over the image and score each spot. Building it exposes just why raw link fails, why normed cross-link is the fix. The limits that no amount of norming repairs.
example_01.pyNumPy
import numpy as np
rng = np.random.default_rng(7)
H, W = 16, 34
img = rng.normal(70, 6, (H, W))
tpl = np.array([[20., 90., 20.], [90., 140., 90.], [20., 90., 20.]])
def paste(a, t, r, c, gain=1.0, offset=0.0):
a[r:r+t.shape[0], c:c+t.shape[1]] = t * gain + offset
paste(img, tpl, 3, 5)
paste(img, tpl, 9, 20, gain=0.45, offset=85)
img[6:11, 27:32] = 240.0
img = np.clip(img, 0, 255)
ramp = " .:-=+*#%@"
def show(a, label, hi=255):
print(" %s" % label)
for row in a:
print(" " + "".join(ramp[min(9, int(9*max(0,v)/hi))] for v in row))
show(img, "the image")
print(" two plus signs: exact at (3,5), faint at (9,20). bright blob at (6,27).")
print()
th, tw = tpl.shape
def score_map(fn):
m = np.full((H-th+1, W-tw+1), -np.inf)
for r in range(H-th+1):
for c in range(W-tw+1):
m[r,c] = fn(img[r:r+th, c:c+tw])
return m
print("METHOD 1 -- RAW CORRELATION: sum(patch * template).")
raw = score_map(lambda p: float((p*tpl).sum()))
print(" scores at the three interesting places:")
for label, (r, c) in (("exact plus",(3,5)),("faint plus",(9,20)),("bright blob",(6,28))):
print("%20s %12.0f" % (label, raw[r, c]))
print(" the blob WINS. raw correlation rewards brightness: 240 everywhere")
print(" times any positive template gives a big number. it is a")
print(" brightness detector with extra steps.")
Output
The next block tries sum of squared differences, which fixes the brightness bug but fails on a faint copy of the same shape.
example_02.pyNumPy
import numpy as np
rng = np.random.default_rng(7)
H, W = 16, 34
img = rng.normal(70, 6, (H, W))
tpl = np.array([[20., 90., 20.], [90., 140., 90.], [20., 90., 20.]])
def paste(a, t, r, c, gain=1.0, offset=0.0):
a[r:r+t.shape[0], c:c+t.shape[1]] = t * gain + offset
paste(img, tpl, 3, 5)
paste(img, tpl, 9, 20, gain=0.45, offset=85)
img[6:11, 27:32] = 240.0; img = np.clip(img, 0, 255)
th, tw = tpl.shape
def score_map(fn):
m = np.full((H-th+1, W-tw+1), -np.inf)
for r in range(H-th+1):
for c in range(W-tw+1):
m[r,c] = fn(img[r:r+th, c:c+tw])
return m
print("METHOD 2 -- SUM OF SQUARED DIFFERENCES (negated, bigger=better):")
ssd = score_map(lambda p: -float(((p-tpl)**2).sum()))
for label, (r,c) in (("exact plus",(3,5)),("faint plus",(9,20)),("bright blob",(6,28))):
print("%20s %12.0f" % (label, ssd[r, c]))
bg_mask = np.ones_like(ssd, bool)
bg_mask[2:5, 4:7] = False; bg_mask[8:11, 19:22] = False; bg_mask[5:12, 26:33] = False
print("%20s %12.0f" % ("best background", ssd[bg_mask].max()))
print(" the exact copy wins and the blob is rejected. but the faint plus")
print(" at %.0f is WORSE than best background at %.0f."
% (ssd[9, 20], ssd[bg_mask].max()))
print(" SSD asks 'same pixels?' not 'same shape?'. a different exposure")
print(" produces the same shape at different values, and SSD rejects it.")
Output
The next block applies normalised cross-correlation, which strips out brightness and contrast and measures pure shape.
example_03.pyNumPy
import numpy as np
rng = np.random.default_rng(7)
H, W = 16, 34
img = rng.normal(70, 6, (H, W))
tpl = np.array([[20., 90., 20.], [90., 140., 90.], [20., 90., 20.]])
def paste(a, t, r, c, gain=1.0, offset=0.0):
a[r:r+t.shape[0], c:c+t.shape[1]] = t * gain + offset
paste(img, tpl, 3, 5)
paste(img, tpl, 9, 20, gain=0.45, offset=85)
img[6:11, 27:32] = 240.0; img = np.clip(img, 0, 255)
th, tw = tpl.shape
t0 = (tpl - tpl.mean()) / tpl.std()
def ncc(p):
s = p.std()
if s < 1e-9: return 0.0
return float((((p - p.mean()) / s) * t0).sum() / t0.size)
def score_map(fn):
m = np.full((H-th+1, W-tw+1), -np.inf)
for r in range(H-th+1):
for c in range(W-tw+1):
m[r,c] = fn(img[r:r+th, c:c+tw])
return m
nc = score_map(ncc)
print("METHOD 3 -- NORMALISED CROSS-CORRELATION:")
print("%20s %8s" % ("", "NCC"))
for label, (r,c) in (("exact plus",(3,5)),("faint plus",(9,20)),("bright blob",(6,28))):
print("%20s %8.3f" % (label, nc[r, c]))
print(" both plus signs score 1.000 EXACTLY. the blob scores 0: its")
print(" std is 0, so no shape to match.")
print(" NCC is invariant to a*x + b -- exactly what exposure and gain do.")
print(" the faint plus was 0.45x contrast + 85 offset, and NCC does not")
print(" care: subtracting the mean cancels +85, dividing by std cancels 0.45.")
bg_mask = np.ones_like(nc, bool)
bg_mask[2:5, 4:7] = False; bg_mask[8:11, 19:22] = False; bg_mask[5:12, 26:33] = False
bg_nc = nc[bg_mask]
print(" but %.1f%% of pure-background positions clear NCC > 0.75."
% (100.0 * (bg_nc > 0.75).mean()))
print(" a 3x3 template has only 9 numbers, so noise matches by chance.")
print(" NCC reliability comes from template SIZE: a 15x15 template has")
print(" 225 constraints and false positives become vanishingly rare.")
Output
The next block tests the limits no normalisation can fix: rotation, scale and perspective all break template matching entirely.
example_04.pyNumPy
import numpy as np
rng = np.random.default_rng(7)
H, W = 16, 34
img = rng.normal(70, 6, (H, W))
tpl = np.array([[20., 90., 20.], [90., 140., 90.], [20., 90., 20.]])
def paste(a, t, r, c, gain=1.0, offset=0.0):
a[r:r+t.shape[0], c:c+t.shape[1]] = t * gain + offset
paste(img, tpl, 3, 5)
img = np.clip(img, 0, 255)
patch = img[3:6, 5:8]
print("THE LIMITS. transform the template, re-score against the exact match:")
print("%-34s %8s" % ("template variant", "NCC"))
variants = [("as given", tpl), ("rotated 90 degrees", np.rot90(tpl)),
("transposed", tpl.T), ("inverted (dark on light)", 160 - tpl)]
for name, t in variants:
tt = (t - t.mean()) / (t.std() + 1e-9)
pp = (patch - patch.mean()) / (patch.std() + 1e-9)
print("%-34s %8.3f" % (name, float((pp * tt).sum() / tt.size)))
print(" rotation leaves 1.000 only because THIS template is symmetric.")
print(" inversion gives -1.000: NCC measures correlation, not similarity.")
print()
print("%-30s %s" % ("image change", "does NCC survive?"))
for k, v in (("brightness / exposure", "yes"),
("contrast / gain", "yes"),
("additive noise", "degrades smoothly"),
("rotation", "no"), ("scale", "no"),
("perspective", "no"), ("non-rigid deformation", "no")):
print("%-30s %s" % (k, v))
print()
print("that table is why the field moved on. template matching still owns")
print("jobs where appearance is fixed -- factory inspection, GUI automation.")
print("the moment the object can turn or move nearer, you need features")
print("that survive those changes: SIFT, or a convolutional network.")
Output
Check yourself
0 of 3
Answer without scrolling back up.
Why is normalised cross-correlation preferred over sum of squared differences?
Brightening the image by a constant grows every squared difference even where the shapes match perfectly. Subtracting means and dividing by standard deviations removes that.
Why is the peak in a response map a blob rather than a point?
That is why non-maximum suppression is needed: several nearby positions score highly and would otherwise be returned as separate detections of the same thing.
Which of these does template matching handle natively?
Position is the only unknown it searches over. Rotation and scale require searching those dimensions too, and deformation cannot be represented by a rigid patch at all.
Cheat sheet
Template Matching
Slide a template across an image, compute the match score at every position, and see why changes in scale and rotation break the method.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.