A flat region looks the same from everywhere, an edge looks the same along itself, and a corner does not. That difference is computable.
Overview
The question behind the detector
To match two photos of the same scene, you need points you can recognise in both. Which points are those?
The Harris insight is to ask what happens when you shift a small window slightly in each direction.
Flat region. Shift it anywhere and the contents barely change. There's nothing to lock onto: this window could be almost anywhere.
Edge. Shift it along the edge and nothing changes. Shift it across and all does. It's locatable in one direction and free to slide in the other — the aperture problem.
Corner. Shift it in any direction at all and the contents change. Its spot is pinned in both directions, and that is just what a landmark needs to be.
Harris Corners and Keypoints
This module needs JavaScript: the
images are computed in the page rather than downloaded.
Worth knowing
Shift a small window in any direction. On flat ground nothing changes; along an edge nothing changes; at a corner everything changes.
The structure tensor summarises how the gradient behaves in a window. Two large eigenvalues means a corner.
Harris computes det(M) − k·trace(M)², which is large when both eigenvalues are, without computing either.
Corners are stable under rotation and moderate lighting change, which is what makes them useful as landmarks.
Harris Corners and Keypoints
What makes a point in an image worth remembering. The determinant that measures it.
Making it arithmetic
The change under a small shift is captured by how the image grad behaves across the window. Collect the grads into the structure tensor:
M = [ sum(Ix*Ix) sum(Ix*Iy) ]
[ sum(Ix*Iy) sum(Iy*Iy) ]
Its two eigenvals say how strongly the intensity varies along the two principal directions:
λ1
λ2
Meaning
small
small
flat
large
small
edge
large
large
corner
finding eigenvals at each pixel would be slow. So Harris uses a combination that behaves the same way without them:
R = det(M) - k * trace(M)^2
= (l1*l2) - k*(l1+l2)^2
R is large and plus only when both eigenvals are large, strongly minus at an edge, and near zero on flat ground. The determinant times the eigenvals, so one small eigenval kills it. The trace term subtracts a fine that grows when the two are unbalanced.
k sets how strictly that balance is enforced — usually between 0.04 and 0.06. Drag it high in the visualisation and detections thin out to the sharpest corners. Drag it low and edges start to live.
Reading the visualisation
Highlighted pixels are those whose response clears the threshold.
Set the threshold low and whole edges light up. This Is the failure mode R is designed to avoid, appearing because the threshold is admitting weakly minus and near-zero responses. Raise it and only the rectangle's four corners and the triangle's points remain — the truly distinctive spots.
The window radius controls how much context each choice uses. A small window is sensitive to noise and finds many tiny corners. A large one is stable but blurs nearby corners into a single response and misses fine structure.
What it is and is not invariant to
Rotation: yes. The eigenvals of the structure tensor don't depend on the coord frame, so a rotated corner is still a corner with the same response. This is the property that makes Harris handy for matching.
Illumination: mostly. A constant brightness offset doesn't change grads at all. A contrast scaling times them. This Scales R, so a scaled threshold is needed rather than an abs one.
Scale: no. This is the major gap. A corner viewed from twice as far is a smaller corner, and a fixed window either sees only part of it or swamps it with context.
That gap is what SIFT handled, by searching over scales and spots and recording the scale at which each keypoint is most distinctive, then showing the local grad pattern so keypoints can be matched rather than merely found. ORB does a thing similar much faster using FAST corners and binary descriptors, which is why it turns up in real-time systems.
Where corners are used
Panorama stitching, structure from motion, visual SLAM, camera calibration and image registration all rest on finding the same physical points in a few images and solving for the transform between them. Spotting is the first step; template matching is the option that doesn't live rotation.
Flat, edge, corner, by the numbers
The whole finder reduces to the two eigenvals of the structure tensor, and Harris's shortcut R = det(M) - k·trace(M)² reads them without solving for them. With k = 0.04:
region
λ₁
λ₂
R
flat
0.1
0.1
+0.008 (near zero)
edge
5.0
0.1
−0.54 (strongly negative)
corner
5.0
4.0
+16.76 (large positive)
The determinant times the eigenvals. So One small eigenval drags it toward zero. The trace term subtracts a fine that grows when the two are unbalanced. The result is large and plus only when *both* eigenvals are large — the corner case, where the intensity changes in two free directions and the window can be pinned down. An edge changes in one direction only, so it comes out strongly minus and is rejected. A flat region changes in neither and sits near zero. That is why R separates corners from edges without ever finding an eigenval, and why k (0.04-0.06) tunes how strictly the balance is enforced. The eigenvals are rotation-rule, so a rotated corner scores the same — but not scale-rule, which is the gap SIFT was built to close.
What Harris is and isn't rule to decides where it can be used. Rotation: yes — the eigenvals don't depend on the coord frame, so a rotated corner scores the same, which is what makes it usable for matching two views. Illumination: mostly — a brightness offset leaves grads untouched, but a contrast scaling times R, which is why the threshold must be scaled to the image max rather than abs. Scale: no. A corner seen from twice as far is a smaller corner a fixed window can't capture, and that gap is just what SIFT closed by searching over scales and recording the one at which each keypoint is most distinctive. Corners feed panorama stitching, structure-from-motion, visual SLAM and calibration, all of which rest on finding the same physical point in a few images.
Where it goes wrong with Harris Corners and Keypoints
An abs threshold. Response scales with contrast. So A threshold tuned on one image fails on a darker one. Threshold scaled to the max, as this page does.
No non-max suppression. A corner makes a cluster of high responses, not one pixel.
Expecting scale steadiness. If the camera distance varies, use a scale-rule finder.
Corners on a blurred image. Blur destroys the grads the whole method depends on, so denoise gently or not at all before detecting.
Questions people ask about Harris Corners and Keypoints
What is a corner, to a computer? A point where the image changes strongly in every direction when you shift a small window. Along an edge it changes in only one direction, and on flat areas it barely changes.
What is the Harris response? R = det(M) - k * trace(M) squared, where M is the matrix of summed gradient products in a window. Large positive R means corner, large negative means edge, small means flat.
What value should k take? Typically 0.04 to 0.06. A smaller k admits more corner-like points and a larger k makes the detector stricter.
Is Harris invariant to rotation and scale? Rotation, yes, since the eigenvalues of M do not change. Scale, no, so a corner at one size can look like an edge at another.
Why do I get clusters of detections around one corner? Neighbouring pixels all have high responses. Apply non-maximum suppression so only the local peak remains.
How does Shi-Tomasi differ? It scores a point by the smaller eigenvalue of M instead. It is simple, tends to pick better points to track and is what goodFeaturesToTrack uses.
Two eigenvalues that tell an edge from a corner
A corner is a place where the image changes in two free directions. That sentence turns right into linear algebra. This builds the structure tensor, reads its eigenvals. Shows why Harris uses a determinant-and-trace shortcut instead of finding them.
example_01.pyNumPy
import numpy as np
rng = np.random.default_rng(6)
H, W = 14, 20
img = np.full((H, W), 50.0)
img[3:10, 3:9] = 200.0; img[:, 14:16] = 200.0
img += rng.normal(0, 2.0, (H, W))
ramp = " .:-=+*#%@"
def show(a, label, hi=None):
hi = a.max() if hi is None else hi
print(" %s" % label)
for row in a:
print(" " + "".join(ramp[min(9, int(9*max(0.,v)/(hi+1e-9)))] for v in row))
show(img, "the image", 255)
print(" a solid square (4 corners) and a vertical strip (edges only).")
print()
KX = np.array([[-1., 0, 1], [-2, 0, 2], [-1, 0, 1]]) / 8.0
p = np.pad(img, 1, mode="edge")
gx = np.array([[float((p[i:i+3, j:j+3]*KX).sum()) for j in range(W)] for i in range(H)])
gy = np.array([[float((p[i:i+3, j:j+3]*KX.T).sum()) for j in range(W)] for i in range(H)])
print("THE STRUCTURE TENSOR at each pixel is a 2x2 matrix:")
print(" M = sum over window of [[gx*gx, gx*gy], [gx*gy, gy*gy]]")
print(" its two eigenvalues tell you: flat, edge, or corner.")
def tensor(r, c, win=3):
h = win // 2
r0, r1 = max(0, r-h), min(H, r+h+1)
c0, c1 = max(0, c-h), min(W, c+h+1)
a, b = gx[r0:r1, c0:c1].reshape(-1), gy[r0:r1, c0:c1].reshape(-1)
return np.array([[float(a@a), float(a@b)], [float(a@b), float(b@b)]])
probes = [("flat interior", 6, 5), ("flat background", 12, 10),
("vertical edge", 6, 3), ("horizontal edge", 3, 6),
("strip's edge", 6, 14), ("top-left corner", 3, 3),
("bottom-right corner", 9, 8)]
print("%-26s %8s %8s %10s" % ("location", "l1", "l2", "type"))
for label, r, c in probes:
ev = np.sort(np.linalg.eigvalsh(tensor(r, c)))[::-1]
if ev[0] < 200: kind = "flat"
elif ev[1] < ev[0] * 0.15: kind = "edge"
else: kind = "CORNER"
print("%-26s %8.0f %8.0f %10s" % (label, ev[0], ev[1], kind))
print(" both small = flat; one large = edge; both large = CORNER.")
Output
The next block shows Harris's shortcut: instead of computing eigenvalues, score each pixel with det(M) - k * trace(M)^2, which separates the three cases by sign alone.
example_02.pyNumPy
import numpy as np
rng = np.random.default_rng(6)
H, W = 14, 20
img = np.full((H, W), 50.0)
img[3:10, 3:9] = 200.0; img[:, 14:16] = 200.0
img += rng.normal(0, 2.0, (H, W))
KX = np.array([[-1., 0, 1], [-2, 0, 2], [-1, 0, 1]]) / 8.0
p = np.pad(img, 1, mode="edge")
gx = np.array([[float((p[i:i+3, j:j+3]*KX).sum()) for j in range(W)] for i in range(H)])
gy = np.array([[float((p[i:i+3, j:j+3]*KX.T).sum()) for j in range(W)] for i in range(H)])
def tensor(r, c, win=3):
h = win // 2
r0, r1 = max(0, r-h), min(H, r+h+1)
c0, c1 = max(0, c-h), min(W, c+h+1)
a, b = gx[r0:r1, c0:c1].reshape(-1), gy[r0:r1, c0:c1].reshape(-1)
return np.array([[float(a@a), float(a@b)], [float(a@b), float(b@b)]])
K = 0.05
print("HARRIS'S SHORTCUT: R = det(M) - k * trace(M)^2")
print(" det(M) = l1*l2, trace(M) = l1+l2. no square roots needed.")
print("%-26s %12s %12s %12s" % ("location", "l1*l2", "det(M)", "R"))
probes = [("vertical edge", 6, 3), ("horizontal edge", 3, 6),
("strip's edge", 6, 14), ("top-left corner", 3, 3),
("bottom-right corner", 9, 8)]
for label, r, c in probes:
M = tensor(r, c)
ev = np.sort(np.linalg.eigvalsh(M))[::-1]
R = np.linalg.det(M) - K * np.trace(M)**2
print("%-26s %12.4g %12.4g %12.4g" % (label, ev[0]*ev[1], np.linalg.det(M), R))
print(" det matches the eigenvalue product exactly.")
print(" R's sign: edge -> negative, corner -> positive, flat -> near 0.")
print(" one number, three cases, distinguished by SIGN.")
Output
The next block computes the full Harris response map, thresholds it and extracts local maxima to find the four corners of the square.
example_03.pyNumPy
import numpy as np
rng = np.random.default_rng(6)
H, W = 14, 20
img = np.full((H, W), 50.0)
img[3:10, 3:9] = 200.0; img[:, 14:16] = 200.0
img += rng.normal(0, 2.0, (H, W))
KX = np.array([[-1., 0, 1], [-2, 0, 2], [-1, 0, 1]]) / 8.0
p = np.pad(img, 1, mode="edge")
gx = np.array([[float((p[i:i+3, j:j+3]*KX).sum()) for j in range(W)] for i in range(H)])
gy = np.array([[float((p[i:i+3, j:j+3]*KX.T).sum()) for j in range(W)] for i in range(H)])
def tensor(r, c, win=3):
h = win // 2
r0, r1 = max(0, r-h), min(H, r+h+1)
c0, c1 = max(0, c-h), min(W, c+h+1)
a, b = gx[r0:r1, c0:c1].reshape(-1), gy[r0:r1, c0:c1].reshape(-1)
return np.array([[float(a@a), float(a@b)], [float(a@b), float(b@b)]])
K = 0.05
Rmap = np.zeros((H, W))
for r in range(H):
for c in range(W):
M = tensor(r, c)
Rmap[r, c] = np.linalg.det(M) - K * np.trace(M)**2
ramp = " .:-=+*#%@"
print("THE RESPONSE MAP (positive values only):")
for row in np.maximum(Rmap, 0):
print(" " + "".join(ramp[min(9, int(9*max(0.,v)/(Rmap.max()+1e-9)))] for v in row))
print(" four corners light up; edges and the strip do not.")
thr = Rmap.max() * 0.25; peaks = []
for r in range(1, H-1):
for c in range(1, W-1):
if Rmap[r,c] > thr and Rmap[r,c] >= Rmap[r-1:r+2, c-1:c+2].max():
peaks.append((r, c, Rmap[r, c]))
print(" %d local maxima above 25%% of max:" % len(peaks))
for r, c, v in peaks:
print(" (%2d,%2d) R = %.4g" % (r, c, v))
Output
The next block summarises what the Harris response is and is not invariant to, and why SIFT was invented to fix the scale gap.
example_04.pyNumPy
print("WHAT HARRIS IS AND IS NOT INVARIANT TO:")
print("%-26s %s" % ("change", "does R survive?"))
for k, v in (("translation", "yes -- computed locally"),
("rotation", "yes -- det and trace are rotation invariant"),
("brightness (+b)", "yes -- gradients ignore a constant"),
("contrast (*a)", "NO -- R scales as a^4"),
("scale (zoom)", "NO -- a corner becomes a curve")):
print("%-26s %s" % (k, v))
print(" contrast: R is thresholded relative to the image's own maximum.")
print(" scale: SIFT was invented to fix this -- search over scales as")
print(" well as positions. Harris finds WHERE; scale-space finds where")
print(" AND how big.")
Output
Check yourself
0 of 3
Answer without scrolling back up.
What distinguishes a corner from an edge in the structure tensor?
An edge has one large eigenvalue and one small one, so the window can slide along it freely. A corner is pinned in both directions.
Why does Harris compute det(M) - k*trace(M)^2 instead of the eigenvalues?
The determinant is the product of the eigenvalues, so one small one kills it, and the trace term penalises imbalance. k sets how strictly.
What is Harris NOT invariant to?
A corner viewed from twice the distance is a smaller corner, and a fixed window either sees part of it or swamps it. SIFT addressed this by searching over scales.
Cheat sheet
Harris Corners and Keypoints
Compute the Harris corner response at every pixel, threshold the result, and see why corners survive rotation and small shifts.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.