Reading a kernel
The identity kernel is the easiest to reason about:
0 0 0
0 1 0
0 0 0
Every neighbour is multiplied by zero, the centre pixel by one, so the output is the input unchanged. Now change the centre to 5 and set the four orthogonal neighbours to −1:
0 -1 0
-1 5 -1
0 -1 0
The centre pixel is amplified and its neighbours are subtracted from it. Where the neighbourhood is flat, the subtraction cancels the amplification exactly and nothing happens. Where the centre differs from its surroundings — at an edge — the difference is exaggerated. That is sharpening, and it explains why sharpening amplifies noise: a noisy pixel is, by definition, a pixel that differs from its neighbours.
The sum tells you what kind of filter it is
There are two families, and you can tell them apart by adding the weights up.
Weights summing to 1 preserve average brightness. A box blur of nine ones divided by nine is the arithmetic mean of the neighbourhood; a Gaussian blur weights the centre more heavily than the corners. Both keep the image at the same overall exposure because the total contribution of every pixel is unchanged.
Weights summing to 0 produce a difference image. Flat regions become zero, because a constant multiplied by weights that cancel gives nothing. Only where the image changes does anything survive. Since a result can be negative, these filters are usually displayed with 128 added, which is why edge images have that flat grey background.
| Kernel | Sum | What survives |
|---|
| Identity | 1 | everything, unchanged |
| Box blur | 1 (after division) | low-frequency detail |
| Sharpen | 1 | everything, with edges exaggerated |
| Sobel | 0 | edges in one direction |
| Laplacian | 0 | edges in every direction |
| Emboss | 1 | a directional shadow |
Sobel: why there are two of them
The Sobel operator comes as a pair, and the pair is one kernel rotated by ninety degrees:
Sobel X Sobel Y
-1 0 1 -1 -2 -1
-2 0 2 0 0 0
-1 0 1 1 2 1
Sobel X subtracts the column on the left from the column on the right. A vertical edge — where left and right differ — produces a large value. A horizontal edge produces nothing at all, because left and right are identical there. Sobel Y is the same argument with rows.
Run each on the visualisation above and the asymmetry is obvious: the vertical sides of the rectangle appear under Sobel X and vanish under Sobel Y. In practice the two are combined, usually as the square root of the sum of their squares, to get an edge strength that does not care about direction.
The middle row being −2, 0, 2 rather than −1, 0, 1 is a small piece of smoothing built in: the pixel in line with the centre is weighted more than the diagonals, so a single noisy pixel has less influence than it would in a pure difference.
What happens at the border
A kernel centred on a corner pixel has five of its nine cells hanging over the edge of the image. Something has to be decided, and the three common answers are to leave the border out (producing a smaller output), to treat the missing pixels as zero, or to repeat the nearest real pixel outwards. The visualisation here repeats the edge pixel, which is why the frame stays clean; zero-padding would darken it. The [padding module](padding_in_cnn.html) works through what that choice costs inside a network.
Why this matters for CNNs
Everything above is fixed: someone chose the nine numbers and wrote them down. A convolutional layer in a neural network performs the identical operation and differs in exactly one respect — the nine numbers are parameters, and gradient descent chooses them.
That is worth sitting with. When people say a CNN "learns to detect edges in the first layer", they mean that training drives some of the learned kernels towards something that looks very much like Sobel, because edges turn out to be a useful thing to measure. Nobody put Sobel there. The architecture only supplies the sliding window; the content of the window is discovered.
Where it goes wrong
Sharpening noise. Any kernel with negative weights amplifies pixel-to-pixel variation, and noise is pixel-to-pixel variation. Denoise first, sharpen second.
Forgetting to normalise. A blur kernel of nine ones, applied without dividing by nine, produces an image nine times too bright — which, after clipping at 255, is a white rectangle.
Assuming bigger is better. A 3×3 kernel sees a 3×3 region. To see further you either use a larger kernel, which costs quadratically more multiplications, or stack several small ones, which is what modern architectures do and why [receptive field](feature_map_in_cnn.html) is a concept worth having.