Visualize how adding a border of zeros (padding) preserves spatial dimensions and edge information during convolution.
Overview
The Shrinking Problem
In a standard convolutional layer, a kernel (like a 3x3 grid) slides across an image. However, the center of a 3x3 kernel cannot be placed on the absolute edge pixel of the input image without the rest of the kernel "falling off" the edge. Therefore, the kernel must start one pixel inward.
The result? The output feature map is smaller than the input image. If you pass an 8x8 image through a 3x3 kernel, you get a 6x6 output. If you stack 10 of these layers in a deep neural network, your image shrinks rapidly to nothing! This is called Valid convolution.
Padding Amount (P)
Output Size Formula
O = (W - K + 2P) / S + 1
O = (8 - 3 + 2(0)) / 1 + 1 = 6
Filter Kernel (K = 3)
Understanding Padding in CNNs
Why do we surround our images with zeros? Solving the shrinking problem and preserving edge information.
The Solution: Zero-Padding
To fix this, we artificially expand the original image by adding a border of zeros around it before applying the convolution. This is called Padding.
By adding a 1-pixel border of zeros (Padding = 1), our 8x8 image becomes 10x10. Now, when the 3x3 kernel slides over it, the output feature map is exactly 8x8. Because the input and output dimensions are the same, this specific padding amount is commonly referred to as "Same" Padding.
The edges lose out
Slide a 3×3 filter over a 5×5 image and it only fits in 3×3 positions. The output is smaller than the input, and the reason is at the borders: a pixel in the corner has no neighbours on two sides, so the filter cannot be centred there.
Two consequences follow, and the second is the one that matters.
The image shrinks. Each 3×3 convolution removes one pixel from every side. After ten such layers a 32×32 image is down to 12×12, and depth becomes limited by arithmetic rather than by design.
Edge pixels are undersampled. A pixel in the middle is covered by nine different filter positions; a corner pixel by one. Without padding, the network sees the borders far less often than the centre, and information there is systematically underused.
Padding fixes both by adding a border of extra values around the input before convolving.
The three modes
Mode
What it does
Output size
valid
No padding at all
Shrinks
same
Pad enough to preserve size
Unchanged (stride 1)
full
Pad so every input pixel gets full coverage
Grows
same is the everyday choice, and it is what almost every modern architecture uses in its convolution blocks. The required padding is (kernel − 1) / 2, so 1 for a 3×3 kernel and 2 for a 5×5 — which incidentally is why odd kernel sizes are preferred: even ones cannot be padded symmetrically.
valid remains useful when you deliberately want to reduce size without pooling, and in the final layers of some detection heads.
The output size formula covers all cases:
out = floor( (in + 2×pad − kernel) / stride ) + 1
32×32 with a 3×3 kernel, padding 1, stride 1 gives (32 + 2 − 3) + 1 = 32. With padding 0 it gives 30.
What to put in the padded border
The default is zeros, and it is nearly always fine. But zeros are a fabrication: the network sees a black frame that is not part of the image, and filters near the border learn from content that does not exist.
Alternatives, all available in modern frameworks:
Reflect mirrors the pixels near the edge. The border looks like plausible image content, which matters for tasks where edge artefacts are visible — denoising, super-resolution, style transfer.
Replicate repeats the outermost pixel outwards. Simple and smooth.
Circular wraps around, appropriate for genuinely periodic data such as panoramas or spectrograms along the frequency axis.
For classification, zero padding is the standard and the difference is negligible. For dense prediction tasks that output an image, reflection padding often visibly reduces edge artefacts.
Why keeping the size matters architecturally
Preserving spatial dimensions through the convolutions lets you separate two decisions that would otherwise be tangled: how deep the network is, and how much downsampling it does.
With same padding, convolution layers do feature extraction and pooling (or strided convolution) does the downsampling, deliberately and where you choose. Without it, every convolution silently shrinks the map, and a deep network runs out of pixels.
It also matters for architectures that must produce full-resolution output. A U-Net concatenates encoder feature maps with decoder ones of matching size — and same padding is what makes those sizes match cleanly. The original U-Net used valid padding and had to crop the skip connections, which is a real complication its successors avoid.
Residual connections have the same requirement: x + F(x) needs both terms to have the same shape.
Guided Experiments with This Interactive
Observe "Valid" Shrinkage (P=0):
Make sure Padding is set to P = 0. Look at the Output Feature Map on the right. Notice it is physically smaller (6x6) than the base 8x8 input. Click Slide Window and watch how the kernel box is restricted; it can never reach the very corners of your drawing.
Switch to "Same" Padding (P=1):
Select P = 1. Watch the Input canvas dynamically expand. A dark border containing "0"s appears around your shape. The Output canvas now perfectly matches your base 8x8 dimension! Click Slide Window again. Notice the kernel now beautifully sweeps over the entire base image, centering on the extreme edge pixels by borrowing space from the padded zeros.
The "Information Loss" Test:
Set padding to P = 0. Draw a bright dot in the absolute top-left corner of the input. Notice how weakly it registers in the output feature map (because the kernel only ever touches it with its bottom-right corner). Now switch to P = 1. The corner dot is now proudly represented in the output, because the kernel was able to center directly on top of it. Padding prevents information loss at the borders!
Check the Math:
Look at the formula box in the center. Play with the padding radio buttons and watch the math update in real time. For a kernel ($K$) of size 3, $P=1$ is required to maintain the dimension. If we used a 5x5 kernel, we would need $P=2$ to achieve "Same" padding.
The Universal Formula
The dimension of the output feature map ($O$) can always be calculated using:
O = [(W - K + 2P) / S] + 1
W = Input width/height
K = Kernel size
P = Padding amount
S = Stride (step size, default 1)
Why every convolution shrinks, and the three ways to stop it
A 3x3 convolution loses a pixel from every edge. That compounds with depth until there is nothing left, and the padding you choose to prevent it changes what the border pixels mean.
example_01.pyNumPy
import numpy as np
def out_size(n, k, p=0, s=1):
return (n + 2 * p - k) // s + 1
print("the output size formula, which is the whole topic:")
print(" out = floor((in + 2*padding - kernel) / stride) + 1")
print()
print("with no padding and stride 1, a 3x3 kernel costs one pixel per side:")
print("%10s %10s %12s %10s" % ("input", "kernel", "no padding", "lost"))
for n, k in ((32, 3), (32, 5), (32, 7), (224, 3)):
o = out_size(n, k)
print("%10d %10d %12d %10d" % (n, k, o, n - o))
print()
print("AND IT COMPOUNDS. a 32x32 image through repeated 3x3 convolutions:")
n = 32
print("%10s %14s" % ("layer", "size"))
for layer in range(20):
if layer in (0, 1, 5, 10, 15):
print("%10d %14s" % (layer, "%dx%d" % (n, n)))
n = out_size(n, 3)
if n <= 0:
print("%10d %14s" % (layer + 1, "nothing left"))
break
print(" a network cannot be deeper than the image is wide. VGG-16 has 13")
print(" convolutions; without padding a 32x32 input would be 6x6 by the")
print(" end, and a 224x224 one would lose %d pixels of context." % (2 * 13))
print()
img = np.arange(1, 26, dtype=float).reshape(5, 5)
print("THE THREE PADDING MODES, on a 5x5 image:")
print(" original:")
for r in img:
print(" %s" % "".join("%5.0f" % v for v in r))
print()
modes = [("zero", lambda a: np.pad(a, 1, mode="constant", constant_values=0)),
("edge (replicate)", lambda a: np.pad(a, 1, mode="edge")),
("reflect", lambda a: np.pad(a, 1, mode="reflect"))]
for name, fn in modes:
p = fn(img)
print(" %s padding -> %dx%d:" % (name, p.shape[0], p.shape[1]))
for r in p:
print(" %s" % "".join("%5.0f" % v for v in r))
print()
print("WHAT EACH ONE TELLS THE NETWORK about the border:")
print(" zero -- 'beyond the edge everything is black'. that is a lie,")
print(" and it creates an artificial dark edge the first layer")
print(" will happily learn to detect.")
print(" edge -- 'the border continues'. no invented content, but it")
print(" flattens the gradient at the boundary.")
print(" reflect -- 'the image mirrors'. keeps texture statistics, which is")
print(" why it is preferred for denoising and super-resolution.")
print()
def convolve(a, k, pad_mode):
kh = k.shape[0]
p = kh // 2
if pad_mode == "valid":
padded, oh, ow = a, a.shape[0] - kh + 1, a.shape[1] - kh + 1
else:
padded = np.pad(a, p, mode=pad_mode)
oh, ow = a.shape
out = np.zeros((oh, ow))
for i in range(oh):
for j in range(ow):
out[i, j] = (padded[i:i + kh, j:j + kh] * k).sum()
return out
flat = np.full((7, 7), 100.0)
edge_k = np.array([[0, 1, 0], [1, -4, 1], [0, 1, 0]], float)
print("THE ZERO-PADDING ARTEFACT, measured. run an edge detector over an")
print("image that is COMPLETELY FLAT -- there are no edges in it at all:")
for mode in ("constant", "edge", "reflect"):
out = convolve(flat, edge_k, mode)
border = np.concatenate([out[0], out[-1], out[:, 0], out[:, -1]])
print(" %-10s padding: interior response %+8.2f, border response up to %+8.2f"
% (mode, np.abs(out[1:-1, 1:-1]).max(), np.abs(border).max()))
print(" zero padding invents an edge around the entire image. the other")
print(" two do not. that phantom border is real enough that networks have")
print(" been shown to use it to infer absolute position.")
print()
print("'SAME' PADDING is the size you want, and the arithmetic that gets")
print("you there. for stride 1, p = (k-1)/2:")
print("%10s %14s %16s" % ("kernel", "padding needed", "output for 32x32"))
for k in (1, 3, 5, 7, 9):
p = (k - 1) // 2
print("%10d %14d %16s" % (k, p, "%dx%d" % (out_size(32, k, p), out_size(32, k, p))))
print(" which is why kernels are almost always ODD. an even kernel cannot")
print(" be centred, so 'same' padding has to be asymmetric:")
for k in (2, 4):
total = k - 1
print(" %dx%d kernel needs %d pixels of padding -- %d on one side and"
% (k, k, total, total // 2))
print(" %d on the other. no symmetric choice exists."
% (total - total // 2))
print()
print("and the cost, which is small but not zero:")
for n, k in ((224, 3), (224, 7)):
p = (k - 1) // 2
real = n * n
padded = (n + 2 * p) ** 2
print(" %dx%d image, %dx%d kernel: %d padded pixels, %.2f%% of the"
% (n, n, k, k, padded - real, 100 * (padded - real) / padded))
print(" total are invented")
Output
Summing up
Valid Padding (P=0): No padding. The image shrinks. Only pixels where the kernel fits entirely are computed.
Same Padding: Padding is added so the output size matches the input size. For K=3, P=1. For K=5, P=2.
Padding solves the shrinking problem in deep networks.
Padding ensures edge pixels are given equal processing weight as center pixels.
By manipulating padding, machine learning engineers control the spatial dimensions flowing through a neural network's architecture.
Padding is not free
Each padded border adds computation and memory. For a 3×3 kernel on a 224×224 image the overhead is about 2%, which is negligible. On small feature maps deep in the network — 7×7 — a padding of 1 adds nearly 30% more positions, which is not.
There is also a subtler cost: the network can learn to use the padded border as a cue for position. Because zero padding appears only at the edges, a CNN can infer roughly where in the image a patch sits — which is sometimes useful and sometimes a shortcut that will not generalise to different image sizes.
The practical advice remains simple: use same padding with odd kernels in the feature-extraction blocks, and change it only when you have a specific reason.
Common mistakes
Using an even kernel size with same padding. Symmetric padding is impossible, so frameworks pad asymmetrically and the output is subtly offset.
Forgetting padding shrinks the map. Ten unpadded 3×3 layers cost 20 pixels in each dimension, which is fatal on small inputs.
Mismatched shapes in a skip connection. The commonest cause is valid padding in one branch and same in the other.
Zero padding on data where zero is meaningful. In some medical and scientific imaging, zero is a real value, and a black border is a real signal. Reflect padding avoids the confusion.
Assuming padding preserves size regardless of stride.same preserves size only at stride 1; with stride 2 the output halves whatever the padding.
Questions people ask
Does padding add information? No — it adds space so the filter can reach the edges. The padded values carry no content.
Which padding mode should I use? Zeros for classification, reflect for tasks that produce images, circular for genuinely periodic data.
Why do frameworks default to valid? Historical convention in some, same in others. Always check rather than assume — PyTorch's Conv2d defaults to padding=0, which is valid.
Can padding be different on each side? Yes — frameworks accept per-side padding, which is how even kernels and odd input sizes are handled.
Does padding affect the receptive field? It affects which input regions each output position covers at the edges, but not the size of the receptive field itself.
Should I pad the pooling layers too? Usually not. Pooling is meant to reduce size, and padding it adds artificial zeros to the max or average.
Recap in one screen
Without padding, convolution shrinks the image and undersamples the border pixels.
same padding preserves size at stride 1 and is the everyday default; valid means none.
Odd kernels are preferred because they can be padded symmetrically.
Zeros are the standard filling; reflect and replicate suit image-to-image tasks.
Preserving size is what makes skip and residual connections line up.
Recall check
0 of 3
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “The Shrinking Problem”?
In a standard convolutional layer, a kernel (like a 3x3 grid) slides across an image. However, the center of a 3x3 kernel cannot be placed on the absolute edge pixel of the input image without the rest of the kernel "falling off" the edge. Therefore, the kernel must start one pixel inward.
What does this module say about “The Solution: Zero-Padding”?
To fix this, we artificially expand the original image by adding a border of zeros around it before applying the convolution. This is called Padding .
What does this module say about “The edges lose out”?
Slide a 3×3 filter over a 5×5 image and it only fits in 3×3 positions. The output is smaller than the input, and the reason is at the borders: a pixel in the corner has no neighbours on two sides, so the filter cannot be centred there.
Cheat sheet
Padding in CNN
In a standard convolutional layer, a kernel (like a 3x3 grid) slides across an image. However, the center of a 3x3 kernel cannot be placed on the absolute edge pixel of the input image without the rest of the kernel "falling off" the edge. Therefore, the kernel must start one pixel inward.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.