Visualize how adding a border of zeros (padding) preserves spatial dimensions and edge information during convolution.
In a standard convolutional layer, a kernel (like a 3x3 grid) slides across an pic. However, the center of a 3x3 kernel cannot be placed on the absolute edge pixel of the input pic without the rest of the kernel "falling off" the edge. Therefore, the kernel must start one pixel inward.
The result? The output feature map is less than the input pic. If you pass an 8x8 pic through a 3x3 kernel, you get a 6x6 output. If you stack 10 of these layers in a deep neural network, your pic shrinks rapidly to nothing! This is called Valid convolution. Same (or "half") padding adds a border of zeros so the output keeps the input's spatial dimensions — for a 3x3 kernel that means one pixel of padding on each side. Full padding adds enough zeros that each pixel is visited equally, giving an output *more* than the input. The formula is output = (input + 2*pad - kernel) / stride + 1. And choosing pad = (kernel - 1) / 2 with stride 1 gives "same" just. Padding also preserves edge information: without it, corner pixels add to just one output item while centre pixels add to nine. So the network under-represents the borders of each feature map. This page shows all three modes side by side, lets you change kernel size and stride. And plots the output dimensions and per-pixel contribution counts.
Visualize how adding a border of zeros (padding) preserves spatial dimensions and edge information during convolution.
Why do we surround our images with zeros? Solving the shrinking problem and preserving edge information.
In a standard convolutional layer, a kernel (like a 3x3 grid) slides across an pic. However, the center of a 3x3 kernel cannot be placed on the absolute edge pixel of the input pic without the rest of the kernel "falling off" the edge. Therefore, the kernel must start one pixel inward.
The result? The output feature map is less than the input pic. If you pass an 8x8 pic through a 3x3 kernel, you get a 6x6 output. If you stack 10 of these layers in a deep neural network, your pic shrinks rapidly to nothing! This is called Valid convolution. Same (or "half") padding adds a border of zeros so the output keeps the input's spatial dimensions — for a 3x3 kernel that means one pixel of padding on each side. Full padding adds enough zeros that each pixel is visited equally, giving an output *more* than the input. The formula is output = (input + 2*pad - kernel) / stride + 1. And choosing pad = (kernel - 1) / 2 with stride 1 gives "same" just. Padding also preserves edge information: without it, corner pixels add to just one output item while centre pixels add to nine. So the network under-represents the borders of each feature map. This page shows all three modes side by side, lets you change kernel size and stride. And plots the output dimensions and per-pixel contribution counts.
To fix this, we artificially expand the first pic by adding a border of zeros around it before applying the convolution. This is called Padding.
By adding a 1-pixel border of zeros (Padding = 1), our 8x8 pic turns 10x10. Now, when the 3x3 kernel slides over it, the output feature map is just 8x8. Since the input and output dimensions are the same, this set padding amount is often referred to as "Same" Padding.
Slide a 3×3 filter over a 5×5 pic and it just fits in 3×3 positions. The output is less than the input. And the reason is at the borders: a pixel in the corner has no neighbours on two sides. So the filter cannot be centred there.
Two consequences follow, and the second is the one that matters.
The pic shrinks. Each 3×3 convolution removes one pixel from each side. After ten such layers a 32×32 pic is down to 12×12. And depth turns limited by arithmetic rather than by design.
Edge pixels are undersampled. A pixel in the middle is covered by nine other filter positions. A corner pixel by one. Without padding, the network sees the borders far less often than the centre. And information there is systematically underused.
Padding fixes both by adding a border of extra vals around the input before convolving.
| Mode | What it does | Output size |
|---|---|---|
valid | No padding at all | Shrinks |
same | Pad enough to preserve size | Unchanged (stride 1) |
full | Pad so every input pixel gets full coverage | Grows |
same is the everyday choice. And it is what almost each modern design uses in its convolution blocks. The needed padding is (kernel − 1) / 2. So 1 for a 3×3 kernel and 2 for a 5×5 — which incidentally is why odd kernel sizes are preferred: still ones cannot be padded symmetrically.
valid stays useful when you deliberately want to reduce size without pooling. And in the final layers of some detection heads.
The output size formula covers all cases:
out = floor( (in + 2×pad − kernel) / stride ) + 1
32×32 with a 3×3 kernel, padding 1, stride 1 gives (32 + 2 − 3) + 1 = 32. With padding 0 it gives 30.
The default is zeros, and it is nearly always fine. But zeros are a fabrication: the network sees a black frame that is not part of the pic. And filters near the border learn from content that does not exist.
Options, all on hand in modern frameworks:
For classification, zero padding is the standard and the gap is negligible. For dense prediction tasks that output an pic, reflection padding often visibly reduces edge artefacts.
Preserving spatial dimensions through the convolutions lets you split two decisions that would otherwise be tangled: how deep the network is. And how much downsampling it does.
With same padding, convolution layers do feature extraction and pooling (or strided convolution) does the downsampling, deliberately and where you choose. Without it, each convolution silently shrinks the map. And a deep network runs out of pixels.
It also matters for designs that must give full-resolution output. A U-Net concatenates encoder feature maps with decoder ones of matching size — and same padding is what makes those sizes match cleanly. The first U-Net used valid padding and had to crop the skip connections. It is a real complication its successors avoid.
Residual connections have the same requirement: x + F(x) needs both terms to have the same shape.
Make sure Padding is set to P = 0. Look at the Output Feature Map on the right. Notice it is physically less (6x6) than the base 8x8 input. Click Slide Window and watch how the kernel box is restricted. It can never reach the very corners of your drawing.
Select P = 1. Watch the Input canvas dynamically expand. A dark border holding "0"s appears around your shape. The Output canvas now perfectly matches your base 8x8 dimension! Click Slide Window again. Notice the kernel now beautifully sweeps over the whole base pic, centering on the extreme edge pixels by borrowing space from the padded zeros.
Set padding to P = 0. Draw a bright dot in the absolute top-left corner of the input. Notice how weakly it registers in the output feature map (since the kernel just ever touches it with its bottom-right corner). Now switch to P = 1. The corner dot is now proudly shown in the output, since the kernel was able to center right on top of it. Padding stops information loss at the borders!
Look at the formula box in the center. Play with the padding radio buttons and watch the math update in real time. For a kernel ($K$) of size 3, $P=1$ is needed to keep the dimension. If we used a 5x5 kernel, we would need $P=2$ to reach "Same" padding.
The dimension of the output feature map ($O$) can always be calculated with:
O = [(W - K + 2P) / S] + 1
A 3x3 convolution loses a pixel from each edge. That compounds with depth until there is nothing left. And the padding you choose to stop it changes what the border pixels mean.
The three padding modes fill that border differently. And what goes into it changes the signal at the edges.
Zero padding can create a phantom edge that is not in the real pic. It a simple experiment measures right.
"Same" padding is the size that preserves the spatial dimensions. And the arithmetic to get there explains why odd kernels are the standard.
By manipulating padding, machine learning engineers control the spatial dimensions flowing through a neural network's design.
Each padded border adds computation and memory. For a 3×3 kernel on a 224×224 pic the cost is about 2%, which is negligible. On small feature maps deep in the network — 7×7 — a padding of 1 adds nearly 30% more positions, which is not.
There is also a subtler cost: the network can learn to use the padded border as a cue for position. Since zero padding appears just at the edges, a CNN can infer roughly where in the pic a patch sits — which is sometimes useful and sometimes a shortcut that will not generalise to other pic sizes.
The real advice stays simple: use same padding with odd kernels in the feature-extraction blocks. And change it just when you have a set reason.
same padding. Symmetric padding is impossible. So frameworks pad asymmetrically and the output is subtly offset.valid padding in one branch and same in the other.same preserves size just at stride 1. With stride 2 the output halves whatever the padding.Does padding add information? No — it adds space so the filter can reach the edges. The padded vals carry no content.
Which padding mode should I use? Zeros for classification, reflect for tasks that give pics, circular for genuinely periodic data.
Why do frameworks default to valid? Historical convention in some, same in others. Always check rather than assume — PyTorch's Conv2d defaults to padding=0, which is valid.
Can padding be other on each side? Yes — frameworks accept per-side padding. It is how still kernels and odd input sizes are handled.
Does padding affect the receptive field? It affects which input regions each output position covers at the edges. But not the size of the receptive field it.
Should I pad the pooling layers too? Often not. Pooling is meant to reduce size. And padding it adds artificial zeros to the max or average.
same padding preserves size at stride 1 and is the everyday default. valid means none.Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “The Shrinking Problem”?
What does this module say about “The Solution: Zero-Padding”?
What does this module say about “The edges lose out”?
Add zero-padding around an image and watch the output keep its spatial dimensions instead of shrinking with every convolution layer.