Slide a window across a feature map, take the max or the average at each position, and watch the output shrink.
Overview
What pooling does
A convolutional layer produces a feature map the same size as its input (when padded). Stack several layers and the maps stay large, the memory stays high, and every later operation is expensive. Pooling fixes that by summarising small regions into single values.
The idea is simple. Divide the feature map into non-overlapping windows, usually 2x2. Replace each window with one number. The output is half the width and half the height, one quarter the area. That quarter carries most of the signal, because nearby pixels in a feature map tend to carry similar information.
Two rules dominate: take the maximum of the window, or take the average. Both shrink the map. They differ in what survives the shrinking.
Max Pooling & Average Pooling
Pool size:
Worth knowing
Max pooling keeps the strongest activation in each window. If one value is high, it survives. The rest are discarded.
Average pooling takes the mean of the window. It preserves the overall energy but dilutes sharp peaks.
The standard setting is a 2x2 window with stride 2. Each pool layer quarters the spatial area and has zero learnable parameters.
Pooling gives translation invariance: move a feature by one pixel and the same window still captures it. The output does not change.
Pooling Layers in CNNs: Max and Average Pooling
After a convolution extracts features, pooling shrinks the map so the next layer sees more context with less compute.
Max pooling
Max pooling keeps the strongest activation in each window. If one pixel in a 2x2 block fired at 0.9 and the other three were near zero, the output is 0.9. The feature was detected somewhere in that region, and max pooling records that fact.
This makes max pooling tolerant to small shifts. Move a feature one pixel to the right and the same window still captures it. The output does not change. That property is called translation invariance, and it is one reason pooling layers exist.
Max pooling was the default for years. VGG-16 uses five 2x2 max-pool layers, each halving the spatial dimensions. LeNet-5, the architecture that started modern CNNs, also used a form of it.
Average pooling
Average pooling takes the mean of every value in the window. If the four pixels are 0.9, 0.1, 0.0, and 0.2, the output is 0.3. The strong activation is diluted by its quiet neighbours.
Average pooling preserves the overall energy of the map better than max pooling. When the goal is to keep a smooth summary rather than a sharp peak, it is the better choice. Global average pooling is the extreme case: one average per channel across the entire spatial extent, used at the end of modern networks to replace the flatten-and-dense head.
For intermediate layers, max pooling is more common than average pooling, because classification networks care more about whether a feature appeared than about its average strength.
How the window slides
The pooling window slides across the feature map exactly like a convolution filter, controlled by stride. The standard setting is stride equal to pool size. A 2x2 pool with stride 2 covers every pixel exactly once. A 3x3 pool with stride 3 does the same at a coarser scale.
You can set stride smaller than the pool size to get overlapping windows. A 3x3 pool with stride 2 overlaps by one column and one row at each step. AlexNet used this (3x3 pool, stride 2), and it gave a small accuracy gain over non-overlapping pooling at the cost of a larger output.
The output dimension follows the same formula as strided convolution:
output = floor((input - pool_size) / stride) + 1
A 32x32 map with a 2x2 pool and stride 2 gives 16x16. A 32x32 map with a 3x3 pool and stride 2 gives 15x15.
Why pooling matters
Shrinks the map. Each 2x2 pool layer quarters the spatial area. Two pool layers reduce it to a sixteenth. That cuts the compute and memory for every later layer by the same factor.
Builds translation invariance. Small shifts in the input move features within a pooling window but do not change the pooled output. This means the network does not need to learn the same feature at every possible pixel offset.
Grows the receptive field. After pooling, each unit in the next convolutional layer covers a larger region of the original image. A 3x3 filter after one 2x2 pool sees a 6x6 patch of the input. After two pools it sees a 12x12 patch. Pooling is one of the two ways a network gains a wide enough view to recognise whole objects (depth is the other).
Has no learnable parameters. The operation is fixed: take the max or the mean. No weights, no gradients for the pooling layer itself. This makes pooling cheap and impossible to overfit.
Where pooling goes wrong
Too aggressive pooling loses detail. A single 2x2 pool is fine. Three in a row reduce a 64x64 map to 8x8. Five reduce it to 2x2. At that point the network cannot distinguish objects that differ only by spatial arrangement. Tasks like segmentation and detection need that detail, which is why U-Net and feature pyramid networks add skip connections to recover the resolution pooling destroyed.
Pooling too early throws away spatial structure the early layers have not yet finished extracting. The standard pattern is convolution, activation, then pool, in that order.
Ignoring stride settings. If your stride is smaller than the pool size, you get overlapping windows and a larger output than you expected. If stride equals 1, pooling barely shrinks anything.
Pooling versus strided convolution
Modern architectures sometimes replace pooling with a strided convolution. A stride of 2 in a convolution also halves each dimension, and the filter weights are learned. The argument is that downsampling is a decision, and a learned decision beats a fixed rule.
The counter-argument is simplicity. Pooling has zero parameters, is fast, and gives a small amount of built-in shift tolerance. In practice both approaches work. ResNet uses strided convolutions; EfficientNet mixes both. The gap between the two is smaller than the design around them.
example_01.pyNumPy
import numpy as np
# 6x6 feature map with a bright patch in the top-left
fmap = np.array([
[9, 7, 3, 1, 0, 2],
[8, 6, 4, 0, 1, 1],
[2, 3, 5, 8, 7, 6],
[1, 0, 4, 9, 8, 5],
[0, 1, 2, 3, 6, 7],
[1, 0, 1, 2, 5, 9],
], dtype=float)
def pool2d(x, size, mode):
h, w = x.shape
oh = (h - size) // size + 1
ow = (w - size) // size + 1
out = np.zeros((oh, ow))
for i in range(oh):
for j in range(ow):
patch = x[i*size:(i+1)*size, j*size:(j+1)*size]
out[i, j] = patch.max() if mode == "max" else patch.mean()
return out
print("input 6x6:")
print(fmap.astype(int))
print()
for mode in ("max", "average"):
result = pool2d(fmap, 2, mode)
print("%s pool 2x2 -> %dx%d:" % (mode, result.shape[0], result.shape[1]))
print(np.round(result, 2))
print()
print("max pooling kept the 9 in the top-left;")
print("average pooling blended it down to %.1f." % pool2d(fmap, 2, "average")[0, 0])
Output
Questions people ask about Pooling Layers in CNNs
Is max pooling or average pooling better? Max pooling is more common for intermediate layers because it preserves the strongest activations. Average pooling works better when you want a smooth summary, especially at the end of a network (global average pooling).
Does pooling have any learnable parameters? No. The operation is fixed: take the max or the mean. No weights are trained during pooling.
What pool size should I use? 2x2 with stride 2 is the standard. 3x3 with stride 2 (overlapping) was used in AlexNet and gives marginally better results at higher compute cost. Sizes above 3 are rare.
Can I pool across channels? Standard pooling operates on each channel independently. The spatial dimensions shrink but the channel count stays the same. Channel-wise pooling is a separate idea (channel attention).
Why not just use stride 2 in the convolution instead? You can, and modern networks often do. Strided convolution learns what to keep. Pooling applies a fixed rule. Both downsample. The practical difference is small.
Does pooling help with overfitting? Indirectly. By shrinking the map, pooling reduces the number of values flowing into later layers, which means fewer parameters downstream and less room to memorise training data.
Recall check
0 of 3
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “What pooling does”?
What does this module say about “Max pooling”?
What does this module say about “Average pooling”?
Cheat sheet
Pooling Layers in CNNs
Slide a 2x2 or 3x3 window across a feature map, take the max or average at each position, and watch the output shrink.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.