Counting it, one layer at a time
Drag the layers control above and watch the highlighted band widen as it moves down the diagram. Each row shows how many input pixels the layer above can be influenced by.
With stride 1 the rule is simple. A single unit sees 1 pixel of its own input. One k×k convolution makes that k. Another adds k - 1 more, because the window's centre already covers what the previous layer covered and each side extends by (k-1)/2. So:
r = 1
for each layer:
r = r + (k - 1)
Three 3×3 layers: 1 → 3 → 5 → 7. The growth is linear in depth, and that is slow. A network of twenty 3×3 layers at stride 1 has a receptive field of 41 pixels — less than a fifth of a 224-pixel image.
Stride changes the arithmetic
Set the stride control to 2 and the widening accelerates sharply.
The reason is that stride changes the *spacing* between the positions a layer looks at, measured in original pixels. Call that spacing the jump. At stride 1 the jump stays 1 forever. At stride 2 it doubles every layer, so a step of one unit in layer three corresponds to a step of four pixels in the input.
r = 1; jump = 1
for each layer:
r = r + (k - 1) * jump
jump = jump * stride
Now the additions themselves grow, and the receptive field expands geometrically rather than linearly. This is the real reason architectures downsample. Pooling and strided convolutions are usually explained as reducing computation, which they do — but the more important effect is that they are the only affordable way to get a deep unit to see the whole image.
Why 3×3 won everything
Two stacked 3×3 convolutions have the same 5×5 receptive field as one 5×5 convolution. Three stacked have the same 7×7 as one 7×7. So why not just use the big kernel?
Count the weights, per input/output channel pair:
| Arrangement | Receptive field | Weights |
|---|
| One 5×5 | 5×5 | 25 |
| Two 3×3 | 5×5 | 18 |
| One 7×7 | 7×7 | 49 |
| Three 3×3 | 7×7 | 27 |
The stack is cheaper, and it has a second advantage that matters more: there is a non-linearity between the layers. One 7×7 convolution is a single linear function of its 49 inputs. Three 3×3 convolutions with ReLUs between them is a composition of three linear functions separated by non-linearities, which can represent things a single linear map cannot.
VGG made this argument explicitly in 2014 and effectively ended large kernels in general-purpose vision architectures. Everything since is 3×3 stacks, with the occasional [1×1](one_by_one_convolutions.html) for channel work.
Effective versus theoretical
The number the formula gives is the *theoretical* receptive field: the set of pixels that could in principle affect the output.
The effective receptive field is smaller and softer. Contributions from the edge of the theoretical field pass through fewer paths than contributions from the centre, and the number of paths falls off roughly like a Gaussian. In practice a unit is strongly influenced by the middle of its field and barely influenced by the rim.
The practical consequence is that a network usually needs a theoretical receptive field noticeably larger than the objects it has to recognise, not merely equal to them. Dilated convolutions exist largely as a way to buy receptive field without buying downsampling, which matters when the output has to stay at full resolution — segmentation, most obviously.
Where it goes wrong
Assuming depth alone is enough. Twenty stride-1 layers of 3×3 still only see 41 pixels. Without downsampling or dilation, a deep network can be blind to anything large.
Forgetting the input resolution. A receptive field of 100 pixels covers half a 224-pixel image and a twentieth of a 2000-pixel one. Resizing the input silently changes what the architecture can see.
Reading detection failures as a data problem. If the model consistently misses large objects, check the receptive field before collecting more images.