One pixel deep in a network sees a patch of the original image. Work out how big that patch is, and why stacking beats widening.
Overview
The question
Take one number in the output of a convolutional network's third layer. Change a pixel in the input image. Does that number change?
For most pixels, no. A convolution is local. Each output depends only on a small window of its input. And stacking local ops gives an output that depends on a larger. But Still bounded, region of the first image. That region is the unit's receptive field, and it's the honest answer to "what can this feature possibly be detecting".
A unit with a 7×7 receptive field can't detect a face in a 224×224 photo. It has never seen a face. It has seen a 7×7 patch, and what it reacts to has to be visible in one.
Receptive Field
This module needs JavaScript: the
images are computed in the page rather than downloaded.
Worth knowing
The receptive field is how much of the original input a single unit can be influenced by.
With stride 1, each layer adds k − 1 to the receptive field. Growth is linear in depth.
With stride greater than 1, later layers add more, because each step covers more original pixels. Growth becomes geometric.
Three stacked 3×3 layers see 7×7 using 27 weights per channel pair. One 7×7 layer sees the same using 49.
Receptive Field
How much of the first image a single deep unit can really see. Why the answer shapes architecture.
Counting it, one layer at a time
Drag the layers control above and watch the highlighted band widen as it moves down the diagram. Each row shows how many input pixels the layer above can be influenced by.
With stride 1 the rule is simple. A single unit sees 1 pixel of its own input. One k×k convolution makes that k. Another adds k - 1 more, because the window's centre already covers what the previous layer covered and each side extends by (k-1)/2. So:
r = 1
for each layer:
r = r + (k - 1)
Three 3×3 layers: 1 → 3 → 5 → 7. The growth is linear in depth, and that is slow. A network of twenty 3×3 layers at stride 1 has a receptive field of 41 pixels — less than a fifth of a 224-pixel image.
Stride changes the arithmetic
Set the stride control to 2 and the widening accelerates sharply.
stride changes the *spacing* between the spots a layer looks at, measured in first pixels. Call that spacing the jump. At stride 1 the jump stays 1 forever. At stride 2 it doubles each layer. A step of one unit in layer three matches to a step of four pixels in the input.
r = 1; jump = 1
for each layer:
r = r + (k - 1) * jump
jump = jump * stride
Now the additions themselves grow, and the receptive field expands geometrically rather than linearly. This is the real reason architectures downsample. Pooling and strided convolutions are often shown as reducing work, which they do — but the more key effect is that they are the only affordable way to get a deep unit to see the whole image.
Why 3×3 won everything
Two stacked 3×3 convolutions have the same 5×5 receptive field as one 5×5 convolution. Three stacked have the same 7×7 as one 7×7. So why not just use the big kernel?
Count the weights, per input/output channel pair:
Arrangement
Receptive field
Weights
One 5×5
5×5
25
Two 3×3
5×5
18
One 7×7
7×7
49
Three 3×3
7×7
27
The stack is cheaper, and it has a second edge that matters more: there's a non-linearity between the layers. One 7×7 convolution is a single linear function of its 49 inputs. Three 3×3 convolutions with ReLUs between them is a composition of three linear functions separated by non-linearities, which can represent things a single linear map can't.
VGG made this arg by name in 2014 and in effect ended large kernels -purpose vision architectures. All since is 3×3 stacks, with the occasional 1×1 for channel work.
Effective versus theoretical
The number the formula gives is the *theoretical* receptive field: the set of pixels that could in principle affect the output.
The effective receptive field is smaller and softer. parts from the edge of the theoretical field pass through fewer paths than parts from the centre, and the number of paths falls off roughly like a Gaussian. In practice a unit is strongly influenced by the middle of its field and barely influenced by the rim.
The hands-on consequence is that a network often needs a theoretical receptive field noticeably larger than the objects it has to recognise, not merely equal to them. Dilated convolutions exist largely as a way to buy receptive field without buying downsampling. This Matters when the output has to stay at full detail — segmentation, most obviously.
Why two 3×3s beat one 5×5
Stacking small kernels reaches as far as one big kernel for fewer weights and more non-linearity, which is why 3×3 stacks won. The receptive field of a stride-1 stack grows by k-1 per layer, so:
arrangement
receptive field
weights (per channel pair)
one 5×5
5×5
25
two 3×3
5×5
18
one 7×7
7×7
49
three 3×3
7×7
27
Same reach, fewer weights — and, more importantly, a ReLU sits between the stacked layers, so three 3×3s are a composition of three non-linear functions where one 7×7 is a single linear map of its 49 inputs. That representational gap is the arg VGG made in 2014. It ended large kernels. The table also shows why depth alone isn't enough: at stride 1 the field grows only linearly, so twenty 3×3 layers still see just 41 pixels. Under a fifth of a 224-pixel image. Growing it affordably needs stride. This Makes the per-layer jump compound geometrically, or dilation. This Is how segmentation networks reach far while keeping full detail. And the *effective* field is smaller and softer than this theoretical one, falling off like a Gaussian from the centre, so a network often needs a theoretical field noticeably larger than the objects it must see.
Where it goes wrong with Receptive Field
given depth alone is enough. Twenty stride-1 layers of 3×3 still only see 41 pixels. Without downsampling or dilation, a deep network can be blind to any of it large.
Losing the input detail. A receptive field of 100 pixels covers half a 224-pixel image and a twentieth of a 2000-pixel one. Resizing the input silently changes what the architecture can see.
Reading spotting failures as a data problem. If the model always misses large objects, check the receptive field before collecting more images.
Questions people ask about Receptive Field
What is a receptive field? The region of the input image that can affect one output unit. It grows as you stack layers.
How do I calculate it? Layer by layer: add (kernel - 1) times the product of earlier strides. Stride multiplies how much each later layer adds.
Why do small kernels beat large ones? Two 3x3 layers cover 5x5 with 18 weights instead of 25, plus an extra non-linearity. Stacking small kernels is cheaper and more expressive.
What is the difference between theoretical and effective receptive field? The theoretical one is every pixel that could matter. The effective one is the smaller central region where influence is actually strong, roughly Gaussian in shape.
Why does the receptive field matter for my task? A unit cannot recognise anything bigger than it can see. Segmentation of large objects needs a receptive field that covers them.
How can I enlarge it without losing resolution? Use dilated convolutions, or add attention or global context blocks. Pooling and striding work too but shrink the feature map.
How much of the image one output pixel can see
A single neuron deep in a network reacts to a patch of the first image. finding that patch's size layer by layer shows most architectural choices -- kernel sizes, depth, dilation and pooling.
example_01.pyNumPy
import numpy as np
print("the recurrence, and it is the whole calculation:")
print(" RF_out = RF_in + (kernel - 1) * jump_in")
print(" jump_out = jump_in * stride")
print()
def trace(layers, label):
rf, jump = 1, 1
print(label)
print("%8s %10s %8s %8s %14s %12s"
% ("layer", "type", "kernel", "stride", "receptive field", "jump"))
print("%8s %10s %8s %8s %14d %12d" % ("input", "-", "-", "-", rf, jump))
for i, (kind, k, s) in enumerate(layers, 1):
rf = rf + (k - 1) * jump
jump = jump * s
print("%8d %10s %8d %8d %14d %12d" % (i, kind, k, s, rf, jump))
return rf
rf = trace([("conv", 3, 1)] * 8, "EIGHT 3x3 CONVOLUTIONS, stride 1:")
print(" each layer adds 2. after 8 layers a neuron sees %dx%d." % (rf, rf))
print(" growth is LINEAR -- to see 224 pixels you need %d layers."
% ((224 - 1) // 2))
print()
rf = trace([("conv", 3, 1), ("conv", 3, 1), ("pool", 2, 2),
("conv", 3, 1), ("conv", 3, 1), ("pool", 2, 2),
("conv", 3, 1), ("conv", 3, 1), ("pool", 2, 2)],
"VGG PATTERN -- two convs then a pool, three times:")
print(" %dx%d from 9 layers, against 17x17 from 8 without pooling."
% (rf, rf))
print(" stride-2 layers make growth geometric rather than arithmetic.")
Output
Stacking small kernels matches the receptive field of a large one for fewer parameters, and dilation grows the field without any downsampling.
example_02.pyNumPy
import numpy as np
print("TWO 3x3 CONVOLUTIONS vs ONE 5x5 -- same receptive field:")
for label, layers in (("one 5x5", [("conv", 5, 1)]),
("two 3x3", [("conv", 3, 1), ("conv", 3, 1)])):
rf_, jump_, params = 1, 1, 0
for _, k, s in layers:
rf_ += (k - 1) * jump_; jump_ *= s; params += k * k * 64 * 64
print(" %-10s field %dx%d, %s params (64->64 ch)"
% (label, rf_, rf_, "{:,}".format(params)))
print(" same field for %.0f%% of the parameters, and two non-linearities."
% (100 * (2 * 9) / 25))
print()
for label, layers in (("one 7x7", [("conv", 7, 1)]),
("three 3x3", [("conv", 3, 1)] * 3)):
rf_, jump_, params = 1, 1, 0
for _, k, s in layers:
rf_ += (k - 1) * jump_; jump_ *= s; params += k * k * 64 * 64
print(" %-10s field %dx%d, %s params"
% (label, rf_, rf_, "{:,}".format(params)))
print()
print("DILATION grows the field without adding parameters or losing")
print("resolution. a dilated kernel has gaps:")
for d in (1, 2, 4):
eff = 3 + (3 - 1) * (d - 1)
row = " ".join("x" if i % d == 0 else "." for i in range(eff))
print(" dilation %d: 3 weights spanning %d pixels %s" % (d, eff, row))
print()
rf_, jump_ = 1, 1
print(" a stack with doubling dilation, all stride 1:")
print("%8s %10s %16s" % ("layer", "dilation", "receptive field"))
for i, d in enumerate((1, 2, 4, 8, 16), 1):
eff = 3 + (3 - 1) * (d - 1)
rf_ += (eff - 1) * jump_
print("%8d %10d %16d" % (i, d, rf_))
print(" %d pixels from 5 layers and %d weights, no downsampling."
% (rf_, 5 * 9))
print(" that is why dilation is the standard tool for segmentation.")
Output
The effective receptive field is much smaller than the theoretical one because corner pixels contribute far less than centre pixels.
example_03.pyNumPy
import numpy as np
print("THE EFFECTIVE FIELD IS SMALLER THAN THE THEORETICAL ONE.")
print("not every input pixel contributes equally -- the centre is reached")
print("by many paths and the corners by one:")
w = np.array([[1.0]])
for layer in range(4):
k = np.ones((3, 3))
new = np.zeros((w.shape[0] + 2, w.shape[1] + 2))
for i in range(w.shape[0]):
for j in range(w.shape[1]):
new[i:i + 3, j:j + 3] += w[i, j] * k
w = new
w = w / w.max()
print(" contribution weight after 4 layers of 3x3 (field %dx%d):" % w.shape)
for r in w:
print(" " + "".join("%6.2f" % v for v in r))
centre = w[w.shape[0] // 2, w.shape[1] // 2]
corner = w[0, 0]
print(" centre weight %.2f; corner %.4f -- a factor of %.0f."
% (centre, corner, centre / corner))
print(" the field is gaussian-ish, not a flat square, so the EFFECTIVE")
print(" field is roughly the square root of the theoretical one.")
print(" that is why very deep networks still benefit from explicit")
print(" long-range mechanisms rather than depth alone.")
Output
Check yourself
0 of 3
Answer without scrolling back up.
With stride 1, how does the receptive field grow as layers are added?
Each layer's window centre already covers the previous field, so only the (k-1)/2 on each side is new. Three 3x3 layers give 1, 3, 5, 7.
Why are three stacked 3x3 convolutions usually preferred over one 7x7?
27 weights against 49 for the same 7x7 field, plus two ReLUs between the layers - so the stack can represent functions a single linear map cannot.
Why is the effective receptive field smaller than the theoretical one?
The number of paths from an input pixel to the output falls off roughly like a Gaussian away from the centre, so the rim of the theoretical field contributes very little.
Cheat sheet
Receptive Field
Trace one unit's receptive field back through conv and pooling layers and see how depth, stride and dilation decide what it can observe.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.