Visualize why Convolutional Neural Networks are so efficient. See how a single set of shared weights is reused across the entire image, drastically reducing the number of parameters compared to a Dense layer.
Overview
What is Parameter Sharing?
In a traditional Fully Connected (Dense) neural network layer, every output node is connected to every input node with a unique weight (parameter). If you process an image, a specific pixel in the top-left corner has a completely different weight than a pixel in the bottom-right corner.
Parameter Sharing (or Weight Sharing) is the core innovation of Convolutional Neural Networks (CNNs). Instead of learning a separate weight for every pixel, the network learns a small set of weights (a filter/kernel) and shares that exact same set of weights across the entire image by sliding it around.
The Shared Parameters
These 4 parameters are shared and applied everywhere.
Total CNN Weights Used:4
Dense Equivalent:144
Current Calculation
Click "Slide Filter" to view shared operations.
Understanding Parameter Sharing
Why Convolutional layers are drastically more efficient than Fully Connected layers.
The Massive Efficiency Gain
Let's look at the math for the simple interactive above. We have a 4x4 input grid (16 pixels) and we are generating a 3x3 output grid (9 pixels).
Without Parameter Sharing (Dense Layer): Every one of the 9 output pixels needs a connection to all 16 input pixels. 16 inputs × 9 outputs = 144 unique parameters (weights).
With Parameter Sharing (CNN): We use a 2x2 filter. The filter only has 4 weights. We simply slide those same 4 weights to 9 different locations. Total parameters = 4 unique parameters (weights).
For a real-world 1080p image, a Dense layer would require trillions of parameters. A CNN might only require a few hundred. Parameter sharing makes image processing computationally possible.
One filter, used everywhere
A dense layer gives every input pixel its own weight to every output unit. A convolutional layer does something quite different: it learns one small filter and applies that same filter at every position in the image.
The arithmetic is where the argument lives. For a 224×224×3 input into 1,000 units:
Layer type
Parameters
Dense
150,528 × 1,000 = 150,528,000
Conv, 64 filters of 3×3×3
64 × (27 + 1) = 1,792
Four orders of magnitude, and the convolutional version performs better. That is not a trade-off; it is a strictly better design for images, and the reason is that it encodes a true fact about them.
The assumption behind it
Parameter sharing works because of translation invariance of statistics: a vertical edge in the top-left corner of a photograph looks exactly like a vertical edge in the bottom-right. The pattern is the same wherever it appears.
If that is true, learning a separate edge detector for every position is not just wasteful — it is actively harmful, because each position's detector sees only the examples that happened to fall there. Sharing means every position's examples train the same filter, so the filter sees the whole dataset rather than a slice of it.
Three consequences follow directly:
Fewer parameters. Less memory, less computation, and far less overfitting for a given amount of data.
Translation equivariance. Shift the input and the feature map shifts identically. A network does not need to see cats in every position to detect them everywhere.
Better data efficiency. Every occurrence of a pattern anywhere in any image contributes to learning that one filter.
Where the assumption fails
It is worth knowing the cases, because they explain several real architectures.
Position genuinely matters in some domains. In a face-alignment task where images are cropped and centred, eyes are always near the top and mouths near the bottom. Locally connected layers — local receptive fields without sharing — were used in early face recognition work for exactly this reason, and they need far more data.
Vision transformers need position told to them. Attention has no built-in notion of location at all, so transformers add explicit positional embeddings. They trade the convolutional prior for flexibility, which is why they need substantially more data to reach the same accuracy without pretraining.
Coordinate information is sometimes required. Tasks that must reason about absolute position benefit from appending coordinate channels to the input — the CoordConv trick — because pure convolution deliberately discards that information.
Sharing in other architectures
The same principle, applied along a different axis, appears throughout deep learning:
Recurrent networks share weights across time steps. One set of weights processes every position in a sequence, which is why an RNN handles variable-length input.
Transformers share the same feed-forward and attention weights across all token positions within a layer.
Graph neural networks share weights across nodes, so one learned message function applies to every vertex.
The unifying idea: wherever the same kind of structure repeats, share the parameters that process it. It is one of the most reliable ways to build a useful inductive bias into a model.
Guided Experiments with This Interactive
Watch the Colors:
Click the Slide Filter button. Watch the formula in the center column. The 4 weights are color-coded (Red, Blue, Yellow, Purple). Notice that as the green highlight box slides across different areas of the Input Image, the exact same colored weights are used over and over again to do the multiplication.
Translation Invariance:
Because the weights are shared, the filter is looking for the exact same pattern everywhere. If the weights represent a "vertical edge detector", it will find a vertical edge whether it's in the top-left of the image or the bottom-right. This property is called Translation Invariance—a cat is recognized as a cat regardless of where it is in the photo.
Change the Shared Rules:
Change the weights in the center kernel box to something extreme (like 10, 0, 0, 0). Click slide again. You'll see that changing the parameters once immediately alters how the network interprets the entire image, because the rule applies globally.
Are All Parameters Shared?
In a CNN, weights are shared spatially (across the width and height of the image), but they are not shared across different filters (channels).
A single Convolutional layer usually has many filters (e.g., 32 filters). Filter #1 (looking for edges) has its own 9 shared weights. Filter #2 (looking for colors) has a different set of 9 shared weights.
In one line
Parameter sharing reuses the same weights across different spatial locations.
It drastically reduces memory and computation requirements.
It forces the network to learn global patterns (Translation Invariance).
It prevents severe overfitting by heavily restricting the number of variables the model can memorize.
The +1 is the bias, one per filter. The crucial detail people miss is that each filter spans all input channels — a 3×3 filter on a 64-channel input has 3×3×64 = 576 weights, not 9.
Worked through for a small network on 32×32×3 input:
Layer
Shape
Parameters
Conv 3×3, 32 filters
32×32×32
(3×3×3 + 1) × 32 = 896
Conv 3×3, 64 filters
16×16×64
(3×3×32 + 1) × 64 = 18,496
Conv 3×3, 128 filters
8×8×128
(3×3×64 + 1) × 128 = 73,856
Global average pool
128
0
Dense to 10 classes
10
1,290
Total: about 94,000 parameters for a working CIFAR-scale classifier. The equivalent dense network on the same input would run into the tens of millions.
Notice where the parameters concentrate: in the deeper convolutions, because the channel count grows. And notice that global average pooling contributes nothing — which is exactly why it replaced the flatten-and-dense head that once held most of a network's weights.
Sharing does not mean shared computation
A common misunderstanding: parameter sharing reduces the number of weights, not the number of operations.
A 3×3 convolution with 64 filters on a 224×224 map performs 224 × 224 × 64 × 9 × in_channels multiply-adds. The filter is small; it is applied 50,000 times.
That is why convolutional networks are memory-light and compute-heavy, and why GPUs suit them: the same small set of weights is reused across thousands of parallel positions, which is exactly the access pattern GPU memory hierarchies are built for.
It also explains depthwise separable convolutions, used in MobileNet and EfficientNet. Splitting a convolution into a per-channel spatial filter followed by a 1×1 channel mixer cuts both parameters and operations by roughly a factor of the kernel area — around 8–9× for 3×3 — with a small accuracy cost. On phones, that trade is usually worth making.
One kernel, applied everywhere
A convolution reuses the same weights at every position. That single decision buys a parameter count independent of image size and a detector that works wherever the thing appears -- both measured here against a dense layer.
example_01.pyNumPy
import numpy as np
rng = np.random.default_rng(0)
print("THE PARAMETER COUNT. a dense layer needs one weight per input pixel")
print("per output unit. a convolution needs one per kernel position:")
print("%14s %20s %20s %14s"
% ("image", "dense (64 units)", "conv (64 filters, 3x3)", "ratio"))
for n in (28, 64, 224, 512):
dense = n * n * 64
conv = 3 * 3 * 1 * 64
print("%14s %20s %20s %13.0fx"
% ("%dx%d" % (n, n), "{:,}".format(dense), "{:,}".format(conv),
dense / conv))
print(" the conv column does not move. the same %d weights work on a"
% (3 * 3 * 64))
print(" 28x28 thumbnail and a 512x512 photograph, because the kernel is")
print(" applied at every position rather than wired to specific pixels.")
print()
print("TRANSLATION EQUIVARIANCE, which is the other half. train a detector")
print("to find a pattern at one location, then move the pattern:")
N, SIZE = 400, 12
pattern = np.array([[0, 1, 0], [1, 1, 1], [0, 1, 0]], float) * 5.0
def make(n, pos):
X = rng.normal(0, 0.5, (n, SIZE, SIZE))
y = rng.integers(0, 2, n)
r, c = pos
for i in np.where(y == 1)[0]:
X[i, r:r + 3, c:c + 3] += pattern
return X, y
Xtr, ytr = make(N, (2, 2))
# a dense model: flatten and fit a linear classifier
flat = Xtr.reshape(N, -1)
A = np.column_stack([flat, np.ones(N)])
w_dense = np.linalg.lstsq(A, ytr * 2.0 - 1, rcond=None)[0]
# a convolutional model: correlate with the pattern, take the max response
def conv_score(X):
k = pattern / np.linalg.norm(pattern)
out = np.zeros(len(X))
for i, img in enumerate(X):
best = -1e9
for r in range(SIZE - 2):
for c in range(SIZE - 2):
best = max(best, (img[r:r + 3, c:c + 3] * k).sum())
out[i] = best
return out
thresh = np.median(conv_score(Xtr))
print(" both models are fitted with the pattern always at (2,2).")
print("%18s %16s %16s" % ("pattern moves to", "dense accuracy", "conv accuracy"))
for pos in ((2, 2), (2, 5), (6, 6), (8, 3)):
Xte, yte = make(300, pos)
d = ((np.column_stack([Xte.reshape(300, -1), np.ones(300)]) @ w_dense) > 0)
c = conv_score(Xte) > thresh
print("%18s %16.4f %16.4f"
% (str(pos), (d == (yte == 1)).mean(), (c == (yte == 1)).mean()))
print(" the dense model learned 'look at these specific pixels'. move the")
print(" pattern and it is at chance. the convolution learned the pattern")
print(" itself, and finds it anywhere.")
print()
print("WHAT SHARING ASSUMES. it is a real assumption, and it is sometimes")
print("wrong:")
print(" 'a feature that is useful at one position is useful at every")
print(" position'")
print()
print(" true for -- natural photographs. an edge is an edge wherever it is.")
print(" false for -- aligned faces, where the eye region is always in the")
print(" same place and a position-specific detector would be")
print(" better. locally-connected layers exist for exactly")
print(" this and are rarely worth it.")
print(" false for -- spectrograms along the frequency axis, where 200Hz")
print(" and 4000Hz mean genuinely different things.")
print()
print("AND WHAT IT DOES NOT BUY. sharing gives EQUIVARIANCE -- move the")
print("input, the feature map moves with it -- not INVARIANCE:")
img = np.zeros((8, 8))
img[2, 1] = 10.0
# a peaked kernel, so the response has ONE maximum rather than a
# 3x3 plateau of tied values that argmax would break arbitrarily
k = np.array([[1.0, 2.0, 1.0], [2.0, 4.0, 2.0], [1.0, 2.0, 1.0]])
def conv(a):
out = np.zeros((6, 6))
for i in range(6):
for j in range(6):
out[i, j] = (a[i:i + 3, j:j + 3] * k).sum()
return out
SHIFT = 3
shifted = np.zeros((8, 8))
shifted[2, 1 + SHIFT] = 10.0
o1, o2 = conv(img), conv(shifted)
p1 = np.unravel_index(o1.argmax(), o1.shape)
p2 = np.unravel_index(o2.argmax(), o2.shape)
print(" input pixel at column 1 -> feature map peak at %s" % (p1,))
print(" input pixel at column %d -> feature map peak at %s" % (1 + SHIFT, p2))
print(" the peak moved %d columns for an input shift of %d. that is"
% (p2[1] - p1[1], SHIFT))
print(" equivariance: the map does not stay the same, it TRANSLATES.")
print(" the two maps are not equal:", np.allclose(o1, o2))
print()
print(" invariance -- the same ANSWER regardless of position -- comes from")
print(" pooling and from the global average at the end:")
print(" max over the whole map, original : %.1f" % o1.max())
print(" max over the whole map, shifted : %.1f" % o2.max())
print(" equal:", o1.max() == o2.max())
print(" that is the division of labour: convolution finds the thing")
print(" anywhere, pooling forgets where it was.")
Output
Questions people ask
Do all positions really need the same filter? For natural images, yes — and the empirical evidence is overwhelming. For centred, aligned images where position is meaningful, less so.
Does sharing hurt accuracy? It constrains the model, and that constraint is what prevents overfitting. On image data the constraint matches reality, so it helps.
How does backpropagation work with shared weights? The gradients from every position where the filter was applied are summed into one update. That is why a filter effectively learns from the whole image at once.
Why do deeper layers have more filters? Because spatial size shrinks and the number of distinct patterns worth detecting grows. Doubling channels as size halves keeps the computation per layer roughly constant.
Is a 1×1 convolution still parameter sharing? Yes — the same channel-mixing weights are applied at every spatial position.
Do transformers share parameters? Across positions within a layer, yes. Across layers, usually not, though some efficient variants tie layer weights deliberately.
Recap in one screen
One filter is applied at every position, so a convolutional layer has thousands of weights where a dense layer would have millions.
The justification is that image statistics are the same everywhere — an edge is an edge wherever it appears.
Sharing gives translation equivariance and much better data efficiency.
It reduces parameters, not operations: convolutions are compute-heavy and memory-light.
The same idea drives recurrent networks, transformers and graph networks along their own axes.
Recall check
0 of 3
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “What is Parameter Sharing”?
In a traditional Fully Connected (Dense) neural network layer, every output node is connected to every input node with a unique weight (parameter). If you process an image, a specific pixel in the top-left corner has a completely different weight than a pixel in the bottom-right corner.
What does this module say about “The Massive Efficiency Gain”?
Let's look at the math for the simple interactive above. We have a 4x4 input grid (16 pixels) and we are generating a 3x3 output grid (9 pixels).
What does this module say about “One filter, used everywhere”?
A dense layer gives every input pixel its own weight to every output unit. A convolutional layer does something quite different: it learns one small filter and applies that same filter at every position in the image.
Cheat sheet
Parameter Sharing in CNN
Visualize why Convolutional Neural Networks are so efficient. See how a single set of shared weights is reused across the entire image, drastically reducing the number of parameters compared to a Dense layer.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.