Visualize why Convolutional Neural Networks are so efficient. See how a single set of shared weights is reused across the entire image, drastically reducing the number of parameters compared to a Dense layer.
The Shared Parameters
These 4 parameters are shared and applied everywhere.
Total CNN Weights Used:4
Dense Equivalent:144
Current Calculation
Click "Slide Filter" to view shared operations.
Overview
What is Param Sharing?
In a classic Fully Connected (Dense) neural network layer, each output node is connected to each input node with a unique weight (param). If you process an pic, a set pixel in the top-left corner has a fully other weight than a pixel in the bottom-right corner.
Param Sharing (or Weight Sharing) is the core innovation of Convolutional Neural Networks (CNNs). Instead of learning a split weight for each pixel, the network learns a small set of weights (a filter/kernel) and shares that exact same set of weights across the whole pic by sliding it around. The assumption behind sharing is **translation equivariance**: a feature that matters in the top-left corner matters equally in the bottom-right. So the detector should be the same. A 3×3 filter has 9 params no matter whether the pic is 32×32 or 4096×4096. The same 9 weights are reused at each spatial position. A fully connected layer connecting the same input to the same output would need one weight per input-output pair. It for still a small pic is orders of magnitude more. This page shows the param count for a convolutional layer and the equivalent dense layer side by side, shows how the same filter detects the same edge at each position. And covers the cases where sharing is deliberately broken (locally connected layers in face recognition).
Visualize why Convolutional Neural Networks are so efficient.
Why Convolutional layers are drastically more efficient than Fully Connected layers.
What is Param Sharing?
In a classic Fully Connected (Dense) neural network layer, each output node is connected to each input node with a unique weight (param). If you process an pic, a set pixel in the top-left corner has a fully other weight than a pixel in the bottom-right corner.
Param Sharing (or Weight Sharing) is the core innovation of Convolutional Neural Networks (CNNs). Instead of learning a split weight for each pixel, the network learns a small set of weights (a filter/kernel) and shares that exact same set of weights across the whole pic by sliding it around. The assumption behind sharing is **translation equivariance**: a feature that matters in the top-left corner matters equally in the bottom-right. So the detector should be the same. A 3×3 filter has 9 params no matter whether the pic is 32×32 or 4096×4096. The same 9 weights are reused at each spatial position. A fully connected layer connecting the same input to the same output would need one weight per input-output pair. It for still a small pic is orders of magnitude more. This page shows the param count for a convolutional layer and the equivalent dense layer side by side, shows how the same filter detects the same edge at each position. And covers the cases where sharing is deliberately broken (locally connected layers in face recognition).
The Massive Efficiency Gain
Let's look at the math for the simple interactive above. We have a 4x4 input grid (16 pixels) and we are making a 3x3 output grid (9 pixels).
Without Param Sharing (Dense Layer): Each one of the 9 output pixels needs a connection to all 16 input pixels. 16 inputs × 9 outputs = 144 unique params (weights).
With Param Sharing (CNN): We use a 2x2 filter. The filter just has 4 weights. We simply slide those same 4 weights to 9 other locations. Total params = 4 unique params (weights).
For a real-world 1080p pic, a Dense layer would need trillions of params. A CNN might just need a few hundred. Param sharing makes pic handling computationally possible.
One filter, used all over
A dense layer gives each input pixel its own weight to each output unit. A convolutional layer does something quite other: it learns one small filter and applies that same filter at each position in the pic.
The arithmetic is where the argument lives. For a 224×224×3 input into 1,000 units:
Layer type
Parameters
Dense
150,528 × 1,000 = 150,528,000
Conv, 64 filters of 3×3×3
64 × (27 + 1) = 1,792
Four orders of magnitude, and the convolutional version does better. That is not a trade-off. It is a strictly better design for pics. And the reason is that it encodes a true fact about them.
The assumption behind it
Param sharing works since of translation invariance of stats: a vertical edge in the top-left corner of a photograph looks just like a vertical edge in the bottom-right. The pattern is the same wherever it appears.
If that is true, learning a split edge detector for each position is not just wasteful. It is actively harmful, since each position's detector sees just the cases that happened to fall there. Sharing means each position's cases train the same filter. So the filter sees the whole dataset rather than a slice of it.
Three consequences follow right:
Fewer params. Less memory, less math. And far less overfitting for a given amount of data.
Translation equivariance. Shift the input and the feature map shifts identically. A network does not need to see cats in each position to detect them all over.
Better data efficiency. Each occurrence of a pattern anywhere in any pic adds to learning that one filter.
Where the assumption fails
It is worth knowing the cases, since they explain a few real designs.
Position genuinely matters in some domains. In a face-alignment task where pics are cropped and centred, eyes are always near the top and mouths near the bottom. Locally connected layers — local receptive fields without sharing. Were used in early face recognition work for just this reason. And they need far more data.
Vision transformers need position told to them. Attention has no built-in notion of location at all. So transformers add explicit positional embeddings. They trade the convolutional prior for flexibility. It is why they need much more data to reach the same score without pretraining.
Coordinate information is sometimes needed. Tasks that must reason about absolute position gain from appending coordinate channels to the input. The CoordConv trick — since pure convolution deliberately discards that information.
Sharing in other designs
The same principle, applied along a other axis, appears throughout deep learning:
Recurrent networks share weights across time steps. One set of weights processes each position in a sequence. It is why an RNN handles var-length input.
Transformers share the same feed-forward and attention weights across all token positions within a layer.
Graph neural networks share weights across nodes. So one learned message function applies to each vertex.
The unifying idea: wherever the same kind of structure repeats, share the params that process it. It is one of the most reliable ways to build a useful inductive bias into a net.
Guided Tests with This Interactive
Watch the Colors:
Click the Slide Filter button. Watch the formula in the center column. The 4 weights are color-coded (Red, Blue, Yellow, Purple). Notice that as the green highlight box slides across other areas of the Input Pic, the exact same colored weights are used over and over again to do the multiplication.
Translation Invariance:
Since the weights are shared, the filter is looking for the exact same pattern all over. If the weights represent a "vertical edge detector", it will find a vertical edge whether it's in the top-left of the pic or the bottom-right. This property is called Translation Invariance—a cat is recognized as a cat no matter where it is in the photo.
Change the Shared Rules:
Change the weights in the center kernel box to something extreme (like 10, 0, 0, 0). Click slide again. You'll see that changing the params once at once alters how the network interprets the whole pic, since the rule applies globally.
Are All Params Shared?
In a CNN, weights are shared spatially (across the width and height of the pic). But they are not shared across other filters (channels).
A single Convolutional layer often has many filters (e.g., 32 filters). Filter #1 (looking for edges) has its own 9 shared weights. Filter #2 (looking for colors) has a other set of 9 shared weights.
In one line
Param sharing reuses the same weights across other spatial locations.
It cuts memory and math needs by orders of scale.
It forces the network to learn global patterns (Translation Invariance).
It stops severe overfitting by heavily restricting the count of vars the net can memorize.
The +1 is the bias, one per filter. The key detail people miss is that each filter spans all input channels. A 3×3 filter on a 64-channel input has 3×3×64 = 576 weights, not 9.
Worked through for a small network on 32×32×3 input:
Layer
Shape
Parameters
Conv 3×3, 32 filters
32×32×32
(3×3×3 + 1) × 32 = 896
Conv 3×3, 64 filters
16×16×64
(3×3×32 + 1) × 64 = 18,496
Conv 3×3, 128 filters
8×8×128
(3×3×64 + 1) × 128 = 73,856
Global average pool
128
0
Dense to 10 classes
10
1,290
Total: about 94,000 params for a working CIFAR-scale sorter. The equivalent dense network on the same input would run into the tens of millions.
Notice where the params concentrate: in the deeper convolutions, since the channel count grows. And notice that global average pooling adds nothing. Which is just why it replaced the flatten-and-dense head that once held most of a network's weights.
Sharing does not mean shared computation
A common misunderstanding: param sharing reduces the count of weights, not the count of steps.
A 3×3 convolution with 64 filters on a 224×224 map does 224 × 224 × 64 × 9 × in_channels multiply-adds. The filter is small. It is applied 50,000 times.
That is why convolutional networks are memory-light and compute-heavy. And why GPUs suit them: the same small set of weights is reused across thousands of parallel positions. It is just the access pattern GPU memory hierarchies are built for.
It also explains depthwise separable convolutions, used in MobileNet and EfficientNet. Splitting a convolution into a per-channel spatial filter followed by a 1×1 channel mixer cuts both params and steps by roughly a factor of the kernel area. Around 8–9× for 3×3 — with a small score cost. On phones, that trade is often worth making.
One kernel, applied all over
A convolution reuses the same weights at each position. That single decision buys a param count free of pic size and a detector that works wherever the thing appears -- both measured here against a dense layer.
example_01.pyNumPy
import numpy as np
rng = np.random.default_rng(0)
print("THE PARAMETER COUNT. a dense layer needs one weight per input pixel")
print("per output unit. a convolution needs one per kernel position:")
print("%14s %20s %20s %14s"
% ("image", "dense (64 units)", "conv (64 filters, 3x3)", "ratio"))
for n in (28, 64, 224, 512):
dense = n * n * 64
conv = 3 * 3 * 1 * 64
print("%14s %20s %20s %13.0fx"
% ("%dx%d" % (n, n), "{:,}".format(dense), "{:,}".format(conv),
dense / conv))
print(" the conv column does not move. the same %d weights work on a"
% (3 * 3 * 64))
print(" 28x28 thumbnail and a 512x512 photograph, because the kernel is")
print(" applied at every position rather than wired to specific pixels.")
print()
print("TRANSLATION EQUIVARIANCE, which is the other half. train a detector")
print("to find a pattern at one location, then move the pattern:")
N, SIZE = 400, 12
pattern = np.array([[0, 1, 0], [1, 1, 1], [0, 1, 0]], float) * 5.0
def make(n, pos):
X = rng.normal(0, 0.5, (n, SIZE, SIZE))
y = rng.integers(0, 2, n)
r, c = pos
for i in np.where(y == 1)[0]:
X[i, r:r + 3, c:c + 3] += pattern
return X, y
Xtr, ytr = make(N, (2, 2))
Output
Building on the same setup, the next block shows the left cases.
example_02.pyNumPy
import numpy as np
# a dense model: flatten and fit a linear classifier
flat = Xtr.reshape(N, -1)
A = np.column_stack([flat, np.ones(N)])
w_dense = np.linalg.lstsq(A, ytr * 2.0 - 1, rcond=None)[0]
# a convolutional model: correlate with the pattern, take the max response
def conv_score(X):
k = pattern / np.linalg.norm(pattern)
out = np.zeros(len(X))
for i, img in enumerate(X):
best = -1e9
for r in range(SIZE - 2):
for c in range(SIZE - 2):
best = max(best, (img[r:r + 3, c:c + 3] * k).sum())
out[i] = best
return out
thresh = np.median(conv_score(Xtr))
print(" both models are fitted with the pattern always at (2,2).")
print("%18s %16s %16s" % ("pattern moves to", "dense accuracy", "conv accuracy"))
for pos in ((2, 2), (2, 5), (6, 6), (8, 3)):
Xte, yte = make(300, pos)
d = ((np.column_stack([Xte.reshape(300, -1), np.ones(300)]) @ w_dense) > 0)
c = conv_score(Xte) > thresh
print("%18s %16.4f %16.4f"
% (str(pos), (d == (yte == 1)).mean(), (c == (yte == 1)).mean()))
print(" the dense model learned 'look at these specific pixels'. move the")
print(" pattern and it is at chance. the convolution learned the pattern")
print(" itself, and finds it anywhere.")
print()
Output
The next block covers the left steps.
example_03.pyNumPy
import numpy as np
print("WHAT SHARING ASSUMES. it is a real assumption, and it is sometimes")
print("wrong:")
print(" 'a feature that is useful at one position is useful at every")
print(" position'")
print()
print(" true for -- natural photographs. an edge is an edge wherever it is.")
print(" false for -- aligned faces, where the eye region is always in the")
print(" same place and a position-specific detector would be")
print(" better. locally-connected layers exist for exactly")
print(" this and are rarely worth it.")
print(" false for -- spectrograms along the frequency axis, where 200Hz")
print(" and 4000Hz mean genuinely different things.")
print()
print("AND WHAT IT DOES NOT BUY. sharing gives EQUIVARIANCE -- move the")
print("input, the feature map moves with it -- not INVARIANCE:")
img = np.zeros((8, 8))
img[2, 1] = 10.0
# a peaked kernel, so the response has ONE maximum rather than a
# 3x3 plateau of tied values that argmax would break arbitrarily
k = np.array([[1.0, 2.0, 1.0], [2.0, 4.0, 2.0], [1.0, 2.0, 1.0]])
def conv(a):
Output
The next block continues with the left demonstrations.
example_04.pyNumPy
out = np.zeros((6, 6))
for i in range(6):
for j in range(6):
out[i, j] = (a[i:i + 3, j:j + 3] * k).sum()
return out
SHIFT = 3
shifted = np.zeros((8, 8))
shifted[2, 1 + SHIFT] = 10.0
o1, o2 = conv(img), conv(shifted)
p1 = np.unravel_index(o1.argmax(), o1.shape)
p2 = np.unravel_index(o2.argmax(), o2.shape)
print(" input pixel at column 1 -> feature map peak at %s" % (p1,))
print(" input pixel at column %d -> feature map peak at %s" % (1 + SHIFT, p2))
print(" the peak moved %d columns for an input shift of %d. that is"
% (p2[1] - p1[1], SHIFT))
print(" equivariance: the map does not stay the same, it TRANSLATES.")
print(" the two maps are not equal:", np.allclose(o1, o2))
print()
print(" invariance -- the same ANSWER regardless of position -- comes from")
print(" pooling and from the global average at the end:")
print(" max over the whole map, original : %.1f" % o1.max())
print(" max over the whole map, shifted : %.1f" % o2.max())
print(" equal:", o1.max() == o2.max())
print(" that is the division of labour: convolution finds the thing")
print(" anywhere, pooling forgets where it was.")
Output
Questions people ask
Do all positions really need the same filter? For natural pics, yes. And the empirical evidence is overwhelming. For centred, aligned pics where position is meaningful, less so.
Does sharing hurt score? It constrains the net. And that constraint is what stops overfitting. On pic data the constraint matches reality, so it helps.
How does backpropagation work with shared weights? The gradients from each position where the filter was applied are summed into one update. That is why a filter effectively learns from the whole pic at once.
Why do deeper layers have more filters? Since spatial size shrinks and the count of distinct patterns worth detecting grows. Doubling channels as size halves keeps the computation per layer roughly constant.
Is a 1×1 convolution still param sharing? Yes. The same channel-mixing weights are applied at each spatial position.
Do transformers share params? Across positions within a layer, yes. Across layers, often not, though some fast variants tie layer weights deliberately.
Recap in one screen
One filter is applied at each position. So a convolutional layer has thousands of weights where a dense layer would have millions.
The justification is that pic stats are the same all over. An edge is an edge wherever it appears.
Sharing gives translation equivariance and much better data efficiency.
It reduces params, not steps: convolutions are compute-heavy and memory-light.
The same idea drives recurrent networks, transformers and graph networks along their own axes.
Recall check
0 of 3
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “What is Param Sharing”?
What does this module say about “The Massive Efficiency Gain”?
What does this module say about “One filter, used all over”?
Cheat sheet
Parameter Sharing in CNN
Slide one small filter across every spatial position and compare its 9 parameters to the millions a fully connected layer would need.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.