Explore how the size and number of filters, input channels, and biases dictate the total number of trainable parameters in a Convolutional layer.
A Single Filter (3D)
A filter is not just a 2D square! It extends through all input channels.
KxK
C_in
Parameters in ONE Filter:
3 × 3 × 3 = 27 weights
+ 1 bias term
Overview
The formula
For a convolutional layer with kernel size k×k, Cin input channels and F filters:
params = (k × k × Cin × F) + F
Each filter is a small 3-D volume of size k×k×Cin. And there are F of them. The trailing + F is one bias per filter.
The part worth pausing on is what is absent: the height and width of the input never appear. A layer applied to a 32×32 pic and the same layer applied to a 4096×4096 pic have just the same count of params. This is since the same filter slides across each spatial position — param sharing — so the count of learned weights depends just on the filter's shape, not the pic's. A fully connected layer with the same input and output would have H × W × C_in × H_out × W_out × C_out params. It for still a modest 32×32 pic is millions. The convolution formula gives a few thousand. That gap is the reason CNNs can process high-resolution pics at all. The param count also tells you how many training cases you need: a layer with 100,000 params trained on 1,000 pics is almost certainly overfitting, while the same layer on 100,000 pics is fine. This page lets you build a network layer by layer, shows the per-layer and cumulative param counts. And checks them to the equivalent fully connected layer.
Explore how the size and number of filters, input channels, and biases dictate the total number of trainable parameters in a Convolutional layer.
A convolutional layer's parameter count depends on the filter size, the input channels and the number of filters - and not at all on the size of the image. That independence is the whole reason CNNs are practical.
The formula
For a convolutional layer with kernel size k×k, Cin input channels and F filters:
params = (k × k × Cin × F) + F
Each filter is a small 3-D volume of size k×k×Cin. And there are F of them. The trailing + F is one bias per filter.
The part worth pausing on is what is absent: the height and width of the input never appear. A layer applied to a 32×32 pic and the same layer applied to a 4096×4096 pic have just the same count of params. This is since the same filter slides across each spatial position — param sharing — so the count of learned weights depends just on the filter's shape, not the pic's. A fully connected layer with the same input and output would have H × W × C_in × H_out × W_out × C_out params. It for still a modest 32×32 pic is millions. The convolution formula gives a few thousand. That gap is the reason CNNs can process high-resolution pics at all. The param count also tells you how many training cases you need: a layer with 100,000 params trained on 1,000 pics is almost certainly overfitting, while the same layer on 100,000 pics is fine. This page lets you build a network layer by layer, shows the per-layer and cumulative param counts. And checks them to the equivalent fully connected layer.
Filters are three-dimensional
A “3×3 filter” on an RGB input is not 9 weights, it is 3×3×3 = 27. A filter always spans each input channel, since it looks for a pattern across all of them at once — an edge detector on colour pics must consider red, green and blue together.
Work one through. First conv layer, 3×3 kernel, RGB input, 64 filters:
(3 × 3 × 3 × 64) + 64 = 1728 + 64 = 1792
Second layer, 3×3 kernel, taking those 64 channels, giving 128:
(3 × 3 × 64 × 128) + 128 = 73728 + 128 = 73,856
Forty times more, from the same kernel size — since Cin and F both grew. The product of those two dominates. It is why deep layers with many channels hold most of a CNN’s weights.
Weight sharing, and the comparison that makes the point
A fully connected layer on a 224×224×3 pic with 64 outputs would need 224 × 224 × 3 × 64 ≈ 9.6 million params. The convolutional layer above does a comparable job with 1,792.
The saving comes from weight sharing: the same filter slides across each position rather than each position having its own weights. That encodes a real assumption — a vertical edge is a vertical edge wherever it appears — and it buys two things at once. Far fewer params, and translation equivariance, since a feature is detected the same way no matter where it sits.
Counting weights, layer by layer
Knowing where a network's params live tells you what will overfit, what will be slow. And where a design is wasteful. Three formulas cover almost all.
Convolution:
(kh × kw × Cin + 1) × Cout
Dense:
(inputs + 1) × outputs
Batch normalisation:
2 × C (a scale and a shift per channel; the running statistics are not trained)
Pooling, ReLU, dropout and flattening have zero params — they compute. But they learn nothing.
The detail that catches everyone: a convolution filter spans all input channels. A 3×3 filter applied to a 256-channel map holds 3×3×256 = 2,304 weights.
A full worked case
A small classifier on 32×32×3 pics:
Layer
Output shape
Parameters
Input
32×32×3
0
Conv 3×3, 32
32×32×32
(3×3×3+1)×32 = 896
BatchNorm
32×32×32
64
MaxPool 2×2
16×16×32
0
Conv 3×3, 64
16×16×64
(3×3×32+1)×64 = 18,496
BatchNorm
16×16×64
128
MaxPool 2×2
8×8×64
0
Conv 3×3, 128
8×8×128
(3×3×64+1)×128 = 73,856
Flatten
8192
0
Dense 256
256
(8192+1)×256 = 2,097,408
Dense 10
10
(256+1)×10 = 2,570
Total: 2,193,418 — and 96% of it is in one layer, the dense layer after the flatten.
That single observation drove a real architectural change. Replace the flatten with global average pooling and the 8×8×128 tensor turns 128 vals. So the dense layer needs (128+1)×256 = 33,024 params instead of 2.1 million. The network drops to about 127,000 params, trains faster, overfits less. And often scores the same or better.
Params are not the whole cost
Two other counts matter, and they behave differently.
Steps (FLOPs) depend on how many times each filter is applied:
FLOPs ≈ 2 × kh × kw × Cin × Cout × Hout × Wout
The first convolution above has 896 params and about 1.8 million multiply-adds, since it is applied at 1,024 positions. Early layers are param-light and compute-heavy. Late layers are the reverse.
Activation memory is what often limits batch size in training, since each intermediate feature map must be kept for the backward pass. A 224×224×64 map is 3.2 million floats — 12.8MB at 32-bit precision, per pic, for one layer. Multiply by batch size and by the count of layers and it dwarfs the weights.
That is why a net with 25 million params (about 100MB) can still exhaust a 16GB GPU: the weights are not the problem, the activations are.
Interactive Exploration Guide
Grow the kernel. Raise 3x3 from 3 to 7 and watch the count. It grows with the square of the kernel size. So 7×7 costs more than five times a 3×3 — which is why modern networks stack small kernels instead.
Grow the channels. Raise Input Channels and then Count of Filters. Each scales the count linearly, and together they scale it multiplicatively.
Toggle the bias. Switch the bias checkbox and note the change is just F — a rounding error next to the weights. It is why layers followed by batch normalisation often drop it fully.
Confirm the pic size is irrelevant. Nothing in the math means input height or width. That is the property that makes a CNN usable on large pics at all.
What trips people up
Forgetting the depth dimension. Counting a 3×3 filter as 9 weights instead of 9 × Cin is the most common error. And it understates the layer by a large factor.
Confusing params with activations. Params are fixed and independent of pic size. Activation memory scales with height × width × channels × batch. Out-of-memory errors in training are almost always activations, not weights.
With large kernels. Two stacked 3×3 layers cover the same receptive field as one 5×5 with fewer params and an extra non-linearity. VGG made this the standard.
Keeping the bias before batch norm. The normalisation subtracts the mean and cancels it just. So those params do nothing.
In one line
A convolutional layer holds (k × k × Cin × F) + F params, with filters spanning each input channel and the pic dimensions appearing nowhere. Weight sharing is what buys the enormous reduction against a fully connected layer, along with translation equivariance. Cost grows with the square of the kernel and the product of input and output channels — which is why small kernels and staged channel growth are the standard design.
Checking it in code
import torch
from torchvision.models import resnet18
model = resnet18()
total = sum(p.numel() for p in model.parameters())
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"{total:,} parameters, {trainable:,} trainable")
for name, module in model.named_children():
n = sum(p.numel() for p in module.parameters())
if n:
print(f"{name:<12} {n:>12,}")
Two things worth checking with this. The trainable count should match your intent — if you froze a backbone for transfer learning and it still reports 11 million trainable params, the freezing did not take. And the per-module breakdown shows at once whether one layer dominates. It is nearly always fixable.
Storage follows right: 4 bytes per param in 32-bit precision, 2 in half precision, 1 in int8. A 25-million-param net is 100MB, 50MB or 25MB depending on precision. It is the arithmetic behind each quantisation decision.
Rules of thumb
Flatten-then-dense is where params explode. Global average pooling is the standard fix.
Doubling the channels roughly quadruples that layer's params, since both C_in and C_out appear in the formula.
Early layers cost compute. Late layers cost params. Optimise the right one for your constraint.
More params is not more capacity in any useful sense if they sit in a badly placed dense layer.
Batch normalisation is nearly free in params and considerably improves trainability.
1×1 convolutions are the cheap way to change channel count, which is why bottleneck blocks use them either side of the costly 3×3.
Count each weight in a real design
The formula is one line per layer type. Applying it to a whole network shows that almost all the params sit somewhere people rarely look. And almost all the compute sits somewhere else fully.
example_01.pyNumPy
import numpy as np
print("the two formulas, and there are only two:")
print(" conv : (kernel_h * kernel_w * in_channels + 1) * out_channels")
print(" dense : (in_features + 1) * out_features")
print("the +1 is the bias, one per output.")
print()
def conv_params(kh, kw, cin, cout, bias=True):
return (kh * kw * cin + (1 if bias else 0)) * cout
def dense_params(fin, fout):
return (fin + 1) * fout
print("WHY THE INPUT SIZE IS ABSENT from the conv formula:")
for n in (32, 224, 1024):
print(" a 3x3, 64->128 conv on a %4dx%-4d input: %s parameters"
% (n, n, "{:,}".format(conv_params(3, 3, 64, 128))))
print(" identical. the kernel is reused at every position, so image size")
print(" changes the COMPUTE and never the parameter count.")
Output
Now apply those formulas to a full VGG-style network and see where the params really sit.
The table reveals the key insight: where params sit versus where compute sits.
example_03.pyNumPy
import numpy as np
def conv_params(kh, kw, cin, cout):
return (kh * kw * cin + 1) * cout
def dense_params(fin, fout):
return (fin + 1) * fout
conv_specs = [(3,3,3,64,224),(3,3,64,64,224),(3,3,64,128,112),(3,3,128,128,112),
(3,3,128,256,56),(3,3,256,256,56),(3,3,256,512,28),(3,3,512,512,28)]
dense_specs = [(14*14*512,4096),(4096,4096),(4096,1000)]
conv_p = sum(conv_params(kh,kw,ci,co) for kh,kw,ci,co,_ in conv_specs)
conv_m = sum(hw*hw*kh*kw*ci*co for kh,kw,ci,co,hw in conv_specs)
dense_p = sum(dense_params(fi,fo) for fi,fo in dense_specs)
dense_m = sum(fi*fo for fi,fo in dense_specs)
total_p = conv_p + dense_p
total_m = conv_m + dense_m
print("AND HERE IS THE POINT:")
print("%22s %18s %10s %18s %10s" % ("", "parameters", "share", "multiply-adds", "share"))
print("%22s %18s %9.1f%% %18s %9.1f%%"
% ("convolutions", "{:,}".format(conv_p), 100*conv_p/total_p,
"{:,}".format(conv_m), 100*conv_m/total_m))
print("%22s %18s %9.1f%% %18s %9.1f%%"
% ("dense layers", "{:,}".format(dense_p), 100*dense_p/total_p,
"{:,}".format(dense_m), 100*dense_m/total_m))
print()
print(" dense layers hold %.0f%% of parameters, do %.1f%% of the arithmetic."
% (100*dense_p/total_p, 100*dense_m/total_m))
print(" convolutions are the opposite.")
print()
first_dense = dense_params(14*14*512, 4096)
print(" ONE layer dominates: the first dense layer alone:")
print(" %s parameters -- %.0f%% of the entire network"
% ("{:,}".format(first_dense), 100*first_dense/total_p))
print(" because flattening 14x14x512 = %s inputs, all connecting to 4096 outputs."
% "{:,}".format(14*14*512))
Output
Global average pooling solves that bottleneck. BatchNorm and bias removal also affect the count.
example_04.pyNumPy
import numpy as np
def conv_params(kh, kw, cin, cout, bias=True):
return (kh * kw * cin + (1 if bias else 0)) * cout
def dense_params(fin, fout):
return (fin + 1) * fout
dense_p = dense_params(14*14*512, 4096) + dense_params(4096, 4096) + dense_params(4096, 1000)
gap_dense = dense_params(512, 1000)
print("GLOBAL AVERAGE POOLING replaces flattening:")
print(" flatten + 3 dense layers : %s parameters" % "{:,}".format(dense_p))
print(" global avg pool + dense : %s parameters" % "{:,}".format(gap_dense))
print(" a %.0fx reduction, and the network generalises BETTER." % (dense_p / gap_dense))
print()
print("BATCHNORM adds 2 learnable parameters per channel:")
for c in (64, 256, 512):
print(" %3d channels -> %d trainable, %d buffers" % (c, 2*c, 2*c))
conv_channels = [64, 64, 128, 128, 256, 256, 512, 512]
bn_total = sum(2*c for c in conv_channels)
total_p = sum(conv_params(3,3,ci,co) for ci,co in
[(3,64),(64,64),(64,128),(128,128),(128,256),(256,256),(256,512),(512,512)]) + dense_p
print(" across this network: %s -- about 1 in %s. negligible count, important effect."
% ("{:,}".format(bn_total), "{:,}".format(int(total_p / bn_total))))
print()
print("BIAS is often disabled before BatchNorm (its shift parameter does the same job):")
print(" 3x3 512->512 with bias: %s, without: %s -- saves %d parameters."
% ("{:,}".format(conv_params(3,3,512,512,True)),
"{:,}".format(conv_params(3,3,512,512,False)),
conv_params(3,3,512,512,True) - conv_params(3,3,512,512,False)))
Output
Questions people ask
Why does my net have more params than I calculated? Biases, batch normalisation. And any auxiliary heads. Also check whether an embedding or a large final classifier is included.
Do frozen layers count as params? They count in the total and not in the trainable count. And they still consume memory and compute in the forward pass.
How many params do I need? It depends on the data, not on a target. A few hundred thousand is plenty for CIFAR-scale problems. ImageNet-scale nets run to tens of millions.
Does more params mean better accuracy? Just with enough data. On a small dataset a large net overfits. And transfer learning from a pretrained one beats training a big net from scratch.
What limits my batch size? Activation memory, almost always, not the weights. Gradient checkpointing trades compute for memory when this binds.
How do I reduce params without losing much accuracy? Global average pooling instead of flatten, depthwise separable convolutions, fewer channels. And quantisation after training.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.