vizlearn
Modules / Deep Learning / Network Architecture

Calculating Parameters in CNNs

Computer VisionBy Updated

Explore how the size and number of filters, input channels, and biases dictate the total number of trainable parameters in a Convolutional layer.

Overview

The formula

For a convolutional layer with kernel size k×k, Cin input channels and F filters:

parameters = (k × k × Cin × F) + F

Each filter is a small 3-D volume of size k×k×Cin, and there are F of them. The trailing + F is one bias per filter.

The part worth pausing on is what is absent: the height and width of the input never appear. A layer applied to a 32×32 image and the same layer applied to a 4096×4096 image have exactly the same number of parameters.

A Single Filter (3D)

A filter is not just a 2D square! It extends through all input channels.

KxK
C_in
Parameters in ONE Filter:
3 × 3 × 3 = 27 weights
+ 1 bias term

Calculating Parameters in CNN: A Practical Guide

A convolutional layer's parameter count depends on the filter size, the input channels and the number of filters - and not at all on the size of the image. That independence is the whole reason CNNs are practical.

Filters are three-dimensional

A “3×3 filter” on an RGB input is not 9 weights, it is 3×3×3 = 27. A filter always spans every input channel, because it looks for a pattern across all of them at once — an edge detector on colour images must consider red, green and blue together.

Work one through. First conv layer, 3×3 kernel, RGB input, 64 filters:

(3 × 3 × 3 × 64) + 64 = 1728 + 64 = 1792

Second layer, 3×3 kernel, taking those 64 channels, producing 128:

(3 × 3 × 64 × 128) + 128 = 73728 + 128 = 73,856

Forty times more, from the same kernel size — because Cin and F both grew. The product of those two dominates, which is why deep layers with many channels hold most of a CNN’s weights.

Weight sharing, and the comparison that makes the point

A fully connected layer on a 224×224×3 image with 64 outputs would need 224 × 224 × 3 × 64 ≈ 9.6 million parameters. The convolutional layer above does a comparable job with 1,792.

The saving comes from weight sharing: the same filter slides across every position rather than each position having its own weights. That encodes a real assumption — a vertical edge is a vertical edge wherever it appears — and it buys two things at once. Far fewer parameters, and translation equivariance, since a feature is detected the same way regardless of where it sits.

Counting weights, layer by layer

Knowing where a network's parameters live tells you what will overfit, what will be slow, and where a design is wasteful. Three formulas cover almost everything.

Convolution:

(kh × kw × Cin + 1) × Cout

Dense:

(inputs + 1) × outputs

Batch normalisation:

2 × C  (a scale and a shift per channel; the running statistics are not trained)

Pooling, ReLU, dropout and flattening have zero parameters — they compute, but they learn nothing.

The detail that catches everyone: a convolution filter spans all input channels. A 3×3 filter applied to a 256-channel map holds 3×3×256 = 2,304 weights.

A full worked example

A small classifier on 32×32×3 images:

LayerOutput shapeParameters
Input32×32×30
Conv 3×3, 3232×32×32(3×3×3+1)×32 = 896
BatchNorm32×32×3264
MaxPool 2×216×16×320
Conv 3×3, 6416×16×64(3×3×32+1)×64 = 18,496
BatchNorm16×16×64128
MaxPool 2×28×8×640
Conv 3×3, 1288×8×128(3×3×64+1)×128 = 73,856
Flatten81920
Dense 256256(8192+1)×256 = 2,097,408
Dense 1010(256+1)×10 = 2,570

Total: 2,193,418 — and 96% of it is in one layer, the dense layer after the flatten.

That single observation drove a real architectural change. Replace the flatten with global average pooling and the 8×8×128 tensor becomes 128 values, so the dense layer needs (128+1)×256 = 33,024 parameters instead of 2.1 million. The network drops to about 127,000 parameters, trains faster, overfits less, and typically scores the same or better.

Parameters are not the whole cost

Two other numbers matter, and they behave differently.

Operations (FLOPs) depend on how many times each filter is applied:

FLOPs ≈ 2 × kh × kw × Cin × Cout × Hout × Wout

The first convolution above has 896 parameters and about 1.8 million multiply-adds, because it is applied at 1,024 positions. Early layers are parameter-light and compute-heavy; late layers are the reverse.

Activation memory is what usually limits batch size during training, because every intermediate feature map must be kept for the backward pass. A 224×224×64 map is 3.2 million floats — 12.8MB at 32-bit precision, per image, for one layer. Multiply by batch size and by the number of layers and it dwarfs the weights.

That is why a model with 25 million parameters (about 100MB) can still exhaust a 16GB GPU: the weights are not the problem, the activations are.

Interactive Exploration Guide

  1. Grow the kernel. Raise 3x3 from 3 to 7 and watch the count. It grows with the square of the kernel size, so 7×7 costs more than five times a 3×3 — which is why modern networks stack small kernels instead.
  2. Grow the channels. Raise Input Channels and then Number of Filters. Each scales the count linearly, and together they scale it multiplicatively.
  3. Toggle the bias. Switch the bias checkbox and note the change is exactly F — a rounding error next to the weights, which is why layers followed by batch normalisation usually drop it entirely.
  4. Confirm the image size is irrelevant. Nothing in the calculation refers to input height or width. That is the property that makes a CNN usable on large images at all.

What trips people up

  • Forgetting the depth dimension. Counting a 3×3 filter as 9 weights instead of 9 × Cin is the most common error, and it understates the layer by a large factor.
  • Confusing parameters with activations. Parameters are fixed and independent of image size; activation memory scales with height × width × channels × batch. Out-of-memory errors during training are almost always activations, not weights.
  • Using large kernels. Two stacked 3×3 layers cover the same receptive field as one 5×5 with fewer parameters and an extra non-linearity. VGG made this the standard.
  • Keeping the bias before batch norm. The normalisation subtracts the mean and cancels it exactly, so those parameters do nothing.

In one line

A convolutional layer holds (k × k × Cin × F) + F parameters, with filters spanning every input channel and the image dimensions appearing nowhere. Weight sharing is what buys the enormous reduction against a fully connected layer, along with translation equivariance. Cost grows with the square of the kernel and the product of input and output channels — which is why small kernels and staged channel growth are the standard design.

Checking it in code

import torch
from torchvision.models import resnet18

model = resnet18()
total = sum(p.numel() for p in model.parameters())
trainable = sum(p.numel() for p in model.parameters() if p.requires_grad)
print(f"{total:,} parameters, {trainable:,} trainable")

for name, module in model.named_children():
    n = sum(p.numel() for p in module.parameters())
    if n:
        print(f"{name:<12} {n:>12,}")

Two things worth checking with this. The trainable count should match your intent — if you froze a backbone for transfer learning and it still reports 11 million trainable parameters, the freezing did not take. And the per-module breakdown shows immediately whether one layer dominates, which is nearly always fixable.

Storage follows directly: 4 bytes per parameter in 32-bit precision, 2 in half precision, 1 in int8. A 25-million-parameter model is 100MB, 50MB or 25MB depending on precision, which is the arithmetic behind every quantisation decision.

Rules of thumb

  • Flatten-then-dense is where parameters explode. Global average pooling is the standard fix.
  • Doubling the channels roughly quadruples that layer's parameters, because both C_in and C_out appear in the formula.
  • Early layers cost compute; late layers cost parameters. Optimise the right one for your constraint.
  • More parameters is not more capacity in any useful sense if they sit in a badly placed dense layer.
  • Batch normalisation is nearly free in parameters and considerably improves trainability.
  • 1×1 convolutions are the cheap way to change channel count, which is why bottleneck blocks use them either side of the expensive 3×3.

Count every weight in a real architecture

The formula is one line per layer type. Applying it to a whole network shows that almost all the parameters sit somewhere people rarely look, and almost all the compute sits somewhere else entirely.

example_01.pyNumPy
Output

                

                

                    
Questions people ask

                    
Why does my model have more parameters than I calculated? Biases, batch normalisation, and any auxiliary heads. Also check whether an embedding or a large final classifier is included.
Do frozen layers count as parameters? They count in the total and not in the trainable count, and they still consume memory and compute in the forward pass.
How many parameters do I need? It depends on the data, not on a target. A few hundred thousand is plenty for CIFAR-scale problems; ImageNet-scale models run to tens of millions.
Does more parameters mean better accuracy? Only with enough data. On a small dataset a large model overfits, and transfer learning from a pretrained one beats training a big model from scratch.
What limits my batch size? Activation memory, almost always, not the weights. Gradient checkpointing trades compute for memory when this binds.
How do I reduce parameters without losing much accuracy? Global average pooling instead of flatten, depthwise separable convolutions, fewer channels, and quantisation after training.

                

                

                    
Recap in one screen

                    
  • Conv: (k×k×C_in + 1) × C_out. Dense: (inputs + 1) × outputs. BatchNorm: 2 per channel.
  • Pooling, activations and dropout have no parameters.
  • Each filter spans every input channel — that is the term people forget.
  • The dense layer after a flatten usually holds most of a naive network's weights; global average pooling removes it.
  • Parameters are not the only cost: early layers dominate compute, and activation memory dominates GPU usage.

                

            


        

    



        
    



Recall check
0 of 4
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
  1. What is meant by “Flatten-then-dense is where parameters explode” here?
  2. What is meant by “Doubling the channels roughly quadruples that layer's parameters,” here?
  3. What is meant by “Early layers cost compute; late layers cost parameters” here?
  4. What is meant by “More parameters is not more capacity in any useful sense” here?





 Cheat sheet
Calculating Parameters in CNN
Explore how the size and number of filters, input channels, and biases dictate the total number of trainable parameters in a Convolutional layer.
COMPUTER VISION · vizlearn.in/computer_vision/calculating_parameters_in_cnn.html
More in Computer Vision
Computer VisionCNN ArchitectureComputer VisionData Loaders in CNNComputer VisionImage Data AugmentationComputer VisionTransfer Learning with CNN
About the author
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.
About VizLearnGitHubLinkedInContact