Understanding Convolutional Neural Networks

Computer Vision·By Ashish Jangra·Updated 11 September 2026

Convolutional Neural Networks (CNNs) are specialized deep learning architectures fundamentally designed to process structured grid data, such as images. By sliding "filters" across an image, CNNs systematically recognize spatial hierarchies—starting from simple edges and assembling them into complex concepts like faces or handwritten digits.

1. The Convolutional Layer

The core building block of a CNN is the Convolution Operation. Instead of connecting every input pixel to every hidden node (which is computationally impossible for large images), a small matrix called a kernel or filter (e.g., 3x3) slides across the input image.

At each position, we compute the dot product between the filter weights and the underlying pixels. This operation is defined mathematically as:

Where $I$ is the input image, $K$ is the kernel matrix, and $S$ is the resulting feature map. The network learns the optimal values for $K$ during training to extract meaningful features automatically.

2. Activation (ReLU)

After convolution, we introduce non-linearity using an activation function, typically the Rectified Linear Unit (ReLU). This function simply zeroes out negative values and passes positive values through unchanged:

Without non-linearity, a neural network, no matter how many layers it has, would essentially just behave like a single linear regression model.

3. Pooling (Downsampling)

Following activation, Max Pooling layers reduce the spatial dimensions of the feature maps. A common strategy is taking the maximum value over a 2x2 window with a stride of 2.

This dramatically reduces the number of parameters and computational load in the network, while simultaneously making the detected features translation invariant (i.e., the network recognizes the feature even if it shifts slightly in the image).

4. The Dense Classifier

After several rounds of Convolution and Pooling, the 3D output volume is Flattened into a 1D array. This array is fed into standard Dense (Fully Connected) layers. The final layer typically uses a Softmax activation function to output a probability distribution over the possible classes (in our visualizer, the digits 0-9).

This forces the network's outputs to sum exactly to $1.0$ ($100\%$), converting raw network confidence scores into strict class probabilities.

5. Padding, Stride and Output Size

Two settings control how the filter moves, and together they decide the size of every feature map in the network. Stride is how far the filter jumps between positions — a stride of 1 slides one pixel at a time, a stride of 2 skips every other position and halves the output. Padding adds a border of zeros around the input so the filter can be centred on the edge pixels.

Without padding, every convolution shrinks the image slightly, because the filter cannot hang over the edge. A 3x3 filter on a 28x28 image produces a 26x26 output, and after ten such layers there is nothing left. This is why "same" padding — enough zeros to keep the output size equal to the input — is the common default in deep networks.

Here $I$ is the input side length, $K$ the kernel size, $P$ the padding and $S$ the stride. A 28x28 input with a 3x3 kernel, no padding and stride 1 gives $\lfloor (28 - 3 + 0)/1 \rfloor + 1 = 26$. Add one pixel of padding on each side and the output returns to 28. Set the stride to 2 instead and it drops to 13.

Working this out by hand for each layer is worth doing once. Almost every shape-mismatch error in a CNN comes from a stride or padding value that produced a different size than expected, and the formula above tells you exactly what a layer will emit before you run it.

6. Counting the Parameters

A convolutional layer's parameter count depends on the filter size and the number of channels — not on the size of the image. That single fact explains most of what makes CNNs practical.

The $+1$ is the bias, one per output channel. A layer taking a 3-channel colour image and producing 32 feature maps with 3x3 filters has $(3 \times 3 \times 3 + 1) \times 32 = 896$ weights. The same layer applied to a 4-megapixel photograph still has 896 weights, because the filter is reused at every position.

This reuse is called parameter sharing, and it carries an assumption worth stating: a feature worth detecting in one part of the image is worth detecting everywhere. An edge detector is useful in the top-left corner and in the centre, so one set of weights can serve both. That assumption holds well for photographs and holds badly for data where absolute position matters — which is why CNNs are not automatically the right tool for every grid-shaped input.

The dense layers at the end are usually where the parameters actually live. Flattening a $7 \times 7 \times 64$ volume gives 3,136 values, and connecting those to 128 units costs over 400,000 weights — several hundred times more than the convolutional layer above. When a model is too large, the classifier head is normally the first place to look.

7. The Receptive Field

A single neuron in the first convolutional layer sees only a 3x3 patch of the original image. A neuron in the second layer sees a 3x3 patch of the first layer's output, and each of those values already summarised a 3x3 region — so it indirectly sees a 5x5 region of the input. This growing window is the receptive field.

Stacking layers grows the receptive field slowly; pooling and strided convolutions grow it quickly, because each of their outputs already covers a wider area. This is the real reason depth matters in a CNN: the network cannot recognise a face until some neuron somewhere can see a whole face at once.

It also explains the standard shape of a CNN. Early layers have small receptive fields and learn local, generic patterns — edges, corners, colour transitions. Middle layers combine those into textures and parts. Late layers, with receptive fields covering most of the image, respond to whole objects. Nobody designs this hierarchy; it emerges from training, and it is remarkably consistent across networks and datasets.

8. Why Not Just Use a Dense Network?

A fully connected layer on a modest 224x224 colour image would need $224 \times 224 \times 3 = 150{,}528$ inputs. Connecting those to just 1,000 hidden units costs over 150 million weights — for one layer. Beyond the memory cost, a model that large will memorise its training set long before it learns anything general.

Convolution replaces that with three structural assumptions, each of which removes parameters:

• Locality. Nearby pixels are related; distant ones are mostly not. So each neuron looks at a small patch instead of the whole image.
• Parameter sharing. The same filter is applied everywhere, so one set of weights serves every position.
• Translation invariance. Pooling means a feature detected slightly to the left still registers, so the network does not need separate evidence for every possible position.

These are exactly the properties images happen to have, which is why the architecture works so well on them — and why the same architecture applied to tabular data, where columns have no spatial relationship, performs no better than a plain dense network.

9. A Worked Example: One Digit, End to End

Take the 28x28 grayscale digit you can draw above and follow its shape through a small network. Each line is one layer, and the numbers come straight from the output-size and parameter formulas.

• Input: $28 \times 28 \times 1$ — one channel, since the image is grayscale.
• Conv 3x3, 32 filters, same padding: $28 \times 28 \times 32$. Parameters: $(3 \times 3 \times 1 + 1) \times 32 = 320$.
• ReLU: shape unchanged, zero parameters. Negative activations become 0.
• Max pool 2x2, stride 2: $14 \times 14 \times 32$. No parameters — pooling has nothing to learn.
• Conv 3x3, 64 filters, same padding: $14 \times 14 \times 64$. Parameters: $(3 \times 3 \times 32 + 1) \times 64 = 18{,}496$.
• Max pool 2x2, stride 2: $7 \times 7 \times 64$.
• Flatten: 3,136 values in a single vector.
• Dense 128: $3{,}136 \times 128 + 128 = 401{,}536$ parameters — by far the largest layer.
• Dense 10 with softmax: $128 \times 10 + 10 = 1{,}290$ parameters, producing ten probabilities that sum to 1.

Total: about 421,000 parameters, of which 95% sit in that one dense layer. Notice how the spatial dimensions shrink while the channel count grows — the network trades where information is for what it is. By the final convolution, each of the 64 channels at each of the 49 positions represents a fairly abstract pattern rather than a pixel.

10. Training in Practice

Training a CNN is ordinary gradient descent: run a batch forward, compare the predicted distribution to the true label with cross-entropy loss, backpropagate, and adjust every weight slightly. What differs from a dense network is which practical techniques matter.

• Data augmentation. Random shifts, small rotations, flips and crops multiply the effective dataset size for free. For image models this is usually the single most effective regulariser.
• Batch normalisation. Normalising activations within each batch lets you use higher learning rates and makes deep stacks trainable at all.
• Dropout in the dense head. Since most parameters live there, that is where overfitting starts.
• Transfer learning. Start from weights trained on a large dataset and fine-tune. The early layers have already learned edges and textures, and those transfer to almost any image task.
• Learning-rate scheduling. A high rate early to make progress, decayed later to settle into a minimum.

The training signal to watch is the gap between training and validation accuracy. A model that fits the training set well and generalises poorly needs more data, more augmentation or more regularisation — not more layers.

11. Where CNNs Struggle

Pooling gives approximate translation invariance, and nothing more. A CNN is not naturally invariant to rotation or scale — a digit rotated ninety degrees is, as far as the network is concerned, an unfamiliar shape. Augmentation is how that robustness gets added, by showing the network rotated examples during training rather than by building the invariance into the architecture.

The other well-known limitation is that pooling discards spatial relationships along with position. A network that has learned to detect eyes, a nose and a mouth may respond strongly to an image containing all of them in the wrong arrangement, because the evidence is present even though the configuration is nonsense.

Since 2020, Vision Transformers have matched or exceeded CNNs on large datasets by dropping the locality assumption entirely and letting attention decide which parts of the image relate to which. They need considerably more data to reach that point, which is precisely because they lack the built-in assumptions that make CNNs so efficient on small datasets. Convolution remains the better choice when data is limited, and the two approaches are increasingly combined.

12. Recap

• Convolution slides a small learned filter across the image and computes a dot product at every position, producing a feature map.
• Output size is $\lfloor (I - K + 2P)/S \rfloor + 1$; padding preserves size, stride shrinks it.
• Parameters are $(K_h \times K_w \times C_{in} + 1) \times C_{out}$ — independent of image size, because filters are reused everywhere.
• ReLU supplies the non-linearity; without it the whole stack collapses to a single linear map.
• Pooling shrinks the feature maps, grows the receptive field, and buys approximate translation invariance.
• Depth matters because the receptive field must eventually cover a whole object for the network to recognise it.
• Most parameters usually sit in the dense classifier, which is where overfitting and model bloat begin.