Understanding Convolutional Neural Networks
Convolutional Neural Networks (CNNs) are specialized deep learning architectures fundamentally designed to process structured grid data, such as images. By sliding "filters" across an image, CNNs systematically recognize spatial hierarchies—starting from simple edges and assembling them into complex concepts like faces or handwritten digits.
1. The Convolutional Layer
The core building block of a CNN is the Convolution Operation. Instead of connecting every input pixel to every hidden node (which is computationally impossible for large images), a small matrix called a kernel or filter (e.g., 3x3) slides across the input image.
At each position, we compute the dot product between the filter weights and the underlying pixels. This operation is defined mathematically as:
Where $I$ is the input image, $K$ is the kernel matrix, and $S$ is the resulting feature map. The network learns the optimal values for $K$ during training to extract meaningful features automatically.
2. Activation (ReLU)
After convolution, we introduce non-linearity using an activation function, typically the Rectified Linear Unit (ReLU). This function simply zeroes out negative values and passes positive values through unchanged:
Without non-linearity, a neural network, no matter how many layers it has, would essentially just behave like a single linear regression model.
3. Pooling (Downsampling)
Following activation, Max Pooling layers reduce the spatial dimensions of the feature maps. A common strategy is taking the maximum value over a 2x2 window with a stride of 2.
This dramatically reduces the number of parameters and computational load in the network, while simultaneously making the detected features translation invariant (i.e., the network recognizes the feature even if it shifts slightly in the image).
4. The Dense Classifier
After several rounds of Convolution and Pooling, the 3D output volume is Flattened into a 1D array. This array is fed into standard Dense (Fully Connected) layers. The final layer typically uses a Softmax activation function to output a probability distribution over the possible classes (in our visualizer, the digits 0-9).
This forces the network's outputs to sum exactly to $1.0$ ($100\%$), converting raw network confidence scores into strict class probabilities.