Visualize how stepping the kernel by multiple pixels (Strides) acts as an efficient downsampling mechanism in Neural Networks.
How taking bigger steps across an image reduces computational load and extracts higher-level features.
In a Convolutional Layer, a kernel (filter) slides across the input image to produce a feature map. The Stride specifies exactly how many pixels this kernel shifts each time it moves.
By default, the stride is 1. The kernel moves 1 pixel to the right, and when it reaches the end of a row, it moves 1 pixel down. But what if we tell the kernel to jump 2 or 3 pixels at a time? This is called strided convolution.
If you process a massive high-resolution image pixel-by-pixel, the resulting feature map will also be massive. This requires enormous amounts of memory and computational power.
By increasing the stride, the kernel skips over sections of the image. This dramatically shrinks the width and height of the resulting feature map. We call this downsampling. Downsampling forces the network to summarize local features (like corners or edges) into higher-level, broader concepts, making the network faster and more robust to slight shifts in the image.
Leave Stride at S = 1. Click Slide Window. Watch the red box (kernel) move smoothly, pixel by pixel, across the input. The output feature map is 7x7. It's a dense, highly detailed representation of the input.
Select S = 2. Notice the Output canvas instantly shrinks to 4x4. Click Slide Window again. Now watch the red box literally skip every other pixel. Because it's taking larger steps, it requires fewer operations to cross the image, producing a smaller, summarized output.
Select S = 3. The output shrinks to a tiny 3x3 grid. The kernel now only stops at 9 specific locations on the entire input board. This is incredibly fast to compute, but notice how "blocky" and low-resolution the resulting feature map has become.
Look at the formula box in the center. Notice the division operation: $(W - K) / S$. The larger the Stride ($S$) in the denominator, the drastically smaller the final Output ($O$) becomes. The $⌊ ⌋$ brackets mean we round down (floor) if the kernel doesn't fit perfectly at the end of a row.
Assuming Padding (P) is 0, the output dimension ($O$) is calculated as:
O = ⌊(W - K) / S⌋ + 1
By manipulating strides, we force neural networks to transition from looking at individual pixels to recognizing high-level abstract shapes.
In a Convolutional Layer, a kernel (filter) slides across the input image to produce a feature map. The Stride specifies exactly how many pixels this kernel shifts each time it moves.