Modules / Deep Learning / Image Processing

How Neural Networks Process Images

Visualizing the process of "flattening" a 2D image grid into a 1D sequence of numbers to feed into Dense Layers.

Overview

The idea in brief

When you look at an image, you see a 2D grid of colors. However, standard Artificial Neural Networks (specifically Dense or Fully-Connected Layers) are structured to accept only a one-dimensional (1D) array or vector of numbers as input. Before a network can "look" at an image, the image must undergo Flattening.

Neural Network Input

LIVE PROCESSING
Step 1: 2D Matrix 16 x 16 x 3
Flattening Row by Row
Step 2: 1D Input Vector 768 nodes
Scroll horizontally to view all nodes

Understanding Image Flattening

Why and how we transform 2D grids of pixels into 1D lists for Neural Networks.

The Core Idea: Unrolling the Grid

Think of a digital image as a spreadsheet where each cell is a pixel holding a color value. "Flattening" is simply taking the first row of this spreadsheet, placing it down, taking the second row and appending it to the end of the first, and continuing until the whole grid forms a single, long line of numbers.

If the image is colored, it contains three channels: Red, Green, and Blue (RGB). The network needs separate input nodes for every color value of every pixel. The math for the total number of inputs is simple: Width × Height × Channels = Total Nodes.

An image is a grid of numbers

Before anything else, a picture has to become arithmetic. A greyscale image is a grid of brightness values from 0 (black) to 255 (white). A colour image is three such grids stacked — red, green and blue.

A 224×224 colour photograph is therefore a block of numbers with shape (224, 224, 3), or 150,528 values. That is what the network actually receives; there is no image in there, only a tensor.

Almost every pipeline rescales those values to 0–1 by dividing by 255, or standardises them per channel using the dataset's mean and standard deviation. The reason is the same as for tabular data: gradients and initialisation schemes assume inputs of roughly unit scale, and raw 0–255 values make the first layer's activations far too large.

Why a dense layer is the wrong tool

Feed that image to an ordinary fully connected layer with 1,000 units and you need 150,528 × 1,000 = 150 million weights in the first layer alone. Three problems follow immediately.

It is unaffordable. That many parameters need enormous data and memory, and most of them will be fitting noise.

It throws away the structure. Flattening the grid into a list destroys the fact that neighbouring pixels are related. To a dense layer, pixel (0,0) and pixel (0,1) are no more related than pixel (0,0) and pixel (200,150).

It has to relearn everything everywhere. A dense layer that learns to detect an edge in the top-left corner has learned nothing about edges in the bottom-right. Every position needs its own weights.

Convolution fixes all three at once by making two assumptions about images that happen to be true.

Local connections and shared weights

Locality. A small filter — typically 3×3 — looks at a tiny patch rather than the whole image. Meaningful visual structure is local: an edge, a corner or a texture is defined by a handful of neighbouring pixels.

Weight sharing. That same filter slides across the entire image. One set of nine weights is applied everywhere, so a vertical-edge detector learned in one place works in every place.

The arithmetic is startling. A 3×3 filter over 3 input channels is 27 weights plus a bias. Use 64 such filters and the layer has 1,792 parameters — against 150 million for the dense equivalent, and it works better.

This also gives the network translation equivariance: move the cat two pixels right and the feature map moves two pixels right. The network does not have to see cats in every position to recognise them.

The hierarchy that emerges

Stacking convolution layers builds features of increasing abstraction, and this is not a metaphor — it is visible when you inspect trained filters.

DepthWhat the filters respond to
Layer 1Edges at various orientations, colour blobs
Layer 2Corners, curves, simple textures
Layers 3–4Repeated patterns, parts — eyes, wheels, letters
Deep layersObject parts and whole objects

Each layer sees a wider region of the original image than the last, because each of its inputs already summarised a patch. That growing window is the receptive field, and it is why depth matters: a network needs enough layers for its deepest neurons to see the whole object.

A complete classifier is then: several convolution and pooling blocks to build features, followed by pooling or flattening, and finally one or two dense layers to turn those features into class scores.

Guided Experiments with This Interactive

  1. Observe the Math:

    Look at the right column. For a 16x16 RGB image, you'll see the 2D matrix labeled 16 x 16 x 3. This means there are 768 individual color values. Scroll the 1D Input Vector horizontally to see all 768 "nodes" laid out end-to-end!

  2. Toggle Grayscale:

    Check the Convert to Grayscale box. Notice how the node count drops by exactly a factor of 3 (e.g., from 768 to 256). Grayscale images only have 1 channel (brightness) instead of 3 (RGB), making them much easier for simple neural networks to process.

  3. Adjust the Resolution:

    Slide the Resolution to 32x32. Even for this tiny icon-sized image, an RGB format creates 3,072 nodes. This rapid growth in data is why modern deep learning requires powerful GPUs.

  4. Live Webcam Test:

    Switch to the Live Webcam tab. Try moving your hand. Watch how the 2D image changes and how those changes ripple through the 1D flattened array. Every movement changes hundreds of values simultaneously across the vector.

The Problem with Flattening

While flattening allows an MLP to process an image, it destroys spatial relationships. A pixel right above another pixel in 2D might be dozens of indices apart in the 1D array. Standard dense networks struggle to understand that these pixels were originally neighbors.

This exact problem led to the invention of Convolutional Neural Networks (CNNs), which process images in 2D first before flattening them at the very end!

What to remember

  • Dense neural network layers only accept 1D lists (vectors) as input.
  • Flattening is the process of stringing rows of pixels together into a line.
  • The size of the input layer is $W \times H \times C$ (Channels).
  • High resolution images create massive input layers, making them computationally expensive.

Flattening is the bridge between raw image files and the mathematical matrix operations that power Artificial Intelligence.

What each piece contributes

Convolution detects patterns and produces feature maps — one per filter, each highlighting where its pattern was found.

ReLU zeroes negative responses. Without a non-linearity, stacked convolutions collapse into a single convolution, and depth buys nothing.

Pooling (or a strided convolution) shrinks the spatial dimensions, which reduces computation, widens the receptive field, and adds a little tolerance to small shifts.

Batch normalisation keeps activations in a stable range, making deeper networks trainable and allowing higher learning rates.

Global average pooling collapses each feature map to a single number, replacing the huge flatten-then-dense step used by older architectures and removing most of their parameters.

The classifier head maps the final features to class scores, which a softmax turns into probabilities.

Where the assumptions break

Convolution works because of assumptions about images, and it is worth knowing when they do not hold.

Rotation and scale are not free. Convolution is equivariant to translation, not to rotation or resizing. A network trained on upright faces will struggle with faces at 90°, which is exactly why rotation and scaling are standard data augmentations.

Global relationships need depth or attention. A 3×3 filter sees three pixels; relating opposite corners of an image requires many layers of accumulated receptive field. Vision transformers take the other route, comparing all patches directly with attention from the first layer — which is why they need more data but capture long-range structure more easily.

Not all grids are images. Convolution assumes neighbouring positions are related. Applied to a table of unrelated columns, that assumption is false and the inductive bias is wasted.

From a grid of numbers to a class score

A network never sees a picture -- it sees an array, and every layer is arithmetic on that array. This follows one small image all the way from pixels to a prediction, so that nothing in the pipeline is left as a metaphor.

example_01.pyNumPy
Output

Questions people ask

Why 3×3 filters almost everywhere? Two stacked 3×3 layers see the same region as one 5×5 but use fewer parameters and include an extra non-linearity. VGG established this and it stuck.

Does the input have to be a fixed size? Convolutions do not care, but a flatten-then-dense head does. Global average pooling removes the constraint, which is why modern architectures accept varying sizes.

Why normalise pixel values? Because 0–255 inputs produce large activations and awkward gradients. Scaling to 0–1 or standardising per channel is standard.

How many filters should a layer have? Typically doubling with depth — 32, 64, 128, 256 — as spatial size shrinks and the number of distinct patterns to represent grows.

Do CNNs see like humans? Not really. They are far more sensitive to texture than to shape, and small adversarial changes invisible to a person can flip a prediction entirely.

Are CNNs obsolete now that transformers exist? No. They remain more data-efficient, faster on small images, and the standard choice on limited hardware. Hybrid designs are common.

Recap in one screen

  • An image is a grid of numbers; colour adds three channels.
  • A dense layer on raw pixels needs hundreds of millions of weights and ignores spatial structure.
  • Convolution assumes locality and shares one small filter across the whole image, cutting parameters by orders of magnitude.
  • Stacked layers build a hierarchy: edges, then textures, then parts, then objects.
  • Translation is handled naturally; rotation and scale must come from augmentation.

Recall check

0 of 3

Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.

  1. What is meant by “Flattening” here?

  2. What does this module say about “The idea in brief”?

  3. What does this module say about “Preprocessing Parameters”?

Cheat sheet

How Neural Networks Process Images

When you look at an image, you see a 2D grid of colors. However, standard Artificial Neural Networks (specifically Dense or Fully-Connected Layers) are structured to accept only a one-dimensional (1D) array or vector of numbers as input. Before a network can "look" at an image, the image must undergo Flattening.

COMPUTER VISION · vizlearn.in/computer_vision/how_neural_network_process_images.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.