vizlearn
Modules / Deep Learning / Classification

How Do Fully Connected Layers Work in CNN?

Computer VisionBy Updated

Visualize how a flattened 1D array of features connects to every node in the output layer using weights, biases, and Softmax to make final classifications.

Overview

The Transition to 1D

Throughout the early stages of a CNN, images are processed as 2D grids (or 3D volumes with color channels). Convolutional and pooling layers extract spatial features like edges, textures, and shapes. However, to make a final classification (e.g., "Is this a cat or a dog?"), the network must flatten this 2D/3D data into a single 1D list of numbers. This is the input vector you see on the left.

Dense Network Graph

Dot Product Formula

Zj = Σ (Xi · Wij) + Bj
Click "Forward Pass" to view calculations.

Understanding Fully Connected (Dense) Layers

The decision-making powerhouse at the end of a Convolutional Neural Network.

Why "Fully Connected"?

It is called a Fully Connected (or Dense) layer because every single node in the input vector is connected to every single node in the output vector. If you have 4 inputs and 3 outputs, there are $4 \times 3 = 12$ distinct connections (weights), plus 3 biases.

Each weight represents the "importance" of a specific feature for a specific class. For example, if Input Feature 2 strongly represents "pointy ears", the weight connecting Input 2 to the "Cat" output node will likely be a high positive number, while its connection to "Bird" might be negative.

Where the convolutions stop and the decision starts

Convolutional layers produce feature maps: a stack of grids saying where each learned pattern was found. That is not yet an answer to "which of these ten classes is it?"

The dense (fully connected) layer is what turns the evidence into a decision. Every input value connects to every output unit, with its own weight:

outputj = Σi (wij × inputi) + bj

Because every input touches every output, a dense layer can combine evidence from anywhere in the image at once — something a convolution, with its small local window, deliberately cannot do. "Whiskers in the upper left and a tail in the lower right" is a combination only a global layer can express directly.

The final dense layer has one unit per class, and its outputs are logits — unbounded scores that a softmax converts into probabilities.

Getting from a 3-D tensor to a vector

Convolutions produce something like 7×7×512. A dense layer needs a flat vector. Two ways to get there, and the choice matters more than it looks.

Flatten lays the whole tensor out in order: 7×7×512 = 25,088 values. A dense layer of 4,096 units on top of that holds 25,088 × 4,096 ≈ 103 million parameters — in one layer.

Global average pooling reduces each feature map to its mean: 512 values, one per channel. A dense layer to 1,000 classes then needs about 513,000 parameters.

Two hundred times fewer, and it usually generalises better. Three reasons it won:

  • Far fewer parameters, so far less overfitting.
  • No dependence on input size, since the number of channels does not change with resolution.
  • Each output corresponds to "how much of this feature was present overall", which is directly interpretable.

VGG used flatten and carried 138 million parameters, most of them in the classifier head. ResNet uses global average pooling and has 25 million in total, while being both deeper and more accurate. That single change is one of the clearest architectural improvements in the field's history.

Dense layers as 1×1 convolutions

There is a useful equivalence: a dense layer applied at every spatial position is exactly a 1×1 convolution.

That is what makes fully convolutional networks possible. Replace the dense head of a classifier with 1×1 convolutions and the network accepts any input size, producing a coarse map of class scores rather than a single vector — which is where segmentation architectures began.

It also explains why 1×1 convolutions appear all over modern architectures: they mix information across channels at each position without touching spatial structure, which is exactly a per-pixel dense layer. ResNet's bottleneck blocks use them to shrink and restore channel counts cheaply.

Practical guidance

  • Prefer global average pooling to flatten. It is the modern default for good reasons.
  • One hidden dense layer is usually plenty after good convolutional features; two is occasionally justified, three almost never.
  • Dropout belongs here. Dense layers hold most of a naive network's parameters and overfit first; 0.5 dropout between dense layers was standard practice for exactly this reason.
  • The output layer has no activation in most frameworks — the loss function applies softmax internally, and applying it twice is a real and quiet bug.
  • Check the flattened size when changing input resolution. A hard-coded dense input size is the most common shape error when adapting an architecture.

Guided Experiments with This Interactive

  1. Observe the Forward Pass:

    Click the Forward Pass button. Watch the center canvas. The network computes the output one node at a time. It highlights the connections, multiplies each input by its corresponding weight, sums them up, and adds a bias. This raw sum is called the Logit ($Z$).

  2. Tweak the Weights:

    Look at the Weights table on the right. Change the weights connecting to the "Cat" column to high positive numbers (e.g., 2.0). Click Forward Pass again. Notice how the raw score (Logit) for the Cat class shoots up!

  3. The Role of Bias:

    At the bottom of the weights table are the Biases. A bias is an extra value added to the final sum. It shifts the activation function left or right. Try setting the Bird bias to a high value like 5.0. It will artificially inflate the Bird's probability regardless of the inputs.

  4. Softmax Activation:

    Raw Logits can be any number (e.g., -5.2, 0.4, 12.8). To interpret them as probabilities, we apply the Softmax function. Softmax squashes the logits into values between 0 and 1, ensuring they all sum up to exactly 1.0 (or 100%). Notice how the progress bars in the Output section always balance each other out.

The Dense Formula

For an output node $j$, the raw Logit ($Z_j$) is the dot product of the inputs ($X$) and weights ($W$), plus bias ($b$):

Zj = (X1·W1j + X2·W2j + ...) + bj

Softmax is then applied to turn $Z$ into probabilities ($P$):

Pj = eZj / Σ(eZ)

The layer that finally forgets where things were

After the convolutions comes a dense layer, and it is the first part of the network with no notion of position at all. This builds one by hand, shows exactly what flatten does to the geometry, and measures the parameter blow-up that made everyone stop using them.

example_01.pyNumPy
Output

                

                

                    
Summing up

                    
  • Dense layers sit at the very end of a CNN to perform classification.
  • They require 1D flattened input vectors.
  • Every input is connected to every output, creating a massive number of weight parameters.
  • Softmax transforms raw dot-product scores (Logits) into readable probabilities.

                

                

                    
What each part contributes

                    
ComponentJob
Convolution layersDetect patterns, locally and repeatedly
Pooling / strideReduce size, widen the view
Global average poolingSummarise each feature map into one number
Dense hidden layerCombine features globally
DropoutPrevent the dense layers from memorising
Output dense layerOne logit per class
Softmax (in the loss)Turn logits into probabilities
Reading a trained network's dense weights is occasionally informative: a large positive weight from a particular channel to a particular class means that feature is evidence for that class. With global average pooling this is unusually interpretable, and it is exactly the mechanism Class Activation Mapping exploits to produce heatmaps showing where the evidence came from.

                

                

                    
Why not use dense layers throughout?

                    
The natural question, given that dense layers are more general than convolutions. Three answers.
Parameters. A dense first layer on a 224×224×3 image needs 150 million weights per 1,000 units. A convolution needs a few thousand.
No spatial prior. A dense layer treats neighbouring pixels as unrelated, so it has to learn from data that they are related — which requires far more data.
No translation equivariance. A feature learned in one corner does not transfer to another, so every position needs its own examples.
Convolutions win not because they are more powerful but because they are more constrained, and the constraint matches how images actually work. That is what an inductive bias is, and it is why CNNs still beat vision transformers on small datasets, where the transformer has to learn the same structure from scratch.

                

                

                    
Questions people ask

                    
How many units should the hidden dense layer have? Somewhere between the feature count and the class count — 128 to 512 is typical. Bigger rarely helps once the convolutional features are good.
Do I need a hidden dense layer at all? Often not. Modern architectures frequently go straight from global average pooling to the output layer.
Why is my model overfitting badly? Check whether a flatten-and-dense head holds most of the parameters. Replacing it with global average pooling usually fixes it outright.
Should I apply softmax in the model? No, in PyTorch — CrossEntropyLoss expects logits. In Keras, from_logits=True does the same thing. Applying softmax twice trains slowly and quietly badly.
What is the difference between dense and linear? Nothing — Keras calls it Dense, PyTorch calls it Linear.
Can dense layers handle variable input sizes? Not directly. Global average pooling before them removes the constraint.

                

                

                    
Recap in one screen

                    
  • Dense layers connect everything to everything, which is what allows a global decision from local features.
  • Flatten-then-dense is where a naive CNN's parameters explode; global average pooling replaces it at a fraction of the cost.
  • A dense layer applied per position is exactly a 1×1 convolution — the basis of fully convolutional networks.
  • Put dropout between dense layers; leave the output layer unactivated and let the loss apply softmax.
  • Convolutions beat dense layers on images because their constraints match how images are structured.

                

                

                    
A worked size calculation

                    
Tracing shapes through a small network makes the dense layer's cost concrete. Input 224×224×3:
StageOutput shapeValues
After block 1 (stride 2)112×112×64802,816
After block 256×56×128401,408
After block 328×28×256200,704
After block 414×14×512100,352
After block 57×7×51225,088
Flatten25,08825,088
Global average pool512512
Both final rows describe the same tensor, summarised differently. A dense layer to 1,000 classes costs 25 million parameters from the flatten and half a million from the pooled version.
That is the entire argument, and it also explains a common failure when adapting an architecture: change the input from 224 to 256 and the flattened size changes from 25,088 to 32,768, so a hard-coded dense layer errors out. With global average pooling the size is 512 either way, and nothing breaks.

                

            


        

    



        
    



Recall check
0 of 3
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
  1. What does this module say about “The Transition to 1D”?
  2. What does this module say about “Why "Fully Connected"”?
  3. What does this module say about “Where the convolutions stop and the decision starts”?





 Cheat sheet
Fully Connected Layer in CNN
Visualize how a flattened 1D array of features connects to every node in the output layer using weights, biases, and Softmax to make final classifications.
COMPUTER VISION · vizlearn.in/computer_vision/how_dense_layer_works_in_cnn.html
More in Computer Vision
Computer Vision1x1 ConvolutionsComputer VisionDepthwise Separable ConvolutionComputer VisionDilated ConvolutionsComputer VisionGlobal Average Pooling against Flatten
About the author
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.
About VizLearnGitHubLinkedInContact