1x1 Convolutions

A convolution with no spatial extent sounds pointless. It is one of the most useful layers in modern architectures.

Overview

The operation that looks like nothing

A 3×3 convolution combines a pixel with its eight neighbours. A 1×1 convolution has no neighbours to combine. At first reading it appears to multiply each pixel by a number, which is a scaling and hardly worth a layer.

That reading forgets the channel dimension, and the channel dimension is where everything happens.

A feature map is not a grid of numbers; it is a stack of grids, one per channel. A layer with 256 channels holds 256 values at every spatial position. A 1×1 convolution stands at one position, takes all 256 values there, and computes a weighted sum of them — then does it again for each output channel, and then repeats the whole thing at every position with the same weights.

So it does not mix neighbours. It mixes channels. Drag the two controls above and watch the connection pattern: every input channel reaches every output channel, and nothing spatial happens at all.

1x1 Convolutions

This module needs JavaScript: the images are computed in the page rather than downloaded.

Worth knowing

A 1×1 convolution looks at one pixel position and all of its channels. It mixes channels, never neighbours.
It is a fully-connected layer applied identically at every spatial position, which is why it is written as a convolution at all.
Its main job is changing the channel count — usually reducing it before an expensive layer, which is what a bottleneck is.
Weights: Cin × Cout, against 9 × Cin × Cout for a 3×3.

1x1 Convolutions

A kernel with no spatial extent, and why almost every modern architecture is full of them.

It is a fully-connected layer in disguise

Fix one spatial position and the operation is exactly a dense layer: Cin inputs, Cout outputs, Cin × Cout weights. What makes it a convolution is that the same dense layer is applied, unchanged, at every position in the map.

That is the same weight-sharing argument that motivates convolution in the first place. If mixing channels in a particular way is useful at one location, it is probably useful at all of them, and sharing the weights makes the layer independent of the input's spatial size.

What it is actually for

Changing the channel count. This is the common case. A 1×1 convolution is the cheapest possible way to turn 256 channels into 64, or 64 into 256, and it is why the layer is sometimes called a projection.

Bottlenecks. Put a channel reduction before an expensive spatial convolution and an expansion after it. ResNet's bottleneck block is exactly this: 1×1 down to 64 channels, 3×3 at 64, 1×1 back up to 256. The 3×3 — by far the costliest part — runs on a quarter of the channels, and the two 1×1s cost almost nothing by comparison.

Count it. A 3×3 straight from 256 to 256 channels needs 9 × 256 × 256 ≈ 590,000 weights. The bottleneck version needs 256×64 + 9×64×64 + 64×256 ≈ 70,000. Same input and output shape, an eighth of the parameters, and an extra two non-linearities thrown in.

Adding non-linearity without touching resolution. Each 1×1 is followed by an activation, so a stack of them increases representational depth at constant spatial size and negligible cost. This was the "network in network" idea that named the technique.

Replacing the classifier head. Global average pooling followed by a 1×1 convolution does the job of a large dense layer with a fraction of the parameters, and works for any input size.

The arithmetic

The readout above compares the two counts directly. For Cin input and Cout output channels:

KernelWeightsSees
1×1Cin × Coutone position, all channels
3×39 × Cin × Cout3×3 positions, all channels
Depthwise 3×39 × Cin3×3 positions, one channel each

That last row is worth noticing. A depthwise separable convolution splits the work in two: a depthwise 3×3 that mixes neighbours but not channels, followed by a 1×1 that mixes channels but not neighbours. Together they approximate a full 3×3 at roughly a ninth of the cost. MobileNet is built almost entirely from that pair, and half of it is 1×1 convolutions.

Where it goes wrong

Expecting spatial work from it. It cannot smooth, sharpen or find an edge. If the receptive field needs to grow, a 1×1 contributes nothing — its contribution to the [receptive field](receptive_field.html) is exactly zero.

Reducing channels too aggressively. The bottleneck is a genuine information bottleneck. Squeezing 256 channels to 8 before the spatial convolution saves computation and can cost more accuracy than it is worth.

Forgetting the activation. A 1×1 with no non-linearity after it, stacked on another linear layer, collapses into a single linear map. Two matrices multiplied together are one matrix.

Check yourself

0 of 3

Answer without scrolling back up.

  1. What does a 1x1 convolution actually combine?

  2. Why does a ResNet bottleneck put 1x1 convolutions around the 3x3?

  3. What happens to two stacked 1x1 convolutions with no activation between them?

Cheat sheet

1x1 Convolutions

A 3×3 convolution combines a pixel with its eight neighbours. A 1×1 convolution has no neighbours to combine. At first reading it appears to multiply each pixel by a number, which is a scaling and hardly worth a layer.

COMPUTER VISION · vizlearn.in/computer_vision/one_by_one_convolutions.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.