Select a pre-trained model, choose a strategy, and selectively unfreeze layers to see how fine-tuning impacts the learning curve and final accuracy.
Training a Deep Convolutional Neural Network from scratch needs millions of pics (like the ImageNet dataset), massive amounts of computing power (GPUs). And weeks of training time. Also, early layers in a CNN generally learn the exact same things no matter the dataset: generic edges, colors. And basic textures.
Transfer Learning is the practice of taking a pre-trained net (whose weights have now been optimized on a massive dataset) and repurposing it for a new, tied task. You are transferring the "knowledge" of visual features to a new domain. The early layers of any CNN trained on natural pics learn the same features — edges, textures, colour gradients — no matter the task. Those features are general enough to be useful for almost any pic classification problem, from medical X-rays to satellite imagery. The standard recipe is: take a net pretrained on ImageNet (ResNet, EfficientNet, VGG), remove its final classification head (which outputs 1,000 ImageNet classes), attach a new head matching your count of classes, freeze the pretrained layers. And train just the new head on your data. This is called feature extraction. If your dataset is large enough, you can then unfreeze some or all of the pretrained layers and fine-tune them at a low learning rate. It adapts the general features to your set domain. The two-phase way — freeze then unfreeze — avoids destroying the pretrained weights with large random gradients from the untrained head. This page loads a pretrained net, replaces its head, and shows how freezing layers affects the trainable param count and the resulting accuracy.Select a pre-trained model, choose a strategy, and selectively unfreeze layers to see how fine-tuning impacts the learning curve and final accuracy.
Why reinvent the wheel? Learn how to reuse powerful models trained by giant tech companies for your own custom tasks.
Training a Deep Convolutional Neural Network from scratch needs millions of pics (like the ImageNet dataset), massive amounts of computing power (GPUs). And weeks of training time. Also, early layers in a CNN generally learn the exact same things no matter the dataset: generic edges, colors. And basic textures.
Transfer Learning is the practice of taking a pre-trained net (whose weights have now been optimized on a massive dataset) and repurposing it for a new, tied task. You are transferring the "knowledge" of visual features to a new domain. The early layers of any CNN trained on natural pics learn the same features — edges, textures, colour gradients — no matter the task. Those features are general enough to be useful for almost any pic classification problem, from medical X-rays to satellite imagery. The standard recipe is: take a net pretrained on ImageNet (ResNet, EfficientNet, VGG), remove its final classification head (which outputs 1,000 ImageNet classes), attach a new head matching your count of classes, freeze the pretrained layers. And train just the new head on your data. This is called feature extraction. If your dataset is large enough, you can then unfreeze some or all of the pretrained layers and fine-tune them at a low learning rate. It adapts the general features to your set domain. The two-phase way — freeze then unfreeze — avoids destroying the pretrained weights with large random gradients from the untrained head. This page loads a pretrained net, replaces its head, and shows how freezing layers affects the trainable param count and the resulting accuracy.A standard pre-trained net (like ResNet-50 or VGG-16) is split into two conceptual parts:
After removing the first head, we attach a new, randomly set up Dense layer matching our target class count. Now, we have to decide how to train this new hybrid net.
We ignore pre-trained weights and set up the whole network randomly. The network has no prior knowledge. The accuracy climbs very slowly. And on a small dataset, heavy nets like VGG-16 will likely overfit and fail to reach high performance.
We load the pre-trained weights into the Conv Base and freeze them (set them to untrainable). In training, just the weights of our newly added Dense Head are updated. The frozen base acts as a powerful feature extractor. Training is very fast. And accuracy jumps up quickly since the network now knows how to "see".
First, we do Feature Extraction. Then, we unfreeze set layers of the Convolutional Base (often the top layers) and train them alongside the Dense Head with a very small learning rate. (In the app, select Fine-Tune and click layers to unlock them). We keep early layers frozen (generic edges). But let later layers adapt their set shape-detectors to our custom pics. This pushes the accuracy to its absolute max limit.
In Fine-Tuning mode, try unfreezing all the Convolutional layers. You will notice the accuracy curve might drop or cap less than expected. If you unfreeze the whole network and train it on a small dataset, the massive gradients from the randomly set up Dense Head propagate backward and wreck the delicate, pre-trained weights of the whole base. This is called Catastrophic Forgetting.
Training a vision net from scratch needs a great deal of data. ImageNet has 1.2 million labelled pics. Most projects have a few hundred.
Transfer learning closes that gap. Take a network now trained on a large dataset, keep the features it learned. And adapt just the parts that are set to your task.
The reason it works is what the layers hold. Early layers learn edges, colours and textures — visual primitives that are useful for in effect any pic task, whether the subject is animals, X-rays or circuit boards. Just the final layers are set to the first classes.
So the recipe is: keep the general part, replace the set part.
| Layer depth | What it learned | Reusable? |
|---|---|---|
| First | Edges, colour transitions | Almost always |
| Early middle | Textures, corners, simple shapes | Usually |
| Late middle | Object parts | Often, with fine-tuning |
| Final | The original 1,000 classes | No — replace it |
Feature extraction. Freeze the whole pretrained network, replace the classifier head. And train just the head. Fast, needs little data, and very hard to overfit.
Fine-tuning. Train the head first, then unfreeze some or all of the backbone and continue with a much less learning rate. Slower, needs more data. And reaches more accuracy when your pics differ from the first dataset.
The choice depends on two things — how much data you have. And how close it is to the pretraining data:
| Similar to ImageNet | Very different (X-rays, satellite) | |
|---|---|---|
| Little data | Freeze everything, train the head | Freeze early layers, train the later ones |
| Plenty of data | Fine-tune the last few blocks | Fine-tune everything, or train from scratch |
The learning rate is what makes fine-tuning work or fail. Use something like 10–100× less than you would for training from scratch — the pretrained weights are now good. And a large step destroys them in the first few batches. Discriminative learning rates, where earlier layers get less rates than later ones, are a standard refinement.
import torch
import torch.nn as nn
from torchvision import models
model = models.resnet50(weights="IMAGENET1K_V2")
for p in model.parameters(): # freeze the backbone
p.requires_grad = False
model.fc = nn.Linear(model.fc.in_features, num_classes) # new head, trainable
opt = torch.optim.AdamW(model.fc.parameters(), lr=1e-3)
# ... train the head for a few epochs ...
for p in model.layer4.parameters(): # then unfreeze the last block
p.requires_grad = True
opt = torch.optim.AdamW([
{"params": model.layer4.parameters(), "lr": 1e-5},
{"params": model.fc.parameters(), "lr": 1e-4},
])Two details that are easy to get wrong. Replacing net.fc creates a new layer with requires_grad=True by default. It is what you want — but check the trainable param count rather than assuming. And keep batch normalisation layers in test mode while the backbone is frozen. Or their running statistics will drift on your small dataset and quietly degrade the features.
A pretrained network expects its input to look statistically like what it was trained on. Three things have to match:
Normalisation. ImageNet nets expect per-channel standardisation with mean [0.485, 0.456, 0.406] and standard deviation [0.229, 0.224, 0.225]. Feeding 0–1 pics without this shifts each activation and costs a few points of accuracy for no obvious reason.
Input size. Often 224×224. Modern designs tolerate other sizes thanks to global average pooling. But the features were learned at a particular scale and work best near it.
Channel order. Torchvision expects RGB; OpenCV loads BGR. This mismatch gives a net that works but does oddly. And it is a genuinely common bug.
The frameworks now ship the correct transforms alongside the weights — weights.transforms() in torchvision — and with that is safer than copying constants from a blog post.
| Backbone | Character |
|---|---|
| ResNet-50 | The reliable default; well understood, widely supported |
| EfficientNet | Better accuracy per FLOP; good on constrained hardware |
| MobileNetV3 | Built for phones and edge devices |
| ConvNeXt | Modern convolutional design, transformer-inspired |
| ViT / Swin | Strong with large data and large pretraining |
| CLIP encoders | Excellent general-purpose features, and zero-shot capable |
For a first attempt, ResNet-50 fine-tuned from ImageNet weights is hard to beat as a baseline. CLIP-pretrained backbones have turn a strong option since their features transfer unusually well to domains far from ImageNet.
Self-supervised pretraining (SimCLR, DINO, MAE) matters when you have a lot of unlabelled domain pics: pretrain on those without labels, then fine-tune on your small labelled set. In specialised domains this often beats ImageNet initialisation outright.
Very other data. Medical scans, satellite imagery, microscopy and audio spectrograms differ enough from natural photographs that the gain shrinks — though it is often still positive. And still worth trying first.
Very small pics. A network built for 224×224 downsamples 32×32 CIFAR pics to nothing. Use an design designed for the resolution.
Plenty of data. With millions of domain pics, training from scratch can exceed a fine-tuned net. Below that, it rarely does.
Catastrophic forgetting. Fine-tuning with too high a learning rate wipes out the pretrained features in a few hundred steps. If accuracy drops sharply at the start of fine-tuning, the learning rate is the first thing to check.
Transfer learning replaces the last layer of a trained network and reuses all before it. This measures why that works, when it stops working. And the two settings -- which layers to freeze and what learning rate to use -- that decide whether you get a good net or a broken one.
The next block continues with the left demonstrations.
The next block continues with the left demonstrations.
The next block continues with the left demonstrations.
The next block continues with the left demonstrations.
The next block continues with the left demonstrations.
The next block continues with the left demonstrations.
How many pics do I need? With a frozen backbone, a few dozen per class can work. For fine-tuning, a few hundred per class is a reasonable target.
Should I freeze batch normalisation? Yes when the backbone is frozen — keep those layers in eval mode. When fine-tuning the whole network on a reasonable batch size, let them update.
Which layers should I unfreeze first? The last block. Work backwards if accuracy is still improving and you have the data to support it.
Can I transfer between very other tasks? Yes — ImageNet features are a good starting point for detection, segmentation and retrieval, not just classification.
Is a bigger backbone always better? No. With a small dataset a large net overfits faster. And inference cost may matter more than the last point of accuracy.
Why is my fine-tuned net worse than the frozen one? Almost always the learning rate is too high. Or the dataset is too small to support updating that many params.
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “What is Transfer Learning”?
What does this module say about “How it Works: Changing the Design”?
What does this module say about “The Three Plans”?
Swap the classification head of a pretrained CNN, freeze the base, and fine-tune on your dataset with a fraction of the data.