Explore how various augmentation techniques generate diverse training data from a single input source.
Pic Data Tweaking is a critical trick in training deep learning nets for computer vision. The core idea is to artificially expand your training dataset by creating changed copies of existing pics. By showing a net the same pic—but rotated, zoomed, shifted. Or with altered brightness—we teach it to recognize the core subject matter no matter these variations. This process makes the net more reliable and helps stop overfitting. The standard augmentations are horizontal flip, random rotation, random crop, colour jitter. And random erasing — each teaches the net that the label does not change under that change. More aggressive tricks like Mixup (blending two pics and their labels) and CutMix (pasting a patch from one pic onto another) have turn standard in modern training pipelines. The augmentations are applied on the fly in training, not stored as split files. So the dataset stays the same size on disk but the net never sees the exact same pic twice. In PyTorch this is handled by torchvision.transforms. In TensorFlow by tf.pic or the Keras preprocessing layers. The choice of which augmentations to use depends on what invariances are true for your task: horizontal flip is safe for most natural pics but wrong for text recognition. Vertical flip is fine for aerial photographs but not for faces. This page applies a few augmentations to a sample pic, shows the transformed versions side by side. And covers which changes are safe for which domains.
Explore how various augmentation techniques generate diverse training data from a single input source.
Teaching models to generalize by showing them infinite variations of the same thing.
Pic Data Tweaking is a critical trick in training deep learning nets for computer vision. The core idea is to artificially expand your training dataset by creating changed copies of existing pics. By showing a net the same pic—but rotated, zoomed, shifted. Or with altered brightness—we teach it to recognize the core subject matter no matter these variations. This process makes the net more reliable and helps stop overfitting. The standard augmentations are horizontal flip, random rotation, random crop, colour jitter. And random erasing — each teaches the net that the label does not change under that change. More aggressive tricks like Mixup (blending two pics and their labels) and CutMix (pasting a patch from one pic onto another) have turn standard in modern training pipelines. The augmentations are applied on the fly in training, not stored as split files. So the dataset stays the same size on disk but the net never sees the exact same pic twice. In PyTorch this is handled by torchvision.transforms. In TensorFlow by tf.pic or the Keras preprocessing layers. The choice of which augmentations to use depends on what invariances are true for your task: horizontal flip is safe for most natural pics but wrong for text recognition. Vertical flip is fine for aerial photographs but not for faces. This page applies a few augmentations to a sample pic, shows the transformed versions side by side. And covers which changes are safe for which domains.
Imagine you're training a net to recognize cats. If all your training photos show cats perfectly centered and facing forward, the net might fail to recognize a cat that's slightly off-center or tilted. Data tweaking solves this. It teaches the net the concept of "cat-ness" is invariant to changes in spot, scale, and orientation.
By applying these random changes in training, we ensure the net never sees the exact same pic twice. It is forced to learn the underlying patterns of what makes a cat a cat, rather than just memorizing the set pixels of the training pics. This leads to better transfer. The ability to do well on new, unseen data. The standard augmentations are horizontal flip, random rotation (often up to 15–30 degrees), random crop, colour jitter (brightness, contrast, saturation, hue). And random erasing (cutting out a rectangular patch). Each one teaches the net that the label does not change under that change. More aggressive tricks like Mixup (blending two pics and their labels) and CutMix (pasting a patch from one pic onto another) have turn standard in modern training pipelines. The augmentations are applied on the fly in training, not stored as split files. So the dataset stays the same size on disk. In PyTorch this is handled by torchvision.transforms. In TensorFlow by tf.pic or the Keras preprocessing layers. This page applies a few augmentations to a sample pic, shows the transformed versions side by side. And covers which augmentations are safe for which tasks (vertical flip is fine for aerial photos but wrong for faces).
A net that has seen one photograph of a cat has seen that cat, at that angle, in that light. Tweaking shows it many plausible variations of the same pic. So it learns the cat rather than the photograph.
Each variation is created on the fly in training. So the net rarely sees just the same input twice. Nothing is stored. And the dataset on disk is unchanged.
The standard changes for natural pics:
| Transformation | What it teaches |
|---|---|
| Horizontal flip | Left-right orientation does not matter |
| Random crop / resize | Position and scale do not matter |
| Rotation (small) | Slight tilts are the same object |
| Brightness / contrast | Lighting varies |
| Colour jitter | Camera and white balance vary |
| Gaussian noise / blur | Image quality varies |
| Random erasing | Partial occlusion still leaves the object recognisable |
The rule that governs all of them: the change must preserve the label. That is the whole design constraint. And it is where most tweaking mistakes come from.
Vertical flips are fine for satellite and microscope pics. And wrong for street scenes. Upside-down cars are not a category your net needs to handle.
Horizontal flips are fine for animals and wrong for text, digits and any task involving handedness. A flipped "b" is a "d".
Heavy rotation turns a 6 into a 9. For digit recognition, keep rotations small.
Colour jitter destroys the signal when colour is the signal. Ripeness, traffic lights, medical staining.
Aggressive cropping can remove the object fully, leaving an pic labelled "cat" that holds just carpet.
The real procedure: apply your tweaking pipeline to a batch, look at the results. And ask whether you would still give the same label. It takes two minutes and catches almost each mistake of this kind.
Mixup blends two pics and their labels in the same share. 70% cat plus 30% dog, labelled 0.7/0.3. It gives soft targets, improves tuning and reduces overconfidence.
CutMix pastes a rectangular patch of one pic into another and mixes the labels by area. It keeps local detail sharp, which mixup blurs.
RandAugment and AutoAugment apply a random sequence of steps chosen from a fixed pool, with the strength controlled by one or two params. RandAugment in given removed the need to hand-tune a policy and is a standard part of modern training recipes.
Random erasing blanks a random rectangle, forcing the net to rely on more than one region of the object.
For detection and splitting, the change must be applied to the annotations as well as the pixels. Boxes and masks have to move with the pic. Albumentations handles this correctly and is the standard choice for those tasks.
Start by drawing a simple, asymmetric shape, like the letter 'F'. This will make changes like rotation and flipping easy to spot.
Select Rotate from the dropdown. Use the slider to set a central rotation angle. The tweaked grid shows variations around that central angle. Notice how the net would learn that an 'F' is still an 'F', still when tilted.
Switch to Zoom. Move the slider to the right (zoom in) and left (zoom out). This teaches the net scale invariance—recognizing the object whether it's close to the camera or far away.
Select Cutout. This trick randomly removes a patch from the pic. Look at the tweaked versions. This forces the net to make decisions based on the whole object, still if parts of it are occluded or hidden. It can't just rely on one set feature (like the top bar of the 'F').
In a real training pipeline, these augmentations are often chained together. An pic might be randomly rotated, then zoomed, then have its brightness adjusted, all before being fed to the net. This interactive shows them one at a time for clarity. But their true power comes from combination.
Tweaking makes new training cases out of old ones by applying transforms the label is supposed to survive. The whole craft is in that phrase -- and this measures what happens on the transforms where it is not true.
Building on the same setup, the next block shows the left cases.
The next block covers the left steps.
The next block continues with the left demonstrations.
Building on the same setup, the next block shows the left cases.
Data tweaking is one of the most effective and widely used tools for improving the performance of any computer vision net.
Three rules, all of which are violated often.
Training data just. Tweaking validation or test data makes the metrics measure something other than real performance. And makes results irreproducible between runs.
Applied on the fly, not saved to disk. Random change each epoch gives effectively unlimited variation. Pre-making a fixed set of tweaked files gives you a more dataset with the same limited diversity. And uses far more storage.
Normalisation last. Colour and geometric changes operate on the raw pic. Standardisation comes after them, at once before the tensor goes to the net.
import albumentations as A
from albumentations.pytorch import ToTensorV2
train_tf = A.Compose([
A.RandomResizedCrop(224, 224, scale=(0.7, 1.0)),
A.HorizontalFlip(p=0.5),
A.ColorJitter(0.2, 0.2, 0.2, 0.05, p=0.5),
A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
ToTensorV2(),
])
val_tf = A.Compose([ # no randomness at all
A.Resize(256, 256), A.CenterCrop(224, 224),
A.Normalize(mean=(0.485, 0.456, 0.406), std=(0.229, 0.224, 0.225)),
ToTensorV2(),
])One exception to "training just" is test-time tweaking: deliberately predicting on a few tweaked copies of a test pic and averaging the results. It reliably adds a fraction of a percent of score at a few times the test time cost. It is worth it in competitions and rarely in production.
Tweaking is a smoother. And like any smoother it can be overdone. The symptoms differ:
The reasonable starting point for natural pics is a random resized crop, a horizontal flip and mild colour jitter. Add strength just if the net is overfitting. And check that the tweaked pics still look like plausible members of their class.
Note also that tweaking and dataset size interact. With very large datasets the gain shrinks, since the data now covers the variation. With a few hundred pics it is often the single most effective change on hand.
Does tweaking slow training? The CPU does the work while the GPU trains. So with enough dataloader workers it is often free. If GPU use is low, the tweaking pipeline is the first thing to check.
Should I augment the validation set? No. Just apply deterministic resizing and normalisation.
Is mixup always helpful? It helps most on large datasets and long training runs. On small datasets it can slow convergence noticeably.
Can I use tweaking with transfer learning? Yes. And you should — the small datasets that motivate transfer learning are just the ones that overfit without it.
What about synthetic data from generative nets? Increasingly used. And it carries a risk tweaking does not: the net learns the generator's idea of the class rather than the world's.
How do I augment for splitting? Apply the identical geometric change to the mask, with nearest-neighbour interpolation so labels are not blended into invalid vals.
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
Without scrolling back — what is the one-line takeaway from this module?
What does this module say about “Quick Context”?
What does this module say about “The Core Idea: Invariance and Transfer”?
Flip, rotate, zoom and add noise to an image, and see how augmentation multiplies a small training set into a larger one.