Home / Deep Learning

Model Training Curves

By Updated

Simulate and understand how Loss and Accuracy evolve over time. Tune the architecture mathematically to observe underfitting, ideal fits, and overfitting.

Overview

The four shapes

  • Both high and flat — underfitting. The model has not got enough capacity, or has not trained long enough, to capture the pattern.
  • Both falling and close together — healthy. Keep going.
  • Training falling, validation turning upward — overfitting. The turning point is where you should have stopped.
  • Validation below training — usually not a miracle. Dropout is active during training but not evaluation, or the validation split happens to be easier.
5
5
100
System Diagnosis

ANALYZING...

Collecting initial training metrics...
Current Epoch 0
Train Loss 0.000
Val Loss 0.000
Train Acc 0.00%
Val Acc 0.00%
Loss Curve (Lower is Better)
Train
Val
Accuracy Curve (Higher is Better)
Train
Val

Model Training Curves: A Practical Guide

Two lines tell you almost everything about a training run. Learning to read the gap between them is the fastest diagnostic skill in machine learning.

Reading real numbers

Training loss 0.08, validation loss 0.34. The gap of 0.26 is the model's memorisation of the training set — that is overfitting, no matter how good 0.08 looks in isolation.

Training loss 0.31, validation loss 0.33. A tiny gap, but both are high. The model is not overfitting; it simply is not good enough yet. These two situations need opposite responses, which is exactly why looking at one curve is not enough.

Reading the gap between the curves

The two curves matter less individually than the distance between them, and how that distance changes over time.

A gap that stays narrow while both curves fall is a healthy run. A gap that opens steadily — training continuing down while validation flattens or rises — is overfitting, and the epoch where validation turns is where early stopping should trigger. Both curves flat and high means the model has not got started: check the learning rate before assuming the architecture is too small.

The size of the gap is not itself a problem. A large but stable gap on a model whose validation loss is still the best you have achieved is fine; it is the trend in validation loss that decides whether to stop.

Curves that mean something is broken

  • Loss becomes NaN. Exploding gradients or a numerical error such as log(0). Lower the learning rate, clip the gradient norm, and check for unscaled inputs.
  • Loss is completely flat from step one. Nothing is learning. Usually a learning rate near zero, a disconnected graph, or gradients not flowing — verify the optimiser is actually stepping the parameters you think it is.
  • Loss oscillates violently without trending down. Learning rate too high. Divide by ten.
  • Validation loss much lower than training loss. Normally dropout and batch norm being active during training only. If the gap is large and persistent, suspect a leak or a validation split that is easier than the training set.
  • Loss drops sharply at each epoch boundary. The data is not being reshuffled, so the model is memorising batch order.

What a healthy run looks like

A steep initial drop as the model learns the easy structure, then a long shallow decline as it refines. Validation tracks training closely at first and gradually separates. Both curves are noisy step to step — that is mini-batch sampling, not instability — and the trend over a window of epochs is what to judge.

If loss is still falling meaningfully when training ends, you stopped too early. If validation has been flat or rising for many epochs, you trained too long and the useful checkpoint is behind you.

The most informative plot in machine learning

Two lines — training loss and validation loss against epochs — diagnose more problems faster than any other single output.

Read them together, never separately. The level of the training curve measures bias: how well the model can fit at all. The gap between the curves measures variance: how much of that fit was memorisation.

ShapeDiagnosisAction
Both falling, close togetherHealthy, still learningKeep training
Both flat and highUnderfittingBigger model, better features, higher rate
Training falls, validation risesOverfittingRegularise, augment, stop earlier
Training falls, validation flatMild overfitting or a ceilingMore data or regularisation
Both flat from step oneLearning rate far too low, or a bugCheck gradients and the rate
Loss jumps to NaNRate too high, or log(0)Lower the rate, clip gradients
Validation below trainingAugmentation or dropout artefact, or a leakCheck the pipeline

That last row is worth expanding. Validation loss below training loss is usually benign: dropout and augmentation are active during training and disabled at validation, so the training number is measured under harder conditions. If neither is in use, suspect a leak.

Reading the shape of the descent

Beyond the broad diagnosis, the curve's shape carries specific information.

A sharp initial drop then a long plateau is normal. The model learns the easy structure — class priors, obvious features — in the first epoch or two, and the rest is slow refinement.

A staircase means a step learning-rate schedule; each drop is a rate reduction letting the model settle into a narrower minimum.

A sudden improvement in the final epochs is the cosine schedule's low-rate phase working as intended. Expect it, and set early-stopping patience long enough not to cut it off.

A sudden permanent jump upwards means the learning rate was too high for the sharper region the model had entered. The weights have been thrown somewhere worse. A decaying schedule prevents it.

Noise batch to batch, smooth across epochs is expected. Judge the trend over several epochs; a single bad epoch is not a signal.

Curves worth plotting beyond the loss

Learning rate. Plot it alongside the loss and the connection between schedule and progress becomes visible immediately. It also catches the common bug of a scheduler stepped at the wrong interval.

Gradient norm per layer. A steady decay from output to input means vanishing gradients; values in the thousands mean exploding. This is the fastest diagnosis available for a network that will not train.

The task metric. Accuracy or F1 alongside the loss, because they can diverge — loss rising while accuracy holds means growing overconfidence rather than worse classification.

The fraction of zero activations. Around 50% after ReLU is healthy; 90% and rising means units are dying.

Logging all four costs almost nothing and turns debugging from guesswork into reading.

Try this above

  1. Set Model Complexity low and Data Noise low. Both curves flatten out high — the underfitting signature.
  2. Raise Model Complexity and let it run the full Total Epochs. Find the epoch where validation stops falling and starts to climb.
  3. Raise Data Noise and repeat. The turning point arrives earlier, because there is more noise available to memorise.
  4. Set Learning Rate too high and watch both curves become jagged rather than smooth.

What usually goes wrong

Judging on training loss alone. It almost always keeps falling. A model can drive training loss to nearly zero while getting steadily worse at its actual job.Training for a fixed epoch count. The right number is wherever validation loss bottoms out, and that moves with every change to the data, the model or the learning rate.Reading noise as trend. With a small validation set, a wobble of a couple of percent between epochs is sampling noise. Wait for a sustained rise before concluding anything.

In one line

The gap between the curves is the diagnosis; either curve alone is not.

Learning curves: a different plot with a similar name

A learning curve plots performance against the amount of training data rather than against epochs, and it answers a question no other diagnostic does: would more data help?

Train on 10%, 20%, … 100% of the data and plot both scores:

Both curves converging to a high error means you are limited by the model, not the data. More rows will not help; more capacity or better features will.

A large remaining gap, with validation error still falling means you are limited by data. Collecting more is the highest-value action available.

Validation error flat as data grows

That distinction is worth an afternoon of compute before committing to an expensive annotation project.

A practical checklist

When a training run looks wrong, work through this in order:

  1. Can the model overfit 20 examples? If not, it is a bug — wrong labels, a broken loss, gradients not reaching the parameters.
  2. Is the initial loss about ln(C)? If not, the output layer or the labels are misaligned.
  3. Is the learning rate right? A short exponential sweep answers this in a minute.
  4. Are the inputs scaled? Unscaled features look like every other problem.
  5. Are gradients reaching the early layers? Log per-layer norms.
  6. Is the validation set representative and large enough? A few hundred examples give metrics that swing several percent.
  7. Is the data shuffled? Batches sorted by class break batch normalisation.

Most training failures are caught by the first four.

Six curves and what each one is telling you

Training curves are the main diagnostic you get, and most of their shapes have a specific cause. Here are six, generated from those causes, with the reading for each.

example_01.pyNumPy
Output

Questions people ask

How many epochs should I train for? Until validation stops improving, with early stopping deciding rather than a fixed number.

Should the training loss reach zero? On real data, no — noise sets a floor. Reaching zero usually means memorisation.

Why is my validation loss so noisy? A small validation set, or a high learning rate. Increase the set size or smooth over several epochs.

My training loss is lower than validation from the start. Overfitting? Not necessarily — some gap is normal from the first epoch. Watch whether it widens.

Should I plot loss or accuracy? Both. Loss is the smoother signal; accuracy is what you report, and the two can disagree informatively.

What if both curves are flat at exactly the same value? The model is predicting a constant. Check the learning rate, the initialisation, and whether the optimiser is stepping at all.

Recap in one screen

  • The level of the training curve measures bias; the gap between the curves measures variance.
  • Falling together is healthy; diverging is overfitting; both flat and high is underfitting.
  • A final-epoch improvement is the learning-rate schedule working — do not stop before it.
  • Log the learning rate, per-layer gradient norms and the task metric alongside the loss.
  • A learning curve against dataset size is what tells you whether more data would help.

Check yourself

0 of 3

Answer without scrolling back up.

  1. Training loss keeps falling while validation loss rises. The right reading is:

  2. Both curves are flat and high from the very first epoch. Most likely:

  3. Validation loss sits consistently *below* training loss. The usual explanation is:

Cheat sheet

Model Training Curves

Simulate and understand how Loss and Accuracy evolve over time. Tune the architecture mathematically to observe underfitting, ideal fits, and overfitting.

DEEP LEARNING · vizlearn.in/deep_learning/model_training_curve.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.