Two lines that tell you whether to collect more data or change the model — and they give different answers.
Overview
What is plotted
A learning curve answers a single question: will more data help? The answer saves real money. Collecting and labelling another 50,000 rows is expensive. If the model has already stopped improving, that cost is wasted. The curve tells you before you spend it.
Train the model on 40 examples, record the error on those examples and on a held-out validation set. Repeat with 80, 120, and so on. Plot both against training-set size. Grey is training error, orange is validation error.
The shape of the pair is the diagnosis, and the diagnosis is more useful than either number alone. A gap that keeps closing says more data will help. A gap that has already flattened says it will not. Two lines that sit high and close together say the model itself is the problem, not the data. This page walks through each of those shapes and what to do about them.
Learning Curves
This module needs JavaScript: the
numbers are computed in the page rather than recorded.
Worth knowing
Plot training and validation error against the number of training examples. The shape is the diagnosis.
Both high and converged: bias. The model is too simple, and more data will not help.
Low training error with a large gap: variance. More data or less capacity will help.
A validation curve that has flattened is the honest answer to 'should we collect more data'. It is no.
Learning Curves
The plot that answers whether to collect more data. Saves you from collecting it pointlessly.
Underfitting
Set room to 1 or 2. Both curves rise quickly to a plateau, sit close together, and stay high.
Training error is high. This Is the tell. The model can't fit the data it has already seen, so nothing about unseen data is the problem. This is bias: the model is too simple for the structure present.
The key consequence is minus. More data won't help. The curves have already settled. Adding examples moves neither. Money spent on collection here is wasted, and this plot is the cheapest way to find that out before spending it.
What helps instead: more room, better features, less smoothing, a different model class.
Overfitting
Set room to 8 or 9. Training error drops close to zero, validation error stays well above it, and a wide gap opens between them.
The model fits its training data nearly just right and generalises poorly. This is variance: capacity is being spent memorising noise.
Here more data *does* help, and the curve says so. The validation line is still falling at the right-hand edge. each extra example makes the noise harder to store. Other options: reduce room, add smoothing, boost the data, or ensemble.
About right
Set room to 4 or 5. The curves settle to a small gap at a low error, and both have flattened.
The readout says as much. There's no obvious bias and no obvious variance, and the flattening is the answer to "should we collect more data": no, not for this model. Improvement would have to come from features or from a different model class, not from volume.
The summary
Training error
Gap
Diagnosis
Do
High
Small
Bias
more capacity, better features
Low
Large
Variance
more data, regularisation, less capacity
Low
Small
Fine
features or a different model
High
Large
Something is wrong
check the split and the labels
That last row is worth keeping. High training error *and* a large gap shouldn't happen from bias or variance alone, and often means a bug — leakage in reverse, mislabelled data, or a validation set drawn from a different spread.
Where the curve is still falling
The single most valuable thing here is being able to answer "will more data help" with evidence.
If the validation curve is still descending at your current dataset size, collecting more will improve the model, and the slope gives a rough sense of how much. If it has been flat for the last few points, more data buys nothing, and the effort belongs elsewhere.
That question is or else answered by intuition, and it often involves real money.
Practical notes
The curves are noisy at small sizes. A model trained on 40 examples depends heavily on which 40. Mean over a few splits, and plot the spread if you can.
Use the same validation set at each size. Growing it alongside the training set confounds two changes.
And don't confuse this with a validation curve. This Plots error against a setting at fixed data size. Both are handy and they answer different questions: learning curves ask about data, validation curves ask about a setting.
Reading the four shapes
The diagnosis is the *pair* of curves, not either number, and four combinations cover each case:
training error
gap to validation
diagnosis
what helps
high
small
bias (underfit)
more capacity, better features
low
large
variance (overfit)
more data, regularisation, less capacity
low
small
about right
features or a different model, not volume
high
large
a bug
check the split, the labels, for leakage
The one that saves real money is the top row against the second. Both look like a model that could be better. But They answer "should we collect more data" in opposite directions. In the bias case both curves have already flattened and sit close together and high — the model can't even fit what it has, so more of the same data moves nothing, and the plot is the cheapest way to learn that before paying for collection. In the variance case the validation curve is still falling at the right-hand edge, so each extra example makes the noise harder to store and the gap narrows. The rule is precise: a *settled* gap is variance you can't buy your way out of. A still-descending validation curve is the one that justifies the data budget.
Two hands-on cautions keep the diagnosis honest. The curves are noisy at small training sizes, because a model fitted on 40 examples depends heavily on *which* 40, so mean over a few splits before reading a shape into them. And use the same validation set at each training size. Growing it alongside the training set confounds two changes and muddies the very gap you're trying to read. The learning curve, which varies the amount of data, is also not to be confused with a validation curve. This Varies a setting at fixed data size: both are handy and they answer different questions, one about collection and one about a setting.
Where it goes wrong with Learning Curves
Reading a single run. Small-sample noise looks like structure.
Plotting fit on imbalanced data. The curves will be flat and uninformative. Plot the metric you really care about.
Concluding 'more data' from a gap alone. Check the validation curve is still falling. A settled gap means variance you can't fix with volume.
Questions people ask about Learning Curves
What does a learning curve show? Training and validation scores as the training set grows. It tells you whether more data, a bigger model or a simpler model is likely to help.
How do I recognise overfitting? Training score stays high while validation score sits well below it. A wide gap that does not close with more data points to overfitting.
How do I recognise underfitting? Both scores are low and close together. More data will not help, so try a more flexible model or better features.
Will more data help? If the validation curve is still rising at the right edge, yes. If it has flattened, more data of the same kind will give little.
How is this different from a validation curve? A learning curve varies training set size. A validation curve varies one hyperparameter, such as tree depth, at a fixed size.
Why are the curves noisy? Each point comes from a finite cross-validation run on a small subset. Use more folds or repeats, and look at the shaded spread.
The chart that tells you whether more data will help
Two models on the same problem, scored as the training set grows. One curve says collect more data. The other says don't bother. Reading the difference is the whole skill.
example_01.pyscikit-learn
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import learning_curve, StratifiedKFold
from sklearn.linear_model import LogisticRegression
from sklearn.tree import DecisionTreeClassifier
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler
X, y = make_classification(n_samples=4000, n_features=12, n_informative=6,
n_clusters_per_class=2, flip_y=0.08, random_state=0)
cv = StratifiedKFold(5, shuffle=True, random_state=0)
sizes = np.array([50, 100, 250, 600, 1500, 3200])
def curve(name, model, verdict):
n, tr, te = learning_curve(model, X, y, train_sizes=sizes, cv=cv,
scoring="accuracy")
print(name)
print("%10s %10s %10s %8s" % ("rows", "train", "val", "gap"))
for i in range(len(n)):
print("%10d %10.4f %10.4f %8.4f"
% (n[i], tr[i].mean(), te[i].mean(), tr[i].mean()-te[i].mean()))
print(" verdict: %s" % verdict)
print()
curve("UNDERFITTING -- linear on a non-linear problem",
make_pipeline(StandardScaler(), LogisticRegression(max_iter=2000)),
"both low and flat. more data changes nothing.")
curve("OVERFITTING -- unbounded decision tree",
DecisionTreeClassifier(random_state=0),
"train near 1.0, val far behind and still rising. more data helps.")
Output
The next block adds a well-fitted model and summarises the three shapes.
example_02.pyscikit-learn
import numpy as np
from sklearn.datasets import make_classification
from sklearn.model_selection import learning_curve, StratifiedKFold
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import StandardScaler, PolynomialFeatures
X, y = make_classification(n_samples=4000, n_features=12, n_informative=6,
n_clusters_per_class=2, flip_y=0.08, random_state=0)
cv = StratifiedKFold(5, shuffle=True, random_state=0)
sizes = np.array([50, 100, 250, 600, 1500, 3200])
def curve(name, model, verdict):
n, tr, te = learning_curve(model, X, y, train_sizes=sizes, cv=cv,
scoring="accuracy")
print(name)
print("%10s %10s %10s %8s" % ("rows", "train", "val", "gap"))
for i in range(len(n)):
print("%10d %10.4f %10.4f %8.4f"
% (n[i], tr[i].mean(), te[i].mean(), tr[i].mean()-te[i].mean()))
print(" verdict: %s" % verdict)
print()
curve("GOOD FIT -- degree-2 features on a linear model",
make_pipeline(StandardScaler(), PolynomialFeatures(2),
LogisticRegression(max_iter=5000)),
"small gap, val flat. you have enough data.")
print("the three shapes:")
print(" both low, together -> underfitting. get a better model.")
print(" big gap, val still rising -> overfitting. get more data or regularise.")
print(" small gap, both flat -> done. remaining error is label noise.")
print()
print("plot this BEFORE collecting data. the curve answers 'would another")
print("10,000 rows help' using data you already have.")
Output
Check yourself
0 of 3
Answer without scrolling back up.
Training error is high and the two curves have converged. What does that mean?
The model cannot fit data it has already seen, so unseen data is not the problem. The curves have already converged, so adding examples moves neither.
What tells you whether collecting more data will help?
A still-descending validation curve means more examples will improve the model, and its slope suggests by how much. A flat one means the effort belongs elsewhere.
High training error AND a large gap suggests what?
Bias and variance alone do not produce that combination. It usually means mislabelled data or a validation set drawn from somewhere else.
Cheat sheet
Learning Curves
Plot training and validation error against dataset size. Read the gap to decide whether more data or more model capacity will help.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.