Explicitly control Bias and Variance to understand how models generalize. Use the Resample button to visualize the instability of High Variance models!
Bias describes systematic error from a model that is too restrictive; variance describes how much its predictions change across training samples. The distinction helps explain why a model can underfit or overfit, and why adding complexity is not always an improvement.
Imagine training the same kind of model on many different samples from one population. Bias is how far its average prediction is from the target pattern. Variance is how much those fitted predictions differ from one another. A rigid straight line trying to fit a curved relationship may have high bias. A very flexible curve that changes sharply whenever the sample changes may have high variance.
For squared-error regression, under the usual assumptions, expected prediction error can be written as:
Expected squared error = bias² + variance + irreducible noise
Irreducible noise is variation in the outcome that the available features cannot explain. This decomposition is not a universal formula for every model or metric; it is a way to reason about expected squared error.
| Pattern | Likely diagnosis | What to consider |
|---|---|---|
| Both errors are high and close | Underfitting, often high bias | Add useful features or model flexibility; reduce excessive regularisation. |
| Training error is low but validation error is much higher | Overfitting, often high variance | Simplify, regularise, gather representative data, or use cross-validation. |
| Both are low and close | The model generalises well on this split | Check whether the validation data represents deployment. |
These patterns are clues, not proof. Data leakage, a distribution shift, a small validation set, or a mismatched metric can also create misleading curves. See learning curves for how error changes as training data grows.
The visualiser is a teaching model, not a diagnostic estimator. In a real workflow, compare training and validation performance using the same split strategy and a metric suited to the task.
More flexible models can represent a wider range of patterns, which may reduce bias. The same flexibility can also fit random quirks in the training data, increasing variance. Regularisation, model selection and additional representative data change this balance; none guarantees that both sources of error will fall at once.
Continue with learning curves to diagnose whether more data may help, and cross-validation to compare models across multiple splits.
About to run: set Data Noise to its minimum (0.0). Before it does — what happens to the readout?
Committing to an answer first is the point — the reveal runs the experiment on the visualisation above and reads the real value back, so nothing here is scripted.
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “The two sources of model error”?
What does this module say about “Read the training and validation errors”?
What does this module say about “Explore resampling”?
Dial bias and variance up and down, resample the data, and watch a stable underfit turn into an unstable overfit.