vizlearn

The Central Limit Theorem

Maths for Machine LearningBy Updated

Sample from a population that looks nothing like a bell curve. The averages come out as one anyway.

Overview

What the theorem says

Take repeated samples of size n from any population with a finite mean and variance. Compute each sample's mean. Those means have a distribution of their own, and as n grows it approaches a normal distribution — with mean equal to the population mean and standard deviation sigma / sqrt(n).

The population's own shape does not appear in that statement anywhere. It can be skewed, bimodal, discrete, or wildly irregular.

The population above is deliberately unpleasant: two separate humps and a long tail. Nothing about it is bell-shaped. Set n to 1 and the right-hand panel is that same shape, because a sample of one *is* a draw from the population.

Now drag n upward. By 5 the two humps have merged. By 30 the right-hand panel is a clean bell, and the fitted normal curve sits on top of the resampled histogram. Nothing about the population changed.

The Central Limit Theorem

This module needs JavaScript: the numbers are computed in the page rather than recorded.

Worth knowing

The theorem is about the distribution of the sample mean, not about the population and not about the sample.
Whatever the population's shape, the sample means tend to normal as n grows — provided the population has a finite variance.
The spread of the means is σ/√n, which is why quadrupling n only halves the uncertainty.
At n = 1 the means are just the population. Drag n up and watch the shape reorganise itself.

The Central Limit Theorem

Why the normal distribution turns up everywhere, stated precisely enough to know when it does not.

Three things it is not

Not "large samples are normally distributed." The sample is not normal; it looks like the population, because that is what it is drawn from. It is the *mean* that goes normal.

Not "everything is normal." Heights are roughly normal because they are the sum of many small independent influences. Incomes are not normal and no amount of data makes them so.

Not a licence to ignore the population. It describes the sampling distribution of a statistic, not the data.

The square root

The standard deviation of the sample mean — the standard error — is sigma / sqrt(n). The square root is the practical heart of the theorem.

To halve your uncertainty you must quadruple your sample. To divide it by ten you need a hundred times the data. Precision gets expensive fast, and this single fact governs how large a survey has to be, how long an A/B test must run and why polls stubbornly report margins of around three percent.

[The sampling distribution module](sampling_distributions.html) plots that decay directly.

How large is large enough

The usual rule of thumb is n = 30, and like all such rules it is a summary of something more specific.

Near-symmetric population: convergence is fast, and 10 may do.

Strongly skewed: 30 is not enough. Heavily skewed populations can need hundreds before the mean is convincingly normal.

Heavy tails: worse still, and there is a limit case below.

The honest procedure is the one on this page: resample and look. Drag n and find where the histogram stops looking lumpy for *your* population.

When it fails outright

The theorem requires a finite variance, and that condition is not decorative.

The Cauchy distribution has none — nor a finite mean. Average n Cauchy draws and you get another Cauchy, with exactly the same spread as one draw. No amount of averaging concentrates anything. The sample mean of a million draws is no better than the first.

Heavy-tailed distributions that do have a finite variance still converge, but slowly enough to matter. Financial returns are the standard cautionary example: methods that assume normality of averages understate the chance of extreme outcomes, because convergence has not really happened at the sample sizes in use.

Where it shows up

Confidence intervals and t-tests assume the sampling distribution of the mean is normal. That assumption is this theorem.

A/B testing: a conversion rate is a mean of Bernoulli draws.

Bootstrapping exists partly to avoid needing it when it does not hold.

Measurement error is often modelled as normal because it is the sum of many small independent errors.

Where it goes wrong

Applying it to the data instead of the statistic. Extremely common.

Trusting n = 30 on skewed data. Check.

Assuming it applies to the maximum or the variance. It is about sums and means. Extreme values have their own limit theory, and a different family of limiting distributions.

Ignoring dependence. The classical version needs independent draws. Time series data usually is not independent, and the effective sample size is smaller than the count.

Check yourself

0 of 3

Answer without scrolling back up.

  1. What exactly becomes normal?

  2. You want to halve your uncertainty about a mean. What must you do?

  3. For which population does the theorem fail entirely?

Cheat sheet

The Central Limit Theorem

Take repeated samples of size n from any population with a finite mean and variance. Compute each sample's mean. Those means have a distribution of their own, and as n grows it approaches a normal distribution — with mean equal to the population mean and standard deviation sigma / sqrt(n).

MATHS · vizlearn.in/maths/central_limit_theorem.html

More in Maths

MathsQuantiles and PercentilesMathsSampling Distributions and Standard ErrorMathsConfidence IntervalsMathsHypothesis Testing and p-values

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.

About VizLearnGitHubLinkedInContact