A statistic computed from a sample is itself random. Its spread has a name, a formula, and a square root that governs every survey ever run.
Overview
The statistic is random too
You draw a sample and find its mean. Draw a different sample and you get a different mean. The statistic is itself a random quantity, and it has a spread: the sampling spread.
This is the conceptual step all in inference rests on. It's also the one that is easy to skip, because you only ever draw one sample and never see the spread at all. You reason about what *would* happen across the samples you didn't take.
The left panel makes it visible by really doing it: 800 samples drawn, each mean plotted. That histogram is the sampling distribution of the mean, and it's the object trust intervals show.
Sampling Distributions and Standard Error
This module needs JavaScript: the
numbers are computed in the page rather than recorded.
Worth knowing
A sampling distribution is the distribution of a statistic across every sample you could have drawn.
The standard error is its standard deviation. For a mean it is σ/√n.
Standard deviation describes the data. Standard error describes an estimate. They are not interchangeable.
The right-hand curve is why precision is expensive: it flattens, and every further gain costs four times the data.
Sampling Distributions and Standard Error
The idea that makes each trust interval and each p-value mean a thing.
Standard error
The standard deviation of a sampling spread is the standard error. For a sample mean:
SE = sigma / sqrt(n)
This follows right from the rules for variance: the variance of a sum of n free draws is n * sigma^2, cutting by n to make the mean scales variance by 1/n^2, leaving sigma^2 / n, and the square root gives the formula.
Watch the readout as you drag n: the seen spread of the 800 means tracks the predicted sigma / sqrt(n) closely. That isn't a coincidence being illustrated — it's an identity being checked.
Standard deviation is not standard error
These get confused constantly, and the gap isn't subtle.
Standard deviation shows the spread of the data. It's a property of the group, and collecting more data doesn't reduce it — more careful reading might, but more readings won't.
Standard error shows the spread of an guess. It shrinks as n grows, because a larger sample pins the guess down better.
A handy test: if the number would change when you collect more data *of the same kind*, it's a standard error. If it wouldn't, it's a standard deviation.
An error bar on a chart could be either, and the two say fully different things. Charts that don't label which are unreadable.
The square root, again
The right-hand panel plots sigma / sqrt(n) against n, and its shape is the hands-on content of this page.
It falls steeply at first. Going from 4 samples to 16 halves the error. Then it flattens. Going from 100 to 400 halves it again, at the cost of three hundred more readings.
each halving of uncertainty costs four times the data. That is why national polls settle around 1,000 respondents: the marginal precision per person merges past that point, and the left error is dominated by sampling bias rather than sampling noise anyway.
More data can't fix a biased sample. A poll of 100,000 people who all answer the same badly-worded question is just wrong.
Beyond the mean
each statistic has a sampling spread, not only the mean. Medians, variances, links and regression coeffs all have one, and the formulas are often harder or down.
The bootstrap exists for just this. Resample your data with swap, recompute the statistic, repeat thousands of times, and the spread of the results guesses the sampling spread — with no formula and almost no assumptions. It's the same op this page performs, run on your own sample instead of on a known group.
Where it shows up
trust intervals are built right from the standard error.
t-tests and z-tests compare an seen difference against the standard error of that difference.
Cross-validation variance is the sampling distribution of a speed guess, which is why a single fold's fit is a point with error bars around it.
A/B test duration is a standard error math before it's any of it else.
The cost of precision, tabulated
The sigma / √n formula has a consequence worth seeing as a table, because it governs how big each survey and each A/B test has to be. With a group standard deviation of 10:
sample size n
standard error
4
5.00
16
2.50
64
1.25
256
0.625
Each row quadruples n and the standard error halves — the √n in the denominator made literal. Going from 4 to 16 buys a lot. Going from 64 to 256 buys the same *factor* of improvement for four times the abs cost. Precision has diminishing returns, and they diminish predictably: to add one more decimal place of fit you need a hundred times the data. This is the whole reason national polls settle near a thousand respondents rather than ten thousand — past that point each extra person shrinks the error by too little to be worth collecting, and any left error is bias. This More data can't touch.
The table also draws the line between the two quantities people confuse. The group's own spread — its standard deviation — is 10 in each row; collecting more data never moves it. The standard error is the column that shrinks, because it shows the *guess*, not the data. A handy test: if a number would change when you collect more readings of the same kind, it's a standard error; if it wouldn't, it's a standard deviation. An error bar that doesn't say which it's could mean either, and the two tell fully different stories.
Where it goes wrong
Reporting SD when you mean SE, or the reverse. Label your error bars.
given freedom. The sqrt(n) assumes free draws. Clustered or time-series data has a smaller effective n.
Treating a large sample as accurate. It's precise. Fit is a separate question, and bias doesn't shrink with n.
Using the formula for statistics that don't have one. Bootstrap instead.
Confusing the spread of the data with the spread of the statistic. The data can be wildly scattered and the mean can still be pinned down tightly, as long as the sample is large enough. The two spreads answer different questions.
Questions people ask
What is a sampling distribution? It is the distribution of a statistic, such as the sample mean, across many repeated samples from the same population.
What is the standard error? The standard deviation of a statistic's sampling distribution. For the mean it is sigma divided by the square root of n.
How is standard error different from standard deviation? Standard deviation describes the spread of the data. Standard error describes how much an estimate would move between samples, and it shrinks as n grows.
Why does a bigger sample help only slowly? Because of the square root. Four times the data halves the error, and a hundred times the data cuts it by ten.
Can I get a sampling distribution without theory? Yes, with the bootstrap. Resample your data with replacement many times and look at the spread of the statistic.
Does this apply to statistics other than the mean? Yes, every statistic has one, including the median, a correlation or a model coefficient. Their shapes and formulas differ.
The distribution of a statistic, not of the data
Draw a sample, find its mean, write it down, repeat ten thousand times. What you get is a different spread from the one you sampled.
example_01.pyNumPy
import numpy as np
rng = np.random.default_rng(0)
# the population: heavily skewed, nothing like a bell curve
population = rng.exponential(scale=10.0, size=1_000_000)
print("population: mean %.3f, sd %.3f, median %.3f"
% (population.mean(), population.std(), np.median(population)))
print(" it is skewed -- 90th percentile is %.1f, max is %.1f"
% (np.percentile(population, 90), population.max()))
print()
for n in (2, 10, 50, 200):
means = population[rng.integers(0, len(population), size=(10_000, n))].mean(axis=1)
predicted_se = population.std() / np.sqrt(n)
print("samples of %3d: mean of means %6.3f sd of means %6.3f "
"(predicted %6.3f)" % (n, means.mean(), means.std(), predicted_se))
print()
print("two separate facts here:")
print(" the sampling distribution centres on the population mean;")
print(" its spread shrinks like 1/sqrt(n), so 4x the data halves the error.")
print()
print("that sd of the means is what 'standard error' means. it is not the")
print("spread of your data -- it is the spread of the answer you computed.")
Output
Check yourself
0 of 3
Answer without scrolling back up.
What is a sampling distribution?
You only ever draw one sample, so you reason about what would happen across the samples you did not take. It is the object a confidence interval describes.
Which shrinks as you collect more data?
SD describes the data and is a property of the population. SE describes an estimate. If the number would change when you collect more data of the same kind, it is a standard error.
Why do national polls settle around 1,000 respondents?
Every halving of uncertainty costs four times the data. Past that point the remaining error is dominated by sampling bias, which more data cannot fix.
Cheat sheet
Sampling Distributions and Standard Error
Simulate thousands of samples and plot the distribution of the sample mean. Watch the standard error shrink as the sample size grows.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.