Draw a thousand intervals from a known population and count how many contain the answer. The confidence is a property of the procedure.
Overview
The construction
Take a sample, find its mean, and put an interval around it:
mean +/- z * (sample SD / sqrt(n))
The z depends on the trust level. 1.96 for 95%. The rest is the standard error.
What the confidence refers to
Here is the statement people often give: *there's a 95% chance the true mean lies in this interval.*
That is wrong, and the visualisation shows why. The group mean is fixed at 10 — the dashed line. It isn't random. Your interval is either side of it or it isn't. There's no chance left once both are found.
What is random is the interval, because it's found from a random sample. Draw again and you get a different one. The 95% shows how often that method lands on the truth.
The panel draws forty intervals, from a thousand made. Most cross the dashed line. The highlighted few don't. Nothing sets apart a missing interval from a covering one when you can only see it from the inside — and you've just one, and no idea which kind you've.
Confidence Intervals
This module needs JavaScript: the
numbers are computed in the page rather than recorded.
Worth knowing
A 95% interval does not mean the parameter is 95% likely to be inside your interval. It is inside or it is not.
It means the procedure produces intervals that contain the parameter 95% of the time.
Higher confidence buys a wider interval and nothing else. A 99% interval says less about where the answer is, not more.
Confidence Intervals
What the ninety-five percent really refers to, shown by drawing a thousand of them.
The count is real
The readout gives the actual coverage of all thousand. At n = 30 with a 95% level it lands near 94%, and each part of that gap is instructive.
Drag n down to 5 and coverage falls to about 89%. That isn't sampling noise. This page uses a z key value with the sample's *guessed* standard deviation, and at small n that estimate is unreliable enough that the interval comes out too narrow.
That is just why the t-spread exists. Gosset's correction widens the interval to account for having guessed the SD, and the correction is large at small n and negligible past about 30. The multiplier it puts in place of 1.96 is worth seeing, because it shows both halves of that sentence:
n
t multiplier
wider than z by
5
2.776
42%
10
2.262
15%
30
2.045
4%
100
1.984
1%
At n = 5 the honest interval is nearly half as wide again as the one this page draws, which is where the missing coverage went. By n = 30 the correction is 4% and the gap stops mattering. This Is where the rule of thumb comes from. Drag n upward and watch the gap close on its own — the page is showing the problem the t-spread solves, rather than quietly using t and hiding it.
Width and what buys it
Two things set the width.
The trust level. Switch from 95% to 99% and the intervals lengthen noticeably. A higher level doesn't locate the param better. It hedges more. A 100% interval would be "a place between minus infinity and infinity" — just right solid and just right useless.
The sample size, through sqrt(n). Quadrupling the sample halves the width.
Reading one properly
"Between 42% and 48%" means the method that made it covers the truth 95% of the time.
A wide interval is a result. It says the data doesn't pin the answer down, which is genuine info rather than a failed test.
shared intervals don't imply no difference. Two 95% intervals can share while a direct test of their difference is major. Test the difference. Don't eyeball the bars.
An interval holding zero matches to a two-sided test not rejecting at that level. That correspondence is exact, and it's why many people prefer reporting intervals to reporting p-values: the interval carries the effect size and the uncertainty together. Here A p-value carries neither.
Two intervals that overlap, and a difference that is real
The rule above — don't eyeball the bars — is worth doing rather than taking on trust, because the math is short and the result is truly counter-natural.
An test with 100 readings in each arm and a standard deviation of 10, so the standard error in each arm is 10 / √100 = 1.
arm
mean
95% interval
A
10.0
[8.04, 11.96]
B
13.2
[11.24, 15.16]
Those intervals share, between 11.24 and 11.96. Draw them as error bars and each instinct says the arms are the same.
Now test the difference right. The difference is 3.2, and its standard error is not the two standard errors added:
SEdiff = √(1² + 1²) = √2 = 1.414, not 2
which gives a 95% interval for the difference of 3.2 ± 1.96 × 1.414 = [0.43, 5.97]. That excludes zero, and the matching two-tailed p is 0.024. The difference is major while the single intervals share.
The reason is in that square root. Two free errors don't both go the wrong way at once as often as they each go wrong. They partly cancel, and the difference is pinned down more tightly than either mean is. Adding the errors assumes the worst case each time.
The consequence is a whole band where the eye and the math disagree. With equal standard errors, a difference turns major once it exceeds 1.96√2 = 2.77 standard errors, but the two intervals keep touching until it exceeds 2 × 1.96 = 3.92. Any gap between those two numbers shares *and* is major.
gap between the means
bars overlap?
significant?
2.0 SE
yes
no
3.2 SE
yes
yes
4.5 SE
no
yes
The middle row is the one that catches people, and it's why the check you report has to be a test of the difference rather than a picture of two intervals side by side. The picture is answering a question no one asked.
Where it goes wrong with Confidence Intervals
Saying "95% chance the mean is in here." The param is fixed. The interval is random.
Using z at small n. Use t. This page shows you the cost of not.
given freedom. Clustered or time-series data has a smaller effective sample size, and the interval will be too narrow.
Reporting an interval for a biased estimator. Coverage is about sampling variability. A systematically wrong reading makes a tight interval around the wrong answer.
Questions people ask about Confidence Intervals
What does a 95 percent confidence interval mean? If you repeated the study many times and built an interval each time, about 95 percent of those intervals would contain the true value. It is a statement about the procedure.
Does it mean a 95 percent chance the parameter is inside? Not strictly. Once computed, the interval either contains the true value or does not. The probability describes the method over repeats.
How can I make an interval narrower? Collect more data, reduce the noise in your measurements, or accept a lower confidence level. Width shrinks with the square root of n.
What is the formula for a mean? The sample mean plus or minus a critical value times the standard error. The critical value is about 1.96 for 95 percent with a large sample.
Do overlapping intervals mean no difference? Not necessarily. Two intervals can overlap and the difference between the means can still be significant. Build an interval for the difference directly.
When should I use a t value rather than 1.96? When the sample is small and you estimate the standard deviation from it. The t distribution has heavier tails, so the interval is wider.
What "95% confident" actually counts
Build a trust interval a thousand times from a known group and count how often it holds the true mean. The number the interval shows is the method, not your one result.
example_01.pyNumPy
import numpy as np
rng = np.random.default_rng(0)
TRUE_MEAN, TRUE_SD, N = 100.0, 15.0, 40
def interval(sample):
se = sample.std(ddof=1) / np.sqrt(len(sample))
half = 1.96 * se
return sample.mean() - half, sample.mean() + half
print("one sample of %d from a population with mean %.0f:" % (N, TRUE_MEAN))
s = rng.normal(TRUE_MEAN, TRUE_SD, N)
lo, hi = interval(s)
print(" sample mean %.2f, 95%% interval (%.2f, %.2f)" % (s.mean(), lo, hi))
print(" contains the true mean:", lo <= TRUE_MEAN <= hi)
print()
hits = 0
trials = 10_000
widths = []
for _ in range(trials):
s = rng.normal(TRUE_MEAN, TRUE_SD, N)
lo, hi = interval(s)
hits += (lo <= TRUE_MEAN <= hi)
widths.append(hi - lo)
print("%d intervals built the same way:" % trials)
print(" %.2f%% of them contained the true mean" % (100 * hits / trials))
print(" average width %.2f" % np.mean(widths))
print()
print("any single interval contains the mean or it does not -- no 95%% about")
print("it once computed. the 1.96 multiplier assumes a known sd; estimating")
print("it from %d points costs coverage. the t distribution corrects that:" % N)
t39, hits_t = 2.0227, 0
for _ in range(trials):
s2 = rng.normal(TRUE_MEAN, TRUE_SD, N)
half = t39 * s2.std(ddof=1) / np.sqrt(N)
hits_t += abs(s2.mean() - TRUE_MEAN) <= half
print(" with t instead of 1.96: %.2f%% coverage" % (100 * hits_t / trials))
print()
print("more data makes the interval narrower, at the usual 1/sqrt(n) rate:")
for n in (10, 40, 160, 640):
ws = [np.diff(interval(rng.normal(TRUE_MEAN, TRUE_SD, n)))[0] for _ in range(400)]
print(" n=%4d average width %.2f" % (n, np.mean(ws)))
Output
Check yourself
0 of 3
Answer without scrolling back up.
What does the 95% in a 95% confidence interval describe?
The parameter is fixed - it is inside or it is not. What is random is the interval, because it is computed from a random sample.
Why does a nominal 95% interval cover only about 89% at n = 5 here?
The correction is large at small n and negligible past about 30. Dragging n upward closes the gap on its own.
Two 95% intervals overlap. What follows?
Overlap of separate intervals is not a test of their difference. Test the difference directly rather than eyeballing the bars.
Cheat sheet
Confidence Intervals
Draw a thousand confidence intervals and count how many capture the true mean. See what the ninety-five percent actually refers to.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.