Confidence Intervals

Draw a thousand intervals from a known population and count how many contain the answer. The confidence is a property of the procedure.

Confidence Intervals

This module needs JavaScript: the numbers are computed in the page rather than recorded.

Worth knowing

A 95% interval does not mean the parameter is 95% likely to be inside your interval. It is inside or it is not.
It means the procedure produces intervals that contain the parameter 95% of the time.
The width is z · SE, so it shrinks like √n.
Higher confidence buys a wider interval and nothing else. A 99% interval says less about where the answer is, not more.

Confidence Intervals

What the ninety-five percent actually refers to, demonstrated by drawing a thousand of them.

The construction

Take a sample, compute its mean, and put an interval around it:

mean  +/-  z * (sample SD / sqrt(n))

The z depends on the confidence level — 1.96 for 95%. The rest is the [standard error](sampling_distributions.html).

What the confidence refers to

Here is the statement people usually give: *there is a 95% probability the true mean lies in this interval.*

That is wrong, and the visualisation shows why. The population mean is fixed at 10 — the dashed line. It is not random. Your interval is either side of it or it is not; there is no probability left once both are determined.

What is random is the interval, because it is computed from a random sample. Draw again and you get a different one. The 95% describes how often that procedure lands on the truth.

The panel draws forty intervals, from a thousand generated. Most cross the dashed line; the highlighted few do not. Nothing distinguishes a missing interval from a covering one when you can only see it from the inside — and in practice you have exactly one, and no idea which kind you have.

The count is real

The readout gives the actual coverage of all thousand. At n = 30 with a 95% level it lands near 94%, and every part of that gap is instructive.

Drag n down to 5 and coverage falls to about 89%. That is not sampling noise. This page uses a z critical value with the sample's *estimated* standard deviation, and at small n that estimate is unreliable enough that the interval comes out too narrow.

That is exactly why the t-distribution exists. Gosset's correction widens the interval to account for having estimated the SD, and the correction is large at small n and negligible past about 30. Drag n upward and watch the gap close on its own — the page is demonstrating the problem the t-distribution solves, rather than quietly using t and hiding it.

Width and what buys it

Two things set the width.

The confidence level. Switch from 95% to 99% and the intervals lengthen noticeably. A higher level does not locate the parameter better; it hedges more. A 100% interval would be "somewhere between minus infinity and infinity" — perfectly reliable and perfectly useless.

The sample size, through sqrt(n). Quadrupling the sample halves the width.

Reading one properly

"Between 42% and 48%" means the procedure that produced it covers the truth 95% of the time.

A wide interval is a result. It says the data does not pin the answer down, which is genuine information rather than a failed experiment.

Overlapping intervals do not imply no difference. Two 95% intervals can overlap while a direct test of their difference is significant. Test the difference; do not eyeball the bars.

An interval containing zero corresponds to a two-sided test not rejecting at that level. That correspondence is exact, and it is why many people prefer reporting intervals to reporting [p-values](hypothesis_testing_and_p_values.html): the interval carries the effect size and the uncertainty together, where a p-value carries neither.

Where it goes wrong

Saying "95% chance the mean is in here." The parameter is fixed; the interval is random.

Using z at small n. Use t, and this page shows you the cost of not.

Assuming independence. Clustered or time-series data has a smaller effective sample size, and the interval will be too narrow.

Reporting an interval for a biased estimator. Coverage is about sampling variability. A systematically wrong measurement produces a tight interval around the wrong answer.

Check yourself

0 of 3

Answer without scrolling back up.

  1. What does the 95% in a 95% confidence interval describe?

  2. Why does a nominal 95% interval cover only about 89% at n = 5 here?

  3. Two 95% intervals overlap. What follows?

Cheat sheet

Confidence Intervals

That is wrong, and the visualisation shows why. The population mean is fixed at 10 — the dashed line. It is not random. Your interval is either side of it or it is not; there is no probability left once both are determined.

MATHS · vizlearn.in/maths/confidence_intervals.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.