Hypothesis Testing and p-values

The p-value is a shaded area under an assumption. Move the observation and watch the area, and the assumption, stay exactly where they were.

Overview

The machinery

The null hypothesis is a specific, boring claim: no effect, no difference, nothing happening. It is specific on purpose, because a specific claim implies a distribution for the test statistic.

That distribution is the curve. It is what the statistic would look like across repeated samples if the null were true. Every use of a p-value inherits that conditional.

The p-value is the shaded tail: the probability of a statistic at least as extreme as the one observed, under the null.

Drag the observed statistic and watch the area shrink as it moves outward. A large statistic is unusual under the null, so its tail is small — and a small tail is what "unlikely, if the null were true" means.

Hypothesis Testing and p-values

This module needs JavaScript: the numbers are computed in the page rather than recorded.

Worth knowing

The curve is the distribution of the test statistic assuming the null hypothesis is true. Everything else follows from that.
The p-value is the shaded area: the probability of a result at least this extreme, if the null held.
It is not the probability the null is true, and it is not the probability your result was a fluke.
0.05 is a convention Fisher suggested as a rule of thumb, not a property of nature.

Hypothesis Testing and p-values

The most used and most misread number in statistics, defined precisely enough to stop misreading it.

What it is not

Three misreadings, all common enough to have names.

Not the probability that the null is true. The p-value is computed *assuming* the null. It cannot then tell you how likely the assumption was. Getting from P(data | null) to P(null | data) needs [Bayes' theorem](bayes_theorem.html) and a prior, and the two quantities can differ by orders of magnitude.

Not the probability the result was chance. Same error, differently worded.

Not a measure of effect size. With enough data, a difference of no practical consequence produces a tiny p-value. The p-value answers "is there evidence of *any* effect", never "is the effect large enough to matter". This is why [confidence intervals](confidence_intervals.html) are increasingly preferred: they carry the size and the uncertainty in one object.

Failing to reject is not accepting

Set the observed statistic near zero. The p-value is large, and the conclusion is that the data is consistent with the null.

It is not that the null is true. A large p-value is produced both by a real absence of effect and by a study too small to detect one. Distinguishing those requires [power](type_i_and_type_ii_errors.html), and it is why "no significant difference" in an underpowered study means almost nothing.

One tail or two

Switch the alternative. The two-tailed p-value is exactly double the one-tailed one for the same statistic, because the same area is being counted on both sides.

The choice must be made before seeing the data. Deciding afterwards that you only cared about one direction halves your p-value for free, and that is a recognised form of cheating rather than a modelling choice.

Why 0.05 is under attack

The threshold is a convention. Fisher suggested it as a rough guide and explicitly did not intend it as a decision rule.

The problems are structural.

p-hacking. Test enough hypotheses and something crosses 0.05 by chance. At 0.05, one test in twenty does so with no effect present at all.

Publication bias. Significant results get published, so the literature over-represents them, and the effect sizes it reports are inflated.

The cliff. p = 0.049 and p = 0.051 are indistinguishable as evidence and are treated as opposites.

The responses in circulation: report exact p-values rather than thresholds, report effect sizes and intervals alongside, pre-register the analysis, and correct for multiple comparisons. Some fields have moved the threshold to 0.005; others argue for abandoning the dichotomy entirely.

Where it goes wrong

Reading p as the probability the null is true. The central error.

Treating "not significant" as "no effect." Check the power.

Choosing the tail after seeing the data.

Running many tests without correction. Bonferroni is crude and better than nothing; false discovery rate control is usually the better tool.

Check yourself

0 of 3

Answer without scrolling back up.

  1. A p-value is the probability of:

  2. A study reports p = 0.4. What can you conclude?

  3. Why must the choice of one or two tails be made before seeing the data?

Cheat sheet

Hypothesis Testing and p-values

The p-value is a shaded area under an assumption. Move the observation and watch the area, and the assumption, stay exactly where they were.

MATHS · vizlearn.in/maths/hypothesis_testing_and_p_values.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.