Roll dice thousands of times and watch the messy experimental frequencies settle onto the exact theoretical answer. Probability is what happens in the long run — not what happens next.
The chart compares the experimental frequency with the theoretical probability as trials accumulate.
Counting outcomes, combining events, and why a fair coin owes you nothing.
Probability measures how likely something is, on a scale from 0 (impossible) to 1 (certain). When every outcome is equally likely it is pure counting:
The sample space is the complete list of things that could happen — the tiles shown under the chart. An event is any subset of it: "even number", "greater than 4". Selecting events A and B highlights their tiles, and tiles in both are the intersection.
Getting the sample space right is most of the work. For two dice there are 36 equally likely outcomes but only 11 possible sums — and those sums are not equally likely, which is why the two-dice chart is a triangle rather than a flat line.
That subtraction is the part people forget. Adding P(A) and P(B) double-counts every outcome living in both events, so you remove one copy. Pick two overlapping events and check the arithmetic against the live numbers.
When events cannot happen together — mutually exclusive — the overlap is 0 and the rule simplifies to plain addition.
Two events are independent when knowing one happened tells you nothing about the other. The test is arithmetic:
The app checks this for your chosen events and reports the verdict. It is a genuinely surprising result on a single die: "even" and "greater than 3" are not independent, while "even" and "greater than 4" happen to be — independence is a property of the numbers, not of intuition.
Press Run continuously. Early on the experimental line jumps around wildly; after a few thousand trials it settles onto the theoretical value and stays there.
That is the law of large numbers: averages converge in the long run. What it emphatically does not say is that outcomes "balance out". After ten heads, the next flip is still exactly 50/50 — the coin has no memory. Convergence happens because early flukes are diluted by the growing pile of trials, not because anything corrects them.
A probability is a number between 0 and 1 saying how likely an outcome is. Zero means impossible, one means certain, and everything interesting lives in between.
For situations where every outcome is equally likely, it is a counting exercise:
P(event) = favourable outcomes / total outcomes
A fair die: P(rolling a 4) = 1/6 ≈ 0.167. P(rolling an even number) = 3/6 = 0.5.
Three rules follow immediately and cover most of what you need:
Worked example: two dice, what is P(at least one six)? Directly it is fiddly. Via the complement: P(no six) = (5/6)×(5/6) = 25/36, so the answer is 1 − 25/36 = 11/36 ≈ 0.306.
Two events are independent when knowing one tells you nothing about the other. Coin flips are independent; drawing cards without replacement is not.
Independence matters because it makes the multiplication rule simple — multiply the probabilities and stop. Without it you need conditional probabilities for every combination, and the number of those grows exponentially.
This is exactly the trade Naive Bayes makes: assume every feature is independent given the class, so a joint probability over 20,000 words becomes a product of 20,000 easy numbers. The assumption is false and the classifier works anyway, because the errors it causes affect the magnitudes more than the ranking.
Checking independence is a real modelling step, not a formality. Repeated measurements on the same patient, sensor readings a second apart, and words in the same sentence are all correlated, and treating them as independent produces overconfident models.
A distribution describes the probabilities of every possible outcome at once.
| Distribution | Models | Example |
|---|---|---|
| Bernoulli | One yes/no trial | A single coin flip |
| Binomial | Successes in n trials | Heads in 10 flips |
| Poisson | Events in a fixed interval | Support tickets per hour |
| Uniform | All outcomes equally likely | A fair die |
| Normal | Values clustered around a mean | Heights, measurement error |
| Exponential | Time until the next event | Time between arrivals |
Two properties summarise most of them: the mean (where the centre is) and the variance (how spread out it is).
Discrete distributions assign probability to individual values; continuous ones assign probability to ranges, which is why the probability of any exact value is zero for a continuous variable. P(height = 180.000...) is zero; P(179.5 < height < 180.5) is not.
Probability as a long-run frequency, measured rather than asserted.
Count the outcomes, define events as subsets, subtract the overlap when combining with OR, and multiply only when events are genuinely independent. Probability describes long-run behaviour — it makes no promise whatsoever about your next roll.
The expected value is the long-run average outcome — each value weighted by its probability.
E[X] = Σ x × P(x)
A fair die: (1+2+3+4+5+6)/6 = 3.5. Note that 3.5 is not a possible roll; an expected value is an average, not a prediction.
The reason it matters is that it turns a set of uncertain outcomes into a single comparable number:
A model flags 1,000 transactions. Each flag costs £5 to review, and each fraud caught saves £200. At a 12% precision, the expected value is 1,000 × (0.12×200 − 5) = 1,000 × 19 = £19,000. Raise the threshold to reach 30% precision but only 400 flags: 400 × (0.3×200 − 5) = £22,000. The second is better, and no accuracy metric would have told you that.
Every threshold decision, every "should we ship this model", and every A/B test readout is an expected-value calculation underneath.
What is the difference between probability and statistics? Probability reasons from a known model to expected data; statistics reasons from observed data back to a plausible model.
Can a probability be greater than 1? No. A probability density can exceed 1, because it is a density over a range rather than a probability of a point — the area under it still totals 1.
What are odds? The ratio of "happens" to "does not": a probability of 0.75 is odds of 3 to 1. Logistic regression works in log-odds because they are unbounded and additive.
How many samples do I need? It depends on the effect you want to detect. A power calculation gives the number; guessing gives an inconclusive experiment.
Why is the normal distribution everywhere? The central limit theorem: sums and averages of many independent contributions tend towards a normal shape, whatever the individual distributions look like.
Is a p-value the probability that my hypothesis is wrong? No — it is the probability of seeing data this extreme if the null hypothesis were true. The two are routinely confused and are not the same thing.
Answer without scrolling back up.
Two fair coin flips. What is the probability of two heads?
Independent events multiply: 1/2 x 1/2 = 1/4. The four equally likely outcomes are HH, HT, TH, TT, and only one of them qualifies.
P(A|B) means:
The bar is 'given'. Conditioning narrows the world to the cases where B is true and asks how often A holds within that smaller set. Swapping the two sides gives a different number - assuming otherwise is the base-rate fallacy.
A test is 99% accurate for a disease affecting 1 in 10,000 people. You test positive. Roughly how worried should you be?
Among 10,000 people there is about 1 true case and about 100 false positives, so a positive result is right roughly 1 time in 100. The base rate dominates, which is exactly what Bayes' rule formalises.
Roll dice thousands of times and watch the messy experimental frequencies settle onto the exact theoretical answer. Probability is what happens in the long run — not what happens next.