Load a die face by face and watch the two numbers that summarise any random quantity move.
Overview
Expectation
The expectation of a random variable is each value it can take, weighted by how likely that value is:
E[X] = sum over x of x * P(X = x)
The mean, mode and median module covers the mean of a dataset you've. This is the mean of a spread you haven't sampled — what the mean would settle to given enough draws.
Load the die above and watch E[X] move. Put all the weight on 6 and it turns 6. Spread it evenly and it turns 3.5.
3.5 is the case worth pausing on: a fair die has expectation 3.5, and no die has ever rolled a 3.5. The expectation is a balance point, not a guess. The spread would balance on a pin at that value, and that is all the word means.
Expectation and Variance
This module needs JavaScript: the
numbers are computed in the page rather than recorded.
Worth knowing
Expectation is the probability-weighted average: E[X] = Σ x · P(x).
It is a balance point, not a prediction. A fair die has E[X] = 3.5, a value it can never roll.
Variance is the expected squared distance from the mean: E[(X − μ)²].
Expectation is linear no matter what. Variance only adds across independent variables — a distinction that matters constantly.
Expectation and Variance
The two numbers that summarise a random quantity, and the rules for mixing them.
Variance
Expectation alone says nothing about spread. Two very different dice can share one. Variance measures the spread:
Var(X) = E[(X - mu)^2]
The expected squared distance from the mean. Squaring is what stops the plus and minus deviations cancelling, and it's why the units come out squared — variance of a height in metres is in metres squared, which is meaningless as a length. Taking the square root gives the standard deviation, back in the first units. This Is why that is the number often reported.
The identity worth storing is:
Var(X) = E[X^2] - (E[X])^2
The mean of the squares minus the square of the mean. It's how variance is found in one pass over data, and it's also the source of a classic numeric bug: with large values and small variance, the two terms are nearly equal and subtracting them destroys precision. Welford's method exists to avoid just this.
Combining them
The rules for expectation are as good as rules get:
E[aX + b] = a E[X] + b
E[X + Y] = E[X] + E[Y] always
That second line holds whether or not X and Y are free. Linearity of expectation is plain, and it's the reason a surprising number of counting args are one line long.
Variance is stricter:
Var(aX + b) = a^2 Var(X) note: b vanishes, a is squared
Var(X + Y) = Var(X) + Var(Y) only if X and Y are independent
Shifting a spread doesn't change its spread. This Is why b disappears. Scaling by a scales the variance by a², because variance is in squared units.
The freedom rule on the sum is where mistakes live. If X and Y move together, the covariance term reappears:
The standard error. The mean of *n* free draws has variance σ²/n, straight from the rules above. Its square root is σ/√n, which is the whole content of the sampling spread.
Bias-variance breakdown. Expected squared error splits into bias squared plus variance plus irreducible noise, using the identity above.
Portfolio diversification. Mixing assets reduces variance only when they aren't just right linked — the covariance term again.
Bagging. Blending *n* models reduces variance by *n* only if their errors are free, which is why random forests work to decorrelate their trees.
Where it goes wrong with Expectation and Variance
Treating the expectation as a likely value. It need not be attainable at all.
Adding variances of linked variables. The commonest error in this material.
Losing the square on the scale factor. Doubling a variable quadruples its variance.
given a variance exists. Some spreads have none — the Cauchy has neither a finite variance nor a finite mean, and sample means of Cauchy draws never settle.
Both numbers, worked by hand
Five cards, drawn one at a time with each card equally likely. They're marked 1, 1, 2, 5 and 11.
Build the table. P(x) is how often that value appears out of five, and the last two columns are what the two formulas need:
x
P(x)
x P(x)
x² P(x)
1
2/5
0.4
0.4
2
1/5
0.4
0.8
5
1/5
1.0
5.0
11
1/5
2.2
24.2
total
1
4.0
30.4
The third column sums to the expectation: E[X] = 4.0. Note that no card is marked 4. The fourth sums to E[X²] = 30.4, and the one-pass identity gives the variance straight away:
Var(X) = E[X^2] - (E[X])^2 = 30.4 - 16 = 14.4
Now the same number from the meaning, E[(X - mu)^2], squaring each distance from 4 and weighting it:
The same 14.4, by a different route. The standard deviation is √14.4 = 3.79 — almost as large as the mean itself, which is the honest summary of this deck: the mean is 4. The single 11 does most of the work.
The two routes cost differently. The meaning needs the mean before it can start, so it's two passes over the data. The identity needs only running totals of x and x², so it's one. Which is why it's the one that gets built, and why the precision trap above is worth knowing about.
Questions people ask about Expectation and Variance
Why square the deviations instead of taking abs values? You can, and that gives the mean abs deviation. This Is a just right good measure of spread. Squaring wins on grounds of convenience rather than truth: it's differentiable all over, it makes the variance of a sum break down into a sum of variances, and it makes the identity above. abs values give none of those, which is why almost each result in this material is stated for variance.
Does the expectation have to be a value the variable can take? No, and often it isn't. A fair die means 3.5. The deck above means 4 without holding a 4. Expectation is a balance point of the spread, not a forecast of any single draw.
Why does sample variance divide by n − 1? Because the sample mean is itself guessed from the same data, and it sits closer to the data than the true mean does — it's the value that *cuts* the sum of squared deviations, so squared distances measured from it are systematically too small. Cutting by n - 1 instead of n corrects that bias just. With the true mean known, n would be right.
When can I add variances? Only when the variables are free, or at least unlinked. Or else the covariance term is truly there and dropping it will mislead you, almost always in the direction of thinking you have less risk than you do.
Variance or standard deviation in a report? Standard deviation, nearly always, because it's in the same units as the data and can be checked to the mean. Variance is the quantity the algebra is easier in. This Is why it appears in the derivations and rarely in the conclusions.
A bet worth taking that usually loses
Expectation and variance found from the meaning, then measured over 400,000 plays. The two numbers say different things, and both matter.
example_01.pyNumPy
import numpy as np
# a bet: 1/6 chance to win 80, else lose 12
outcomes = np.array([80.0, -12.0])
probs = np.array([1 / 6, 5 / 6])
ev = (outcomes * probs).sum()
var = (probs * (outcomes - ev) ** 2).sum()
print("E[X] = %.4f" % ev)
print("Var[X] = %.4f sd = %.4f" % (var, np.sqrt(var)))
print()
rng = np.random.default_rng(1)
draws = rng.choice(outcomes, size=400_000, p=probs)
print("400,000 plays: average %.4f, variance %.4f" % (draws.mean(), draws.var()))
print(" total won: %.0f" % draws.sum())
print()
print("a bet worth taking that still loses most of the time:")
print(" fraction of plays that lost money: %.3f" % (draws < 0).mean())
print()
print("the sd is %.1f -- far bigger than the edge of %.2f."
% (np.sqrt(var), ev))
print("expectation says play; variance says do not bet the rent on one round.")
Output
Check yourself
0 of 3
Answer without scrolling back up.
A fair die has E[X] = 3.5. What does that tell you?
No die has ever rolled a 3.5. The expectation is where the distribution would balance on a pin, not a prediction of any single outcome.
When does Var(X + Y) = Var(X) + Var(Y)?
Expectation adds unconditionally; variance does not. Otherwise a covariance term appears - which is why diversification and bagging both depend on decorrelation.
Why is Var(X) = E[X^2] - (E[X])^2 numerically dangerous?
It is the one-pass formula, and the subtraction of two nearly equal large numbers is what Welford's algorithm exists to avoid.
Cheat sheet
Expectation and Variance
Calculate expected value and variance from a probability distribution. Apply the linearity and independence rules that simplify real problems.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.