Variance
Expectation alone says nothing about spread. Two very different dice can share one. Variance measures the spread:
Var(X) = E[(X - mu)^2]
The expected squared distance from the mean. Squaring is what stops the positive and negative deviations cancelling, and it is why the units come out squared — variance of a height in metres is in metres squared, which is meaningless as a length. Taking the square root gives the standard deviation, back in the original units, which is why that is the number usually reported.
The identity worth memorising is:
Var(X) = E[X^2] - (E[X])^2
The mean of the squares minus the square of the mean. It is how variance is computed in one pass over data, and it is also the source of a classic numerical bug: with large values and small variance, the two terms are nearly equal and subtracting them destroys precision. Welford's algorithm exists to avoid exactly this.
Combining them
The rules for expectation are as good as rules get:
E[aX + b] = a E[X] + b
E[X + Y] = E[X] + E[Y] always
That second line holds whether or not X and Y are independent. Linearity of expectation is unconditional, and it is the reason a surprising number of counting arguments are one line long.
Variance is stricter:
Var(aX + b) = a^2 Var(X) note: b vanishes, a is squared
Var(X + Y) = Var(X) + Var(Y) only if X and Y are independent
Shifting a distribution does not change its spread, which is why b disappears. Scaling by a scales the variance by a², because variance is in squared units.
The independence condition on the sum is where mistakes live. If X and Y move together, the covariance term reappears:
Var(X + Y) = Var(X) + Var(Y) + 2 Cov(X, Y)
The [covariance module](covariance_and_correlation.html) covers that term.
Where it shows up
The standard error. The mean of *n* independent draws has variance σ²/n, straight from the rules above. Its square root is σ/√n, which is the whole content of [the sampling distribution](sampling_distributions.html).
Bias-variance decomposition. Expected squared error splits into bias squared plus variance plus irreducible noise, using the identity above.
Portfolio diversification. Combining assets reduces variance only when they are not perfectly correlated — the covariance term again.
Bagging. Averaging *n* models reduces variance by *n* only if their errors are independent, which is why random forests work to decorrelate their trees.
Where it goes wrong
Treating the expectation as a likely value. It need not be attainable at all.
Adding variances of correlated variables. The commonest error in this material.
Forgetting the square on the scale factor. Doubling a variable quadruples its variance.
Assuming a variance exists. Some distributions have none — the Cauchy has neither a finite variance nor a finite mean, and sample averages of Cauchy draws never settle.