Covariance and Correlation
Stretch and tilt a cloud of points. Covariance changes when you change the units; correlation does not — and neither notices a curve.
The Cloud
e.g. metres → centimetres
The Scatter
r = 0.70Green points push covariance up, orange pull it down. The dashed lines are the two means.
Measures
Covariance Matrix
The off-diagonal entries are equal, so the matrix is symmetric — which guarantees the ellipse axes are perpendicular. That guarantee is what PCA is built on.
Covariance and Correlation
Covariance and correlation describe how two variables vary together, but they measure different things.
Interpret with care
Correlation does not establish causation, and outliers can strongly affect it. A change of units rescales covariance but leaves correlation unchanged, while nonlinear transformations can alter both. Independence implies zero covariance when the relevant moments exist, but zero covariance does not generally imply independence. See probability basics and the normal distribution.
Interpret the sign, magnitude, and units
Suppose study hours and scores tend to be above their respective means together. Paired deviations have positive products, so covariance and correlation are positive. If high values of one tend to pair with low values of the other, the products tend to be negative. Correlation standardises covariance, so it has no units and lies between −1 and 1. A correlation of −0.8 indicates a strong negative *linear* association; it does not mean one variable falls by 0.8 units for every one-unit increase in the other.
Covariance depends on units: converting height from metres to centimetres multiplies covariance with another variable by 100. Correlation is unchanged by positive rescaling, but reversing one variable's scale flips its sign. A perfect straight-line relationship has correlation +1 or −1 when both variables vary. A curved relationship can have correlation near zero despite a clear pattern. Plot the paired observations, look for outliers and subgroups, and describe the result as association rather than cause. Independence implies zero covariance when the moments exist, but zero covariance alone does not prove independence.
Try this above
Switch between a rising, falling, and curved pattern. Compare covariance and correlation, and notice the effect of changing one variable’s units.
For observed pairs, sample covariance usually divides the sum of paired deviations by n−1; sample Pearson correlation then divides by both sample standard deviations. The divisor convention matters when estimating a population quantity but cancels in common definitions of correlation if used consistently. Neither coefficient describes all dependence. A U-shaped pattern can have near-zero Pearson correlation, and a subgroup mixture can create an association that is absent within groups. Plot the data and consider domain context before summarising with one number.
A correlation matrix collects pairwise correlations and must be positive semidefinite to represent a valid joint covariance structure. Choosing pairwise coefficients independently can create an impossible matrix, even if every individual value lies between −1 and 1. In multivariable modelling, check the whole matrix, not just each pair. Correlation also captures only linear association; nonlinear dependence may require plots or other dependence measures.
Sources and next steps
- NIST: Measures of Association describes covariance and correlation.
- Continue with expectation and variance to derive covariance from deviations around means.
Overview
Direction and strength of linear association
Covariance is the average product of paired deviations from their means. Its sign indicates whether the variables tend to be above or below their means together. Its magnitude depends on both variables’ units, so it is hard to compare across datasets.
Pearson correlation standardises covariance by the two standard deviations:
Correlation is unitless and lies between −1 and 1 when both standard deviations are nonzero. Values near ±1 indicate a strong linear relationship; a value near zero means weak linear association, not necessarily no relationship. For example, a symmetric U-shaped relationship can have correlation near zero.