Eigenvalues and Eigenvectors
Sweep a vector around the circle and watch it swing away from its own direction — except at two angles, where it does not turn at all.
The Matrix
Which Directions Survive?
turned 0°White is the probe, orange is where the matrix sends it. Green lines are the eigenvector directions.
Eigenvalues
Cross-checks
Eigenvalues and Eigenvectors: A Practical Guide
The directions a transformation leaves alone, and why PCA is built on them.
Quick Context
A matrix moves space. Almost every vector it touches gets both stretched and rotated — it comes out pointing somewhere new.
But for most matrices there are a small number of special directions where the rotation does not happen. A vector along one of those directions comes out pointing exactly the same way, only longer or shorter. Those directions are the eigenvectors, and the amount each is stretched by is its eigenvalue.
The definition, and what it is really saying
A v = λ v
Read it as an equation of behaviours rather than symbols. The left-hand side is "apply the whole transformation to v". The right-hand side is "just scale v by a number". A vector satisfying this is one for which the entire matrix — all four entries, all that rotating and shearing — collapses into a single multiplication.
That is why they matter. Along an eigenvector, a complicated transformation becomes arithmetic.
Finding them
For a 2×2 matrix the eigenvalues come straight out of two numbers you can read off by eye:
trace = a + d det = ad − bc
λ = [ trace ± √(trace² − 4·det) ] / 2
Two consequences fall out immediately, and both are shown live in the cross-checks panel:
- λ₁ + λ₂ = trace. The eigenvalues sum to the diagonal.
- λ₁ × λ₂ = determinant. They multiply to the area scale factor — which makes sense, since stretching by λ₁ in one direction and λ₂ in the other scales area by their product.
The quantity under the square root decides everything. Positive gives two real eigenvalues, zero gives one repeated, and negative gives none at all in the real plane.
Interactive Exploration Guide
- Hunt for them by hand. Drag the Probe Angle slider slowly and watch the "turned by" readout. For most angles the orange image points somewhere different from the white probe; at two angles it drops to zero and the two arrows lie on top of each other. Those are the eigenvectors, found the hard way.
- Let it find them. Click Snap to an Eigenvector and the probe jumps to a direction where the turn is exactly 0°. Click again to jump to the other one.
- See the whole picture. Click Sweep All Directions and watch the probe rotate through a full circle while the image swings around at a different rate, crossing it exactly twice.
- Check the identities. Drag any entry slider and watch the cross-checks panel. trace always equals λ₁+λ₂ and det always equals λ₁·λ₂, for every matrix you can reach.
- Make them perpendicular. Set the Preset to Symmetric. The two eigenvectors are now at exactly 90° to each other. Any matrix with b = c has this property, and it is the reason PCA produces axes at right angles.
- Make every direction an eigenvector. Set the Preset to Uniform scale. Now nothing rotates at all, whatever the probe angle, because a uniform scale stretches every direction by the same factor.
- Take them away entirely. Set the Preset to Rotation. Real eigenvalues vanish and Real? reads no. A rotation turns every direction, so there is nothing left for it to leave alone.
Why symmetric matrices are special
When b = c the matrix is symmetric, and three things become guaranteed rather than lucky: the eigenvalues are always real, the eigenvectors are always perpendicular, and the matrix can always be diagonalised.
This is not a curiosity. Covariance matrices are symmetric by construction, because the covariance of x with y is the same number as the covariance of y with x. PCA is precisely the eigendecomposition of a covariance matrix, and the guarantee above is what makes its principal components perpendicular and its variances real, positive numbers. The whole method rests on this property.
Where else they turn up
- PCA — eigenvectors of the covariance matrix are the directions of greatest variance; the eigenvalues are how much variance each one carries.
- PageRank — the ranking is the dominant eigenvector of the link matrix.
- Stability — repeatedly applying a matrix makes vectors blow up if the largest |λ| exceeds 1 and shrink to nothing if it is below. This is the exploding-and-vanishing-gradient story in a different vocabulary.
- Conditioning — the ratio of the largest to smallest eigenvalue tells you how stretched the loss surface is, which is what makes plain gradient descent zig-zag.
What usually goes wrong
- Expecting every matrix to have real eigenvectors. Rotations have none. A negative discriminant is not an error.
- Forgetting eigenvectors have no fixed length. If v is an eigenvector so is 2v and −v. Only the direction is determined, which is why implementations return unit vectors and the sign can differ between libraries.
- Assuming perpendicularity in general. That is guaranteed only for symmetric matrices. The Generic preset shows two eigenvectors that are not at right angles.
- Running PCA on unscaled features. The covariance matrix is dominated by whichever feature has the largest units, so the first component just points at it. Standardise first.
Key Takeaway
An eigenvector is a direction a transformation does not rotate, and its eigenvalue is the factor it is stretched by, so along an eigenvector the whole matrix collapses into a single multiplication. For a 2×2 matrix they follow from the trace and determinant alone, and always satisfy λ₁+λ₂ = trace and λ₁λ₂ = det; a negative discriminant means no real eigenvectors exist, which is exactly the case for a rotation. Symmetric matrices are guaranteed real eigenvalues and perpendicular eigenvectors, and since covariance matrices are symmetric, that guarantee is what PCA is built on.