A Matrix is a Transformation

The columns are the whole story

Start with the two basis vectors: î = (1, 0) pointing along x, and ĵ = (0, 1) pointing along y. Every vector is built from them — (3, 2) means "3 of î plus 2 of ĵ".

A linear transformation is one that keeps the grid lines straight, evenly spaced, and the origin fixed. That constraint has a consequence: if you know where î and ĵ land, you know where everything lands, because (3, 2) must still be "3 of the new î plus 2 of the new ĵ".

So a transformation only needs to record two vectors. Write them as columns and you have the matrix:

[ a   b ]    first column (a, c) = where î lands
[ c   d ]    second column (b, d) = where ĵ lands

Check it against the panel on the right while you drag the sliders. The two readouts are always exactly the two columns, because that is what the columns are.

Why matrix-vector multiplication looks like that

Now the formula stops being arbitrary. To transform (x, y), take x copies of where î went and y copies of where ĵ went, and add:

M(x, y) = x·(a, c) + y·(b, d) = (ax + by, cx + dy)

That is the rule you were taught, derived rather than memorised. And matrix multiplication is just doing this twice — applying one transformation and then another — which is also why order matters: rotating then shearing is genuinely not the same as shearing then rotating.

A matrix is a function on space

The most useful way to read a matrix is not as a grid of numbers but as an instruction: take every point in space and move it, in a consistent way.

Multiply a matrix by a vector and you get a new vector. Do that to every point in a square grid and the grid becomes stretched, rotated, sheared or flipped — but it stays a grid. Lines stay lines, parallel lines stay parallel, and the origin stays put. Those three properties define a linear transformation, and matrices are exactly the transformations with them.

The quickest way to read a matrix is to look at what it does to the basis vectors:

A = [[a, b], [c, d]]

The first column is where [1, 0] lands; the second column is where [0, 1] lands. That is all. Everything else follows, because any vector is a combination of those two and the transformation preserves combinations.

The transformations worth recognising

That last one is worth dwelling on. Its two columns are multiples of each other, so both basis vectors land on the same line, and the whole plane is flattened onto it. The transformation destroys information and cannot be undone — the matrix is singular, its determinant is zero, and it has no inverse.

Composition, and why order matters

Applying one transformation after another is a matrix product, and the order matters because the operations do not commute.

Rotate 90° then stretch horizontally, versus stretch then rotate: the results differ, and you can verify it by tracking where [1, 0] ends up in each case.

The convention with column vectors is that the rightmost matrix acts first:

(A B) v = A (B v)  — B first, then A

That reversal catches everyone once. It is why a graphics pipeline written as M = translate · rotate · scale applies scaling first.

Because matrix multiplication is associative, a chain of transformations can be multiplied into a single matrix once, and then applied to millions of points cheaply. That is the basis of every rendering pipeline and of fused operations in neural network inference.

Reading the numbers back out

Three summaries tell you what a matrix does without applying it to anything:

The determinant is the factor by which areas (or volumes) are scaled. A determinant of 3 triples areas; 0.5 halves them; 0 collapses them; and a negative determinant means the space has been flipped over.

The eigenvectors are the directions that do not rotate — the axes of the transformation. The eigenvalues say how much each is stretched.

The rank is the number of dimensions that survive. A 3×3 matrix of rank 2 flattens three-dimensional space onto a plane.

Those three between them describe almost everything a linear transformation can do, which is why they appear in every linear algebra course before anything else.

Watch the unit square get bent

A matrix is what it does to space. Each one here is applied to the same unit square, and the determinant tells you what happened to its area.

import numpy as np

square = np.array([[0.0, 1.0, 1.0, 0.0],     # x
                   [0.0, 0.0, 1.0, 1.0]])    # y

mats = {
    "identity":        [[1, 0], [0, 1]],
    "scale x2, y3":    [[2, 0], [0, 3]],
    "scale x2, y0.5":  [[2, 0], [0, 0.5]],
    "rotate 90":       [[0, -1], [1, 0]],
    "shear x by y":    [[1, 1], [0, 1]],
    "flip across y":   [[-1, 0], [0, 1]],
    "squash to line":  [[1, 2], [2, 4]],
}

print("%-16s %6s %11s %5s  corners" % ("matrix", "det", "area scale", "rank"))
for name, M in mats.items():
    M = np.asarray(M, dtype=float)
    det = np.linalg.det(M)
    print("%-16s %6.2f %11.2f %5d  %s"
          % (name, det, abs(det), np.linalg.matrix_rank(M),
             np.round((M @ square).T, 2).tolist()))
print()
print("the determinant is the area factor, and nothing else:")
print("  x2 and y3 makes the square six times bigger  -> det 6")
print("  x2 and y0.5 stretches one way and squeezes the other -> det 1")
print("  rotation and shear move the square without resizing it -> det 1")
print("  a negative determinant means the square was flipped over")
print("  a zero determinant means it was flattened onto a line, and that")
print("  cannot be undone: two different squares now land on the same place.")
print()
a = np.array([1.0, 0.0]); b = np.array([0.0, 1.0])
S = np.array([[1.0, 2.0], [2.0, 4.0]])
print("proof for the last one -- two different inputs, one output:")
print("  [1,0] and [-1,1] both map to", (S @ a), "and", (S @ np.array([-1.0, 1.0])))

Things to try

1. Start with nothing. Set the Preset to Identity. The grid does not move, the determinant is 1, and the columns are (1,0) and (0,1) — the basis vectors, unmoved.
2. Confirm the columns claim. Set Entry a to 2 and Entry c to 1. The green vector jumps to (2, 1), which is the first column exactly. Drag any slider and watch the two readouts track the four numbers.
3. Watch it happen. Click Animate the Transform and watch space slide continuously from untouched to fully transformed. The grid lines stay straight and evenly spaced the whole way — that is what makes it linear.
4. Double the area. Set the Preset to Scale by 2. Every length doubles, so area quadruples, and the determinant reads exactly 4.
5. Rotate without distorting. Set the Preset to Rotate 45°. The determinant is 1 — rotation moves everything but stretches nothing, so area is untouched.
6. Flip it over. Set the Preset to Reflect in the y-axis. The determinant goes to −1. The magnitude is still 1, because no area was gained or lost, but the sign flipped: space was turned over.
7. Destroy a dimension. Set the Preset to Squash flat. The determinant hits 0, the grid collapses onto a single line, rank drops to 1, and Invertible reads no. Once two dimensions have been flattened into one, no transformation can pull them apart again — the information is gone.

The determinant, in one sentence

The unit square has area 1. Apply the matrix and it becomes a parallelogram. The determinant is that parallelogram's area, with a sign telling you whether space was flipped.

• det = 1 — area preserved. Rotations and shears.
• det = 3 — everything triples in area.
• det = −1 — area preserved but orientation reversed, like looking in a mirror.
• det = 0 — space collapsed to a line or a point. Nothing can undo it, which is precisely why a zero determinant means no inverse exists.

The formula ad − bc is not a magic incantation; it is the area of the parallelogram spanned by the two columns.

Where this shows up

• A neural network layer is a matrix followed by a non-linearity. The matrix moves space; the activation is what bends it, which is why stacking matrices alone buys nothing.
• PCA looks for the directions along which a transformation stretches most — which is exactly what eigenvectors describe.
• Rotation matrices in graphics and robotics are chosen precisely because their determinant is 1: they move objects without resizing them.

Traps worth knowing

• Reading the rows as the basis images. It is the columns. A very common slip, and it transposes every result.
• Assuming multiplication commutes. AB and BA are different transformations applied in different orders and generally give different answers.
• Ignoring a near-zero determinant. Not quite singular is often worse than singular: the inverse exists but is enormous, and small input errors get amplified. This is what "ill-conditioned" means.
• Expecting the determinant to measure size. A matrix can stretch enormously in one direction and squash in another and still have determinant 1. It measures area, not magnitude.

The short of it

A matrix is a record of where the basis vectors land, stored as its columns, and because linear transformations keep grid lines straight and evenly spaced that record determines where every other vector goes. This makes the multiplication rule a derivation rather than a convention: transforming (x, y) means taking x copies of the first column plus y copies of the second. The determinant is the area of the transformed unit square, signed by whether space was flipped, so a determinant of zero means space collapsed onto a line and no inverse can exist.

Where this reading pays off in machine learning

A neural network layer is Wx + b: a linear transformation followed by a shift. The weight matrix rotates, scales and projects the input space; the bias moves it. The activation function then bends it — and that bending is the only non-linear step, which is why activations are what give depth its power. Without them, a hundred stacked layers collapse into a single matrix.

PCA is a rotation: the eigenvectors of the covariance matrix form a matrix that rotates the data onto new axes aligned with its own spread. Dropping components afterwards is a projection onto a lower-dimensional subspace.

Embeddings are learned linear maps from a sparse one-hot space into a dense one where distances mean something.

Attention in transformers applies three learned matrices to each token — queries, keys and values — each one a transformation into a space where the comparison it needs is easy.

Data augmentation for images applies rotation, scaling and shear matrices to pixel coordinates — literally the transformations in the table above.

Non-square matrices change the dimension

A square matrix maps a space to itself. A non-square one moves between spaces:

• A 2×3 matrix takes 3-D vectors and returns 2-D ones — a projection, losing a dimension.
• A 3×2 matrix takes 2-D vectors into 3-D space, but the output can only ever lie on a plane through the origin.

This is exactly what network layers do when they change width. A layer mapping 512 features to 128 is a projection into a smaller space, and information is necessarily discarded. A layer mapping 128 to 512 cannot create information — the outputs lie in a 128-dimensional subspace of the larger space, which is why bottleneck architectures and autoencoders work the way they do.

Questions people ask

Why must the origin stay fixed? Because a linear transformation satisfies T(0) = 0. Translation is not linear, which is why the bias term is added separately, and why graphics uses homogeneous coordinates to fold translation into a bigger matrix.

What does a negative determinant mean? The transformation flipped orientation — a reflection is involved.

How do I picture a 512-dimensional transformation? You do not. Work with the summaries: rank, determinant, eigenvalues, and what the shapes are before and after.

Is every matrix invertible? No. If the determinant is zero, the transformation collapsed a dimension, and the lost information cannot be recovered.

Why do the columns tell you where the basis vectors go? Because multiplying by [1, 0] picks out the first column by construction — the arithmetic is the explanation.

Does this help with debugging models? Yes — a great deal of framework shape errors, and every question about which layer discards information, is easier to reason about in terms of what maps into what.

Recap in one screen

• A matrix is a linear transformation: lines stay lines, parallels stay parallel, the origin stays fixed.
• The columns are where the basis vectors land, which is the fastest way to read what a matrix does.
• Products compose transformations, and the rightmost acts first on a column vector.
• Determinant is the area scale factor, eigenvectors are the unrotated directions, rank is the surviving dimensions.
• Network layers, PCA, embeddings and attention are all this idea with learned matrices.