Click any cell in the result and watch exactly which row and column produced it. Every entry is a dot product — and this single operation is what every neural network layer actually does.
Shapes
rows of A (m)2
cols of A / rows of B (n)3
cols of B (p)2
Click any cell in C to see the row and column that build it.
A × B = C
click a cell in C
How That Cell Is Built
Multiplications
0
Additions
0
Output cells
0
Total ops
0
Matrix Multiplication
A grid of dot products — and the reason deep learning needs GPUs.
The problem it solves
Multiplying two matrices produces a new matrix in which every single entry is a dot product — one row of the left matrix against one column of the right. Nothing more complicated is happening.
The Rule for a Single Cell
Click any cell in C and the app highlights the blue row and amber column that feed it, then writes out the arithmetic term by term. Every cell is independent of every other — which is precisely why this operation parallelises so well on a GPU.
The Shape Rule
You cannot multiply arbitrary matrices. The inner dimensions must match:
Drag the sliders and watch the shape line update. This single rule is behind most tensor-shape errors you will ever hit: a mismatch is not a subtle numerical problem, it is an operation that simply has no definition.
Order Matters
Matrix multiplication is not commutative: AB ≠ BA in general. Often BA is not even a legal operation — a (2×3) times a (3×4) works, but (3×4) times (2×3) does not. When people say the order of layers matters, this is the literal reason.
It is associative, though: (AB)C = A(BC). Choosing the cheaper grouping can save enormous amounts of computation, which is exactly the trick behind LoRA's low-rank update.
Why This Is the Bottleneck
Multiplying two n×n matrices needs about n³ multiply-add operations. Doubling the size makes it eight times more work. A single transformer layer with d = 4096 performs billions of these per token.
Watch the "Total ops" counter as you raise the sliders. That cubic growth is the entire reason GPUs — chips designed to do thousands of independent multiply-adds at once — became the hardware of deep learning.
What multiplying two matrices means
Matrix multiplication looks like an arbitrary rule until you see what it is for: applying a transformation to a collection of vectors, all at once.
Every row of the first matrix meets every column of the second, and each meeting produces one number — the dot product of that row and that column.
A (2x3) B (3x2) AB (2x2)
[1 2 3] [7 8] [1*7+2*9+3*11 1*8+2*10+3*12]
[4 5 6] [9 10] = [4*7+5*9+6*11 4*8+5*10+6*12]
[11 12]
Working the top-left entry through: 1×7 + 2×9 + 3×11 = 7 + 18 + 33 = 58. The top-right: 1×8 + 2×10 + 3×12 = 8 + 20 + 36 = 64. The full result is [[58, 64], [139, 154]].
The shape rule, and why it exists
(m × n) × (n × p) = (m × p)
The inner dimensions must match; the outer ones become the result's shape. A 2×3 times a 3×2 works and gives 2×2. A 2×3 times a 2×3 does not work at all.
The rule is not arbitrary. Each output entry is a dot product between a row of A and a column of B, and a dot product needs both vectors to have the same length. A's rows have n entries; B's columns have n entries. Match, or there is nothing to compute.
Checking shapes before writing any code is the fastest debugging habit in linear algebra. In practice:
import numpy as np
A = np.array([[1, 2, 3], [4, 5, 6]]) # (2, 3)
B = np.array([[7, 8], [9, 10], [11, 12]]) # (3, 2)
A @ B # (2, 2) - the @ operator is matrix multiplication
A.shape, B.shape, (A @ B).shape
A * B # ERROR - * is elementwise, and these shapes disagree
The distinction between @ and * is the single most common source of NumPy bugs: * multiplies corresponding entries, @ does the row-by-column operation above.
Order matters, and that is the point
AB ≠ BA
Sometimes BA is not even a legal shape. When both are legal, they are usually different matrices.
This is not a defect; it reflects reality. Matrices represent transformations, and transformations do not generally commute. Rotate a book 90° then flip it, versus flip it then rotate: the book ends up in different orientations. The matrices behave the same way.
What matrix multiplication does keep:
Associative: (AB)C = A(BC). The grouping is free, which matters enormously for efficiency — multiplying in the cheaper order can save orders of magnitude of work.
Distributive: A(B + C) = AB + AC.
Identity: AI = IA = A, where I has ones on the diagonal.
And a fact worth carrying: (AB)ᵀ = BᵀAᵀ — transposing a product reverses the order.
Why this is the core operation of machine learning
A single neural network layer is one matrix multiplication and one addition:
output = input × W + b
If a batch of 32 examples with 100 features each is a 32×100 matrix, and the layer has 50 units, then W is 100×50 and the output is 32×50. One operation, the entire batch processed.
That is why GPUs matter. They are built to perform thousands of multiply-and-add operations simultaneously, and matrix multiplication is exactly that pattern. Training a large model is, at the arithmetic level, mostly this operation repeated billions of times.
The same shape reasoning explains most deep learning error messages. "Expected input of size (*, 100) but got (*, 128)" is the shape rule above, refusing to multiply mismatched dimensions.
Multiply two, both ways round
The clearest way to see that order matters: the same two matrices, multiplied in each order, giving different answers.
example_01.pyNumPy
import numpy as np
A = np.array([[1, 2], [3, 4]])
B = np.array([[0, 1], [1, 0]])
print("A @ B ="); print(A @ B)
print("B @ A ="); print(B @ A)
print()
print("not the same - matrix multiplication does not commute")
print()
print("entry (0,0) is row 0 of A dotted with column 0 of B:")
print(" ", A[0], ".", B[:, 0], "=", A[0] @ B[:, 0])
print()
C = np.arange(6).reshape(2, 3)
print("shapes: (2,2) @ (2,3) ->", (A @ C).shape)
Output
Guided tour
Click the top-left cell of C. Only the first row of A and first column of B light up — the rest of both matrices is irrelevant to that number.
Press "Walk every cell" to sweep through the whole output and see the pattern of row-column pairings.
Set n to 1. Each cell becomes a single product — this is the outer product, and it is exactly what LoRA's B·A builds at rank 1.
Push all three sliders to 5 and watch the operation count climb far faster than the matrices grow.
Summing up
Row against column, multiply and add, repeat for every output cell. The inner dimensions must agree, the order cannot be swapped, and the cost grows cubically — three facts that explain tensor errors, layer ordering and the entire GPU industry.
Composition: several transformations as one
Because matrix multiplication is associative, a chain of transformations can be collapsed into a single matrix before it is ever applied to data.
Rotate, then scale, then translate: multiply the three matrices together once, and apply the single result to a million points. Computer graphics pipelines are built on this, and so is every framework that fuses layers for inference.
The order in the product is the reverse of the order the operations happen in when they act on a column vector: M = T · S · R applies R first. That reversal catches everyone once, and it comes straight from M·v = T·(S·(R·v)).
Cost, and why the order of a chain matters
Multiplying an (m×n) by an (n×p) matrix costs roughly m×n×p multiply-add operations. So the associativity above has real consequences.
Same answer, one hundred times the work. This is why the order in which you group a chain of matrix products is a real optimisation, and why libraries such as np.linalg.multi_dot choose the grouping for you.
The naive algorithm is cubic for square matrices, and asymptotically faster algorithms exist (Strassen and its successors). In practice, the constant factors and cache behaviour of the straightforward algorithm on optimised hardware usually win for realistic sizes.
Questions people ask
Why is A * B different from A @ B in NumPy?* multiplies element by element and requires matching shapes (or broadcastable ones); @ is the row-by-column matrix product.
Can I multiply a matrix by a vector? Yes — treat the vector as an n×1 matrix. (m×n) @ (n×1) gives an m×1 result, which is the standard "apply a transformation to a point".
Why does my deep learning code complain about shapes? Because the inner dimensions do not match. Print the shapes at each step; the mismatch is always visible.
Is matrix multiplication ever commutative? For particular pairs, yes — a matrix and its inverse, a matrix and the identity, two diagonal matrices. In general, no.
What is the identity matrix? Ones on the main diagonal, zeros elsewhere. Multiplying by it changes nothing, which makes it the matrix equivalent of 1.
How do I know if a product is possible? Write the shapes side by side: (m×n)(n×p). If the two inner numbers match, it works.
Recap in one screen
Each entry of the product is the dot product of a row from the left and a column from the right.
Inner dimensions must match; the outer ones give the result's shape.
Order matters — AB is not BA — but grouping is free, and choosing the grouping can change the cost enormously.
@ is matrix multiplication, * is elementwise. Confusing them is the classic NumPy bug.
A neural network layer is one matrix multiplication, which is why GPUs are built for this operation.
Recall check
0 of 3
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “The problem it solves”?
Multiplying two matrices produces a new matrix in which every single entry is a dot product — one row of the left matrix against one column of the right. Nothing more complicated is happening.
What does this module say about “The Rule for a Single Cell”?
Click any cell in C and the app highlights the blue row and amber column that feed it, then writes out the arithmetic term by term. Every cell is independent of every other — which is precisely why this operation parallelises so well on a GPU.
What does this module say about “Order Matters”?
Matrix multiplication is not commutative : AB ≠ BA in general. Often BA is not even a legal operation — a (2×3) times a (3×4) works, but (3×4) times (2×3) does not. When people say the order of layers matters, this is the literal reason.
Cheat sheet
Matrix Multiplication
Multiplying two matrices produces a new matrix in which every single entry is a dot product — one row of the left matrix against one column of the right. Nothing more complicated is happening.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.