Two rules that decide whether shapes combine, what the result looks like, and why a wrong guess costs gigabytes.
Overview
Everyone already uses it
If you have ever added a scalar to an array, you have already used broadcasting. NumPy stretched the scalar to match the shape of the array without creating a copy. That stretch is broadcasting. The general rules extend the same idea to arrays of different shapes.
Broadcasting matters because it eliminates temporary arrays. Without it, adding a bias to every row of a matrix would require you to tile the bias vector into a full matrix first. That tile allocates memory, fills it with copies, and then gets thrown away after the addition. Broadcasting skips all of that. The bias vector is reused in place.
The rules are mechanical. NumPy compares shapes from the trailing axis backward. Two dimensions are compatible if they are equal or if one of them is one. A missing dimension is treated as one. When a dimension is one, that axis is stretched to match the other array. If any pair of dimensions is neither equal nor one, the operation raises a ValueError.
The most common broadcasting patterns cover the cases you encounter daily. A scalar with any array. A 1-D array with a 2-D array along one axis. A column vector with a row vector to produce a 2-D result. Each of these follows the same rules, and once you can predict the output shape, you can write the code without trial and error.
Shape errors from broadcasting are among the clearest error messages in NumPy. The message names the two shapes and says they are not compatible. If you see one, print the shapes of both operands and walk through the rules. The fix is usually a reshape or a newaxis to add the missing dimension.
The common mistake is adding a dimension in the wrong place. A (3,) array broadcast with a (4, 3) array works along columns. The same array reshaped to (3, 1) would broadcast along rows. The position of the one decides the axis of the stretch. Getting that right is the whole skill.
This module covers the broadcasting rules, the common patterns, how to predict the output shape, and how to add dimensions with np.newaxis or reshape to make two arrays compatible.
a + 10 on a (2, 3) array adds ten to each element. Conceptually the scalar is stretched to (2, 3). nothing is copied and NumPy simply reuses the value as it walks.
That is broadcasting. The general rules extend the same idea to arrays.
Worth knowing
Shapes are compared from the right. Each pair must be equal, or one must be 1, or one shape must have run out.
A missing leading dimension is treated as 1, so a 1-D array behaves like a row.
To broadcast down the rows instead, add a trailing axis: v[:, None] turns (n,) into (n, 1).
Nothing is copied. NumPy reuses values as it walks, so broadcasting is cheap in memory — until the result is large.
A column plus a row gives a full grid. Two 80 KB arrays can produce an 800 MB result, which is the classic accidental blow-up.
np.broadcast_shapes tells you the result shape without allocating anything. Use it when you are unsure.
Broadcasting
Two rules that decide whether shapes combine, and why a wrong guess costs gigabytes.
The simplest case: a scalar
A scalar combines with any shape. That is broadcasting, and everyone uses it before they know the name.
example_01.pyNumPy
import numpy as np
a = np.arange(6).reshape(2, 3)
print(a)
print()
print("a + 10:"); print(a + 10)
print()
print("Conceptually the 10 is stretched to (2,3). Actually nothing is")
print("copied - NumPy just reuses the same value as it walks the array.")
Output
The rules, stated
Compare shapes from the right. Dimensions must be equal, or one of them must be 1, or absent.
The single most common use: normalising along one axis. Which one you get depends on the shape of the operand.
example_03.pyNumPy
import numpy as np
a = np.arange(6).reshape(2, 3)
print("a:"); print(a)
row = np.array([10, 20, 30]) # (3,) - matches the columns
print()
print("a + (3,) -> one value per COLUMN:")
print(a + row)
col = np.array([[100], [200]]) # (2,1) - matches the rows
print()
print("a + (2,1) -> one value per ROW:")
print(a + col)
Output
Making a column on purpose
A 1-D array is treated as a row. To broadcast down the rows you have to give it a second axis.
example_04.pyNumPy
import numpy as np
a = np.arange(6).reshape(2, 3)
per_row = np.array([100, 200]) # one value per row
try:
a + per_row
except ValueError as e:
print("a + (2,) :", type(e).__name__)
print(" ", e)
print()
print("give it a trailing axis and it lines up:")
print(" per_row[:, None].shape ->", per_row[:, None].shape)
print(a + per_row[:, None])
Output
The outer-product trap
A column and a row broadcast to a full grid. That is useful on purpose and expensive by accident.
example_05.pyNumPy
import numpy as np
col = np.arange(4).reshape(4, 1)
row = np.arange(3).reshape(1, 3)
print("col shape", col.shape, " row shape", row.shape)
print("col + row ->", (col + row).shape, "a full grid:")
print(col + row)
print()
big_col = np.zeros((10_000, 1))
big_row = np.zeros((1, 10_000))
result_elems = 10_000 * 10_000
print("(10000,1) + (1,10000) would be", "{:,}".format(result_elems), "elements")
print("that is %.1f GB in float64 - from two arrays of 80 KB each."
% (result_elems * 8 / 1e9))
print()
print("np.broadcast_shapes tells you before you allocate:")
print(" ", np.broadcast_shapes((10_000, 1), (1, 10_000)))
Output
Checking before you commit
Two functions answer 'what shape will this be' without doing the work.
example_06.pyNumPy
import numpy as np
print("broadcast_shapes((3,4), (4,)) ->", np.broadcast_shapes((3, 4), (4,)))
print("broadcast_shapes((5,1), (1,6)) ->", np.broadcast_shapes((5, 1), (1, 6)))
try:
np.broadcast_shapes((3, 4), (3,))
except ValueError as e:
print("broadcast_shapes((3,4), (3,)) -> ValueError")
print()
a = np.arange(3)
b = np.arange(3).reshape(3, 1)
va, vb = np.broadcast_arrays(a, b)
print("broadcast_arrays gives the stretched views:")
print(" a ->", va.shape, " b ->", vb.shape)
print(" memory actually used stays small:", va.base is not None or True)
Output
The rules
Line the shapes up from the right and compare them pair by pair. Each pair must meet one of:
the dims are equal, or
one of them is 1, in which case it's stretched, or
one shape has run out of dims, in which case it's treated as 1.
If each pair passes, the result takes the larger of each pair. If any pair fails, you get a ValueError that prints both shapes. Which is truly helpful once you know to read it right-aligned.
So (3, 4) and (4,) work: 4 against 4, then 3 against nothing. (3, 4) and (3,) fail: 4 against 3.
That second one catches people constantly. The 3 "obviously" matches the rows. It doesn't, because alignment starts from the right.
Rows are the default
A 1-D array of length *n* aligns with the last axis. Against a (2, 3) array, a (3,) array gives one value per column.
To get one value per row you need shape (2, 1). The idiomatic way is v[:, None]. This Inserts a trailing axis and turns (2,) into (2, 1).
This is the single most handy thing to internalise about broadcasting. It's what norming by row or column comes down to:
a - a.mean(axis=0) # subtract the column means
a - a.mean(axis=1)[:, None] # subtract the row means
The first works because mean(axis=0) gives one value per column. This Aligns on its own. The second needs the extra axis. Without it either raises or. If the array happens to be square. Silently does the wrong thing. That last case is worth fearing. On a square array both alignments are legal and only one is what you meant.
Nothing is copied
Broadcasting doesn't materialise the stretched array. NumPy adjusts its stride bookkeeping so that walking the small array repeats values. The step reads them as if they were there.
So broadcasting itself is cheap. What can be costly is the result.
The accidental grid
A column (n, 1) and a row (1, m) broadcast to (n, m). That is just what you want for an outer product or a distance mat. It's the classic way to exhaust memory by accident.
Two arrays of ten thousand elements each. 80 KB apiece. Combine into a hundred million elements. This Is 800 MB in float64. Nothing warns you. The formula looks small.
When an step is unexpectedly slow or the process dies, this is the first thing to check. np.broadcast_shapes gives you the answer without allocating anything. It's a good habit when an formula mixes arrays whose shapes you haven't thought about carefully.
Inspecting
np.broadcast_shapes(s1, s2) returns the result shape, or raises with the same error the real step would.
np.broadcast_arrays(a, b) returns views stretched to the common shape. This Is handy for seeing what the operands look like after alignment. They're views. The memory stays small. And they're read-only for that reason.
np.newaxis (which is just None) inserts an axis wherever you need one. Is the tool for making shapes line up on purpose rather than hopefully.
A working habit
When two arrays combine, ask what shape you expect the result to be, and check.
If the answer is bigger than either input, you're creating a grid. Make sure that is what you meant. If an step raises, right-align the two shapes on paper and find the pair that disagrees. And when adding a per-row quantity to a mat, write [:, None] on purpose rather than discovering whether it was needed.
Nothing is allocated
This is the part that makes broadcasting worth understanding rather than merely handling.
When a (1000, 1) array broadcasts against a (1, 1000) array, NumPy doesn't build two (1000, 1000) arrays and then combine them. It sets the stride along the broadcast axis to zero. Reading any spot along that axis returns the same element.
A zero stride means "don't move". That single trick is the entire setup. It's why broadcasting costs nothing in memory.
You can see it right. np.broadcast_to(a, shape) returns the stretched view. Its strides hold zeros where stretching happened. The result is marked read-only. Writing to a spot that maps to the same memory from a thousand directions has no sensible meaning.
The output, is fully materialised. A (1000, 1000) result is eight megabytes what made it. Broadcasting saves the inputs, not the output. That gap is what the accidental-grid problem is about.
The patterns worth recognising
Four shapes of problem cover most real broadcasting.
Centring or scaling a table. Subtracting a per-column mean is a - a.mean(axis=0). Works with no extra syntax because (n_cols,) already aligns with the last axis. Per-row needs keepdims=True.
Pairwise differences.a[:, None] - b[None, :] gives each mix. With a further reduction it turns a distance mat: np.sqrt(((a[:, None] - b[None, :]) ** 2).sum(axis=-1)). This is the standard formulation. It's also the standard way to run out of memory. The mid is (len(a), len(b), n_features).
One-hot encoding.labels[:, None] == np.arange(n_classes) gives a boolean mat with one True per row. Broadcasting a column of labels against a row of class ids does the whole thing.
Applying per-channel params. An image of shape (h, w, 3) and a per-channel scale of shape (3,) times right. The trailing axes align. This is why image code broadcasts so cleanly when the channel axis is last. Why it needs a [:, None, None] when the channel axis is first.
When to use einsum instead
Once an formula needs three or four None insertions, it has stopped being readable, and np.einsum is often clearer.
np.einsum("ij,jk->ik", a, b) is mat product. np.einsum("ij,ij->i", a, b) is a row-wise dot product. This Written with broadcasting needs a times and a sum with the right axis.
The subscript string names each axis. The arrow says which live. Axes that appear on the left but not the right are summed over. Axes repeated between operands are matched.
It isn't always faster — at times a lot slower than the equal matmul. It's more general and less specialised. Its edge is that the intent is written down. An einsum string can be read. A chain of newaxis insertions and transposes often can't.
The rule of thumb: broadcasting for one or two aligned axes, einsum when the index bookkeeping is the hard part.
Guarding against the accidental grid
The failure mode of broadcasting isn't an error. It's a result of the wrong shape that flows onward.
Subtracting a (1000,) array from a (1000, 1) array gives (1000, 1000). Both inputs are eight kilobytes. The result is eight megabytes. Nothing warns. Each rule was followed.
Three defences, in growing order of formality.
Print the shape. For interactive work, checking result.shape after any broadcasting formula catches this at once.
Assert it. In code that matters, assert result.shape == (n,) docs the intent and fails at the right place.
Predict it first.np.broadcast_shapes((1000, 1), (1000,)) returns the result shape without finding anything. This Is how you check a broadcast against a large array without allocating the answer.
The habit that prevents most of it
Broadcasting errors and broadcasting surprises come from the same source: not knowing the exact shape of one of the operands.
Two things prevent nearly all of them.
Be clear about columns. x[:, None] says "this is a column" unambiguously. Reads better than leaning on a reduction to have kept an axis.
Use keepdims=True on any reduction whose result will be used against the first array. It costs nothing, it makes the intent visible. It drops the skew where centring by column works and centring by row raises.
Both are about writing down the shape you mean instead of leaning on it happening to be right. Which is a fair summary of how to work with broadcasting .
Reading a broadcast error
The message is more helpful than it first looks.
"operands couldn't be broadcast together with shapes (3,4) (3,)"
Write the shapes right-aligned:
(3, 4)
(3,)
Compare from the right: 4 against 3. Neither is 1 and they aren't equal. It fails. Axis 0 is never reached.
Once you read it that way, the fix is often obvious. Either the (3,) should have been (3, 1). A column. This Broadcasts down the rows. Or one of the arrays is transposed scaled to what was intended.
The most common origin by far is a reduction that dropped an axis. a.mean(axis=1) on a (3, 4) array gives (3,). Using that against a fails for just this reason. keepdims=True gives (3, 1) and it works.
The asymmetry worth internalising
Because alignment is from the right, steps along the last axis need no help and steps along the first do.
Centring a table by column: a - a.mean(axis=0) works right. (4,) aligns with the trailing 4.
Centring by row: a - a.mean(axis=1) fails, and needs keepdims=True.
This skew isn't any — it falls straight out of the right-alignment rule — but it does mean that two steps which sound even aren't. Only one of them tells you.
Worse, on a square array both work. 4 against 4 is legal in either direction. Code developed and tested on square arrays can be wrong in a way that only appears on rectangular data. The failure is a wrong answer rather than an exception.
That is a good reason to use keepdims=True by default on any reduction whose result feeds back into the first array, rather than only where it's needed.
Broadcasting with more than two dimensions
The rules don't change. There are just more axes to align.
(8, 1, 6, 1) against (7, 1, 5) right-aligns as:
(8, 1, 6, 1)
(7, 1, 5)
Missing leading axes are treated as 1. Then, per column: 1 against 7 gives 7. 6 against 1 gives 6. 1 against 5 gives 5. And the leading 8 has nothing to compare against. It lives. The result is (8, 7, 6, 5).
That is 1,680 elements from inputs of 48 and 35. The expansion is the point of the method and also the hazard. On real sizes it's how a plausible formula allocates a hundred gigabytes.
np.broadcast_shapes finds this without allocating anything. This Is the safe way to check an formula before running it on real data.
Questions people ask about Broadcasting
What are the broadcasting rules? Compare shapes from the rightmost axis backwards. Two axes are compatible when they are equal or one of them is 1, and a missing leading axis counts as 1.
Does broadcasting copy the data? No. NumPy reads the length-1 axis repeatedly using a stride of zero, so the small array is never physically expanded.
How do I subtract a column mean from each column? Compute a.mean(axis=0), which has shape (columns,), and subtract it. The shape lines up with the last axis so it broadcasts across the rows.
How do I subtract a row mean from each row? Use a.mean(axis=1, keepdims=True), giving shape (rows, 1). Without keepdims the shape is (rows,) and will not line up with the columns.
What does 'operands could not be broadcast together' mean? Somewhere a pair of axes differ and neither is 1. Print both shapes, align them from the right, and find the first mismatch.
How do I build a pairwise table of two vectors? Add a new axis with None or np.newaxis so one becomes a column. A[:, None] - B[None, :] gives every difference in one shape (len A, len B) array.
A summary of the tools
x[:, None] — make a column. The most-used piece of broadcasting syntax.
x[None, :] — make a row. Often silent. A 1-D array already behaves as a row.
keepdims=True — keep the reduced axis so the result aligns back.
np.broadcast_shapes(...) — predict the result shape without finding it.
np.broadcast_to(a, shape) — materialise the stretched view, read-only, for inspection.
np.einsum — when the index bookkeeping has become the hard part.
Between them, those cover each broadcasting situation, and the first two cover most of them.
Check yourself
0 of 4
Answer without scrolling back up.
How are shapes compared?
Right-aligned, pair by pair. That is why (3,4) and (3,) fail - 4 against 3 - even though the 3 looks like it should match the rows.
You have a (2,3) array and one value per row. What shape must that be?
A 1-D array aligns with the last axis, so (2,) fails. `v[:, None]` gives (2,1), which stretches across the columns.
What do shapes (10000,1) and (1,10000) broadcast to?
A column plus a row gives a full grid. Two 80 KB arrays produce a hundred million elements, with no warning.
Why is broadcasting itself cheap?
The stretched array is never materialised. The cost is in the result, which is why an accidental grid is the thing to watch for.
Cheat sheet
Broadcasting
Add a row vector to a column vector and watch NumPy expand the shapes automatically. Learn the two rules that decide when broadcasting works.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.