Where the time and the bytes actually go, and the handful of changes that move the needle.
Overview
The order to try things
Performance tuning in NumPy follows a predictable order, and the biggest gains come from the simplest changes. The most expensive mistake is to optimise the wrong thing first.
The first question is whether there is still a Python loop. A for-loop over array elements drops from compiled code into the Python interpreter, which is roughly a hundred times slower. Replacing the loop with a vectorised operation gives you that factor back. Most NumPy performance problems are this problem, and most people skip it to look for fancier solutions.
The second question is dtype width. An array of small integers stored as int64 uses eight times the memory of int8. An array of moderate floats stored as float64 uses twice the memory of float32. Smaller dtypes mean more elements fit in CPU cache at once, and that speeds up every operation that touches the array. The speedup is typically 1.5x to 2x, on top of the memory savings.
The third question is memory layout. Operations that walk through contiguous memory are faster than operations that jump around. A transposed array or a column slice of a C-contiguous array is not contiguous along the access direction. Making a contiguous copy with np.ascontiguousarray before a tight loop can help, but only if the loop is long enough to repay the copy.
The fourth question is temporary arrays. An expression like a * b + c creates a temporary array for a * b, then creates another for the addition. For large arrays, those temporaries cost time and memory. np.multiply(a, b, out=result) reuses an existing array and avoids the allocation. numexpr can fuse the whole expression into one pass.
The fifth and last question is whether NumPy is the right tool. For very large data that does not fit in memory, memory-mapped arrays or chunked processing help. For operations that are hard to vectorise, writing a small C or Cython extension, or using Numba's JIT compiler, can close the gap.
This module works through the list in order, with timing measurements that show the payoff of each step. The point is not just what to do, but what to do first.
Most NumPy speed work comes down to four questions, roughly in order of payoff:
Is there still a Python loop? Dropping it's worth 10–100x.
Is the dtype larger than it needs to be? Halving it's worth up to 2x, and halves memory.
Are temporaries being allocated in a hot path? In-place steps drop them.
Is the memory access pattern fighting the cache? Layout changes can be worth 2–5x.
The first is covered in the introductory module and dwarfs the rest. This module is about the other three. This Matter once the obvious loop is gone.
Worth knowing
Choosing float32 over float64 halves memory and usually speeds things up, because less data has to move.
Every intermediate expression allocates a full-size temporary array. a * 2 + 1 allocates twice.
In-place operators (*=, +=) and out= reuse memory you already have.
C order makes the last axis contiguous. Reading strided memory (copying a transpose) is several times slower than reading consecutive bytes.
Preallocate with np.empty and assign, rather than growing an array — np.append in a loop is quadratic.
Measure with nbytes and a timing loop, and take the best of several runs rather than the average.
Performance and Memory
Where the time and the bytes really go.
dtype is the cheapest saving there is
Half the width, half the memory, and usually faster because less has to travel.
example_01.pyNumPy
import numpy as np
n = 200_000
for dt in [np.float64, np.float32, np.int64, np.int32, np.int8, np.bool_]:
a = np.ones(n, dtype=dt)
print("%-9s itemsize %d total %6.2f MB"
% (np.dtype(dt).name, a.itemsize, a.nbytes / 1e6))
print()
print("float32 halves a float64 array and keeps ~7 significant digits.")
print("For measurements that came from a sensor with 3, that is plenty.")
Output
Temporaries are the hidden cost
Every intermediate expression allocates a full-size array you never see.
example_02.pyNumPy
import numpy as np
import time
a = np.random.default_rng(0).random(1_000_000)
t = time.perf_counter()
b = a * 2 + 1 # allocates (a*2), then allocates the result
r1 = time.perf_counter() - t
c = a.copy()
t = time.perf_counter()
c *= 2 # no allocation at all
c += 1
r2 = time.perf_counter() - t
print("a * 2 + 1 : %.4f s (2 temporary arrays)" % r1)
print("in place *= += : %.4f s (0)" % r2)
print("same answer :", np.allclose(b, c))
print()
print("Each temporary here is 8 MB. On a chain of five operations")
print("that is 40 MB of allocation for one line of arithmetic.")
Output
out= writes into memory you already have
The explicit form of the same idea, and it works with any ufunc.
example_03.pyNumPy
import numpy as np
a = np.arange(6, dtype=float)
dest = np.empty_like(a)
np.multiply(a, 3, out=dest)
print("dest after multiply:", dest)
np.add(dest, 1, out=dest)
print("dest after add :", dest, "<- read and written in one buffer")
print()
print("useful in a loop where the destination is reused:")
acc = np.zeros(4)
for row in np.arange(12, dtype=float).reshape(3, 4):
np.add(acc, row, out=acc)
print(" accumulated:", acc)
Output
Strided access costs more than contiguous access
Same number of bytes, same result - only the order they are read in differs.
example_04.pyNumPy
import numpy as np
import time
def best(fn, reps=5):
b = 1e9
for _ in range(reps):
t = time.perf_counter(); fn(); b = min(b, time.perf_counter() - t)
return b
a = np.ones((1200, 1200))
t1 = best(lambda: a.copy()) # consecutive bytes
t2 = best(lambda: np.ascontiguousarray(a.T)) # jumps a row every element
print("copy of a : %.5f s" % t1)
print("copy of a.T : %.5f s" % t2)
print("strided is %.1fx slower" % (t2 / t1))
print()
print("Both move 11 MB and produce 11 MB. The transpose reads one value")
print("from each row in turn, so almost every read is a cache miss.")
print()
print("Reductions are not a good test of this - NumPy blocks them so both")
print("directions run at similar speed. It shows up when data is MOVED.")
Output
Preallocate instead of growing
The same lesson as concatenating in a loop, stated as a habit.
example_05.pyNumPy
import numpy as np
import time
n = 3000
t = time.perf_counter()
grow = np.array([])
for i in range(n):
grow = np.append(grow, i)
t1 = time.perf_counter() - t
t = time.perf_counter()
pre = np.empty(n)
for i in range(n):
pre[i] = i
t2 = time.perf_counter() - t
t = time.perf_counter()
best = np.arange(n, dtype=float)
t3 = time.perf_counter() - t
print("np.append in a loop : %.4f s" % t1)
print("preallocated : %.4f s" % t2)
print("vectorised : %.6f s" % t3)
print("all equal :", np.array_equal(grow, pre) and np.array_equal(pre, best))
Output
Measuring rather than guessing
nbytes for size, and a timing loop for anything you are about to optimise.
example_06.pyNumPy
import numpy as np
import time
a = np.ones((500, 500))
print("shape %s dtype %s" % (a.shape, a.dtype))
print("nbytes : %.2f MB" % (a.nbytes / 1e6))
print("as float32: %.2f MB" % (a.astype(np.float32).nbytes / 1e6))
print()
def timeit(fn, reps=5):
best = 1e9
for _ in range(reps):
t = time.perf_counter(); fn(); best = min(best, time.perf_counter() - t)
return best
print("sqrt : %.5f s" % timeit(lambda: np.sqrt(a)))
print("a ** 2 : %.5f s" % timeit(lambda: a ** 2))
print("a * a : %.5f s" % timeit(lambda: a * a))
print()
print("Take the BEST of several runs, not the average - the slow runs")
print("are measuring the machine, not the code.")
Output
dtype
float64 is the default and is often more precision than the data justifies. Sensor readings good to three large figures don't gain from fifteen.
float32 keeps about seven large digits, halves the memory. Often runs faster. Not because the math is cheaper. Because half as much data has to travel from memory to the processor, and memory bandwidth is often the limit.
The same applies to ints. An array of small counts doesn't need int64. An array of flags should be bool_. This Is one byte, not int64, which is eight.
The caution from the dtypes module still applies. Narrow ints overflow silently. Choose the smallest type that can't overflow for your actual data, not the smallest that happens to fit today's sample.
Temporaries
a * 2 + 1 on a million-element array doesn't do one pass. It allocates an 8 MB temporary for a * 2, then allocates another 8 MB for the final result, then frees the first.
You never see them, and for a single formula it rarely matters. In a loop, or in a chain of five steps on a large array, the allocation and the extra memory traffic dominate.
Two ways to avoid it.
In-place ops.a *= 2 modifies the existing buffer and allocates nothing. note that it also modifies anything sharing that buffer. This Is the views-versus-copies problem again.
**out=.** each ufunc accepts out. This Names the destination by name. np.times(a, 3, out=dest) writes into dest. out may be one of the inputs. This is the clearest form when you're building up into a buffer across loops.
Neither is worth doing all over. a * 2 + 1 is more readable than two statements. Readability wins until you've measured that this line matters.
Layout and the cache
A C-sorted array stores the last axis contiguously. in a row elements along that axis sit next to each other in memory.
Processors fetch memory in cache lines. Reading in a row bytes is far cheaper than jumping a row at a time.
Reductions are a poor demo of this. NumPy blocks them inside. sum(axis=0) and sum(axis=1) often run at similar speed and at times in the order you wouldn't predict. It's worth knowing that. The "always reduce along the last axis" advice is repeated widely and doesn't live reading.
Where it shows clearly is when data is really moved. Copying a (1200, 1200) array and copying its transpose move the same number of bytes and give the same number of bytes. The transpose reads one value from each row in turn. Runs a few times slower.
Two hands-on results. If you've a choice about which axis holds the thing you iterate over, put it last. And if you're about to make many strided passes over the same data, one np.ascontiguousarray up front can pay for itself. A single copy against many slow reads.
Preallocation
np.append and np.concatenate don't append. They allocate a new array and copy all. In a loop that is quadratic. It's the single most common accidental slowdown in NumPy code after the plain Python loop.
If you know the final size, np.empty(n) and set by index is the direct answer. np.empty doesn't reset. It's faster than np.zeros when you're going to overwrite all. And dangerous if you aren't. The contents are what was in that memory.
If you don't know the size, collect into a Python list and convert once at the end. Lists are designed for growth. Arrays aren't.
Measuring
Guessing is unreliable, and the intuitions transferred from pure Python are often wrong.
a.nbytes gives the real size of the data. a.itemsize gives the per-element cost.
For time, run the step a few times and take the min, not the mean. The slow runs are measuring interference from the rest of the machine. The fastest run is the closest guess of the actual cost. A single timed run of a fast step measures mostly noise.
And measure the thing you intend to change. Optimising an step that accounts for 2% of the runtime is effort spent for a 2% ceiling. Meeting the local speedup looks.
Views and memory that will not go away
A view holds a link to the array it came from. The entire base buffer stays alive if the view does.
Extract one row from an 8 MB array, keep it, and you're keeping 8 MB.
This is the most common source of NumPy memory that never comes back. It's invisible. row.nbytes reports four kilobytes while the process holds eight megabytes.
The fix is copy() at the point you decide to keep a thing small from a thing large. The cost is one small allocation. The saving is the whole base.
a.base shows what a view derives from. a.base.nbytes shows what is really being held. That is the diagnostic when memory in a long-running process grows without an obvious cause.
Threads and the GIL
NumPy releases the GIL during most array steps. This Means threads truly run in parallel inside a NumPy math. Unlike most pure Python code.
That makes concurrent.futures.ThreadPoolExecutor a real option for parallelising array work, with none of the pickling and process-startup costs of multiprocessing.
Two caveats. steps that touch Python objects. Object arrays, anything calling back into Python. Don't release it. And BLAS routines are often already multi-threaded inside. Adding your own threads on top can oversubscribe the machine and run slower. OMP_NUM_THREADS and threadpoolctl control that layer.
Measure before given threading helps. For large mat steps it often does nothing. BLAS was already using each core.
Chunking
When an step would allocate more than fits in memory, the answer is often to do it in pieces.
The classic case is a pairwise distance mat. For 100,000 points, the mid is 10 billion entries and no machine will hold it. handling 1,000 rows at a time keeps the mid at 100 million and gives the same answer.
The general shape:
for start in range(0, n, chunk):
block = data[start:start + chunk]
out[start:start + chunk] = expensive(block)
The slices are views, so the chunking itself allocates nothing. The output is preallocated once.
This is one of the cases where a loop is correct. The loop runs n / chunk times, not n times. The interpreter cost is negligible and the memory ceiling is the thing being controlled.
When to reach past NumPy
NumPy has a ceiling, and recognising it saves effort spent optimising against it.
Numba compiles a Python function to machine code with a decorator. It's the right answer for truly sequential methods. The ones that can't be vectorised. And often gets within range of C for a few lines of change.
Cython gives more control at the cost of a build step. Is what a few scientific libraries use inside.
SciPy already holds a compiled setup of a great many things people write by hand. Distance mats, sparse mats, signal filters, tuning, blending. Checking whether SciPy has it's cheaper than writing it.
Dask handles arrays larger than memory with a NumPy-like interface, splitting work into chunks on its own.
CuPy / PyTorch move the work to a GPU, with an interface close enough to NumPy that porting is often rote. Worth it for large array workloads, not worth it for small ones where the transfer cost dominates.
The order to consider them: is it vectorisable in NumPy, is it in SciPy, is it sequential and worth Numba, is it too big for memory, is it big enough for a GPU.
Profiling before optimising
The instinct about where time goes is unreliable, and array code is no exception.
cProfile gives function-level timings and finds the hot function. line_profiler gives per-line timings within it. This Is what you really need when one function holds the whole math.
memory_profiler tracks allocation line by line. a.nbytes summed over the arrays you're holding is a quick manual version.
Two rules make profiling worth the time.
Profile the real workload. A toy input can have fully other traits. Other cache action, other branch of the method, other memory pressure.
Fix the top item, then measure again. The bottleneck moves. Optimising the second item on the first list is often wasted. After the first fix it's no longer second.
And keep the ceiling in view: a function taking 5% of runtime can't give more than a 5% gain but fully you optimise it.
Where the wins actually are, in order
Drop the Python loop. 10–100x, and it dominates all else. If a loop over elements is still there, nothing below this line matters yet.
Avoid quadratic growth.np.append or concatenate in a loop turns a linear job into a quadratic one. The fix is preallocation or a list.
Right-size the dtype. Up to 2x on bandwidth-bound work, and it halves memory.
Drop temporaries in hot paths. In-place ops and out=. Worth doing where reading says it matters, not all over.
Fix access patterns. Contiguity, and avoiding repeated strided gathers.
Reach past NumPy. Numba, SciPy, a GPU. When the method truly doesn't vectorise or the data truly doesn't fit.
Working down that list in order means the large wins come first. It avoids the common trap of micro-optimising an formula inside a loop that shouldn't exist.
Things that look slow and are not
Slicing. A view. Free, what the array size.
Transposing. A view. Free.
Reshaping a contiguous array. A view. Free.
**Reversing with [::-1].** A view. Free.
Broadcasting the inputs. No allocation. The strides are set to zero. The *output* is allocated. This Is a other thing.
Rewriting any of these to "avoid the copy" is effort spent on a copy that was never happening. Confirm with np.shares_memory before optimising a thing that may already be free.
Things that look cheap and are not
Boolean masking and fancy indexing. Both allocate, and both gather from scattered spots.
**astype.** Allocates a full second array, even when the dtype is unchanged, unless copy=False.
Chained expressions on large arrays. each mid is a full-size allocation.
**np.append.** Not an append. A full copy, each call.
Keeping a small view of a large array. Holds the whole base alive.
The discipline
Measure before optimising, and profile the real workload rather than a small stand-in. Cache action and memory pressure don't scale down predictably.
Take the min of a few timed runs, not the mean.
Fix the top item and measure again, because the bottleneck moves.
Keep the ceiling in view: an step taking 5% of the runtime can't yield more than 5%. Thoroughly it's optimised.
And weigh readability honestly. A vectorised one-liner that no one can safely modify has a maintenance cost that doesn't show up in a benchmark. The version a colleague can read is often the better engineering answer even when it's slower.
Questions people ask about Performance and Memory
Why is a NumPy loop so much faster than a Python loop? The loop runs in compiled C over contiguous memory, with no per-element Python object overhead or type checks.
How much memory does an array use? Read a.nbytes, which is the element count times the item size. The shape and dtype fields add only a small fixed overhead.
How do I avoid temporary arrays? Use in-place operators like a += b, or the out= argument on ufuncs. Chained expressions each allocate an intermediate array.
Does memory layout matter? Yes. Walking along the contiguous axis is faster because of CPU caching. In a C-ordered array that means iterating over the last axis.
When should I use float32? When memory or bandwidth is the limit and you can accept about seven digits of precision. It halves the footprint and often speeds up big operations.
How do I time code properly? Use timeit or %timeit, which repeats the run and takes the best result. A single time.time() measurement is noisy.
The lock behind two of these timings
Some of the numbers above only make sense in the presence of the interpreter lock. It deserves a sentence rather than a hand-wave. CPython holds one lock around bytecode run. Two Python-level threads never execute Python also. But a NumPy call that spends its time in C can *release* that lock while it works. This Is why a large dot or a big elementwise step shares across threads and a Python loop over the same array doesn't.
So the vectorised version isn't merely faster. It's differently parallel. That is a property of where the time is spent rather than of NumPy's API.
The GIL, and what it really locks. Measured. Threads share waiting and don't share finding, and what "releases the GIL" means at the C boundary.
Why is `float32` often faster than `float64`, not just smaller?
Most array operations are bandwidth-bound rather than compute-bound, so halving the bytes roughly halves the time.
How many temporary arrays does `a * 2 + 1` allocate?
On a million float64 elements that is 8 MB you never see. In-place operators or `out=` remove them - but only bother once you have measured that the line matters.
Copying an array and copying its transpose move the same bytes. Why is the transpose slower?
Processors fetch cache lines, so consecutive reads are far cheaper than jumping. Reductions are a poor test of this - NumPy blocks them and both directions run at similar speed.
When timing an operation, should you take the mean or the minimum of several runs?
Slow runs measure interference from the rest of the machine. The fastest run is the closest estimate of the actual cost.
Cheat sheet
Performance and Memory
Profile a NumPy workflow to find where time and memory go. Replace Python loops, cut unnecessary copies and pick smaller dtypes.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.