Race Conditions: Why x += 1 Is Three Operations
The GIL makes one bytecode atomic. An increment is not one bytecode, and the gap is where a count goes missing.
The GIL makes one bytecode atomic. An increment is not one bytecode, and the gap is where a count goes missing.
This explorer needs JavaScript: every interleaving, cost and speed-up on it is computed in the page rather than downloaded as an image.
Read the middle three. LOAD_GLOBAL reads the current value, BINARY_OP adds one to it, and STORE_GLOBAL writes the result back. A read, a modify, and a write — three separate steps, with two gaps between them, and a thread switch is allowed to land in either gap.
The previous page ended on a fact worth repeating, because everything here follows from it: a thread switch can only happen between bytecodes, so a single bytecode is atomic with respect to other threads.
counter += 1 compiles to:
RESUME
LOAD_GLOBAL counter
LOAD_CONST 1
BINARY_OP +=
STORE_GLOBAL counter
RETURN_CONST NoneThat is the whole mechanism. There is nothing subtle left; the rest of this page is about what happens when the switch lands there.
The editors here run in a browser interpreter with no OS threads, so the two-thread versions are shown with their CPython output and the interleaving itself is simulated. That turns out to be an advantage: a real race is intermittent, but a simulated one happens every time.
Take two threads, each running counter += 1 exactly once, starting from zero. The expected answer is 2. Here are the same six steps in two different orders:
no switch (T1 finishes first)
T1 reads counter = 0
T1 adds -> 1 (in its own register)
T1 writes counter = 1
T2 reads counter = 1
T2 adds -> 2 (in its own register)
T2 writes counter = 2
=> counter = 2 after two increments
switch after T1 reads
T1 reads counter = 0
T2 reads counter = 0
T2 adds -> 1 (in its own register)
T2 writes counter = 1
T1 adds -> 1 (in its own register)
T1 writes counter = 1
=> counter = 1 after two incrementsIn the second order both threads read zero. Each then computes 1 in its own register, entirely correctly, and each writes 1. Two increments happened and the counter advanced once: the second write did not build on the first, it replaced it. Nothing was corrupted and no operation was interrupted half-way — every individual step did exactly what it should.
That is what makes this class of bug hard. There is no broken line to find. The defect is in the *interleaving*, which is not written down anywhere in the program.
With two increments the damage is at most one. With two threads each doing a hundred thousand increments, the damage is however many times the switch happened to land in a gap:
import threading
counter = 0
def bump_many(n):
global counter
for _ in range(n):
counter += 1
threads = [threading.Thread(target=bump_many, args=(100_000,)) for _ in range(2)]
for t in threads: t.start()
for t in threads: t.join()
print("expected:", 200_000)
print("actual: ", counter)expected: 200000
actual: 137216Sixty thousand increments gone, and the number is different on every run. Two properties of that output are the reason races survive code review.
It is non-deterministic: the same program gives 200000, 137216 or 199998 depending on when the operating system happened to switch. A test that asserts the total can pass a hundred times and fail on the hundred and first.
And it is load-dependent: with the loop set to 1,000 rather than 100,000, most runs come out exactly right, because the whole loop often completes inside one 5 ms slice. Small tests hide it and production finds it.
A lost update is the famous version. The more common one in real code is check-then-act, which does not look like an increment at all:
no switch
T1 checks 'k' in store -> False
T1 acts on that: store['k'] = []
T1 appends -> ['T1']
T2 checks 'k' in store -> True
T2 acts on that: nothing to do
T2 appends -> ['T1', 'T2']
=> store = {'k': ['T1', 'T2']}
switch after both checks
T1 checks 'k' in store -> False
T2 checks 'k' in store -> False
T1 acts on that: store['k'] = []
T1 appends -> ['T1']
T2 acts on that: store['k'] = []
T2 appends -> ['T2']
=> store = {'k': ['T2']}Both threads looked, both correctly saw the key was missing, and both created it. T2's fresh list replaced the one T1 had already appended to, so T1's item is gone — and note that nothing was *overwritten* in the increment sense. The value that disappeared was written into a container that was then thrown away.
This shape hides everywhere: a lazily-built cache, a "create the directory if it does not exist", a counter dictionary, an "insert if not present" against a database. The giveaway is a gap between deciding and doing, and the decision going stale inside it.
Two of these have a one-line fix that removes the gap rather than locking it. dict.setdefault(k, []) does the check and the create in one C call, and collections.defaultdict(list) does it at lookup time. Both are safe for exactly the reason append is: the whole thing is one trip into C with no Python running in the middle.
Not every line needs protecting, and the rule is mechanical: an operation implemented as a single bytecode — or as one call into C that does not run Python in the middle — cannot be interleaved.
append_one 4 ops: ['LOAD_FAST', 'LOAD_ATTR', 'LOAD_CONST', 'CALL']
index_incr 9 ops: ['LOAD_FAST', 'LOAD_CONST', 'COPY', 'COPY', 'BINARY_SUBSCR', 'LOAD_CONST', 'BINARY_OP', 'SWAP', 'SWAP', 'STORE_SUBSCR']
swap 3 ops: ['LOAD_FAST', 'LOAD_FAST', 'BUILD_TUPLE']list.append ends in a single CALL into C that appends and returns without running any Python, so the append itself cannot be split — which is why appending to a list from several threads does not lose items. d["n"] += 1 is nine operations with a BINARY_SUBSCR read and a STORE_SUBSCR write at opposite ends, and it is exactly as unsafe as the global version.
The practical list that follows:
| Operation | Safe from other threads? |
|---|---|
lst.append(x), lst.pop() | yes, one C call |
d[k] = v | yes |
d.setdefault(k, v) | yes |
x += 1, d[k] += 1 | no, read-modify-write |
if k not in d: d[k] = v | no, check then act |
lst[0], lst[1] = lst[1], lst[0] | no |
The second and fifth rows are the same shape as the increment and are worth recognising: check-then-act is a read and a write with a gap, whatever it looks like. Relying on this table is also fragile, because it describes the current compiler rather than the language. The reliable habit is to protect shared mutable state with a lock and stop reasoning about bytecode counts.
Do not share the state. A thread that owns its data has no race. Give each worker its own accumulator and combine at the end, or hand results back through a queue. This removes the problem rather than managing it and is almost always the right first answer.
Use a lock. Wrapping the read-modify-write in with lock: makes the three steps one indivisible unit. It is the general answer and it is the next page.
Use something already atomic. itertools.count() hands out distinct integers safely because next() is one C call. A queue.Queue is internally locked. collections.deque has thread-safe appends and pops at both ends.
Here is the same two-thread program with the three steps made indivisible, on CPython:
import threading
counter = 0
lock = threading.Lock()
def bump_many(n):
global counter
for _ in range(n):
with lock:
counter += 1
threads = [threading.Thread(target=bump_many, args=(100_000,)) for _ in range(2)]
for t in threads: t.start()
for t in threads: t.join()
print("expected:", 200_000)
print("actual: ", counter)expected: 200000
actual: 200000Exactly right, and exactly right on every run, which is the property the unlocked version could not offer. with lock: means a thread must hold the lock to be inside the block, so the read, the add and the write happen with no other thread able to interleave between them. The switch can still land mid-block — the GIL does not care about your lock — but the other thread will immediately block on acquire and hand the interpreter straight back.
What it costs is worth knowing before you reach for it everywhere. Acquiring and releasing a lock two hundred thousand times is real work, and the loop above is measurably slower than the broken one. It is also *correct*, which is the trade being made, and the way to get most of the speed back is to hold the lock around the loop rather than around the line — which is the granularity question the next page measures.
Assuming the GIL protects statements. It protects bytecodes. A statement is usually several.
Testing with small numbers. A short loop finishes inside one time slice and the race never fires. Push the iteration count up by a hundred and it appears.
Fixing it by adding a sleep. Changing the timing hides a race rather than removing it; the interleaving is still legal and will happen under different load.
Protecting the write but not the read. A lock only helps if *every* access to the shared state takes it, including the ones that only read and then decide.
Assuming a debugger will show you. Running under a debugger or a profiler changes the scheduling, so the class of bug most in need of one is the class least likely to reproduce under it.
Say the answer out loud before you reveal it — recalling it is what makes it stick, and rereading it is not.
What does this module say about “One statement, several bytecodes”?
What does this module say about “The lost update, step by step”?
What does this module say about “At scale, and why testing misses it”?
Run x += 1 from two threads, watch the count drift, and trace the three bytecodes that open the gap where a value goes missing.