Reading a path
people[0]["langs"][1]
Left to right: index the list, look up the key, index the inner list. Each step returns an object. The next step operates on that object. If you are unsure what a line does, break it apart one piece at a time: people[0], then people[0]["langs"]. That is exactly what a debugger shows you.
Nested data structures appear the moment you work with real-world data. A list of students, each represented as a dictionary with a name, an age, and a list of courses. A JSON response from a web API, where the top level is a dictionary, one of its values is a list, and each list element is another dictionary. A game board represented as a list of lists. These are not exotic examples. They are the default shape of most data you will work with outside a textbook.
Reading a nested structure is a matter of chaining access operations. Each bracket or dot takes you one level deeper. The type at each level determines what operation you use next. A list needs an integer index. A dictionary needs a string key. An object needs an attribute name. Getting comfortable with this chaining is one of the most practical skills in Python.
Writing to a nested structure works the same way. people[0]["langs"].append("rust") reaches into the first person's language list and adds an item. The chain of access operations locates the inner object, and then the method call modifies it in place.
Walking a nested structure usually requires nested loops or recursion. A list of dictionaries needs a for loop over the list, with dictionary access inside. A tree-shaped structure needs recursion. The depth of the nesting determines the number of loops or recursive calls.
This page covers reading nested paths, modifying inner objects, looping over nested structures, handling missing keys safely, and the common patterns for working with JSON-shaped data.
Walking it
for person in people:
for lang in person["langs"]:
The outer loop takes records, the inner takes the list inside each one. That is the shape of most processing you'll do. The flatten version of it's a nested comprehension:
[lang for person in people for lang in person["langs"]]
Same clause order as the loops. Worth using only when it fits on one line.
The missing-key problem
data["user"]["settings"]["theme"]
raises as soon as any level is absent. The KeyError names only the key that failed, not the path you were walking. Two ways to be safe:
data.get("user", {}).get("settings", {}).get("theme", "default")
Each get returns {} rather than None when absent. The next get still has a dict to call. That {} default is the trick that makes the chain work.
For any of it deeper than two or three levels, a small helper is clearer than a long chain. A try/except KeyError around the whole path is clearer still when a missing value really is exceptional.
Building nested shapes
Flat rows into groups is the most common change:
teams.setdefault(team, []).append(name)
and summing across two levels is a single gen formula:
sum(item["price"] for o in orders for item in o["items"])
The shape of data that arrives
Most real data is a list of records, and most records are dicts. API responses, parsed JSON, CSV rows read with DictReader, DB results - all the same shape.
Recognising this makes the code predictable. The outer loop takes records. A key lookup takes a field. An inner loop takes a list-valued field. Three moves cover almost all, and they compose to any depth.
The depth itself is the thing to keep an eye on. Two levels are easy to hold in your head. Four levels of mixed lists and dicts is where a small helper function - def city_of(record): - pays for itself right away, because it gives the path a name and one place to fix when the shape changes.
Modifying nested structures
Reading a nested value is safe. Writing one is where aliasing matters. If you copied the outer list shallowly and then edit a dict inside it, both copies change, because they hold the same dicts.
That isn't a reason to deepcopy all. It's a reason to decide on purpose: either the structure is shared and everyone knows it, or you take a deep copy at the boundary where it enters your code.
Flattening and grouping
The two changes that come up constantly are inverses of each other.
Flattening turns a nested structure into a flat one - all languages anyone knows, all items across all orders - and a nested comprehension or itertools.chain does it in a line.
Grouping turns a flat list into a nested one, keyed by some field. setdefault or defaultdict(list) is the idiom, and it's worth writing out once until it feels auto, because half of the reporting code anyone ever writes is a grouping followed by an aggregation.
JSON is the same shape
json.load makes just these structures - dicts, lists, strings, numbers, booleans and None - which is why working with parsed JSON needs no new skills beyond this page. json.dumps(data, indent=2) prints them back readably, and is the fastest way to understand a thing you've just fetched.
The one skew: JSON object keys are always strings. A dict keyed by ints lives the round trip as string keys. This Is a small surprise that shows up when a lookup that worked before saving fails after loading.
A worked example: from rows to a report
Most work with nested data is the same journey — records in, grouped summary out. Here it's end to end:
orders = [
{"customer": "ana", "items": [{"name": "pen", "price": 2},
{"name": "pad", "price": 5}]},
{"customer": "bo", "items": [{"name": "pen", "price": 2}]},
]
totals = {}
for order in orders:
total = sum(item["price"] for item in order["items"])
totals[order["customer"]] = totals.get(order["customer"], 0) + total
for customer, total in sorted(totals.items(), key=lambda kv: -kv[1]):
print(f"{customer:<6}{total:>4}")
ana 7
bo 2
Three moves, each from elsewhere in the track. The gen formula sums a list-valued field without building an mid list. totals.get(key, 0) builds up without a first-time special case. And sorted with a negated key puts the largest first.
What makes it readable is that each level is handled at one depth. The inner sum works on one order's items and knows nothing about customers. The outer loop works on orders and knows nothing about prices. When nested-data code turns hard to follow, it's almost always because one formula is reaching through three levels at once.
Naming the path
The single most handy habit with nested data is refusing to repeat a long path. record["user"]["profile"]["display_name"] written in four places is four places to update when the API changes. Four chances to typo a key into a KeyError that names only the last one.
A one-line function fixes it:
def display_name(record):
return record["user"]["profile"]["display_name"]
The path now exists once. It has a name that says what it means, so the call sites read as intent rather than as navigation. It has a place to put a default or a try. And when the shape changes. And it will — there's one line to edit.
This is worth doing at two levels of nesting, not four. The instinct to wait until it gets bad is why it often does.
Where the shape comes from, and what it costs
Nested data almost always arrives rather than being designed: it's the shape some API returns, or the shape JSON has, or the shape a DB join made. That has a consequence worth being deliberate about.
Data in the shape it arrived in is convenient for reading once and awkward for all else. Nothing validates it. So A missing key is found at the point of use rather than at the point of parsing. Nothing names it, so each function that touches it has to know the layout. And nothing stops two parts of the program disagreeing about whether a field is optional.
The option is to convert at the boundary: read the nested structure once, pull out what you need, and build objects. Dataclasses, named tuples, or just flatter dicts with the names you chose. All downstream then works with a shape you defined, validated once, at a known place.
For a script that reads a file and prints a summary, this is overkill and the raw structure is fine. For any of it long-lived, converting at the edge is the difference between a program where a shape change breaks one function and one where it breaks eleven.
Depth, and when to stop nesting
Reading nested data is unavoidable. *Building* deeply nested data is a choice, and often a poor one past two levels.
A dict of dicts of lists is hard to inspect, hard to iterate without three loops, and hopeless to query except by walking. The option is often a flat list of records where the nesting turns fields: instead of by_region[region][city] = [names], a list of {"region": ..., "city": ..., "name": ...} rows.
Flat records are longer to write and enormously easier to work with. They sort by any field, filter with one comprehension, group by whichever key the current question needs rather than the one you committed to when you built the structure, and convert right to CSV or a dataframe. The nesting you really need can be made on demand with a grouping. This Is three lines.
The heuristic: nest when the structure reflects genuine containment that will never be queried the other way round. Flatten when you can imagine wanting a different grouping later — which, for any of it resembling a report, you will.
Failing at the boundary rather than in the middle
The trait problem with nested data is that a shape mistake surfaces far from where it entered. A key that is missing from an API response is found three functions later, as a KeyError naming a key that looks correct, in code that has nothing to do with fetching.
The remedy is to check the shape once. Here The data arrives, rather than defending against it all over afterwards. What that check looks like depends on how much the data matters.
At the simplest, a few lines that pull out the fields you need and raise a clear error if they're absent. The error then says "response is missing user.profile.name" at the point of parsing. This Is a diagnosis rather than a symptom.
A step up, build the values into a dataclass or a NamedTuple. build fails right away if a field is missing, the resulting object has known fields rather than any keys, and each function downstream can be written against a shape that is ensured rather than hoped for.
Further still, a validation library — Pydantic being the common choice — declares the expected shape as types and reports each problem at once, with the path to each. That is worth it when the data comes from outside your control and the cost of processing a thing malformed is high.
All three are the same idea at different sizes: convert once, at the edge, and let all inside the boundary assume the data is what it claims to be. The option — a .get chain at each use site — spreads the uncertainty through the whole program and never really resolves it.
Questions people ask about Nested Data Structures
How do I get a value a few levels down safely? Chain .get with {} defaults, or wrap the direct path in try/except KeyError when lack is truly exceptional.
Why does .get("a", {}).get("b") work but .get("a").get("b") not? Because the second returns None when the key is missing, and None has no .get. The {} keeps a dict in the chain.
How do I see the shape of a thing I just parsed? print(json.dumps(data, indent=2)) for JSON-compatible data, and pprint.pprint for any of it else.
Can I flatten an arbitrarily deep structure? Yes, with recursion or a stack — but if the depth is unknown, that is often a sign the data wants to be flat records instead.
Are nested comprehensions read inside-out? No. The for clauses read left to right in the same order as the equal nested loops.
Why did my copy of a nested structure change? Because the copy was shallow and the inner structures are shared.
Do JSON keys stay ints? No. They come back as strings, which is a common surprise after a save-and-load round trip.
Should I use dotted access instead of brackets? Libraries that turn dicts into field access look convenient and hide typos, since a missing field and a missing key report differently. A dataclass gives you the same ergonomics with the checking.
What is the fastest way to search nested data? Build an index once — a dict keyed by what you search on — rather than walking the structure on each lookup.
Is there a standard way to walk a structure of unknown depth? Recursion, with a check on each value for whether it's a dict, a list, or a leaf. The standard library has no general walker. What to do at each node depends fully on the task.
Nested Data Structures: recap in one screen
- Most real data is a list of records. The moves are index, key lookup, and an inner loop, composed to what depth is needed.
- Chain
.get(key, {}) for optional paths, and try/except KeyError when a missing value is exceptional rather than expected. - Give a repeated path a named function at two levels of nesting, not four.
- Convert at the boundary into shapes you defined, unless the script is small enough that the raw structure will do.
- Prefer flat records over deep nesting for any of it you might want to group a different way later.