Run it: Group records and invert a dictionary in code
The three grouping idioms agreeing, groupby getting it wrong on unsorted input, and both inversion failures caught in the act.
Inverting has its own two traps: duplicate values collapse silently, and values that cannot be hashed cannot become keys.
The defaultdict trap
A defaultdict creates entries on read. Merely looking a key up inserts it:
That surprises people debugging a size or an iteration. Two ways to avoid it: use d.get(key, []) for reads, or convert to a plain dict once building is finished with dict(d).
The related habit worth having: never pass a defaultdict out of a function as a return value if callers will read arbitrary keys. Convert it first, and the mutation-on-read behaviour stays inside the code that wanted it.
Inverting a dictionary
Duplicate values collide, and the last one wins. alice and carol are both 30, so only carol survives. This is the single most common bug with inversion, and the fix is to invert into lists:
Which is the same operation as grouping — grouping the keys by their value. The two topics are one topic.
The other requirement: values must be hashable to become keys. Inverting {"a": [1, 2]} raises TypeError, and converting the list to a tuple first fixes it.
Three ways to group
setdefault(key, []).append(x) — no import, and it makes the default explicit. It does construct an empty list on every call, which is why the next form is usually preferred.
defaultdict(list) — the idiomatic answer. Remember that reading a missing key creates it, so it is not safe to inspect casually.
itertools.groupby — the one that catches people. It groups consecutive equal keys only, so it needs the input sorted by the same key first. It is not the SQL GROUP BY its name suggests.
Two ways inversion breaks
Duplicate values. {v: k for k, v in d.items()} keeps only the last key for each repeated value, silently. If duplicates are possible, invert to lists instead — which is just grouping again, by value.
Unhashable values. A value that was fine as a value cannot necessarily be a key: a dict mapping names to lists of scores cannot be inverted directly, because a list is unhashable.
Where this shows up
Grouping is the shape behind most "organise this data" questions. Anagram groups keyed on sorted letters. Files keyed on their hash. Log lines keyed on their level. Spot the shape and you skip the nested loop that scans for an existing group.
The one-pass defaultdict build is O(n). A nested loop that re-scans the groups already built is O(n²). On a hundred thousand records that is ten thousand operations versus ten billion. The gap is why this idiom is worth reaching for on sight.
The standard library has a groupby, and it does not do what the name suggests to anyone coming from SQL or pandas.
itertools.groupby groups consecutive runs only. It does not collect scattered items. To get SQL-like behaviour the input must be sorted by the same key first:
Which costs O(n log n), while the defaultdict version is O(n). Prefer defaultdict unless the data is already sorted or the runs themselves are what you want. Compressing repeated values, detecting consecutive duplicates, run-length encoding.
The other sharp edge: the group is an iterator sharing the underlying stream. It is exhausted as soon as you advance to the next group, so list(group) must happen before moving on. groups = [(k, g) for k, g in groupby(x)] yields empty groups for exactly this reason.
The grouping patterns worth having ready
Group by a computed key.
Group by several keys — use a tuple:
Count instead of collect — Counter is the specialised form:
Aggregate while grouping — defaultdict(int) for sums, defaultdict(set) for deduplicated members:
Group anagrams, the classic interview form — the key is a canonical form:
groups = defaultdict(list)
for w in words:
groups["".join(sorted(w))].append(w)
return list(groups.values())
| Need | Tool |
|---|
| Collect items | defaultdict(list) |
| Count items | Counter |
| Sum values | defaultdict(int) |
| Deduplicated members | defaultdict(set) |
| Nested grouping | defaultdict(lambda: defaultdict(list)) |
| Consecutive runs only | itertools.groupby |
The nested case needs a lambda because defaultdict(defaultdict(list)) is wrong — the argument must be a callable that produces the default, not an instance.
For the underlying ideas, see dictionaries, dictionary methods and dict comprehensions.