pandas, by Running It

About this track

A DataFrame is not a spreadsheet and not a two-dimensional array. It is a set of columns, each with its own dtype, sharing one index - and almost everything that surprises people about pandas comes from that index doing work they did not ask it to do.

This track works up from the Series and the index to selection, missing data, group-by, joins, reshaping and time series. Every idea arrives as a small program you can run and change on the page.

Every example runs in the browser against pandas itself, and several demonstrate a failure rather than describing it - the assignment that silently does nothing, the merge that multiplies rows, the group-by whose totals no longer add up.

What you will be able to do

• Explain what the index is doing in an operation you did not ask it to take part in.
• Choose between loc and iloc without guessing, and know why their slices differ.
• Recognise chained assignment on sight, and write the version that works.
• Reshape a frame into the form a group-by, a join or a chart actually wants.
• Spot the silent failures - a dropped NaN key, a duplicated merge key, an integer column turned float.

How the track is ordered

The Series and the index come first, because nearly every pandas surprise traces back to the index doing work you did not ask for. Selection follows, and then the copy warning, which gets a module to itself because nothing else in the library wastes as much of people's time. Cleaning comes next - dtypes, missing values, duplicates, text and dates - since that is where most real work goes. Only then the aggregation everyone thinks of as the point: group-by, joins, reshaping, time series, and the performance rules that decide whether any of it finishes.

Where this leads

pandas sits on NumPy and feeds matplotlib, scikit-learn and most data pipelines. The NumPy track explains the array layer underneath; the matplotlib track picks up where df.plot stops being enough.