vizlearn

Method Chaining

pandasBy Updated

Writing a pipeline as one expression - what it buys, and where to stop.

Overview

What it buys

Method chaining is a style where each step in a data pipeline is a method call on the result of the previous step. Instead of assigning intermediate variables, you write one expression that flows from input to output. The result reads top to bottom, in the order the operations happen.

The practical benefit is readability. A chained pipeline shows the sequence of transformations in one place. You can see what goes in, what happens at each step, and what comes out. Intermediate variables break that flow. They force you to invent names, and each name is one more thing to track.

The second benefit is safety. Each step in a chain operates on the result of the previous step. There is no risk of accidentally using a stale variable from an earlier version of the pipeline. In a script with many intermediate frames, reordering steps can introduce a bug where one step reads a variable that was overwritten by another. Chaining avoids that because there are no variables to overwrite.

The cost is debugging. A chain that fails gives you one traceback for the whole expression. Breaking the chain into named steps lets you inspect each intermediate result. The practical compromise is to develop in steps, then chain once the logic is settled.

pandas supports chaining because most methods return a new object. query, assign, sort_values, groupby, rename, reset_index -- all return a DataFrame or Series that the next method can act on. The assign method exists specifically for chaining. It adds or replaces columns and returns the modified frame.

pipe extends chaining to functions that are not DataFrame methods. Any function that takes a DataFrame as its first argument and returns one can be called inside a chain via pipe. That includes your own functions, which means a chain can mix built-in pandas methods with custom logic.

This module covers the basics of chaining, assign, pipe, and the patterns that make chains readable. It also covers the cases where chaining is the wrong choice: when the pipeline needs conditional branches or when a step is easier to read as a named variable.

Most pandas methods return a new object. This Means they compose:

out = (df
       .query("year == 2024")
       .assign(doubled=lambda d: d["sales"] * 2)
       .groupby("city", as_index=False)["doubled"].sum())

Three things follow, and they're the actual arg for the style.

No mid names. The tmp, tmp2, df_clean, df_clean2 sequence is where stale-var bugs live. Above all in a notebook where cells run out of order. df is no longer what the cell above assumed.

Nothing is mutated. The first frame is untouched. Re-running a cell gives the same answer. That alone drops a large class of notebook confusion.

Each line is one step. The chain reads top to bottom in the order the work happens. This Nested function calls don't.

The wrapping parentheses are what allow the line breaks. Without them the formula has to fit on one line or carry backslashes.

Worth knowing

Chaining removes intermediate names, and with them the chance of using a stale one.
assign takes a lambda so it can see the frame at that point in the chain, including columns earlier steps created.
pipe(f, args) is just f(df, args), written so a custom step reads in order rather than inside out.
A pipe that prints and returns unchanged lets you inspect a chain without taking it apart.
Long chains hide which link failed — four steps is comfortable, fifteen is a debugging problem.
The pattern suits load-clean-aggregate work, where each line is one idea and nothing is mutated.

Method Chaining

Writing a pipeline as one formula.

The same work, two ways

Intermediate names against one expression.

example_01.pypandas
Output

                

                

                    
lambda is what makes assign chain

                    
It sees the frame at that point, not the one you started with.
example_02.pypandas
Output

                

                

                    
pipe puts your own function in the chain

                    
For steps that are not a pandas method.
example_03.pypandas
Output

                

                

                    
Debugging a chain

                    
The usual objection, and the usual answer.
example_04.pypandas
Output

                

                

                    
Where chaining stops helping

                    
Long chains hide where things went wrong.
example_05.pypandas
Output

                

                

                    
A realistic pipeline

                    
Load, clean, aggregate - as one readable expression.
example_06.pypandas
Output

                

                

                    
assign and the lambda

                    
assign is what makes math chainable, and the lambda is what makes assign work mid-chain.
d.assign(b=d["a"] * 10) finds against the frame as it was before the chain started. Inside a chain that is often wrong. If the column it needs was created two steps earlier it raises KeyError.
d.assign(b=lambda d: d["a"] * 10) receives the frame at that point. It sees all earlier steps made.
Use the lambda form by default in a chain. It costs eight characters and drops the class of error fully.

                

                

                    
pipe

                    
Not each step is a pandas method. pipe inserts your own function without breaking the chain:
df.pipe(drop_small, threshold=10).pipe(add_share)
df.pipe(f, x) is just f(df, x). The gain is order. The nested form, add_share(drop_small(df, 10)), reads inside out and gets worse with each step.
For this to work, your functions should take a frame as the first arg and return a frame. That is a good shape for them anyway. It makes them testable in alone.

                

                

                    
Debugging

                    
The standard objection to chaining is that you can't see what is happening in the middle. Can't set a breakpoint on a step.
The answer is a pipe that inspects and returns unchanged:
def show(d, label):
    print(label, d.shape, list(d.columns))
    return d
Dropped into the chain, it prints the shape at each stage without changing the result. Most chain bugs are a shape or a column name, and that catches both.
For a heavier version, log d.head() or write the mid to a file. The principle is the same. A function that returns its input can go anywhere in a chain.

                

                

                    
Where to stop

                    
Chaining is a style, not a virtue, and it stops paying at some length.
The traceback problem. When a fifteen-step chain raises KeyError: 'typo', the traceback points at the whole formula. You know it failed. You don't at once know where. The last-but-one editor shows this.
The reading problem. A chain is one formula. It has to be understood as a whole. Past a certain length that is harder than reading five named steps.
The reuse problem. An mid result needed twice has to be found twice, or the chain has to be broken anyway.
A workable rule: chain a coherent stage. Loading and cleaning, or aggregating and formatting. And give each stage a name.
clean = (raw.pipe(normalise_columns).dropna(subset=["id"]))
summary = (clean.groupby("city").agg(...).sort_values("total"))
Two names instead of ten, and each chain is short enough to debug.

                

                

                    
Where it fits

                    
The style suits load, clean, aggregate work above all well. That work is on its own a sequence of changes with no branching.
It suits exploratory notebook work, because immutability makes re-running cells safe.
It suits less well anything with branching logic, loops, or steps whose output feeds two other places. At which point named intermediates are simply clearer. There's nothing wrong with using them.

                

                

                    
Which methods chain

                    
Anything returning a DataFrame or Series can be chained. This Is most of the API:
Pick — query, loc with a callable, filter, head, sample, nlargest.
change — assign, rename, drop, astype, fillna, replace, round.
Reshaping — sort_values, set_index, reset_index, melt, pivot, stack, unstack, explode.
Aggregation — groupby(...).agg(...), value_counts, show.
Anything else — pipe.
The methods that break a chain are the ones returning None. Anything with inplace=True. sort / shuffle-style in-place methods. That is one more reason to avoid inplace.

                

                

                    
Naming the stages

                    
The hands-on shape for a real script is a few named stages rather than one long chain:
raw = pd.read_csv(path, dtype=..., parse_dates=[...])

clean = (raw
         .pipe(normalise_columns)
         .dropna(subset=["id"])
         .astype({"id": "string"}))

summary = (clean
           .query("year == 2024")
           .groupby("city", as_index=False)
           .agg(total=("sales", "sum")))
Each stage is short enough to debug, each name says what the data is at that point. The intermediates are on hand for inspection without breaking anything apart.
That structure also tests well: normalise_columns and each stage can be checked on its own.

                

                

                    
Chaining and memory

                    
Each step in a chain allocates. A ten-step chain on a large frame allocates ten frames, though earlier ones are freed as it proceeds. Peak memory is roughly two at a time rather than ten.
That is acceptable for most work and worth knowing when the frame is large enough that a single copy is large.
Filtering early in the chain reduces each allocation after it. This Is the same advice as all over else and matters more here because there are more of them.

                

                

                    
Common chaining mistakes

                    
**Losing the lambda in assign.** The formula is then checked against the pre-chain frame. Raises if it needs a column made mid-chain.
**Using inplace=True in a chain.** Returns None, and the next method raises AttributeError on NoneType.
given the index lives. reset_index(drop=True) at the point it matters, not at the end.
A chain that is really two steps. If an mid is needed twice, the chain has to find it twice or be broken. Break it.
Debugging by deleting lines. Better to insert a pipe that prints. This Doesn't change the structure.

                

                

                    
When not to chain

                    
Chaining suits linear change. It doesn't suit:
Branching — other handling based on a rule.
Loops — over files, groups or params.
Reuse — an mid needed by two downstream steps.
Error handling — a try/except around one step.
In all four cases, named intermediates are clearer. Reaching for a chain anyway makes code that is harder to read than the thing it replaced.
The style is a tool for the common case of "load, clean, aggregate, output". Here It truly reads better than the option. It isn't a standard to hold all code to.

                

                

                    
A realistic shape for a script

                    
Chaining works best as a few named stages rather than one long formula or a hundred split statements:
def load(path):
    return pd.read_csv(path, dtype=DTYPES, parse_dates=["date"])

def clean(d):
    return (d
            .rename(columns=str.lower)
            .dropna(subset=["id"])
            .assign(city=lambda x: x["city"].str.strip().str.lower())
            .drop_duplicates(subset=["id"], keep="last"))

def summarise(d):
    return (d
            .query("date >= @CUTOFF")
            .groupby("city", as_index=False)
            .agg(total=("sales", "sum"), orders=("sales", "size"))
            .sort_values("total", ascending=False))

result = summarise(clean(load(path)))
Each function is testable on its own, each chain is short enough to debug. The top-level line reads as what the script does.
load(path).pipe(clean).pipe(summarise) expresses the same thing in chain form. This Reads in order rather than inside out.

                

                

                    
Chaining and notebooks

                    
The style suits notebooks above all well, for a reason worth stating: cells get re-run, out of order, again and again.
A chain doesn't mutate its input. Re-running a cell gives the same result. A sequence of in-place changes doesn't. Running it twice applies the change twice. The frame is now wrong in a way nothing shows.
That failure is common enough that "restart and run all" is standard advice. Chaining drops most of the need for it.

                

                

                    
Readability in practice

                    
A few rules make chains easier to read:
One step per line.
Wrapping parentheses, so no backslashes are needed.
A blank line between logical stages within a long chain.
pipe for anything that isn't a pandas method, rather than breaking out.
Names for intermediates at genuine stage boundaries.
And a limit: if you can't see the whole chain on one screen, split it. The point of the style is clarity. A chain that has to be scrolled has stopped giving it.

                

                

                    
A summary

                    
Chaining drops mid names and the stale-var bugs that come with them.
Nothing is mutated, so re-running is safe.
assign needs a lambda to see the frame mid-chain.
pipe inserts your own functions in reading order.
A pipe that prints and returns its input lets you debug without breaking the chain.
Long chains hide which link failed — name coherent stages instead.
And it's a style for linear change, not a rule for all code. Branching, loops and reuse are clearer with names.

                

                

                    
A closing note on Method Chaining

                    
Method chaining is one of the few stylistic choices in pandas that changes how many bugs you write rather than merely how the code looks.
The method isn't elegance. It's that mid names are where stale state lives: df_clean that was cleaned by an earlier version of the cell above, tmp2 that is a filtered copy of tmp from before the filter changed. A chain has nowhere for that to hide.
The immutability matters for the same reason. Code that doesn't modify its input makes the same answer on the second run as the first. This Is the property notebooks most often lack.
Against that, a chain is one formula. One formula fails as a unit. The balance is struck by keeping chains to a coherent stage and giving each stage a name. Which is plain good structure, applied to data change.

                

                

                    
Two more things worth knowing

                    
pipe has a second form for functions whose frame arg isn't first: df.pipe((func, "data"), other_arg) tells pandas which param receives the frame. It's rarely needed. It exists so that third-party functions with awkward signatures can still be chained.
assign accepts plain values and lambdas. Mixing the two in one call is legal. The rule is that a plain value is checked once, against the frame as it was before the chain started. A plain value referring to a column made mid-chain will fail. Using lambdas uniformly avoids having to hold that gap in mind.
Finally, chains and comments coexist badly. There's no natural place to explain a step. That is a real arg for named stages. A function name is a comment that can't drift out of date.

                

                

                    
Chaining and testing

                    
One hands-on gain of the named-stage structure is that it makes a pipeline testable without any test framework ceremony.
Each stage takes a frame and returns a frame. Each can be checked on a small hand-built input:
def test_clean():
    raw = pd.DataFrame({"ID": [1, 1, None], "City": [" Pune ", "pune", "goa"]})
    out = clean(raw)
    assert len(out) == 1
    assert list(out["city"]) == ["pune"]
That is a genuine unit test of a data change. It's possible only because clean is a function rather than a sequence of statements operating on a global df.
The same structure makes the pipeline reusable across scripts and notebooks. Makes it obvious where a new step belongs. Chaining, in this reading, is less about elegance than about pushing data changes into functions that can be named, tested and reused. Which is plain software practice arriving one spot it's often skipped.

                

                

                    
Questions people ask about Method Chaining

                    
What is method chaining? It is writing a series of operations as one expression, with each method returning a new frame for the next to use. Wrapped in parentheses, it reads like a recipe from top to bottom.
What does pipe do? pipe passes the frame into your own function inside a chain: df.pipe(clean, col='x'). It lets custom steps fit in the chain.
How do I use a column I just created in the same chain? Use assign with a lambda: .assign(total=lambda d: d.a + d.b). The lambda receives the current frame at that step.
How do I debug a long chain? Insert a pipe step that prints or checks the shape, such as .pipe(lambda d: print(d.shape) or d). You can also run the chain in parts and inspect each result.
Does chaining use more memory? Each step creates a new object, but earlier ones are freed once nothing refers to them. Peak memory is usually about one extra copy.
When should I not chain? When a step needs an explanation, when you need to inspect intermediate results often, or when the chain runs to dozens of lines. Split it into named steps.

                

                

                    
Method Chaining, in summary

                    
Chaining is a style that suits linear change. This Is most of what data cleaning is.
It drops mid names and the stale-state bugs that live in them, it doesn't mutate its input so re-running is safe. It reads in the order the work happens.
Its costs are real: a failure points at the whole formula, an mid needed twice forces a break. Branching or looping doesn't fit at all.
The detail is named stages of a handful of steps each. This Keeps the gains and makes each stage short enough to debug and small enough to test.

                

            


        

    


    



Check yourself
0 of 4
Answer without scrolling back up.
  1. Why does `assign` take a lambda inside a chain?
  2. What is `df.pipe(f, x)` equivalent to?
  3. How do you inspect the middle of a chain without breaking it apart?
  4. What is the main practical cost of a very long chain?





 Cheat sheet
Method Chaining
Write a pandas analysis as one chained expression. Compare chained and step-by-step styles and see when pipe and assign keep the chain clean.
PANDAS · vizlearn.in/pandas/method_chaining.html
More in pandas
pandasWhat pandas Is ForpandasThe IndexpandasCreating DataFramespandasLooking at Data
About the author
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.
About VizLearnGitHubLinkedInContact