The escape hatch, what it costs, and the vectorised form that usually replaces it.
Overview
Three different things
pandas has several methods that apply a function to data, and their names are easy to confuse. map, apply and applymap (now called DataFrame.map) look similar in small examples but differ in what they operate on, how they handle missing values, and how fast they run.
The confusion matters because choosing the wrong one is either slow, incorrect, or both. Series.map with a dict is a fast lookup. Series.apply with a lambda is a slow Python loop disguised as a pandas call. DataFrame.apply with axis=1 iterates over every row in Python, which is the slowest operation you can write in pandas. Knowing which one you are using, and whether a vectorised alternative exists, is the difference between a script that finishes in a second and one that takes a minute.
The first question to ask is whether the operation has a vectorised form. Multiplying by two, comparing to a threshold, extracting the year from a date -- all of these have array-level implementations that run in compiled code. Using apply for any of them is correct but unnecessarily slow.
The second question is what the function operates on. map works on single values. apply on a Series works on single values. apply on a DataFrame works on whole columns or whole rows, depending on the axis parameter. The axis parameter is a common source of mistakes because the default is columns, not rows.
This module covers map for substitution, apply for per-element and per-row computation, and the vectorised replacements that make apply unnecessary in most cases. It also covers applymap and its renaming, for completeness.
The examples measure the speed difference between apply and the vectorised alternative. That makes the cost of the wrong choice concrete rather than theoretical. If you take one thing from this module, let it be the habit of always checking whether a vectorised form exists before reaching for apply.
Series.map looks each value up in a dict, a Series, or a function. It's for substitution.
Series.apply calls a function on each value. It's for math that has no array form.
DataFrame.apply calls a function on each column (axis=0) or each row (axis=1).
DataFrame.map — formerly applymap — calls a function on each single cell.
They look the same in small examples and aren't. The differences show up in both speed and action with missing values.
Worth knowing
map looks values up in a dict or Series; unmatched keys become NaN, so add a fallback if that is not what you want.
apply on a Series calls your function once per element — a Python loop, however it is spelled.
apply(axis=1) builds a Series per row and is the slowest common pattern in pandas.
Most row-wise applies have a vectorised equivalent: np.where, np.select, map, pd.cut, .str.
apply is legitimate for per-row logic with no array equivalent, for wrapping an existing scalar function, and on small frames.
The test is size: on 200 rows apply is fine; on 200,000 it is usually the bottleneck.
apply, map and Vectorising
The escape hatch, what it costs, and the form that often replaces it.
map replaces values from a dict
The simplest of the three, and it is a lookup rather than a loop you write.
example_01.pypandas
import pandas as pd
s = pd.Series(["a", "b", "c", "z"])
lookup = {"a": "apple", "b": "banana", "c": "cherry"}
print("map with a dict:")
print(list(s.map(lookup)))
print(" 'z' was not in the dict and became NaN")
print()
print("na_action='ignore' skips missing input instead of calling on it:")
print(list(pd.Series(["a", None]).map(lookup, na_action="ignore")))
print()
print("keep unmatched values with a fallback:")
print(list(s.map(lookup).fillna(s)))
Output
apply on a Series runs your function per value
Which is a Python loop, however it is spelled.
example_02.pypandas
import pandas as pd
s = pd.Series([1, 2, 3, 4])
print("apply:", list(s.apply(lambda v: v * 2 + 1)))
print("vectorised:", list(s * 2 + 1))
print("same:", list(s.apply(lambda v: v * 2 + 1)) == list(s * 2 + 1))
print()
print("apply is not doing anything clever - it calls the function")
print("once per element and collects the results.")
print()
print("it can return anything, which is its only real advantage:")
print(" ", list(s.apply(lambda v: "even" if v % 2 == 0 else "odd")))
Output
What it costs
The same result, an order of magnitude apart.
example_03.pypandas
import pandas as pd
import numpy as np
import time
n = 200_000
s = pd.Series(np.arange(n))
t = time.perf_counter(); a = s.apply(lambda v: v * 2 + 1); t_apply = time.perf_counter() - t
t = time.perf_counter(); b = s * 2 + 1; t_vec = time.perf_counter() - t
t = time.perf_counter(); c = pd.Series([v * 2 + 1 for v in s]); t_list = time.perf_counter() - t
print("apply : %.4f s" % t_apply)
print("list comprehension: %.4f s" % t_list)
print("vectorised : %.4f s" % t_vec)
print("apply is %.0fx slower than vectorised" % (t_apply / max(t_vec, 1e-9)))
print()
print("same values:", a.tolist() == b.tolist() == c.tolist())
print("dtypes : apply %s, vectorised %s" % (a.dtype, b.dtype))
print(" equals() would say False - it compares dtype too, and going")
print(" through Python ints can widen the result.")
Output
apply on a DataFrame gets whole rows or columns
axis=1 is the row-wise form, and it is the slowest thing here.
example_04.pypandas
import pandas as pd
df = pd.DataFrame({"a": [1, 2, 3], "b": [10, 20, 30]})
print("axis=0 (default): the function gets each COLUMN")
print(df.apply(lambda col: col.max() - col.min()))
print()
print("axis=1: the function gets each ROW as a Series")
print(list(df.apply(lambda row: row["a"] * row["b"], axis=1)))
print()
print("but that same result is a single vectorised expression:")
print(list(df["a"] * df["b"]))
print()
print("axis=1 builds a Series object per row. It is the form most")
print("worth removing when a script is slow.")
Output
Replacing the common apply patterns
Almost every row-wise apply has a vectorised equivalent.
import pandas as pd
df = pd.DataFrame({"text": ["a b c", "d e", "f"]})
# genuinely awkward to vectorise: a per-row parse returning several fields
def parse(row):
parts = row["text"].split()
return pd.Series({"first": parts[0], "n": len(parts)})
print(df.apply(parse, axis=1))
print()
print("Reasonable uses:")
print(" - calling an existing scalar function you cannot rewrite")
print(" - per-row logic with no array equivalent")
print(" - small frames, where clarity beats speed")
print()
print("The test: is the frame big enough for the difference to matter?")
print("On 200 rows, apply is fine. On 200,000 it is the bottleneck.")
Output
map
s.map({"a": "apple"}) replaces values by lookup.
Anything not in the mapping turns NaN. That is a fair default and often not what people want. A typo in a key silently blanks a column.
s.map(lookup).fillna(s) keeps the first where there was no match. This Is often the intended action for a partial rename.
na_action="skip" skips missing input rather than passing NaN to the function. This Matters when the function would fail on it.
map with a dict is fast. It's a lookup rather than a call. map with a function is just as slow as apply.
What apply costs
apply doesn't vectorise anything. It calls your function once per element and assembles the results.
The third editor measures it: on 200,000 rows, apply runs roughly an order of magnitude slower than the equal math. Is no faster than a plain list comprehension. Because that is what it's.
There's a persistent belief that apply is "the pandas way" and so fast. It's neither. It's the escape hatch for when there's no array formula.
axis=1 is the expensive one
df.apply(func, axis=1) makes a Series object for each row and passes it to your function. That is object creation per row on top of the Python call.
It's the single most common reason a pandas script is slow, and it's nearly always replaceable.
The tell is a lambda that indexes into the row: lambda row: row["a"] * row["b"]. That is df["a"] * df["b"]. This Operates on whole columns in compiled code.
The replacements
Nearly each row-wise apply is one of a handful of patterns:
A gated — np.where(cond, a, b). The vectorised if/else.
a few rules — np.select([c1, c2], [v1, v2], default=v3). checked in order, first match wins.
A lookup — map with a dict.
Numeric bands — pd.cut.
String work — the .str accessor.
Row-wise math — plain column math.
A group statistic per row — groupby(...).transform(...).
Between them these cover the large majority of real cases. When you find yourself writing apply(axis=1), it's worth thirty seconds to check this list first.
When apply is right
It truly earns its place in a few situations.
Wrapping an existing scalar function you can't or shouldn't rewrite. A parser, a checker, a call into another library.
Per-row logic with no array equal. A thing that branches on a few columns in a way np.select can't express cleanly, or that returns a var number of fields.
Returning a few columns at once, by returning a Series from the function, as the last editor shows.
Small frames, where the whole step takes microseconds either way and clarity is the only thing that matters.
That last point deserves emphasis. Optimising an apply over 200 rows is wasted effort. A readable apply can be better code than a clever vectorised formula no one can modify. The question is always whether the frame is large enough for the gap to matter.
If it must stay a loop
When the logic really is sequential and per-row. The frame is large, pandas isn't the tool. The options are to move the loop into NumPy, compile it with Numba, or restructure the problem.
What is never right is iterrows. It's slower than apply(axis=1), it loses dtypes by converting each row to an object Series. There's no situation where it's the best on hand option. itertuples is a lot faster if you truly need to iterate, and preserves types.
DataFrame.map and the applymap rename
DataFrame.map(func) applies a function to each cell. It was called applymap until pandas 2.1. The old name is deprecated.
It's rarely the right tool. Applying a function to each cell of a table often means either the step belongs on exact columns, or the frame should have been reshaped first.
The legitimate uses are formatting for display and elementwise type coercion across a homogeneous frame. Both of which are end-of-pipeline steps.
Returning several values
A function that returns a Series gives a few columns:
That works and is slow twice over. A Python call and a Series build per row.
The faster shape is often to find each output column apart with vectorised expressions, even if that means traversing the input more than once. Three passes over a column in compiled code beat one pass in Python by a wide margin.
When the math truly makes a few values at once and can't be split, zip(*df.apply(...)) or building lists and assigning at the end avoids the Series build.
result_type and the empty-frame trap
df.apply(func, axis=1) on an empty frame doesn't call the function at all. Pandas can't infer the result shape. The result is often an empty DataFrame where the code expected an empty Series. The next step fails with a confusing error.
Code that runs apply on a frame that may be empty should handle that case by name. The failure appears only when the input happens to be empty. Which is just the edge case least likely to be tested.
result_type= controls how a list-returning function is read: "expand" makes columns, "reduce" keeps a Series.
Choosing the right escape hatch
When a thing truly can't be vectorised, apply isn't the only option. Often not the best one.
**np.vectorize** — despite the name, a loop. It handles broadcasting and dtypes for you, and offers no speed gain.
**A list comprehension over zip of the columns**. Often *faster* than apply(axis=1). It skips the per-row Series build. Less idiomatic, measurably quicker.
**itertuples** — fast loop with dtypes preserved, when you need the whole row.
Numba — compiles a numeric loop to machine code. The right answer when the logic is truly sequential and the frame is large.
Restructuring — often the real answer. A loop over rows to look a thing up is a merge. A loop to find a group statistic is a transform. A loop with a rule is np.select.
A worked replacement
A row-wise function with branching:
def band(row):
if row["score"] >= 80:
return "A"
elif row["score"] >= 50:
return "B" if row["city"] == "pune" else "C"
return "F"
df["band"] = df.apply(band, axis=1)
The rules are checked in order and the first match wins. This Is just the semantics of the if/elif chain. It's longer to read the first time and orders of magnitude faster. The rules can be built programmatically in a way the function can't.
The honest summary
apply isn't forbidden. It's a Python loop with a pandas-shaped interface. It should be used when that is what you want. Awkward per-row logic, an existing function you can't rewrite, or a frame small enough that the gap is unmeasurable.
The mistake is reaching for it reflexively for math, conditionals and lookups, all of which have vectorised forms that are shorter and faster.
Mapping with a Series
map accepts a Series and a dict. This Makes a lookup table out of another frame:
This is often better than a merge for a simple one-column lookup. It can't times rows, it needs no how or validate. Unmatched codes become NaN rather than dropping the row.
The need is that the lookup's index is unique. If it isn't, map raises. Which is a better failure than a merge silently duplicating rows.
For a lookup that must bring a few columns, a merge is the right tool.
Missing values in apply and map
map passes NaN to the function unless na_action="skip". A function that calls .lower() or does math will fail on it. The traceback points at your function rather than at the data.
apply does the same.
Two options: guard inside the function, or filter first and assign back through .loc. The second is often cleaner. It keeps the function simple:
ok = df["text"].notna()
df.loc[ok, "parsed"] = df.loc[ok, "text"].apply(parse)
Measuring before optimising
apply is slow scaled to vectorised steps and fast scaled to nothing at all. Whether it matters depends fully on size.
On a thousand rows, an apply takes about a millisecond. Replacing it with np.select saves a millisecond and costs readability if the logic is truly branchy.
On a million rows, the same apply takes seconds and dominates the script.
The choice rule is to measure the actual frame, not to apply a blanket rule. %timeit on the real data answers it in seconds. Often the answer is that this given apply is irrelevant and a other line is the problem.
A summary
map for lookups — dict or Series — and note unmatched keys become NaN.
apply on a Series is a loop. On a frame with axis=1 it's a slower loop.
DataFrame.map (formerly applymap) touches each cell and is rarely the right tool.
np.where, np.select, pd.cut, .str and transform replace most row-wise applies.
A list comprehension over zip of columns often beats apply(axis=1).
iterrows is never the right choice. itertuples if you must iterate.
And the size of the frame decides whether any of this matters.
A closing note on apply, map and Vectorising
apply occupies a strange place: it's the most reached-for method in pandas and rarely the right one.
it looks like the pandas way of doing a thing per row. Pandas has a reputation for speed. It feels like it should be fast. It's a Python loop with a method-call interface, and it's no faster than the list comprehension it replaces.
Nearly each row-wise apply is one of a handful of patterns with a vectorised form. A gated is np.where or np.select, a lookup is map, a numeric band is pd.cut, string work is .str. A group statistic is transform.
That said, the honest spot isn't that apply is forbidden. On a small frame the gap is unmeasurable. A readable apply beats a clever formula no one can modify. The mistake is reaching for it by default rather than deciding. Never measuring whether it matters.
One more thing
Series.map accepts a function with a default through sets.defaultdict. This Is the neat way to map known values and leave all else at a fallback without a split fillna.
And apply on a groupby is where include_groups=False now matters. In pandas 2.2 the grouping columns are passed to the function by default and that action is being changed. Passing the arg by name is how you write code that behaves the same before and after the change.
Questions people ask about apply, map and Vectorising
What is the difference between map, apply and applymap? map works on a Series element by element, apply works on a Series or along the rows or columns of a DataFrame, and DataFrame.map (formerly applymap) works on every cell.
Why is apply slow? It calls a Python function once per row or element, so pandas cannot use its compiled loops. Vectorised operations on whole columns are typically tens to hundreds of times faster.
How do I replace values using a dictionary? Use df['col'].map({'a': 1, 'b': 2}). Values not in the dictionary become NaN, so use replace if you want to leave them unchanged.
What can I use instead of apply with axis=1? Use vectorised arithmetic, np.where or np.select for conditions, and str or dt accessors for text and dates.
When is apply the right tool? When the logic cannot be expressed on whole columns, such as parsing a complicated string or calling a function that works on one object at a time.
Why does apply return a different shape than I expected? The shape depends on what your function returns: a scalar gives a Series, a Series gives a DataFrame. Use result_type to control it for row-wise applies.
apply, map and Vectorising, in summary
map substitutes, apply finds, and both call Python once per element.
Nearly each row-wise apply has a vectorised equal. np.where, np.select, map, pd.cut, .str, transform. That is shorter and faster.
apply remains the right answer for truly awkward per-row logic, for wrapping a function you can't rewrite. On frames small enough that the gap is unmeasurable. The mistake is reaching for it without deciding. Never measuring whether it matters.
Check yourself
0 of 4
Answer without scrolling back up.
What happens to a value not present in the dict passed to `map`?
A typo in a key silently blanks a column. Use .map(lookup).fillna(s) to keep the original where there was no match.
Why is `df.apply(func, axis=1)` the slowest common pattern?
The tell is a lambda indexing into the row - that is column arithmetic, which runs in compiled code.
What is the vectorised replacement for a two-branch conditional apply?
For several conditions, np.select evaluates in order with first match winning. For numeric bands, pd.cut.
When is `apply` a reasonable choice?
Optimising an apply over 200 rows is wasted effort. The question is whether the frame is large enough for the difference to matter.
Cheat sheet
apply, map and Vectorising
Use apply, map and applymap on a pandas DataFrame. Measure the speed cost and see the vectorised alternatives that usually replace them.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.