assign, drop, rename - and the alignment that decides what a new column actually contains.
Overview
Bracket assignment
Adding and removing columns is the most common mutation you perform on a DataFrame. New columns hold computed results, flags, cleaned versions of existing data, or values from a lookup table. Dropped columns remove noise from downstream operations.
The simplest form is bracket assignment. It writes a new column if the name does not exist and overwrites an existing one if it does. There is no confirmation, no warning, and no undo. That silent overwrite is a real hazard in longer scripts where you cannot see the full column list.
The assigned value can be a Series, a scalar, or an array. A scalar broadcasts to every row. A Series aligns on the index, which means mismatched indexes produce NaN in the gaps rather than raising an error. An array must match the frame length exactly or pandas raises a ValueError.
Understanding alignment during assignment prevents a common surprise. If you compute a value from a filtered subset and assign it back, the result has NaN wherever the filter excluded a row. That is correct behavior, but it catches people who expected the values to fill in order rather than by label.
This module covers bracket assignment, the assign method for chaining, insert for positional placement, drop for removal, and rename for changing column names. Each returns a new object by default, which matters for method chaining and for avoiding accidental mutation of a shared frame.
The examples show each operation on small frames so you can see the before and after. Pay attention to how alignment works during assignment, because that is where most surprises come from.
df["total"] = df["qty"] * df["price"] adds a column at the end.
Two actions are worth stating because neither warns.
A scalar broadcasts: df["currency"] = "INR" fills each row.
An existing name is replaced, silently. There's no protection against overwriting a column by reusing its name. This Is a real hazard in a long script where the column list isn't in front of you.
Column order is insertion order. insert is the only way to place one one spot exact.
Worth knowing
df["new"] = ... adds a column at the end, or silently replaces one of the same name.
assign returns a new frame and takes lambdas, so it chains — and each lambda sees the frame as it is at that point.
Assigning a Series aligns on the index; assigning a list goes in by position.
Assigning a subset-derived Series to a whole column overwrites every unmatched row with NaN, even one that held a value. df.loc[mask, "col"] = ... touches only the selected rows.
drop(columns=[...]) and drop(index=[...]) return a new frame; a missing name raises unless errors="ignore".
rename ignores keys that do not exist, so a typo does nothing. insert is the only way to choose a column's position.
Adding and Removing Columns
assign, drop, rename, and the alignment that decides what a new column holds.
Assigning a new column
Bracket assignment adds it at the end, or replaces it if the name exists.
example_01.pypandas
import pandas as pd
df = pd.DataFrame({"qty": [3, 1, 2], "price": [10.0, 250.0, 400.0]})
df["total"] = df["qty"] * df["price"]
print(df)
print()
df["currency"] = "INR" # a scalar fills every row
print("a scalar broadcasts:", list(df["currency"]))
print()
print("assigning an existing name replaces it, silently:")
df["currency"] = "USD"
print(" ", list(df["currency"]))
print()
print("column order is insertion order:", list(df.columns))
Output
assign returns a new frame
Which is what makes it chainable, and what makes it safe.
example_02.pypandas
import pandas as pd
df = pd.DataFrame({"qty": [3, 1, 2], "price": [10.0, 250.0, 400.0]})
out = (df
.assign(total=lambda d: d["qty"] * d["price"])
.assign(cheap=lambda d: d["total"] < 100))
print(out)
print()
print("the original is untouched:", list(df.columns))
print()
print("the lambda sees the frame AS IT IS AT THAT POINT, so the second")
print("assign can use the column the first one created.")
Output
Alignment decides what lands in the column
Assigning a Series matches on the index, not on position.
example_03.pypandas
import pandas as pd
df = pd.DataFrame({"a": [1, 2, 3]}, index=["x", "y", "z"])
s = pd.Series([100, 200, 300], index=["z", "y", "x"])
df["aligned"] = s
print("assigning a Series with a shuffled index:")
print(df)
print(" x got 300, because alignment is by LABEL")
print()
partial = pd.Series([9, 9], index=["x", "q"])
df["partial"] = partial
print()
print("labels that do not match become NaN:")
print(df[["a", "partial"]])
print()
print("a plain list has no index, so it goes in by POSITION:")
df["by_pos"] = [7, 8, 9]
print(list(df["by_pos"]))
Output
The filtered-assignment trap
Computing from a subset and assigning back wipes every row that was not in it.
example_04.pypandas
import pandas as pd
def fresh():
d = pd.DataFrame({"city": ["pune", "delhi", "pune"], "sales": [10, 20, 30]})
d["doubled"] = 0 # a column that already has values
return d
a = fresh()
sub = a[a["city"] == "pune"]
a["doubled"] = sub["sales"] * 2
print("whole-column assign from a subset:")
print(a)
print(" delhi was 0 and is now NaN - the assignment replaced the")
print(" ENTIRE column, and alignment had nothing for that row.")
b = fresh()
b.loc[b["city"] == "pune", "doubled"] = b["sales"] * 2
print()
print("loc assignment touches only the selected rows:")
print(b)
print(" delhi kept its 0.")
Output
The most common real instance:
sub = df[df["city"] == "pune"]
df["doubled"] = sub["sales"] * 2
sub has only the Pune rows. The found Series has only those labels. Alignment fills every other row with NaN.
pandas did just what it promises. The author expected "find for these rows and leave the others alone". This Is a other step:
The gap is visible when the column already exists. Whole-column set replaces the entire column. Rows absent from the subset are overwritten with NaN. Even if they held a just right good value a moment ago. The .loc form touches only the selected rows and leaves the rest alone.
When the column does *not* already exist, both forms leave NaN in the unselected rows. Creating a column has to put a thing in each row. If you want a default there, create the column first and then overwrite the subset. Which is just what the editor above does.
Dropping columns and rows
The same method, steered by axis - and it returns a new frame by default.
example_05.pypandas
import pandas as pd
df = pd.DataFrame({"a": [1, 2], "b": [3, 4], "c": [5, 6]})
print("drop a column:", list(df.drop(columns=["b"]).columns))
print("drop a row :", list(df.drop(index=[0]).index))
print("original :", list(df.columns), "<- unchanged")
print()
print("axis= works too, but columns=/index= say what you mean:")
print(" ", list(df.drop("b", axis=1).columns))
print()
print("a missing name raises, unless you say otherwise:")
try:
df.drop(columns=["zzz"])
except KeyError as e:
print(" KeyError:", str(e)[:40])
print(" errors='ignore':", list(df.drop(columns=["zzz"], errors="ignore").columns))
Output
Renaming, and inserting in a position
rename takes a mapping; insert is the only way to choose where a column goes.
example_06.pypandas
import pandas as pd
df = pd.DataFrame({"a": [1, 2], "b": [3, 4]})
print("rename by mapping:", list(df.rename(columns={"a": "alpha"}).columns))
print("rename by function:", list(df.rename(columns=str.upper).columns))
print()
print("unknown keys are ignored silently - a typo just does nothing:")
print(" ", list(df.rename(columns={"zzz": "nope"}).columns))
print()
df.insert(1, "middle", [9, 9])
print("insert(1, ...) puts it at position 1:")
print(df)
print()
print("reorder by selecting in the order you want:")
print(list(df[["b", "middle", "a"]].columns))
Output
assign
df.assign(total=...) returns a new frame rather than modifying in place.
That makes it chainable, which is its main purpose:
The lambda receives the frame as it's at that point in the chain. The second assign can use the column the first one created. Passing a value right rather than a lambda works too. But Then it's found against the first frame. This Breaks in a chain.
assign also keeps the first untouched. This Is worth a thing in a notebook where cells get re-run out of order.
The cost is a copy per call. In a hot loop that matters. In plain analysis code it doesn't. The readability of a chain often wins.
Alignment decides the contents
This is the part that surprises people, and it's the index rule again.
Assigning a Series aligns on the index. If the Series has the same labels in a other order, pandas reorders it to match. Which is correct and isn't what spot-based intuition expects. If the Series is missing some labels, those rows get NaN. If it has extra labels, they're dropped.
Assigning a list or a NumPy array has no index to align on. It goes in by spot. The length must match just or it raises.
That gap means df["x"] = some_series and df["x"] = some_series.values can give other columns from the same data. The first is often what you want. The second is the escape hatch when you know the order is right and the labels aren't.
Dropping
df.drop(columns=["b"]) and df.drop(index=[0]) both return a new frame.
The axis= form works and is older. columns=/index= say what they mean and are worth preferring.
A name that doesn't exist raises KeyError. That is often helpful. It catches typos and stale column lists. And errors="skip" turns it off when dropping optional columns.
To drop in place, reassign: df = df.drop(columns=["b"]). The inplace=True arg exists. Is discouraged: it doesn't well avoid a copy, it breaks chaining. It returns None. This Makes df = df.drop(..., inplace=True) a silent way to destroy your frame.
Renaming
df.rename(columns={"a": "alpha"}) takes a mapping, and columns=str.upper takes a function applied to each name.
The key quirk: keys that don't match anything are skipped silently. A typo in the old name does nothing at all and gives no indication. When a rename appears not to have worked, a misspelled key is the first thing to check.
df.columns = [...] replaces each name at once and needs the right length. It's blunter and, for a full rename, clearer.
For cleaning up messy headers, a function is often better than a mapping:
For column names that aren't valid identifiers. A space, a leading digit. Keyword args won't work. Bracket set is the only option. That is a fair arg for norming column names early.
Assigning a few columns from one step is the case assign handles less well. When a function returns a few values per row, a common pattern is:
The right-hand side must have the same number of columns as the left, and the index must align.
Reordering
Column order is insertion order, and there's no reorder method.
Three ways, in growing strength:
df[["b", "a", "c"]] — select in the order you want. Fails loudly if a name is wrong. This Is often a feature.
df.insert(pos, name, values) — place one column at a spot, in place.
Building the order from what exists:
first = ["id", "date"]
df = df[first + [c for c in df.columns if c not in first]]
That pins the columns you care about to the front and keeps the rest, without breaking when the schema changes.
Removing columns safely
drop(columns=[...]) raises on a name that doesn't exist. This Catches typos and stale lists.
errors="skip" suppresses that, and is right when the columns are truly optional. Dropping debug fields that may or may not be present.
df.drop(columns=df.filter(like="tmp_").columns) drops by pattern. This Lives changes in the exact names.
For keeping rather than dropping, selecting is clearer: df[["a", "b"]] states what you want rather than what you don't. Doesn't need updating when new unwanted columns appear.
A note on inplace
Most of these methods take inplace=True. It's worth being clear about why it's discouraged.
It does not well avoid a copy. The speed arg for it's largely false.
It returns None, so df = df.drop(columns=["a"], inplace=True) silently sets df to None. A mistake that is easy to make and confusing to debug.
It breaks method chaining.
And it's on the way out: the pandas team has discussed dropping it. Copy-on-write makes its left rationale weaker.
Reassignment — df = df.drop(columns=["a"]) — is clearer and no slower .
Renaming, at scale
For one or two columns, a mapping is fine.
For a whole frame that arrived with human-written headers, a function over the Index is better:
This handles trailing spaces, capitals, punctuation and units-in-parentheses in one pass. Doesn't need to know what the columns are called.
Doing it at once after loading means each later link uses predictable names. It drops the class of bug where a column name has an invisible trailing space.
df.rename(columns=..., errors="raise") makes a mapping strict. A key that matches nothing raises instead of doing nothing silently. That is worth using when the rename is key.
Creating columns conditionally
The three standard shapes, none of which needs apply:
Two branches — np.where(cond, a, b).
a few branches — np.select([c1, c2], [v1, v2], default=v3), checked in order.
Numeric bands — pd.cut.
A lookup — df["k"].map(mapping).
For a column that exists only for some rows, create it with a default first and then overwrite the subset:
That avoids the NaN that appears when a column is created right for a subset. It makes the default clear rather than implied.
Types when adding a column
A new column takes its dtype from what you assign. Two cases are worth watching.
Assigning a Python list of ints gives int64. Assigning one with a None in it gives object or float64. If the column is meant to be a nullable int, say so: pd.array([1, None], dtype="Int64").
Assigning a string to a subset of a numeric column upcasts the whole column to object, silently. This Makes each later numeric step on it slow or wrong. If a column may hold either, that is often a sign it should be two columns.
Dropping rows
drop(index=[...]) drops by label. This Is fine for a handful of known labels and awkward else.
For a rule, filtering is clearer: df[~mask] says "all except". Doesn't need knowing the labels.
df.drop(df[mask].index) works and is a longer way of writing the same thing.
dropna, drop_duplicates and query cover the common cases right. Each says what it does in its name.
A summary
df["new"] = ... adds or silently replaces.
assign returns a new frame and chains. Use a lambda inside a chain.
Assigning a Series aligns on the index. A list goes in by spot.
Whole-column set from a subset overwrites the unmatched rows with NaN. .loc touches only the pick.
drop(columns=...) raises on a missing name unless errors="skip".
rename skips keys that match nothing. So A typo does nothing at all.
Norm column names once, at load, with a function over df.columns.
And prefer reassignment to inplace=True. This Doesn't do what its name suggests.
A closing note on Adding and Removing Columns
Adding a column looks like the simplest thing in pandas and carries two traps.
The first is silence: assigning to an existing name replaces it without a word. A rename with a misspelled key does nothing without a word. Neither raises. Both are easy to miss in a long script.
The second is alignment. A Series assigned to a column is matched by label, not spot. A value found from a filtered subset fills the unmatched rows with NaN. And if the column already existed, it overwrites what was there. A plain list, having no index, goes in positionally instead. Two lines that look equal behave differently.
The defences are small: use .loc when you mean "these rows only", check the column list after a rename. Norm column names once at load so that later links are predictable.
One more thing
df.pop("col") drops a column and returns it. This Is at times the clearest way to move a column out of a frame and into a var in one step.
And df.drop accepts level= for a MultiIndexed frame. A whole outer group can be removed by label without building a mask. Both are small conveniences. Both read better than the two-step options when the intent is just what they show.
Questions people ask about Adding and Removing Columns
What is the difference between df['new'] = ... and df.assign(new=...)? Bracket assignment modifies the frame in place. assign returns a new frame, which makes it suitable for method chains.
Why does my new column come out full of NaN? Assigning a Series aligns it on the index. If its labels do not match the frame's index, the unmatched rows get NaN, so use .values or reset the index if you want position-based assignment.
How do I drop a column? Use df.drop(columns=['a', 'b']), or df.drop('a', axis=1). Both return a new frame unless you reassign.
Should I use inplace=True? Generally no. It rarely saves memory, it breaks method chaining, and it is being phased out in favour of reassigning the result.
How do I add a column based on a condition? Use np.where(cond, a, b) for two outcomes or np.select for several. Avoid apply with a row-wise function, which is much slower.
How do I rename columns? Use df.rename(columns={'old': 'new'}) for a few, or assign a full list to df.columns to rename them all.
Adding and Removing Columns, in summary
Adding and dropping columns is mechanically simple. The two things that bite are both about silence.
Nothing warns when you overwrite a column by reusing its name. Nothing warns when a rename key matches nothing. Both leave the frame in a state that looks correct.
And set is governed by alignment: a Series matches on labels, a list matches on spot. A whole-column set found from a subset overwrites each unmatched row with NaN. Including rows that held a just right good value a moment before.
Using .loc when you mean "these rows only", norming column names once at load. Checking the column list after a rename cover all of it.
Check yourself
0 of 4
Answer without scrolling back up.
What happens when you assign to an existing column name?
There is no protection against overwriting a column by reusing its name - a real hazard in a long script.
You assign a Series whose index is in a different order. What lands in the column?
Assigning a Series aligns by label; assigning a plain list goes in by position. That is why .values sometimes gives a different result.
An existing column holds 0 everywhere. What does `df['x'] = df[mask]['y'] * 2` do to the unmatched rows?
Whole-column assignment replaces the entire column and alignment has nothing for those labels. df.loc[mask, 'x'] = ... touches only the selected rows.
What does `df.rename(columns={'typo': 'new'})` do when 'typo' does not exist?
Unmatched keys are ignored. When a rename appears not to have worked, a misspelled key is the first thing to check.
Cheat sheet
Adding and Removing Columns
Add, rename and drop columns in a pandas DataFrame with assign and drop. See how index alignment decides what a new column contains.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.