From dicts, records, arrays and files - and which orientation each one assumes.
Overview
The two orientations
Almost every confusion about constructing a DataFrame comes from one question: does this input describe columns or rows?
pd.DataFrame({"a": [1,2,3], "b": [4,5,6]}) describes columns. Each key is a column name and each value is the entire column. This is the most common form and the one to reach for when you have parallel lists.
pd.DataFrame([{"a":1,"b":4}, {"a":2,"b":5}]) describes rows. Each dict is one record. This is the shape data arrives in from an API or a database cursor, and pandas handles it directly.
Both produce the same frame. Knowing which one you are writing prevents a transposed result.
Worth knowing
A dict of columns is the usual form: keys become column names, values become whole columns, and a scalar is broadcast.
A list of dicts is one record per row — missing keys become NaN rather than raising.
A list of lists gives integer column names unless you pass columns=.
pd.DataFrame(dict_of_scalars) raises. Wrap it in a list for one row, or use from_dict(orient="index") for one column.
read_csv infers a dtype per column, and takes dtype=, index_col= and usecols= to steer it.
Never build a frame row by row — each assignment reallocates it. Collect into a list and construct once.
Creating DataFrames
From dicts, records, arrays and files, and which orientation each assumes.
From a dict of columns
The most common form. Each key is a column name, each value the whole column.
example_01.pypandas
Output
From a list of records
One dict per row - the shape data usually arrives in from an API.
example_02.pypandas
Output
From a list of lists, and why columns= matters
Without names you get integer columns, which is rarely what you want.
example_03.pypandas
Output
Reading a CSV from text
read_csv is the usual entry point, and it takes any file-like object.
example_04.pypandas
Output
From a Series, and the orientation trap
A dict of scalars gives one row only if you tell it so.
example_05.pypandas
Output
Building rows in a loop is the slow way
The same lesson as NumPy: collect, then construct once.
example_06.pypandas
Output
What each form tolerates
Dict of columns requires equal lengths and raises otherwise, which is a useful check. A scalar value is the exception: it broadcasts to every row, so {"a": [1,2,3], "flag": True} gives a flag column of three True values.
List of dicts does not require the keys to match. Missing keys become NaN, and the column order follows first appearance across all records. That tolerance is convenient for messy input and worth watching: a typo in one record's key silently creates a new mostly-empty column rather than raising.
List of lists works but names the columns 0, 1, 2. Pass columns= unless you genuinely want integer names. Integer column names are legal and lead to real confusion later, because df[0] then means a column and looks like positional indexing.
Dict of scalars raises, because pandas cannot tell whether you meant one row or one column. pd.DataFrame([d]) means one row; pd.DataFrame.from_dict(d, orient="index") means one column. The error is good — guessing here would be worse.
read_csv
For real data this is the usual entry point, and it does a lot by default: infers a dtype per column, treats the first line as a header, and parses common missing-value markers.
The arguments worth knowing early:
index_col sets a column as the index during the read rather than afterwards.
usecols reads only the columns you name, which matters on wide files — it saves both time and memory.
dtype overrides inference. This is how you stop an identifier column being read as an integer and losing its leading zeros, and how you read a low-cardinality column straight into category.
parse_dates converts date columns during the read, which is faster than converting afterwards.
nrows reads a sample, which is the right way to inspect a large file before committing to it.
It accepts any file-like object, so io.StringIO works for testing and for data that is already in memory.
Type inference and its costs
read_csv and the constructors infer types per column. That is usually helpful and occasionally wrong in expensive ways.
A column of integers with one missing value becomes float64, because integer arrays cannot hold NaN. Identifiers then print as 1001.0.
A column that is mostly numbers with one stray non-numeric value becomes object, and every subsequent numeric operation on it either fails or silently operates on strings.
Both are worth checking with df.dtypes immediately after loading. It takes one line and catches a category of bug that otherwise surfaces much later.
Never grow a frame
acc.loc[len(acc)] = row inside a loop reallocates the entire frame on every iteration. So did df.append, which is why it was removed in pandas 2.0 rather than merely deprecated.
The correct pattern is the same as NumPy's: collect into a Python list and construct once at the end. Lists append cheaply; DataFrames do not append at all.
If the pieces are already frames, pd.concat(list_of_frames) once at the end is the right call — and pd.concat inside a loop is exactly the same mistake in a different costume.
The last editor measures it. The gap grows with the number of rows, so a pattern that seems acceptable on a hundred records becomes unusable on a hundred thousand.
Choosing
Parallel lists you already have: dict of columns.
Records from an API or a cursor: list of dicts.
A file: read_csv, with dtype and usecols set deliberately.
A NumPy array: pd.DataFrame(arr, columns=[...]), remembering that the array's single dtype applies to every column until you convert.
One row from a dict: pd.DataFrame([d]).
And in every case, print df.dtypes and df.shape immediately afterwards. Those two lines catch most of what goes wrong at the boundary between raw data and pandas.
From a NumPy array
pd.DataFrame(arr, columns=[...]) wraps an array. The array has one dtype, so every column starts with that dtype until you convert.
That matters when the array came from something that widened it. An array of mixed data is object, and a DataFrame built from it has every column as object — numeric-looking columns that do not behave numerically.
index= sets the labels at the same time.
For the reverse direction, df.to_numpy() collapses back to a single dtype, and df.values is the older spelling of the same thing.
From a database or an API
pd.read_sql(query, connection) runs a query and returns a frame, with column names taken from the result. It accepts a SQLAlchemy connection or a raw DBAPI one.
pd.read_json handles JSON, and pd.json_normalize is the one worth knowing: it flattens nested JSON into columns, turning {"user": {"name": "ana"}} into a user.name column. Most API responses need it, and building the frame by hand from nested dicts is the long way round.
pd.read_html(url_or_html) scrapes every table on a page into a list of frames. It is startlingly effective for simple pages and needs lxml or bs4 installed.
Setting the index at creation
Any of the constructors accept index=, and read_csv accepts index_col=.
Doing it at creation is slightly cheaper than set_index afterwards, and more importantly it documents that the column is a key rather than data.
If the index should be a DatetimeIndex, parse the dates in the same call: read_csv(path, parse_dates=["date"], index_col="date") gives a time-indexed frame in one step.
Empty frames, and why they cause trouble
pd.DataFrame() is legal and occasionally useful as a placeholder.
It is a poor starting point for accumulation, for two reasons. Growing it row by row is quadratic, as the last editor shows. And its columns have no meaningful dtype, so the first concat into it can widen everything to object — at which point the frame is slow and the numeric columns are no longer numeric.
pd.DataFrame(columns=["a", "b"]) has the same problem with more ceremony.
The pattern to use instead is always the same: build a list of rows or of frames, and construct once at the end.
Checking what you built
Four lines, immediately after construction, catch most problems at the boundary:
df.shape # is this the number of rows I expected?
df.dtypes # did anything become object or float unexpectedly?
df.head() # does the data look like the data?
df.isna().sum() # where are the gaps?
The second is the one that earns its keep. Type inference is where most silent damage happens, and it happens exactly once, at construction.
Choosing a constructor
Parallel lists you already have — a dict of columns.
Records from an API or a cursor — a list of dicts, or json_normalize if they are nested.
A file — read_csv, with dtype, usecols and parse_dates set deliberately.
A query — read_sql, which names the columns for you.
A NumPy array — the constructor with columns=, remembering the single dtype.
One row from a dict — pd.DataFrame([d]).
Accumulated pieces — a list, then pd.DataFrame(rows) or pd.concat(frames) once.
In every branch, set the dtype at creation rather than converting afterwards. astype allocates a second full frame, and getting it right the first time avoids both the copy and the class of bugs where a column silently is not the type you assumed.
Reproducible test frames
Small frames written by hand are how you check that an operation does what you think, and it is worth having a habit for them.
A dict of columns is the most readable form for a handful of rows. pd.DataFrame({"a": [1,2,3], "b": list("xyz")}) fits on one line and shows the columns clearly.
For a frame with a specific index, pass index= rather than setting it afterwards.
For random test data, seed it: np.random.default_rng(0). Unseeded random test data makes a failing check unreproducible, which is the opposite of what a test is for.
pd.util.testing used to provide frame generators; the supported route now is pd.testing.assert_frame_equal for comparisons, and building the frames yourself.
Checking two frames match
pd.testing.assert_frame_equal(a, b) is the precise comparison: values, dtypes, index and column order all have to agree, and the error message says which differed.
check_dtype=False relaxes the type comparison, which is often necessary when one side came through a CSV.
check_like=True ignores row and column order.
This is what belongs in a test, rather than a.equals(b), because when it fails it tells you *what* failed.
Copy semantics at construction
pd.DataFrame(some_array) does not always copy. If the array's dtype and layout are usable directly, pandas may wrap it, and writing to the frame then writes to the array.
copy=True forces a copy. Under copy-on-write this stops mattering, which is another reason that change is welcome.
pd.DataFrame(dict_of_series) aligns the Series on their indexes, which is easy to forget: three Series with different labels produce a frame with the union of them and NaN in the gaps, not three columns side by side.
If you meant "these are parallel columns", reset the indexes first or pass plain lists.
The construction checklist
Decide whether the input describes rows or columns.
Pass columns= if the source has no names.
Set dtype at construction, especially for identifiers and for integers that may have gaps.
Set index= if a column is really a key.
Then check shape, dtypes, head() and isna().sum().
Never grow a frame in a loop.
That is six lines of discipline that prevent most of what the cleaning modules exist to repair.
A closing note
Construction is where a frame's types are decided, and types decided badly here cause problems everywhere afterwards.
The two questions worth answering deliberately are whether the input describes rows or columns, and what the dtypes should be. Getting the first wrong gives a transposed frame that is immediately obvious. Getting the second wrong gives a frame that looks correct and behaves oddly much later.
dtype= at construction costs nothing. astype afterwards allocates a second copy of every column it touches, and by then the leading zeros may already be gone.
The other rule is the one shared with NumPy: never grow a frame in a loop. Collect into a list and construct once. df.loc[len(df)] = row and pd.concat inside a loop are the same quadratic mistake wearing different syntax, and df.append was removed rather than merely deprecated because of it.
One more thing
pd.DataFrame.from_records handles an iterable of tuples with a columns argument, and accepts an index naming one of the fields. It is the constructor that fits a database cursor most directly, since cursors yield tuples rather than dicts.
And pd.concat of many small frames is usually faster than building one large list of dicts when the pieces are already frames — the rule is to avoid the loop, not to prefer one container over the other.
In summary
Decide whether the input describes rows or columns, set the dtypes at construction, and check shape, dtypes and head immediately afterwards.
Never grow a frame in a loop — collect into a list and build once. df.append was removed rather than deprecated because of exactly that pattern.
Check yourself
0 of 4
Answer without scrolling back up.
What does `pd.DataFrame({'a': [1,2,3], 'flag': True})` produce?
A scalar value is broadcast to every row, while list values must all have matching lengths.
Why does `pd.DataFrame({'a': 1, 'b': 2})` raise?
Wrap it in a list for one row, or use from_dict(orient='index') for one column. Guessing would be worse than the error.
A CSV column of integers has one missing value. What dtype does it get?
This is why identifiers sometimes print as 1001.0. Check df.dtypes right after loading - it takes one line and catches the bug early.
What happened to `df.append()`?
Collect rows into a Python list and construct once. pd.concat inside a loop is the same mistake in different clothing.
Cheat sheet
Creating DataFrames
pd.DataFrame({"a": [1,2,3], "b": [4,5,6]}) describes columns. Each key is a column name and each value is the entire column. This is the most common form and the one to reach for when you have parallel lists.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.