Files and with

Opening a file so it always closes, reading it a line at a time, and the modes that truncate what was there.

Overview

Opening a file so it always closes, reading it a line at a time, and the modes that truncate what was there.

In this guide
  1. A note about this page
  2. with, and why
  3. Three ways to read
  4. The modes
  5. Missing files raise
  6. Paths, and why pathlib is worth the switch
  7. Encoding is not optional in practice
  8. Writing safely
  9. Binary mode
  10. A worked example, start to finish
  11. What with actually is
  12. Do not parse standard formats by hand
  13. Where file code goes wrong in practice
  14. Line endings, and the translation you did not ask for
  15. Questions people ask about Files and with
  16. Files and with: recap in one screen
  17. with is a protocol, and files are only its most common user

files.py

files.py Python 3
Output

                    

file_modes.py

file_modes.py Python 3
Output

                    

Worth knowing

with closes the file when the block ends, including when it raises.
"w" truncates the file immediately. "a" appends.
Iterating the file object reads one line at a time - the right way for large files.
Lines keep their trailing newline. Use .rstrip().

Files and with: A Practical Guide

Opening a file gives you an object to read or write. The with statement makes sure it's closed afterwards. This Is the part that is easy to skip and costly to get wrong.

A note about this page

These editors run Python in your browser against an in-memory filesystem. The files are real while the program runs and vanish afterwards. Everything below behaves just as it would on your machine. It just does not persist between runs.

Working with files is one of the most practical skills in programming. Programs read configuration, process log files, load data for analysis, and write results to disk. Python makes file operations straightforward, but there is one critical pattern you need to learn from the start: always close a file when you are done with it.

An open file holds system resources. The operating system tracks it, buffers its writes, and keeps a lock on it. If your program opens files without closing them, it eventually runs out of file handles and new open calls fail. Closing a file also flushes any buffered writes to disk. If you skip the close, the last chunk of data you wrote might never reach the file.

The with statement solves this. It opens the file, runs your code, and closes the file automatically when the block ends, even if your code raises an exception. You do not need to remember to call .close(), and you do not need to write a try/finally block. The with statement handles both.

The basic pattern is with open("data.txt") as f: followed by an indented block that reads or writes. The file is open inside the block and closed the moment the block ends. This is how virtually all modern Python code handles files.

Reading has several options. f.read() loads the entire file as one string. f.readline() reads one line. for line in f: iterates line by line without loading the whole file into memory, which matters for large files. Each line includes its trailing newline character, so you often see line.strip() or line.rstrip(" ") to remove it.

Writing uses f.write(text), which writes exactly the text you give it with no added newline. If you want a newline, include it: f.write(text + " "). Or use print(text, file=f), which adds the newline automatically.

This page covers the with statement, reading files, writing files, file modes, and the common mistakes beginners make when working with file input and output.

with, and why

with open("notes.txt") as f:
    content = f.read()

When the block ends the file is closed, whether it ended most times or by raising. Doing it by hand means f.close() in a finally. Losing it leaves the handle open. Which on a long-running program eventually exhausts the operating system's limit. On a write leaves data sitting in a buffer that never reaches the disk.

with isn't a style preference here. It's the correct way to open a file.

Three ways to read

f.read()       # the whole thing as one string
f.readlines()  # a list of lines
for line in f: # one line at a time

The third is the one to reach for by default. It holds a single line in memory even so of file size. It works on a file larger than your RAM. It reads no worse than the others.

Lines keep their trailing \n, which is why line.rstrip() appears in almost every loop over a file.

The modes

"r"  read, the default — raises if the file is missing
"w"  write — truncates right away
"a"  append — writes to the end
"x"  create — raises if it already exists

"w" is the dangerous one. It empties the file the moment it's opened, before you write any of it. An open(path, "w") that then raises leaves you with nothing. When you mean "add to this", "a" is the mode.

"x" is worth remembering when overwriting would be a bug: it refuses rather than destroying.

Missing files raise

open("nope.txt") raises FileNotFoundError, it doesn't return an empty file. Handle it, or let it pass. Both are reasonable. Don't check with os.path.exists first. The file can disappear between the check and the open. The try handles that right anyway.

Paths, and why pathlib is worth the switch

String concatenation for paths breaks across operating systems and on edge cases like trailing slashes. pathlib handles both and reads better:

from pathlib import Path

data = Path("content") / "articles" / "notes.txt"
text = data.read_text(encoding="utf-8")

read_text and write_text open, read or write, and close in one call, which covers the common case where you want the whole file and don't need to stream it. For any of it iterative, data.open() gives you the same file object open would.

Path also carries the questions you'd or else ask the os module: .exists(), .suffix, .stem, .parent, .glob("*.txt").

Encoding is not optional in practice

Text mode decodes bytes into a string using an encoding, and if you don't name one, Python uses the platform default. That default differs between machines, which is how a script that works on one computer makes UnicodeDecodeError on another, with data that hasn't changed.

Naming it drops the whole class of problem:

open(path, encoding="utf-8")

If you're handling files you didn't create and can't assume, errors="replace" lets you read a thing rather than crashing, at the cost of substituting the bytes it couldn't decode.

Writing safely

Writing right over the only copy of a file has an obvious failure mode: if the program dies halfway, the first is gone and the swap is incomplete. The standard remedy is to write beside it and rename:

tmp = path.with_suffix(".tmp")
tmp.write_text(new_content, encoding="utf-8")
tmp.replace(path)

replace is atomic on the same filesystem. So A reader sees either the old file or the new one and never a half-written one.

Binary mode

Adding b - open(path, "rb") - skips decoding and gives bytes. That is what you want for images, archives, and any of it you're copying rather than reading. The mistake in the other direction is more common: opening a text file in binary mode and then being puzzled that checks against strings all fail, because b"abc" and "abc" are different types that never compare equal.

A worked example, start to finish

Writing a file, reading it back and summarising it, using the tools from this page rather than the ones from the os module:

from pathlib import Path

path = Path("scores.txt")
path.write_text("ana 91\nbo 78\ncy 91\n", encoding="utf-8")

total = count = 0
for line in path.read_text(encoding="utf-8").splitlines():
    name, score = line.split()
    total += int(score)
    count += 1

print(count, "rows, mean", round(total / count, 1))
3 rows, mean 86.7

splitlines() rather than split("\n") because it handles the line endings of files written on other systems, and because it doesn't leave an empty string at the end from the final newline. encoding="utf-8" appears on both calls, because the default is the platform's and the platform's isn't yours.

For a file of this size, reading it whole is fine. The version that scales is the loop over the open file. This Holds one line at a time:

with path.open(encoding="utf-8") as f:
    for line in f:
        name, score = line.split()

Both are correct. The difference only matters when the file stops fitting in memory, which is the point at which it matters a great deal.

What with actually is

with isn't special syntax for files. It works with any object building two methods, and knowing that turns it from a rule into a tool.

An object is a context manager if it has __enter__, called on the way in, and __exit__, called on the way out. open returns one. So Does a lock, a DB link, a temporary directory, and any of it from contextlib. __exit__ runs whether the block finished most times, returned, or raised, which is the ensure the whole make exists to provide.

The as name receives what __enter__ returns. For a file, the file object itself. That is why with open(...) as f gives you f and why the name is optional when the object is only needed for its side effect.

Writing one is a decorator and a yield:

from contextlib import contextmanager

@contextmanager
def timer(label):
    import time
    start = time.perf_counter()
    yield
    print(label, round(time.perf_counter() - start, 3))

All before the yield is setup, all after is cleanup, and the yield is where the body of the with runs. This is the same gen machinery from elsewhere in the track, used for a fully different purpose.

a few managers can share one statement: with open(a) as f, open(b, "w") as g: opens both and closes both, in reverse order, even if the second open raises.

Do not parse standard formats by hand

The single most common mistake with files is treating a structured format as plain text. CSV is the usual victim, because it looks like split(",") will work, and it does until a field holds a comma inside quotes — at which point the parse silently makes the wrong number of columns.

The standard library has a module for each of the formats you're likely to meet. csv handles quoting, embedded newlines and different delimiters, and csv.DictReader gives you dicts keyed by the header row. json handles escaping, Unicode and nesting, and round-trips to Python types. configparser reads INI files. sqlite3 is there when a file of records has started to want queries.

Each of them costs one import and drops an entire category of bug you'd or else discover on somebody else's data. The rule is worth stating plainly: if the format has a name, a thing in the standard library already reads it.

Where file code goes wrong in practice

Four failures account for most of it, and none of them are exotic.

The path is scaled to the wrong place. A scaled path is resolved against the process's working directory, not the script's location, so the same program works when run from its own folder and fails from anywhere else. If a file lives beside the script, Path(__file__).parent / "data.txt" says so.

The file is open longer than intended. A file opened without with and never closed keeps a handle and, on writes, keeps data in a buffer that hasn't reached the disk. On a short script the interpreter cleans up on exit and the bug is invisible. In a long-running program it builds up until the process hits the operating system's limit.

The encoding was assumed. This is the failure that travels: it works on the machine that wrote the file and raises UnicodeDecodeError on a colleague's. Name the encoding on each text-mode open.

The write wasn't atomic. A program that dies mid-write leaves a truncated file where the good one used to be. The write-then-rename pattern earlier on this page costs two lines and drops the possibility.

Line endings, and the translation you did not ask for

Text mode does more than decode bytes into characters: it also translates line endings, and knowing that shows a family of confusing results.

Windows ends lines with carriage-return plus newline. All else uses a newline alone. Python's text mode hides the difference. On reading, any of the variants turns a plain newline in your string. This Is why a loop over lines behaves the same way on each platform. On writing, a newline is translated back into what the platform uses.

That is almost always what you want, and it has two consequences worth knowing. First, the string you read isn't byte-for-byte what is in the file, so a length found from the text can differ from the file size. Second, if you open a file in binary mode you get no translation at all, and lines end with what the file really holds. Which is why text read as binary often shows a trailing carriage return that seems to have come from nowhere.

The one place it actively causes trouble is the csv module, which does its own line-ending handling. Opening a CSV without newline="" lets both layers translate, and the result is a blank line between each row on Windows. The documented fix is just that arg, and it's worth passing habitually rather than discovering the need for it later.

newline="" is also what you want when reading a file whose exact line endings matter — a diff tool, a formatter, any of it that must write back what it read without silently changing it.

Questions people ask about Files and with

Do I still need f.close() if I use with? No. That is the entire point of with.

What is the difference between "w" and "a"? "w" empties the file the moment it opens. "a" keeps what is there and writes at the end.

How do I check whether a file exists? Path(p).exists() — but for opening, prefer to just open it and handle FileNotFoundError, because the file can vanish between the check and the open.

Why does my file have blank lines between rows? On Windows, opening a CSV without newline="" makes doubled line endings. Pass newline="" to open when using the csv module.

Can I read a file backwards? Not right. Read the lines and reverse them, or seek from the end if the file is too big for that.

What does errors="replace" do? Substitutes a placeholder for bytes that can't be decoded. So You get a thing rather than an exception. handy for salvage, not for data you care about.

Is pathlib slower than string paths? Marginally, and never enough to matter next to the file system call that follows.

Can I open the same file twice? Yes, and on most systems you can even open it for reading and writing at once. Whether that is a good idea is a different question.

What does f.seek(0) do? Moves the spot back to the start, which is how you read a file a second time without reopening it.

Why is my file empty until the program ends? Because the data is still in a buffer. Closing the file. Which with does — or calling f.flush() writes it out.

Should I use os.path or pathlib? pathlib for new code. os.path isn't deprecated and you'll keep meeting it, so both are worth reading fluently.

How do I append to a file that might not exist? Mode "a" creates it if it's missing, so no check is needed.

Files and with: recap in one screen

  • with closes the file whether the block ends most times or raises. It's the correct way to open one, not a style choice.
  • Iterating the file object reads one line at a time and works on files larger than memory.
  • "w" truncates on open. "a" appends. "x" refuses to overwrite.
  • Name encoding="utf-8" on each text-mode open, because the default varies by machine.
  • If the format has a name. CSV, JSON, INI. Use the module rather than splitting strings.

with is a protocol, and files are only its most common user

Nothing in with is about files. It calls __enter__, runs the block, and calls __exit__ but the block ends. Which makes it the right shape for any of it with a matching pair of ops. Open and close. Begin and commit. Acquire and release.

  • Locks, and the ways they go wrong. with lock: is this page's pattern with a different resource, and the reason to prefer it over acquire() and release() is one you've already met. An exception in the middle of the block still releases.

There's one place with won't save you. Opening and reading a file is a blocking call, and inside an async def it stops the event loop rather than yielding to it — the with block is scoped just right right and the whole loop is frozen for its duration.

Check yourself

0 of 3

Answer without scrolling back up.

  1. What does `with` do for a file?

  2. Opening an existing file with mode "w" does what?

  3. Why iterate `for line in f` rather than use readlines()?

Cheat sheet

Files and with

Open, read and write files using the with statement so Python closes them even when your code raises an exception.

PYTHON · vizlearn.in/python/files_and_with.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.

Teaching a course with this? Individual teachers can use it freely; for a whole department or programme, see institutional licences.