Optional does not mean optional. The three-way difference between missing, null and defaulted, and the mutable-default trap.
Overview
The rule, first
Every field on a model is in one of three states. It is required, so the caller must provide it. It has a default, so the model fills it in when the caller does not. Or it can be None, which is a separate idea that people often mix up with the other two.
The confusion comes from the word Optional. In Python's typing module it means "this value may be None". It does not mean "you may leave this out". A field typed as Optional with no default is still required. That one fact causes more wrong assumptions than almost anything else in the library.
This article separates the three ideas and shows how Pydantic treats each. You will see the simple rule for what makes a field required, and why Optional does not change it. It then looks at the difference between a field that is missing and a field that is present but null, since an API often needs to tell those apart. The last part covers mutable defaults, such as lists, which are a famous trap in plain Python and behave differently here. Read it before you design any model that accepts input from outside.
Nothing here is hard, but it is easy to get wrong by assumption. Most of the surprises come from a single word. Once you have seen how each state behaves on real input, you will stop second-guessing your own models. You will also read other people's models faster, because you will know what to look for first.
A field is needed when it has no default value. That is the entire rule. The type annotation plays no part in it.
class Module(BaseModel):
title: str # required
minutes: int = 10 # optional, defaults to 10
Leave out title and the model refuses to be built, with an error whose type is missing. Leave out minutes and you get 10.
all confusing about this topic comes from one word being borrowed for two jobs.
Worth knowing
A field is required when it has no default. The type is irrelevant to that decision.
Optional[str] means str | None. To make a field genuinely optional you must also write = None.
model_fields_set tells you which fields the caller actually supplied — the only way to distinguish “omitted” from “sent as null”.
model_dump(exclude_unset=True) returns only what was supplied. This is the correct shape for a PATCH request, where absent means “leave alone”.
Unlike plain Python, a mutable default such as [] is safe here — Pydantic deep-copies it per instance. Use default_factory when the value must be computed each time.
Defaults are not validated unless you set validate_default=True. It is a deliberate speed trade, and it will hide a typo in a default.
Required, Optional and Defaults: The Three-Way Distinction
Why Optional doesn't mean optional. How to tell "not sent" from "sent as null".
A field with no default is required
That is the whole rule. Leave it out and the model refuses to exist, with the error type missing.
example_01.pyPydantic
from pydantic import BaseModel, ValidationError
class Module(BaseModel):
title: str
minutes: int
print(Module(title="Norms", minutes=8))
try:
Module(title="Norms")
except ValidationError as e:
err = e.errors()[0]
print("refused :", err["loc"], "->", err["msg"], "|", err["type"])
Output
Optional[int] is not an optional field
This is the single most common misreading in the library. Optional[int] means “int or None”. It says nothing about whether the field must be supplied.
example_02.pyPydantic
from typing import Optional
from pydantic import BaseModel, ValidationError
class Strict(BaseModel):
note: Optional[str] # nullable, but STILL REQUIRED
class Truly(BaseModel):
note: Optional[str] = None # nullable AND optional
print("explicit None :", Strict(note=None))
try:
Strict()
except ValidationError as e:
print("omitted : refused ->", e.errors()[0]["type"])
print("with a default:", Truly())
print("still nullable:", Truly(note=None))
# The default is what makes a field optional. Not the type.
Output
Three states, not two
“Not sent” and “sent as null” are different facts, and an API often needs to tell them apart. model_fields_set is how you ask.
example_03.pyPydantic
from typing import Optional
from pydantic import BaseModel
class Update(BaseModel):
title: Optional[str] = None
minutes: Optional[int] = None
absent = Update()
nulled = Update(title=None)
given = Update(title="Projections")
for name, m in [("omitted", absent), ("sent as null", nulled), ("sent", given)]:
print("%-13s fields_set=%-12s dump=%s"
% (name, m.model_fields_set, m.model_dump()))
print()
print("only what was actually sent:", given.model_dump(exclude_unset=True))
print("this is how a PATCH endpoint avoids overwriting untouched fields")
Output
The mutable default trap
In ordinary Python a mutable default is shared by every call. Pydantic does not have that bug — but the habit it teaches, default_factory, is still what you want for anything computed.
example_04.pyPydantic
from typing import List
from pydantic import BaseModel, Field
# First, the classic Python bug, so the contrast is clear:
def add_tag(tag, tags=[]): # <- shared list, created once
tags.append(tag)
return tags
print("plain python:", add_tag("maths"))
print("plain python:", add_tag("python"), "<- the first call leaked in")
# Pydantic copies the default per instance, so this is safe:
class Module(BaseModel):
title: str
tags: List[str] = []
a = Module(title="A")
b = Module(title="B")
a.tags.append("maths")
print()
print("a.tags:", a.tags)
print("b.tags:", b.tags, "<- unaffected")
# Use default_factory when the default must be COMPUTED per instance:
import itertools
counter = itertools.count(1)
class Draft(BaseModel):
order: int = Field(default_factory=lambda: next(counter))
print()
print("factory:", Draft().order, Draft().order, Draft().order)
Output
Defaults are not validated by default
Pydantic trusts what you wrote in the class body and skips checking it. That is a speed decision, and it means a wrong default sits there quietly until you ask for it to be checked.
example_05.pyPydantic
from pydantic import BaseModel, ConfigDict, ValidationError
class Sloppy(BaseModel):
minutes: int = "not a number" # nonsense, and never noticed
class Careful(BaseModel):
model_config = ConfigDict(validate_default=True)
minutes: int = "not a number"
s = Sloppy()
print("sloppy :", repr(s.minutes), "<- a str in an int field")
print("and so :", type(s.minutes).__name__)
try:
Careful()
except ValidationError as e:
print("careful : caught it ->", e.errors()[0]["msg"])
Output
Putting it together
A realistic model: some fields the caller must supply, some with sensible defaults, one genuinely nullable, and a list that starts empty.
example_06.pyPydantic
from typing import List, Optional
from pydantic import BaseModel, Field
class Module(BaseModel):
title: str # required
track: str # required
minutes: int = 10 # defaulted
published: bool = False # defaulted
summary: Optional[str] = None # nullable and optional
tags: List[str] = Field(default_factory=list)
minimal = Module(title="Determinant", track="maths")
print("minimal :", minimal.model_dump())
print()
full = Module(title="Determinant", track="maths", minutes=13,
published=True, summary="Area, and what a matrix does to it.",
tags=["linear-algebra", "geometry"])
print("full :", full.model_dump())
print()
print("what the caller actually set:", full.model_fields_set)
print("only their input :", minimal.model_dump(exclude_unset=True))
Output
Optional does not mean optional
Optional[str] comes from the typing module. Here It means just one thing: str or None. It's a statement about which *values* are allowed. It says nothing whatsoever about whether the field has to be supplied.
class Strict(BaseModel):
note: Optional[str] # nullable, and STILL REQUIRED
Strict(note=None) works. Strict() raises missing. This surprises nearly all once. The surprise is fully the fault of English rather than of Pydantic. Optional was named for type theory, not for form fields.
To get the action people often mean, add the default:
class Truly(BaseModel):
note: Optional[str] = None # nullable AND optional
It's worth internalising as two split questions. *Can this value be null?* is answered by the type. *Must the caller provide it?* is answered by whether there's a default. They're split. All four mixes are legitimate and at times handy.
Pydantic v1 blurred this: it made Optional fields default to None on its own. Pydantic v2 stopped, just because the silent default hid the gap. If you find old code that leans on it, this is one of the changes that will bite during a migration.
Missing and null are different facts
For most models the gap doesn't matter. For anything that updates existing data, it matters enormously.
Consider a PATCH endpoint. A client sends {"title": "New Name"}. What should happen to summary? Obviously nothing. They didn't mention it. Now a client sends {"summary": null}. What should happen? Just as obviously, summary should be cleared. Those are opposite intentions. After check both fields are None in the model.
model_fields_set recovers the gap. It's a set of the field names the caller really supplied:
And model_dump(exclude_unset=True) uses it, returning only the fields that were given. That output is the correct thing to apply to an existing record. Absent keys are absent. An clear None is present and clears the value.
Getting this wrong is a real and common bug. An update endpoint that dumps the whole model and writes each field will happily overwrite six columns with None because the client only mentioned one.
The mutable default trap, and why it is not one here
each Python programmer eventually learns this the hard way:
The list is created once, when the function is defined. Shared by each call that doesn't pass one. The second call sees the first call's data.
Pydantic doesn't have this bug. A default like tags: List[str] = [] is deep-copied for each new instance. Two models never share a list. You can write the natural thing and it's safe.
default_factory still matters, though, for defaults that must be *found* rather than copied. A timestamp, a made id, a counter. Anything whose value should reflect the moment of creation:
Written as = datetime.now() that would freeze the time the class was defined. Each model would claim to have been created at import.
Defaults are not checked
This one is quiet and at times costly. By default, Pydantic doesn't validate default values:
class Sloppy(BaseModel):
minutes: int = "not a number"
That class meaning raises nothing, and Sloppy().minutes is the string. The thinking is speed. You wrote the default yourself. Checking it on each instantiation is work with no expected payoff.
The cost is that a mistyped default is invisible until a thing downstream does math. If you'd rather be told, model_config = ConfigDict(validate_default=True) turns the checking on, at a small cost per instance.
For a model with hand-written defaults that never change, the default action is fine. For a model whose defaults come from config or a constant defined elsewhere, turning check on is cheap insurance.
Choosing well
A few habits that keep models honest.
Prefer needed fields. each default is a choice made on behalf of a caller who didn't make it. A model where all is optional docs nothing. A reader can't tell what is really sure.
Don't use None as a stand-in for a real default. If a missing duration means ten minutes, write minutes. Int = 10, not Optional[int] = None with a or 10 scattered through the code that follows.
Reserve None for values that are truly absent in the domain. A summary no one has written yet, an end date for a thing still running. Then None carries meaning rather than marking a gap in the model.
And when you're modelling an update rather than a creation, reach for exclude_unset early. It's easier to build the endpoint right than to work out later why six columns went blank.
The ellipsis, and other spellings of "required"
You'll meet Field(...) in older code and in a lot of docs:
title: str = Field(..., min_length=3)
The literal Ellipsis was Pydantic v1's way of saying "there's no default, this is needed". Field needed a thing in the default spot. It still works in v2. It's redundant: omitting the default says the same thing.
title: str = Field(min_length=3) # identical, and clearer
Prefer the second. The first makes readers who haven't met the rule stop and look it up, and it buys nothing.
Defaults that depend on the environment
A default doesn't have to be a literal. default_factory takes any callable. This Means a default can come from config, the clock, or a gen:
The third one is worth a caution. Reading config inside a default factory works. It happens at model-build time rather than at import. This Makes it harder to reason about and hard to replace in tests. For anything that is really config, a settings model is the better home. That is a module in the last tier.
There's also a form of default_factory that receives the already-checked data, letting a default depend on other fields. It's powerful and easy to overuse. A value found from other fields is often better expressed as a computed_field. This Doesn't pretend to be an input.
How this appears in the schema
The gap between needed, optional and nullable isn't just a Python concern. It shows up in the made JSON Schema. So in your API docs and any client made from it.
A field with no default appears in the schema's needed array. A field with a default doesn't. Its default is recorded. A nullable field's type turns an anyOf including null.
So the four mixes give four truly other contracts for a consumer:
str — must be sent, can't be null. str = "x" — may be omitted, can't be null. Optional[str] — must be sent, may be null. Optional[str] = None — may be omitted, may be null.
Reading them as sentences like that is the fastest way to check you've written what you meant. If the sentence sounds wrong for your API, the annotation is wrong.
Designing create and update models
The place all of this comes together is a resource with more than one shape.
Create takes what a caller may supply. The server-assigned id is absent fully. Not optional, absent. Because including it invites a caller to try setting it.
Update, for a PATCH, has each field optional with a None default. Is dumped with exclude_unset=True so that untouched fields stay untouched.
Output has all the caller is allowed to see, with server-assigned fields needed. By the time you're returning one they exist.
class ModuleCreate(BaseModel):
title: str
minutes: int = 10
class ModuleUpdate(BaseModel):
title: Optional[str] = None
minutes: Optional[int] = None
class ModuleOut(BaseModel):
id: int
title: str
minutes: int
Three small classes rather than one clever one. Each says just what it means. None of them needs a comment showing which fields apply when.
The temptation is always to collapse them into a single model with all optional. It looks like less code and it's. It's also a model that docs nothing, makes useless API docs. Can't tell a client what is sure in a response.
A note on validation order
Fields are checked in declaration order. Defaults are filled as part of that pass. This matters once you write a checker that reads another field. It can only see fields declared *above* it. The ones below haven't been processed yet.
If a rule needs the whole object, that is what model_validator(mode="after") is for. It runs once, after each field is in place. Trying to express a cross-field rule as a field checker on whichever field happens to come last works until somebody reorders the class.
A checklist for a field you are about to write
Four questions, in order, and the annotation falls out of the answers.
Can this be absent? If yes, it needs a default. If no, leave the default off and let the model refuse.
Can this be null, usefully? Only if "no value" is a real state in your domain. An unwritten summary, an unfinished end date. If None would just mean "no one bothered", it isn't nullable. It's defaulted.
Is the default a constant or does it depend on when we are? A constant goes in right. Anything found. A time, an id, a fresh holder — goes in default_factory.
Am I creating or updating? Creating means needed fields are needed. Updating means all is optional and you dump with exclude_unset=True.
Most confusing models are the result of answering the second question with "I suppose so" instead of thinking about it. Optional[X] = None is the annotation people reach for when they haven't decided. A model full of them has quietly recorded that nothing was ever decided.
What this buys downstream
The payoff for being precise here shows up one spot else fully: in the code that reads the model.
If summary: Optional[str] = None truly means "may not have been written yet", then if module.summary: is a real branch about the domain. If it means "we weren't sure", each reader has to defend against None on each field. The type system has stopped helping.
needed fields are a promise to the rest of your program. Each one you make lets code downstream stop checking. That is the actual product of this module. Not the syntax. The discipline of deciding what is sure and then writing it down where the compiler, the schema and the next reader can all see it.
One last distinction
There's a fourth state people at times need, beyond needed, defaulted and nullable. A field that may be absent but has no sensible default. Here You truly want to know whether it was supplied.
That is what model_fields_set and exclude_unset exist for. It's worth naming by name because the instinct is to invent a sentinel. A magic string, a -1, a custom UNSET object. And thread it through the code. Pydantic already tracks the answer. Reach for the sentinel only when the value has to live output. This Is rare. Think hard before you do. Each consumer of that data now has to know about your magic value.
Questions people ask about Required, Optional and Defaults
Does Optional[str] make a field optional? No. It only means the value may be None. Without a default, the key must still be present.
How do I make a field both optional and nullable? Write name: str | None = None. The default makes it omittable and the union allows an explicit null.
Is a default value validated? No, defaults are not validated unless validate_default=True. A bad default will slip through until it is used.
Is it safe to use a list as a default? Yes, in Pydantic. Each instance gets its own copy, unlike a plain Python function default. You can also use Field(default_factory=list).
How can I tell a missing field from one set to None? Check model_fields_set. It lists fields the caller actually supplied, including explicit None.
How do I write an update model where every field is optional? Give each field a default of None, and use model_dump(exclude_unset=True) to apply only the fields the caller sent.
Where to go after Required, Optional and Defaults
The next module is about the moment a model says no. How to read a ValidationError in full, what each part of an entry means. How to turn one into a message a user can act on.
Check yourself
0 of 4
Answer without scrolling back up.
A field is declared `note: Optional[str]` with no default. Is it required?
`Optional[str]` means `str or None`, which is about allowed values. Requiredness is decided by whether a default exists. Pydantic v1 added the default implicitly; v2 deliberately does not.
Which tells you whether a caller actually supplied a field?
After validation an omitted field and an explicitly null one both read as `None`. `model_fields_set` is the only record of what was actually sent.
What does `model_dump(exclude_unset=True)` produce, and why does it matter?
An update that dumps every field will overwrite untouched columns with defaults or None. Excluding unset fields keeps 'not mentioned' meaning 'leave alone'.
Why use `default_factory=datetime.now` instead of `= datetime.now()`?
`datetime.now()` is evaluated once, when the class body runs, so every instance would claim the same creation time. A factory is called per instance.
Cheat sheet
Required, Optional and Defaults
Learn why Optional does not mean optional in Pydantic, and how to distinguish required, defaulted and nullable fields correctly.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.