Partitioning by Range and Hash

One logical table, several physical pieces. The query planner skips the pieces it can prove are irrelevant.

Overview

One table, several files

Partitioning splits a table into physical pieces by a rule on one or more columns. Applications keep querying the single logical table; the engine decides which pieces are involved.

This is not sharding. Every partition lives in the same database on the same server. [Sharding](sharding_in_databases.html) spreads data across separate machines and is a much larger commitment.

The query above builds three range partitions and a view that unions them, which is the shape a partitioned table has underneath. Run the variants to see what each strategy costs.

Partitioning by Range and Hash

Compare the access patterns

query.sql SQLite
Result

Worth knowing

Partitioning splits one table into pieces by a rule on a column. The table still looks like one table.
Range partitions by an ordered value, usually a date. Hash spreads rows evenly by hashing a key.
The win is partition pruning: a query with a predicate on the partition key touches only the partitions that can match.
Dropping a partition is instant. Deleting the equivalent rows is not, which is why time-series data is almost always range-partitioned.

Partitioning by Range and Hash

Cutting one table into pieces the planner can skip, and the choice of where to cut.

Range partitioning

Pick an ordered column — almost always a timestamp — and give each partition a bounded interval.

orders_2024  :  order_date >= '2024-01-01' AND < '2025-01-01'
orders_2025  :  order_date >= '2025-01-01' AND < '2026-01-01'
orders_2026  :  order_date >= '2026-01-01' AND < '2027-01-01'

A query with WHERE order_date >= '2026-01-01' can be proved to need only the last partition, so the engine reads one file instead of three. That proof is partition pruning, and it is where nearly all the benefit is.

The second benefit is administrative and, for time-series data, often the bigger one. Deleting a year of orders with DELETE writes a row version per deleted row, generates enormous WAL traffic, and leaves the table bloated. DROP TABLE orders_2024 unlinks a file. Retention policies are the standard reason to partition.

Hash partitioning

Hash the key, take the remainder modulo the partition count, and store the row there. Rows spread evenly regardless of what the values look like.

This is the right choice when there is no natural ordering to exploit and the goal is to spread contention — many concurrent writers hitting one hot page, for instance.

The trade-off is that hash partitioning prunes only on equality. A query for one customer_id reaches one partition; a query for a *range* of customer ids reaches all of them, because hashing destroys order. Range partitioning is the opposite: it prunes ranges beautifully and can leave you with one very hot partition if recent data is where all the traffic goes.

RangeHash
Prunes equalityyesyes
Prunes rangesyesno
Even distributionnoyes
Drop old data instantlyyesno
Typical keytimestampid

Choosing the key

The partition key must appear in the WHERE clause of the queries you care about. This is the whole game, and it is where partitioning schemes go wrong.

Partition by order_date and then run reports by customer_id, and every report reads every partition — you have added complexity and gained nothing. Worse than nothing: each partition has its own indexes, so a query that scans them all does more index work than it would have on a single table.

A second constraint follows: a unique constraint has to include the partition key, because the engine cannot cheaply enforce uniqueness across pieces it is trying not to read.

Sizing

Too few partitions and pruning barely helps. Too many and planning cost grows, because the planner considers each one. Somewhere between a few dozen and a few hundred is the usual advice, with monthly partitions over a couple of years landing naturally in that band.

Partitions can also be subpartitioned — range by month, then hash by customer within each month — when both access patterns matter.

Where it goes wrong

Partitioning a small table. Under a few million rows, a good index is simpler and usually faster.

A partition key the queries do not filter on. No pruning, more index overhead.

Hash partitioning then querying ranges. Every partition, every time.

Forgetting the default partition. A row that matches no partition is an error unless a catch-all exists, and a data-loading job that fails at midnight on New Year's Day is the classic version of this.

Check yourself

0 of 3

Answer without scrolling back up.

  1. What is partition pruning?

  2. Why can hash partitioning not prune a range query?

  3. Why is time-series data almost always range-partitioned?

Cheat sheet

Partitioning by Range and Hash

Partitioning splits a table into physical pieces by a rule on one or more columns. Applications keep querying the single logical table; the engine decides which pieces are involved.

DATABASE · vizlearn.in/database/partitioning_in_databases.html

About the author

Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.