Grouping data with no labels at all - and the three assumptions it makes that decide whether the groups mean anything.
Overview
The algorithm
K-means is the most common clustering algorithm. It groups data points into k clusters by minimising the distance from each point to its cluster centre. You choose k in advance. The algorithm does the rest.
Clustering is unsupervised learning. There is no target column. The model finds structure in the features alone. That makes it useful for segmentation, anomaly detection and exploratory analysis. It also makes it harder to evaluate, because there is no ground truth to compare against.
K-means works best on data where the clusters are roughly spherical and roughly equal in size. It struggles with elongated clusters, clusters of very different densities, and clusters that overlap. Knowing these limits helps you decide whether k-means is the right tool or whether you need something like DBSCAN or Gaussian mixture models instead.
The algorithm is sensitive to feature scales. A column measured in thousands will dominate a column measured in single digits. Scaling before clustering is almost always necessary. It is also sensitive to the initial placement of the centres, which is why scikit-learn runs the algorithm several times with different starting positions and keeps the best result.
Choose k. Place k centres. Assign each point to its nearest centre, move each centre to the mean of the points assigned to it, and repeat until nothing moves.
That is the whole thing, and it settles quickly. inertia_ is what it cuts. The total squared distance from each point to its centre.
The interface is the familiar one with a piece missing. fit(X) takes no y. There's none. labels_ holds the group number for each training row, cluster_centers_ holds the centres. predict assigns new points to the nearest existing centre.
The labels are any. Cluster 0 in one run may be cluster 2 in the next, with the same grouping. Checking two clusterings so needs a metric that skips the numbering. This Is what adjusted_rand_score is for.
Worth knowing
Unsupervised: fit(X) with no y. The output is a group number per row, and the numbering is arbitrary.
You must choose n_clusters - the algorithm cannot discover how many groups there are.
inertia_ falls monotonically with k, so look for the bend; silhouette_score has an actual maximum.
It measures distance, so it must have scaled features - unscaled, it clusters whichever column has the largest units.
It assumes clusters are round, similar in size and similar in density; DBSCAN handles shapes it cannot.
The result depends on the random start, which is why n_init repeats it and keeps the best.
k-Means Clustering: A Practical Guide
each model so far was told the answers. This one is given features and asked to find structure - which is a truly different job. One whose results are much harder to check.
Fitting without a target
No y anywhere - the estimator is given features and asked to find groups.
example_01.pyscikit-learn
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
X, true_labels = make_blobs(n_samples=300, centers=3, cluster_std=1.0,
random_state=0)
km = KMeans(n_clusters=3, n_init=10, random_state=0).fit(X)
print("labels_ (first 12):", km.labels_[:12])
print("cluster_centers_:")
print(km.cluster_centers_.round(2))
print()
print("no y was passed - this is unsupervised. the labels are")
print("group numbers, and which group got which number is arbitrary.")
Output
Inertia cannot choose k for you
It falls forever, so its minimum is always the largest k you tried.
example_02.pyscikit-learn
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=400, centers=4, cluster_std=1.0, random_state=0)
print(" k inertia")
for k in range(1, 8):
km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
print("%3d %10.1f" % (k, km.inertia_))
print()
print("inertia always falls as k rises - at k = n it is zero.")
print("the bend is the signal, not the minimum.")
Output
Silhouette does have a maximum
Which makes it the more useful of the two for choosing k.
example_03.pyscikit-learn
from sklearn.cluster import KMeans
from sklearn.metrics import silhouette_score
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=400, centers=4, cluster_std=1.0, random_state=0)
print(" k silhouette")
best = None
for k in range(2, 8):
km = KMeans(n_clusters=k, n_init=10, random_state=0).fit(X)
s = silhouette_score(X, km.labels_)
print("%3d %11.4f" % (k, s))
if best is None or s > best[1]:
best = (k, s)
print()
print("highest silhouette at k =", best[0], "- and unlike inertia,")
print("this one has a maximum rather than always falling.")
Output
Without scaling it clusters the units
One noise column measured in a larger unit, and the grouping it finds is worthless.
example_04.pyscikit-learn
from sklearn.cluster import KMeans
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import adjusted_rand_score
import numpy as np
rng = np.random.RandomState(0)
y = np.array([0] * 150 + [1] * 150)
# Column 0 carries the grouping. Column 1 is noise, measured in a unit
# 500 times larger - so it dominates every distance.
signal = np.concatenate([rng.normal(0, 1, 150), rng.normal(5, 1, 150)])
noise = rng.normal(0, 1, 300) * 500.0
X = np.column_stack([signal, noise])
raw = KMeans(2, n_init=10, random_state=0).fit_predict(X)
scaled = KMeans(2, n_init=10, random_state=0).fit_predict(
StandardScaler().fit_transform(X))
print("agreement with the true grouping (0.0 is chance, 1.0 is perfect):")
print(" unscaled:", round(adjusted_rand_score(y, raw), 4))
print(" scaled :", round(adjusted_rand_score(y, scaled), 4))
print()
print("unscaled, the clustering is entirely decided by the noise column.")
Output
It only finds round, similar-sized groups
Two interleaving crescents are the shape it cannot see.
example_05.pyscikit-learn
from sklearn.cluster import KMeans, DBSCAN
from sklearn.datasets import make_moons
from sklearn.metrics import adjusted_rand_score
X, y = make_moons(n_samples=400, noise=0.06, random_state=0)
km = KMeans(2, n_init=10, random_state=0).fit_predict(X)
db = DBSCAN(eps=0.2, min_samples=5).fit_predict(X)
print("agreement with the true crescents:")
print(" KMeans:", round(adjusted_rand_score(y, km), 4))
print(" DBSCAN:", round(adjusted_rand_score(y, db), 4))
print()
print("k-means draws straight boundaries around round blobs, so two")
print("interleaving crescents are exactly what it cannot do.")
Output
The starting point matters
Eight random starts, and they do not all land in the same place.
example_06.pyscikit-learn
from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
X, _ = make_blobs(n_samples=300, centers=5, cluster_std=1.4, random_state=2)
inertias = []
for seed in range(8):
km = KMeans(5, n_init=1, init="random", random_state=seed).fit(X)
inertias.append(km.inertia_)
print("n_init=1, eight seeds:", [round(v, 1) for v in inertias])
print(" best %.1f, worst %.1f" % (min(inertias), max(inertias)))
good = KMeans(5, n_init=10, random_state=0).fit(X)
print()
print("n_init=10 (the default) keeps the best of ten:", round(good.inertia_, 1))
Output
Choosing k is your problem
The method can't tell you how many groups exist. It will happily split one blob into five or merge five into two.
inertia_ looks like it should help and can't. It falls monotonically. More centres always means points are closer to a centre. At k = n it's zero. The second editor shows the curve dropping from 4219 to 469 with no min in sight. The elbow is the heuristic. The point where the improvement flattens. And on this data the drops are 2213, 831, 472, 89, 78, 67. This Bends after 4.
silhouette_score is the better tool because it has a genuine max. For each point it checks the distance to its own cluster against the distance to the nearest other cluster. Means. Values run from -1 to 1. Higher means better-separated groups. The third editor peaks at k=4. This Is the number of blobs the data was made with.
Neither is authoritative. Both are heuristics on an ill-posed question. The honest answer often comes from outside. How many segments the business can act on, how many categories the taxonomy allows, what the groups turn out to mean when you look at them.
Scale it, or cluster the units
k-means is fully distance-based, so an unscaled feature with large units decides all.
The fourth editor makes this stark. One column carries a real grouping. The other is pure noise measured in a unit 500 times larger. Unscaled, the match with the true grouping is -0.0032. Chance. Scaled, it's 0.9867.
The clustering wasn't merely worse. It was fully found by a column holding nothing. This is the same arg the scaling module made for k-NN. It applies with more force here because there's no label to notice the problem with. A guided model that clusters on noise scores badly and tells you. An unsupervised one makes groups. The groups look like a result.
StandardScaler in a pipeline before KMeans should be auto.
What it assumes
Three assumptions, all baked into "assign each point to the nearest centre".
Clusters are round. The boundary between two centres is a straight line. Each cluster is a convex region. The fifth editor puts this against two interleaving crescents. K-means scores 0.27 and DBSCAN scores 1.00. No amount of tuning fixes it. The shape is outside what the method can express.
Clusters are similar in size. k-means tends to split a large cluster and merge small ones. That reduces total squared distance.
Clusters have similar density. A tight group and a diffuse one confuse the distance check.
When these hold, k-means is fast, simple and hard to beat. When they don't, the options are DBSCAN for any shapes and outliers, GaussianMixture for elliptical clusters of different sizes with soft sets. AgglomerativeClustering when a hierarchy is more handy than a flat split.
The random start
The centres are was reset at random, and the method settles to a local min. So different starts give different answers.
The last editor runs eight single-start fits with random init and gets inertias from 1034.9 to 1188.6. Three of the eight found the good answer and five didn't.
n_init runs the whole thing a few times and keeps the lowest inertia. This Is why the default is 10 rather than 1. init="k-means++", also the default, spreads the initial centres out rather than placing them at random. This Makes good answers much more likely and is the reason the default configuration mostly avoids this.
Both defaults are sensible and both are worth knowing about. A clustering that changes between runs is often one of these two being overridden.
What the groups mean
Nothing, until you look.
k-means returns a split whether or not there's structure to find. Ask for four clusters in uniform noise and you get four. So the first question after fitting is always whether the groups are real. The second is what sets apart them.
The hands-on checks are checking the silhouette against what random data of the same shape scores, looking at the cluster centres in the first units to see what characterises each group. Checking the sizes for a cluster holding three rows.
The step that gives clustering its value is naming the groups. showing each in terms someone can act on. A split with no reading is a column of ints. The method can't supply the reading.
Judging a clustering when there are no labels
Guided models are checked against the truth. Clustering has none. The metrics split into two families and it's worth knowing which you're allowed to use.
inner measures use only the data and the labels the method made. silhouette_score checks within-cluster distance against nearest-other-cluster distance. calinski_harabasz_score and davies_bouldin_score are options with different biases. All three reward compact, well-separated groups. Which means they systematically favour just the round, equal-sized clusters that k-means makes. Would score a correct DBSCAN clustering of two crescents poorly. They measure the shape you asked for, not whether the grouping is real.
External measures compare against known labels. adjusted_rand_score and normalized_mutual_info_score both skip the any numbering. This Is what makes them usable at all. Adjusted Rand is adjusted for chance. 0.0 is what random labelling scores and minus values are worse than random. Which is how the unscaled run above landed at -0.0032.
These are for the situation this track has been using throughout. Made data where the truth is known, or a subset somebody has labelled. In genuine unsupervised work there's nothing to compare against. The honest test is whether the groups turn out to be handy.
Where clustering earns its place
Three uses, of which only one is what people often mean by clustering.
Segmentation. Finding groups of customers, docs or sessions that behave alike. That each can be treated differently. This is the familiar use. It's the one where the reading matters more than the method. A split no one can name isn't actionable.
Compression and feature build. The cluster label turns a cat feature for a guided model, or the distances to each centre become numeric ones. This often helps. It's subject to the usual rule. The clustering must be fitted inside the folds, or the labels carry info from the test rows.
Finding what isn't in any group. Points far from each centre are unusual. This Makes clustering a crude outlier finder. DBSCAN does this by name by labelling outliers -1 rather than forcing them into a cluster. This Is one of its real edges over k-means.
The use to be careful with is treating clusters as though they were found categories. The method splits what it's given. Whether the split matches to any of it real is a separate question that the output can't answer.
Can I use k-means on cat data? Not sensibly - the mean of one-hot columns isn't a category. KModes outside scikit-learn, or a distance-based method with a cat metric.
What about many rows?MiniBatchKMeans fits on small random batches and is sharply faster on large data, at a small cost in quality.
Why does my clustering change between runs? Either n_init=1 or no random_state. The defaults handle the first. Set the second for reproducibility.
Should the clustering go in a pipeline? Yes, if its output feeds a guided model - or else the cluster labels are learned from the test rows too.
Reading the clusters once you have them
Fitting takes one line. Understanding the result takes rather more. Skipping it's how clustering projects end with a column of ints no one uses.
Look at the centres in the first units.cluster_centers_ is in what space the model was fitted in. This After scaling is standard deviations rather than pounds or minutes. scaler.inverse_transform(km.cluster_centers_) puts them back. Reading the result column by column is what turns cluster 2 into "high spend, low rate, recent".
Check the sizes.np.bincount(km.labels_) in one line. A cluster with four members is often not a segment. It's an outlier group that k-means was forced to place a place because you asked for that many centres.
Compare each cluster against the total mean. Which features are most different from the global mean is a faster route to a description than reading the raw centres, above all with many columns.
Check steadiness. Refit on a random 80% and see whether the same groups appear. Clusters that dissolve when a fifth of the data changes are artefacts of the split rather than structure in the data. This test costs one loop.
The output of that work is a name and a description per group. That is the deliverable. The labels are an mid.
Is there a way to have k chosen automatically? Not by k-means. DBSCAN and HDBSCAN find the number of groups from the density instead. This Is a genuine edge when you've no prior view.
Does predict() work on new data? Yes - it assigns each new point to the nearest existing centre without refitting. This Is what makes a clustering usable as a feature in production.
Why is my silhouette score low though the clusters look right? Silhouette rewards compact, well-separated, roughly spherical groups. A correct clustering of elongated or touching groups scores poorly on it. This Is a limitation of the metric rather than of the clustering.
Does k-means work in high dims? Less well. Distances between points become more similar as dims grow. "nearest centre" carries less info. Reducing dims first - with PCA, say - is common practice and often helps.
Can two clusters end up empty? With random init, yes. Scikit-learn relocates empty clusters rather than leaving them. This Is one more thing k-means++ makes unlikely.
Things to try with k-Means Clustering
Watch inertia fail. The second editor's numbers fall forever. Find the differences and find the bend yourself.
Break it with scale. The fourth editor's unscaled run scores below chance. Change 500 to 5 and see where the crossover is.
Try the wrong shape. In the fifth editor, raise noise to 0.15 and see whether DBSCAN still manages.
Cluster noise. Fit k=4 on np.random.rand(300, 2) and look at the silhouette. It won't be zero.
Questions people ask about k-Means Clustering
How do I choose k? Try several values and compare the elbow in inertia and the silhouette score. Also ask which k gives groups you can use.
Do I need to scale for KMeans? Yes. It uses Euclidean distance, so a feature with a large range will dominate the clusters.
Why do results change between runs? The starting centroids are random. Set random_state and keep n_init above 1 for a steadier result.
What shapes does KMeans assume? Roughly round, similarly sized blobs. For odd shapes or noise, try DBSCAN or a Gaussian mixture.
Can I check the clusters without labels? Use the silhouette score and inspect cluster profiles. Stability across different seeds is also a useful sign.
Can KMeans find outliers? Not directly. Outliers drag centroids, so remove or handle them first.
Where this leaves you
Scale first, choose k with silhouette rather than inertia, keep n_init at its default. Check that the groups mean a thing before reporting them. When the clusters aren't round, the method is the wrong one rather than badly tuned.
Check yourself
0 of 4
Answer without scrolling back up.
Why can inertia_ not be used to choose k?
Its minimum is always the largest k you tried. The elbow is a heuristic on the curve; silhouette has an actual maximum.
What happens if you run k-means on unscaled features?
In the editor, a pure-noise column with a 500x larger unit took the agreement with the true grouping to -0.0032, which is chance.
Why does k-means fail on two interleaving crescents?
Assigning each point to the nearest centre gives straight boundaries. DBSCAN scored 1.00 where k-means scored 0.27.
What does n_init do?
k-means converges to a local minimum, and eight single starts in the editor ranged from 1034.9 to 1188.6.
Cheat sheet
k-Means Clustering
Group data points into clusters with k-means, measure fit with inertia and silhouette score, and see how the algorithm converges.
Ashish Jangra builds and maintains VizLearn. Every module here is written and the visualisation behind it hand-built, so the numbers in a readout come from the same code that draws the picture. Corrections are genuinely welcome and get priority over everything else — if a page states something wrong, or an animation misrepresents what the algorithm does, get in touch.