K-Means is one of the most popular and intuitive unsupervised learning algorithms, designed to find groups (or "clusters") in a dataset. With unsupervised learning, we don't have predefined labels; the algorithm's job is to discover the underlying structure on its own. This lab lets you step through the K-Means algorithm to see exactly how it partitions data into a specified number of clusters (K).
The Two-Step Dance of K-Means
The algorithm is surprisingly simple and elegant. It iteratively repeats two main steps until the cluster assignments no longer change.
This two-step process continues until the centroids stop moving, which means the assignments are stable. At this point, the algorithm has converged.
Guided Experiments
Use the controls to see how K-Means behaves under different conditions.
-
Step-by-Step Execution:
Start with the default "Gaussian Blobs" dataset. Click the "Reset Positions" button to see the randomly placed initial centroids. Now, click the "Step Algorithm" button. You'll see the points get assigned colors (Assignment Step). Click it again, and the centroids will move to the center of their new clusters (Update Step). Keep stepping to see the process unfold until the "CONVERGED" message appears.
-
The Importance of 'K':
With the "Gaussian Blobs" dataset, the data is clearly grouped into 5 clusters. Set the "Clusters (K)" slider to 5 and click "Auto-Run". The algorithm should find the clusters perfectly. Now, set K to 3 and reset. Run it again. Notice how the algorithm is forced to group distinct blobs together, resulting in less meaningful clusters. This highlights that choosing the right K is crucial.
-
When K-Means Fails:
K-Means works best on well-separated, spherical clusters. To see its limitations, select the "Nested Moons" or "Concentric Circles" dataset from the "Initial Layout" dropdown. Run the algorithm. You'll see that K-Means fails to identify these non-convex shapes because it can only create linear boundaries between clusters. This is a key weakness of the algorithm.
-
Understanding Inertia (WCSS):
The "WCSS (Inertia)" metric measures how tightly packed the clusters are. It's the sum of squared distances of samples to their closest cluster center. With the "Gaussian Blobs" and K=5, run the algorithm and note the final WCSS. Now, increase K to 8 and run it again. The WCSS will be lower. While a lower WCSS is generally better, it always decreases as K increases, so it can't be the only metric for choosing K (this is the motivation behind methods like the "Elbow Method").
Key Takeaways
- Sensitivity to Initialization: Since the initial centroids are placed randomly, you might get different final clusters if you run the algorithm multiple times. This is why in practice, K-Means is often run several times with different random initializations.
- Speed and Scalability: K-Means is very fast and computationally efficient, making it an excellent choice for large datasets.
- Assumptions: The algorithm assumes that clusters are spherical, of similar size, and have similar density. When these assumptions are violated (like with the moons or circles), it performs poorly.