K-Means Clustering Explained: Finding Groups in Unlabelled Data (with Python)

What you'll learn

Everything so far has been supervised — we had the right answers to learn from. K-Means is unsupervised: it finds structure in data that has no labels at all. Give it customer purchase histories and it discovers segments; give it the pixels of an image and it finds the dominant colours.

• How the K-Means assign-and-update loop works
• How to choose the number of clusters, k
• Why scaling and smart initialisation (k-means++) matter
• Where K-Means breaks and what to use instead

The intuition in plain English

You want to sort a pile of dots into k groups. K-Means does it by trial and refinement: drop k 'group centres' (centroids) at random, assign every dot to its nearest centre, then move each centre to the middle of the dots assigned to it. Repeat, and the centres slide into the heart of natural clusters and stop moving.

The algorithm, step by step

1. Choose k and place k centroids (ideally with k-means++, see below).
2. Assign: attach each point to its nearest centroid (Euclidean distance).
3. Update: move each centroid to the mean of the points assigned to it.
4. Repeat assign + update until the assignments stop changing (convergence).

What it is really minimising is inertia — the total squared distance from each point to its centroid:

inertia = Σ (distance from point to its centroid)²

Lower inertia means tighter, more compact clusters.

Choosing k: the elbow and silhouette methods

K-Means cannot tell you how many clusters exist — you must choose k. Two standard tools:

• Elbow method: plot inertia against k. Inertia always falls as k rises, but there is usually a 'kink' where the gains flatten. That elbow is a good k.
• Silhouette score: measures how well each point sits in its cluster versus the next-nearest one, from -1 to 1. Pick the k with the highest average silhouette.

Implementing it in Python

We generate four blobs and let both metrics find k:

from sklearn.cluster import KMeans
from sklearn.datasets import make_blobs
from sklearn.metrics import silhouette_score

X, _ = make_blobs(n_samples=500, centers=4, cluster_std=0.8, random_state=42)

# The elbow + silhouette method to choose k.
for k in range(2, 7):
    km = KMeans(n_clusters=k, n_init=10, random_state=42).fit(X)
    print(f"k={k}: inertia={km.inertia_:.0f}, silhouette={silhouette_score(X, km.labels_):.3f}")

Output:

k=2: inertia=15386, silhouette=0.607
k=3: inertia=3125,  silhouette=0.782
k=4: inertia=607,   silhouette=0.834
k=5: inertia=548,   silhouette=0.700
k=6: inertia=492,   silhouette=0.564

Inertia keeps dropping, but the silhouette peaks sharply at k=4 — exactly how many blobs we created. That agreement is the signal you want. In real data the answer is rarely this clean.

Why initialisation matters: k-means++

K-Means can converge to a poor solution if the initial centroids are unlucky (e.g. two in the same real cluster). k-means++ spreads the initial centroids out intelligently, which gives faster, more reliable convergence. scikit-learn uses it by default, and n_init=10 runs the whole thing several times and keeps the best result — always leave that on.

Preparing your data

• Scale your features. Like KNN, K-Means is distance-based, so features on larger scales dominate. Standardise first.
• K-Means assumes round, similar-sized clusters. It will force elongated or nested shapes into blobs, often wrongly.
• It is sensitive to outliers because centroids are means; consider removing them or using K-Medoids.
• Reduce dimensions (e.g. PCA) first on very wide data.

Limitations and alternatives

K-Means is fast and scalable but rigid. When it disappoints, reach for:

• DBSCAN — finds arbitrarily shaped clusters and labels outliers as noise; you do not pre-specify k.
• Gaussian Mixture Models — soft, elliptical clusters with probabilities of membership.
• Hierarchical clustering — builds a tree of clusters (a dendrogram) you can cut at any level.

Key takeaways

• K-Means groups unlabelled data by iteratively assigning points to the nearest centroid and recomputing centroids.
• It minimises within-cluster squared distance (inertia).
• Choose k with the elbow and silhouette methods; use k-means++ and n_init for stable results.
• Scale features, and switch to DBSCAN or GMMs when clusters are non-spherical or noisy.

Further reading: scikit-learn clustering and the visual guide.