Hierarchical Clustering

This unlocks the entire path to unsupervised learning and beyond. Once you can build a tree of clusters, you can explore any dataset without knowing ahead of time how many groups it has. You can then move on to k-means, DBSCAN, and dimensionality reduction — and later see how the same tree-building idea powers the embeddings that make retrieval-augmented generation and agentic systems work. You will also be able to read papers and documentation that casually say 'we used hierarchical clustering to group the data', and know exactly what they did.

Think of a scattered collection of points on a map. Hierarchical clustering starts by treating each point as its own tiny cluster, then repeatedly finds the two clusters that are closest together and merges them. Each merge creates a larger cluster. Repeat until everything is one big cluster. The result is a tree, called a dendrogram, that shows every merge from the bottom (all singletons) to the top (one cluster). The tree is the product, not a flat list of clusters. You can 'cut' the tree at any height to get a clustering with any number of clusters you want: cut low (near the bottom) and you get many small clusters; cut high and you get a few big ones. You never need to decide the number of clusters in advance — you decide after seeing the tree. The same idea works the other way, too: you could start with one big cluster and split it recursively, but the merging version is the one most commonly used.

You are organising a large extended family for a reunion. You begin with every person as their own separate family unit. Then you look for the two people who are closest — say, two siblings who both live in the same town — and you group them. Now you have a new unit: that sibling pair. Next, you find the closest pair among all your units, which might be the sibling pair and a cousin who lives nearby, so you merge again. Over many steps, you merge the closest units until the whole extended family is one tree. The key: you never said 'I want five families' at the start. You just kept merging the closest. The resulting family tree — a dendrogram — lets you decide later, after seeing the whole structure, that you want to group at the level of immediate families, or at the level of the whole clan. Now, the analogy breaks down: in real families, 'closeness' might not be geographic, and merging is not always symmetric — a family might be closer to another in one direction but not the other. Also, when you merge, you lose information about the individual family units — you have to decide how to measure the distance between merged groups, which is the 'linkage' choice. And critically, the merging order depends entirely on how you define distance between clusters, which can change the shape of the tree. The family tree is fixed by blood; the dendrogram is fixed by your distance measurement.

Hierarchical clustering builds a tree of nested clusters by iteratively merging (or splitting) the two clusters with the smallest distance between them, using a linkage criterion to define cluster-to-cluster distance, and visualises this process as a dendrogram.

This is one of the three major families of clustering. From your library: k-means partitions the data into k groups, but you have to fix k in advance, and it assumes spherical, similar-sized clusters. Hierarchical clustering drops both of those assumptions — you get a tree, and you can cut it at any level. DBSCAN and OPTICS, which you have notes on, drop k-means' assumptions entirely and label noisy points as belonging to no cluster; hierarchical clustering instead places every point in a nested structure, with no point labelled as noise. Dimensionality reduction (PCA) compresses many features into a few; hierarchical clustering organises the points themselves. This sits under the Unsupervised Learning parent concept, where there is no target to predict and validation is harder because there is no ground truth.