1. CSE 4705 Artificial Intelligence
Jinbo Bi
Department of Computer Science & Engineering

2. Machine learning (2) Cluster analysis – hierarchical clustering

3. Types of hierarchical clustering
Two main types of hierarchical clustering
Agglomerative:
Start with the points as individual clusters
At each step, merge the closest pair of clusters until only one cluster (or k clusters) left
Divisive:
Start with one, all-inclusive cluster
At each step, split a cluster until each cluster contains a point (or there are k clusters)
Traditional hierarchical algorithms use a similarity or distance matrix
Merge or split one cluster at a time

4. Divisive hierarchical clustering
Start with all points in a single cluster
Create subsequent clusters by splitting some clusters
Find a cluster and bi-section that cluster
Build a spanning tree which gives a tree structure that links all data points (a connected component that has all points)
Then find an edge in the tree to break so a connected component is broken into two connected components

5. Divisive hierarchical clustering
Build MST (Minimum Spanning Tree)
Start with a tree that consists of any point
In successive steps, look for the closest pair of points (p, q) such that one point (p) is in the current tree but the other (q) is not
Add q to the tree and put an edge between p and q

6. Divisive hierarchical clustering
Use MST for constructing hierarchy of clusters

7. Divisive hierarchical clustering
Example X1 X2 X3 X4 X5

8. Divisive hierarchical clustering
Example X1 X2 X3 X4 X5

9. Divisive hierarchical clustering
Example X1 X2 X3 X4 X5

10. Divisive hierarchical clustering
Example X1 X2 X3 X4 X5

11. Divisive hierarchical clustering
Example X1 X2 X3 X4 X5

12. Density-based clustering (DBSCAN)
DBSCAN is a density-based algorithm.
Density = number of points within a specified radius (Eps)
A point is a core point if it has more than a specified number of points (MinPts) within Eps
These are points that are at the interior of a cluster
A border point has equal or fewer than MinPts within Eps, but is in the neighborhood of a core point
A noise point is any point that is not a core point or a border point.

13. Density-based clustering (DBSCAN)
DBSCAN is a density-based algorithm.
Density = number of points within a specified radius (Eps)
A point is a core point if it has more than a specified number of points (MinPts) within Eps
These are points that are at the interior of a cluster
A border point has equal or fewer than MinPts within Eps, but is in the neighborhood of a core point
A noise point is any point that is not a core point or a border point.

14. DBSCAN: core, border, and noise points

15. DBSCAN algorithm Eliminate noise points
Perform clustering on the remaining points

16. DBSCAN algorithm Eliminate noise points
Perform clustering on the remaining points

17. DBSCAN (more examples)
Original Points
Point types: core, border and noise
Eps = 10, MinPts = 4

18. When DBSCAN works well Original Points Clusters Resistant to Noise
Can handle clusters of different shapes and sizes

19. When DBSCAN does not work well
(MinPts=4, Eps=9.75).
Original Points
Varying densities
High-dimensional data
(MinPts=4, Eps=9.92)

20. When DBSCAN does not work well
(MinPts=4, Eps=9.75).
Original Points
Varying densities
High-dimensional data
(MinPts=4, Eps=9.92)

21. DBSCAN: determining EPS and MinPts
Idea is that for points in a cluster, their kth nearest neighbors are at roughly the same distance
Noise points have the kth nearest neighbor at farther distance
So, plot sorted distance of every point to its kth nearest neighbor

22. DBSCAN: in-class practice
EPS
MinPts = 3

23. Questions?