Unsupervised Classification CHAPTER 14 CLASSIFICATION Clustering and Unsupervised Classification A. Dermanis

Clustering = dividing of N pixels into K classes ω1, ω2, …, ωK scatter matrix of class ωi : mean of class ωi :  xi Si = (x – mi)(x – mi)T mi = x  xi 1 ni covariance matrix of class ωi : Ci = Si 1 ni total scatter matrix: global mean   i xi ST = (x – mi)(x – mi)T m = x 1 N   i xi total covariance matrix: CT = ST 1 N A. Dermanis

     Sin = Si = (x – mi)(x – mi)T Sex = ni (mi – m)(mi – m)T Clustering criteria overall compactness of the clusters  internal scatter matrix   i xi Sin = Si = (x – mi)(x – mi)T  i degree of distinction between the clusters  external scatter matrix Sex = ni (mi – m)(mi – m)T  i ST = Sin + Sex = constant Optimal algorithm: Sin = min and Sex = max (simultaneously) Problem: How many clusters ? (K = ?) Extreme choice: K = N (each pixel a different class) k = {xk} mk = xk, Sk = 0, Sin = Sk = 0 = min, Sex = ST =max  k Extreme choice: K = 1 (all pixels in a single class) Sin = ST, Sex = 0 A. Dermanis

Hierarchical Clustering 1 2 3 4 5 6 A Agglomerative clustering: Unifying at each step the two closest clusters B AGGLOMERATIVE DIVISIVE C Divisive clustering : Dividing at each step the most disperse cluster into two new clusters D E F Needed: Unification criteria. Division criteria and procedures. A. Dermanis

Hierarchical Clustering 1 2 3 4 5 6 1 2 3 4 5 6 A B AGGLOMERATIVE DIVISIVE C D E F A B C D E F A. Dermanis

Distance between two clusters (alternatives): mean distance: minimum distance: maximum distance: Used in agglomerative and divisive clustering A. Dermanis

The K-means or migrating means algorithm 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 A. Dermanis

The K-means or migrating means algorithm 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Step 0: Selection of K = 3 pixels as initial positions of means A. Dermanis

The K-means or migrating means algorithm 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Step 1: Assignment each pixels to the cluster of its closest mean Calculation of the new means for each cluster A. Dermanis

The K-means or migrating means algorithm 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Step 2: Assignment each pixels to the cluster of its closest mean Calculation of the new means for each cluster A. Dermanis

The K-means or migrating means algorithm 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Step 3: Assignment each pixels to the cluster of its closest mean Calculation of the new means for each cluster A. Dermanis

The K-means or migrating means algorithm 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 Step 4: Assignment each pixels to the cluster of its closest mean All pixels remain in the same cluster. Means remain the same. Termination of the algorithm ! A. Dermanis

A variant of the K means algorithm. The Isodata Algorithm A variant of the K means algorithm. In each step one of 3 additional procedures can be used: 1. Cluster ELIMINATION Eliminate clusters with very few pixels 2. Cluster UNIFICATION Unify pairs of clusters Very close to each other 3. Cluster DIVISION Divide large clusters which are elongated Into two clusters A. Dermanis

The Isodata Algorithm 1. Cluster ELIMINATION Eliminate clusters with very few pixels A. Dermanis

The Isodata Algorithm 2. Cluster UNIFICATION Unify pairs of clusters Very close to each other A. Dermanis

The Isodata Algorithm 3. Cluster DIVISION Divide large clusters which are elongated Into two clusters A. Dermanis

m2+kσ2 m2 m2–kσ2 m1 The Isodata Algorithm The unification process The division process m2+kσ2 m2–kσ2 m2 m1 A. Dermanis

Examples of classifiction using the K-mean algorithm K-means: 3 classes K-means: 5 classes K-means: 7 classes K-means: 9 classes A. Dermanis

Examples of classifiction using the ISODATA algorithm ISODATA : 3 classes ISODATA : 5 classes ISODATA : 7 classes ISODATA : 9 classes A. Dermanis