Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology A novel genetic algorithm for automatic clustering Advisor ： Dr. Hsu Graduate ： Sheng-Hsuan Wang Authors ： Gautam Garai B.B. Chaudhuri Department of Information Management Pattern Recognition Letters 25 (2004)

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Outline Motivation Objective Introduction Basic concept of Classical Genetic Algorithm Clustering with Genetic Algorithm Experimental results Discussion and conclusion Personal opinions Review

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Motivation Some problems of the clustering. Automatic clustering. If one cluster is confined fully or partly within another cluster. If clusters are present in noisy data.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Objective A new genetically guided algorithm for solving the clustering problem, which have two-phase process. Cluster Decomposition Algorithm (CDA). Hierarchical Cluster Merging Algorithm (HCMA). Adjacent Cluster Checking Algorithm (ACCA).

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Introduction These clustering methods can broadly be classified into two categories: Hierarchical agglomerative divisive Non-hierarchical k-means

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Introduction Some researchers have used GA based on split-and- merge method in defining clusters. Tseng and Yang (2001). Other algorithms: DBScan CURE Chameleon

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Introduction Genetically based Clustering Algorithm (GCA) which is basically a two-stage split-and-merge algorithm for finding the clusters. Splitting of clusters with CDA. Cluster merging with HCMA. Adjacency checking between two fragmented clusters with ACCA.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Basic concept of Classical Genetic Algorithm Encoding schemas Fitness evaluation Testing the end of the algorithm Parent selection Crossover operators Mutation operators NO Halt YES

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Clustering with Genetic Algorithm n vectors X = {x 1, x 2, …, x n } to be clustered into k groups. The clustering approach has two steps Cluster Decomposition Algorithm (CDA). Hierarchical Cluster Merging Algorithm (HCMA).

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Splitting of clusters with CDA First decomposes the entire data set into m groups of clusters.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. The progress of the CDA process Step 1. For each object x i, find the nearest neighbor x j. Step 2. Compute d av.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. The progress of the CDA process Step 3. Consider x i as the center of a circular region with radius r. Step 4. Set p = 1. Step 5. Extract B p and modify the data set X such that X = |X - B p |. Step 6. Terminate the algorithm if. Otherwise, p = p + 1 and go to step 5.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Cluster merging with HCMA The second stage to merge the fragmented clusters, B i PiPi m … u

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Cluster merging with HCMA The algorithm, HCMA consists of all three phases of CGA. P a and P b are chosen randomly from the pool of individuals. Corssover probability,, using single point corssover operation. Adaptive mutation probability.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Cluster merging with HCMA (example) B1B1 m0m0 pipi Merge until B 0 is null B0B0 m’ CiCi

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Cluster merging with HCMA Let the seed of the fragmented cluster B i be. The center S j of each C j : The fitness function,.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Adjacency checking between two fragmented clusters The ACCA is used along with HCMA if One cluster is confined fully or partly within another cluster. Clusters are present in noisy data. The ACCA uses two thresholds for deciding merging of pair of clusters. : The threshold of boundary points. : The threshold of data density difference.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. The progress of the ACCA process Step 1. Define suitably the value of the radius. Step 2. Select two fragmented clusters,, which satisfy the merging condition. Step 3. Count the number of boundary points of which resides within radius r’. Let it be N b and the object density of be. Step 4. If then are adjacent to each other. Step 5. Terminate the algorithm.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Experimental results Parameter setting Population size, = 50. The number of clusters, m, is inversely proportional to the value of r. 2 <= u <= 4. k is pre-specified by the user. Crossover probability Initial mutation probability G max =100 times in each cycle, 30 runs. T b =4 ; T d =0.4

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Cluster partitioning in R 2 feature space

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Cluster partitioning in R 2 feature space

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Cluster partitioning in R 2 feature space The noise is represented as the third cluster.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Cluster separation in Iris data 4-D Iris dataset.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Discussion and conclusion GCA is composed of two algorithms CDA HCMA After several GA cycles when k clusters are found. Identify clusters accurately (ACCA) Either partly or fully enclosed by another cluster. Noise.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Personal Opinions It may be used in SOM 2-D map to automatic clustering.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. Review Using GCA to automatic clustering. Split : CDA Merge : HCMA + ACCA