Intelligent Database Systems Lab 國立雲林科技大學 National Yunlin University of Science and Technology 1 A survey of kernel and spectral methods for clustering Author: Maurizio Filippone, Francesco Camastra Francesco Masulli, and StefanoRovetta Reporter: Wen-Cheng Tsai 2007/10/17 PR (PATTERN RECOGNITION), 2008

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 2 Outline  Motivation  Objective  Method ─ Kernel clustering methods ─ Batch K-means VS. Kernel K-means ─ SOM VS. Kernel SOM ─ Spectral clustering ─ Kernel clustering methods objective ─ Spectral clustering methods objective ─ A unified view of the two approaches  Conclusion  Comments

Intelligent Database Systems Lab N.Y.U.S.T. I. M.  The focus of this paper is the partitioning clustering problem with an interest in two recent approaches. 3 Motivation ` Batch K-means, SOM, Fuzzy clustering (c-means), Possibilistic clustering (c-means). Kernel clusteringSpectral clustering Partitioning clustering

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 4 Objective  The aim of this paper is to present a survey of kernel and spectral clustering methods that they can produce nonlinear separating hypersurfaces between clusters.  an explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation. ` Batch K-means, SOM, Fuzzy clustering (c-means), Possibilistic clustering (c-means). Kernel clusteringSpectral clustering Partitioning clustering

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 5 Kernel clustering methods  One of the most relevant aspects in application is that it is possible to compute Euclidean distances in F without knowing explicitly Φ. This can be done using the so called distance kernel trick. High dimension using the distance kernel trick

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 6 Batch K-means

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 7 Kernel K-means

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 8 SOM

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 9 Kernel SOM

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 10 Spectral clustering High complexity High computed cost Dimensionality reduction Low computed cost Reduction dimension using the Laplacian matrix

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 11 ` Batch K-means, SOM, Fuzzy clustering (c-means), Possibilistic clustering (c-means). Kernel clusteringSpectral clustering Partitioning clustering

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 12 Kernel clustering methods objective  We introduce the weighted version of the kernel K-means.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 13 Spectral clustering methods objective  Definition of association between two sets of edges S and T of a weighted graph is the following ： High complexity High computed cost Dimensionality reduction Low computed cost

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 14 A unified view of the two approaches

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 15 Conclusion  An explicit proof of the fact that these two paradigms have the same objective is reported since it has been proven that these two seemingly different approaches have the same mathematical foundation.

Intelligent Database Systems Lab N.Y.U.S.T. I. M. 16 Comments  Advantages ─ Kernel and spectral methods of the same mathematical foundation. ─ Deal with the nonlinear separating hypersurfaces.  Disadvantages ─ In real world of the high computational cost.  Application ─ All kind of the partitioning clustering.