A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts

Slides:

Advertisements

Similar presentations

Consistent Bipartite Graph Co-Partitioning for High-Order Heterogeneous Co-Clustering Tie-Yan Liu WSM Group, Microsoft Research Asia Joint work.

Advertisements

Partitional Algorithms to Detect Complex Clusters

Principal Component Analysis Based on L1-Norm Maximization Nojun Kwak IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008.

O(N 1.5 ) divide-and-conquer technique for Minimum Spanning Tree problem Step 1: Divide the graph into  N sub-graph by clustering. Step 2: Solve each.

Fast SDP Relaxations of Graph Cut Clustering, Transduction, and Other Combinatorial Problems (JMLR 2006) Tijl De Bie and Nello Cristianini Presented by.

The Stability of a Good Clustering Marina Meila University of Washington

Carolina Galleguillos, Brian McFee, Serge Belongie, Gert Lanckriet Computer Science and Engineering Department Electrical and Computer Engineering Department.

Distance Metric Learning with Spectral Clustering By Sheil Kumar.

Normalized Cuts and Image Segmentation

Online Social Networks and Media. Graph partitioning The general problem – Input: a graph G=(V,E) edge (u,v) denotes similarity between u and v weighted.

Clustering II CMPUT 466/551 Nilanjan Ray. Mean-shift Clustering Will show slides from:

10/11/2001Random walks and spectral segmentation1 CSE 291 Fall 2001 Marina Meila and Jianbo Shi: Learning Segmentation by Random Walks/A Random Walks View.

Analysis of Network Diffusion and Distributed Network Algorithms Rajmohan Rajaraman Northeastern University, Boston May 2012 Chennai Network Optimization.

One-Shot Multi-Set Non-rigid Feature-Spatial Matching

Determinants Bases, linear Indep., etc Gram-Schmidt Eigenvalue and Eigenvectors Misc

Lecture 21: Spectral Clustering

Learning in Spectral Clustering Susan Shortreed Department of Statistics University of Washington joint work with Marina Meilă.

A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan, and Brian Kulis.

Normalized Cuts and Image Segmentation Jianbo Shi and Jitendra Malik, Presented by: Alireza Tavakkoli.

Region Segmentation. Find sets of pixels, such that All pixels in region i satisfy some constraint of similarity.

Unsupervised Learning of Categories from Sets of Partially Matching Image Features Dominic Rizzo and Giota Stratou.

Normalized Cuts Demo Original Implementation from: Jianbo Shi Jitendra Malik Presented by: Joseph Djugash.

Prénom Nom Document Analysis: Data Analysis and Clustering Prof. Rolf Ingold, University of Fribourg Master course, spring semester 2008.

Segmentation Graph-Theoretic Clustering.

Clustering In Large Graphs And Matrices Petros Drineas, Alan Frieze, Ravi Kannan, Santosh Vempala, V. Vinay Presented by Eric Anderson.

Lecture#6: segmentation Anat Levin Introduction to Computer Vision Class Fall 2009 Department of Computer Science and App math, Weizmann Institute of Science.

Clustering with Bregman Divergences Arindam Banerjee, Srujana Merugu, Inderjit S. Dhillon, Joydeep Ghosh Presented by Rohit Gupta CSci 8980: Machine Learning.

Application of Graph Theory to OO Software Engineering Alexander Chatzigeorgiou, Nikolaos Tsantalis, George Stephanides Department of Applied Informatics.

Diffusion Maps and Spectral Clustering

Graph-based consensus clustering for class discovery from gene expression data Zhiwen Yum, Hau-San Wong and Hongqiang Wang Bioinformatics, 2007.

Domain decomposition in parallel computing Ashok Srinivasan Florida State University COT 5410 – Spring 2004.

Image Segmentation Rob Atlas Nick Bridle Evan Radkoff.

8.1 Vector spaces A set of vector is said to form a linear vector space V Chapter 8 Matrices and vector spaces.

Spectral coordinate of node u is its location in the k -dimensional spectral space: Spectral coordinates: The i ’th component of the spectral coordinate.

Presenter ： Kuang-Jui Hsu Date ： 2011/5/3(Tues.).

Modal Shape Analysis beyond Laplacian (CAGP 2012) Klaus Hildebrandt, Christian Schulz, Christoph von Tycowicz, Konrad Polthier (brief) Presenter: ShiHao.Wu.

Segmentation using eigenvectors

Segmentation using eigenvectors Papers: “Normalized Cuts and Image Segmentation”. Jianbo Shi and Jitendra Malik, IEEE, 2000 “Segmentation using eigenvectors:

Image Segmentation February 27, Implicit Scheme is considerably better with topological change. Transition from Active Contours: –contour v(t) 

Machine Learning Problems Unsupervised Learning – Clustering – Density estimation – Dimensionality Reduction Supervised Learning – Classification – Regression.

Chapter 14: SEGMENTATION BY CLUSTERING 1. 2 Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary.

Low-Rank Kernel Learning with Bregman Matrix Divergences Brian Kulis, Matyas A. Sustik and Inderjit S. Dhillon Journal of Machine Learning Research 10.

Spectral Analysis based on the Adjacency Matrix of Network Data Leting Wu Fall 2009.

Andreas Papadopoulos - [DEXA 2015] Clustering Attributed Multi-graphs with Information Ranking 26th International.

Jad silbak -University of Haifa. What we have, and what we want: Most segmentations until now focusing on local features (K-Means). We would like to extract.

Detecting Communities Via Simultaneous Clustering of Graphs and Folksonomies Akshay Java Anupam Joshi Tim Finin University of Maryland, Baltimore County.

Learning Spectral Clustering, With Application to Speech Separation F. R. Bach and M. I. Jordan, JMLR 2006.

Gaussian Mixture Models and Expectation-Maximization Algorithm.

A Convergent Solution to Tensor Subspace Learning.

Domain decomposition in parallel computing Ashok Srinivasan Florida State University.

About Me Swaroop Butala  MSCS – graduating in Dec 09  Specialization: Systems and Databases  Interests:  Learning new technologies  Application of.

 In the previews parts we have seen some kind of segmentation method.  In this lecture we will see graph cut, which is a another segmentation method.

Optimal Reverse Prediction: Linli Xu, Martha White and Dale Schuurmans ICML 2009, Best Overall Paper Honorable Mention A Unified Perspective on Supervised,

Presented by Alon Levin

A Tutorial on Spectral Clustering Ulrike von Luxburg Max Planck Institute for Biological Cybernetics Statistics and Computing, Dec. 2007, Vol. 17, No.

Clustering Usman Roshan CS 675. Clustering Suppose we want to cluster n vectors in R d into two groups. Define C 1 and C 2 as the two groups. Our objective.

Normalized Cuts and Image Segmentation Patrick Denis COSC 6121 York University Jianbo Shi and Jitendra Malik.

Document Clustering with Prior Knowledge Xiang Ji et al. Document Clustering with Prior Knowledge. SIGIR 2006 Presenter: Suhan Yu.

Spectral Methods for Dimensionality

Clustering Clustering definition: Partition a given set of objects into M groups (clusters) such that the objects of each group are ‘similar’ and ‘different’

Clustering Usman Roshan.

Matrices and vector spaces

Degree and Eigenvector Centrality

Segmentation Graph-Theoretic Clustering.

Spectral Clustering Eric Xing Lecture 8, August 13, 2010

3.3 Network-Centric Community Detection

Linear Algebra Lecture 32.

 = N  N matrix multiplication N = 3 matrix N = 3 matrix N = 3 matrix

Spectral clustering methods

Clustering Usman Roshan CS 675.

Presentation transcript:

A Unified View of Kernel k-means, Spectral Clustering and Graph Cuts Dhillon, Inderjit S., Yuqiang Guan and Brian Kulis

Outline (Kernel) kmean, weighted kernel kmean Spectral clustering algorithms The connect of kernel kmean and spectral clustering algorithms The Uniformed Problem and the ways to solve the problem Experiment results

K means and Kernel K means

Weighted Kernel k means Distance from ai to cluster c Matrix Form

Spectral Methods Represent the data by a graph Each data points corresponds to a node on the graph The weight of the edge between two nodes represent the similarity between the two corresponding data points The similarity can be a kernel function, such as the RBF kernel Use spectral theory to find the cut for the graph: Spectral Clustering

Spectral Methods

Spectral Methods Similar in the cluster Difference between clusters

Represented with Matrix Ratio assoc Ratio cut L for Ncut Norm assoc

Weighted Graph Cut Weighted association Weighted cut

Conclusion Spectral Methods are special case of Kernel K means

Solve the unified problem A standard result in linear algebra states that if we relax the trace maximizations, such that Y is an arbitrary orthonormal matrix, then the optimal Y is of the form Vk Q, where Vk consists of the leading k eigenvectors of W1/2KW1/2 and Q is an arbitrary k × k orthogonal matrix. As these eigenvectors are not indicator vectors, we must then perform postprocessing on the eigenvectors to obtain a discrete clustering of the point

From Eigen Vector to Cluster Indicator 1 2 Normalized U with L2 norm equal to 1

The Other Way Using k means to solve the graph cut problem: (random start points+ EM, local optimal). To make sure k mean converge, the kernel matrix must be positive definite. This is not true for arbitrary kernel matrix

The effect of the regularization ai is in ai is not in

Experiment results

Results (ratio association)

Results (normalized association)

Image Segmentation

Thank you. Any Question?