Download presentation

Presentation is loading. Please wait.

Published byColin Hurst Modified over 2 years ago

1
Spectral Clustering Eyal David Image Processing seminar May 2008

2
2 Lecture Outline Motivation Graph overview and construction Demo Spectral Clustering Demo Cool implementations

3
3 A Tutorial on Spectral Clustering\Arik Azran

4
4 Spectral Clustering Example – 2 Spirals Dataset exhibits complex cluster shapes K-means performs very poorly in this space due bias toward dense spherical clusters. K-means performs very poorly in this space due bias toward dense spherical clusters. In the embedded space given by two leading eigenvectors, clusters are trivial to separate. Spectral Clustering - Derek Greene

5
5 Lecture Outline Motivation Graph overview and construction Graph demo Spectral Clustering Spectral Clustering demo Cool implementation

6
6 Matthias Hein and Ulrike von Luxburg August 2007

7
7

8
8

9
9

10
10 Matthias Hein and Ulrike von Luxburg August 2007

11
11 Matthias Hein and Ulrike von Luxburg August 2007

12
12 Matthias Hein and Ulrike von Luxburg August 2007

13
13 Matthias Hein and Ulrike von Luxburg August 2007

14
14 Matthias Hein and Ulrike von Luxburg August 2007

15
15 Matthias Hein and Ulrike von Luxburg August 2007

16
16 Matthias Hein and Ulrike von Luxburg August 2007

17
17 Matthias Hein and Ulrike von Luxburg August 2007

18
Demo (Live example)

19
19 Lecture Outline Motivation Graph overview and construction Demo Spectral Clustering Demo Cool implementations

20
20 Matthias Hein and Ulrike von Luxburg August 2007

21
21 Matthias Hein and Ulrike von Luxburg August 2007

22
22 Matthias Hein and Ulrike von Luxburg August 2007

23
23 Matthias Hein and Ulrike von Luxburg August 2007

24
24 Matthias Hein and Ulrike von Luxburg August 2007

25
25 Matthias Hein and Ulrike von Luxburg August 2007

26
26 Matthias Hein and Ulrike von Luxburg August 2007

27
27 Matthias Hein and Ulrike von Luxburg August 2007

28
28 Matthias Hein and Ulrike von Luxburg August 2007

29
29 Eigenvectors & Eigenvalues

30
30 Matthias Hein and Ulrike von Luxburg August 2007

31
31 Matthias Hein and Ulrike von Luxburg August 2007

32
Demo (Live example)

33
33 Spectral Clustering Algorithm Ng, Jordan, and Weiss Motivation Given a set of points We would like to cluster them into k subsets Slides from Spectral Clustering by Rebecca Nugent, Larissa Stanberry based on Ng et al On Spectral clustering: analysis and algorithm

34
34 Algorithm Form the affinity matrix Define if Scaling parameter chosen by user Define D a diagonal matrix whose (i,i) element is the sum of As row i Slides from Spectral Clustering by Rebecca Nugent, Larissa Stanberry based on Ng et al On Spectral clustering: analysis and algorithm

35
35 Algorithm Form the matrix Find, the k largest eigenvectors of L These form the the columns of the new matrix X Note: have reduced dimension from nxn to nxk Slides from Spectral Clustering by Rebecca Nugent, Larissa Stanberry based on Ng et al On Spectral clustering: analysis and algorithm

36
36 Algorithm Form the matrix Y Renormalize each of Xs rows to have unit length Y Treat each row of Y as a point in Cluster into k clusters via K-means Slides from Spectral Clustering by Rebecca Nugent, Larissa Stanberry based on Ng et al On Spectral clustering: analysis and algorithm

37
37 Algorithm Final Cluster Assignment Assign point to cluster j iff row i of Y was assigned to cluster j Slides from Spectral Clustering by Rebecca Nugent, Larissa Stanberry based on Ng et al On Spectral clustering: analysis and algorithm

38
38 Why? If we eventually use K-means, why not just apply K-means to the original data? This method allows us to cluster non-convex regions Slides from Spectral Clustering by Rebecca Nugent, Larissa Stanberry based on Ng et al On Spectral clustering: analysis and algorithm

39
39 Some Examples

40
40 Ng et al On Spectral clustering: analysis and algorithm

41
41 Ng et al On Spectral clustering: analysis and algorithm

42
42 Ng et al On Spectral clustering: analysis and algorithm

43
43 Ng et al On Spectral clustering: analysis and algorithm

44
44 Ng et al On Spectral clustering: analysis and algorithm

45
45 Ng et al On Spectral clustering: analysis and algorithm

46
46 Ng et al On Spectral clustering: analysis and algorithm

47
47 Ng et al On Spectral clustering: analysis and algorithm

48
48 Users Prerogative Affinity matrix construction Choice of scaling factor Realistically, search over and pick value that gives the tightest clusters Choice of k, the number of clusters Choice of clustering method Slides from Spectral Clustering by Rebecca Nugent, Larissa Stanberry based on Ng et al On Spectral clustering: analysis and algorithm

49
49 Largest eigenvalues of Cisi/Medline data λ1λ1 λ2λ2 How to select k ? Eigengap: the difference between two consecutive eigenvalues. Most stable clustering is generally given by the value k that maximises the expression Choose k=2 Choose k=2 Spectral Clustering - Derek Greene

50
50 Matthias Hein and Ulrike von Luxburg August 2007 Recap – The bottom line

51
51 Summary Spectral clustering can help us in hard clustering problems The technique is simple to understand The solution comes from solving a simple algebra problem which is not hard to implement Great care should be taken in choosing the starting conditions

52
The End

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google