Download presentation

Presentation is loading. Please wait.

Published byAriel Bilberry Modified over 2 years ago

1
1 Learning the topology of a data set Michaël Aupetit – Researcher Engineer (CEA) Pierre Gaillard – Ph.D. Student Gerard Govaert – Professor (University of Technology of Compiegne) PASCAL BOOTCAMP, Vilanova, July 2007

2
2 12/07/2007 BootCamp PASCAL Introduction Given a set of M data in R D, the estimation of the density allow solving various problems : classification, clustering, regression

3
3 12/07/2007 BootCamp PASCAL A question without answer… The generative models cannot aswer this question: Which is the « shape » of this data set ?

4
4 12/07/2007 BootCamp PASCAL An subjective answer 1 point and 1 curve not connected to each other The expected answer is : The problem : what is the topology of the principal manifolds

5
5 12/07/2007 BootCamp PASCAL Why learning topology : (semi)-supervised applications Estimate the complexity of the classification task [Lallich02, Aupetit05Neurocomputing] Add a topological a priori to design a classifier [Belkin05Nips] Add topological features to statistical features Classify through the connected components or the intrinsic dimension. [Belkin]

6
6 12/07/2007 BootCamp PASCAL Why learning topology : unsupervised applications [Zeller, Schulten -IEEE ISIC1996] Clusters defined by the connected components Data exploration (e.g. shortest path) Robotic (Optimal path, inverse kinematic)

7
7 12/07/2007 BootCamp PASCAL Gaussian Mixture Generative manifold learning Problems: fixed or incomplete topology GTM [Bishop] Revisited Principal Curves [Hastie,Stuetzle] MPPCA [Bishop]

8
8 12/07/2007 BootCamp PASCAL Computational Topology M1M1 M2M2 All the previous work about topology learning has been grounded on the result of Edelsbrunner and Shah (1997) which proved that given a manifold and a set of N prototypes nearby M, it exists a subgraph* of the Delaunay graph of the prototypes which has the same topology as M * more exactly a subcomplex of the Delaunay complex

9
9 12/07/2007 BootCamp PASCAL Computational Topology M1M1 M2M2 All the previous work about topology learning has been grounded on the result of Edelsbrunner and Shah (1997) which proved that given a manifold and a set of N prototypes nearby M, it exists a subgraph of the Delaunay graph of the prototypes which has the same topology as M

10
10 12/07/2007 BootCamp PASCAL Computational Topology M1M1 M2M2 All the previous work about topology learning has been grounded on the result of Edelsbrunner and Shah (1997) which proved that given a manifold and a set of N prototypes nearby M, it exists a subgraph of the Delaunay graph of the prototypes which has the same topology as M

11
11 12/07/2007 BootCamp PASCAL Computational Topology M1M1 M2M2 All the previous work about topology learning has been grounded on the result of Edelsbrunner and Shah (1997) which proved that given a manifold and a set of N prototypes nearby M, it exists a subgraph of the Delaunay graph of the prototypes which has the same topology as M

12
12 12/07/2007 BootCamp PASCAL Computational Topology M1M1 M2M2 All the previous work about topology learning has been grounded on the result of Edelsbrunner and Shah (1997) which proved that given a manifold and a set of N prototypes nearby M, it exists a subgraph of the Delaunay graph of the prototypes which has the same topology as M

13
13 12/07/2007 BootCamp PASCAL Computational Topology M1M1 M2M2 All the previous work about topology learning has been grounded on the result of Edelsbrunner and Shah (1997) which proved that given a manifold and a set of N prototypes nearby M, it exists a subgraph of the Delaunay graph of the prototypes which has the same topology as M Extractible topology O(DN 3 )

14
14 12/07/2007 BootCamp PASCAL Application : known manifold Topology of molecules [Edelsbrunner1994]

15
15 12/07/2007 BootCamp PASCAL Approximation : manifold known throught a data set

16
16 12/07/2007 BootCamp PASCAL Topology Representing Network Topology Representing Network [Martinetz, Schulten 1994] Connect the 1 st and 2 nd NN of each data

17
17 12/07/2007 BootCamp PASCAL Topology Representing Network Topology Representing Network [Martinetz, Schulten 1994] 1 er 2 nd

18
18 12/07/2007 BootCamp PASCAL Topology Representing Network Topology Representing Network [Martinetz, Schulten 1994] 1 er 2 nd

19
19 12/07/2007 BootCamp PASCAL Topology Representing Network Topology Representing Network [Martinetz, Schulten 1994] 1 er 2 nd

20
20 12/07/2007 BootCamp PASCAL Topology Representing Network Topology Representing Network [Martinetz, Schulten 1994]

21
21 12/07/2007 BootCamp PASCAL Topology Representing Network Topology Representing Network [Martinetz, Schulten 1994]

22
22 12/07/2007 BootCamp PASCAL Topology Representing Network

23
23 12/07/2007 BootCamp PASCAL Topology Representing Network Good points : 1- O(DNM) 2- If there are enough prototypes and if they are well located then resulting graph is « good » in practice. Some drawbacks from the machine learning point of view

24
24 12/07/2007 BootCamp PASCAL Topology Representing Network : some drawbacks Noise sensitivity

25
25 12/07/2007 BootCamp PASCAL Topology Representing Network : some drawbacks Not self-consistent [Hastie]

26
26 12/07/2007 BootCamp PASCAL Topology Representing Network : some drawbacks No quality measure –How to measure the quality of the TRN if D >3 ? –How to compare two models ? For all these reasons, we propose a generative model

27
27 12/07/2007 BootCamp PASCAL General assumptions on data generation Unknown principal manifolds

28
28 12/07/2007 BootCamp PASCAL Unknown principal manifolds …from which are drawn data with a unknown pdf General assumptions on data generation

29
29 12/07/2007 BootCamp PASCAL Unknown principal manifolds …from which are drawn data with a unknown pdf …corroputed with some unknown noise leading to the observarion General assumptions on data generation

30
30 12/07/2007 BootCamp PASCAL General assumptions on data generation Unknown principal manifolds …from which are drawn data with a unknown pdf …corroputed with some unknown noise leading to the observarion The goal is to learn from the observed data, the principal manifolds such that their topological features can be extracted

31
31 12/07/2007 BootCamp PASCAL 3 assumptions…1 generative model The manifold is close to the DG of some prototypes

32
32 12/07/2007 BootCamp PASCAL 3 assumptions…1 generative model The manifold is close to the DG of some prototypes we associate to each component a weighted uniform distribution

33
33 12/07/2007 BootCamp PASCAL 3 assumptions…1 generative model we convolve the components by an isotropic Gaussian noise we associate to each component a weighted uniform distribution The manifold is close to the DG of some prototypes

34
34 12/07/2007 BootCamp PASCAL A Gaussian-point and a Gaussian-segment A A B How to define a generative model based on points and segments ? can be expressed in terms of « erf »

35
35 12/07/2007 BootCamp PASCAL Hola !

36
36 12/07/2007 BootCamp PASCAL Proposed approach : 3 steps 1. Initialization Location of the prototypes with a « classical » isotropic GM …and then building of the Delaunay Graph Initialize the generative model (equiprobability of the components)

37
37 12/07/2007 BootCamp PASCAL Number of prototypes min BIC ~ - Likelihood + Complexity of the model

38
38 12/07/2007 BootCamp PASCAL Proposed approach : 3 steps 2. Learning update the variance of the Gaussian noise, the weights of the components, and the location of the prototypes with the EM algorithm in order to maximize the Likelihood of the model w.r.t the N observed data :

39
39 12/07/2007 BootCamp PASCAL EM updates

40
40 12/07/2007 BootCamp PASCAL EM updates

41
41 12/07/2007 BootCamp PASCAL Proposed approach : 3 steps Some components have a (quasi-) nul probability (weights): They do not explain the data and can be prunned from the initial graph 3. After the learning

42
42 12/07/2007 BootCamp PASCAL Threshold setting ~ « Cattell Scree Test »

43
43 12/07/2007 BootCamp PASCAL Toy Experiment Seuillage sur le nombre de witness

44
44 12/07/2007 BootCamp PASCAL Toy Experiment

45
45 12/07/2007 BootCamp PASCAL Toy Experiment

46
46 12/07/2007 BootCamp PASCAL Other applications + O

47
47 12/07/2007 BootCamp PASCAL Comments There is « no free lunch » –Time Complexity O(DN 3 ) (initial Delaunay graph) –Slow convergence (EM) –Local optima

48
48 12/07/2007 BootCamp PASCAL Key Points Statistical learning of the topology of a data set Assumption : »Initial Delaunay graph is rich enough to contain a sub-graph having the same topology as the principal manifolds Based on a statistical criterion (the likelihood) available in any dimension « Generalized » Gaussian Mixture Can be seen as a generalization of the « Gaussian mixture » (no edges) Can be seen as a finite mixture (number of « Gaussian-segment ») of an infinite mixture (Gaussian-segment) This preliminary work is an attempt to bridge the gap between Statistical Learning Theory and Computational Topology

49
49 12/07/2007 BootCamp PASCAL Open questions Validity of the assumption « good » penalized-likelihood = « good » topology Theorem of «universal approximation» of manifold ?

50
50 12/07/2007 BootCamp PASCAL Related works Publications NIPS 2005 (unsupervised) and ESANN 2007 (supervised: analysis of the iris and oil flow data sets) Workshop submission at NIPS on this topic in collaboration with F. Chazal (INRIA Futurs), D. Cohen-Steiner (INRIA Sophia), S. Canu and G.Gasso (INSA Rouen)

51
51 Thanks

52
52 12/07/2007 BootCamp PASCAL Equations

53
53 12/07/2007 BootCamp PASCAL Topology d i =0 d i =1 d i =2 … … Number of holes (Betti), connectedness… Homeomorphism: topological equivalence = = intrinsic dimension

Similar presentations

OK

CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 10: The Bayesian way to fit models Geoffrey Hinton.

CSC321: 2011 Introduction to Neural Networks and Machine Learning Lecture 10: The Bayesian way to fit models Geoffrey Hinton.

© 2018 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on uses of plants for grade 1 Ppt on healthy food and junk food Ppt on airbag working principle of ups Ppt on synthesis and degradation of purines and pyrimidines are components Ppt on houses around the world Ppt on professional development of teachers Ppt on sports day speech Download ppt on coordinate geometry for class 9th biology Ppt on microsoft excel formulas Ppt on branches of biology