Download presentation

Presentation is loading. Please wait.

Published byLyndsey Appleby Modified over 2 years ago

1
1 Using Bayesian Network for combining classifiers Leonardo Nogueira Matos Departamento de Computação Universidade Federal de Sergipe

2
2 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

3
3 Why combining classifiers? Classifiers can colabore with each other Minimizes computational effort for training Maximizes global recognition rate

4
4 Why not to do so? Because combining individual preditions can be so difficult as divising a robust single classifier

5
5 Why not to do so? Because combining individual preditions can be so difficult as divising a robust single classifier Decision Classifiers Combiner

6
6 Approaches for combining classifiers L1. Data LevelL3. Decision Level L2. Feature Level Fixed rules Trainable rules

7
7 Approaches for combining classifiers L1. Data LevelL3. Decision Level L2. Feature Level Fixed rules Trainable rules

8
8 Why not to do so? Because combining individual preditions can be so difficult as divising a robust single classifier Decision Classifiers Combiner p(w|x)

9
9 Approaches for combining classifiers L1. Data LevelL3. Decision Level L2. Feature Level Fixed rules Trainable rules

10
10 Existent scenarios Pattern space Pattern 2 1 classifiers

11
11 Our scenery Pattern space classifiers

12
12 A closed look

13
13 A closed look – discriminant function

14
14 A closed look – using multiple classifiers

15
15 A closed look – using multiple classifiers The challegers: How can we combine classifier's output? How can we identify regions in pattern space?

16
16 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

17
17 Bayesian network principles A B C Those circles represent binary random variables

18
18 Bayesian network principles A B C Those circles represent binary random variables

19
19 Bayesian network principles A B C Those circles represent binary random variables dataset

20
20 Bayesian network principles A B C Those circles represent binary random variables instance

21
21 Bayesian network principles A B C Jointly probability inference is a combinatorial problem 2 possibilities 4 possibilities

22
22 Bayesian network principles A B C Jointly probability inference is a combinatorial problem Independence makes computation a little more simple

23
23 Bayesian network principles A B C Arest – indicates statistical dependence between variables

24
24 Bayesian network principles A B C Arc – represents causality

25
25 Bayesian network principles A B C A Bayesian network is a DAG (Direct Aciclic Graph) where nodes represent random variables and arcs represent causality relatioship

26
26 Bayesian network principles A B C There are polinomial time algorithms to compute inference in BN

27
27 Bayesian network principles A B C There are polinomial time algorithms to compute inference in BN Evidence

28
28 Bayesian network principles A B C There are polinomial time algorithms to compute inference in BN Evidence messages

29
29 Bayesian network principles A B C There are polinomial time algorithms to compute inference in BN Evidence

30
30 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

31
31 A Fundamental Goal

32
32 Another insight From a statistical point-of-view a Bayesian network is also a graphical model to represents a complex and factored probability distribution function

33
33 Another insight From a statistical point-of-view a Bayesian network is also a graphical model to represents a complex and factored probability distribution function

34
34 Another insight From a statistical point-of-view a Bayesian network is also a graphical model to represents a complex and factored probability distribution function The challegers: How can we combine classifier's output? How can we identify regions in pattern space?

35
35 How can we combine classifier's output? We use a BN as a graphical model of the pdf P(w|x) We assume that classifier participate in computing that function Each classifier must be a statistical classifier

36
36 How can we identify regions in pattern space?

37
37 Splitting pattern space

38
38 Defining a region

39
39 Patterns in a region

40
40 Algorithm

41
41 Bayesian Network Structure

42
42 Bayesian networks for combining classifiers

43
43 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

44
44 Results with UCI databases

45
45 Results with NIST database

46
46 System I classifiers

47
47 Preliminaries

48
48 Results with the complete dataset

49
49 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

50
50 Future works

51
51 Future works

52
52 Future works

53
53 Future works

54
54 Future works Pattern space Pattern 2 1 classifiers

55
55 Conclusions We have developed a method for combining classifiers using a Bayesian network A BN act as trainable ensemble of statistical classifiers The method is not suitable for small size dataset Experimental results reveal a good performance with a large dataset As a future work we intend to use a similar approach for splitting the feature vector and combine classifiers specialized on each piece of it.

56
56 Thank you!

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google