Presentation is loading. Please wait.

Presentation is loading. Please wait.

1 Using Bayesian Network for combining classifiers Leonardo Nogueira Matos Departamento de Computação Universidade Federal de Sergipe.

Similar presentations


Presentation on theme: "1 Using Bayesian Network for combining classifiers Leonardo Nogueira Matos Departamento de Computação Universidade Federal de Sergipe."— Presentation transcript:

1 1 Using Bayesian Network for combining classifiers Leonardo Nogueira Matos Departamento de Computação Universidade Federal de Sergipe

2 2 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

3 3 Why combining classifiers? Classifiers can colabore with each other Minimizes computational effort for training Maximizes global recognition rate

4 4 Why not to do so? Because combining individual preditions can be so difficult as divising a robust single classifier

5 5 Why not to do so? Because combining individual preditions can be so difficult as divising a robust single classifier Decision Classifiers Combiner

6 6 Approaches for combining classifiers L1. Data LevelL3. Decision Level L2. Feature Level Fixed rules Trainable rules

7 7 Approaches for combining classifiers L1. Data LevelL3. Decision Level L2. Feature Level Fixed rules Trainable rules

8 8 Why not to do so? Because combining individual preditions can be so difficult as divising a robust single classifier Decision Classifiers Combiner p(w|x)

9 9 Approaches for combining classifiers L1. Data LevelL3. Decision Level L2. Feature Level Fixed rules Trainable rules

10 10 Existent scenarios Pattern space Pattern 2 1 classifiers

11 11 Our scenery Pattern space classifiers

12 12 A closed look

13 13 A closed look – discriminant function

14 14 A closed look – using multiple classifiers

15 15 A closed look – using multiple classifiers The challegers: How can we combine classifier's output? How can we identify regions in pattern space?

16 16 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

17 17 Bayesian network principles A B C Those circles represent binary random variables

18 18 Bayesian network principles A B C Those circles represent binary random variables

19 19 Bayesian network principles A B C Those circles represent binary random variables dataset

20 20 Bayesian network principles A B C Those circles represent binary random variables instance

21 21 Bayesian network principles A B C Jointly probability inference is a combinatorial problem 2 possibilities 4 possibilities

22 22 Bayesian network principles A B C Jointly probability inference is a combinatorial problem Independence makes computation a little more simple

23 23 Bayesian network principles A B C Arest – indicates statistical dependence between variables

24 24 Bayesian network principles A B C Arc – represents causality

25 25 Bayesian network principles A B C A Bayesian network is a DAG (Direct Aciclic Graph) where nodes represent random variables and arcs represent causality relatioship

26 26 Bayesian network principles A B C There are polinomial time algorithms to compute inference in BN

27 27 Bayesian network principles A B C There are polinomial time algorithms to compute inference in BN Evidence

28 28 Bayesian network principles A B C There are polinomial time algorithms to compute inference in BN Evidence messages

29 29 Bayesian network principles A B C There are polinomial time algorithms to compute inference in BN Evidence

30 30 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

31 31 A Fundamental Goal

32 32 Another insight From a statistical point-of-view a Bayesian network is also a graphical model to represents a complex and factored probability distribution function

33 33 Another insight From a statistical point-of-view a Bayesian network is also a graphical model to represents a complex and factored probability distribution function

34 34 Another insight From a statistical point-of-view a Bayesian network is also a graphical model to represents a complex and factored probability distribution function The challegers: How can we combine classifier's output? How can we identify regions in pattern space?

35 35 How can we combine classifier's output? We use a BN as a graphical model of the pdf P(w|x) We assume that classifier participate in computing that function Each classifier must be a statistical classifier

36 36 How can we identify regions in pattern space?

37 37 Splitting pattern space

38 38 Defining a region

39 39 Patterns in a region

40 40 Algorithm

41 41 Bayesian Network Structure

42 42 Bayesian networks for combining classifiers

43 43 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

44 44 Results with UCI databases

45 45 Results with NIST database

46 46 System I classifiers

47 47 Preliminaries

48 48 Results with the complete dataset

49 49 Agenda Why combining classifiers? Bayesian network principles Bayesian network as an ensemble of classifiers Experimental results Future works and conclusions

50 50 Future works

51 51 Future works

52 52 Future works

53 53 Future works

54 54 Future works Pattern space Pattern 2 1 classifiers

55 55 Conclusions We have developed a method for combining classifiers using a Bayesian network A BN act as trainable ensemble of statistical classifiers The method is not suitable for small size dataset Experimental results reveal a good performance with a large dataset As a future work we intend to use a similar approach for splitting the feature vector and combine classifiers specialized on each piece of it.

56 56 Thank you!


Download ppt "1 Using Bayesian Network for combining classifiers Leonardo Nogueira Matos Departamento de Computação Universidade Federal de Sergipe."

Similar presentations


Ads by Google