Presentation is loading. Please wait.

Presentation is loading. Please wait.

Shared Ensemble Learning using Multi-trees 전자전기컴퓨터공학과 G201249003 김영제 Database Lab.

Similar presentations


Presentation on theme: "Shared Ensemble Learning using Multi-trees 전자전기컴퓨터공학과 G201249003 김영제 Database Lab."— Presentation transcript:

1 Shared Ensemble Learning using Multi-trees 전자전기컴퓨터공학과 G 김영제 Database Lab

2 Introduction What is a decision tree? Each node in the tree specifies a test for some attribute of the instance Each branch corresponds to an attribute value Each leaf node assigns a classification Decision Tree for PlayTennis

3 Cost Associated with Machine Learning Generation costs Computational costs i.e. computer resource consumption Give better solutions for provided resources

4 Cost Associated with Machine Learning Application costs First Models are accurate in average This does not mean seamless, and confident Model can be highly accurate for frequent cases Extremely inaccurate for infrequent, critical situations i.e. diagnosis, fault detection

5 Cost Associated with Machine Learning Application costs Second Even accurate models can be useless If the purpose is to obtain some new knowledge not expressed in the form of rules the number of rules is too high The interpretation of results significant costs it may even be impossible

6 Construction of Decision Tree Tree construction Driven by a splitting criterion that selects the best split The selected split is applied to generate new branches The rest of splits are discarded Algorithm stops when the examples that fall into a branch belong to the same class

7 Construction of Decision Tree Pruning Removal of not useful parts of the tree in order to avoid over- fitting Pre-pruning performed during the construction of the tree Post-pruning performed by analyzing the leaves once the tree has been built

8 Merit and Demerit of Decision Tree Merit Allows the quick construction of a model Because decision trees are built in a eager way (greedy) Demerit It may produce bad models because of bad decisions

9 Multi-tree Structure Rejected splits are not removed But stored as suspended nodes Two new criteria required for the construction of a single decision tree Suspended node selection To populate a multi-tree, need to specify a criterion that selects one of the suspended nodes Selection of a model Select one or more comprehensible models according to a selection criterion

10 Multi-tree Structure

11 Shared Ensembles Combination Combination of a set of classifiers improves the accuracy of simple classifiers Combination methods Boosting, Bagging, Randomization, Stacking, Windowing The large amount of memory required to store Shared the common parts of the components of the ensemble Using the multi-tree

12 Shared Ensembles Combination

13

14 Original Good loser Bad loser MajorityDifference {40, 10, 30} {80, 0, 0} {40, 0, 0} {1, 0, 0} {0, -60, -20} {7, 2, 10} {0, 0, 19} {0, 0, 10} {0, 0, 1} {-5, -15, 1}

15 Experiments #DatasetSizeClasses Nom. Attr. Num. Attr. 1Balance-scale Cars Dermatology Ecoli Iris House-votes Monks Monks Monks New-thyroid Post-operative Soybean-small Tae Tic-tac Wine Information about datasets used in the experiments.

16 Experiments Arit.Sum.Prod.Max.Min. #Acc.Dev.Acc.Dev.Acc.Dev.Acc.Dev.Acc.Dev Geomean Comparison between fusion techniques

17 Experiments Max+OrigMax+GoodMax+BadMax+Majo.Max+Diff. #Acc.DevAcc.DevAcc.DevAcc.DevAcc.Dev Gmean Comparison between vector transformation methods

18 Experiments #Acc.Dev.Acc.Dev.Acc.Dev.Acc.Dev Gmean Influence of the size of the multi-tree

19 Experiments

20

21 References 54-escicucrri.pdf 54-escicucrri.pdf Shared Ensemble Learning using Multi-trees V. Estruch, C. Ferri, J. Hernandez-Orallo, M.J. Ramirez-Quintana Wikipedia

22 Thank you for listening


Download ppt "Shared Ensemble Learning using Multi-trees 전자전기컴퓨터공학과 G201249003 김영제 Database Lab."

Similar presentations


Ads by Google