Download presentation

Presentation is loading. Please wait.

Published byYuliana Ketchum Modified about 1 year ago

1
Oblique Decision Trees Using Householder Reflection Chitraka Wickramarachchi Dr. Blair Robertson Dr. Marco Reale Dr. Chris Price Prof. Jennifer Brown

2
Introduction Literature Review Methodology Results and Discussion Outline of the Presentation

3
Example: A bank wants to predict the potential status (Default or not) of a new credit card customer For the existing customers the bank has following data Introduction Salary No of Credit cards Total Credit Amount No of Loans Total Loan Installment Value of Other earnings GenderAge Possible approach - Generalized Linear models with binomial errors Model become complex if the structure of the data is complex.

4
Decision tree is a tree structured classifier. Decision Tree (DT) Salary <= s TCA < tc TLA < tl DNDD Root Node Non-Terminal Node Terminal Node Test based on features

5
Recursively partition the feature space into disjoint sub- regions until each sub-region becomes homogeneous with respect to a particular class Partitions X1 X2

6
Choosing the best split X2 <= X1<= X1<=

7
Types of DTs Decision Trees Univariate DT Multivariate DT Linear DT Non-Linear DT Axis parallel splits Oblique splits

8
Axis parallel splits Easy to implement Computer complexity is low Easy to interpret Advantages Disadvantage When the true boundaries are not axis parallel it produces complicated boundary structure

9
Axis parallel boundaries X1 X2

10
Oblique splits Advantage - Simple boundary structure X1 X2

11
Disadvantages Implementation is challenging Computer complexity is high Therefore computationally less expensive oblique tree induction method would be desirable X1 X2 Oblique splits

12
Literature Review Oblique splits search for splits in the form of CART – LC Starts with the best axis parallel split Perturb each coefficient until find the best split Breiman et al. (1984) Can get trapped in local mimina Limitations No upper bound on the time spent at any node

13
Literature Review Heath et al. (1993) Simulated annealing Decision Trees (SADT) First places a hyperplane in a canonical location Perturb each coefficient randomly By randomization - try to escape from the local mimima Algorithm runs much slower than CART- LC Limitations

14
Literature Review Murthy et al. (1994) Oblique Classifier 1 (OC1) Start with the best axis parallel split Perturb each coefficient At a local mimima, perturb the hyperlane randomly Since 1994, there are many ODT induction methods have been developed based on EA algorithms and neural network concept

15
Proposed Methodology Our approach is to Transform the data set parallel to one of the feature axes Implement axis parallel splits Back-transform them in to the original space Transformation is done using Householder reflection.

16
Householder Reflection Let X and Y are vectors with the same norm there exists orthogonal symmetric matrix P such that where

17
Orientation of a cluster can be represented by the dominant Eigen vector of its variance covariance matrix. X1 X2 Householder Reflection

18

19
X1 X2 Householder Reflection

20
To avoid over-fitting Number of Terminal Nodes Accuracy Cost-complexity pruning

21
Results and Discussion Data sets - UCI Machine Learning Repository Data set Number of examples Number of features Number of Classes Iris Data15043 Boston Housing Data Estimate of the accuracy was obtained by ten 5-fold cross validation experiments.

22
Results and Discussion ClassiferIris DataHousing Data Householder Method CART-LC OC1 C4.5 Results High accuracy Computationally inexpensive

23

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google