Presentation is loading. Please wait.

Presentation is loading. Please wait.

Oblique Decision Trees Using Householder Reflection Chitraka Wickramarachchi Dr. Blair Robertson Dr. Marco Reale Dr. Chris Price Prof. Jennifer Brown.

Similar presentations


Presentation on theme: "Oblique Decision Trees Using Householder Reflection Chitraka Wickramarachchi Dr. Blair Robertson Dr. Marco Reale Dr. Chris Price Prof. Jennifer Brown."— Presentation transcript:

1 Oblique Decision Trees Using Householder Reflection Chitraka Wickramarachchi Dr. Blair Robertson Dr. Marco Reale Dr. Chris Price Prof. Jennifer Brown

2  Introduction  Literature Review  Methodology  Results and Discussion Outline of the Presentation

3  Example: A bank wants to predict the potential status (Default or not) of a new credit card customer  For the existing customers the bank has following data Introduction Salary No of Credit cards Total Credit Amount No of Loans Total Loan Installment Value of Other earnings GenderAge Possible approach - Generalized Linear models with binomial errors Model become complex if the structure of the data is complex.

4 Decision tree is a tree structured classifier. Decision Tree (DT) Salary <= s TCA < tc TLA < tl DNDD Root Node Non-Terminal Node Terminal Node Test based on features

5 Recursively partition the feature space into disjoint sub- regions until each sub-region becomes homogeneous with respect to a particular class Partitions X1 X2

6 Choosing the best split X2 <= X1<= X1<=

7 Types of DTs Decision Trees Univariate DT Multivariate DT Linear DT Non-Linear DT Axis parallel splits Oblique splits

8 Axis parallel splits  Easy to implement  Computer complexity is low  Easy to interpret Advantages Disadvantage  When the true boundaries are not axis parallel it produces complicated boundary structure

9 Axis parallel boundaries X1 X2

10 Oblique splits Advantage - Simple boundary structure X1 X2

11 Disadvantages  Implementation is challenging  Computer complexity is high Therefore computationally less expensive oblique tree induction method would be desirable X1 X2 Oblique splits

12 Literature Review Oblique splits search for splits in the form of CART – LC  Starts with the best axis parallel split  Perturb each coefficient until find the best split Breiman et al. (1984)  Can get trapped in local mimina Limitations  No upper bound on the time spent at any node

13 Literature Review Heath et al. (1993) Simulated annealing Decision Trees (SADT)  First places a hyperplane in a canonical location  Perturb each coefficient randomly By randomization - try to escape from the local mimima Algorithm runs much slower than CART- LC Limitations

14 Literature Review Murthy et al. (1994) Oblique Classifier 1 (OC1)  Start with the best axis parallel split  Perturb each coefficient  At a local mimima, perturb the hyperlane randomly Since 1994, there are many ODT induction methods have been developed based on EA algorithms and neural network concept

15 Proposed Methodology Our approach is to  Transform the data set parallel to one of the feature axes  Implement axis parallel splits  Back-transform them in to the original space Transformation is done using Householder reflection.

16 Householder Reflection Let X and Y are vectors with the same norm there exists orthogonal symmetric matrix P such that where

17 Orientation of a cluster can be represented by the dominant Eigen vector of its variance covariance matrix. X1 X2 Householder Reflection

18

19 X1 X2 Householder Reflection

20 To avoid over-fitting Number of Terminal Nodes Accuracy Cost-complexity pruning

21 Results and Discussion Data sets - UCI Machine Learning Repository Data set Number of examples Number of features Number of Classes Iris Data15043 Boston Housing Data Estimate of the accuracy was obtained by ten 5-fold cross validation experiments.

22 Results and Discussion ClassiferIris DataHousing Data Householder Method CART-LC OC1 C4.5 Results  High accuracy  Computationally inexpensive

23


Download ppt "Oblique Decision Trees Using Householder Reflection Chitraka Wickramarachchi Dr. Blair Robertson Dr. Marco Reale Dr. Chris Price Prof. Jennifer Brown."

Similar presentations


Ads by Google