Decision Trees Example:

Slides:

Advertisements

Similar presentations

Data Mining using Decision Trees Professor J. F. Baldwin.

Advertisements

Data Mining Lecture 9.

1 Machine Learning: Lecture 3 Decision Tree Learning (Based on Chapter 3 of Mitchell T.., Machine Learning, 1997)

CHAPTER 9: Decision Trees

More Set Definitions and Proofs 1.6, 1.7. Ordered n-tuple The ordered n-tuple (a1,a2,…an) is the ordered collection that has a1 as its first element,

C4.5 algorithm Let the classes be denoted {C1, C2,…, Ck}. There are three possibilities for the content of the set of training samples T in the given node.

Paper By - Manish Mehta, Rakesh Agarwal and Jorma Rissanen

Huffman code and ID3 Prof. Sin-Min Lee Department of Computer Science.

IT 433 Data Warehousing and Data Mining

Hunt’s Algorithm CIT365: Data Mining & Data Warehousing Bajuna Salehe

Decision Tree Approach in Data Mining

Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation Lecture Notes for Chapter 4 Part I Introduction to Data Mining by Tan,

1 Data Mining Classification Techniques: Decision Trees (BUSINESS INTELLIGENCE) Slides prepared by Elizabeth Anglo, DISCS ADMU.

Application of Decision Tree: Bankruptcy Prediction 2004/05/07.

Lecture Notes for Chapter 4 Introduction to Data Mining

Final Exam: May 10 Thursday. If event E occurs, then the probability that event H will occur is p ( H | E ) IF E ( evidence ) is true THEN H ( hypothesis.

Fall 2002CMSC Discrete Structures1 Permutations How many ways are there to pick a set of 3 people from a group of 6? There are 6 choices for the.

Data Quality Class 9. Rule Discovery Decision and Classification Trees Association Rules.

ID3 Algorithm Abbas Rizvi CS157 B Spring What is the ID3 algorithm? ID3 stands for Iterative Dichotomiser 3 Algorithm used to generate a decision.

About ISoft … What is Decision Tree? Alice Process … Conclusions Outline.

Data Mining with Decision Trees Lutz Hamel Dept. of Computer Science and Statistics University of Rhode Island.

Decision Trees (2). Numerical attributes Tests in nodes are of the form f i > constant.

Classification and Prediction by Yen-Hsien Lee Department of Information Management College of Management National Sun Yat-Sen University March 4, 2003.

Classification.

Ordinal Decision Trees Qinghua Hu Harbin Institute of Technology

Machine Learning Lecture 10 Decision Trees G53MLE Machine Learning Dr Guoping Qiu1.

Chapter 7 Decision Tree.

Classification and Prediction — Slides for Textbook — — Chapter 7 —

Lecture Notes for Chapter 4 Introduction to Data Mining

Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation COSC 4368.

Decision Trees. Decision trees Decision trees are powerful and popular tools for classification and prediction. The attractiveness of decision trees is.

Training begins in… 15:00 minutes Training begins in… 14:00 minutes.

CS690L Data Mining: Classification

ID3 Algorithm Michael Crawford.

Chapter 6 Classification and Prediction Dr. Bernard Chen Ph.D. University of Central Arkansas.

Lecture 12, CS5671 Decisions, Decisions Concepts Naïve Bayesian Classification Decision Trees –General Algorithm –Refinements Accuracy Scalability –Strengths.

DECISION TREE Ge Song. Introduction ■ Decision Tree: is a supervised learning algorithm used for classification or regression. ■ Decision Tree Graph:

Data Mining Classification: Basic Concepts, Decision Trees, and Model Evaluation Lecture Notes for Chapter 4 Introduction to Data Mining by Tan, Steinbach,

ID3 Algorithm Amrit Gurung. Classification Library System Organise according to special characteristics Faster retrieval New items sorted easily Related.

1 Classification: predicts categorical class labels (discrete or nominal) classifies data (constructs a model) based on the training set and the values.

DECISION TREES Asher Moody, CS 157B. Overview  Definition  Motivation  Algorithms  ID3  Example  Entropy  Information Gain  Applications  Conclusion.

Iterative Dichotomiser 3 By Christopher Archibald.

ISQS 7342 Dr. Zhangxi Lin By: Tej Pulapa. Recommendation System Predict consumer behavior - Is to discover the relationship between one’s personal characteristics,

Chapter 3 Data Mining: Classification & Association Chapter 4 in the text box Section: 4.3 (4.3.1),

Review of Decision Tree Learning Bamshad Mobasher DePaul University Bamshad Mobasher DePaul University.

By N.Gopinath AP/CSE.  A decision tree is a flowchart-like tree structure, where each internal node (nonleaf node) denotes a test on an attribute, each.

10. Decision Trees and Markov Chains for Gene Finding.

Chapter 6 Decision Tree.

DECISION TREES An internal node represents a test on an attribute.

Decision Trees an introduction.

Introduction to Combinatorics

C4.5 algorithm Let the classes be denoted {C1, C2,…, Ck}. There are three possibilities for the content of the set of training samples T in the given node.

Lecture Notes for Chapter 4 Introduction to Data Mining

Chapter 6 Classification and Prediction

Information Management course

SAD: 6º Projecto.

Classification and Prediction

COSC 6368 Machine Learning Organization

Chapter 8 Tutorial.

Classification by Decision Tree Induction

Computing the Entropy (H-Function)

Data Mining – Chapter 3 Classification

BNFO 602 Phylogenetics – maximum likelihood

BNFO 602 Phylogenetics Usman Roshan.

COSC 6368 Machine Learning Organization

©Jiawei Han and Micheline Kamber

Promising “Newer” Technologies to Cope with the

Decision Trees Example:

Presentation transcript:

Decision Trees Example: Conducted survey to see what customers were interested in new model car Want to select customers for advertising campaign training set

Basic Information Gain Computations Result: I_Gain_Ratio: city>age>car Result: I_Gain: age > car=city Gain(D,city=)= H(1/3,2/3) – ½ H(1,0) – ½ H(1/3,2/3)=0.45 D=(2/3,1/3) G_Ratio_pen(city=)=H(1/2,1/2)=1 city=sf city=la D1(1,0) D2(1/3,2/3) Gain(D,car=)= H(1/3,2/3) – 1/6 H(0,1) – ½ H(2/3,1/3) – 1/3 H(1,0)=0.45 G_Ratio_pen(car=)=H(1/2,1/3,1/6)=1.45 D=(2/3,1/3) car=van car=merc car=taurus D1(0,1) D2(2/3,1/3) D3(1,0) G_Ratio_pen(age=)=log2(6)=2.58 Gain(D,age=)= H(1/3,2/3) – 6*1/6 H(0,1) = 0.90 D=(2/3,1/3) age=22 age=25 age=27 age=35 age=40 age=50 D1(1,0) D2(0,1) D3(1,0) D4(1,0) D5(1,0) D6(0,1)

C5.0/ID3 Test Selection Assume we have m classes in our classification problem. A test S subdivides the examples D= (p1,…,pm) into n subsets D1 =(p11,…,p1m) ,…,Dn =(p11,…,p1m). The qualify of S is evaluated using Gain(D,S) (ID3) or GainRatio(D,S) (C5.0): Let H(D=(p1,…,pm))= Si=1 (pi log2(1/pi)) (called the entropy function) Gain(D,S)= H(D) - Si=1 (|Di|/|D|)*H(Di) Gain_Ratio(D,S)= Gain(D,S) / H(|D1|/|D|,…, |Dn|/|D|) Remarks: |D| denotes the number of elements in set D. D=(p1,…,pm) implies that p1+…+ pm =1 and indicates that of the |D| examples p1*|D| examples belong to the first class, p2*|D| examples belong to the second class,…, and pm*|D| belong the m-th (last) class. H(0,1)=H(1,0)=0; H(1/2,1/2)=1, H(1/4,1/4,1/4,1/4)=2, H(1/p,…,1/p)=log2(p). C5.0 selects the test S with the highest value for Gain_Ratio(D,S), whereas ID3 picks the test S for the examples in set D with the highest value for Gain (D,S). m n