Induction: Discussion Sources: –Chapter 3, Lenz et al Book: Case-based Reasoning Technology –www.aic.nrl.navy.mil/~aha/research/applications.html.

Slides:

Advertisements

Similar presentations

Learning from Observations Chapter 18 Section 1 – 3.

Advertisements

Association Analysis (Data Engineering). Type of attributes in assoc. analysis Association rule mining assumes the input data consists of binary attributes.

CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 15 Nov, 1, 2011 Slide credit: C. Conati, S.

Decision Tree Approach in Data Mining

ICS320-Foundations of Adaptive and Learning Systems

Lazy vs. Eager Learning Lazy vs. eager learning

Decision Tree Learning 主講人：虞台文大同大學資工所智慧型多媒體研究室.

1 Knowledge Engineering for Bayesian Networks Ann Nicholson School of Computer Science and Software Engineering Monash University.

Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Spring 2004.

Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Fall 2005.

Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18 Fall 2004.

Learning from Observations Copyright, 1996 © Dale Carnegie & Associates, Inc. Chapter 18.

LEARNING DECISION TREES

Learning decision trees derived from Hwee Tou Ng, slides for Russell & Norvig, AI a Modern Approachslides Tom Carter, “An introduction to information theory.

Learning decision trees

Learning decision trees derived from Hwee Tou Ng, slides for Russell & Norvig, AI a Modern Approachslides Tom Carter, “An introduction to information theory.

On the Application of Artificial Intelligence Techniques to the Quality Improvement of Industrial Processes P. Georgilakis N. Hatziargyriou Schneider ElectricNational.

Induction of Decision Trees (IDT) CSE 335/435 Resources: – –

Decision Tree Models in Data Mining

Software Evolution Planning CIS 376 Bruce R. Maxim UM-Dearborn.

Basic Data Mining Techniques

Issues with Data Mining

DATA MINING : CLASSIFICATION. Classification : Definition  Classification is a supervised learning.  Uses training sets which has correct answers (class.

Fall 2004 TDIDT Learning CS478 - Machine Learning.

Machine Learning Chapter 3. Decision Tree Learning

INTRODUCTION TO MACHINE LEARNING. $1,000,000 Machine Learning  Learn models from data  Three main types of learning :  Supervised learning  Unsupervised.

Inductive learning Simplest form: learn a function from examples

Mohammad Ali Keyvanrad

Modeling & Simulation: An Introduction Some slides in this presentation have been copyrighted to Dr. Amr Elmougy.

LEARNING DECISION TREES Yılmaz KILIÇASLAN. Definition - I Decision tree induction is one of the simplest, and yet most successful forms of learning algorithm.

Basic Data Mining Technique

Learning from observations

Learning from Observations Chapter 18 Through

CHAPTER 18 SECTION 1 – 3 Learning from Observations.

Categorical data. Decision Tree Classification Which feature to split on? Try to classify as many as possible with each split (This is a good split)

Data Mining – Intro. Course Overview Spatial Databases Temporal and Spatio-Temporal Databases Multimedia Databases Data Mining.

CS 8751 ML & KDDDecision Trees1 Decision tree representation ID3 learning algorithm Entropy, Information gain Overfitting.

Decision Trees. What is a decision tree? Input = assignment of values for given attributes –Discrete (often Boolean) or continuous Output = predicated.

Chapter 6 Classification and Prediction Dr. Bernard Chen Ph.D. University of Central Arkansas.

CS 5751 Machine Learning Chapter 3 Decision Tree Learning1 Decision Trees Decision tree representation ID3 learning algorithm Entropy, Information gain.

Decision Trees Binary output – easily extendible to multiple output classes. Takes a set of attributes for a given situation or object and outputs a yes/no.

Classification And Bayesian Learning

An Introduction Student Name: Riaz Ahmad Program: MSIT( ) Subject: Data warehouse & Data Mining.

Decision Tree Learning Presented by Ping Zhang Nov. 26th, 2007.

Data Mining and Decision Support

Computational Biology Group. Class prediction of tumor samples Supervised Clustering Detection of Subgroups in a Class.

1 Classification: predicts categorical class labels (discrete or nominal) classifies data (constructs a model) based on the training set and the values.

Chapter 18 Section 1 – 3 Learning from Observations.

Outline Decision tree representation ID3 learning algorithm Entropy, Information gain Issues in decision tree learning 2.

Machine Learning Lecture 1: Intro + Decision Trees Moshe Koppel Slides adapted from Tom Mitchell and from Dan Roth.

Learning From Observations Inductive Learning Decision Trees Ensembles.

Anifuddin Azis LEARNING. Why is learning important? So far we have assumed we know how the world works Rules of queens puzzle Rules of chess Knowledge.

Learning from Observations

Learning from Observations

Introduce to machine learning

Presented By S.Yamuna AP/CSE

Chapter 6 Classification and Prediction

Example Example Alternate Type Patrons Target_wait x1 Yes Thai Full

Classification and Prediction

One-Sample Tests of Hypothesis

Taxonomy of Problem Solving and Case-Based Reasoning (CBR)

Machine Learning Chapter 3. Decision Tree Learning

Classification and Prediction

Machine Learning Chapter 3. Decision Tree Learning

Learning from Observations

©Jiawei Han and Micheline Kamber

Learning from Observations

Machine Learning: Decision Tree Learning

Presentation transcript:

Induction: Discussion Sources: –Chapter 3, Lenz et al Book: Case-based Reasoning Technology –

Information Gain Formula Patrons? none X7(-),x11(-) some X1(+),x3(+),x6(+),x8(+) full X4(+),x12(+), x2(-),x5(-),x9(-),x10(-) Gain(A) = I(p/(p+n),n/(p+n)) – Remainder(A) Reminder(A) = p(A,1) I(p 1 /(p 1 + n 1 ), n 1 /(p 1 + n 1 )) + p(A,2) I(p 2 /(p 2 + n 2 ), n 2 /(p 2 + n 2 )) + p(A,3) I(p 3 /(p 3 + n 3 ), n 3 /(p 3 + n 3 )) The standard Expected Value Formula

The IDT Example Patrons? none X7(-),x11(-) some X1(+),x3(+),x6(+),x8(+) full X4(+),x12(+), x2(-),x5(-),x9(-),x10(-) Gain(Patrons) = 1 – ((2/12)I(0,1)+(4/12)I(1,0)+(6/12)I(2/6,4/6)) = 0.541

The IDT Example (II) Type? french X1(+), x5(-) italian X6(+), x10(-) burger X3(+),x12(+), x7(-),x9(-) X4(+),x12(+) x2(-),x11(-) thai Gain(Type) = 1 – ((2/12)I(1/2,1/2)+(2/12)I(1/2,1/2)+ (4/12)I(2/4,2/4)+(4/12)I(2/4,2/4)) = 0 Thus Parents is a better choice than Type

Induction: Fielded Applications 1. Westinghouse: Transforming uranium gas 2. Hartford Steam Boiler: Transformer diagnosis 3. Steel Works Jesenice: Oil/lubricant properties 4. American Express UK: credit cards applicant 5. Siemens (BMT): Equipment configuration 6. USAF school: Thallium diagnosis 7. Boeing (Gold-digger): Manufacturing flaws 8. R.R. Donelly and Sons (APOS): Banding 9. Enichem (Enigma): Trouble shooting motor pumps 10. Palomar Observation (SKICAT): Astronomical cataloging 11. Continuum (Shopping): WWW shopping …

Classifying Credit Card Applications (from (Aha, 1996)) Credit card application yes (10% of 10 4 ) Induced Rule System Accept? Borderline? no American Express UK Problem: Expert accuracy was below average (48%) Evaluation: system was iteratively refined with experts 18 attributes (age, years of residence, etc) Improved accuracy: 75%+

Reduce Process Delays of Rotogravure Printers Problem: Bandwidth often appears on chrome cylinders causing a shutdown or costly replacement of cylinders. Cause unknown Use of inductive process to predict setting of control parameters (e.g., ink viscosity) Rules were posted on shop floor Gain: less downtime and lower replacement costs

Developing Cycle of IDT Applications (Adapted from (Langley, 1995)) Problem formulation Data collection Induction of Decision Trees/rules Evaluation of DT/rules Fielding and acceptance Maintenance

When to Consider Decision Trees Examples describable by attribute-value pairs Target function is discrete valued Disjunctive hypothesis might be required Possible noise in data Some functions are not amenable to be represented with decision trees: Parity function (returns true if input has an even number of 1’s)

Induction: Advantages Building a decision tree is a straightforward process The information gain measure is built on a sound basis During consultation, only a few tests are necessary before a classification is obtained For industrial applications, the consultation system can be delivered in a runtime system

Induction: Limitations DTs are not incremental: cannot be modified in runtime Consultation system is static Handling of unknown values for attributes is problematic The inductive approach cannot distinguish between various classes of users (e.g., experts vs non experts)