Friday’s Deliverable As a GROUP, you need to bring 2N+1 copies of your “initial submission” –This paper should be a complete version of your paper – something you would be willing to turn in for your final grade in no worse than a worst-case scenario Also bring another timecard –PLEASE read the instructions

When do Schafer and East play golf?

Learning Definition: A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience.

Machine Learning Models Classification Regression Clustering Time series analysis Association Analysis Sequence Discovery ….

Classification example Weight Height o x x x x x x x x x x x o o o o o o o o o x oo x x x - weight-lifters o - ballet dancers Features: height, weight

Classification example - Simple Model Weight Height o x x x x x x x x x x x o o o o o o o o o x oo x x x - weight-lifters o - ballet dancers Decision boundary Features: height, weight

Classification example - Complex model Weight Height o x x x x x x x x x x x o o o o o o o o o x oo x x x - weight-lifters o - ballet dancers Complex Decision boundary Features: height, weight Note: A simple decision boundary is better than a complex one - It GENERALIZES better.

Classification example Model Test set Train set Learning system New data Loan Yes/No

Learning Paradigms Supervised learning - with teacher inputs and correct outputs are provided by the teacher Reinforced learning - with reward or punishment an action is evaluated Unsupervised learning - with no teacher no hint about correct output is given

Supervised Learning Activity

Supervised Learning This IS a “Frinkle”

Supervised Learning This IS a “Frinkle”

Supervised Learning This IS NOT a “Frinkle”

Supervised Learning This IS NOT a “Frinkle”

Supervised Learning Is this a “Frinkle”??

Supervised Learning

Machine Learning Methods Artificial Neural Networks Decision Trees Instance Based Methods (CBR, k-NN) Bayesian Networks Evolutionary Strategies Support Vector Machines..

Machine Learning Methods Artificial Neural Networks Decision Trees Instance Based Methods (CBR, k-NN) Bayesian Networks Evolutionary Strategies Support Vector Machines..

When do Schafer and East play golf?

Decision Tree for PlayGolf Outlook SunnyOvercastRain Humidity HighNormal Wind StrongWeak NoYes No

Decision Tree for PlayGolf Outlook SunnyOvercastRain Humidity HighNormal NoYes Each node tests an attribute Each branch corresponds to an attribute value Each leaf node assigns a classification

No Decision Tree for PlayGolf Outlook SunnyOvercastRain Humidity HighNormal Wind StrongWeak NoYes No Outlook Temperature Humidity Wind PlayGolf Sunny Hot High Weak ?

Decision Tree for Conjunction Outlook SunnyOvercastRain Wind StrongWeak NoYes No Outlook=Sunny  Wind=Weak No

Decision Tree for Disjunction Outlook SunnyOvercastRain Yes Outlook=Sunny  Wind=Weak Wind StrongWeak NoYes Wind StrongWeak NoYes

Decision Tree for XOR Outlook SunnyOvercastRain Wind StrongWeak YesNo Outlook=Sunny XOR Wind=Weak Wind StrongWeak NoYes Wind StrongWeak NoYes

Decision Tree Outlook SunnyOvercastRain Humidity HighNormal Wind StrongWeak NoYes No decision trees represent disjunctions of conjunctions (Outlook=Sunny  Humidity=Normal)  (Outlook=Overcast)  (Outlook=Rain  Wind=Weak)

Expressiveness of Decision Trees Decision trees can express ANY function of the input attributes. Trivially, there exists a decision tree for any consistent training set (one path to a leaf for each example).

When to consider Decision Trees Instances describable by attribute-value pairs Target function is discrete valued Disjunctive hypothesis may be required Possibly noisy training data Missing attribute values Examples: –Medical diagnosis –Credit risk analysis –Object classification for robot manipulator (Tan 1993)

Expressiveness of Decision Trees Decision trees can express ANY function of the input attributes. Trivially, there exists a decision tree for any consistent training set (one path to a leaf for each example). –But it probably won’t generalize to new examples. –We prefer to find more compact decision trees.

Think about it… Which tree would you rather use… AB CD CD Y F E F E F E F YYYYNYN EF AB NY E Y

Decision tree learning How do you select a small tree consistent with the training examples? Idea: (recursively) choose the “most significant” attribute as root of (sub)tree.