1. Greg GrudicIntro AI1 Introduction to Artificial Intelligence CSCI 3202 Fall 2007 Introduction to Classification Greg Grudic

2. Intro AI2 This Class: Classification Models Collect Training data Construct Model: happy = F(feature space) Make a prediction High Dimensional Feature (input) Space

3. Greg GrudicIntro AI3 Binary Classification A binary classifier is a mapping from a set of d inputs to a single output which can take on one of TWO values (e.g. path/no path) In the most general setting Specifying the output classes as -1 and +1 is arbitrary! –Often done as a mathematical convenience

4. Greg GrudicIntro AI4 A Binary Classifier Classification Model Given learning data: A model is constructed: Not in learning set!

5. Greg GrudicIntro AI5 Classification Learning Data… Example 10.950130.582791 Example 20.231140.4235 Example 30.89130.432911 Example 40.0185040.76037 …………

6. Greg GrudicIntro AI6 The Learning Data Matrix Representation of N learning examples of d dimensional inputs

7. Greg GrudicIntro AI7 Graphical Representation of 2D Classification Training Data

8. Greg GrudicIntro AI8 Linear Separating Hyper-Planes: Discriminative Classifiers How many lines can separate these points? NO!

9. Greg GrudicIntro AI9 Linear Separating Hyper-Planes (2 dimensions)

10. Greg GrudicIntro AI10 Linear Separating Hyper-Planes (d dimensions)

11. Greg GrudicIntro AI11 Linear Separating Hyper-Planes The Model: Where: The decision boundary:

12. Greg GrudicIntro AI12 Linear Separating Hyper-Planes The model parameters are: The hat on the betas means that they are estimated from the data Many different learning algorithms have been proposed for determining

13. Is this Data Linearly Separable? Greg GrudicIntro AI13 NO!

14. Is this Data Linearly Separable? Greg GrudicIntro AI14 YES!

15. Is this Data Linearly Separable? Greg GrudicIntro AI15 NO!

16. Is this Data Linearly Separable? Greg GrudicIntro AI16 YES!