1. 1 CSC 8520 Spring 2010. Paula Matuszek Kinds of Machine Learning Machine learning techniques can be grouped into several categories, in several ways: –What kind of input or data do they work on? –What kind of output or result do they produce? –What gets used as a critic? The typical broad categories are –supervised learning –unsupervised learning –reinforcement learning

2. 2 CSC 8520 Spring 2010. Paula Matuszek Supervised Learning In supervised learning, we provide the system with example training data and the result we want to see from those data. –each example, or training case, consists of a set of variables or features describing one case, including the decision that should be made –the system builds a model from the examples and uses the model to make a decision –the critic compares the actual decision to the desired decision –and tweaks the model to make the actual and desired decisions more similar

3. 3 CSC 8520 Spring 2010. Paula Matuszek Supervised Learning Examples Learn to detect spam from example spam and non-spam email Decide whether a house will sell from a list of its features Decide the age and gender of a skeleton Need more!!

4. 4 CSC 8520 Spring 2010. Paula Matuszek Supervised Learning, continued Most commonly used machine learning methods are based on supervised learning. The success of a supervised learning system depends very heavily on the examples it is given. They must be typical or representative. It also depends on the data or features provided, the feature space. The feature space must reflect the domain or field.

5. 5 CSC 8520 Spring 2010. Paula Matuszek Representative Examples The machine learning examples must accurately reflect the field or domain that we want to learn. The examples must be typical of the ones on which we will make decisions. Examples –Spam or non-spam Good: random sample of email from the last year Bad: random sample of email from the last week Bad: email received at a gmail account –need some good and bad examples from various fields here...

6. 6 CSC 8520 Spring 2010. Paula Matuszek Feature Spaces Which features to include in the examples is a major question in developing a supervised learning system: –They should be relevant to the decision to be made –They should be (mostly) observable for every example –They should be as much as possible independent of one another

7. 7 CSC 8520 Spring 2010. Paula Matuszek Relevant Examples We want our system to look at some features and some decision, and find the patterns which led to the decision. This will only work if the features we give the system are in fact related to the decision being made. Examples: –To decide whether a house will sell Probably relevant: price, square footage, age Probably irrelevant: name of the owner, day of the week Most supervised systems will accept a large number of features and successfully identify the relevant ones, but if the most relevant ones aren’t included the system cannot perform well.

8. 8 CSC 8520 Spring 2010. Paula Matuszek Unsupervised Learning In an unsupervised learning application, we do not give the system any a priori decisions. The task instead is to find similarities among the examples given and group them The critic is some measure of similarity among the cases in a group compared to those in a different group The data we provide define the kind of similarities and groupings we will find. Still important to have representative examples.

9. 9 CSC 8520 Spring 2010. Paula Matuszek Unsupervised Learning The goal in unsupervised learning is often focused more on discovery than on specific decisions. Some examples: –do my search results have some natural grouping? (eg, “bank” should give results related to finance and results related to rivers) –can I identify categories or genres of books based on what people purchase?

10. 10 CSC 8520 Spring 2010. Paula Matuszek Reinforcement Learning Reinforcement learning systems learn a series of actions or decisions, rather than a single decision, based on feedback given at the end of the series. –For instance, a system learning to play a game will make multiple moves, but the critic gives only whether the game was won or lost. A reinforcement learner has a goal, and carries out trial-and-error search to find the best paths toward that goal

11. 11 CSC 8520 Spring 2010. Paula Matuszek Reinforcement Learning A typical reinforcement learning system is an active agent, interacting with its environment. It must balance –exploration: trying different actions and sequences of actions to discover which ones work best –achievement: using sequences which have worked well so far It must also learn successful sequences of actions in an uncertain environment Typical current applications are in artificial intelligence and in engineering.