1. Incremental Learning Chris Mesterharm Fordham University

2. Artificial Intellegence Make machines as smart or smarter than humans Intelligent machines can automate many tasks performed by humans Research started at beginning of modern computer era

3. Programming AI Difficult to program machines to be intelligent Some success with well structured problems Less success with “simpler” skills such as perception and vision

4. Machine Learning Teach instead of program machines Similar to humans and animals But how do we model learning?

5. Induction The sun has risen every day in the past The sun will rise tomorrow Learning from examples

6. Induction Examples Predict promoter sequences in DNA that specify where a gene is starting Predict the amount of change in the price of a particular stock in the market Predict whether or not an image contains a picture of a bicycle Predict tomorrow’s weather

7. Stock Market Example Goal is to predict how price of IBM stock will change tomorrow Learning algorithm looks for a prediction rule using any information that might be relevant –Previous prices and sales volume –Economic indicators and index fund prices –Analysts predictions and recent new reports All the information is collected together as an instance of the learning problem.

8. Stock Market Example Goal is to label a new instance with a price that is close to the future price We already have many labeled instances from previous days We use the previously labeled instances to learn a rule that works well on future instances

9. Batch Learning Instancelabel Algorithmrule Instancelabel

10. Incremental Learning Assume you get feedback on your predictions Refine your rule with label feedback Allows the rule to adjust to changing conditions

11. Online Learning Algorithm has no instances at start Algorithm gets a single instance each trial A trial is composed of three steps –Get instance –Predict label –Discover true label and use that information to refine the prediction rule

12. Online Learning rule InstancePredicted label Algorithm True label

13. Online Learning Examples Financial market Weather prediction Sports Predicting the future Spam detection Cooking

14. Winnow Learns a hyperplane to separate instances x x x x x x x o o o o o o o o o o x x x

15. Winnow Strong theoretical guarantees Performs well even with large instances Robust to noise Can learn a changing hyperplane

16. Acquisitions: Reuters

17. Tracking Winnow Example

18. Expanding and Testing Algorithm should allow –Delays in label feedback –New ways to use information in instances –Ability to internally generate labels Test online algorithms on –Financial problems –Robotics problems

19. Conclusion Online learning is a theoretically strong model for learning Online learning allows an incremental and adaptive style of learning that might be closer to the way humans and animals learn Winnow is a successful example of online learning that works well in practice