Introduction Introduction Dr. Khaled Wassif Spring Machine Learning

Outline Why Machine Learning? Why Machine Learning? Relevant disciplines Relevant disciplines What is a well-defined learning problem? What is a well-defined learning problem? How to design a learning system? How to design a learning system? What issues arise in machine learning problems? What issues arise in machine learning problems? Machine Learning Research Machine Learning Research Machine Learning By Dr. Khaled Wassif Slide 1- 2

Adding new kind of capability for computers: Adding new kind of capability for computers: –Data mining »E.g. extracting new information from medical records, maintenance records, etc. –Self-customizing programs »A learning newsreader/browser that learns what you like and seeks it out. –Applications we can’t program by hand »E.g. speech recognition and autonomous driving Why Machine Learning? Machine Learning By Dr. Khaled Wassif Slide 1- 3

Understanding human learning and teaching: Understanding human learning and teaching: –Mature mathematical models might contribute approaching into biological ones. The time is right: The time is right: –Recent progress in algorithms and theory –Enormous amounts of data and applications »Increasing online data, increasing networking to share it –Substantial computational power –Promising industry Why Machine Learning? Machine Learning By Dr. Khaled Wassif Slide 1- 4

Typical Data Mining Task Machine Learning By Dr. Khaled Wassif Slide 1- 5 Data: Given: ─9714 patient records, each describing a pregnancy and birth. ─Each patient record contains 215 features, some unspecified (we have little control over data). ─Learn to predict classes of future patients at high risk for emergency cesarean section.

Data Mining Result Machine Learning By Dr. Khaled Wassif Slide 1- 6 One of 18 learned rules: If No previous vaginal delivery, and Abnormal 2nd Trimester Ultrasound, and Malpresentation at admission Then Probability of Emergency C-Section is 0.6 Over training data: 26/41 =.63, Over test data: 12/20 =.60 Data:

Credit Risk Analysis Machine Learning By Dr. Khaled Wassif Slide 1- 7 Rules learned: If Other-Delinquent-Accounts > 2, and Number-Delinquent-Billing-Cycles > 1 Then Profitable-Customer? = No [Deny Credit Card application] If Other-Delinquent-Accounts = 0, and (Income > $30k) OR (Years-of-Credit > 3) Then Profitable-Customer? = Yes [Accept Credit Card application] Data:

Software that Customizes to User Machine Learning By Dr. Khaled Wassif Slide 1- 8 Adaptive Web Sites

Now most pocket Speech Recognizers or Translators are running on some sort of learning device --- the more you play/use them, the smarter they become! Now most pocket Speech Recognizers or Translators are running on some sort of learning device --- the more you play/use them, the smarter they become! Natural Language Processing and Speech Recognition Machine Learning By Dr. Khaled Wassif Slide 1- 9

Behind a security camera, most likely there is a computer that is learning and/or checking! Behind a security camera, most likely there is a computer that is learning and/or checking! Object Recognition Machine Learning By Dr. Khaled Wassif Slide 1- 10

The best helicopter pilot is now a computer! The best helicopter pilot is now a computer! –it runs a program that learns how to fly and make acrobatic military exercises by itself! –no taped instructions, joysticks, or things like … Robotic Control Machine Learning By Dr. Khaled Wassif Slide 1- 11

Now cars can find their own ways! Now cars can find their own ways! Robotic Control Machine Learning By Dr. Khaled Wassif Slide 1- 12

Reading, digesting, and categorizing a vast text database is too much for human! Reading, digesting, and categorizing a vast text database is too much for human! Text Mining Machine Learning By Dr. Khaled Wassif Slide We want:

Understanding Brain Activities Machine Learning By Dr. Khaled Wassif Slide 1- 14

Gacatcgctgcgtttcggcagctaattgccttttagaaattattttcccatttcgagaaactcgtgtgggatgccggatgcggctttcaatcacttctggcccgggatcggattgggtcacattgtcgcgggctctattgtctcgatccgcggcgcagttcgcgtgcttagcggtcagaaaggcagagattcggttcggattgatgcgctggcagcagggcacaaagatcta atgactggcaaatcgctacaaataaattaaagtccggcggctaattatgagcggactgaagccactttggattaaccaaaaaacagcagataaacaaaaacggcaaagaaaattgccacagagttgtcacgctttgttgcacaaacatttgtgcagaaaagtgaaaagcttttagccattattaagtttttcctcagtcgctggcagcacttgcgaatgtactgatgttcctc ataaatgaaaattaatgtttgctctacgctccaccgaactcgcttgtttgggggattggctggctaatcgcggctagatcccaggcggtataaccttttcgcttcatcagttgtgaaccagatggctggtgttttggcacagcggactcccctcgaacgctctcgaaatcaagtggctttccagccggcccgctgggccgctcgcccactggaccggtattcccaggccag gccacactgtaccgcaccgcataatcctcgccagatcggcgctgataaggcccaatgtcactccgcaggcgtctatttatgccaaggaccgttcttcttcagctttcggctcgagtatttgttgtgccatgttggttacgatgccaatcgcggtacagttatgcaaatgagcagcgaataccgctcctgacaatgaacggcgtcttgtcatattcatgctgacattcatattcat tcctttggttttttgtcttcgacggactgaaaagtgcggagagaaacccaaaaacagaagcgcgcaaagcgccgttaatatgcgaactcagcgaactcattgagttatcacaacaccatatccatacatatccatatcaatatcaatatcgctattattaacgatcatgctctgctgatcaagtattcagcgctgcgctagattcgacagattgaatcgagctcaatagactca acagactccactcgacagagcgcaatgccaaggacaattgccgtggagtaaacgaggcgtatgcgcaacctgcacctggcggacgcggcgtatgcgcaatgtgcaattcgcttaccttctcgttgcgggtcaggaactcccagatgggaatggccgatgacgagctgatcgaatgtggaaggcgcccagcaggcaagattactttcgccgcagtcgtcatggtgt cgttgctgcttttatgttgcgtactccgcactacacggagagttcaggggattcgtgctccgtgatctgtgatccgtgttccgtgggtcaattgcaggttcggttgtgtaaccttcgtgttctttttttttagggcccaataaaagcgcttttgtggcggcttgatagattatcacttggtttcggtggctagccaagtggctttcttctgtccgacgcacttaattgaattaaccaaac aacgagctggccaattcgtattatcgctgtttacgtgtgtctcagcttgaaacgcaaaagcttgtttcacacatcggtttctcggcaagatgggggagtcagtcggtctagggagaggggcgcccaccagtcgatcacgaaaacggcgaattccaagcaaacggaaacggagcgagcactatagtactatgtcgaacaaccgatcgcggcgatgtcagtgagtcgtc ttcggacagcgctggcgctccacacgtatttaagctctgagatcggctttgggagagcgcagagagcgccatcgcacggcaggcgaaagcggcagtgagcgaaagcgagcggcagcgggtgggggatcgggagccccccgaaaaaaacagaggcgcacgtcgatgccatcggggaattggaacctcaatgtgtgggaatgtttaaatattctgtgttaggta gtgtagtttcaagactatagattctcatacagattgagtccttcgagccgattatacacgacagcaaaatatttcagtcgcgcttgggcaaaaggcttaagcacgactcccagtccccccttacatttgtcttcctaagcccctggagccactatcaaacttttctacgcttgcactgaaaatagaaccaaagtaaacaatcaaaaagaccaaaaacaataacaaccagcacc gagtcgaacatcagtgaggcattgcaaaaatttcaaagtcaagtttgcgtcgtcatcgcgtctgagtccgatcaagccggcttgtaattgaagttgttgatgagttactggattgtggcgaattctggtcagcatacttaacagcagcccgctaattaagcaaaataaacatatcaaattccagaatgcgacggcgccatcatcctgtttgggaattcaattcgcgggcagtc gtttaattcaattaaaaggtagaaaagggagcagaagaatgcgatcgctggaatttcctaacatcacggaccccataaatttgataagcccgagctcgctgcgttgagtcagccaccccacatccccaaatccccgccaaaagaagacactgggttgttgactcgccagattgattgcagtggagtggacctggtcaaagaagcaccgttaatgtgctgattccattcg attccatccgggaatgcgataaagaaaggctctgatccaagcaactgcaatccggatttcgattttctcttccatttggttttgtatttacgtacnnnnnnnnhjhjhjhjhjhjhjcbashyudtsscfs\xnbxncjuauxvxuaafxgxjbxnvxfaquixaxbahxvybvbbnvnbvbnvbnvvvbnvvbnvnbvagdqsgddqsaachjdchxklCV XOIDUQUIFUYVFJHVFJHVEDFQWW;ODJKJBFDJHFJFKJHFKJHFKJEHFEHFKJEHFKJHWFHFKJHEFaagcattctaatgaagacttggagaagacttacgttatattcagaccatcgtgcgatagaggatgagtcatttccatatggccgaaatttattatgtttactatcgtttttagaggtgttttttggattac caaaagaggcatttgttttcttcaactgaaaagatatttaaattttttcttggaccattttcaaggttccggatatatttgaaacacactagctagcagtgttggtaagttacatgtatttctataatgtcatattcctttgtccgtttcaaatcgaatactccacatctcttgtacttgaggaattggcgatcgtagcgatttcccccgccgtaaagttcctgatcctcgttgtttttgtacatc ataaagtccggattctgctcgtcgccgaagatgggaacgaagctgccaaagcgagagtctgcttgaggtgctggtcgtcccagctggataaccttgctgtacagatcggcatctgcctggagggcacgatcgaaatccttccagtggacgaacttcacctgctcgctgggaatagcgttgttgtcaagcagctcaaggagcgtttcgagttgacgggctgcaccacg ctgctccttcgctggggattcccctgcgggtaagcgccgcttgcttggactcgtttccaaatcccatagccacgccagcagaggagtaacagagctcwhereisthegenetgattaaaaatatccttaagaaagcccatgggtataacttactgcgtcctatgcgaggaatggtctttaggttctttatggcaaagttctcgcctcgcttgcccagccgcggtacgttctt ggtgatctttaggaagaatcctggactactgtcgtctgcctggctttggccacaagacccaccaagagcgaggactgttatgattctcatgctgatgcgactgaagcttcacctgactcctgctccacaattggtggcctttatatagcgagatccacccgcatcttgcgtggaatagaaatgcgggtgactccaggattagcattatcgatcggaaagtgataaaactgaa ctaacctgacctaaatgcctggccataattaagtgcatacatacacattacattacttacatttgtataagaactaaattttatagtacataccacttgcgtatgtaaatgcttgtttttctcttatatacgttttataacccagcatattttacgtaaaaacaaaacggtaatgcgaacataacttatttattggggcccggaccgcaaaccggccaaacgcgtttgcacccataaaaa cataagggcaacaaaaaaattgttaagtgttgtttatttttgcaatcgaaacgctcaaatagctgcgatcactcgggagcagggtaaagtcgcctcgaaacaggaagctgaagcatcttctataaatacactcaaagcgatcattccgaggcgagtctggttagaaatttacatggacgcaaaaaggtatagccccacaaaccacatcgctgcgtttcggcagctaattgc cttttagaaattattttcccatttcgagaaactcgtgtgggatgccggatgcggctttcaatcacttctggcccgggatcggattgggtcacattgtcgcgggctctattgtctcgatccgcggcgcagttcgcgtgcttagcggtcagaaaggcagagattcggttcggattgatgcgctggcagcagggcacaaagatctaatgactggcaaatcgctacaaataaatta aagtccggcggctaattatgagcggactgaagccactttggattaaccaaaaaacagcagataaacaaaaacggcaaagaaaattgccacagagttgtcacgctttgttgcacaaacatttgtgcagaaaagtgaaaagcttttagccattattaagtttttcctcagtcgctggcagcacttgcgaatgtactgatgttcctcataaatgaaaattaatgtttgctctacgct ccaccgaactcgcttgtttgggggattggctggctaatcgcggctagatcccaggcggtataaccttttcgcttcatcagttgtgaaccagatggctggtgttttggcacagcggactcccctcgaacgctctcgaaatcaagtggctttccagccggcccgctgggccgctcgcccactggaccggtattcccaggccaggccacactgtaccgcaccgcataatcct cgccagatcggcgctgataaggcccaatgtcactccgcaggcgtctatttatgccaaggaccgttcttcttcagctttcggctcgagtatttgttgtgccatgttggttacgatgccaatcgcggtacagttatgcaaatgagcagcgaataccgctcctgacaatgaacggcgtcttgtcatattcatgctgacattcatattcattcctttggttttttgtcttcgacggactga aaagtgcggagagaaacccaaaaacagaagcgcgcaaagcgccgttaatatgcgaactcagcgaactcattgagttatcacaacaccatatccatacatatccatatcaatatcaatatcgctattattaacgatcatgctctgctgatcaagtattcagcgctgcgctagattcgacagattgaatcgagctcaatagactcaacagactccactcgacagagcgcaat gccaaggacaattgccgtggagtaaacgaggcgtatgcgcaacctgcacctggcggacgcggcgtatgcgcaatgtgcaattcgcttaccttctcgttgcgggtcaggaactcccagatgggaatggccgatgacgagctgatcgaatgtggaaggcgcccagcaggcaagattactttcgccgcagtcgtcatggtgtcgttgctgcttttatgttgcgtactccgc actacacggagagttcaggggattcgtgctccgtgatctgtgatccgtgttccgtgggtcaattgcaggttcggttgtgtaaccttcgtgttctttttttttagggcccaataaaagcgcttttgtggcggcttgatagattatcacttggtttcggtggctagccaagtggctttcttctgtccgacgcacttaattgaattaaccaaacaacgagctggccaattcgtattatcgct gtttacgtgtgtctcagcttgaaacgcaaaagcttgtttcacacatcggtttctcggcaagatgggggagtcagtcggtctagggagaggggcgcccaccagtcgatcacgaaaacggcgaattccaagcaaacggaaacggagcgagcactatagtactatgtcgaacaaccgatcgcggcgatgtcagtgagtcgtcttcggacagcgctggcgctccacacgt atttaagctctgagatcggctttgggagagcgcagagagcgccatcgcacggcaggcgaaagcggcagtgagcgaaagcgagcggcagcgggtgggggatcgggagccccccgaaaaaaacagaggcgcacgtcgatgccatcggggaattggaacctcaatgtgtgggaatgtttaaatattctgtgttaggtagtgtagtttcaagactatagattctcatac agattgagtccttcgagccgattatacacgacagcaaaatatttcagtcgcgcttgggcaaaaggcttaagcacgactcccagtccccccttacatttgtcttcctaagcccctggagccactatcaaacttttctacgcttgcactgaaaatagaaccaaagtaaacaatcaaaaagaccaaaaacaataacaaccagcaccgagtcgaacatcagtgaggcattgcaa aaatttcaaagtcaagtttgcgtcgtcatcgcgtctgagtccgatcaagccggcttgtaattgaagttgttgatgagttactggattgtggcgaattctggtcagcatacttaacagcagcccgctaattaagcaaaataaacatatcaaattccagaatgcgacggcgccatcatcctgtttgggaattcaattcgcgggcagtcgtttaattcaattaaaaggtagaaaagg gagcagaagaatgcgatcgctggaatttcctaacatcacggaccccataaatttgataagcccgagctcgctgcgttgagtcagccaccccacatccccaaatccccgccaaaagaagacactgggttgttgactcgccagattgattgcagtggagtggacctggtcaaagaagcaccgttaatgtgctgattccattcgattccatccgggaatgcgataaagaaa ggctctgatccaagcaactgcaatccggatttcgattttctcttccatttggttttgtatttacgtacaagcattctaatgaagacttggagaagacttacgttatattcagaccatcgtgcgatagaggatgagtcatttccatatggccgaaatttattatgtttactatcgtttttagaggtgttttttggattaccaaaagaggcatttgttttcttcaactgaaaagatatttaaattttt tcttggaccattttcaaggttccggatatatttgaaacacactagctagcagtgttggtaagttacatgtatttctataatgtcatattcctttgtccgtttcaaatcgaatactccacatctcttgtacttgaggaattggcgatcgtagcgatttcccccgccgtaaagttcctgatcctcgttgtttttgtacatcataaagtccggattctgctcgtcgccgaagatgggaacgaag ctgccaaagcgagagtctgcttgaggtgctggtcGacatcgctgcgtttcggcagctaattgccttttagaaattattttcccatttcgagaaactcgtgtgggatgccggatgcggctttcaatcacttctggcccgggatcggattgggtcacattgtcgcgggctctattgtctcgatccgcggcgcagttcgcgtgcttagcggtcagaaaggcagagattcggttcg gattgatgcgctggcagcagggcacaaagatctaatgactggcaaatcgctacaaataaattaaagtccggcggctaattatgagcggactgaagccactttggattaaccaaaaaacagcagataaacaaaaacggcaaagaaaattgccacagagttgtcacgctttgttgcacaaacatttgtgcagaaaagtgaaaagcttttagccattattaagtttttcctcag tcgctggcagcacttgcgaatgtactgatgttcctcataaatgaaaattaatgtttgctctacgctccaccgaactcgcttgtttgggggattggctggctaatcgcggctagatcccaggcggtataaccttttcgcttcatcagttgtgaaccagatggctggtgttttggcacagcggactcccctcgaacgctctcgaaatcaagtggctttccagccggcccgctggg ccgctcgcccactggaccggtattcccaggccaggccacactgtaccgcaccgcataatcctcgccagatcggcgctgataaggcccaatgtcactccgcaggcgtctatttatgccaaggaccgttcttcttcagctttcggctcgagtatttgttgtgccatgttggttacgatgccaatcgcggtacagttatgcaaatgagcagcgaataccgctcctgacaatgaa cggcgtcttgtcatattcatgctgacattcatattcattcctttggttttttgtcttcgacggactgaaaagtgcggagagaaacccaaaaacagaagcgcgcaaagcgccgttaatatgcgaactcagcgaactcattgagttatcacaacaccatatccatacatatccatatcaatatcaatatcgctattattaacgatcatgctctgctgatcaagtattcagcgctgcgct agattcgacagattgaatcgagctcaatagactcaacagactccactcgacagagcgcaatgccaaggacaattgccgtggagtaaacgaggcgtatgcgcaacctgcacctggcggacgcggcgtatgcgcaatgtgcaattcgcttaccttctcgttgcgggtcaggaactcccagatgggaatggccgatgacgagctgatcgaatgtggaaggcgcccag caggcaagattactttcgccgcagtcgtcatggtgtcgttgctgcttttatgttgcgtactccgcactacacggagagttcaggggattcgtgctccgtgatctgtgatccgtgttccgtgggtcaattgcaggttcggttgtgtaaccttcgtgttctttttttttagggcccaataaaagcgcttttgtggcggcttgatagattatcacttggtttcggtggctagccaagtggct ttcttctgtccgacgcacttaattgaattaaccaaacaacgagctggccaattcgtattatcgctgtttacgtgtgtctcagcttgaaacgcaaaagcttgtttcacacatcggtttctcggcaagatgggggagtcagtcggtctagggagaggggcgcccaccagtcgatcacgaaaacggcgaattccaagcaaacggaaacggagcgagcactatagtactatgtc gaacaaccgatcgcggcgatgtcagtgagtcgtcttcggacagcgctggcgctccacacgtatttaagctctgagatcggctttgggagagcgcagagagcgccatcgcacggcaggcgaaagcggcagtgagcgaaagcgagcggcagcgggtgggggatcgggagccccccgaaaaaaacagaggcgcacgtcgatgccatcggggaattggaacct caatgtgtgggaatgtttaaatattctgtgttaggtagtgtagtttcaagactatagattctcatacagattgagtccttcgagccgattatacacgacagcaaaatatttcagtcgcgcttgggcaaaaggcttaagcacgactcccagtccccccttacatttgtcttcctaagcccctggagccactatcaaacttttctacgcttgcactgaaaatagaaccaaagtaaaca atcaaaaagaccaaaaacaataacaaccagcaccgagtcgaacatcagtgaggcattgcaaaaatttcaaagtcaagtttgcgtcgtcatcgcgtctgagtccgatcaagccggcttgtaattgaagttgttgatgagttactggattgtggcgaattctggtcagcatacttaacagcagcccgctaattaagcaaaataaacatatcaaattccagaatgcgacggcgc atccttaagaaagcccatgggtataacttactgcgtcctatgcgaggaatggtctttaggttctttatggcaaagttctcgcctcgcttgcccagccgcggtacgttcttggtgatctttaggaagaatcctggactactgtcgtctgcctggctttggccacaagacccaccaagagcgaggactgttatgattctcatgctgatgcgactgaagcttcacctgactcctgctc cacaattggtggcctttatatagcgagatccacccgcatcttgcgtggaatagaaatgcgggtgactccaggattagcattatcgatcggaaagtgataaaactgaactaacctgacctaaatgcctggccataattaagtgcatacatacacattacattacttacatttgtataagaactaaattttatagtacataccacttgcgtatgtaaatgcttgtttttctcttatatacg ttttataacccagcatattttacgtaaaaacaaaacggtaatgcgaacataacttatttattggggcccggaccgcaaaccggccaaacgcgtttgcacccataaaaacataagggcaacaaaaaaattgttaagtgttgtttatttttgcaatcgaaacgctcaaatagctgcgatcactcgggagcagggtaaagtcgcctcgaaacaggaagctgaagcatcttctata aatacactcaaagcgatcattccgaggcgagtctggttagaaatttacatggacgcaaaaaggtatagccccacaaaccacatcgctgcgtttcggcagctaattgccttttagaagtatttgttgtgccatgttggttacgatgccaatcgcggtacagttatgcaaatgagcagcgaataccgctcctgacaatgaacggcgtcttgtcatattcatgctgacattcatatt Bioinformatics Machine Learning By Dr. Khaled Wassif Slide Where is the gene (DNA)?

Evolution Machine Learning By Dr. Khaled Wassif Slide ancestor T years ?

Artificial intelligence: Artificial intelligence: –Learning symbolic representations of concepts. –Using prior knowledge with training data to guide learning. Probability theory: Probability theory: –Computing probabilities of hypotheses and functions. –Algorithms for estimating values of unobserved variables. Computational complexity theory: Computational complexity theory: –Bounds on inherent complexity of learning, e.g. time, data. Control theory: Control theory: –Learning to control processes to optimize performance measures. Relevant Disciplines Machine Learning By Dr. Khaled Wassif Slide 1- 17

Information theory: Information theory: –Measuring information entropy and content. Philosophy: Philosophy: –Analysis of the justification for generalizing beyond observed data. Psychology and neurobiology: Psychology and neurobiology: –Practice improves performance. –Motivating artificial neural network models of learning. Statistics: Statistics: –Characterization of errors (e.g., bias and variance) that occur when estimating the accuracy of a hypothesis. Relevant Disciplines (cont.) Machine Learning By Dr. Khaled Wassif Slide 1- 18

A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. A computer program is said to learn from experience E with respect to some class of tasks T and performance measure P, if its performance at tasks in T, as measured by P, improves with experience E. --- Tom Mitchell Machine Learning : Machine Learning : –Computer program automatically improves »at task T »according to performance measure P »through experience E What is the Learning Problem? Machine Learning By Dr. Khaled Wassif Slide 1- 19

T: Playing checkers T: Playing checkers P: Percentage of games won against an arbitrary opponent P: Percentage of games won against an arbitrary opponent E: Playing practice games against itself E: Playing practice games against itself T: Recognizing hand-written words T: Recognizing hand-written words P: Percentage of words correctly classified P: Percentage of words correctly classified E: Database of human-labeled images of handwritten words E: Database of human-labeled images of handwritten words T: Driving on four-lane highways using vision sensors T: Driving on four-lane highways using vision sensors P: Average distance traveled before a human-judged error P: Average distance traveled before a human-judged error E: A sequence of images and steering commands recorded while observing a human driver. E: A sequence of images and steering commands recorded while observing a human driver. T: Categorize messages as spam or acceptable. T: Categorize messages as spam or acceptable. P: Percentage of messages correctly classified. P: Percentage of messages correctly classified. E: Database of s, some with human-given labels E: Database of s, some with human-given labels Machine Learning Examples Machine Learning By Dr. Khaled Wassif Slide 1- 20

Part of problem specification: Part of problem specification: –T: Play checkers –P: Percent of games won in world competition Within our control: Within our control: –What experience? »Choose the training experience –What exactly should be learned? »Choose the target function –How shall it be represented? »Choose the target function representation –What specific algorithm to learn it? Designing a Learning System E.g. Learning to Play Checkers Machine Learning By Dr. Khaled Wassif Slide 1- 21

Direct or indirect feedback? Direct or indirect feedback? –Direct : Given board states + best move for that state –Indirect: Given move sequences + outcome of game »Infer goodness of a move by whether game won or lost and its contribution. Teacher or not? Teacher or not? –Teacher gives learner board states + best move –Learner asks teacher for best move for particular state –Learner plays against itself and has no teacher, only feedback from environment –If teacher exists, how nice is it? Is training experience representative of performance goal? Is training experience representative of performance goal? –Will the data (games) seen in training reflect those seen in practice? Types of Training Experience Machine Learning By Dr. Khaled Wassif Slide 1- 22

What target function (target concept) is to be learned and how it will be used by the performance system? What target function (target concept) is to be learned and how it will be used by the performance system? –For checkers, assume we are given a function for generating the legal moves for a given board position and want to decide the best move. »Could learn a function: ChooseMove(board, legal-moves) → best-move ChooseMove(board, legal-moves) → best-move (might work with direct info, difficult with indirect) (might work with direct info, difficult with indirect) »Or could learn an evaluation function, V(board) → R, that gives each board position a score for how favorable it is. V can be used to pick a move by applying each legal move, scoring the resulting board position, and choosing the move that results in the highest scoring board position. V can be used to pick a move by applying each legal move, scoring the resulting board position, and choosing the move that results in the highest scoring board position. (better choice for indirect info: learn value of board states) (better choice for indirect info: learn value of board states) Choosing the Target Function Machine Learning By Dr. Khaled Wassif Slide 1- 23

–If b is a final winning board, then V(b) = 100 –If b is a final losing board, then V(b) = –100 –If b is a final draw board, then V(b) = 0 –Otherwise, then V(b) = V(b' ) where b' is the highest scoring final board position that is achieved starting from b and playing optimally until the end of the game (assuming the opponent plays optimally as well). »Can be computed using complete mini-max search of the game tree. Gives correct results, but is non operational, i.e. not efficiently computable. Gives correct results, but is non operational, i.e. not efficiently computable. –It involves searching the complete exponential game tree. Need to learn an operational approximation ( ) to the evaluation function. Need to learn an operational approximation ( ) to the evaluation function. Possible Definition for V(b) Machine Learning By Dr. Khaled Wassif Slide 1- 24

Target function can be represented in many ways: Target function can be represented in many ways: –lookup table –collection of rules –numerical function »Polynomial function of board features –neural network. There is a trade-off between the expressiveness of a representation and the ease of learning. There is a trade-off between the expressiveness of a representation and the ease of learning. The more expressive a representation, the better it will be at approximating an arbitrary function; however, the more examples will be needed to learn an accurate function. The more expressive a representation, the better it will be at approximating an arbitrary function; however, the more examples will be needed to learn an accurate function. Representing the Target Function Machine Learning By Dr. Khaled Wassif Slide 1- 25

For learner to approximate V, need to give the learner a set of training examples. For learner to approximate V, need to give the learner a set of training examples. –each training example is an ordered pair of the form:. Uses training values for the target function to induce a hypothesized definition that fits these examples and hopefully generalizes to unseen examples. Uses training values for the target function to induce a hypothesized definition that fits these examples and hopefully generalizes to unseen examples. In statistics, learning to approximate a continuous function is called regression. In statistics, learning to approximate a continuous function is called regression. Attempts to minimize some measure of error (such as mean squared error). Attempts to minimize some measure of error (such as mean squared error). Choosing a Learning Algorithm Machine Learning By Dr. Khaled Wassif Slide 1- 26

Summary of Design Choices Machine Learning By Dr. Khaled Wassif Slide 1- 27

Summary of Design Choices (cont.) Machine Learning By Dr. Khaled Wassif Slide 1- 28

Evaluation of Learning Systems Experimental Experimental –Conduct controlled cross-validation experiments to compare various methods on a variety of benchmark datasets. –Gather data on their performance, e.g. test accuracy, training-time, testing-time. –Analyze differences for statistical significance. Theoretical Theoretical –Analyze algorithms mathematically and prove theorems about their: »Computational complexity »Ability to fit training data »Sample complexity (number of training examples needed to learn an accurate function) Machine Learning By Dr. Khaled Wassif Slide 1- 29

What algorithms can approximate functions well (and when)? What algorithms can approximate functions well (and when)? How does number of training examples influence accuracy? How does number of training examples influence accuracy? –[How well will it generalize given a training set of particular size?] How does complexity of hypothesis representation impact it? How does complexity of hypothesis representation impact it? How does noisy data [noise in examples and labels] influence accuracy? How does noisy data [noise in examples and labels] influence accuracy? What are the theoretical limits of learnability? What are the theoretical limits of learnability? How can prior knowledge of learner help? How can prior knowledge of learner help? –Choose set of candidate hypotheses and choose search procedure. What evidence can we get from biological learning systems? What evidence can we get from biological learning systems? –E.g. artificial neural networks, evolutionary algorithms How can systems alter their own representations? How can systems alter their own representations? –E.g. choosing from among different types of classifiers, learning to learn Some Issues in Machine Learning Machine Learning By Dr. Khaled Wassif Slide 1- 30

Learning can be viewed as using direct or indirect experience to approximate a chosen target function. Learning can be viewed as using direct or indirect experience to approximate a chosen target function. Function approximation can be viewed as a search through a space of hypotheses (representations of functions) for one that best fits a set of training data. Function approximation can be viewed as a search through a space of hypotheses (representations of functions) for one that best fits a set of training data. Different learning methods assume different hypothesis spaces (representation languages) and/or employ different search techniques. Different learning methods assume different hypothesis spaces (representation languages) and/or employ different search techniques. Some Issues in Machine Learning Machine Learning By Dr. Khaled Wassif Slide 1- 31

Various Function Representations Numerical functions Numerical functions –Linear regression –Neural networks –Support vector machines Symbolic functions Symbolic functions –Decision trees –Rules in propositional logic –Rules in first-order predicate logic Instance-based functions Instance-based functions –Nearest-neighbor –Case-based Probabilistic Graphical Models Probabilistic Graphical Models –Naïve Bayes –Bayesian networks –Hidden-Markov Models (HMMs) –Probabilistic Context Free Grammars (PCFGs) –Markov networks Machine Learning By Dr. Khaled Wassif Slide 1- 32

Various Search Algorithms Gradient descent Gradient descent –Perceptron –Backpropagation Dynamic Programming Dynamic Programming –HMM Learning –PCFG Learning Divide and Conquer Divide and Conquer –Decision tree induction –Rule learning Evolutionary Computation Evolutionary Computation –Genetic Algorithms (GAs) –Genetic Programming (GP) –Neuro-evolution Machine Learning By Dr. Khaled Wassif Slide 1- 33

History of Machine Learning 1950s 1950s –Samuel’s checker player –Selfridge’s Pandemonium 1960s: 1960s: –Neural networks: Perceptron –Pattern recognition –Learning in the limit theory –Minsky and Papert prove limitations of Perceptron 1970s: 1970s: –Symbolic concept induction –Winston’s arch learner –Expert systems and the knowledge acquisition bottleneck –Quinlan’s ID3 –Michalski’s AQ and soybean diagnosis –Scientific discovery with BACON –Mathematical discovery with AM Machine Learning By Dr. Khaled Wassif Slide 1- 34

History of Machine Learning (cont.) 1980s: 1980s: –Advanced decision tree and rule learning –Explanation-based Learning (EBL) –Learning and planning and problem solving –Utility problem –Analogy –Cognitive architectures –Resurgence of neural networks (connectionism, backpropagation) –Valiant’s PAC Learning Theory –Focus on experimental methodology 1990s 1990s –Data mining –Adaptive software agents and web applications –Text learning –Reinforcement learning (RL) –Inductive Logic Programming (ILP) –Ensembles: Bagging, Boosting, and Stacking –Bayes Net learning Machine Learning By Dr. Khaled Wassif Slide 1- 35

History of Machine Learning (cont.) 2000s 2000s –Support vector machines –Kernel methods –Graphical models –Statistical relational learning –Transfer learning –Sequence labeling –Collective classification and structured outputs –Computer Systems Applications »Compilers – Debugging – Graphics »Security (intrusion, virus, and worm detection) – management –Personalized assistants that learn –Learning in robotics and vision Machine Learning By Dr. Khaled Wassif Slide 1- 36

Machine Learning seeks to develop theories and computer systems for Machine Learning seeks to develop theories and computer systems for –representing; –classifying, clustering and recognizing; –reasoning under uncertainty; –predicting; –and reacting to –… complex, real world data, based on the system's own experience with data, and (hopefully) under a unified model or mathematical framework, that –can be formally characterized and analyzed –can take into account human prior knowledge –can generalize and adapt across data and domains –can operate automatically and autonomously –and can be interpreted and perceived by human. Technical Definition of M.L. Machine Learning By Dr. Khaled Wassif Slide 1- 37

How can we build computer systems that automatically improve with experience, and what laws govern learning in general? How can we build computer systems that automatically improve with experience, and what laws govern learning in general? Statistics: Statistics: –What can be inferred from data plus a set of modeling assumptions, with what reliability? Computer Science: Computer Science: –How can we build computers to solve problems, and which problems are inherently tractable/intractable? Animal & Human Learning: Animal & Human Learning: –What mechanisms explain learning in animals, and what teaching strategies are most effective? The Challenge Machine Learning By Dr. Khaled Wassif Slide 1- 38

Applications: Applications: –Intelligent agents –Text analysis –Cell biology –Marketing –Brain imaging –Robotics –Counter-Terrorism –Online tutoring systems –Computer vision – … Machine Learning Research Machine Learning By Dr. Khaled Wassif Slide Core Issues: Core Issues: – Transfer learning –Learning from labeled and unlabeled data –Graphical prob. models –Privacy-preserving data mining –Mixed-initiative learning –Active learning –Time series models –Never-ending learning –…

Modern techniques Modern techniques –Probabilistic graphical models »Bayesian network, dynamic Bayesian network, Markov random fields… –Kernel methods »Support vector machines, kernel PCA, all kinds of kernel machines … –Spectral graph analysis »Normalized cuts, spectral clustering … –Markov decision processes (MDPs) and POMDPs »read Sutton & Barto's positioning book, check out Ng, Dietterich, Parr, Littman … –Metric learning, manifold learning, embedding, source separation »Too many to list –Hierarchical Bayesian models, nonparametric Bayesian analysis »Gaussian processes, Dirichlet processes –Probabilistic relational models »PRM, BLOG –… Major Technical Paradigms Machine Learning By Dr. Khaled Wassif Slide 1- 40

Resources: Datasets UCI Repository: UCI Repository: UCI KDD Archive: UCI KDD Archive: Statlib: Statlib: Delve: Delve: WEKA: WEKA: Machine Learning By Dr. Khaled Wassif Slide 1- 41

Resources: Journals Journal of Machine Learning Research Journal of Machine Learning Research Machine Learning Machine Learning Neural Computation Neural Computation Neural Networks Neural Networks IEEE Transactions on Neural Networks IEEE Transactions on Neural Networks IEEE Transactions on Pattern Analysis and Machine Intelligence IEEE Transactions on Pattern Analysis and Machine Intelligence Annals of Statistics Annals of Statistics Journal of the American Statistical Association Journal of the American Statistical Association Machine Learning By Dr. Khaled Wassif Slide 1- 42

Resources: Conferences International Conference on Machine Learning (ICML) International Conference on Machine Learning (ICML) –ICML07: European Conference on Machine Learning (ECML) European Conference on Machine Learning (ECML) –ECML08: Neural Information Processing Systems (NIPS) Neural Information Processing Systems (NIPS) –NIPS05: Uncertainty in Artificial Intelligence (UAI) Uncertainty in Artificial Intelligence (UAI) –UAI05: Computational Learning Theory (COLT) Computational Learning Theory (COLT) –COLT05: International Joint Conference on Artificial Intelligence (IJCAI) International Joint Conference on Artificial Intelligence (IJCAI) –IJCAI07: International Conference on Neural Networks (Europe) International Conference on Neural Networks (Europe) –ICANN08: Machine Learning By Dr. Khaled Wassif Slide 1- 43