1. Learning Programs Danielle and Joseph Bennett (and Lorelei) 4 December 2007

2. Overview Definitions Types of Learning Why and Why Not? Programming Points Learning Machine Learning Common Applications Questions?

3. Machine Learning, Defined “… the design and development of algorithms and techniques that allow computers to ‘learn’ ” “… these programs develop concepts, infer new concepts from existing concepts and revise incorrect concepts”

4. Key Terms in Machine Learning Knowledge base Noisy inputs Sparse but accurate Symbolic Explanation Based Learning systems (EBL)

5. Types of Learning Supervised Unsupervised Semi-supervised Reinforcement Transduction Learning to learn

6. Why Learning Programs?

7. Differences in learning styles Saves human time Helps make advances in research

8. Why Not Learning Programs? Difficult to program May require advance knowledge Supervision required in some cases Not suited for all tasks Human interaction cannot be eliminated

9. Points to Ponder Algorithm efficiency Programming language Avoid expensive operations (pointers) Limit parsing Don’t copy information Minimize code Limit library functions

10. In a Machine Learning Class… Bayesian theory Modeling conditional probability density functions: regression and classification Artificial neural networks Decision trees Gene expression programming Genetic algorithms Genetic programming Inductive Logic Programming Gaussian process regression Linear discriminant analysis K-nearest neighbor Minimum message length Perceptron Quadratic classifier Radial basis function networks Support vector machines Algorithms for estimating model parameters Dynamic programming Expectation-maximization algorithm Modeling probability density functions through generative models Graphical models including Bayesian networks and Markov Random Fields Generative Topographic Mapping Approximate inference techniques Monte Carlo methods Variational Bayes Variable-order Markov models Variable-order Bayesian networks Loopy belief propagation Optimization Meta-learning (ensemble methods) Boosting Bootstrap aggregating Random forest Weighted majority algorithm Inductive transfer and learning to learn Inductive transfer Reinforcement learning Temporal difference learning

11. Common Applications Natural language processing Syntactic pattern recognition Search engines Medical diagnosis Bioinformatics Cheminformatics Classifying DNA sequences

12. More Common Applications Detecting credit card fraud Stock market analysis (legal) Speech recognition Handwriting recognition Object recognition (in computer vision) Game playing Robot locomotion Intelligent tutoring

13. References http://en.wikipedia.org/wiki/Machine_learning http://www.lisa.org/globalizationinsider/2004/09/teaching_comput.html http://teach-computers.org/ http://hunch.net/?p=290 Learning programs, by Daniel St. Clair http://www.machinelearning.net/

14. Conclusion Definitions Types of Learning Why and Why Not? Programming Points Learning Machine Learning Common Applications

15. Questions?