General Information Course Id: COSC6342 Machine Learning Time: TU/TH 10a-11:30a Instructor: Christoph F. Eick Classroom:AH123 Homepage:

2 What is Machine Learning? Machine Learning is the Machine Learning is the study of algorithms thatstudy of algorithms that improve their performanceimprove their performance at some taskat some task with experiencewith experience Role of Statistics: Inference from a sample Role of Statistics: Inference from a sample Role of Computer science: Efficient algorithms to Role of Computer science: Efficient algorithms to Solve optimization problemsSolve optimization problems Representing and evaluating the model for inferenceRepresenting and evaluating the model for inference

3 Applications of Machine Learning Supervised Learning Supervised Learning ClassificationClassification PredictionPrediction Unsupervised Learning Unsupervised Learning Association AnalysisAssociation Analysis ClusteringClustering Preprocessing and Summarization of Data Preprocessing and Summarization of Data Reinforcement Learning Reinforcement Learning Activities Related to Models Activities Related to Models Learning parameters of modelsLearning parameters of models Choosing/Comparing modelsChoosing/Comparing models …

Prerequisites Background Probabilities Probabilities Distributions, densities, marginalization…Distributions, densities, marginalization… Basic statistics Basic statistics Moments, typical distributions, regressionMoments, typical distributions, regression Basic knowledge of optimization techniques Basic knowledge of optimization techniques Algorithms Algorithms basic data structures, complexity…basic data structures, complexity… Programming skills Programming skills We provide some background, but the class will be fast paced We provide some background, but the class will be fast paced Ability to deal with “abstract mathematical concepts” Ability to deal with “abstract mathematical concepts”

Textbooks Textbook: Ethem Alpaydin, Introduction to Machine Learning, MIT Press, Recommended Textbooks: 1.Christopher M. Bishop, Pattern Recognition and Machine Learning, Tom Mitchell, Machine Learning, McGraw-Hill, 1997.

Grading 3 Exams67-70% Project 18-24% Homeworks 10-15% Attendance 1-2% NOTE: PLAGIARISM IS NOT TOLERATED. Remark: Weights are subject to change

Topics Covered in 2009 (Based on Alpaydin) Topic 1: Introduction Topic 2: Supervised Learning Topic 3: Bayesian Decision Theory (excluding Belief Networks) Topic 4: Using Curve Fitting as an Example to Discuss Major Issues in ML Topic 5: Parametric Model Selection Topic 6: Dimensionality Reduction Centering on PCA Topic 7: Clustering1: Mixture Models, K-Means and EM Topic 8: Non-Parametric Methods Centering on kNN and Density Estimation Topic 9: Clustering2: Density-based Approaches Topic 10: Decision Trees Topic 11: Comparing Classifiers Topic 12: Combining Multiple Learners Topic 13: Linear Discrimination Topic 14: More on Kernel Methods Topic 15: Naive Bayes' and Belief Networks Topic 16: Hidden Markov Models Topic 17: Sampling Topic 18: Reinforcement Learning Topic 19: Neural Networks Topic 20: Computational Learning Theory Remark: Topics 14, 16, 17, 19, and 20 likely will be only briefly covered or skipped---due to the lack of time.

Course Project  The project will center on the application of machine learning techniques to a challenging problem. It will be conducted in the window Feb. 12-April 11. to a challenging problem. It will be conducted in the window Feb. 12-April 11.  You can either conduct some novel experiments by applying machine learning algorithm(s) to a challenging machine learning task or attempt a theoretical algorithm(s) to a challenging machine learning task or attempt a theoretical analysis. analysis.  Findings of the project will be summarized in a report and in a brief presentation. The report must include a short survey of related work with the corresponding list The report must include a short survey of related work with the corresponding list of references. of references.

Tentative ML Spring 2009 Schedule WeekTopic Jan 20 Introduction Jan 27 Supervised Learning/Bayesian Decision Theory Feb. 3 Curve Fitting/Model Estimation---Parametric Approaches Feb. 10 Model Estimation---Parametric Approaches Feb. 17 Parametric Approaches/Clustering1 Feb. 24 Clustering1/Non-param Methods March 3 Non-Param Methods/Exam1 March 10 Clustering2/Dim. Reduction,Decision Trees March 24 Dim. Reduction; DecisionTrees /Exam2 March 31 SVMs/Kernel Methods; Ensemble Methods April 7 Comparing Classifiers/Group1 Presentations April 14 Group2 Presentations/TBDL April 21 Reinforcement Learning/possibly Belief Networks April 28 Review/Exam3 March 31, 2009

Course Elements Total: classes 18 lectures 18 lectures 2-3 classes for review and discussing homework problems 2-3 classes for review and discussing homework problems 2 classes will be allocated for student presentations 2 classes will be allocated for student presentations 3 exams 3 exams homeworks homeworks individual graded individual graded group graded group graded not-graded (solutions will be discussed in lecture 7-9 days later). not-graded (solutions will be discussed in lecture 7-9 days later).

Dates to Remember Dates to rememberEvents March 5, March 26, April 30Exams April 9 and 14Student Project Presentations March 17 /19No class (Spring Break) April 13(Group1)/April 15(Group2) 11p Submit Project Report /Software/…

Exams  Will be open notes/textbook  Will get a review list before the exam  Exams will center (80% or more) on material that was covered in the lecture  There will be a review prior to the second and third exam; first exam will mostly center on basics. center on basics.  Exam scores will be immediately converted into number grades  No sample exams; sorry I haven’t taught this course for a long time…

Other UH-CS Courses with Overlapping Contents 1. COSC 6368: Artificial Intelligence  Strong Overlap: Decision Trees, Bayesian Belief Networks  Medium Overlap: Reinforcement Learning  COSC 6335: Data Mining  Strong Overlap: Decision trees, SVM, kNN, Density- based Clustering based Clustering  Medium Overlap: K-means, Decision Trees, Preprocessing/Exploratory DA, AdaBoost Preprocessing/Exploratory DA, AdaBoost  COSC 6343: Pattern Classification  Medium Overlap: all classification algorithms, feature selection—discusses those topics taking a different perspective. a different perspective.