1. PatReco: Introduction Alexandros Potamianos Dept of ECE, Tech. Univ. of Crete Fall 2004-2005

2. PatReco:Applications  Speech/audio/music/sounds Speech recognition, Speaker verification/id,  Image/video OCR, AVASR, Face id, Fingerpring id, Video segmentation  Text/Language Machine translatoin, document class., lnag mod., text underst.  Medical/Biology Disease diagnosis, DNA sequencing, Gene disease models  Other Data User modeling (books/music), Ling analysis (web), Games

3. Basic Concepts  Why statistical modeling? Variability: differences between two examples of the same class in training Mismatch: differences between two examples of the same class (one in training one in testing)  Learning modes: Supervised learning: class labels known Unsupervised learning: class labels unknown Re-inforced learning: only positive/negative feedback

4. Basic Concepts  Feature selection Separate classes, Low correlation  Model selection Model type, Model order  Prior knowledge E.g., a priori class probability  Missing features/observations  Modeling of time series Correlation in time (model?), segmentation

5. PatReco: Algorithms  Parametric vs Non-Parametric  Supervised vs Unsupervised  Basic Algorithms: Bayesian Non-parametric Discriminant Functions Non-Metric Methods

6. PatReco: Algorithms  Bayesian methods Formulation (describe class characteristics) Bayes classifier Maximum likelihood estimation Bayesian learning Estimation-Maximization Markov models, hidden Markov models Bayesian Nets  Non-parametric Parzen windows Nearest Neighbour

7. PatReco: Algorithms  Discriminant Functions Formulation (describe boundary) Learning: Gradient descent Perceptron MSE=minimum squared error LMS=least mean squares Neural Net generalizations Support vector machines  Non-Metric Methods Classification and Regression Trees String Matching

8. PatReco: Algorithms  Unsupervised Learning: Mixture of Gaussians K-means  Other not-covered Multi-layered Neural Nets Stochastic Learning (Simulated Annealing) Genetic Algorithms Fuzzy Algorithms Etc…

9. PatReco: Problem Solving 1.Data Collection 2.Data Analysis 3.Feature Selection 4.Model Selection 5.Model Training 6.Classification 7.Classifier Evaluation

12. PatReco: Problem Solving 1.Data Collection 2.Data Analysis 3.Feature Selection 4.Model Selection 5.Model Training 6.Classification 7.Classifier Evaluation

16. PatReco: Problem Solving 1.Data Collection 2.Data Analysis 3.Feature Selection 4.Model Selection 5.Model Training 6.Classification 7.Classifier Evaluation

21. PatReco: Problem Solving 1.Data Collection 2.Data Analysis 3.Feature Selection 4.Model Selection 5.Model Training 6.Classification 7.Classifier Evaluation

22. Evaluation  Training Data Set 1234 examples of class 1 and class 2  Testing/Evaluation Data Set 134 examples of class 1 and class 2  Misclassification Error Rate Training: 11.61% (150 errors) Testing: 13.43% (18 errors)  Correct for chance (Training 22%, Testing 26%) Why?