1. Machine Vision and Dig. Image Analysis 1 Prof. Heikki Kälviäinen C50A6100 Lectures 12: Object Recognition Professor Heikki Kälviäinen Machine Vision and Pattern Recognition Laboratory Department of Information Technology Faculty of Technology Management Lappeenranta University of Technology (LUT) Heikki.Kalviainen@lut.fi http://www.lut.fi/~kalviai http://www.it.lut.fi/ip/research/mvpr/

2. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 2 Motivation Task: To recognize an object based on a feature vector Image acquisition => digital image ↓ Preprocessing => better image ↓ Segmentation => basic features ↓ Representation and description => advanced features ↓ Object recognition (classification, clustering)

3. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 3 Content Classification. –Patterns, feature vectors. Clustering. –Pattern classes. Pattern recognition approaches: –Statistical approach. –Structural approach. –Neural approach. Other categorizations: for example, metric and non-metric approaches: –Decision trees, strings, grammars, rule-based methods (logic). Good reference: Duda, R.O., Hart, P.E., Stork, D.G.: Pattern Classification, Wiley, 2001.

4. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 4 Patterns and pattern classes A pattern is an arrangement of descriptors (also called features). A feature vector x = [x_1 x_2 … x_n]^T where x_i is a descriptor (feature). A pattern class is a family of patterns that share some common properties. Pattern classes ω_1, ω_2, …, ω_W where W is the number of classes.

5. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 5 Classification and clustering Feature selection and extraction. Clustering. –To find classes. Classification: –To find class ω _i to where x belongs. Building a classifier: –Learning (training): training set. –Testing: test set.

6. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 6 Statistical pattern recognition Bayesian classification strategies. Based on class samples and class parameters such as the mean and the covariance of the distribution of the features. Decision (discriminant) functions and decision surfaces. Normal (Gaussian) distribution. Parametric estimation: Maximum Likelihood (ML) Estimation, Bayesian estimation. Non-parametric estimation: Parzen windows, k-nn nonparametric estimation (k-nn classifier). Unsupervised learning and clustering: sequential algorithms, hierarchical algorithms, optimization based clustering (e.g., C- means), competitive learning (e.g., Self-Organizing Map, SOM).

7. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 7 Bayesian classification: example eye nostril eye Gaussian mixture model densities (EM estimation) Stability property guarantees approximately the Gaussian form of classes in the feature space. One class may still consist of several sub-clusters (open eye, closed eye, etc.). Bayesian classification of features

8. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 8 Syntactic pattern recognition Grammars: formal grammars, different kinds of grammars, examples how to apply grammars. G = (N, ∑, P, S) where P a set of rewriting rules. Parsing: the Cocke-Younger-Kasami (CYK) parsing algorithm, the CYK table. Tree grammars. Graphs: graph types, graph morphology, graph matching algorithms.

9. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 9 Neural pattern recognition Why do we need neural pattern recognition? Supervised and unsupervised learning: methods for classification and clustering. Supervised learning: MLP algorithm, Backpropagation. Unsupervised learning: the Kohonen net or Self- Organizing Map (SOM) algorithm.

10. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 10 Customer profiles by neural computing: Finnish supermarkets From many itemized receipts to categories of customers

11. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 11 Different customers as different nodes in SOM (similar ones near each other)

12. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 12 Visual inspection on wooden surfaces

13. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 13 Neural computing approach

14. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 14 Applications To be selected: Data. Features. Classifiers. considering the following requirement: Fast. Accurate/reliable. Not too expensive. Easy to implement and use.

15. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 15 Visual quality inspection in steel manufacturing

16. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 16 Vacuum chamber

17. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 17 Bubbling of argon gas in the melt Weak bubbling Good bubbling

18. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 18 Histograms of weak and good bubbling

19. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 19 Hydrogen prediction of vacuum degassing

20. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 20 The user interface of the system

21. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 21 Sorting ceramic tiles: RGB and spectral features

22. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 22 Sorting ceramic tiles: test material Class 1 Class 2 Class 3 …. Class 5 Tile 1Tile 2Tile 3Tile 4Tile 5

23. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 23 Sorting ceramic tiles: classification

24. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 24 Molecular computing: Bacteriorhodopsin-based color sensors Molecule structure Fotocycle

25. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 25 Molecular computing: bacterial camera Matrix elementCamera

26. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 26 Molecular computing: color sensors Responses Self-Organizing Map (SOM)

27. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 27 More examples of applications Medical image processing. –Diabetes and retinal image analysis using machine vision. http://www.it.lut.fi/project/imageret/ Industrial machine vision. –Paper and board printability tests by machine vision in the paper making and printing industry. http://www.it.lut.fi/project/papvision/ Biometrics. –Image-based biometric person authentication. http://www.it.lut.fi/project/facedetect/ For more see, e.g., http://www2.it.lut.fi/mvpr/

28. Machine Vision and Dig. Image Analysis Prof. Heikki Kälviäinen C50A6100 28 Summary Task: To recognize an object based on a feature vector Image acquisition => digital image ↓ Preprocessing => better image ↓ Segmentation => basic features ↓ Representation and description => advanced features ↓ Object recognition (classification, clustering)