1 Image Classification MSc Image Processing Assignment March 2003

2 Summary Introduction Classification using neural networks Perceptron Multilayer perceptron Applications

3 Introduction Definition Assignment of a physical object to one of several pre-specified categories Unsupervised Supervised For more details See Image Processing course

4 Classification Supervised Pattern recognition Algebraic Unsupervised k-means Fuzzy k-mean ParametricNon-parametricNeural nets Bayes Minimum distance K-nearest neighbour Decision trees SVM Classification

5 Neural nets Inspired by the human brain Useful for Classification Regression Optimization …

6 Model x=(x 1 …x n ) input vector w=(w 0 …w n ) weight vector f activation function x1x1 xnxn wnwn w1w1 f  y=f(  w i x i + w 0 )

7 Perceptron f=sign 1 w 1 x 1 +w 2 x 2 +w 0 =0 2 inputs

8 Perceptron (2) Example: AND function x1x1 x2x x1x1 x2x2 w 2 =1 w 1 =1 sign  w 0 =1 w 1 =1 x2x2 -1+x1+x2=0 + - x1x1

9 Algorithm Minimise set of misclassified examples Gradient ascent Converges if data linearly separable Demo Perceptron (3)

10 Perceptron (4) XOR problem Problem when Data non-linearly separable Solution: change activation function For more details Matlab classification toolbox

11 Multilayer Perceptron (MLP) Able to model complex non-linear functions Hidden layers with neurons Backpropagation algorithm inputs outputs

12 MLP (2) f=sigmoid w0w0 w1w1 w2w2 y x1x1 x2x2

13 MLP demo Matlab Classification Toolbox Handwritten digits classification Discriminate between 10 digits

14 MLP demo (2) Pre-processing Feature extraction Choice of neural network Training Test For more details See our program 10 neurons OUTPUTOUTPUT FEATURESFEATURES 8 features Input layer 1 st hidden layer Output layer 2 nd hidden layer

15 MLP performance Able to model complex, nonlinear mapping and classification Can be trained by examples, no mathematical description needed In practice, shows good results

16 MLP limitations Extensive training data must be available Computation time Curse of dimensionality Generalisation Overfitting To go further See Neural Network Toolbox, demo on generalisation

17 A few applications Medicine Defence Radar & Sonar Finance …

18 Thank you.