1. Chapter 2: Pattern Recognition

2. Pattern Recognition
Is currently the dominating are for the application of NN.
It is a large area of computer science in itself.

3. Pattern Recognition problems
Patterns include alphabetic characters, text processing, handwritten text, speech, images, stock market trends, etc.
Classification is one of the simpler pattern recognition tasks.
Simplest classifier – template matching technique – each letter is read into a fixed size frame and the frame is compared to a template of all the possible characters but cannot apply to all sizes.

4. Pattern Recognition – A Definition
The fundamental objective – classification – given an input of some form can we analyse that input to provide a meaningful categorization of its data content?
A pattern recognition system can be considered as a two stage
Feature extraction
Classification
A feature as a measurement taken on the input pattern
feature that provide a definite characteristics of that input type.
Example : to distinguish ‘E’ and ‘F’ – we need to compare the number of vertical and horizontal strokes.

5. Pattern Recognition – A Definition
Often poses the greater part of the recognition problem.
The classifier is supplied with the list of measures features.
Its task is to map these input features onto a classification state – given the input features, the classifier must decide which type of class category they match most closely – i.e. – distance metrics and probability theory.

6. Feature Extraction:
Convert a raw pattern into a feature vector.
Reduce redundancy in the pattern.
e.g. convert image to line drawing.
Classification
Assign the feature vector a class
Pattern space (or feature space) must be partitioned through training.

7. Feature Vectors and Feature Space
Classification is not performed using single measurement or feature.
Usually several measurements to be able to adequately distinguish inputs that belong to different categories.
If we make n measurements on our input pattern, each of which is a unique feature, then we can use algebraic notation to create a set of these features and call it a feature vector.
The dimensionality of the vector – the number of elements in it – creates an n dimensional feature space.

8. Feature Vectors and Feature Space
The simplest way to describe feature space is to consider a simple two-dimensional example – two measurements on the pattern to form the feature vector.
Example – to distinguish between ballet dancers from rugby players – two distinct measures – height and weight – take a series of height and weight measurements then plot a two-dimensional Euclidean plane (x1, x2) that defines our feature spaces – Fig. 2.1.
This plot help us to visualize the concept of our feature space.

9. Discriminant Functions
Are the basis for the majority of pattern recognition techniques.
Fig. 2.1 – rugby-player/ ballet-dancer classification problem – samples form two distinct cluster – we could define boundary/ line to separate them – a decision boundary – discriminating function.
A function that discriminate one class from another class.
The purpose of this function is to decide - any new input falls above line belong to ballet.
measure heights and weights of 12 people.

10. rugby-player/ ballet-dancer classification problem
x rugby player
Weight
x x x
x x x +
+ + +
+ +
Height

11. Discriminant Functions
The function maps our input features onto a classification space.
The simplest function that separate the two clusters is a straight line.
It represents a very widely used category of classifiers known as linear classifiers.

12. Classification Techniques
Pattern classification techniques :
numeric and non-numeric.
Numeric technique –
deterministic and statistical measures – made on the geometric pattern space.
Non-numeric techniques –
based on symbolic processing – i.e. – fuzzy set.
Our concern - the numeric techniques –
followed the same by neural computing since symbolic data manipulation is based on traditional AI approach.

13. Classification Techniques
To implement discriminant function analysis:
Nearest neighbor classification: Distance metrics, Euclidean, City Block, Square distance.
Linear classifier
Statistical techniques: Bayesian classifier
Neural Networks

14. Nearest Neighbor Classification
We wish to decide to which of the two the unclassified pattern, X, belongs.
NNC – based on the shortest distance to the neighboring samples.
We can define DF as:
f(X) = closest(class1) – closest(class2)
f(X) > 0 class2 closest(class2) < closest(class1)
f(X) < 0 class1 closest(class2) > closest(class1)

15. Nearest Neighbor Classification
A pattern is classified as belonging to the class of the training pattern that is closest to it. To measure closeness use a distance metric. For a feature vector x = {x1, x2, x3,.....,xn}
and a training pattern t = {t1, t2, t3,.....,tn}
Euclidean distance: D2=Sumi( (xi - ti)2)
Dot Product Distance: D = Sumi(xi * ti)
Angle between vectors: D = Sumi(xi * ti) /(|xi|*|ti|)

16. Nearest Neighbor Classification
However for rogue pattern, fig. 2.4, that has membership of one class but does in fact lie closer to another class.
Nearest neighbor classification is prone to errors due to rogue patterns. A rogue pattern is a mislabeled pattern.
If our unclassified input is measured against rogue samples, it will invariably result in misclassification.
To overcome rogue pattern – take average measurements or known as “K” nearest neighbor classification.
The “K” is the number of neighboring samples.

17. A Rouge Samples

18. Distance Metrics
Nearest Neighbor pose problem of finding the distance.
Methods of measurement based on distance:
Hamming distance
Euclidean distance
City block distance
Square distance

19. Hamming distance
The most basic measure, and widely used because of its simplicity.
Given two vectors:
X = (x1, x2, …)
Y = (y1, y2, …)
H =
Often used to compare binary vectors.

20. Euclidean distance
One of the most common metrics used.
Fig. 2.5:

21. City block distance (Manhattan)
A simplified version of Euclidean distance measure.
The points of equal distance lie on a square boundary. Fig. 2.6.
Advantage - much faster to compute even though less accuracy (more error).

22. Square distance
Simplifying the Euclidean distance measure still further.
No. summation. And therefore
Fig A larger square boundary than City block.
The error is tolerated as a compromise between speed and accuracy.

23. Linear Classifier A pattern recognition technique.
Encountered time and time again in the field of NN.
Can solve linearly separable problem.
But for non-linear separable problem – such as X-OR problem cannot be used.
Refer Fig. 2.2: How to find boundary-line?

24. To find boundary-line – discriminating function, f(X), f(X) = where
Xi i th. component of an input vector
Wi i th. component of a weight vector
n dimensionality

25. Two-dimension, n = 2
Initial weight vector – fig. 2.8
X = { W1, W2, X1, X2}
Objective is to a final value of Ws so that all input patterns fulfill the following criteria:
X f(X) > 0 then class A
X f(X) < 0 then class B

26. Statistical Techniques
Play a major part in pattern classification.
Analytical technique.
The statistical nature of the data/ the characteristics.
The concept of Bayesian classification.
Relies on the basic statistical theory of probabilities and conditional probabilities.

27. Statistical Techniques
Patterns ( the components of feature vector) – to estimate the likelihood, or probability, of a pattern belonging to a particular class.
Basic definitions:
Let Gi, i = 1,2, …, n be a list of possible group or classes then we can define the probability of a pattern belonging to a class as P(Gi) (where 0  P(Gi)  1).
Using condition probability allows us to include knowledge we already have about the pattern in order to improve our estimate of class membership.