 # Chapter 2 (part 3) Bayesian Decision Theory Discriminant Functions for the Normal Density Bayes Decision Theory – Discrete Features All materials used.

## Presentation on theme: "Chapter 2 (part 3) Bayesian Decision Theory Discriminant Functions for the Normal Density Bayes Decision Theory – Discrete Features All materials used."— Presentation transcript:

Chapter 2 (part 3) Bayesian Decision Theory Discriminant Functions for the Normal Density Bayes Decision Theory – Discrete Features All materials used in this course were taken from the textbook “Pattern Classification” by Duda et al., John Wiley & Sons, 2001 with the permission of the authors and the publisher

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 2 Discriminant Functions for the Normal Density We saw that the minimum error-rate classification can be achieved by the discriminant function g i (x) = ln P(x |  i ) + ln P(  i ) Case of multivariate normal 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 3 Case  i =  2. I ( I stands for the identity matrix) 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 4 –A classifier that uses linear discriminant functions is called “a linear machine” –The decision surfaces for a linear machine are pieces of hyperplanes defined by: g i (x) = g j (x) 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 5 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 6 –The hyperplane separating R i and R j always orthogonal to the line linking the means! 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 7 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 8 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 9 Case  i =  (covariance of all classes are identical but arbitrary!) –Hyperplane separating R i and R j (the hyperplane separating R i and R j is generally not orthogonal to the line between the means!) 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 10 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 11 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 12 Case  i = arbitrary –The covariance matrices are different for each category (Hyperquadrics which are: hyperplanes, pairs of hyperplanes, hyperspheres, hyperellipsoids, hyperparaboloids, hyperhyperboloids) 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 13 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 14 6

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 15 Bayes Decision Theory – Discrete Features Components of x are binary or integer valued, x can take only one of m discrete values v 1, v 2, …, v m Case of independent binary features in 2 category problem Let x = (x 1, x 2, …, x d ) t where each x i is either 0 or 1, with probabilities: p i = P(x i = 1 |  1 ) q i = P(x i = 1 |  2 ) 9

Dr. Djamel BouchaffraCSE 616 Applied Pattern Recognition, Chapter 2, Section 2. 16 The discriminant function in this case is: 9

Download ppt "Chapter 2 (part 3) Bayesian Decision Theory Discriminant Functions for the Normal Density Bayes Decision Theory – Discrete Features All materials used."

Similar presentations