1. Bayesian Decision Theory
ECE 471/571 – Lecture 2
Bayesian Decision Theory

2. Pattern Classification
Statistical Approach
Non-Statistical Approach
Supervised
Unsupervised
Decision-tree
Basic concepts:
Baysian decision rule
(MPP, LR, Discri.)
Basic concepts:
Distance
Agglomerative method
Syntactic approach
Parameter estimate (ML, BL)
k-means
Non-Parametric learning (kNN)
Winner-takes-all
LDF (Perceptron)
Kohonen maps
NN (BP)
Mean-shift
Support Vector Machine
Deep Learning (DL)
Dimensionality
Reduction
FLD, PCA
Performance Evaluation
ROC curve (TP, TN, FN, FP)
cross validation
Stochastic Methods
local opt (GD)
global opt (SA, GA)
Classifier Fusion
majority voting
NB, BKS

3. Bayes’ formula (Bayes’ rule)
From domain knowledge
a-priori probability
(prior probability)
Conditional probability density
(pdf – probability density function)
(likelihood)
a-posteriori probability
(posterior probability)
normalization constant
(evidence)

4. pdf examples Gaussian distribution Uniform distribution
Bell curve
Normal distribution
Uniform distribution
Rayleigh distribution

5. Bayes decision rule x Maximum a-posteriori Probability (MAP):
Stock market x
Maximum a-posteriori Probability (MAP):

6. Conditional probability of error

7. Decision regions
The effect of any decision rule is to partition the feature space into c decision regions

8. Overall probability of error
Or unconditional risk, unconditional probability of error 8

9. The conditional risk
Given x, the conditional risk of taking action ai is:
lij is the loss when decide x belongs to class i while it should be j
lij

10. Likelihood ratio - two category classification

11. Zero-One loss

12. Recap Bayes decision rule  maximum a-posteriori probability
Conditional risk
Likelihood ratio
Decision regions  How to calculate the overall probability of error

13. Recap Maximum a-posteriori Probability Likelihood ratio
Overall probability of error