1. A discriminant function for 2-class problem can be defined as the ratio of class likelihoods
g(x) = p(x|C1)/p(x|C2)
Derive formula for g(x) when class likelihoods are Gaussian

2. A discriminant function for 2-class problem is defined as the
Bayesian odds ratio
g(x) = log(P(C1|x)/P(C2|x))
Derive a formula for g(x) when class likelihoods are Gaussian

3. Discriminant functions for 2-class problem are defined as
gi(x) = log(P(Ci|x)) i=1,2
Derive a formula for gi(x) when class likelihoods are Gaussian
Derive a formula for Bayesian discriminant points by setting g1(x) = g2(x) and solving for x

4. Duda and Hart model applied to 2-class problem
R(a1|x) = l11 P(C1|x) + l12 P(C2|x)
R(a2|x) = l21 P(C1|x) + l22 P(C2|x)
l11 = l22 = 0 No cost for correct decisions
l12 = 10, and l21 = 1 Cost incorrect assignment to C1 is ten times greater than cost of incorrect assignment to C2
Posteriors are normalized
Derive the classification rule in terms of P(C1|x)

5. Example: Bernoulli distribution
Two states, failure & success, x is {0,1}
P (x) = pox (1 – po ) (1 – x)
po = probability of success
L (po|X) = log( ∏t poxt (1 – po ) (1 – xt) )
Show that po = ∑t xt / N = successes/trial
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 5

6. A simple example constrained optimization using Lagrange multipliers
find the stationary point of
f(x1, x2) = 1 - x12 – x22
subject to the constraint
g(x1, x2) = x1 + x2 - 1 = 0

7. X is a training set for a classification problem with K>2 classes
xt is a scalar, rt is Boolean vector
Class likelihoods are assumed to be Gaussian
Write formulas for MLEs of priors, means and variance in terms
of ni, the number of examples in class i

8. For a 1D, 2-class problem with Gaussian class likelihoods,
derive the functional form of P(C1|x) when the following
are true:
(1) variances and priors are equal,
(2) posteriors are normalized
Hint: start with the ratio of posteriors to eliminate priors
and evidence
posterior
likelihood
prior
evidence

9. Maximum likelihood estimation of g(x,q)
Log Likelihood
simplify this log-likelihood function as much as possible
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 9

10. Independent Attributes
If xi are independent, off-diagonals of ∑ are 0,
p(x) reduces to a product of probabilities for each component
Simpliy this expression when d=2
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 10

11. 2 attributes and 2 classes
What can we say from this graph about the value
of r in the covariance matrices of the 2 classes?
Lecture Notes for E Alpaydın 2010 Introduction to Machine Learning 2e © The MIT Press (V1.0) 11

12. 2 attributes and 2 classes:
What is the value of r in each example?

13. 2 attributes 2 classes
Same mean different
Covariance
One contour shown
for each class likelihood
Discriminant is dark blue
Describe the covariance
matrices in each case