1 HMM - Part 2 Review of the last lecture The EM algorithm Continuous density HMM

2 Three Basic Problems for HMMs Given an observation sequence O=(o 1,o 2,…,o T ), and an HMM =(A,B,  ) –Problem 1: How to compute P(O| ) efficiently ?  The forward algorithm –Problem 2: How to choose an optimal state sequence Q=(q 1,q 2,……, q T ) which best explains the observations?  The Viterbi algorithm –Problem 3: How to adjust the model parameters =(A,B,  ) to maximize P(O| ) ?  The Baum-Welch (forward-backward) algorithm cf. The segmental K-means algorithm maximizes P(O, Q * | ) P(up, up, up, up, up| )?

3 The Forward Algorithm The forward variable: –Probability of o 1,o 2,…,o t being observed and the state at time t being i, given model λ The forward algorithm

4 The Viterbi Algorithm 1.Initialization 2.Induction 3.Termination 4.Backtracking is the best state sequence

5 The Segmental K-means Algorithm Assume that we have a training set of observations and an initial estimate of model parameters –Step 1 : Segment the training data The set of training observation sequences is segmented into states, based on the current model, by the Viterbi Algorithm –Step 2 : Re-estimate the model parameters –Step 3: Evaluate the model If the difference between the new and current model scores exceeds a threshold, go back to Step 1; otherwise, return

6 Segmental K-means vs. Baum-Welch

7 The Backward Algorithm The backward variable: –Probability of o t+1,o t+2,…,o T being observed, given the state at time t being i and model The backward algorithm cf.

8 otot The Forward-Backward Algorithm Relation between the forward and backward variables (Huang et al., 2001)

9 The Baum-Welch Algorithm (1/3) Define two new variables:  t (i)= P(q t = i | O, ) –Probability of being in state i at time t, given O and  t ( i, j )=P(q t = i, q t+1 = j | O, ) –Probability of being in state i at time t and state j at time t+1, given O and

10 The Baum-Welch Algorithm (2/3)  t (i)= P(q t = i | O, ) –Probability of being in state i at time t, given O and  t ( i, j )=P(q t = i, q t+1 = j | O, ) –Probability of being in state i at time t and state j at time t+1, given O and

11 The Baum-Welch Algorithm (3/3) Re-estimation formulae for , A, and B are How do you know ?

12 Maximum Likelihood Estimation for HMM However, we cannot find the solution directly. An alternative way is to find a sequence: s.t.

13 Jensen’s inequality Q function Solvable and can be proved that If f is a concave function, and X is a r.v., then E[f(X)]≤ f(E[X])

14 The EM Algorithm EM: Expectation Maximization –Why EM? Simple optimization algorithms for likelihood functions rely on the intermediate variables, called latent data For HMM, the state sequence is the latent data Direct access to the data necessary to estimate the parameters is impossible or difficult For HMM, it is almost impossible to estimate ( A, B,  ) without considering the state sequence –Two Major Steps : E step: compute the expectation of the likelihood by including the latent variables as if they were observed M step: compute the maximum likelihood estimates of the parameters by maximizing the expected likelihood found in the E step

15 Three Steps for EM Step 1. Draw a lower bound –Use the Jensen’s inequality Step 2. Find the best lower bound  auxiliary function –Let the lower bound touch the objective function at the current guess Step 3. Maximize the auxiliary function –Obtain the new guess –Go to Step 2 until converge [Minka 1998]

16 objective function current guess Form an Initial Guess of =(A,B,  ) Given the current guess, the goal is to find a new guess such that

17 Step 1. Draw a Lower Bound lower bound function objective function

18 Step 2. Find the Best Lower Bound objective function lower bound function auxiliary function

19 Step 3. Maximize the Auxiliary Function auxiliary function objective function

20 Update the Model objective function

21 Step 2. Find the Best Lower Bound objective function auxiliary function

22 Step 3. Maximize the Auxiliary Function objective function

23 Step 1. Draw a Lower Bound (cont’d) Apply Jensen’s Inequality A lower bound function of If f is a concave function, and X is a r.v., then E[f(X)]≤ f(E[X]) Objective function p(Q): an arbitrary probability distribution

24 Step 2. Find the Best Lower Bound (cont’d) –Find that makes the lower bound function touch the objective function at the current guess

25 Step 2. Find the Best Lower Bound (cont’d) Take the derivative w.r.t and set it to zero

26 Step 2. Find the Best Lower Bound (cont’d) Q function We can check Define

27 EM for HMM Training Basic idea –Assume we have and the probability that each Q occurred in the generation of O i.e., we have in fact observed a complete data pair ( O, Q ) with frequency proportional to the probability P(O,Q| ) –We then find a new that maximizes –It can be guaranteed that EM can discover parameters of model to maximize the log-likelihood of the incomplete data, logP(O| ), by iteratively maximizing the expectation of the log- likelihood of the complete data, logP(O,Q| ) Expectation

28 Solution to Problem 3 - The EM Algorithm The auxiliary function where and can be expressed as

29 Solution to Problem 3 - The EM Algorithm (cont’d) The auxiliary function can be rewritten as wiwi yiyi wjwj yjyj wkwk ykyk example

30 Solution to Problem 3 - The EM Algorithm (cont’d) The auxiliary function is separated into three independent terms, each respectively corresponds to,, and –Maximization procedure on can be done by maximizing the individual terms separately subject to probability constraints –All these terms have the following form

31 Solution to Problem 3 - The EM Algorithm (cont’d) Proof: Apply Lagrange Multiplier Constraint

32 Solution to Problem 3 - The EM Algorithm (cont’d) wiwi yiyi

33 Solution to Problem 3 - The EM Algorithm (cont’d) wjwj yjyj

34 Solution to Problem 3 - The EM Algorithm (cont’d) wkwk ykyk

35 Solution to Problem 3 - The EM Algorithm (cont’d) The new model parameter set can be expressed as:

36 Discrete vs. Continuous Density HMMs Two major types of HMMs according to the observations –Discrete and finite observation: The observations that all distinct states generate are finite in number, i.e., V={v 1, v 2, v 3, ……, v M }, v k  R L In this case, the observation probability distribution in state j, B={b j (k)}, is defined as b j (k)=P(o t =v k |q t =j), 1  k  M, 1  j  N o t : observation at time t, q t : state at time t  b j (k) consists of only M probability values –Continuous and infinite observation: The observations that all distinct states generate are infinite and continuous, i.e., V={v| v  R L } In this case, the observation probability distribution in state j, B={b j (v)}, is defined as b j (v)=f(o t =v|q t =j), 1  j  N o t : observation at time t, q t : state at time t  b j (v) is a continuous probability density function (pdf) and is often a mixture of Multivariate Gaussian (Normal) Distributions

37 Gaussian Distribution A continuous random variable X is said to have a Gaussian distribution with mean μ and variance σ 2 (σ>0) if X has a continuous pdf in the following form:

38 Multivariate Gaussian Distribution If X=(X 1,X 2,X 3,…,X d ) is an d -dimensional random vector with a multivariate Gaussian distribution with mean vector  and covariance matrix , then the pdf can be expressed as If X 1,X 2,X 3,…,X d are independent random variables, the covariance matrix is reduced to diagonal, i.e.,

39 Multivariate Mixture Gaussian Distribution An d -dimensional random vector X=(X 1,X 2,X 3,…,X d ) is with a multivariate mixture Gaussian distribution if In CDHMM, b j (v) is a continuous probability density function (pdf) and is often a mixture of multivariate Gaussian distributions Covariance matrix of the k- th mixture of the j- th state Mean vector of the k -th mixture of the j -th state Observation vector

40 Solution to Problem 3 – The Segmental K-means Algorithm for CDHMM Assume that we have a training set of observations and an initial estimate of model parameters –Step 1 : Segment the training data The set of training observation sequences is segmented into states, based on the current model, by Viterbi Algorithm –Step 2 : Re-estimate the model parameters –Step 3: Evaluate the model If the difference between the new and current model scores exceeds a threshold, go back to Step 1; otherwise, return

41 Solution to Problem 3 – The Segmental K-means Algorithm for CDHMM (cont’d) 3 states and 4 Gaussian mixtures per state O1O1 State O2O2 1 2 t OtOt s2s2 s3s3 s1s1 s2s2 s3s3 s1s1 s2s2 s3s3 s1s1 s2s2 s3s3 s1s1 s2s2 s3s3 s1s1 s2s2 s3s3 s1s1 s2s2 s3s3 s1s1 s2s2 s3s3 s1s1 s2s2 s3s3 s1s1 Global mean Cluster 1 mean Cluster 2mean K-means {  11,  11,c 11 } {  12,  12,c 12 } {  13,  13,c 13 }{  14,  14,c 14 }

42 Solution to Problem 3 – The Baum-Welch Algorithm for CDHMM Define a new variable  t (j,k) –Probability of being in state j at time t with the k -th mixture component accounting for o t, given O and Observation-independent assumption

43 Solution to Problem 3 – The Baum-Welch Algorithm for CDHMM (cont’d) Re-estimation formulae for are

44 A Simple Example o1o1 State o2o2 o3o Time S1S1 S2S2 S1S1 S2S2 S1S1 S2S2 The Forward/Backward Procedure

45 A Simple Example (cont’d) Total 8 paths q: q: 1 1 2

46 A Simple Example (cont’d) back