1. Hidden Markov Models Usman Roshan CS 675 Machine Learning

2. HMM applications Determine coding and non-coding regions in DNA sequences Separating “bad” pixels from “good” ones in image files Classification of DNA sequences

3. Hidden Markov Models Alphabet of symbols: Set of states that emit symbols from the alphabet: Set of probabilities –State transition: –Emission probabilities:

4. Loaded die problem

5. Loaded die automata F L a FL a LF e F(i) e L(i) a FF a LL

6. Loaded die problem Consider the following rolls: Observed : 21665261 Underlying die :FFLLFLLF What is the probability that the underlying path generated the observed sequence?

7. HMM computational problems Hidden sequence known Hidden sequence unknown Transition and emission probabilities known Model fully specified Viterbi to determine optimal hidden sequence Transition and emission probabilities unknown Maximum likelihood Expected maximization and also known as Baum-Welch

8. Probabilities unknown but hidden sequence known A kl : number of transitions from state k to l E k (b): number of times state k emits symbol b

9. Probabilities known but hidden sequence unknown Problem: Given an HMM and a sequence of rolls, find the most probably underlying generating path. Let be the sequence of rolls. Let V F (i) denote the probability of the most probable path of that ends in state F. (Define V L (i) similarly.)

10. Probabilities known but hidden sequence unknown Initialize : Recurrence: for i=0..n-1

11. Probabilities and hidden sequence unknown Use Expected-Maximization algorithm (also known as EM algorithm) Very popular and many applications For HMMs also called Baum-Welch algorithm Outline: 1.Start with random assignment to transition and emission probabilities 2.Find expected transition and emission probabilities 3.Estimate actual transition and emission probabilities from expected values in previous step 4.Go to step 2 if probabilities not converged

12. HMM forward probabilities Consider the total probability of all hidden sequences under a given HMM. Let f L (i) be the sum of the probabilities of all hidden sequences upto i that end in the state L. Then f L (i) is given by We calculate f F (i) in the same way. We call these forward probabilities: –f(i) = f L (i)+f F (i)+f B (i)

13. HMM backward probabilities Similarly we can calculate backward probabilties b L (i). Let b L (i) be the sum of the probabilities of all hidden sequences from i to the end that start in state L. Then b L (i) is given by We calculate b F (i) in the same way. We call these forward probabilities: –b(i) = b L (i)+b F (i)+b B (i)

14. Baum Welch How do we calculate expected transition and emission probabilities? Consider the fair-loaded die problem. What is the expected transition of fair (F) to loaded (L)? To answer we have to count the number of times F transitions to L in all possible hidden sequences and multiply each by the probability of the hidden sequence

15. Baum Welch For example suppose input is 12 What are the different hidden sequences? What is the probability of each? What is the total probability? What is the probability of all hidden sequences where the first state is F? Can we determine these answers automatically with forward and backward probabilities?

16. Baum Welch General formula for expected number of transitions from state k to l. General formula for expected number of emissions of b from state k. Equations from Durbin et. al., 1998

17. Baum Welch 1.Initialize random values for all parameters 2.Calculate forward and backward probabilities 3.Calculate new model parameters 4.Did the new probabilities (parameters) change by more than 0.001? If yes goto step 2. Otherwise stop.