1. Hidden Markov Models Fundamentals and applications to bioinformatics.

2. Markov Chains Given a finite discrete set S of possible states, a Markov chain process occupies one of these states at each unit of time. The process either stays in the same state or moves to some other state in S. This occurs in a stochastic way, rather than in a deterministic one. The process is memoryless and time homogeneous.

3. Transition Matrix Let S={S 1, S 2, S 3 }. A Markov Chain is described by a table of transition probabilities such as the following: S1S1 S2S2 S3S3 S1S1 010 S2S2 1/32/30 S3S3 1/21/31/6 S1S1 S3S3 S2S2 1 1/32/3 1/6 1/3 1/2

4. A simple example Consider a 3-state Markov model of the weather. We assume that once a day the weather is observed as being one of the following: rainy or snowy, cloudy, sunny. We postulate that on day t, weather is characterized by a single one of the three states above, and give ourselves a transition probability matrix A given by:

5. - 2 - Given that the weather on day 1 is sunny, what is the probability that the weather for the next 7 days will be “sun-sun-rain-rain-sun- cloudy-sun”?

6. - 3 - Given that the model is in a known state, what is the probability it stays in that state for exactly d days? The answer is Thus the expected number of consecutive days in the same state is So the expected number of consecutive sunny days, according to the model is 5.

7. Elements of an HMM What if each state does not correspond to an observable (physical) event? What if the observation is a probabilistic function of the state? An HMM is characterized by the following: 1) N, the number of states in the model. 2) M, the number of distinct observation symbols per state. 3) the state transition probability distribution where 4) the observation symbol probability distribution in state q j,, where b j (k) is the probability that the k-th observation symbol pops up at time t, given that the model is in state E j. 5) the initial state distribution

8. Three Basic Problems for HMMs 1) Given the observation sequence O = O 1 O 2 O 3 …O t, and a model m = (A, B, p), how do we efficiently compute P(O | m)? 2) Given the observation sequence O and a model m, how do we choose a corresponding state sequence Q = q 1 q 2 q 3 …q t which is optimal in some meaningful sense? 3) How do we adjust the model parameters to maximize P(O | m)?

9. Solution to Problem (1) Given an observed output sequence O, we have that This calculation involves the sum of N T multiplications, each being a multiplication of 2T terms. The total number of operations is on the order of 2T N T. Fortunately, there is a much more efficient algorithm, called the forward algorithm.

10. The Forward Algorithm It focuses on the calculation of the quantity which is the joint probability that the sequence of observations seen up to and including time t is O 1,…,O t, and that the state of the HMM at time t is E i. Once these quantities are known,

11. … continuation The calculation of the  (t, i)’s is by induction on t. From the formula we get

12. Backward Algorithm Another approach is the backward algorithm. Specifically, we calculate  (t, i) by the formula Again, by induction one can find the  (t,i)’s starting with the value t = T – 1, then for the value t = T – 2, and so on, eventually working back to t = 1.

13. Solution to Problem (2) Given an observed sequence O = O 1,…,O T of outputs, we want to compute efficiently a state sequence Q = q 1,…,q T that has the highest conditional probability given O. In other words, we want to find a Q that makes P[Q | O] maximal. There may be many Q’s that make P[Q | O] maximal. We give an algorithm to find one of them.

14. The Viterbi Algorithm It is divided in two steps. First it finds max Q P[Q | O], and then it backtracks to find a Q that realizes this maximum. First define, for arbitrary t and i,  (t,i) to be the maximum probability of all ways to end in state S i at time t and have observed sequence O 1 O 2 …O t. Then max Q P[Q and O] = max i  (T,i)

15. - 2 - But Since the denominator on the RHS does not depend on Q, we have We calculate the  (t,i)’s inductively.

16. - 3 - Finally, we recover the q i ’s as follows. Define and put: This is the last state in the state sequence desired. The remaining q t for t < T are found recursively by defining and putting

17. Solution to Problem (3) We are given a set of observed data from an HMM for which the topology is known. We wish to estimate the parameters in that HMM. We briefly describe the intuition behind the Baum- Welch method of parameter estimation. Assume that the alphabet M and the number of states N is fixed at the outset. The data we use to estimate the parameters constitute a set of observed sequences {O (d) }.

18. The Baum-Welch Algorithm We start by setting the parameters p i, a ij, b i (k) at some initial values. We then calculate, using these initial parameter values: 1) p i * = the expected proportion of times in state S i at the first time point, given {O (d) }.

19. - 2 - 2) 3) where N ij is the random number of times q t (d) =S i and q t+1 (d) = S j for some d and t; N i is the random number of times q t (d) = S i for some d and t; and N i (k) equals the random number of times q t (d) = S i and it emits symbol k, for some d and t.

20. Upshot It can be shown that if = (p i, a jk, b i (k)) is substituted by * = (p i *, a jk *, b i * (k)) then P[{O (d) }| * ] P[{O (d) }| ], with equality holding if and only if * =. Thus successive iterations continually increase the probability of the data, given the model. Iterations continue until a local maximum of the probability is reached.