1. . EM with Many Random Variables Another Example of EM Sequence Alignment via HMM Lecture # 10 This class has been edited from Nir Friedman’s lecture. Changes made by Dan Geiger, then Shlomo Moran. Background Readings: chapters 11.6, 3.4, 3.5, 4, in the Durbin et al., 2001, Chapter 3.4 Setubal et al., 1997

2. 2 EM for processes with many dice In the previous class we presented the EM algorithm for the case where the parameters are probabilities associated with a single “die” (i.e., probability space/random variable). In practical applications the model may include many dice (like the HMM model). The generalization of the EM algorithm to many dice is rather straightforward, and given next.

3. 3 EM for processes with many dice The model is defined by the parameters (random variables, or dice) and the simple events. Let the random variables be Z l (l =1,...,r), Z l has m l values z l,1,...z l,m l with probabilities {q lk |k=1,...,m l }. Each simple event y corresponds to a sequence of outcomes (z l 1, k 1,...,z l n, k n ) of the random variables used in y. Let N lk (y) = #(z lk appears in y).

4. Define N lk as the expected value of N lk (y), given x and θ: N lk =E(N lk |x,θ) = ∑ y p(y|x,θ) N lk (y), Then we have: 4 EM for processes with many dices Similarly to the single die case, we have:

5. 5 L  (λ) for processes with many dices N lk

6. 6 EM algorithm for processes with many dice Maximization step Set λ lk =N lk / (∑ k’ N lk’ ) Similarly to the one dice case we get: Expectation step Set N lk to E (N lk (y)|x,θ), ie: N lk = ∑ y p(y|x,θ) N lk (y)

7. 7 EM algorithm for n independent observations x 1,…, x n : Expectation step It can be shown that, if the x j are independent, then:

8. 8 EM – One More Example The DNA of species in planet Melmek contains two letters: A and B. In the evolutionary process in Melmek, a mutation of a letter occurs in two steps: first the letter may be deleted, and in case the letter is not deleted it can be changed to the 2 nd letter. The unknown probabilities of these events are respectively There are two species in Melmek, S (for Son) and its direct ancestor F (Father). Scientists were able to deduce that the DNA of F contained a sequence of two letters "AX", where "X" is equally likely to be A or B (Prob(X=A) = Prob(X=B) = 0.5). 1. Describe the probability space defined by the evolution of the two letters "AX" in F into a sequence (of up to two letters) in S: a.Write down all the "simple events". b.For each event write its probability (as a function of ), c. and its contribution to the six statistics used by the EM algorithm.

9. 9 Example (cont) For writing the simple events, we assume the following order of “dice tossing” : (a)decide if the initial sequence is AA or AB (with probability 0.5 each) (b)Delete/replace the right letter (one or two tossing) (c)Same for left letter. For instance, a simple event of deletion of two letters is:

10. 10 Example (cont) Two more simple events which start with AA: There are altogether 18 simple events: 9 which start with AA and 9 which start with AB

11. 11 Example (cont) 2.Later information showed that the sequence "AX" of F evolved into the sequence "A" in S. For given, write down the probability of the above scenario of evolution (of "AX" in F evolving into "A" in S).

12. 12 Example (cont) 3. Write a single round of the EM algorithm to estimate the parameters which maximize the likelihood of the above scenario, starting with arbitrary initial parameters Show that the outcome is independent on the initial parameters. Regardless the initial parameters: del and remain have exactly the same count, hence they receive each probability 0.5 Also, A→A and B→A both get probability 1. Calculate the counts of each outcome in each simple event:

13. 13 Example (end) 4.What are the parameters which maximize the likelihood of the above scenario? Justify your answer. Solution 2: The above is parameters are obtained after each iteration of the EM algorithm. By the EM correctness theorem, this must be the (unique) maximum.

14. 14 EM for other discrete stochastic processes Where the experiment (x,y) is generated by a general “stochastic process”. The only assumption we make is that the outcome of each experiment consists of a (finite) sequence of samplings of r discrete random variables (dices) Z 1,..., Z r, each of the Z i ‘s can be sampled few times. This can be realized by a probabilistic acyclic state machine, where at each state some Z i is sampled, and the next state is determined by the outcome – until a final state is reached. The EM algorithm is applicable to a general scenario in which we wish to maximize p(x| )=∑ y p(x,y| ).

15. 15 EM in Practice Initial parameters: u Random parameters setting u “Best” guess from other source Stopping criteria: u Small change in likelihood of data u Small change in parameter values Avoiding bad local maxima: u Multiple restarts u Early “pruning” of unpromising ones

16. 16 Sequence Comparison using HMM We now use HMM to extend such log-odds scoring functions to alignments which may contain gaps (indels). Recall: We used log-odds scoring functions for gapless alignments, as follows: s(a,b)= log(p ab / q a q b ), where p ab and q a are the probabilities of the “Match” and “Random” models.

17. 17 Sequence alignment using HMM Each “output symbol” of the HMM is an aligned pair of two letters, or of a letter and a gap. Example: Insertion of a first gap in this model: 17 … XM (G,T) … (C,-) We still need to assign transition/emission probabilities

18. 18 Example: Insertion of a first gap in this model: … XM (G,T) … (C,-) We still need to assign transition/emission probabilities

19. 19 Need to define the hidden states, and the transition and emission probabilities, which define the probability of each aligned pair of sequences. Given two input sequences, we look for an alignment of these sequences of maximum probability.

20. 20 Hidden states and emitted symbols “Hidden” States u Match. u Insertion in x. u insertion in y. Symbols emitted u Match: {(a,b)| a,b in ∑ }. u Insertion in x: {(a,-)| a in ∑ }. u Insertion in y: {(-,a)| a in ∑ }.

21. 21 Transitions and Emission Probabilities ε01- ε 0ε δδ 1-2 δ MX X M Y Y Emission Probabilities u Match: (a,b) with p ab – only from M states u Insertion in x: (a,-) with q a – only from X state u Insertion in y: (-,a).with q a - only from Y state. (Note that the hidden states can be reconstructed from the alignment.) Transitions probabilities (note the forbidden ones).  δ = probability for 1 st gap  ε = probability for tailing gap.

22. 22 Scoring alignments For each pair of sequences x (of length m) and y (of length n), there are many alignments of x and y, each corresponds to a different state-path (the lengths of the paths are between max{m,n} and m+n). Given the transmission and emission probabilities, each alignment has a defined score – the product of the corresponding probabilities. An alignment is “most probable” if it maximizes this score.

23. 23 Finding the most probable alignment Let v M (i,j) be the probability of the most probable alignment of x(1..i) and y(1..j), which ends with a match. Similarly, v X (i,j) and v Y (i,j), the probabilities of the most probable alignment of x(1..i) and y(1..j), which ends with an insertion to x or y. Then using a recursive argument, we get:

24. 24 Most probable alignment By similar argument for v X (i,j) and v Y (i,j), the probabilities of the most probable alignment of x(1..i) and y(1..j), which ends with an insertion to x or y, are:

25. 25 The Probability Space Different alignments of x and y may have different lengths, so the probability space which we used earlier, for HMM of a fixed length L, is not applicable to this alignment HMM model. However, there is a probability space which contains all infinite sequence alignments (finite alignments are compound events in this model). The algorithm of the previous slides compute the correct probability of each alignment in this probability space. Another approach is to define a probability space which contains all alignments of finite length. In the following we adapt our algorithm to this model.

26. 26 Adding termination probabilities Each other state has a transition probability τ to the END state. This results in an expected sequence length of 1/ τ. MXY END M 1-2δ -τ δδτ X 1-ε -τ ε τ Y ετ END 1 Probability space for all finite alignments is obtained by adding an END state, which denotes the end of the alignment The last transition in each alignment is to the END state, with probability τ

27. 27 The log-odds scoring function u We wish to compare the “model” alignment score to the “random” alignment score. u For gapless alignments we used the log-odds ratio: s(a,b) = log (p ab / q a q b ). u To adapt this for the HMM model, we need to model random sequence by HMM, with end state.

28. 28 scoring function for random model XYEND X1- ηη0 Y0 η END001 The transition probabilities for the random model, with termination probability η: (x is the start state) The emission probability for a is q a. Thus the probability of x (of length n) and y (of length m) being random is: And the corresponding score is:

29. 29 Markov Matrices for “Random” and “Model” XYEND X1- ηη Y η END 1 MXY M1-2δ -τ δδτ X1-ε -τ ε τ Y ετ END 1 “Model” “Random”

30. 30 Combining models in the log-odds scoring function In order to compare the M score to the R score of sequences x and y, we can find an optimal M score, and then subtract from it the R score. This is insufficient when we look for local alignments, where the optimal substrings in the alignment are not known in advance. A better way: 1. Define a log-odds scoring function which keeps track of the difference Match-Random scores of the partial strings during the alignment. 2. At the end add to the score (logτ – 2logη) to compensate for the end transitions in both models. We get the following:

31. 31 The log-odds scoring function And at the end add to the score (logτ – 2logη). (assuming that letters at insertions/deletions are selected by the random model)

32. 32 The log-odds scoring function Another way, with uniform scoring for the M state (Durbin et. al., Chapter 4.1): Define scoring function s with penalties d and e for a first gap and a tailing gap, resp. Then modify the algorithm to correct for extra prepayment, as follows:

33. 33 Log-odds alignment algorithm Initialization: V M (0,0)=logτ - 2logη. Termination: V = max{ V M (m,n), V X (m,n)+c, V Y (m,n)+c} Where c= log (1-2δ-τ) – log(1-ε-τ)

34. 34 Total probability of x and y Rather then computing the probability of the most probable alignment, we look for the total probability that x and y are related by our model. Let f M (i,j) be the sum of the probabilities of all alignments of x(1..i) and y(1..j), which end with a match. Similarly, f X (i,j) and f Y (i,j) are the sum of these alignments which end with insertion to x (y resp.). A “forward” type algorithm for computing these functions. Initialization: f M (0,0)=1, f X (0,0)= f Y (0,0)=0 (we start from M, but we could select other initial state).

35. 35 Total probability of x and y (cont.) The total probability of all alignments is: P(x,y|model)= f M [m,n] + f X [m,n] + f Y [m,n]