Download presentation

Presentation is loading. Please wait.

Published byCecelia Hillis Modified over 2 years ago

1
HMM II: Parameter Estimation

2
Reminder: Hidden Markov Model Markov Chain transition probabilities: p(S i+1 = t|S i = s) = a st Emission probabilities: p(X i = b| S i = s) = e s (b) S1S1 S2S2 S L-1 SLSL x1x1 x2x2 X L-1 xLxL M M M M TTTT For each integer L>0, this defines a probability space in which the simple events are HMM of length L.

3
Hidden Markov Model for CpG Islands The states: Domain(S i )={+, -} {A,C,T,G} (8 values) In this representation P(x i | s i ) = 0 or 1 depending on whether x i is consistent with s i. E.g. x i = G is consistent with s i =(+,G) and with s i =(-,G) but not with any other state of s i. A-A- T+T+ A T G + G … … … …

4
Most Probable state path S1S1 S2S2 S L-1 SLSL x1x1 x2x2 X L-1 xLxL M M M M TTTT Given an observation sequence x = (x 1,…,x L ), A most probable path s*= (s * 1,…,s * L ) is one which maximizes p(s|x).

5
Viterbi’s algorithm for most probable path s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi For i=1 to L do for each state l : v l (i) = e l (x i ) MAX k {v k (i-1)a kl } ptr i (l)=argmax k {v k (i-1)a kl } [storing previous state for reconstructing the path] Termination: the probability of the most probable path Initialization: v 0 (0) = 1, v k (0) = 0 for k > 0 0 We add the special initial state 0. p(s 1 *,…,s L * ;x 1,…,x l ) =

6
A-A- C-C- T-T- T+T+ A CTT G + G Predicting CpG islands via most probable path: Output symbols: A, C, G, T (4 letters). Markov Chain states: 4 “-” states and 4 “+” states, two for each letter (8 states total). The most probable path (found by Viterbi’s algorithm) predicts CpG islands. Experiment (Durbin et al, p. 60-61) shows that the predicted islands are shorter than the assumed ones. In addition, quite a few “false negatives” are found.

7
Most probable state s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi Given an output sequence x = (x 1,…,x L ), s i is a most probable state (at location i) if: s i =argmax k p(S i =k |x).

8
Finding most probable state 1. The forward algorithm finds {f k (s i ) = P(x 1,…,x i,s i =k): k = 1,...m}. 2. The backward algorithm finds {b k (s i ) = P(x i+1,…,x L |s i =k ): k = 1,...m}. 3. Return {p(S i =k|x) = f k (s i ) b k (s i ) |k=1,...,m}. To Compute for every i simply run the forward and backward algorithms once, and compute {f k (s i ) b k (s i )} for every i, k. s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi f l (i) = p(x 1,…,x i,s i =l ), the probability of a path which emits (x 1,..,x i ) and in which state s i =l. b l (i)= p(x i+1,…,x L,s i =l), the probability of a path which emits (x i+1,..,x L ) and in which state s i =l.

9
Finding the probability that a letter is in a CpG island via the algorithm for most probable state: The probability that the occurrence of G in the i-th location is in a CpG island (+ state) is: ∑ s + p(S i =s + |x) = ∑ s + F(S i =s + )B(S i =s + ) Where the summation is formally over the 4 “+” states, but actually only state G + need to be considered (why?) A-A- C-C- T-T- T+T+ A CTT G + G i

10
Parameter Estimation for HMM An HMM model is defined by the parameters: a kl and e k (b), for all states k,l and all symbols b. Let θ denote the collection of these parameters. s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi lk b a kl ek(b)ek(b)

11
Parameter Estimation for HMM To determine the values of (the parameters in) θ, use a training set = {x 1,...,x n }, where each x j is a sequence which is assumed to fit the model. Given the parameters θ, each sequence x j has an assigned probability p(x j |θ). s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi

12
Maximum Likelihood Parameter Estimation for HMM The elements of the training set {x 1,...,x n }, are assumed to be independent, p(x 1,..., x n |θ) = ∏ j p (x j |θ). ML parameter estimation looks for θ which maximizes the above. The exact method for finding or approximating this θ depends on the nature of the training set used.

13
Data for HMM Possible properties of (the sequences in) the training set: 1.For each x j, the information on the states s j i 2.The size (number of sequences) of the training set S1S1 S2S2 S L-1 SLSL x1x1 x2x2 X L-1 xLxL M M M M TTTT

14
Case 1: ML when state paths are fully known We know the complete structure of each sequence in the training set {x 1,...,x n }. We wish to estimate a kl and e k (b) for all pairs of states k, l and symbols b. By the ML method, we look for parameters θ* which maximize the probability of the sample set: p(x 1,...,x n | θ*) =MAX θ p(x 1,...,x n | θ). s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi

15
Case 1: State paths are fully known s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi Let m kl = |{i: s i-1 =k,s i =l}| (in x j ). m k (b)=|{i:s i =k,x i =b}| (in x j ). For each x j we have:

16
Case 1 s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi By the independence of the x j ’s, p(x 1,...,x n | θ)=∏ j p(x j |θ). Thus, if A kl = #(transitions from k to l) in the training set, and E k (b) = #(emissions of symbol b from state k) in the training set, we have:

17
Case 1 (cont) So we need to find a kl ’s and e k (b)’s which maximize: Subject to:

18
Case 1 Rewriting, we need to maximize:

19
Case 1 (cont) Then we will maximize also F. Each of the above is a simpler ML problem, which is similar to ML parameters estimation for a die, treated next.

20
MLE for n outcomes The MLE is given by the relative frequencies:

21
Apply the ML method to HMM Let A kl = #(transitions from k to l) in the training set. E k (b) = #(emissions of symbol b from state k) in the training set. We need to: s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi

22
Apply to HMM (cont.) We apply the previous technique to get for each k the parameters {a kl |l=1,..,m} and {e k (b)|b Σ}: s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi Which gives the optimal ML parameters

23
Summary of Case 1: State paths are fully known We know the complete structure of each sequence in the training set {x 1,...,x n }. We wish to estimate a kl and e k (b) for all pairs of states k, l and symbols b. Summary: when everything is known, we can find the (unique set of) parameters θ* which maximizes p(x 1,...,x n | θ*) =MAX θ p(x 1,...,x n | θ). s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi

24
Adding pseudo counts in HMM If the sample set is too small, we may get a biased result. In this case we modify the actual count by our prior knowledge/belief: r kl is our prior belief and transitions from k to l. r k (b) is our prior belief on emissions of b from state k. s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi

25
Case 2: State paths are unknown For a given θ we have: p(x 1,..., x n |θ)= p(x 1 | θ) p (x n |θ) (since the x j are independent) s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi For each sequence x, p(x|θ)=∑ s p(x,s|θ), The sum taken over all state paths s which emit x.

26
Case 2: State paths are unknown Thus, for the n sequences (x 1,..., x n ) we have: p(x 1,..., x n |θ)= ∑ p(x 1,..., x n, s 1,..., s n |θ), Where the summation is taken over all tuples of n state paths (s 1,..., s n ) which generate (x 1,..., x n ). For simplicity, we will assume that n=1. s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi (s 1,..., s n )

27
Case 2: State paths are unknown So we need to maximize p(x|θ)=∑ s p(x,s|θ), where the summation is over all the sequences S which produce the output sequence x. Finding θ * which maximizes ∑ s p(x,s|θ) is hard. [Unlike finding θ * which maximizes p(x,s|θ) for a single sequence (x,s).] s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi

28
ML Parameter Estimation for HMM The general process for finding θ in this case is 1.Start with an initial value of θ. 2.Find θ’ so that p(x|θ’) > p(x|θ) 3.set θ = θ’. 4.Repeat until some convergence criterion is met. A general algorithm of this type is the Expectation Maximization algorithm, which we will meet later. For the specific case of HMM, it is the Baum- Welch training.

29
Baum Welch training We start with some values of a kl and e k (b), which define prior values of θ. Then we use an iterative algorithm which attempts to replace θ by a θ * s.t. p( x |θ * ) > p( x |θ) This is done by “imitating” the algorithm for Case 1, where all states are known: s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi

30
Baum Welch training In case 1 we computed the optimal values of a kl and e k (b), (for the optimal θ) by simply counting the number A kl of transitions from state k to state l, and the number E k (b) of emissions of symbol b from state k, in the training set. This was possible since we knew all the states. S i = lS i-1 = k x i-1 = b … … x i = c

31
Baum Welch training When the states are unknown, the counting process is replaced by averaging process: For each edge s i-1 s i we compute the average number of “k to l” transitions, for all possible pairs (k,l), over this edge. Then, for each k and l, we take A kl to be the sum over all edges. S i = ?S i-1 = ? x i-1 = b x i = c ……

32
Baum Welch training Similarly, For each edge s i b and each state k, we compute the average number of times that s i =k, which is the expected number of “k → b” transmission on this edge. Then we take E k (b) to be the sum over all such edges. These expected values are computed by assuming the current parameters θ: S i = ? x i-1 = b

33
A kl and E k (b) when states are unknown A kl and E k (b) are computed according to the current distribution θ, that is: s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi A kl =∑ s A s kl p(s|x,θ), where A s kl is the number of k to l transitions in the sequence s. E k (b)=∑ s E s k (b)p(s|x,θ), where E s k (b) is the number of times k emits b in the sequence s with output x.

34
Baum Welch: step 1a Count expected number of state transitions For each i, k,l, compute the state transitions probabilities by the current θ: s1s1 SiSi sLsL X1X1 XiXi XLXL S i-1 X i-1.. P(s i-1 =k, s i =l | x,θ) For this, we use the forwards and backwards algorithms

35
Finding state probabilities p(s i =k,x) = F k (s i ) B k (s i ) {F k (i) B k (i)} for every i and k are computed by one run of the backward/forward algorithms. s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi XiXi F k (i) = p(x 1,…,x i,s i =k ), the probability that in a path which emits (x 1,..,x i ), state s i =k. B k (i)= p(x i+1,…,x L |s i =k), the probability that a path which emits (x i+1,..,x L ), given that state s i =k.

36
Baum Welch: Step 1a (cont) Claim: By the probability distribution of HMM s1s1 SiSi sLsL X1X1 XiXi XLXL S i-1 X i-1.. (a kl and e l (x i ) are the parameters defined by , and F k (i-1), B k (i) are the forward and backward algorithms)

37
Step 1a: Computing P(s i-1 =k, s i =l | x,θ) P(x 1,…,x L,s i-1 =k,s i =l| ) = P(x 1,…,x i-1,s i-1 =k| ) a kl e l (x i ) P(x i+1,…,x L |s i =l, ) = F k (i-1) a kl e l (x i ) B l (i) Via the forward algorithm Via the backward algorithm s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL S i-1 X i-1 sisi XiXi x p(s i-1 =k,s i =l | x, ) = F k (i-1) a kl e l (x i ) B l (i)

38
Step 1a For each pair (k,l), compute the expected number of state transitions from k to l, as the sum of the expected number of k to l transitions over all L edges :

39
Step 1a for many sequences: When we have n independent input sequences (x 1,..., x n ), then A kl is given by:

40
Baum-Welch: Step 1b count expected number of symbols emissions for state k and each symbol b, for each i where X i =b, compute the expected number of times that X i =b. s1s1 s2s2 s L-1 sLsL X1X1 X2X2 X L-1 XLXL sisi X i =b

41
Baum-Welch: Step 1b For each state k and each symbol b, compute the expected number of emissions of b from k as the sum of the expected number of times that s i = k, over all i’s for which x i = b.

42
Step 1b for many sequences Exercise: when we have n sequences (x 1,..., x n ), the expected number of emissions of b from k is given by:

43
Summary of Steps 1a and 1b: the E part of the Baum Welch training These steps compute the expected numbers A kl of k,l transitions for all pairs of states k and l, and the expected numbers E k (b) of transmitions of symbol b from state k, for all states k and symbols b. The next step is the M step, which is identical to the computation of optimal ML parameters when all states are known.

44
Baum-Welch: step 2 Use the A kl ’s, E k (b)’s to compute the new values of a kl and e k (b). These values define θ *. The correctness of the EM algorithm implies that: p(x 1,..., x n |θ * ) p(x 1,..., x n |θ) i.e, θ * increases the probability of the data This procedure is iterated, until some convergence criterion is met.

Similar presentations

OK

Hidden Markov Models Usman Roshan CS 675 Machine Learning.

Hidden Markov Models Usman Roshan CS 675 Machine Learning.

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google

Ppt on type 1 diabetes mellitus Ppt on contract labour act 1970 malaysia Ppt on transformational leadership Ppt on network layer in computer networks Ppt on web designing software Ppt on air pollution for students Ppt on rbi reforms for the completion Ppt on arunachal pradesh cultured Ppt on power diode rectifier Ppt on series and parallel combination of resistors experiment