1. 236607 Visual Recognition Tutorial1 Markov models Hidden Markov models Forward/Backward algorithm Viterbi algorithm Baum-Welch estimation algorithm Hidden Markov Models Outline

2. 236607 Visual Recognition Tutorial2 Markov Models

3. 236607 Visual Recognition Tutorial3 Observable states: S 1,S 2,…,S N Below we will designate them simply as 1,2,…,N Actual state at time t is q t. q 1,q 2,…,q t,…,q T First order Markov assumption: Stationarity: Observable Markov Models

4. 236607 Visual Recognition Tutorial4 State transition matrix A: where Constraints on : Markov Models

5. 236607 Visual Recognition Tutorial5 States: 1.Rainy (R) 2.Cloudy (C) 3.Sunny (S) State transition probability matrix: Compute the probability of observing SSRRSCS given that today is S. Markov Models: Example

6. 236607 Visual Recognition Tutorial6 Basic conditional probability rule: The Markov chain rule: Markov Models: Example

7. 236607 Visual Recognition Tutorial7 Observation sequence O: Using the chain rule we get: Markov Models: Example

8. 236607 Visual Recognition Tutorial8 What is the probability that the sequence remains in state i for exactly d time units? Exponential Markov chain duration density. What is the expected value of the duration d in state i? Markov Models: Example

9. 236607 Visual Recognition Tutorial9 Markov Models: Example

10. 236607 Visual Recognition Tutorial10 Markov Models: Example Avg. number of consecutive sunny days = Avg. number of consecutive cloudy days = 2.5 Avg. number of consecutive rainy days = 1.67

11. 236607 Visual Recognition Tutorial11 States are not observable Observations are probabilistic functions of state State transitions are still probabilistic Hidden Markov Models

12. 236607 Visual Recognition Tutorial12 N urns containing colored balls M distinct colors of balls Each urn has a (possibly) different distribution of colors Sequence generation algorithm: 1.Pick initial urn according to some random process. 2.Randomly pick a ball from the urn and then replace it 3.Select another urn according a random selection process asociated with the urn 4.Repeat steps 2 and 3 Urn and Ball Model

13. 236607 Visual Recognition Tutorial13 The Trellis STATES N4321N4321 OBSERVATION 1 2 t-1 t t+1 t+2 T-1 T TIME o 1 o 2 o t-1 o t o t+1 o t+2 o T-1 o T

14. 236607 Visual Recognition Tutorial14 N – the number of hidden states Q – set of states Q={1,2,…,N} M – the number of symbols V – set of symbols V ={1,2,…,M} A – the state-transition probability matrix B – Observation probability distribution:  - the initial state distribution:  – the entire model Elements of Hidden Markov Models

15. 236607 Visual Recognition Tutorial15 1.EVALUATION – given observation O=(o 1, o 2,…,o T ) and model, efficiently compute Hidden states complicate the evaluation Given two models 1 and 1, this can be used to choose the better one. 2.DECODING - given observation O=(o 1, o 2,…,o T ) and model find the optimal state sequence q=(q 1, q 2,…,q T ). Optimality criterion has to be decided (e.g. maximum likelihood) “Explanation” of the data. 3.LEARNING – given O=(o 1, o 2,…,o T ), estimate model parameters that maximize Three Basic Problems

16. 236607 Visual Recognition Tutorial16 Problem: Compute P(o 1, o 2,…,o T | ). Algorithm: –Let q=(q 1, q 2,…,q T ) be a state sequence. –Assume the observations are independent: –Probability of a particular state sequence is: –Also, Solution to Problem 1

17. 236607 Visual Recognition Tutorial17 –Enumerate paths and sum probabilities: N T state sequences and O(T) calculations. Complexity: O(T N T ) calculations. Solution to Problem 1

18. 236607 Visual Recognition Tutorial18 Forward Procedure: Intuition STATES N321N321 v a Nk t t+1 TIME a 3k a 1k k

19. 236607 Visual Recognition Tutorial19 Define forward variable as: is the probability of observing the partial sequence such that the state q t is i. Induction: 1.Initialization: 2.Induction: 3.Termination: 4.Complexity: Forward Algorithm

20. 236607 Visual Recognition Tutorial20 Consider the following coin-tossing experiment: –state-transition probabilities equal to 1/3 –initial state probabilities equal to 1/3 Example State 1State2State3 P(H) P(T) 0.5 0.75 0.25 0.75

21. 236607 Visual Recognition Tutorial21 1.You observe O=(H,H,H,H,T,H,T,T,T,T). What state sequence q, is most likely? What is the joint probability,, of the observation sequence and the state sequence? 2.What is the probability that the observation sequence came entirely of state 1? 3.Consider the observation sequence 4.How would your answers to parts 1 and 2 change? Example

22. 236607 Visual Recognition Tutorial22 4.If the state transition probabilities were: How would the new model ’ change your answers to parts 1-3? Example

23. 236607 Visual Recognition Tutorial23 Backward Algorithm STATES N4321N4321 OBSERVATION 1 2 t-1 t t+1 t+2 T-1 T TIME o 1 o 2 o t-1 o t o t+1 o t+2 o T-1 o T

24. 236607 Visual Recognition Tutorial24 Define backward variable as: is the probability of observing the partial sequence such that the state q t is i. Induction: 1. Initialization: 2. Induction: Backward Algorithm

25. 236607 Visual Recognition Tutorial25 Choose the most likely path Find the path (q 1, q 2,…,q T ) that maximizes the likelihood: Solution by Dynamic Programming Define is the highest prob. path ending in state i. By induction we have: Solution to Problem 2

26. 236607 Visual Recognition Tutorial26 Viterbi Algorithm STATES N4321N4321 OBSERVATION 1 2 t-1 t t+1 t+2 T-1 T TIME o 1 o 2 o t-1 o t o t+1 o t+2 o T-1 o T a Nk k a 1k

27. 236607 Visual Recognition Tutorial27 Initialization: Recursion: Termination: Path (state sequence) backtracking: Viterbi Algorithm

28. 236607 Visual Recognition Tutorial28 Estimate to maximize No analytic method because of complexity – iterative solution. Baum-Welch Algorithm (actually EM algorithm) : 1.Let initial model be  2.Compute new based on  and observation O.  f 4.Else set   and go to step  Solution to Problem 3

29. 236607 Visual Recognition Tutorial29 Baum-Welch: Preliminaries

30. 236607 Visual Recognition Tutorial30 Define as the probability of being in state i at time t, and in state j at time t+1. Define as the probability of being in state i at time t, given the observation sequence Baum-Welch: Preliminaries

31. 236607 Visual Recognition Tutorial31 is the expected number of times state i is visited. is the expected number of transitions from state i to state j. Baum-Welch: Preliminaries

32. 236607 Visual Recognition Tutorial32 expected frequency in state i at time (t=1) (expected number of transitions from state i to state j) / expected number of transitions from state i): (expected number of times in state j and observing symbol k) / (expected number of times in state j ): Baum-Welch: Update Rules

33. 236607 Visual Recognition Tutorial33 L. Rabiner and B. Juang. An introduction to hidden Markov models. IEEE ASSP Magazine, p. 4--16, Jan. 1986. Thad Starner, Joshua Weaver, and Alex Pentland Machine Vision Recognition of American Sign Language Using Hidden Markov Model Thad Starner, Alex Pentland Visual Recognition of American Sign Language Using Hidden Markov Models. In International Workshop on Automatic Face and Gesture Recognition, pages 189--194, 1995. http://citeseer.nj.nec.com/starner95visual.html References