Visual Recognition Tutorial1 Markov models Hidden Markov models Forward/Backward algorithm Viterbi algorithm Baum-Welch estimation algorithm Hidden Markov Models Outline

Visual Recognition Tutorial2 Observable states: 1,2,…,N Observable sequence: q 1,q 2,…,q t,…,q T First order Markov assumption: Stationarity: Markov Models

Visual Recognition Tutorial3 State transition matrix A: where Constraints on : Markov Models

Visual Recognition Tutorial4 States: 1.Rainy (R) 2.Cloudy (C) 3.Sunny (S) State transition probability matrix: Compute the probability of observing SSRRSCS given that today is S. Markov Models: Example

Visual Recognition Tutorial5 Basic conditional probability rule: The Markov chain rule: Markov Models: Example

Visual Recognition Tutorial6 Observation sequence O: Using the chain rule we get: Markov Models: Example

Visual Recognition Tutorial7 What is the probability that the sequence remains in state i for exactly d time units? Exponential Markov chain duration density. What is the expected value of the duration d in state i? Markov Models: Example

Visual Recognition Tutorial8 Markov Models: Example

Visual Recognition Tutorial9 Markov Models: Example Avg. number of consecutive sunny days = Avg. number of consecutive cloudy days = 2.5 Avg. number of consecutive rainy days = 1.67

Visual Recognition Tutorial10 States are not observable Observations are probabilistic functions of state State transitions are still probabilistic Hidden Markov Models

Visual Recognition Tutorial11 N urns containing colored balls M distinct colors of balls Each urn has a (possibly) different distribution of colors Sequence generation algorithm: 1.Pick initial urn according to some random process. 2.Randomly pick a ball from the urn and then replace it 3.Select another urn according a random selection process asociated with the urn 4.Repeat steps 2 and 3 Urn and Ball Model

Visual Recognition Tutorial12 The Trellis STATES N4321N4321 OBSERVATION 1 2 t-1 t t+1 t+2 T-1 T TIME o 1 o 2 o t-1 o t o t+1 o t+2 o T-1 o T

Visual Recognition Tutorial13 N – the number of hidden states Q – set of states Q={1,2,…,N} M – the number of symbols V – set of symbols V ={1,2,…,M} A – the state-transition probability matrix B – Observation probability distribution:  - the initial state distribution:  – the entire model Elements of Hidden Markov Models

Visual Recognition Tutorial14 1.EVALUATION – given observation O=(o 1, o 2,…,o T ) and model, efficiently compute Hidden states complicate the evaluation Given two models 1 and 1, this can be used to choose the better one. 2.DECODING - given observation O=(o 1, o 2,…,o T ) and model find the optimal state sequence q=(q 1, q 2,…,q T ). Optimality criterion has to be decided (e.g. maximum likelihood) “Explanation” of the data. 3.LEARNING – given O=(o 1, o 2,…,o T ), estimate model parameters that maximize Three Basic Problems

Visual Recognition Tutorial15 Problem: Compute P(o 1, o 2,…,o T | ). Algorithm: –Let q=(q 1, q 2,…,q T ) be a state sequence. –Assume the observations are independent: –Probability of a particular state sequence is: –Also, Solution to Problem 1

Visual Recognition Tutorial16 –Enumerate paths and sum probabilities: N T state sequences and O(T) calculations. Complexity: O(T N T ) calculations. Solution to Problem 1

Visual Recognition Tutorial17 Forward Procedure: Intuition STATES N321N321 v a Nk t t+1 TIME a 3k a 1k k

Visual Recognition Tutorial18 Define forward variable as: is the probability of observing the partial sequence such that the state q t is i. Induction: 1.Initialization: 2.Induction: 3.Termination: 4.Complexity: Forward Algorithm

Visual Recognition Tutorial19 Consider the following coin-tossing experiment: –state-transition probabilities equal to 1/3 –initial state probabilities equal to 1/3 1.You observe O=(H,H,H,H,T,H,T,T,T,T). What state Example State 1State2State3 P(H) P(T)

Visual Recognition Tutorial20 1.You observe O=(H,H,H,H,T,H,T,T,T,T). What state sequence q, is most likely? What is the joint probability,, of the observation sequence and the state sequence? 2.What is the probability that the observation sequence came entirely of state 1? 3.Consider the observation sequence 4.How would your answers to parts 1 and 2 change? Example

Visual Recognition Tutorial21 4.If the state transition probabilities were: How would the new model ’ change your answers to parts 1-3? Example

Visual Recognition Tutorial22 Backward Algorithm STATES N4321N4321 OBSERVATION 1 2 t-1 t t+1 t+2 T-1 T TIME o 1 o 2 o t-1 o t o t+1 o t+2 o T-1 o T

Visual Recognition Tutorial23 Define backward variable as: is the probability of observing the partial sequence such that the state q t is i. Induction: 1. Initialization: 2. Induction: Backward Algorithm

Visual Recognition Tutorial24 Choose the most likely path Find the path (q 1, q 2,…,q T ) that maximizes the likelihood: Solution by Dynamic Programming Define is the highest prob. Path ending in state I By induction we have: Solution to Problem 2

Visual Recognition Tutorial25 Viterbi Algorithm STATES N4321N4321 OBSERVATION 1 2 t-1 t t+1 t+2 T-1 T TIME o 1 o 2 o t-1 o t o t+1 o t+2 o T-1 o T a Nk k a 1k

Visual Recognition Tutorial26 Initialization: Recursion: Termination: Path (state sequence) backtracking: Viterbi Algorithm

Visual Recognition Tutorial27 Estimate to maximize No analytic method because of complexity – iterative solution. Baum-Welch Algorithm (actually EM algorithm) : 1.Let initial model be  2.Compute new based on  and observation O.  f 4.Else set   and go to step  Solution to Problem 3

Visual Recognition Tutorial28 Define as the probability of being in state i at time t, and in state j at time t+1. Define as the probability of being in state i at time t, given the observation sequence Baum-Welch: Preliminaries

Visual Recognition Tutorial29 is the expected number of times state i is visited. is the expected number of transitions from state i to state j. Baum-Welch: Preliminaries

Visual Recognition Tutorial30 expected frequency in state i at time (t=1) (expected number of transitions from state i to state j) / expected number of transitions from state i): (expected number of times in state j and observing symbol k) / (expected number of times in state j ): Baum-Welch: Update Rules

Visual Recognition Tutorial31 Covariance of the estimated parameters Convergence rates Properties

Visual Recognition Tutorial32 Continuous density Ergodic State duration Types of HMM

Visual Recognition Tutorial33 Scaling Initial parameters Multiple observation Implementation Issues

Visual Recognition Tutorial34 What is natural distance function? If is large, does it mean that the models are really different? Comparison of HMMs

Visual Recognition Tutorial35 L. Rabiner and B. Juang. An introduction to hidden Markov models. IEEE ASSP Magazine, p , Jan Thad Starner, Joshua Weaver, and Alex Pentland Machine Vision Recognition of American Sign Language Using Hidden Markov Model Thad Starner, Alex Pentland Visual Recognition of American Sign Language Using Hidden Markov Models. In International Workshop on Automatic Face and Gesture Recognition, pages , References