1. Jamal Saboune - CRV10 Tutorial Day 1 Bayesian state estimation and application to tracking Jamal Saboune jsaboune@site.uottawa.ca VIVA Lab - SITE - University of Ottawa

2. Jamal Saboune - CRV10 Tutorial Day 2 Dynamic state estimation A dynamic process described using a number of random variables (state variables) The evolution of the variables follows a model Indication on all or some of the variables (observation) Evaluate at time t in a recursive manner (using t-1) the process represented by its state vector X t, given the history of observations Y t 2

3. Jamal Saboune - CRV10 Tutorial Day 3 Dynamic state estimation The Markov process is defined by its transition model and the initial state vector: The observation is defined by the observation model : X t = f t (X t-1,Π t ) X 0 Y t = h t (X t,v t )

4. Jamal Saboune - CRV10 Tutorial Day 4 Dynamic state estimation- Bayesian approach Estimate the a posteriori probability density function P(X t / Y t ) using the transition model, the observation model and the probability density function P(X t-1 / Y t-1 )

5. Jamal Saboune - CRV10 Tutorial Day 5 Kalman filter Probability density propagation = Theoretical solution not an analytical one Particular case : The observation and process noises distributions are Gaussian + The transition and observation functions are linear  The probability density functions are Gaussian mono-modal

6. Jamal Saboune - CRV10 Tutorial Day 6 Kalman filter A number of equations using the transition/observation functions and covariance matrices  Optimal estimation of the state vector Minimizes the mean square error between the estimated state vector X’ t and the ‘real’ state vector X t E[(X’ t - X t ) 2 ] given the history of observations Y t Extended Kalman Filter (EKF) is the non linear version of the KF = The transition and observation function can be non-linear 6

7. Jamal Saboune - CRV10 Tutorial Day 7 Kalman filter example

8. Jamal Saboune - CRV10 Tutorial Day Condensation algorithm (Isard, Blake 98) Multimodal and non Gaussian probability densities Model the uncertainty Each possible configuration of the state vector is represented by a ‘particle' The likelihood of a certain configuration is called ‘weight’ The posterior (a posteriori) density is represented using N ‘weighted’ particles 8

9. Jamal Saboune - CRV10 Tutorial Day Selection Prediction Measure Particles at t-1 Likelihood function Chosen particle CONDENSATION – time t 9

10. Jamal Saboune - CRV10 Tutorial Day 10 Tracking of a hand movement using an edge detector Condensation algorithm (Isard, Blake 98)

11. Jamal Saboune - CRV10 Tutorial Day 11 Hands and head movement tracking using color models and optical flow (Tung et al. 2008) Condensation algorithm for tracking

12. Jamal Saboune - CRV10 Tutorial Day 12 Head tracking with contour models (Zhihong et al. 2002) Condensation algorithm for tracking

13. Jamal Saboune - CRV10 Tutorial Day13 Interval Particle Filtering for 3D motion capture (Saboune et al. 05,07,08) 3D humanoid model adapted to the height of the person 32 degrees of freedom to simulate the human movement Find the best fitting 3D model configuration 13

14. Jamal Saboune - CRV10 Tutorial Day14 Interval Particle Filtering for 3D motion capture (Saboune et al. 05,07,08) Modify the Condensation algorithm and adapt it to the human motion tracking  Good estimation using a reduced number of particles 14

15. Jamal Saboune - CRV10 Tutorial Day15 Particle Filtering for multi-targets tracking Joint state vector for all targets and joint likelihood function (Isard, MacCormick 2001, Zhao, Nevatia 2004) Multiple particle filters (one/target) and combined global likelihood function (Koller-Maier 2001) The Explorative particle filtering for 3D people tracking (Saboune, Laganiere 09) 15

16. Jamal Saboune - CRV10 Tutorial Day16 Q & A 16