1. OBJECT TRACKING USING PARTICLE FILTERS

2. Table of Contents Tracking Tracking Tracking as a probabilistic inference problem Tracking as a probabilistic inference problem Applications of Tracking Applications of Tracking Different approaches for Object Tracking Different approaches for Object Tracking Particle Filter Particle Filter A Simple Particle Filter Algorithm A Simple Particle Filter Algorithm Basic steps implemented in the project Basic steps implemented in the project Files used in the project Files used in the project Demo Demo

3. TRACKING Tracking is the problem of generating an inference about the motion of an object given a sequence of images. In a typical tracking problem, we have a model for the object’s motion and some set of measurements from a sequence of images.

4. The measurements could be the positions of some image points, the positions and moments of some image regions etc. Tracking is an inference problem. The moving object has some form of internal state, which is measured at each frame. We need to combine our measurements as effectively as possible to estimate the object’s state.

5. Tracking as a probabilistic inference problem Prediction : Prediction : P(X i | Y 0 =y 0, …………..,Y i-1 =y i-1 ). P(X i | Y 0 =y 0, …………..,Y i-1 =y i-1 ). Data Association: Data Association: P(X i | Y 0 =y 0, …………..,Y i-1 =y i-1 ). P(X i | Y 0 =y 0, …………..,Y i-1 =y i-1 ). Correction: Correction: P(X i | Y 0 =y 0, …………..., Y i =y i ). P(X i | Y 0 =y 0, …………..., Y i =y i ).

6. Independence Assumptions Only the immediate past matters: Only the immediate past matters: P(X i | X 1,…………., X i-1 )=P(X i | X i-1 ). P(X i | X 1,…………., X i-1 )=P(X i | X i-1 ). Measurements depend only on the current state: Measurements depend only on the current state: P(Y i |Y j,...,Y k |X i )=P(Y i |X i )P(Y j,….,Y k |X i ).

7. Applications of Tracking Motion Capture Motion Capture Recognition from motion Recognition from motion Surveillance Surveillance Targetting Targetting

8. Different approaches for Object Tracking Correlation – based Correlation – based Feature based Feature based Gradient based Gradient based Color Histograms Color Histograms Kalman Filter Kalman Filter Particle Filter Particle Filter

9. PARTICLE FILTER Particle Filters are powerful tools for bayesian state estimation in non-linear systems. The basic idea of particle filters is to approximate a posterior distribution over unknown state variables by a set of particles, drawn from this distribution.

10. Particle Filters requires two types of information: Data Data Controls Controls Measurements Measurements Probabilistic model of the system Probabilistic model of the system The data is given by z t =z 1,z 2,……,z t and u t =u 1,u 2,…….,u t. u t =u 1,u 2,…….,u t.

11. Particle Filters, like any member of the family of Bayes filters such as kalman filters and HMM’s, estimate the posterior distribution of the state of the dynamical system conditioned on the data,p(x t |z t,u t ). They do so via the following recursive formula P(x t |z t,u t )=  t  p(z t |x t ) I p(x t |u t,x t-1 ) p(x t-1 |z t-1,u t-1 )dx t-1

12. To Calculate this posterior, three probability distributions are required ( Probabilistic model of the dynamical systems): 1) A Measurement model,p(z t |x t ), which describes the probability of measuring z t when the system is in state x t. 2) A Control model, p(x t |u t,x t-1 ), which characterizes the effect of controls u t on the system state by specifying the probability that the system is in state x t after executing control u t in state x t-1. 3) An Intial state distribution, p(x 0 ), which specifies the user’s knowledge about the intial system state.

13. Problems with probabilistic filter In many applications, the key concern in implementing this probabilistic filter is the continous nature of the staes x, controls u, and measurements z. Even in discrete versions, these spaces might be prohibitively large to compute the entire posterior.

14. Particle filter tackles the problem The particle filter addresses these concerns by approximating the posterior using the sets of state samples (particles): X t ={x t [i] } i=1,……..,N X t ={x t [i] } i=1,……..,N The set X t consists of N particles x t [ i ],for some large number of N. Together these particles approximates the posterior p(x t |z t,u t ). X t is calculated recursively.

15. A Simple Particle Filter Algorithm Given a prior p(X 1 ), a transition prior p(X t |X t-1 ) and a likelihood p(Y t |X t ), the algorithm is as follows: p(Y t |X t ), the algorithm is as follows: 1) Initialization, t=1 for i=1,……….,N, sample (X 1 ( i ) )~p(X 1 ) and set t=2. 2) Importance Sampling step For i=1,…….,N sample Xp t ( i ) ~p(Xp t ( i ) |X t-1 ( i ) ) and set Xp 1:t ( i ) =( X t ( i ), X 1:t-1 ( i ) ). For i=1,……,N, Evaluate Importance weights w t ~p(Y t |X t ( i ) ) Normalise the importance weights. Normalise the importance weights.

16. Algorithm contd…… 3) Selection Step Resample with replacement N particles (X i:t ( i ) ; i=1,…….,N) from the set ( Xp 1:t ( i ) ; i= 1,……., N) according to the normalised importance weights. Set t = t+1 and go to step 2

17. Basic steps implemented in the project Intially at time t=0, the particles x 0 [i] are generated from the initial state distribution p(x 0 ). The t-th particle set X t, is the calculated recursively from X t-1 as follows: 1 Set X t =X t aux =0 2 For j=1 to N do 3 pick the j-th sample x t-1 [ j ]  X t-1 4 draw x t [ j ] ~p(x t |u t,x t-1 [ j ] ) 5 set w t [ j ] =p(z t |x t [ j ] ) 6 add (x t [ j ],w t [ j ] ) to X t aux 7 End for

18. Basic Steps contd…. 8 For i= 1 to m do 9 draw x t [i] from X t aux with probability propotional to w t [i] 10 add x t [i] to X t 11 End for

19. Files used in the project A set of image files, which was later converted to a video file. A set of image files, which was later converted to a video file. A matrix file consisting of data about the image. A matrix file consisting of data about the image. pf_proj.m, which contains the main algorithm for particle filters and drawing of the trajectories for the moving object. pf_proj.m, which contains the main algorithm for particle filters and drawing of the trajectories for the moving object. multinomialR.m, which contains the code for resampling. multinomialR.m, which contains the code for resampling.

20. Sample of the code for i = 1:N, for i = 1:N, states(:, t, i) = A(:,:,i) * states(:, t-1, i) + B(:,:,i) * randn(100,1); states(:, t, i) = A(:,:,i) * states(:, t-1, i) + B(:,:,i) * randn(100,1); % Evaluate importance weights. % Evaluate importance weights. w(t, i) = (exp(-0.5*(states(:,t,i))'* (states (:,t,i)))) + 1e-99; w(t, i) = (exp(-0.5*(states(:,t,i))'* (states (:,t,i)))) + 1e-99; w_real(t, i) = w(t, i); w_real(t, i) = w(t, i); end; end; w(t,:) = w(t,:)./ sum(w(t, :)); % Normalise the weights. w(t,:) = w(t,:)./ sum(w(t, :)); % Normalise the weights.

21. DEMO [ Shown separately]

22. Acknowledgement Thanks to Dr. Longin Jan Latecki, for providing me the opportunity to work on this project.