1. BraMBLe: The Bayesian Multiple-BLob Tracker By Michael Isard and John MacCormick Presented by Kristin Branson CSE 252C, Fall 2003

2. Problem The goal is to track an unknown number of blobs from static camera video.

3. Solution The Bayesian Multiple-BLob (BraMBLe) tracker is a Bayesian solution. It estimates State at frame t Image Sequence Number, Positions, Shapes, Velocities, …

4. Bayes Rule Posterior State Distribution Observation Likelihood Prior

5. Sequential Bayes Posterior State Distribution Observation Likelihood Prior Instead of modeling directly, BraMBLe models and.

6. Update Algorithm

7. Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

8. Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

9. Image Observations We want to choose our observations so that we can compute quickly: Individual observations are conditionally independent.

10. Image Observations We want. A bank of filters is applied at each grid point. Y Cr Cb Filter plots from http://www.cs.jhu.edu/~wolff/course600.461/week3.2/sld012.htm Mexican Hat Gaussian

11. Image Observations We want to choose our model so that we can compute quickly: Observation depends on membership of grid point.

12. Image Observations We want to choose our model so that we can compute quickly: We can precompute and quickly evaluate any state x.

13. Appearance Models The appearance models are learned from training data. Training Data

14. Observation Likelihood

15. Observation Likelihood Review We defined our image observations so that We defined our observation model so that We can precompute and quickly evaluate for many choices of x.

16. Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

17. Object Model The blob configuration is Number of objects Object State

18. Object Model The blob configuration is The object state is Identity Location Velocity Shape

19. Person Model Generalized-Cylinder Model Calibrated Camera

20. Prediction Model The number of objects can change:  Each object has a constant probability of remaining in the scene.  There is a constant probability that a new object will enter the scene. In this formulation, hypotheses with different numbers of objects can be compared directly.

21. Prediction Model Damped constant location velocity:

22. Prediction Model Damped constant location velocity: Auto-regressive shape:

23. Model Review The observation likelihood is fast to compute for different hypotheses. The prediction model allows generation of from Estimating requires an efficient way of  Representing.  Computing the multiplications and integrations.

24. Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

25. Efficient Representation is represented by a set of particles, : Sampling from the set using the weights approximates sampling from N Points: N Weights:

26. Efficient Representation is represented by a set of particles, : Sampling from the set using the weights approximates sampling from

27. Efficient Computation The particle set representing is computed from by C ONDENSATION : 1.Apply dynamics to the particle set: 2.Multiply by the observation likelihood:

28. Applying Dynamics Given particle set, compute Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.018.gif

29. Applying Dynamics Given particle set, compute 1.Resample into Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.020.gif

30. Applying Dynamics Given particle set, compute 1.Resample into 2.Predict, generating to give

31. Multiplication by Likelihood Given particle set, compute Weight particles, setting Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.021.gif

32. Efficient Computation Review The particle set representing is computed from by C ONDENSATION: Resample Predict Reweight Image from http://www.hpl.hp.com/personal/John_MacCormick/WOMOT03/cal.giftalk/page.022.gif

33. Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

34. People Tracking

35. Tracking was successful in real time on this 53s clip except when two people crossed in front of a third.

36. Outline Observation likelihood model. Prediction model. Estimation of posterior. Results. Discussion.

37. Algorithm Summary The models chosen  Are a smooth integration of foreground and background models.  Allow hypotheses with differing numbers of objects to be compared directly.  Can be quickly evaluated in a particle filtering implementation.

38. Relationship to Previous Work Static camera blob tracking:  Classifies pixels as foreground or background. Application of Stauffer and Grimson’s Adaptive Background Subtraction to video with compression artifacts Video from http://image.pirl.umd.edu/knkim/research/BGS/compressed_video.htm

39. Relationship to Previous Work Static camera blob tracking:  Classifies pixels as foreground or background.  Static camera blob tracking:  Finds the position in the search area Made up of foreground pixels. Matching the blob in the previous frame. Frame t - 1 Frame t Frame t + 1 Predicted position in frame t + 1 Search area

40. Relationship to Previous Work Improvements over blob tracking:  Integrates the foreground and background modeling.  Multiple objects can be tracked through occlusions. Video from http://robotics.stanford.edu/~birch/headtracker/Video from http://www.robots.ox.ac.uk/~jmac/research/thesis/thesis.html

41. Weaknesses The algorithm is sensitive to reflections. The algorithm sometimes switches the labels when one object passes in front of another. There are a lot of parameters to assign.