1. A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University of Kansas, Lawrence, KS Manjunath NarayanaDonna Haverkamp, Assistant Professor IEEE International Conference on Object Tracking and Classification in and Beyond the Visual Spectrum IEEE Computer Society Conference on Computer Vision and Pattern Recognition Minneapolis, MN, USA, Friday, June 22, 2007

2. The University of Kansas2 Main Objective Problem: Tracking of moving objects in surveillance video Our solution: A Bayesian approach to assign tracks to objects probabilistically, based on color and position observations from objects

3. The University of Kansas3 Outline Motivation Background Method Illustration Results Summary Future work

4. The University of Kansas4 Motivation Accurate tracking very important for surveillance applications Major issue: object data is noisy Objects Appear in the scene Disappear due to exit from scene or occlusion Merge with other objects or the background Break up into two or more objects due to occlusion A probabilistic algorithm to assign track numbers to objects may be very useful

5. The University of Kansas5 Example of tracking problem Fig 2Fig 1 previous framecurrent frame

6. The University of Kansas6 Segmentation and Tracking Motion segmentation (background subtraction) used for object detection Once moving objects (blobs) detected, find correspondence between tracks of previous frame and blobs of current frame Most common method: a Match matrix used to determine correspondences Euclidean distance between blobs commonly used as the measure for a match

7. The University of Kansas7 Our Tracking Approach We propose a Bayesian approach to determine probabilities of match between blobs. Bayesian approach results in a Belief matrix (of probabilities) instead of a Match matrix (of distances)

8. The University of Kansas8 Method track blob What is Probability of track  blob ? Basic principle : Given a track in previous frame, we expect a blob of similar color and position in current frame with some probability Provides basis for Bayesian method Upon observing a blob of given color and position, what is the posterior probability that this blob belongs on one of the tracks from the previous frame?

9. The University of Kansas9 O1O1 O2O2 Illustration (1)  Example: Blobs O 1 and O 2 seen in current frame What is the probability that each of these blobs belongs to the tracks in the previous frame? Color, c 1 ={r 1,g 1,b 1 } Position, d 1 ={y 1,x 1 } Color, c 2 ={r 2,g 2,b 2 } Position, d 2 ={y 2,x 2 } t1t1 t3t3 t2t2

10. The University of Kansas10 Probabilistic network for track assignment

11. The University of Kansas11 Illustration (2) Blob O 1 Blob O 2 “lost” t1t1 0.33 t2t2 t3t3 t1t1 t3t3 O1O1 O2O2 t2t2  Given three tracks t 1, t 2, and t 3 in the previous frame  There are six probabilities:  Each track can either be assigned to blob O 1 or O 2, or may be lost in the frame  Produces initial Belief matrix with equal likelihood for all cases

12. The University of Kansas12  Consider track t 1 and update first element in matrix  Observations: color (c 1 ), position(d 1 ) To find:  Assumption - color and position observations are independent: Illustration (3) t1t1 O1O1

13. The University of Kansas13 Illustration (4) Blob O 1 Blob O 2 “lost” t1t1 0.600.33 t2t2 t3t3  First, color observation for first element in matrix, c 1  By Bayes formula:  The Belief matrix is updated t1t1 O1O1

14. The University of Kansas14 Illustration (5) Blob O 1 Blob O 2 “lost” t1t1 0.600.20 t2t2 0.33 t3t3 After row normalization Blob O 1 Blob O 2 “lost” t1t1 0.600.33 t2t2 t3t3  Row needs to be normalized so that sum of elements is 1

15. The University of Kansas15  Next, position observation for first element in matrix, d 1, is considered  By Bayes formula:  Calculation and row normalization: t1t1 O1O1 Illustration (6) Blob O 1 Blob O 2 “lost” t1t1 0.900.05 t2t2 0.33 t3t3

16. The University of Kansas16 Illustration (7) Blob O 1 Blob O 2 “lost” t1t1 0.750.220.08 t2t2 0.33 t3t3  After the first element update, we move to second element  Similar calculation and update:  Row 1 processing - complete t1t1 O2O2

17. The University of Kansas17 t1t1 Illustration (8) Blob O 1 Blob O 2 “lost” t1t1 0.750.220.08 t2t2 0.250.600.15 t3t3 0.33  Similarly, processing track t 2 and row 2: t3t3 t2t2 Blob O 1 Blob O 2 “lost” t1t1 0.750.220.08 t2t2 0.250.600.15 t3t3 0.200.300.50  Processing track t 3 and row 3:

18. The University of Kansas18 Illustration (9) Blob O 1 Blob O 2 “lost” t1t1 0.750.220.08 t2t2 0.250.600.15 t3t3 0.200.300.50 t1t1 t3t3 O1O1 O2O2 t2t2  Based on the Belief matrix, the following assignments may be made  t 1  O 1  t 2  O 2  t 3  ”lost”

19. The University of Kansas19 Results Blob 1Blob 2“lost” track 030.000.100.90 track 070.001.000.00 track 110.000.390.61 track 121.000.00 Real example - frame 0240 Belief matrix Resulting track assignments Blob 1 Blob 2

20. The University of Kansas20 Results – comparison with Euclidean distance matrix Blob 1 Blob 2 Blob 3

21. The University of Kansas21 Results – comparison with Euclidean distance matrix “ lost” probability can be useful Track 12 (lost in frame 0275) would be erroneously assigned to blob 3 if Euclidean distance matrix used

22. The University of Kansas22 Results - tracking

23. The University of Kansas23 Summary Probabilistic track assignment method using the blob color and position observations Tracks allowed to be “lost” and recovered Provides alternative to Euclidean distance based match matrix Accuracy similar to Euclidean distance based matrix Probability values Easier to interpret Beneficial for further processing

24. The University of Kansas24 Future Work Incorporate blob size as an observation Extend to other spectra Use a learning approach to determine PDF’s for p(c|Assign) and p(d|Assign)

25. The University of Kansas25 Questions

26. The University of Kansas26 Probabilities – Color PDF 200 frames observed The color difference between the track and corresponding blob is observed PDF for Red color shown here Same PDF used for G and B

27. The University of Kansas27 Probabilities – Position PDF 200 frames observed The position difference between the track and corresponding blob is noted PDF for Y position shown here Same PDF used for X position