1. Robust Object Tracking by Hierarchical Association of Detection Responses Present by fakewen

2. outline Introduction Hierarchical Association of Detection Responses ▫Low-Level Association ▫Middle-Level Association ▫High-Level Association Experimental Results conclusion

3. a detection-based three-level hierarchical association approach to robustly track multiple objects in crowded environments from a single camera.

4. low level, reliable tracklets. middle level, these tracklets are further associated to form longer tracklets based on more complex affinity measures. high level, entries, exits and scene occluders are estimated using the already computed tracklets, which are used to refine the final trajectories.

7. outline Introduction Hierarchical Association of Detection Responses ▫Low-Level Association ▫Middle-Level Association ▫High-Level Association Experimental Results conclusion

8. notation detection response position size occurrence frame index color histogram

9. notation object trajectory/tracklet object trajectory/tracklet set association results of the low level, the middle level and the high level respectively

10. outline Introduction Hierarchical Association of Detection Responses ▫Low-Level Association ▫Middle-Level Association ▫High-Level Association Experimental Results conclusion

11. Low-Level Association : set of all detection responses

12. outline Introduction Hierarchical Association of Detection Responses ▫Low-Level Association ▫Middle-Level Association ▫High-Level Association Experimental Results conclusion

13. Middle-Level Association The middle level association is an iterative process: each round takes the tracklets generated in the previous round as the input and does further association

14. First round input tracklet association ▫l k is the number of tracklets in S k. corresponding trajectory of S k tracklet association set.

15. MAP problem

16. Hungarian Algorithm

17. Hungarian Algorithm(1) Arrange your information in a matrix with the "people" on the left and the "activity" along the top, with the "cost" for each pair in the middle.

18. Hungarian Algorithm(2) Ensure that the matrix is square by the addition of dummy rows/columns if necessary. Conventionally, each element in the dummy row/column is the same as the largest number in the matrix.

19. Hungarian Algorithm(3) Reduce the rows by subtracting the minimum value of each row from that row.

20. Hungarian Algorithm(4) Reduce the columns by subtracting the minimum value of each column from that column.

21. Hungarian Algorithm(5) Cover the zero elements with the minimum number of lines it is possible to cover them with. (If the number of lines is equal to the number of rows then go to step 9)

22. Hungarian Algorithm(6) Add the minimum uncovered element to every covered element. If an element is covered twice, add the minimum element to it twice.

23. Hungarian Algorithm(7) Subtract the minimum element from every element in the matrix.

24. Hungarian Algorithm(8) Cover the zero elements again. If the number of lines covering the zero elements is not equal to the number of rows, return to step 6.

25. Hungarian Algorithm(9) Select a matching by choosing a set of zeros so that each row or column has only one selected.

26. Hungarian Algorithm(10) Apply the matching to the original matrix, disregarding dummy rows. This shows who should do which activity, and adding the costs will give the total minimum cost.

30. Implementation Details for each input tracklet, a Kalman Filter is used to refine the positions and sizes of its detection responses and estimate their velocities. refined color histogram by a RANSAC method

31. appearance affinity motion affinity frame gap between the tail of and the head of

32. temporal affinity number of frames in which the tracked object is occluded by other objects number of frames in which the tracked object is visible but missed by the detector

34. Initialization and termination probabilities of each tracklet In the following rounds, tracklets with longer frame gaps are associated by progressively increasing.

35. outline Introduction Hierarchical Association of Detection Responses ▫Low-Level Association ▫Middle-Level Association ▫High-Level Association Experimental Results conclusion

36. High-Level Association During the middle-level association, all tracklets have the same initialization/termination probabilities as there is no prior knowledge about entries and exits at that stage.

37. High-Level Association At the high level, an entry map and an exit map are inferred from T M, which are used to specify the initialization/termination of each tracklet in the scene. a scene occluder map is also inferred from T M to revise the link probabilities.

38. The three maps, as hidden variables, constitute a scene structure model in the high-level association. the scene structure model is estimated in the ground plane coordinates for better accuracy. solve this coupled scene-estimation tracklet- association problem by an EM-like algorithm.

39. E-step Bayesian inference indicator function for entries, exits or scene occluders (q = {en; ex; oc}) at position x on the ground plane.

41. The complete version of a tracklet T k that includes missed detections,, is obtained by filling the gaps between inconsecutive detection responses with interpolated ones.

42. entry/exit map estimated position and velocity at the head/tail of by the Kalman Filter short time span for predicting the positions of the entry and the exit.

43. entry/exit map where is the position of response.

44. occluder map subset of the complete tracklet

47. M-step In the M-step, the tracklets in T M are further associated to form even longer ones. based on the scene structure model obtained from the E-step, the initialization and termination probabilities

48. M-step the frame number of missed detection part between the head (or tail) of T k to the nearest entry (or exit):

49. This is used to reivse the temporal affinity in Equ.13 by considering occlusions by scene occluders when counting the occluded frame number.

51. outline Introduction Hierarchical Association of Detection Responses ▫Low-Level Association ▫Middle-Level Association ▫High-Level Association Experimental Results conclusion

52. Evaluation Metric (1) MOTA: Multiple Object Tracking Accuracy, calculated from the number of false alarms, missed detections, and identity switches; (2) FGTIM: Fraction of Ground Truth Instances Missed; and (3) FAPF: False Alarm Per Frame.

57. outline Introduction Hierarchical Association of Detection Responses ▫Low-Level Association ▫Middle-Level Association ▫High-Level Association Experimental Results conclusion

58. Conclusion we present a robust hierarchical association framework for the multiple object tracking problem effectively associating tracklets with inaccurate detection responses and long-time occlusions.

59. The end Thank you!

60. outline Introduction Hierarchical Association of Detection Responses ▫Low-Level Association ▫Middle-Level Association ▫High-Level Association Experimental Results conclusion

62. Hungarian Algorithms 1.From each line (row or column) subtract its minimum element. 2.Find a maximum set of N’ mutually independent zeroes. 3.if N’ = N such zeroes have been found: output their indices and stop otherwise: cover all zeroes in W with N’ lines and find the minimum uncovered value; subtract it from all uncovered elements, and add it to all doubly covered elements; go to 2.

63. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 114587 Machine 221265 Machine 37839 Machine 424610

64. Hungarian Algorithms 1.From each line (row or column) subtract its minimum element.

65. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 114 5 87 Machine 2 2 1265 Machine 378 3 9 Machine 4 2 4610

66. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 19 0 3 2 Machine 2 0 1043 Machine 345 0 6 Machine 4 0 248

67. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 19030 Machine 201041 Machine 34504 Machine 40246

68. Hungarian Algorithms 1.From each line (row or column) subtract its minimum element. 2.Find a maximum set of N’ mutually independent zeroes. ▫Use minimum lines to cover all zeros

69. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 19030 Machine 201041 Machine 34504 Machine 40246

70. Hungarian Algorithms 1.From each line (row or column) subtract its minimum element. 2.Find a maximum set of N’ mutually independent zeroes. 3.if N’ = N such zeroes have been found: output their indices and stop otherwise: cover all zeroes in W with N’ lines and find the minimum uncovered value; subtract it from all uncovered elements, and add it to all doubly covered elements; go to 2.

71. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 110040 Machine 20940 Machine 34403 Machine 40145

72. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 114587 Machine 221265 Machine 37839 Machine 424610

73. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 19030 Machine 201041 Machine 34504 Machine 40246

74. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 110030 Machine 20930 Machine 35504 Machine 40135

75. Hungarian Algorithm Job 1Job 2Job 3Job 4 Machine 114587 Machine 221265 Machine 37839 Machine 424610