Tracking Groups of People for Video Surveillance Xinzhen(Elaine) Wang Advisor: Dr.Longin Latecki

Agenda Introduction Tracking Module Experimental results Conclusion

Introduction to tracking groups Goal: Given a video sequence, track real groups of people present in the scene. Steps: I Motion detection II Tracking module III Interpretation module ADVISORADVISOR: project overview ADVISOR : project overviewproject overview ADVISOR project overview

Motion detector Goal : Detect mobile objects in the scene and classify them into moving regions. Detection of moving regions Extraction of features Parameters: centre of gravity, position, height and width (calculate both in 2D and in 3D) Classification (labeling) of moving regions Classification (labeling) of moving regions 8 classes of mobile objects (person, occluded person, group, crowd, metro train, scene object, noise, unknown)

Group Tracking A real group: A set of persons who are close to each other. A set of moving regions. Four particularities: Size coherence: each moving region of a group has the dimensions of a person or bigger if several persons partially overlap each other. Size coherence: each moving region of a group has the dimensions of a person or bigger if several persons partially overlap each other. Spatial coherence: all moving regions inside a group are close to each other.

Characteristics (conti): Temporal coherence: the speed of the moving regions inside a group cannot exceed the speed of a person. Structure coherence: The number and the size of moving regions inside a group should be stable.

Steps in tracking algorithm Tracking moving regions from frame to frame. Computing inside the sub-graph all possible paths Compute the group structure that gathers all these paths

Frame to Frame Tracker Goal: Link from frame to frame all moving regions computed by the motion detector. A link: The link between M new and M old is computed depending on their 2D and 3D distance and the similitude between their bounding box sizes. Split: one M old linked to several M new Merge: several M old linked to one M new.

Frame to Frame Tracker O contains old moving regions, all those detected at tc – 1 and also those did not get linked at the previous q frames N contains new moving regions detected at time t c F computes the links between O and N G computes the links between N and O

Computing Paths Goal : Select trajectories of moving regions that can correspond to real persons inside a group during a temporal window. Size coefficient: If the size coefficient is bigger than the size of a person, then the path is likely to corresponding to a real person inside a group. To rank the paths

Creation of Paths

Update and Removing Update of Paths: If M last is the last moving region added in and is linked to the moving region M new detected in the new frame, is duplicated in and extended with M new. If M last is not liked to any new moving region, the path is only duplicated. As a result, the rank of such a path decreases. Removing Paths P i is totally overlapping P j and the size of P j is bigger. P i does not belong to a group anymore

Groups computing Goal : Select the paths of a connected sub-graph of that best match with the trajectories of real persons. A group G m is represented by its N paths P m,k, Description: –Groups are computed with a delay T, which constitutes a temporal window [t c – T, t c ] of size T. –In this window, first compute all possible future trajectories of moving regions detected at time t c – T –Select at time tc – T the moving regions best match a real group that would be observed from time t c – T to t c.

Density of the group over time Group quality coeffi c ient (q.c.): Instantaneous quality coefficient: Proximity between P m,best and P m,k Distance between P m,best and P m,k

Group Operations: Creation of Group –Selecting N max paths with biggest size coef, compute q.c. –Check if the q.c. is higher thank a threshold. Update of G m at t c - T –Adding, extending or removing the paths composing the group. –Remove all paths P m,i too far from P m,best –Select the remaining paths with best size coefficients –Recompute P m,best and q.c. Removing Groups –A group is removed if its quality coefficient is lower than a threshold.

Experimental Results Tested on several metro sequences –Longest sequence has more than 6500 frames –Red box: moving regions classified as PERSON –Green box: moving regions classified as GROUP –Blue box: moving regions tracked as a real group. Main limitation: –An imperfect estimation of real group size due to errors in motion detection. –Over-estimation –Under-estimation

Conclusions Track correctly groups of people from beginning to end. Future development: –Computation of group trajectory, speed and events inside the group in order to recognize abnormal behaviors.