Tutorial: multicamera and distributed video surveillance Third ACM/IEEE International Conference on Distributed Smart Cameras ICDSC /08/2009 Como (Italy) Prof. Rita Cucchiara Università di Modena e Reggio Emilia, Italy

Distributed surveillance  Problem of tracking and distributed consistent labeling : Problem of matching or recognizing objects previously viewed by other cameras.  Some constraints:  Constraints on the motion models and transition times  Scene planarity for both overlapping and not overlapping FOVs  Constraints of recurrent paths [70] V. Kettenker, R. Zabih Bayesian multi camera surveillance CVPR 1999 [54] C.Stauffer, K.Tieu automated multi- camera planar tracking correspondence modeling cvpr 2003

Distributed surveillance (cont.)  Network of (smart) cameras; Not overlapped FoVs; loosely coupled.  Problems of node communication  If moving cameras: problems of calibration and tracking. The simultaneous localization and tracking ( SLAT ) problem, to estimate both the trajectory of the object and the poses of the cameras.  Problem of color calibration [71]Zoltan Safar, John Aa. Sørensen, Jianjun Chen, and K°are J. Kristoffersen MULTIMODAL WIRELESS NETWORKS: DISTRIBUTED SURVEILLANCE WITH MULTIPLE NODES Proc of ICASSP 2005 [72]Funiak, S.; Guestrin, C.; Paskin, M.; Sukthankar, R.; Distributed localization of networked cameras Int conf on Information Processing in Sensor Networks, Information Processing in Sensor Networks, originalIndependent channels Look-up table Full matrix

Color calibration  Methods:  Linear transformation  Independent channels  Full matrix ( M conmputed with LSQ)  Look-up table  for non linear  transformation [73]Roullot, E., "A unifying framework for color image calibration," 15th International Conference on Systems, Signals and Image Processing, IWSSIP 2008, pp , June 2008 [74]K. Yamamoto and J. U “Color Calibration for Multi-Camera System by using Color Pattern Board” Technical Report MECSE

Feature to match  Color (single / multiple)  Shape (geometrical ratios / spline / elliptical models)  Motion (speed, direction)  Gait (Fourier transform)  SIFT +, grey level co-occurrence matrix, Zernike moments and some simple colour features  Polar color histogram + Shape [75]Nicholas J. Redding, Julius Fabian Ohmer1, Judd Kelly1 & Tristrom Cooke Cross-Matching via Feature Matching for Camera Handover with Non-Overlapping Fields of View Proc. Of DICTA2008 [76]Kang, Jinman; Cohen, Isaac; Medioni, Gerard, "Persistent Objects Tracking Across Multiple Non Overlapping Cameras," IEEE Workshop on Motion and Video Computing, WACV/MOTIONS '05, vol.2, no., pp , Jan. 2005

Distributed Surveillance at ImageLab  The problem: a people disappeared in the scene exiting from a camera FoV, where can be detected in the future?  1) tracking within a camera FoV multi hypothesis generation  2) tracking in exit zones  3) Prediction into new cameras’ FoVs  4) matching in the entering zones  Using Particle Filtering + Pathnodes  In computer graphic all the possible avatar positions are represented by nodes and the connecting arcs refers to allowed paths. The sequence of visited nodes is called pathnodes. A weight can be associated to each arc in order to give some measures on it, such as the duration, the likelihood to be chosen with respect to other paths, and so on.  Weights can be defined or learned in a testing phase [ 77]R. Vezzani, D. Baltieri, R. Cucchiara, "Pathnodes integration of standalone Particle Filters for people tracking on distributed surveillance systems" in Proceedings of 25° ICIAP2009, 2009

Exploit the knowledge about the scene  To avoid all-to-all matches, the tracking system can exploit the knowledge about the scene  Preferential paths -> Pathnodes  Border line / exit zones  Physical constraints & Forbidden zones NVR  Temporal constraints

Tracking with pathnode A possible path between Camera1 and Camera 4

Pathnodes lead particle diffusion

Results with PF and pathnodes Single camera tracking: Multicamera tracking Recall=90.27% Recall=84.16% Precision=88.64% Precision=80.00%

Example Frame 431. a man #21 exits and his particles are propagated Frame 452 a person # 22 exits too and also his particles are propagated Frame 471 a people is detected in Camera #2 and the particles of both # 21 and #22 are used but the ones of #22 match and person 22 is recognized