Università degli Studi di Modena e Reggio Emilia Dipartimento di Ingegneria dell’Informazione ImageLab: Attività di ricerca e sviluppo in video-sorveglianza Prof. Ing. Rita Cucchiara ImageLab ImageLab Dipartimento di Ingegneria dell’Informazione Università di Modena e Reggio Emilia, Italy

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Imagelab Imagelab Laboratory of Image Analysis for Computer Vision and Multimedia Dipartimento di Ingegneria dell’Informazione Università di Modena e Reggio Emilia, Italy Staff Prof. Rita Cucchiara, PhD (Faculty staff) Ing.Andrea Prati, PhD (Faculty staff) Ing. Costantino Grana,PhD ( Faculty staff) Ing. Roberto Vezzani,PhD (post-Doc ) Ing. Simone Calderara (PhD student) Ing. Giovanni Gualdi( Ph-d Student) Ing. Daniele Borghesani (fellow) Dr. Amit Tilvankar(visiting researcher) Ing, Rudi Melli, Vision-E Ing. Giuliano Pistoni, Vision-E Ing. Emanuele PeriniVision-E Prof. Massimo Piccardi (University of Technology, Sidney) 10 Un-graduated students…

Research Activities at ImageLab Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Computer Vision for robotic automation Transcoding for Multimedia Image analysis for Medical Imaging Video analysis for telemetry People and vehicle surveillance Video analysis for indoor/outdoor surveillance Multimedia: video annotation Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

CV for robotic automation Projects in collaboration with companies with local and regional support: some examples Vision-Based Navigation for AGV Vehicle ( ) Research activity in vision-based localization, navigation and Obstacle detection for Automated Guided Vehicle. Research activity in vision-based localization, navigation and Obstacle detection for Automated Guided Vehicle. System for automatic creation of ceramic mosaics ( ) Research activity in the development of a complete robotic system for the automatic creation of ceramic mosaics. Research activity in the development of a complete robotic system for the automatic creation of ceramic mosaics. Detecting borders in labels for textile implants ( ) Research activity in the development of a complete robotic system for the automatic creation of ceramic mosaics. Research activity in the development of a complete robotic system for the automatic creation of ceramic mosaics. Analysis of car vibrations ( ) analysis of head vibration in Ferrari cars analysis of head vibration in Ferrari cars Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

Medical Imaging Medical Imaging: Dental tomography ( ) Echocardiography ( Burhnam Institute, CA, USA) ( ) Dermatological image analysis ( ) Segmentation Segmentation Feature analysis Feature analysis Mosaicing Mosaicing Retrieval Retrieval Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

Multimedia Cache performance analysis ( ) MIUR ITALIAN Project ( ) MIUR ITALIAN Project ( ) Video Transcoding ( ) Video Annotation ( ) EU VI FP DELOS Project ( ) EU VI FP DELOS Project ( ) EU VII FP VIDI-VIDEO Project ( ) EU VII FP VIDI-VIDEO Project ( ) Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

Video surveillance- old projects  Experiments since 1996 ( University of Ferrara, University of Bololgna)  Projects with Bologna Municipality ( )  Sakbot project (University of Modena) ( )  “Domotica per Disabili” Project indoor surveillance ( )  Sakbot for Traficon ( ) Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

Video-surveillance now (1/2)  LAICA (Laboratorio di Ambient Intelligence per una Città Amica) funded by Regione Emilia-Romagna, Italy ( ?) Collab. With WTI Techn., Regulus, Comune Reggio Emilia… Surveillance in public parks with distributed and ptz cameras.  BE-SAFE Bheavior lEarning foir Surveillance application with feature extraction –NATO Science for Peace program ( ) With Israel (Hebrew Yniv, Magal)  Automatic real-time detection of infiltrated objects for security of airports and train stations ( ) Australian Research Council with University of tecnlogy, Sidney  FREE-SURF: Free Surveillance in a Privacy Respectful way Funded by Italian MUR PRIN Project ( ) With Univ. of firenze, Palermo, IRST Trento Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

Video-surveillance now (2/2) Video-surveillance now (2/2)  Collaboration with Municipal Policy of Modena for forensis analysis ( )  Mobile remote Surveillance ( ) funded by ER PRIITT project with Sistemi integrati spa  Traffic analysis on highway (2007) with Digitek spa  VISOR Project in Vidivideo ( )  Smoke detection (WTI 2007) Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

People real time remote surveillance  “Domotica per Disabili” FCR Modena  Posture detection for domotic application  2006 PRIITT “Videosorveglianza mobile” Sistemi Integrati remote surveillance with h264 streaming

Remote Surveillance  Mobile surveillance  Int. Symp. On Multimedia 2007 Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

Projects at imagelab  vehicle and traffic surveillance - Traficon (B) S.Diego highway

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Projects at ImageLab  LAICA Project Laboratorio di Ambient Intelligence per una Città Amica Piano Telematico ER  Public park in Reggio Emilia,  Modena Campus

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Projects at ImageLab  Automatic real-time detection of infiltrated objects for security of airports and train stations  ( ) Automatic real-time detection of infiltrated objects for security of airports and train stations Australian Research Council

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Project at imagelab  NEW  Free SURF free surveillance in a privacy respectful way MIUR PRIN Project (national coordinator)  BE SAFE Behavioral lEarning in Surveilled Areas with Feature Extraction NATO Science for Peace project with Israel

Multiple-people Multiple-camera surveillance 1) single fixed camera SAKBOT: Moving object segmentation (PAMI 2003) SAKBOT: Moving object segmentation (PAMI 2003) Shadow analysis (PAMI2003) Shadow analysis (PAMI2003) Tracking with occlusions (PR-submitted) Tracking with occlusions (PR-submitted) Posture analysis(T SMC 2004) Posture analysis(T SMC 2004) Face detection (PRL 2005) Face detection (PRL 2005) 2) single moving PT camera Mosaicing Mosaicing Motion clustering (RTI 2004) Motion clustering (RTI 2004) 3) multiple (overlapped FoVs) cameras HECOL Homography and Epipolar based consistent labeling (PAMI Submitted, CVIU) HECOL Homography and Epipolar based consistent labeling (PAMI Submitted, CVIU) 4) multiple (not overlapped FoVs) cameras Color analysis and retrieval Color analysis and retrieval 5) Trajectory analysis trajectory clustering for abnormality detection trajectory clustering for abnormality detection Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

Video surveillance Surveillance of  Environment  Traffic  People Natural environment (fire detection, fog detection…,) Urban Environment ( crowd monitoring in stations, metro..) Abnormal situations detection Road traffic control: Statistical control ( vehicles monitoring, classification ) Accident detection ( queue, incidents..) Urban control: people at road crossing, privileged path.. Naval, aerial traffic control Security : people detection in specific area people tracking for abnormal behavior detection people interaction analysis Safety : people behavior monitoring in house and offices people behavior analysis in public place people behavior analysis in car

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Human-based video surveillance  Most of commercial available video surveillance systems still provide acquisition, storage (and access) only modelli Data Storage Data Compression internet/ intranet Data Acquistion Data Compression Universal Multimedia Access

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Human-based video surveillance  System generations  1) analog cameras and CCTV video monitors

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Human-based video surveillance  System generations 2) Digital video recorder system 3) Digital PC based or embedded encoding system

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Human-based video surveillance  System generations 4) LAN cameras 5) LAN cameras with streaming Wireless cameras

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Human-based video surveillance  Effort in new hardware solutions Distributed vs centralized Distributed vs centralized New cameras New cameras PTZ, infrared, thermal, omnidirectional panoramic camerasPTZ, infrared, thermal, omnidirectional panoramic cameras  Standard vs. stand alone encoding solutions  User friendly (multimedia) human interface  Exploiting motion detection Improving human attention Improving human attention Storage space saving Storage space saving Data Acquisition modelli DBMS Human/machine based Control center Intra-inter net Knowledge extraction

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia (Automated) computer vision-based video surveillance  Exploiting image processing, pattern recognition and computer vision techniques in order to Extract moving objects Extract moving objects Detect variations of objects’ motion status Detect variations of objects’ motion status Measure object flow Measure object flow Classify people and other objects Classify people and other objects Track people and objects Track people and objects Consistent Track in distributed system Consistent Track in distributed system Recognize Bheavior, posture, interaction Recognize Bheavior, posture, interaction

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Multiple-people Multiple-camera surveillance at ImageLab Sakbot Ad hoc Hecol PPC Projects and prototypes since 1996

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Related Imagelab bibliography TRAFFIC CONTROL R. Cucchiara, P. Mello, M. Piccardi,"Image Analysis and Rule-Based Reasoning for a Traffic Monitoring," IEEE Transactions on Intelligent Transportation Systems, vol. 1, n. 2, pp , 2000 SAKBOT R. Cucchiara, C. Grana, M. Piccardi, A. Prati “Detecting Moving Objects, Ghosts and Shadows in Video Streams“, IEEE Transactions on Pattern Analysis and Machine Intelligence, july 2003 SHADOW DETECTION Prati, I. Mikic, M.M. Trivedi, R. Cucchiara, "Detecting Moving Shadows: Algorithms and Evaluation," IEEE Transactions on Pattern Analysis and Machine Intelligence, July 2003 POSTURE ANALYSIS R. Cucchiara, C. Grana, A. Prati, R. Vezzani, "Probabilistic Posture Classification for Human Behaviour Analysis" in IEEE Transactions on Systems, Man, and Cybernetics, Part A: Systems and Humans, vol. 35, n. 1, pp , 2005 MULTISENSOR R. Cucchiara, A. Prati, R. Vezzani, L. Benini, E. Farella, P. Zappi, "An Integrated Multi-Modal Sensor Network for Video Surveillance" Journal of Ubiquitous Computing and Intelligence (JUCI), 2006 DISTRIBUTED SURVEILLANCE S.Calderara, A.Prati, R. Cucchiara, HECOL Homography and epipolar based consistent labeling for outdoor park surveillance in press on Computer vision and Image Understanding 2007 FACE DETECTION R. Cucchiara, A. Prati, R. Vezzani, "A System for Automatic Face Obscuration for Privacy Purposes" in Pattern Recognition Letters, vol. 27, n. 15, pp , 2006

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Ghost shadow ghost MVO MVO shadow Moving object segmentation: SAKBOT   1) Statistical and knowledge based background modeling   2) color based background suppression with shadow detection   3) Classification and MVOs segmentation Tracking Objects History t Scene Background Suppression Background Update Segmentation I t MVO MVO shadow DB t B t Object validation Shadow detection Ghost Ghost shadow

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia SAKBOT RESULTS   Experiments against PETS2001, PETS2006 VSSN dataset First classified in background suppression competition ACM Multimedia VSSN06 S.Barbara USA   Future implementation in Intel OpenCV   Used in indoor and outdoor applications (people detection) In traffico control In sea surveillance

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Multiple people Tracking with occlusions: Ad Hoc  Ad hoc : Appearance-based, with occlusion classification tracking  Probabilistic model of tracking  Appearance based (probabilistic at pixel level)  Discriminative: detect the best position and appearance at each frame  Handling occlusion Occlusion classification Occlusion classification Keep object appearance also during occlusion Keep object appearance also during occlusion

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Example  Example of tracking without occlusion update

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Example

HECOL: distributed surveillance  HECOL Homography and epipoles based consistent labeling, in a probabilistic framework.. 1) Compute overlapped camera FoV 2) Construct the Camera Transition Graph 3) Sakbot + Ad Hoc tracking for each camera 4) when one or more new objects are in the overlapped area 1) Prune the search space with CTG 2) Construct the hypotheses space 3) Construct the forward and backward probability to link to other single or group of people 4) Solve with a double MAP the consistency buildin g Main entranc e benches park road building sec. entrance c1c1 c1c1 c2c2

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia Homography and epipolar lines

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia A priori and conditional probability Matching With occlusion With multiple people

Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

New and Future works Tracking people with PT cameras Bheavior analyis Face tracking for identification Surveillance video annotation (VIDI Video Project)

Smoke detection Rita Cucchiara- ImageLAB-DII Università di Modena e Reggio Emilia

Future events  ICIAP   International Conference on Pattern recognition and computer vision Pattern recognition and computer vision Multimedia Multimedia Human centered application Human centered application Security and surveillaince Security and surveillaince Industrial applications Industrial applications  Workshops Computational Color imaging Computational Color imaging DELOS Visual and Multimedia digital library DELOS Visual and Multimedia digital library  The Industrial Forum  European project forum For more information… Rita Cucchiara Tel