MULTI-TARGET TRACKING THROUGH OPPORTUNISTIC CAMERA CONTROL IN A RESOURCE CONSTRAINED MULTIMODAL SENSOR NETWORK Jayanth Nayak, Luis Gonzalez-Argueta, Bi.

Slides:



Advertisements
Similar presentations
An Adaptive Learning Method for Target Tracking across Multiple Cameras Kuan-Wen Chen, Chih-Chuan Lai, Yi-Ping Hung, Chu-Song Chen National Taiwan University.
Advertisements

A Hierarchical Multiple Target Tracking Algorithm for Sensor Networks Songhwai Oh and Shankar Sastry EECS, Berkeley Nest Retreat, Jan
Coverage Estimation in Heterogeneous Visual Sensor Networks Mahmut Karakaya and Hairong Qi Advanced Imaging & Collaborative Information Processing Laboratory.
System Integration and Experimental Results Intelligent Robotics Research Centre (IRRC) Department of Electrical and Computer Systems Engineering Monash.
Patch to the Future: Unsupervised Visual Prediction
Hybrid Position-Based Visual Servoing
Tracking Multiple Occluding People by Localizing on Multiple Scene Planes Professor :王聖智 教授 Student :周節.
Object Inter-Camera Tracking with non- overlapping views: A new dynamic approach Trevor Montcalm Bubaker Boufama.
Ghunhui Gu, Joseph J. Lim, Pablo Arbeláez, Jitendra Malik University of California at Berkeley Berkeley, CA
Learning Semantic Scene Models From Observing Activity in Visual Surveillance Dimitios Makris and Tim Ellis (2005) Presented by Steven Wilson.
Oklahoma State University Generative Graphical Models for Maneuvering Object Tracking and Dynamics Analysis Xin Fan and Guoliang Fan Visual Computing and.
ICIP 2000, Vancouver, Canada IVML, ECE, NTUA Face Detection: Is it only for Face Recognition?  A few years earlier  Face Detection Face Recognition 
Modeling Pixel Process with Scale Invariant Local Patterns for Background Subtraction in Complex Scenes (CVPR’10) Shengcai Liao, Guoying Zhao, Vili Kellokumpu,
Motion Tracking. Image Processing and Computer Vision: 82 Introduction Finding how objects have moved in an image sequence Movement in space Movement.
Multi video camera calibration and synchronization.
A Bayesian algorithm for tracking multiple moving objects in outdoor surveillance video Department of Electrical Engineering and Computer Science The University.
Motion based Correspondence for Distributed 3D tracking of multiple dim objects Ashok Veeraraghavan.
Detecting and Tracking Moving Objects for Video Surveillance Isaac Cohen and Gerard Medioni University of Southern California.
1 Integration of Background Modeling and Object Tracking Yu-Ting Chen, Chu-Song Chen, Yi-Ping Hung IEEE ICME, 2006.
Student: Hsu-Yung Cheng Advisor: Jenq-Neng Hwang, Professor
CSE473/573 – Stereo Correspondence
TelosCAM: Identifying Burglar Through Networked Sensor-Camera Mates with Privacy Protection Presented by Qixin Wang Shaojie Tang, Xiang-Yang Li, Haitao.
REALTIME OBJECT-OF-INTEREST TRACKING BY LEARNING COMPOSITE PATCH-BASED TEMPLATES Yuanlu Xu, Hongfei Zhou, Qing Wang*, Liang Lin Sun Yat-sen University,
Overview and Mathematics Bjoern Griesbach
Jason Li Jeremy Fowers Ground Target Following for Unmanned Aerial Vehicles.
Tutorial: multicamera and distributed video surveillance Third ACM/IEEE International Conference on Distributed Smart Cameras.
Fault Tolerant Sensor Network for Border Activity Detection B. Cukic, V. Kulathumani, A. Ross Lane Department of CSEE West Virginia University NC-BSI,
GM-Carnegie Mellon Autonomous Driving CRL TitleAutomated Image Analysis for Robust Detection of Curbs Thrust AreaPerception Project LeadDavid Wettergreen,
3D Fingertip and Palm Tracking in Depth Image Sequences
Reading Notes: Special Issue on Distributed Smart Cameras, Proceedings of the IEEE Mahmut Karakaya Graduate Student Electrical Engineering and Computer.
P. 1/30 Heping Song, Tong Liu, Xiaomu Luo and Guoli Wang Feedback based Sparse Recovery for Motion Tracking in RF Sensor Networks IEEE Inter.
Shape-Based Human Detection and Segmentation via Hierarchical Part- Template Matching Zhe Lin, Member, IEEE Larry S. Davis, Fellow, IEEE IEEE TRANSACTIONS.
Consensus-based Distributed Estimation in Camera Networks - A. T. Kamal, J. A. Farrell, A. K. Roy-Chowdhury University of California, Riverside
A General Framework for Tracking Multiple People from a Moving Camera
3D SLAM for Omni-directional Camera
Chapter 14: SEGMENTATION BY CLUSTERING 1. 2 Outline Introduction Human Vision & Gestalt Properties Applications – Background Subtraction – Shot Boundary.
Combined Central and Subspace Clustering for Computer Vision Applications Le Lu 1 René Vidal 2 1 Computer Science Department, Johns Hopkins University,
Scientific Writing Abstract Writing. Why ? Most important part of the paper Number of Readers ! Make people read your work. Sell your work. Make your.
Efficient Deployment Algorithms for Prolonging Network Lifetime and Ensuring Coverage in Wireless Sensor Networks Yong-hwan Kim Korea.
Handover and Tracking in a Camera Network Presented by Dima Gershovich.
December 9, 2014Computer Vision Lecture 23: Motion Analysis 1 Now we will talk about… Motion Analysis.
Automated Intruder Tracking using Particle Filtering and a Network of Binary Motion Sensors Jeremy Schiff EECS Department University of California, Berkeley.
Motion Analysis using Optical flow CIS750 Presentation Student: Wan Wang Prof: Longin Jan Latecki Spring 2003 CIS Dept of Temple.
Wenqi Zhu 3D Reconstruction From Multiple Views Based on Scale-Invariant Feature Transform.
Vehicle Segmentation and Tracking From a Low-Angle Off-Axis Camera Neeraj K. Kanhere Committee members Dr. Stanley Birchfield Dr. Robert Schalkoff Dr.
1 Research Question  Can a vision-based mobile robot  with limited computation and memory,  and rapidly varying camera positions,  operate autonomously.
Robust Object Tracking by Hierarchical Association of Detection Responses Present by fakewen.
Chapter 5 Multi-Cue 3D Model- Based Object Tracking Geoffrey Taylor Lindsay Kleeman Intelligent Robotics Research Centre (IRRC) Department of Electrical.
Efficient Computing k-Coverage Paths in Multihop Wireless Sensor Networks XuFei Mao, ShaoJie Tang, and Xiang-Yang Li Dept. of Computer Science, Illinois.
Boosted Particle Filter: Multitarget Detection and Tracking Fayin Li.
Segmentation of Vehicles in Traffic Video Tun-Yu Chiang Wilson Lau.
By Naveen kumar Badam. Contents INTRODUCTION ARCHITECTURE OF THE PROPOSED MODEL MODULES INVOLVED IN THE MODEL FUTURE WORKS CONCLUSION.
COMP 417 – Jan 12 th, 2006 Guest Lecturer: David Meger Topic: Camera Networks for Robot Localization.
A Tutorial on using SIFT Presented by Jimmy Huff (Slightly modified by Josiah Yoder for Winter )
Adaptive Tracking in Distributed Wireless Sensor Networks Lizhi Yang, Chuan Feng, Jerzy W. Rozenblit, Haiyan Qiao The University of Arizona Electrical.
OBJECT TRACKING USING PARTICLE FILTERS. Table of Contents Tracking Tracking Tracking as a probabilistic inference problem Tracking as a probabilistic.
Presented by: Idan Aharoni
Visual Tracking by Cluster Analysis Arthur Pece Department of Computer Science University of Copenhagen
Respectful Cameras Jeremy Schiff EECS Department University of California, Berkeley Ken Goldberg, Marci Meingast, Deirdre Mulligan, Pam Samuelson IEOR,
Tracking Groups of People for Video Surveillance Xinzhen(Elaine) Wang Advisor: Dr.Longin Latecki.
Person Following with a Mobile Robot Using Binocular Feature-Based Tracking Zhichao Chen and Stanley T. Birchfield Dept. of Electrical and Computer Engineering.
Zhaoxia Fu, Yan Han Measurement Volume 45, Issue 4, May 2012, Pages 650–655 Reporter: Jing-Siang, Chen.
SEMINAR ON TRAFFIC MANAGEMENT USING IMAGE PROCESSING by Smruti Ranjan Mishra (1AY07IS072) Under the guidance of Prof Mahesh G. Acharya Institute Of Technology.
Traffic Sign Recognition Using Discriminative Local Features Andrzej Ruta, Yongmin Li, Xiaohui Liu School of Information Systems, Computing and Mathematics.
+ SLAM with SIFT Se, Lowe, and Little Presented by Matt Loper
Range Imaging Through Triangulation
Vehicle Segmentation and Tracking in the Presence of Occlusions
Dongwook Kim, Beomjun Kim, Taeyoung Chung, and Kyongsu Yi
Vision Tracking System
Video Compass Jana Kosecka and Wei Zhang George Mason University
Presentation transcript:

MULTI-TARGET TRACKING THROUGH OPPORTUNISTIC CAMERA CONTROL IN A RESOURCE CONSTRAINED MULTIMODAL SENSOR NETWORK Jayanth Nayak, Luis Gonzalez-Argueta, Bi Song, Jayanth Nayak 1, Luis Gonzalez-Argueta 2, Bi Song 2, Amit Roy-Chowdhury, Ertem Tuncel Amit Roy-Chowdhury 2, Ertem Tuncel 2 Department of Electrical Engineering, University of California, Riverside 9/8/2008ICDSC'081 Bourns College of Engineering I nformation P rocessing L aboratory

Overview Introduction Problem Formulation Audio And Video Processing Camera Control Strategy Computing Final Tracks Of All Targets Experimental Results Conclusion Acknowledgements 9/8/2008ICDSC'082

Motivation Obtaining multi-resolution video from a highly active environment requires a large number of cameras. Disadvantages Cost of buying, installing and maintaining Bandwidth limitations Processing and storage Privacy Our goal: minimize the quantity of cameras by a control mechanism that directs the attention of the cameras to the interesting parts. 9/8/2008ICDSC'083

Proposed Strategy Audio sensors direct the pan/tilt/zoom of the camera to the location of the event. Audio data intelligently turns on the camera and video data turns off the camera. Audio and video data are fused to obtain tracks of all targets in the scene. 9/8/2008ICDSC'084

Example Scenario 9/8/2008ICDSC'085 An example scenario where audio can be used to efficiently control two video cameras. There are four tracks that need to be inferred. Directly indicated on tracks are time instants of interest, i.e., initiation and end of each track, mergings, splittings, and cross-overs. The mergings and crossovers are further emphasized by X. Two innermost tracks coincide in the entire time interval (t 2, t 3 ). The cameras C 1 and C 2 need to be panned, zoomed, and tilted as decided based on their own output and that of the audio sensors a 1,..., a M.

Relation To Previous Work Fusion of simultaneous audio and video data. Our audio and video data are captured at disjoint time intervals. Dense network of vision sensors. In order to cover a large field, we focus on controlling a reduced set of vision sensors. Our video and audio data is analyzed from dynamic scenes. 9/8/2008ICDSC'086

Problem Formulation Audio sensors A = {a 1,..., a M } are distributed across ground plane R R is also observable from a set of controllable cameras C = { c 1,..., c L }. However, entire region R may not be covered with one set of camera settings. p-tracks: tracks belonging to targets a-tracks: tracks obtained by clustering audio Resolving p-track ambiguity Camera Control Person Matching 9/8/2008ICDSC'087

Tracking System Overview 9/8/2008ICDSC'088 a-tracks Overall camera control system. Audio sensors A = {a 1,..., a M } are distributed across regions R i. The set of audio clusters are denoted by B t, and K t− represent the set of confirmed a-tracks estimated based on observations before time t. P/T/Z cameras are denoted by C = {c 1,..., c L }. Ground plane positions are denoted by O t k.

Processing Audio and Video a-tracks are clusters of audio data that are above amplitude threshold Tracked using Kalman Filter In video, people are detected using histogram of orientated gradients and tracked using Auxilary Particle Filter 9/8/2008ICDSC'089

Mapping From Image Plane to Ground Plane Learned parameters are used to transform tracks from image plane to ground plane Estimate projective transformation matrix H during a calibration phase Precompute H for each PTZ setting of each camera 9/8/2008ICDSC'0810 vanishing line

Tracking System Overview 9/8/2008ICDSC'0811

Camera Control Camera control Goal: avoid ambiguity or disambiguate when tracks are created or deleted intersect merge Set pan/tilt/zoom parameters 9/8/2008ICDSC'0812

Setting Camera Parameters Heuristic algorithm Cover ground plane by regions R i l R i l in field of view of camera C l Camera parameters Tracking algorithm specifies point of interest x from last known a-track If no camera on, find R i l containing x Reassign a camera and set its parameters if x approaches boundary of current R i l 9/8/2008ICDSC'0813

Camera Control Based on Track Trajectories Intersection 9/8/2008ICDSC'0814 Separation Merger Sudden Appearance Undetected Disappearance Sudden Disappearance Location (Meters) Time (Seconds) Location (Meters) Time (Seconds) Location (Meters) Time (Seconds) Location (Meters) Time (Seconds) Location (Meters) Time (Seconds) Switch to video Location (Meters) Time (Seconds)

Creating Final Tracks Of All Targets Bipartite graph matching over a set of color histograms We collect features as the target enters and exits the scene in video. For every new a-track, features are collected from a small set of frames. The weight of an edge is the distance between the observed video features. Additionally, audio data is enforced on the weights. 9/8/2008ICDSC'0815

Creating Final Tracks Using Bipartite Matching 9/8/2008ICDSC'0816 Location (Meters) Time (Seconds) Audio Video [a+, a-] [b+, b-] [c+] [e+, e-] Tracking in Audio and Video Location (Meters) Time (Seconds) Tracking in Audio Only Three tracks are recovered by matching every node (entry and exit from the scene) where video was capture. Two tracks are recovered. However, red and green show the wrong path. Audio cannot disambiguate independence once the clusters have merged. Video abcdefgabcdefg + - Bipartite Graph Matching abcdefgabcdefg abcdefgabcdefg + - Bipartite Graph Matching Without Audio Constraint abcdefgabcdefg [c-]

Experimental Results 9/8/2008ICDSC'0817 Inter P-Track Distance at a Merge EventInter P-Track Distance at a Crossover Event

Experimental Results (Cont.) 9/8/2008ICDSC'0818

Conclusion Goal: minimize camera usage in a surveillance system Save power, bandwidth, storage and money Alleviate privacy concerns Proposed a probabilistic scheme for opportunistically deploying cameras in a multimodal network. Showed detailed experimental results on real data collected in multimodal networks. Final set of tracks are computed by bipartite matching 9/8/2008ICDSC'0819

Acknowledgements This work was supported by Aware Building: ONR- N C-0311 and the NSF CNS Bi Song 2 and Amit Roy-Chowdhury 2 were additionally supported by NSF-ECCS and ARO-W911NF /8/2008ICDSC'0820

Thank You. Questions? Jayanth Nayak Jayanth Nayak 1 Luis Gonzalez-Argueta, Bi Song, Luis Gonzalez-Argueta 2, Bi Song 2, Amit Roy-Chowdhury, Ertem Tuncel Amit Roy-Chowdhury 2, Ertem Tuncel 2 9/8/2008ICDSC'0821 Bourns College of Engineering I nformation P rocessing L aboratory