1. Using Adaptive Tracking To Classify And Monitor Activities In A Site W.E.L. Grimson, C. Stauffer, R. Romano, L. Lee

2. Objectives to calibrate the distributed sensors, to construct rough site models, to classify detected objects, to learn common patterns of activity for different object classes, and to detect unusual activities

3. Motivation The system should provide statistical descriptions of typical activity patterns, e.g., normal vehicular volume or normal pedestrian traffic paths for a given time of day it should detect unusual events, by spotting activities that are very different from normal patterns, e.g., unusual volumes of traffic, or a specific movement very different from normal observation it should detect unusual interactions between objects, e.g., a person parking a car in front of a building, exiting the car, but not entering the building

4. Hypothesis motion tracking is sufficient to support a range of computations about site activities

5. Hardware The system observes activities with a “forest of sensors” distributed around the site. Each sensor unit is a compact packaging of camera, on-board computational power, local memory, communication capability and possibly locational instrumentation (e.g., GPS)

6. Robust Adaptive Tracker

7. Robust Adaptive Tracker contd.. Color encodes direction and intensity encodes speed

8. Using The Tracker To Calibrate When using multiple cameras it is essential to coordinate the individual video streams and find a common coordinate frame. Procedure -  Each camera independently tracks an object.  Correspondence is established between points in the video frames from different cameras.  For each camera pair, small but sufficient set of points is sampled from a larger set of point correspondences and a least square solution to the homographies is found.  This solution is compared with the current solution by calculating the mean square error on all the point correspondences and the best fit is retained.

9. Using The Tracker To Calibrate contd...

10. Using The Tracker For Site Modeling Objective is to determine the pose of the ground plane relative to a camera. Estimate the height of the object. By using the estimate of the ground plane and the height the distance to the object can be estimated. The portion between the un-occluded object and the camera can be carved out as free space. When the object is occluded, this places a lower bound on the occluding portion in the site. Since the ground plane is known these obstacles can be placed in the world coordinates of the site.

11. Using The Tracker For Site Modeling contd..

12. Using The Tracker to Classify Classify objects – size and the aspect ratio of the tracked objects can be used to classify and label the objects in the site Classify actions – tracks of the moving objects can be used to classify activities by clustering the tracks on the basis of common attributes

13. Using The Tracker to Classify contd.. Clustering Algorithm #1- 1) Numeric Iterative Hierarchical Clustering Algorithm 2) MDL cut

14. Using The Tracker to Classify contd..