2. Motion based Correspondence for Distributed 3D tracking of multiple dim objects Ashok Veeraraghavan

3. Problem Setting

4. Constraints R, T ??

5. Outline Tracking Algorithm  Implemented at each camera node.  Correspondence problem for dim targets.  Motion-Based Correspondence Algorithm  Implemented at central processor  Recovering Camera Position and Orientation  Recovering 3D tracks using triangulation.

6. Experimental Setup  Objective :  Reconstruct the 3D trajectories of the bees so as to study the response of bees to visual stimuli.  Outdoor Bee Tunnel with the surrounding walls texture systematically varied  Study relationship of flight patterns to visual stimulii.  Two Fixed Cameras.  Free Flying bees are the targets to be tracked.  Typically the bees are about 20-50 meters away from the camera.  Multiple Targets: On average each frame contains about 6-8 bees.  Occupy about 5-10 pixels at closet range: Low SNR  Objective : Reconstruct the 3D trajectories of the bees so as to study the response of bees to visual stimuli.

7. Tracking Algorithm  Background Subtraction  Background variations are assumed to be much slower than the target.  Dynamic background estimated using a temporal low pass filter for each pixel.  Connected Component Analysis  Morphological processing to connect pixels belonging to same target.  Probabilistic Data Association  Blob Tracking algorithm.

8. Background Subtraction and Connected Component Analysis

10. Adaptive Velocity Motion Model v r

11. Correspondence Problem for Dim Targets  Correspondence across camera Views  Associating the objects found in various views  Especially tricky for multiple dim objects  Dim Targets  Low SNR  Very Small Targets – (order of few pixels )  Features extraction unreliable  Appearance based correspondence  Appearance varies with view  Unreliable for dim targets

12. Motion Based Correspondence  Rubin and Richards (1985)  Rao, Yilmaz and Shah (2002)-  Maxima of spatio-temporal curvature as Dynamic Instants Courtesy: [Rao2002]

13. Dynamic Instants  Detects any start instant, stop instant, non- smooth change in speed, maximal curvature of 3D tracks. Eg., Start Instants Courtesy: [Rao2002]

14. Detected Dynamic Instants Courtesy: [Rao2002]

15. Correspondence Across Views

17. External Calibration  Internal Camera parameters known.  External Orientation of the cameras to be estimated from correspondence data obtained by matching tracks across views.  Simple non-linear optimization implemented (Levenberg-Marquardt).  Distance between cameras (Baseline) approximately known.  Optimization is local. Requires good initial estimate.

18. 3D flight Paths using Triangulation  Internal camera parameters known.  External camera calibration parameters estimated from point correspondences.  3D tracks obtained using Triangulation.

19. 3D Flight Paths

21. Future Work  Human Surveillance.  Work with multiple (more than 2 cameras) cameras.  Study the trade-off between bandwidth and efficiency.  Especially can we also add some appearance information to each target so that limited view reconstruction of target is possible?

23. Thank You