1. DETECTING AND TRACKING TRACTOR-TRAILERS USING VIEW-BASED TEMPLATES Masters Thesis Defense by Vinay Gidla Apr 19,2010

2. Introduction Object tracking: Sports analysis Games and gesture recognition Retail video mining Automobile driver assistance Traffic surveillance Volume, individual speeds, classification Lane changes, speed violations, congestions

3. Feature-based vehicle tracking Beymer et al. 1997 use feature point approach with motion cues to segment vehicles using homography Kanhere et al. 2008 use features with 3D estimation using multi-level homography Feature_based.avi Feature_based.avi Drawback: These approaches track features on the vehicle, not vehicle as a whole

4. Template-based tracking Model the object by 2D template of image intensities Compare search image with template image Comparison usually by discrete cross-correlation Good: Both spatial and appearance information Able to retrieve shape of the object Bad: Encode vehicle appearance from single viewpoint Do not adapt to changes in appearance of object

5. Proposal Overcome the limitations of a single template by using a template sequence instead of a single template The template sequence encapsulates all of the vehicle’s perspective deformations As a starting step, aim to detect and track contours of tractor-trailers in multi-lane traffic

6. Video Sequences

7. Template creation Training sequence: A portion of traffic video containing a tractor- trailer Process the video frames to create a template sequence

8. Training Sequence

9. Training frame

10. Manual contour selection

11. Template creation

12. Template sequence

13. Algorithm Overview

14. Step 1: Background subtraction

15. Reference background

16. Input Video Frame

17. Background subtracted frame

18. Step 2: Blob-Template match

19. Blob-Template match

20. Plot of Blob-Template match

21. Step 3: Trace contour

22. Results based on template-blob correlation

23. Plot of misalignment

24. Gradient magnitude match Reduce the misalignment by including salient features such as points of high gradient magnitude These points are located at identical spatial locations in every tractor-trailer

25. Training frame

26. Gradient Magnitude

27. Template Gradient Magnitude

28. Template Gradient Sequence

30. Results based on template-frame gradient match

31. Plot of misalignment

32. Test sequences

33. Results(Lane 3)

34. Results(Lane 2)

35. Level set-based tracking for automatic template generation

36. Conclusion The new approach accurately traces the contours of all the tractor-trailers in the traffic video Works for multi-lane traffic Minor misalignment

37. Future extensions Tracking other classes of vehicles such as passenger cars, buses etc Compact template sequence with minimal template redundancy Implement matching using level set techniques

38. Thank You

39. Questions & Discussion