1. Real-time On-Road Vehicle Detection With Optical Flows and Haar-like feature detector
Department of Computer Science,
University of Illinois Urbana-Champaign
December
Jaesik Choi

2. I. On-Road Vehicle Detection Problem
Given
The road images from a moving vehicle (monocular camera)
Provide
Road boundary
Location of vehicles in 3D space
Assumption
Paved, flat (no hill) and straight (no curve) road
How can we solve it in Real-time?
Coming traffic: Optical Flow [Lucas and Kanade 81]
Same lane traffic: [Viola & Jones 03]
Scene context: [Hoiem 06]
[Ferryman et al. 2000]

3. Contents Problem definitions Algorithms & Related works
On Road Vehicle Detection Problem
Algorithms & Related works
Overall Diagram
Detection of cars traveling in the opposite direction : Optical Flows
Detection of cars traveling in the same direction : (Viola & Jones 03)
Estimate 3D geometry (D. Hoiem 06)
Technical details
Times
3D Project
Filtering with Dynamic Bayes Net
Demo Video

4. II. Algorithms – Overall Diagram
Input Image
Cars traveling in the opposite direction
Corner detector (Shi & Tomasi )
Find optical flows (Lucas Kanade Optical Flow)
Cars traveling in the same direction
Verify a car with rectangle filters that is learned with ‘AdaBoost’(Viola, etc al 2003)
Viewpoint filtering
Estimate 3D geometry using vanishing point (D. Hoiem 06)
Kalman Filter (DBN) & Projection into 3D image
Filter invalid rectangles in the consecutive images
Project each rectangle into 3D image

5. II. Algorithms – Optical Flows
Find corners & correspondence between consecutive frames
Cluster the small groups
if dist( (location1, vector1), (location2, vector2 ) ) < threshold 1 2 3 4 5 6

6. II. Algorithms - Viola & Jones Haar-like detector
Learn the set of important ‘N’ features from training images
334 positive car rear images
126 images from ‘Cars 1999 (Rear)2 dataset, CALTECH’
194 images from ‘CBCL CAR DATABASE, MIT’
other 14 images
500 negative images vs

7. II. Algorithms - Viola & Jones Haar-like detector
Set of rectangle features
Given an image (Hypothesis):
Verify the image (Hypothesis) with rectangle features that is trained with AdaBoost from training images
Feature 1:
Feature 2: … -
> 1  -
> 2

8. II. Algorithms - Viola & Jones Haar-like detector
Performance
Training set
Number of detect image / all car images = 88.63%
Number of missed image / all car images = 11.37%
Test set (516 images Cars 2001 (Rear), Caltech)
Number of detect image / all car images = 90.50%
Number of detect image / all car images = 9.50%

9. III. Algorithms – Estimate 3D Geometry
Estimate 3D Geometry (D. Hoiem, 06)
P( Windowwidth| Disty ) ?
Windowwidth: The width of a window of a car
DistY : The vertical distance from the vanishing point

10. IV. Technical Details - Times
Real-time issues – execution times
3GHz 2CPU, 1GB Desktop -> frames/sec

11. IV. Technical Details – 3D projection
Project into 3D Image based on the Probabilistic estimate of 3D Geometry
 
dist = h * tan( + n *∆ )  h

12. IV. Technical Details (Filter outliers using Dynamic Bayes Net)
Estimate the correct locations of cars using Dynamic Bayes Net
P( | )
P( | ) vs
P(Modelt+1|Modelt)
= P(Colort+1|Colort)
* P(Sizet+1|Sizet)
* P(Locationt+1|Locationt)
* P(Sizet+1|Locationt+1)

13. V. Demo Movies

14. Reference
[Betke et al. 2000] M. Betke, E. Haritaglu and L. Davis, “Real-time multiple vehicle detection and tracking from a moving vehicle,” Machine Vision and Applications, vol. 12, no. 2, 2000.
[Sun et al. 2002] Z. Sun, R. Miller, G. Bebis, and D. DiMeo, “A real-time precrash vehicle detection system,” IEEE International Workshop on Application of Computer Vision, Dec., 2002.
[Wedel et al. 2006] A. Wedel, U. Franke, J. Klappstein, T. Brox, and D. Cremers, “Realtime Depth Estimation and Obstacle Detection from Monocular Video”, DAGM 2006, LNCS 4174, pp , 2006.
[Ferryman 1998] J. M. Ferryman, S. J. Maybank, and A. D. Worrall, "Visual surveillance for moving vehicles," Int. J. Comput. Vis., vol. 37, no. 2, pp , June 2000.
[Shi and Tomasi 94] Jianbo Shi and Carlo Tomasi. “Good Features to Track,” IEEE Conference on Computer Vision and Pattern Recognition, pages , 1994.
[Nishigaki 00] M. Nishigaki, M. Saka, T. Aoki, H. Yuhara, and M. Kawai, “Fail Ouput Algorithm of Vision Sensing,” Proc. IEEE Intelligent Vehicle Symp., pp , 2000.
[Sun et al. 2006] Z. Sun, G. Bebis, and R. Miller, “On-Road Vehicle Detection: A Review”, IEEE Trans. Pattern Analysis and Mach. Intelligence, vol. 28, no. 5, 2006.