1. Vehicle Segmentation and Tracking in the Presence of Occlusions
Neeraj K. Kanhere
Dr. Stanley T. Birchfield
Dr. Wayne A. Sarasua
Clemson University

2. Introduction Traffic parameters such as volume, speeds, turning counts
and classification are fundamental for…
Transportation planning
Traffic impact of land use
Transportation engineering applications (e.g. signal timing)
Intelligent Transportation Systems (ITS)

3. Why computer vision?
Different types of sensors can be used to gather data:
Radar or laser based sensors
Inductive loop detectors
Video Camera (with Computer Vision techniques)
No traffic disruption for installation and maintenance
Covers wide area with a single camera
Provides rich visual information for manual inspection

4. Why tracking?
Current systems use localized detection within the detection zones
which is prone to errors when camera placement in not ideal.
Tracking enables prediction of a vehicle’s location in consecutive frames
Can provide more accurate estimates of traffic volumes and speeds
Potential to count turn-movements at intersections
Detect traffic incidents

5. Related research Region/Contour Based Computationally efficient
Good results when vehicles are well separated
3D Model Based
Large number of vehicle models needed
Limited experimental results
Markov Random Field
Good results on low angle sequences
Accuracy drops by 50% when sequence is processed in true order
Feature Tracking Based
Handles partial occlusions
Good accuracy when sufficient features are tracked from entry region to exit region

6. Factors to be considered
High-angle
Mid-angle
Planar motion assumption
Well-separated vehicles
Relatively easy
More depth variation
Occlusions
A difficult problem

7. Estimation of 3-D Location
Overview of the approach
Background model
Offline Calibration
Frame-Block #1
Estimation of 3-D Location
Feature Tracking
Frame-Block #2
Frame-Block #3
Grouping
segmented #3
segmented #2
segmented #1
Counts,
Speeds and
Classification
Block Correspondence
and Post Processing

8. Background model and calibration`
Background model and calibration
Adaptive time domain median filtering for background
Calibration provides mapping from scene to image
Use scene features to estimate correspondences
Lane widths
Truck heights
Approximate calibration is good enough for counts

9. Processing a frame-block
Overlap
frames
Block # n
Block # n+1
Multiple frames are needed for motion information
Tradeoff between number of features and amount of motion
Typically 5-15 frames yield good results
#features in
block
#frames in
block

10. Frame differencing Partially occluded vehicles appear as single blob
Effectively segments well-separated vehicles
Goal is to get filled connected components

11. Estimation using single frame
Box-model for vehicles
Road projection using foreground mask
Works for orthogonal surfaces
camera
vehicle
Road plane

12. Selecting stable features
Shadows, partial occlusions will result into wrong estimates
Planar motion assumption is violated more for features higher up
Select stable features, which are closer to road
Use stable features to re-estimate world coordinates of other features

13. Estimation using motion
Estimate coordinates with respect to each stable feature
Choose coordinates which minimized weighted sum of euclidean distance and trajectory error
Rigid body under translation
Estimate coordinates with respect to each stable feature
Select the coordinates minimizing weighted sum of Euclidean distance and trajectory error
P : Feature with unknown coordinates
Q : Stable feature
R : Backprojection on road
H : Backprojection at maximum height
0 : First frame of the block
t : Last frame of the block
Δ : Translation of corresponding point

14. Affinity matrix
Each element represents the similarity between corresponding features
Three quantities contribute to the affinity matrix
Euclidean distance (AD), Trajectory Error (AE) and Background- Content (AB)
Normalized Cut is used for segmentation (Shi, Malik)
Number of Cuts is not known

15. Incremental normalized cuts
We apply normalized cut to initial A with increasing number of cuts
For each successive cut, segmented groups are analyzed till valid groups are found
Valid group: meets dimensional criteria
Elements corresponding to valid groups are removed from A and process repeated starting from single cut
Avoids specifying a threshold for the number of cuts

16. Correspondence over blocks
Formulated as a problem of finding maximum weight graph
Nodes represent segmented groups
Edge weights represent number features common over two blocks

17. Results

18. Results

19. Conclusion A novel approach based on feature point tracking
Key part of the technique is estimation of 3-D coordinates
Results demonstrate the ability to correctly segment vehicles even under severe partial occlusions
Vehicle count, speeds and classification (car or heavy vehicle) data can be easily obtained for tracked vehicles
Future Work
Robust block-correspondence
Tracking vehicles at intersections
Automatic calibration by detecting lane markings
Explicit shadow suppression

20. Questions ?

21. Thank you !