1. Human Detection in Surveillance Applications
Ashish Desai
EE392J Final Project

2. Problem and Motivation
Had many petty vandals cause damage to cars in my parking garage
Landlord added cameras, but did not deter vandals
What if we could count and have photos of all the people in the garage at any given time?

3. Breakdown into Class Concepts
Simply, this is a foreground / background segmentation problem
Perhaps use color based segmentation
Perhaps use motion based segmentation

4. Methodology
Captured 25 second video at 10 frames per second of 1, 2, or 3 people walking around in my apartment.

5. Methodology Used Staufer and Grimson color based segmentation
Used 5 gaussians in RGB space
Used maximum of 1.5 std. dev for classification
Used alpha = 0.7
Combined with connectivity requirements (each pixel must touch 4 others)
Combined with size requirements (each group of pixels must be larger than 100)

6. Methodology Very simplified motion based segmentation
Used block matching between current and next frame
16x16 block size, +16/-15 full search, SAD criterion
Eliminate blocks with fewer than 8 pixel movement
Combine with size requirements (must be greater than 4 blocks)
Note: did not use K-means clustering (and should have used log search) to improve real-time capabilities

7. Results of Individual Segmentations
Color based segmentation
Worked pretty well (I spent a lot of time tweaking this)
Had problems with large areas of occlusion, shadows
Motion based segmentation
Purposely set to allow “false positives”
Simple methodology could not handle global motion from person bumping the camera
Blinds (possible interlace artifact) caused problems, aspect ratio (block-based)

8. Combinational Methodology
Take both of the initial methodologies and create a confidence weighting (those pixels in the center of a group have higher weighting)
Combine the two weights with more preference to color weights (color had better initial performance)
Apply threshold, connectivity and size constraints.

9. Overall Block Diagram Input Sequence (RGB) RGB -> YUV
Motion Estimation
Color Gaussian Segmentation
Connectivity/ Size Requirements
Connectivity/ Size Requirements
Weighting
Weighting
Threshold
Output

10. Overall Results
Performed pretty well especially with lateral movement and objects further from camera
Still had problems with large occlusions (from color) and aspect ratio changes (from motion) but better than respective individuals
Eliminated major shadow and global motion issues by combining the two

11. Results - Videos

12. Future Enhancements Improvements (if time allows)
Use affine parameters for motion based
Use log search (if still use block) for more real time
Use K means clustering to segment, rather than simple threshold
Use temporal knowledge (i.e. use object identification from previous frame or frames)