Detecting Pedestrians by Learning Shapelet Features Payam Sabzmeydani and Greg Mori Vision and Media Lab School of Computing Science Simon Fraser University The main problem that I worked on during my masters was pedestrian detection and today I’ll talk about detecting pedestrians in still images using learned shape features

Problem Given a still image, we want to find and locate the pedestrians in the image Clothing (color, appearance) Body pose Applications: Automated surveillance systems Image search and retrieval Robotics Intelligent vehicles

Problem

Problem Classification-based detection Different cues Classify a window as pedestrian or non-pedestrian Search exhaustively the scale-space image Different cues Wavelet coefficients (Mohan et al., PAMI 2001) Oriented gradients (Dalal and Triggs, CVPR 2005) SIFT features (Leibe et al., CVPR 2005) Edgelet features (Wu and Nevatia, ICCV 2005) “Shapelet features” (Sabzmeydani and Mori, CVPR 2007)

Datasets MIT : Standing pose, simple background, no occlusion INRIA : Standing pose, complex background, partial occlusions

Previous Work Dalal & Triggs (CVPR 2005) HOG features + SVM

Previous Work Wu & Nevatia (ICCV 2005) Edgelet features: short line and curve segments AdaBoost

Our Method Compute low-level gradient features Oriented filter responses Learn mid-level features for detecting pedestrians “Shapelet features” Build final classifier from shapelet features

Low-level Features Filter responses Image gradient in different directions

Low-level Features Smoothed gradient responses in different directions

Shapelet Features A weighted set of low-level gradient features inside a sub-window of the detection window Characteristics Simple and low-dimensional Learned exclusively for our object classes Highly discriminative Local effective area : useful to model separate parts instead of the whole body

Shapelet Features

Learning Shapelet Features Learned using AdaBoost (Viola and Jones, 2001) Extract low-level features in sub-window Select subset of features using AdaBoost Find those which discriminate between pedestrian and background classes

AdaBoost Algorithm W w

Low-level features as weak classifiers Each low-level feature can provide us many weak classifiers: AdaBoost will combine weak classifiers to form a better classifier:

Shapelet features Train classifiers in sub-windows Use the output of a classifier as the shapelet feature response:

Shapelet Features

Shapelet Features

Final Classifier Take all shapelet features Learned at many sub-windows of detection window Run AdaBoost again to select weighted subset of shapelet features for final classifier

Final Classifier A weighted sub-set of mid-level features

Shapelet Feature Size Small, Medium, and Large features Capture different scales of information

Normalization Why normalize? How to normalize? Different lighting, shadows, different contrast, … How to normalize? Per shapelet feature : L2-norm

Normalization

Results on INRIA Dataset

Error examples Most non-pedestrian-like pedestrians (false negatives) Most pedestrian-like non-pedestrians (false positives)

Future work Detecting other objects Use image context or segmentation Pyramid of features

References N. Dalal and B. Triggs. “Histograms of oriented gradients for human detection”. CVPR 2005. B. Wu and R. Nevatia. “Detection of multiple, partially occluded humans in a single image by bayesian combination of edgelet part detectors”. ICCV 2005. P. Viola and M. Jones. “Rapid object detection using a boosted cascade of simple features”. SCTV 2001.

Problem

Problem

Bootstrapping

Mid-level Features