Human Detection Mikel Rodriguez

Organization 1. Moving Target Indicator (MTI) Background models Background models Moving region detection Moving region detection Target chip generation Target chip generation Results Results Input Frame Object Detection Target Chips Wavelet Features SVM Classifier MTIClassification 2. Target Classification (Human Detection) Target features Target features Support vector machines Support vector machines Results Results

Moving Target Indicator Moving target indicator (MTI) identifies moving objects which can be potential targets

MTI Motivation Becoming increasingly important in military and civilian applications Becoming increasingly important in military and civilian applications To minimize human involvement To minimize human involvement Expensive Expensive Short attention spans Short attention spans Computerized monitoring system Computerized monitoring system Real-time capability Real-time capability 24/7 24/7

MTI Challenges Different sensor modalities Different sensor modalities LADAR, IR, EO LADAR, IR, EO Targets with different dynamics Targets with different dynamics Small targets Small targets Weather conditions Weather conditions Illumination changes, shadows… Illumination changes, shadows…

Input Video MTI Background Modeling Intensity models Gradient models Moving Target Detection Background Subtraction dynamic update Targets ChipsPosition

Hierarchical Approach to Background Modeling Pixel level Pixel level Region level Region level Frame level Frame level

Pixel Level Background Features Intensity, heat index Intensity, heat index Gradient Gradient 2D: magnitude, orientation 2D: magnitude, orientation IREO Magnitude Orientation

Pixel Level Background Features Intensity, heat index Intensity, heat index Per-pixel mixture of Gaussians. Per-pixel mixture of Gaussians. Gradient based subtraction Gradient based subtraction Gradient feature vector  =[  m,  d d ] Gradient feature vector  =[  m,  d d ]

Pixel Level Moving Region Detection Mark pixels that are different from the background intensity model Mark pixels that are different from the background intensity model Mark pixels that are different from the background gradient model Mark pixels that are different from the background gradient model

Color basedImage Gradient Region Level Fusion of Intensity & Gradient Results For each color based region, presence of“edge difference” pixels at the boundaries is checked. Regions with small number of edge difference pixel are removed, color model is updated. Final

Frame Level Model Update Performs a high level analysis of the scene components Performs a high level analysis of the scene components If more > 50% of the intensity based background subtracted image becomes foreground. Frame level processing issues an alert Intensity based subtraction results are ignored

Structure of the MTI Class MTI Background ConnectedComponents() BoundaryEdges() SetNumGaussians() SetAlpha() SetRhoMean() SetWeightThresh() SetActiveRegion() GetNumGaussians() GetAlpha() GetRhoMean() GetWeightThresh() GetActiveRegion() Object SetBoundingBox() SetRhoLocation() SetCentroid() GetBoundingBox() GetRhoLocation() GetCentroid() IsFalseDetection() Chips Centroid() ObjectArea() Height() Width()

Results

Target Classification Classification of objects into two classes: humans and others, from target chips generated by MTI

Challenges Small size Small size Obscured targets Obscured targets Background clutter Background clutter Weather conditions Weather conditions

Classifier Flow Feature Extraction Wavelet Testing MTI Chips NegativePositive Training SVM Support Vectors Decision

Wavelet Based Target Features Blurred Vertical Horizontal Diagonal

Feature Extraction Apply 2D Wavelet Transform Apply 2D Wavelet Transform Daubechies wavelets Daubechies wavelets Apply Inverse 2D Wavelet Transform to each of the coefficient matrices individually Apply Inverse 2D Wavelet Transform to each of the coefficient matrices individually Rescale and vectorize output matrices Rescale and vectorize output matrices

Why Wavelets? Separability among samples Separability among samples can be separated from Humans can be separated from cars and background Correlation using gray levelsCorrelation using gradient mag.

Why Wavelets? Person 11 - DB3 Wavelet Correlation

Support Vector Machines (SVM) Classification of data into two classes Classification of data into two classes N dimensional data. N dimensional data. Linearly separable Linearly separable If not transform data into a higher dimensional space If not transform data into a higher dimensional space Find separating N dimensional hyperplane Find separating N dimensional hyperplane

SVM Linear Classifier hyperplane equation N dimensional data point x i Sample distance to hyperplane

SVM Best Hyperplane? Infinite number of hyperplanes. Infinite number of hyperplanes. Minimize r i over sample set x i Minimize r i over sample set x i Maximize margin  around hyperplane Maximize margin  around hyperplane Samples inside the margin are the support vectors Samples inside the margin are the support vectors

SVM Training Set Let  =1,A training set is a set of tuples: {(x 1,y 1 ),(x 2,y 2 ),…(x m,y m )}. Let  =1,A training set is a set of tuples: {(x 1,y 1 ),(x 2,y 2 ),…(x m,y m )}. For support vectors inequality becomes equality For support vectors inequality becomes equality Unknowns are w and b Unknowns are w and b

SVM Linear Separability Linear programming, Linear programming, Separator line in 2D w 1 x i,1 +w 2 x i,2 +b=0. Separator line in 2D w 1 x i,1 +w 2 x i,2 +b=0. Find w 1, w 2, b such that  is maximized Find w 1, w 2, b such that  is maximized Find w 1, w 2, b such that  (w)=w T w is minimized Find w 1, w 2, b such that  (w)=w T w is minimized

SVM Solution Has the following form: Has the following form: Non-zero  i indicates x i is support vector Non-zero  i indicates x i is support vector Classifying function is: Classifying function is:

Classification Class Human Classification TrainingFunction ReadPositiveImages() ReadNegativeImages() AssemblePositive() AssembleNegative() AssembleMatrices() TestingFunction LoadSVM() ReadImages() ExtractFeatures ConvertToGray() ApplyWaveletFilter() ApplyInverseTrans() ResizeInverse() VectorizeInverse() Concatenate() TrainSVM LIBSVM TestSVM LIBSVM

Classification Baseline Analysis Run time for 3.0GHz dualcore, 2GB RAM Run time for 3.0GHz dualcore, 2GB RAM Training: 276 training samples seconds Training: 276 training samples seconds Testing: chips (25 by 25) per second Testing: chips (25 by 25) per second Classifier size Classifier size Depends on diversity of images Depends on diversity of images For 276 training samples of 25x25, classifier size is MB For 276 training samples of 25x25, classifier size is MB

Classification Baseline Analysis Memory requirements Memory requirements Requires entire set of support vectors Requires entire set of support vectors Current classifier Current classifier

Experiments Vivid DatasetUCF Dataset Vivid DatasetUCF Dataset

Results Training set Training set 300 target chips 300 target chips Testing Testing 3872 human chips 3872 human chips 5605 vehicle and background chips 5605 vehicle and background chips Performance Performance 2.4% false positive (others classified as pedestrians) 2.4% false positive (others classified as pedestrians) 3.2% false negative (pedestrian classified as others) 3.2% false negative (pedestrian classified as others)

Future directions MTI MTI Detection by parts Detection by parts Motion clustering Motion clustering Classification Classification Various kernels for SVM Various kernels for SVM Better target features Better target features Motion, steerable pyramids, shape features (height, width) Motion, steerable pyramids, shape features (height, width) Local wavelet coefficients Local wavelet coefficients Adaboost Adaboost