Automated Face Detection Peter Brende David Black-Schaffer Veni Bourakov

Primary Challenges Scale differences Scale differences Overlapping/obstructed faces Overlapping/obstructed faces Lighting variation Lighting variation

Implementation Overview 1.Color-based skin separation 2.Spatial analysis to generate candidate faces Morphological? Morphological? or Template Matching? or Template Matching? 3.Eliminate false hits Hands & arms, based on shape Hands & arms, based on shape Roof, based on texture Roof, based on texture Neck, based on relative position Neck, based on relative position

Color-based Skin Separation What color space to use? What color space to use? How to separate out the skin color? How to separate out the skin color?

Marginal Color Distributions

Parametric Separation Simple & Fast (h>0.98 or h 0.98 or h<0.01) Problems with non-linearity of HSV space in bright areas Problems with non-linearity of HSV space in bright areas

Full Joint-Probability Distribution We have enough data, so why not? We have enough data, so why not? Provides most accurate per-pixel classification. Provides most accurate per-pixel classification. Allows use to circumvent choosing a decision boundary. We can simply use Bayes rule. Allows use to circumvent choosing a decision boundary. We can simply use Bayes rule.

Slices from 3D Joint Distribution

Skin-probability Image Obtained From Applying Bayes Rule

Color-based Skin Separation What color space to use? What color space to use? HSV if separability of distributions is necessary HSV if separability of distributions is necessary How to separate out the skin color? How to separate out the skin color? Parametric is fast but loose in HSV Parametric is fast but loose in HSV Provides a binary mapping and requires choosing thresholds Provides a binary mapping and requires choosing thresholds Full PDF is accurate in any color space Full PDF is accurate in any color space Can be fast if done correctly (table lookup) Can be fast if done correctly (table lookup) No thresholds: produces a pure probability map No thresholds: produces a pure probability map

Spatial Analysis Method Morphological Morphological Obtain binary mask through thresholding Obtain binary mask through thresholding Perform morphological operations to separate and identify blobs corresponding to faces Perform morphological operations to separate and identify blobs corresponding to faces Difficult due to overlapping faces Difficult due to overlapping faces Template Match Template Match Search a scene for prototypical face image Search a scene for prototypical face image Need to decide which data to work with (luminance vs. skin probability) Need to decide which data to work with (luminance vs. skin probability)

Template Matching Using the skin-probability image: Greatly simplifies information content Greatly simplifies information content Simple information  simple algorithm Allows algorithm to focus on the single best facial clue: oval-shaped skin regions Allows algorithm to focus on the single best facial clue: oval-shaped skin regions Allows us to avoid creating a binary mask Allows us to avoid creating a binary mask

Correlation of simple template with Skin-probability image

Process of Inclusion/Elimination Iteratively pick ‘strong’ regions of the skin- probability image as faces: For each template search for matching face shapes (convolution peaks) For each template search for matching face shapes (convolution peaks) For each detected face, ‘subtract/erase’ the region from image to avoid duplicate detection For each detected face, ‘subtract/erase’ the region from image to avoid duplicate detection Stop when no significant skin regions remaining Stop when no significant skin regions remaining

Positive Detection and Elimination

Template and Threshold Selection Critical step of our algorithm Critical step of our algorithm Potential problems: Potential problems: Template matching face of a different size Template matching face of a different size Small template leads to double hits later Small template leads to double hits later Large template leads to missed faces Large template leads to missed faces Bad thresholds  low sensitivity or low specificity Bad thresholds  low sensitivity or low specificity Solution: Solution: Use templates of many sizes, going from largest to smallest Use templates of many sizes, going from largest to smallest Set threshold as high as possible without sacrificing sensitivity Set threshold as high as possible without sacrificing sensitivity

Algorithm Implementation 1. Load probability and template data 2. Down-sample the image by factor 2:1 3. Calculate the face-probability image by color 4. Remove hands/arms 5. Template match with skin-probability image 6. Eliminate false positive hits on necks of large faces 7. Remove patterned hits

Overall Results

Conclusions Recognition of face-shaped blobs from the skin- probability map works excellently Recognition of face-shaped blobs from the skin- probability map works excellently Requires that the skin colors be well known Requires that the skin colors be well known Requires that the general face sizes be well known Requires that the general face sizes be well known Our set of images was relatively consistent in terms of these factors Our set of images was relatively consistent in terms of these factors