Face Detection Final Presentation Mark Lee Nic Phillips Paul Sowden Andy Tait 9 th May 2006

Overview Detect multiple faces in images and return their locations  Pre-processor to face recognition  People tracking  Expression analysis Challenges include:  Pose, occlusion/beards/glasses etc, expression, orientation, lighting conditions

Our Chosen Techniques Compare 2 methods:  Feature Based Selection of image processing methods  Machine Learning (AdaBoost) Currently the de facto method C++ implementation (with CImg library)

Feature Based Method Uses image processing techniques to extract low-level facial features 1. Identify and segment possible regions of interest  Skin tones and overall proportions of skin regions. 2. Determine if each region contains face  Locate facial features – eyes and mouth  Compare feature locations using rules-based approach

Skin Colour Map Many different implementations Chosen Hsu model – good performance and parameters available Non-linear mapping of YCrCb space Ellipse in CrCb plane defines skin colour Works for multiple skin types (from Hsu et al)

Colour Correction Choose ‘white’ reference level from image based on maximum 5% luminance values Find scale factor to make this pure white in RGB space Apply same scaling to all other image pixels Colour corrected imageImage with red tint

Colour Correction Benefits Original Image Colour Corrected Image

Region Finding Algorithm Two Pass:  Identify local region connectivity and build equivalence map  Re-label equivalences Remove Regions:  Small area (noise)  Min. dimensions  Aspect ratio Output:  Set of bounding boxes containing possible face candidates  Passed to Eye+Mouth feature detection stage Connectivity Region Neighbourhood Detected Face Candidates

Skin Adjacency Problem Overlapping regions of skin colour cause false detection Skin Map

Updated Connectivity Criteria New connectivity requirements:  Is a Skin pixel  Y ±8 between adj regions Still can be problematic with some images with inadequate Y variation Edge criteria tested to augment Y channel connectivity metric  Faces broken into separate regions by glasses, eyebrows, strong lighting  Decided that performance gain was not worth the complexity increase Y Channel

Localisation of facial features Using the skin tone bounding boxes, aim to located facial features. Eye and mouth detection based on colour information and morphological techniques. Using YC b C r colour space. Then use heuristic method to discriminate between Face and Not-Face

Eye Detection - Chrominance Chrominance eyemap:  High C b and low C r values around eyes  High response in C b /C r  Also divide by original luminance – eyes usually dark areas. OriginalChrominance eyemap

Eye Detection - Luminance Greyscale morphological methods to emphasise areas of high luminance change – e.g. eyes Results – highlights eyes, plus sometimes nose, open mouthes Combine with chrominance map to pinpoint eye regions. LuminanceDilated luminance LuminanceLuminance + Crominance

Mouthmap Mouth stronger red and weaker blue  C r higher than C b  Low response in C r /C b  High response in C r 2  η based on ratio of C r 2 to C r /C b Original Mouthmap

Results Mouth – high luminance High chrominance map on dark hair Hair – high chrominance, but low luminance Luminance affected by reflection Blue – based on luminance dilation peaks Red – chrominance eyemap/luminance peaks Green – mouthmap peak

Heuristic Rules Eliminate false eye candidates:  Merge repeated eye points  Eye radial distance from mouth  Distance above mouth  Eye-eye-mouth triangle – min and max angles Removal of false eye candidates

Feature Based Methods Conclusions Face detection correct in many cases. Skin detection and segmentation works well  Sometimes confused by overlapping regions in complex arrangements Eye and mouth detection mostly correct  Red-eye, jewellery, other skin areas with high contrast differences can cause false-positives Heuristic elimination of false positives works well and fast  Additional rules could improve robustness Speed improvements possible for the intensive segmentation stage. Subsequent stages very fast.

[1] S. Cooray and N. O’Connor. A Hybrid Technique for Face Detection in Color Images. In Proc. IEEE Conference on Advanced Video and Signal Based Surveillance, pages , [2] R. Hsu and M. Abdel-Mottaleb. Face Detection in Color Images. In IEEE Transactions on Pattern Analysis and Machine Intelligence, volume 25 issue 5, pages , 2002