ECE738 Advanced Image Processing Face Detection IEEE Trans. PAMI, July 1997

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 2 Problem Statement Given an image, detect the presence of human face- like objects and label them using rectangular regions Mathematical formulation: –Two-class pattern classification problem, –detection problem, –hypothesis testing problem Solution: –Template matching, –Feature matching Difficulties: –Face templates have too much variations: Size, lighting, pose, facial wares (eye glasses), beard, etc.

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 3 Boosted Cascade of Simple Features Feature: –Binary Rectangular masks. Detection: –Convolve feature mask with image –Compare output to a threshold Decision structure –Use Adaboost to select feature –Use cascaded decision tree to enable decision fusion [viola01] Viola, CVPR’01, pp. I

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 4 Adaboost A feature selection method. Given the training data set, and a classifier, select one feature that minimizes the training error. Then select the next feature, etc. In calculating training error, a data sample is given a weight. The weight decreases if it has been correctly classified with the previous feature and remain unchanged if not. Hence additional features are selected to correct the remaining errors after selecting previous features. Initialization: training data: {x i, y i } i=1 n y i  {0, 1}, a given classifier g(x), a set of features {j} w 1,i  1/m (1/ ) if y i = 0, (1), Iteration For t = 1, …, T, Normalize w i s.t.  I w t,i = 1. Find feature j * = arg. Min j e j, denote e t = e j*,  t = e t /(1−e t ) Update w t+1,I = w t,I (  t ) 1-ei, e i = 0 if g(x i ) = y i ; = 1 otherwise. Result let  t = -log  t, final classifier is g(x) = 1 if = 0 otherwise. [viola01] Viola, CVPR’01, pp. I

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 5 Decision Cascade Each classifier is trained to yield very low false negative at the cost of moderate false positive Successive classifier operates on candidate positions detected by preceding classifier A special kind of decision fusion for adaboost procedure. Majority voting and other decision fusion methods may also be used. 1 T F 2 T F 3 T F Further processing All sub- window Reject sub-window [viola01] Viola, CVPR’01, pp. I

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 6 3-step Face Detection

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 7 Improvements of Adaboost Use support vector machine (SVM) with linear kernel as the basic classifier structure Use a modified cascade structure called boosting chain –Classifier of previous stage is used as the first stage classifier in the current boosting stage. –Individual classifiers results are combined using another linear classifier trained with SVM. New rectangular templates

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 8 Skin Color Detection It is known that skin color in chromatic color space is quite distinct and can be used as a key feature to segment human skins. Gray scale value is also used to reduce false positive detection. A SVM classifier with polynomial kernel is used. Often region growing method will be used to identify the region where potentially human face may locate. Useful only for color images.

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 9 Multi-view Face Detection - in-plane rotation correction Pose variations makes face detection difficult. Create two rotated versions of original image (±30 o ), and perform coarse face detection on all three images to estimate in-plane rotation angles between ±45 o. Then perform detailed face detection using the in-plane rotation corrected face image.

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 10 Results Data sets –Self collected from web, stock photos –MIT/CMU database ojects/project_419.html 125 gray scale images consisting of 483 faces (manually labeled) Computation cost –Average detection complexity (ADC)

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 11 Results

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 12 Results

ECE738 Advanced Image Processing (C) 2005 by Yu Hen Hu 13 Challenges and Potential Solutions Rectangular region features is scaling dependent.  multi-resolution analysis Rectangular region features is lighting sensitive.  larger facial mask (Sinha’s work) View-based approach for pose variations requires too much computation  in-plane rotation invariant features Adaboost is a greedy heuristic for feature selection  genetic algorithm? Sub-set selection algorithm, Decision cascade/decision chain is a special case of decision fusion  voting, weighted voting, and other decision fusion methods?