1. Automatic Face Recognition Using Color Based Segmentation and Intelligent Energy Detection Michael Padilla and Zihong Fan Group 16 EE368, Spring 2002-2003

2. Project Objective Given a digital image of attractive and intelligent EE368 students and teaching staff, detect the presence of faces in the image and output their location and (if poss.) gender.

3. Basic System Summary Color-space Based Segmentation Morphological Image Processing Matched Filtering Peak/Face Detector Input Image Face Estimates Final System Initial Design  Reduced Eigenface-based coordinate system defining a “face space”, each possible face a point in space.  Using training images, find coordinates of faces/non-faces, and train a neural net classifier.  Abandoned due to problems with neural network: lack of transparency, poor generalization.  Replaced with our secondary design strategy:

4. H vs. S vs. V (Face vs. Non-Face) For faces, the Hue value is seen to typically occupy values in the range H < 19 H > 240 We use this fact to remove some of the non-faces pixels in the image.

5. Y vs. Cr vs. Cb In the same manner, we found empirically that for the YCbCr space that the face pixels occupied the range 102 < Cb < 128 125 < Cr < 160 Any other pixels were assumed non-face and removed.

6. R vs. G vs. B Finally, we found some useful trends in the RGB space as well. The Following rules were used to further isolate face candidates: 0.836·G – 14 < B < 0.836·G + 44 0.89·G – 67 < B < 0.89·G + 42

7. Removal of Lower Region – Attempt to Avoid Possible False Detections Just as we used information regarding face color, orientation, and scale from The training images, we also allowed ourselves to make the assumption that Faces were unlikely to appear in the lower portion of the visual field: We Removed that region to help reduce the possibility of false detections.

8. Morphological Processing Step 1: Open Operation After removing pixels based on color space considerations, removed specs initially by use of the open operation with a window of size 3x3.

9. Morphological Processing Step 2: Small “Blob” Removal Model the average size of head blobs in the training reference image. Remove blobs below one standard deviation. In addition, we: Convert to grayscale. In our case, no more color information to extract. Apply mean removal+histogram equalization -> flatten and bring out details.

10. Template Design Manually selected a number of quality faces: centered, straight, lighting, diverse. Measured face dimensions and used Matlab to uniformly scale and align them. Efforts resulted in 26 sample faces added to produce the final template. Original Faces Final Face Template

11. Matched Filter Operation Masked Input Image Apply Matched Filter Compare peaks To Threshold, T(n) If Peak > Tn, Declare face Scale and Rotate Face Coordinates Pre-processing When faces are detected, we remove the corresponding portion of the masked input image to try to avoid multiple and false detections For each scale and rotation, the threshold, T(n), decreases Algorithm is sensitive to errors made in the pre-processing stage. While (remaining mask area to analyze) { for s = 1:S { % scale for r = 1:R { % rotation for thrshld = Max:Min template = temp(mother_temp, s, r); peaks = conv(mask_image, template); face = detector(peaks, thrshld); if (face) { adjust mask_image; adjust remaining mask area; }

12. Matched Filtering - Steps Masked Input Image Apply Matched Filter Compare peaks To Threshold, T(n) If Peak > Tn, Declare face Pre-processing Face Coordinates

13. Face Detection Steps and Progressive Masking After detecting peaks at the output of the matched filter, the following steps are taken: Peaks within threshold range -> faces. Face pixels are convolved with oval face mask of appropriate scale. Removes neighborhood of detected face pixel. After all processing, face pixels are consolidated into blobs by dilation. Finally, centroids of blobs deemed to be face centers.

14. General Results For training images, run time 80 – 110 sec. Detection results range from 83% - 100%. Main Strengths: Intuitive and (thus far) accurate. Main Weaknesses: Sensitive to errors in pre-processing. (Example result for Training_7.jpg)

15. Conclusions In most cases, effective use of color space – face color relationships and morphological processing allowed effective pre-processing. For images trained on, able to detect faces with reasonable accuracy and miss and false alarm rates. Adaptive adjustment of template scale, angle, and threshold allowed most faces to be detected. Decision Feedback Masking reduced multiple and false detection rate If additional time, would have liked to: Pursue the Eigenimage approach further with MRC or SVM. Explore use of Wavelet spaces for face/gender detection.

16. References Bernd Girod, EE368 Class Lecture Notes, Spring 2002-2003 R. Gonzalez and R. Woods, “Digital Image Processing – 2 nd Edition”, Prentice Hall, 2002 C. Garcia et al., “Face Detection in Color Images Using Wavelet Packet Analysis”. M. Elad et al., “Rejection Based Classifier for Face Detection”, Pattern Recognition Letters, V.23, 2002.