1. Facial Feature Extraction Yuri Vanzine C490/B657 Computer Vision

2. Potentially Bigger Project Future Goal: Application Component Receive Frontal and Profile Images of User’s Head Localize face, locate position of eyes, mouth to determine face geometry Use face geometry to locate secondary features, eg. nose, facial hair, ears. Extract face geometry and features and transform them into quantative feature vectors. Apply features and geometry to 3-D model in a standard 3-D model format, eg. 3DS, OpenNURBS, DXF, DWG, etc. Need: Provide Ability to import Avatar appearance from web cam Application needs to perform the following:

3. Part of the Project for C490/B657 Face Localization, Locating Eyes and Mouth Face Localization Mouth Location Left and Right Eye Location Process phases:

4. Face Localization Existing Techniques: RGB, YCbCr, HSV Late Fusion Several weak, fast classifiers instead of ‘expensive’ ones Environment is controlled: sufficient lighting, head position, subject is told to place head a certain way, but our classifier is rotation, scale and translation invariant! Skin-tone techniques in various colorspaces Template-based Eigen-faces Other Technique used for Face Detection:

5. RGB, YCbCr, HSV Late Fusion

6. RGB Colorspace Matlab Code: for xc=1:x for yc = 1:y iR = (int)R(xc,yc); iG = (int)G(xc,yc); iB = B(xc,yc); mxRGB = max(max(iR, iG), iB); mnRGB = min(min(iR, iG), iB); if iR>95 && iG>40 && iB>20 && mxRGB-mnRGB>15 && abs(iR-iG) > 15 && iR > iG && iR > iB else F(xc,yc,1)=0; F(xc,yc,2)=0; F(xc,yc,3)=0; end

7. HSV Colorspace Bounding planes, defining skin-color, determined in [3]: Matlab Code: for xc=1:x for yc = 1:y iH = double(H(xc,yc))*255; iS = double(S(xc,yc))*255; iV = double(V(xc,yc))*255; if iH>=0 && iS>=15 && iS>=(0.75*iH+0.3*iV-30) || iS<=-iH-0.1*iV+110 && iH<=- 0.4*iV+75 && iS<=0.08 * (100-iV) *iH +0.6* iV else F(xc,yc,1)=0; F(xc,yc,2)=0; F(xc,yc,3)=0; end

8. YCbCr Colorspace “YCbCr simple” from [3] (could not get it to work):

9. Late Fusion of Colorspaces RGB HSV YCbCrTo be implemented… Bounding Box RGB and HSV in this implementation is luminance-dependent. Frequent false positives are likely. This implementation should be replaced by YCbCr from [4].

10. More examples of Bounded Images

11. Mouth Location (YCbCr) See [4] for further detail.

12. Mouth Location (continued) Mouth MaskConvex Face Bounding Box

13. Eye Location (YCbCr) Eye Map

14. Eye Location (YCbCr) Complexity Map 565 71506 7 6 Average of Differences between pixels in the 8-neighborhood, followed by dilation using a disk of 5 pixels

15. Eye Location (YCbCr) Eye Map and Complexity Map Combined AND Notice problems because bounding box needs to be replaced by Connected Component Mask AND

16. Matlab Demo of Bounding Box, EyeMap and MouthMap

17. Current Problems and Future Work Problem: Current skin-tone clustering is not robust. Solution: YCbCr luma-independent detection needs to be implemented. Problem: Manual thresholding is implemented. Solution: Automatic thresholding is desired. Problem: Bounding Box instead of Skin-Tone Connected component Solution: Improve algorithm extracting just face-mask

18. YCbCr Colorspace (continued)

19. This model alone is sufficient. Taken from [4] (to be implemented in the future). YCbCr is luminance-dependent. Direct use of YCbCr results in false positives. Therefore YCbCr must be made luma-independent.

20. YCbCr Colorspace (continued)

22. References 1.A Survey on Pixel-Based Skin Color Detection Techniques, Vladimir Vezhnevets, Vassili Sazonov, Alla Andreeva, http://www.graphicon.ru/2003/Proceedings/Technical/paper509.pdf 2.Sanjay Kr. Singh, D. S. Chauhan, Mayank Vatsa, Richa Singh, A Robust Skin Color Based Face Detection Algorithm, http://www2.tku.edu.tw/~tkjse/6-4/6-4-6.pdfhttp://www2.tku.edu.tw/~tkjse/6-4/6-4-6.pdf 3.Face Detection in Color Images using Wavelet Packet Analysis, C. Garcia, G. Zikos, G. Tziritas, http://www.csd.uch.gr/~tziritas/papers/ICMCS-99.pdf http://www.csd.uch.gr/~tziritas/papers/ICMCS-99.pdf 4. Face Detection in Color Images, Rein-Lien Hsu, Mohamed Abdel-Mottaleb, Anil K. Jain, http://www.csd.uch.gr/~hy471/papers/FaceDetectionColor.pdf http://www.csd.uch.gr/~hy471/papers/FaceDetectionColor.pdf