1. Using Dimensionality Reduction to Improve Local Regression on Facial Age Estimation 國立台灣大學 資訊工程學系 R02922098 簡嘉宏 指導教授：張智星 博士

2. Outline  Introduction  Related work  Proposed method  Performance evaluation  Conclusions and future work 2/28

3. Outline  Introduction  Motivation  Objective  Challenges  Our framework  Related work  Proposed method  Performance evaluation  Conclusion and future work 3/28

4. Introduction  Motivation Application for face recognition topics, ex: surveillance system, advertising recommendation system. Programing contest, ex: UTMVP  Objective Use dimensionality reduction algorithm to improve local regression performance on facial age estimation system  Challenges Ordinal relationship Dataset imbalance 4/28 3226 62

5. Introduction  Our framework 5/28 Input image Feature extraction Age estimation model Predicted age Distance metric learning Manifold learning Dimensionality reduction Input img : Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label- sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641.

6.  Introduction  Related work  Feature extraction  Manifold learning  Distance metric learning  Age determination model  Proposed method  Performance evaluation  Conclusion and future work Outline 6/28

7. Related work TopicMethod Feature extractionLocal binary pattern (LBP) Gabor filter ASM/AAM Manifold learningLocal sensitive discriminate analysis (LSDA) Maximum margin projection (MMP) Distance metric learningRelevant component analysis (RCA) Discriminative component analysis (DCA) Age estimation modelKNN-SVR SVM-BDT Ordinal hyperplanes Ranker (OHRanker) 7/28

8. Outline  Introduction  Related work  Proposed method  Active appearance model  Maximum margin projection  Label-sensitive DCA  kNN-SVR  Performance evaluation  Conclusion and future work 8/28

9. Active appearance model 9/28

10. Active appearance model 10/28

11. Maximum margin projection  Introduction Manifold learning Semi-supervised dimensionality reduction Objective ◦Nearby points with same label are close to each other ◦Nearby points with different labels are far apart Discovering the local manifold structure by Nearest Neighbor Graph 11/28

12. Maximum margin projection  MMP algorithm 12/28 Laplacian eigenmaps

13. Discriminative component analysis  Introduction Distance metric learning Objective: ◦Find a good distance metric which can be used for similarity measure Overcome the disadvantage of relevant component analysis (RCA) Supervised dimensionality reduction Learn an optimal transformation matrix by feature variance analysis 13/28 Relevant component analysis

14. 14/28 Discriminative component analysis

15. 15/28 … Age label i Age label 1 Age label k

16. Label sensitive - DCA 16/28

17. kNN-SVR  Because of the advantage about MMP and ls-DCA, we use kNN to do local regression  Algorithm 17/28 Input query kNN … … Regression

18. Outline  Introduction  Related work  Proposed method  Performance evaluation  MORPH dataset  Dataset imbalance  Setting  Experiment  Error analysis  Conclusion and future work 18/28

19. MORPH dataset 19/28

20. Dataset imbalance  To deal with this challenge, we modify neighbor size when doing k-nearest neighbor  This method called Neighbor Size Modification [1] which gives a huge weight to insufficient range 20/28 [1] Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label- sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641.

21. Setting 21/28

22. Experiment MethodMAE AAM ＋ kNN-SVR6.3374 AAM ＋ MMP ＋ kNN-SVR5.8872 AAM ＋ DCA ＋ kNN-SVR5.9929 AAM ＋ MMP ＋ DCA ＋ kNN-SVR5.9228 AAM ＋ DCA ＋ MMP ＋ kNN-SVR5.8279 22/28 MethodMAE AAM ＋ *kNN-SVR5.9172 AAM ＋ *MMP ＋ *kNN-SVR5.7279 AAM ＋ ls-DCA ＋ *kNN-SVR5.6931 AAM ＋ *MMP ＋ ls-DCA ＋ *kNN-SVR5.6242 AAM ＋ ls-DCA ＋ *MMP ＋ *kNN-SVR5.7987 Notation: Neighbor size modification (*), Label sensitive (ls-) MethodMAE AAM ＋ OHRanker (without cost-sensitive) 6.4644 (4.48) AAM ＋ SVM-BDT (33/41/51) 6.7607 (4.2) Chao’s method: AAM ＋ ls-RCA ＋ ls-MFA ＋ kNN-SVR 5.8150 (4.44)

23. Error analysis - 1 23/28  The limit of neighbor size modification and its trade off. 61 kNN

24. Error analysis - 2  The image which has beard and the neighbors almost have beard. 24/28 kNN

25. Error analysis – 3,4  Face shape and expression of one’s eyes maybe influence the neighbor search.  Mouth maybe be the clue of judgement. 25/28 kNN

26. Outline  Introduction  Related work  Proposed method  Performance evaluation  Conclusion and future work 26/28

27. Conclusions and future work  Conclusions The local regression accuracy is actually promoted by our method. The images which was found by Neighbor Search look like input query,but we think it will more appropriate for facial expression recognition.  Future work Need to make the best use of the dataset, while 2738/55134 is not a effective usage. Need more experience on how to build a proper AAM model for different applications or feature combination. Deal with the dataset imbalance problem, we also can do some modification on SVR. Build this system on mobile device. 27/28

28. Thanks for listening! 28/28

29. Related work  Feature extraction LBP (Local binary pattern) ◦Ojala, Timo, Matti Pietikainen, and David Harwood. "Performance evaluation of texture measures with classification based on Kullback discrimination of distributions." Pattern Recognition, 1994. Vol. Gabor filter ◦Lee, Tai Sing. "Image representation using 2D Gabor wavelets." Pattern Analysis and Machine Intelligence, IEEE Transactions on 18.10 (1996): 959-971. ASM/AAM ( Active shape/appearance model ) ◦Cootes, Timothy F., et al. "Active shape models-their training and application."Computer vision and image understanding 61.1 (1995): 38-59. ◦Cootes, Timothy F., Gareth J. Edwards, and Christopher J. Taylor. "Active appearance models." Computer Vision—ECCV’98. Springer Berlin Heidelberg, 1998. 484-498. BIF ( Bio-inspired feature ) ◦Guo, Guodong, et al. "Human age estimation using bio-inspired features."Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on. IEEE, 2009. 29/28

30. Related work  Manifold learning LSDA (Locality sensitive discriminant analysis) ◦Cai, Deng, et al. "Locality Sensitive Discriminant Analysis." IJCAI. 2007. MMP (Maximum margin projection) ◦He, Xiaofei, Deng Cai, and Jiawei Han. "Learning a maximum margin subspace for image retrieval." Knowledge and Data Engineering, IEEE Transactions on20.2 (2008): 189-201.  Distance metric learning RCA (Relevant component analysis) ◦Shental, Noam, et al. "Adjustment learning and relevant component analysis."Computer Vision—ECCV 2002. Springer Berlin Heidelberg, 2002. 776-790. DCA (Discriminative component analysis) ◦Hoi, Steven CH, et al. "Learning distance metrics with contextual constraints for image retrieval." Computer Vision and Pattern Recognition, 2006 IEEE Computer Society Conference on. Vol. 2. IEEE, 2006. 30/28

31. Related work  Age determination model kNN-SVR ◦Chao, Wei-Lun, Jun-Zuo Liu, and Jian-Jiun Ding. "Facial age estimation based on label- sensitive learning and age-oriented regression." Pattern Recognition46.3 (2013): 628-641. SVM-BDT ◦Han, Hu, Charles Otto, and Anil K. Jain. "Age estimation from face images: Human vs. machine performance." Biometrics (ICB), 2013 International Conference on. IEEE, 2013. Label distribution learning ◦Geng, Xin, Chao Yin, and Zhi-Hua Zhou. "Facial age estimation by learning from label distributions." Pattern Analysis and Machine Intelligence, IEEE Transactions on 35.10 (2013): 2401-2412. Ordinal hyperplanes Ranker ◦Chang, Kuang-Yu, Chu-Song Chen, and Yi-Ping Hung. "Ordinal hyperplanes ranker with cost sensitivities for age estimation." Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on. IEEE, 2011. 31/28

32.  Chao 127  OHRanker 32/28