Download presentation

Presentation is loading. Please wait.

Published byEmmeline Savary Modified over 2 years ago

1
Face Alignment by Explicit Shape Regression Xudong CaoYichen WeiFang WenJian Sun Microsoft Research Asia

2
Outline Introduction Face Alignment by Shape Regression Implementation details Experiments Discussion and Conclusion

3
Introduction

4
Introduction most alignment approaches can be classiﬁed into two categories ◦ optimization-based minimize another error function that is correlated to instead ◦ regression-based learn a regression function that directly maps image appearance to the target output

5
Introduction the shape constraint is essential in all methods ◦ Most previous works use a parametric shape model to enforce such a constraint a novel regression-based approach without using any parametric shape models ◦ the regressor realizes the shape constraint in an non-parametric manner

6
Introduction a boosted regressor to progressively infer the shape ◦ the early regressors handle large shape variations and guarantee robustness ◦ the later regressors handle small shape variations and ensure accuracy two-level boosted regression ◦ shape-indexed features ◦ correlation-based feature selection method

7
Face Alignment by Shape Regression use boosted regression to combine T weak regressors (R 1,...R t,...,R T ) in an additive manner Given a facial image I and an initial face shape S 0 each regressor computes a shape increment δS from image

8
Face Alignment by Shape Regression where the tth weak regressor R t updates the previous shape S t −1 to the new shape S t Given N training examples, the regressors (R 1,...R t,...,R T ) are sequentially learnt until the training error no longer decreases Where is the estimated shape in previous stage

9
Face Alignment by Shape Regression Two-level cascaded regression ◦ learn each weak regressor R t by a second level boosted regression ◦ the shape-indexed image features are ﬁxed in the second level ◦ they are indexed only relative to S t−1 and no longer change when those r’s are learnt

10
Face Alignment by Shape Regression Primitive regressor ◦ use a fern as primitive regressor r δS b : regression output Ω b : alignment error S i : the estimated shape in the previous step

11
Face Alignment by Shape Regression Primitive regressor (Cont.) ◦ The solution for above is the mean of shape differences ◦ To overcome over-ﬁtting in the case of insufﬁcient training data in the bin, a shrinkage is performed as where β is a free shrinkage parameter

12
Face Alignment by Shape Regression Non-parametric shape constraint ◦ the ﬁnal regressed shape S can be expressed as the initial shape S 0 plus the linear combination of all training shapes ◦ as long as the initial shape S 0 satisﬁes the shape constraint, the regressed shape is always constrained to reside in the linear subspace constructed by all training shapes

13
Face Alignment by Shape Regression

14
Shape-indexed (image) features ◦ use simple pixel-difference features, i.e., the intensity difference of two pixels in the image ◦ A pixel is indexed relative to the currently estimated shape rather than the original image coordinates ◦ for each weak regressor R t in the ﬁrst level, randomly sample P pixels, P 2 pixel-difference features are generated

15
Face Alignment by Shape Regression n-best ◦ randomly generating a pool of ferns and selecting the one with minimum regression error ◦ Correlation-based feature selection ◦ exploit the correlation between features and the regression target

16
Face Alignment by Shape Regression Correlation-based feature selection (Cont.) ◦ a good fern should satisfy two properties each feature in the fern should be highly discriminative to the regression target correlation between features should be low so they are complementary when composed

17
Face Alignment by Shape Regression To ﬁnd features satisfying such properties 1.Project the regression to a random direction to produce a scalar 2.Among P 2 features, select a feature with highest correlation to the scalar 3.Repeat steps 1. and 2. F times to obtain F features 4.Construct a fern by F features with random thresholds

18
Implementation details Training data augmentation Multiple initializations in testing Running time performance Parameter settings

19
Experiments BioID ◦ It consists of 1,521 near frontal face images captured in a lab environment LFPW ◦ Its images are downloaded from internet and contain large variations LFW87 ◦ The images mainly come from the LFW dataset ◦ has 87 annotated landmarks,much more than that in BioID and LFPW

20
Experiments Comparison with previous work ◦ Comparison to [1] on LFPW ◦ Comparison to [12] on LFW87 [1] Localizing parts of faces using a concensus of exemplars. In CVPR, 2011 [12] Face alignment via component-based discriminative search. In ECCV, 2008

21
Experiments Comparison with previous work (Cont.) ◦ Comparison to previous methods on BioID [20] Fully automatic facial feature point detection using gabor feature based boosted classiﬁers [5] Feature detection and tracking with constrained local models. BMVC, 2006 [14] Locating facial features with an extended active shape model. ECCV, 2008 [19] Facial point detection using boosted regression and graph models. In CVPR, 2010

22
Experiments Algorithm validation and discussions ◦ Two-level cascaded regression ◦ Shape indexed feature mean error of local index method, which is much smaller than the mean error of global index method ◦ Feature selection

23
Experiments Algorithm validation and discussions (Cont.) ◦ Feature range the distance between the pair of pixels normalized by the distance between the two pupils

24

25
Discussion and Conclusion By jointly regressing the entire shape and minimizing the alignment error, the shape constraint is automatically encoded. The resulting method is highly accurate, efﬁcient, and can be used in real time applications such as face tracking

26
Thank for your listening

Similar presentations

© 2016 SlidePlayer.com Inc.

All rights reserved.

Ads by Google