Presentation is loading. Please wait.

Presentation is loading. Please wait.

Face Alignment with Part-Based Modeling Vahid Kazemi Josephine Sullivan CVAP KTH Institute of Technology.

Similar presentations

Presentation on theme: "Face Alignment with Part-Based Modeling Vahid Kazemi Josephine Sullivan CVAP KTH Institute of Technology."— Presentation transcript:

1 Face Alignment with Part-Based Modeling Vahid Kazemi Josephine Sullivan CVAP KTH Institute of Technology

2 Objective: Face Alignment Find the correspondences between landmarks of a template face model and the target face. Annotated images (source: IMM dataset)Test image (source: YouTube)

3 Why: Possible Applications The outcome can be used for: -Motion Capture: by determining head pose and facial expressions. -Face Recognition: by comparing registered facial features with a database. -3D Reconstruction: by determining camera parameters using correspondences in an image sequence -Etc.

4 Global Methods Overview: -Create a constrained generative template model -Start with a rough estimate of face position. -Refine the template to match the target face. Properties: -Model deformations more precisely -Arbitrary number of landmarks Examples: -Active Shape Models [Cootes 95] -Active Appearance Model [Cootes 98] -3D Morphable Models [Blanz 99]

5 Part-Based Methods Overview: -Train different classifiers for each part. -Learn constraints on relative positions of parts. Properties: -More robust to partial occlusion -Better generalization ability -Sparse results Examples: -Elastic Bunch Graph Matching [Wiskott 97] -Pictorial Structures [Felzenszwalb 2003]

6 Our approach to face alignment How can we avoid the draw backs of existing models?

7 Our approach to face alignment Find the mapping, q, from appearance to the landmark positions: But q is complex and non-linear…

8 Linearizing the model Use piece-wise linear functions

9 Linearizing the model Use a part based model

10 Linearizing the model Use a suitable feature descriptor Feature Descriptor

11 Part Selection Criteria Detect the parts accurately and reliably -Contain strong features Ensure a simple (linear) model -Minimum variation Capture the global appearance -Cover the whole object

12 Part Selection for the face We chose nose, eyes, and mouth as good candidates Image from IMM dataset

13 Appearance descriptor Variation of PHOG descriptor -Divide the patch into 8 sub-regions -Recursively repeat for square regions

14 Part detection Build a tree-structured model of the face, with nose at the root, and eyes and mouth as the leafs of the tree.

15 Part detection Detect the parts by sliding a patch on image and calculating the Mahalanobis distance of the patch from the mean model

16 Part detection Find the optimal solution by minimizing the pictorial structure cost function: We can solve this efficiently by using generalized distance transform [Felzenszwalb 2003] by limiting the cost function

17 Regression Model the mapping between the patch’s appearance feature (f) and its landmark positions (x) as a linear function: Estimate weights from training set using Ridge regression

18 Regression Comparison of different regression methods

19 Robustify the regression function Why Compensate for bad part detection Deformable parts don’t exactly fit in a box How Extend training set by adding noise to part positions

20 Experiments Use 240 face images from IMM dataset. Dataset contains still images from 40 individual subjects with various facial expressions under the same lighting settings 58 landmarks are used to represent the shape of subjects

21 Results Comparison of localization accuracy of our algorithm comparing to some existing methods on IMM dataset. * Mean error is the mean Euclidean distance between predicted and ground truth location of landmarks in pixels

22 Results The results of cross validation on IMM dataset Predicted Ground truth

23 Demo More videos:

24 Conclusion and future work Part-Based models can be used to simplify complicated models The choice of parts is very important HOG descriptors are not fully descriptive

25 Questions?

Download ppt "Face Alignment with Part-Based Modeling Vahid Kazemi Josephine Sullivan CVAP KTH Institute of Technology."

Similar presentations

Ads by Google