1. Active Appearance Models Dhruv Batra ECE CMU

2. Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 (H.Burkhardt & B. Neumann Ed.s). Vol. 2, pp. 484-498, Springer, 1998 2.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", IEEE PAMI, Vol.23, No.6, pp.681-685, 2001 3.G.J. Edwards, A. Lanitis, C.J. Taylor, T. F. Cootes. “Statistical Models of Face Images Improving Specificity”, BMVC (1996)

3. Essence of the Idea  “Interpretation through synthesis”  Form a model of the object/image (Learnt from the training dataset) I. Matthews and S. Baker, "Active Appearance Models Revisited," International Journal of Computer Vision, Vol. 60, No. 2, November, 2004, pp. 135 - 164.

4. Essence of the Idea (cont.)  Explain a new example in terms of the model parameters

5. So what’s a model Model “Shape” “texture”

6. Active Shape Models training set

7. Texture Models warp to mean shape

8. Random Aside  Shape Vector provides alignment = 43 Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

9. Random Aside  Alignment is the key 1. Warp to mean shape 2. Average pixels Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt

10. Random Aside  Enhancing Gender more same original androgynous more opposite D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp. 70-76

11. Random Aside (can’t escape structure!) Alexei (Alyosha) Efros, 15-463 (15-862): Computational Photography, http://graphics.cs.cmu.edu/courses/15-463/2005_fall/www/Lectures/faces.ppt Antonio Torralba & Aude Oliva (2002) Averages: Hundreds of images containing a person are averaged to reveal regularities in the intensity patterns across all the images.

12. Random Aside (can’t escape structure!) Tomasz Malisiewicz, http://www.cs.cmu.edu/~tmalisie/pascal/trainval_mean_large.png

13. Random Aside (can’t escape structure!) “100 Special Moments” by Jason Salavon Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml

14. Random Aside (can’t escape structure!) “Every Playboy Centerfold, The Decades (normalized)” by Jason Salavon 1960s1970s1980s Jason Salavon, http://salavon.com/PlayboyDecades/PlayboyDecades.shtml

15. Back (sadly) to Texture Models raster scan Normalizations

16. PCA Galore Reduce Dimensions of shape vector Reduce Dimension of “texture” vector They are still correlated; repeat..

17. Object/Image to Parameters modeling ~80

18. Playing with the Parameters First two modes of shape variationFirst two modes of gray-level variation First four modes of appearance variation

19. Active Appearance Model Search  Given: Full training model set, new image to be interpreted, “reasonable” starting approximation  Goal: Find model with least approximation error  High Dimensional Search: Curse of the dimensions strikes again

20. Active Appearance Model Search  Trick: Each optimization is a similar problem, can be learnt  Assumption: Linearity  Perturb model parameters with known amount  Generate perturbed image and sample error  Learn multivariate regression for many such perterbuations

21. Active Appearance Model Search  Algorithm:  current estimate of model parameters:  normalized image sample at current estimate

22. Active Appearance Model Search  Slightly different modeling:  Error term:  Taylor expansion (with linear assumption)  Min (RMS sense) error:  Systematically perturb and estimate by numerical differentiation

23. Active Appearance Model Search (Results)