Active Appearance Models Dhruv Batra ECE CMU

Active Appearance Models 1.T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", in Proc. European Conference on Computer Vision 1998 (H.Burkhardt & B. Neumann Ed.s). Vol. 2, pp , Springer, T.F.Cootes, G.J. Edwards and C.J.Taylor. "Active Appearance Models", IEEE PAMI, Vol.23, No.6, pp , G.J. Edwards, A. Lanitis, C.J. Taylor, T. F. Cootes. “Statistical Models of Face Images Improving Specificity”, BMVC (1996)

Essence of the Idea  “Interpretation through synthesis”  Form a model of the object/image (Learnt from the training dataset) I. Matthews and S. Baker, "Active Appearance Models Revisited," International Journal of Computer Vision, Vol. 60, No. 2, November, 2004, pp

Essence of the Idea (cont.)  Explain a new example in terms of the model parameters

So what’s a model Model “Shape” “texture”

Active Shape Models training set

Texture Models warp to mean shape

Random Aside  Shape Vector provides alignment = 43 Alexei (Alyosha) Efros, (15-862): Computational Photography,

Random Aside  Alignment is the key 1. Warp to mean shape 2. Average pixels Alexei (Alyosha) Efros, (15-862): Computational Photography,

Random Aside  Enhancing Gender more same original androgynous more opposite D. Rowland, D. Perrett. “Manipulating Facial Appearance through Shape and Color”, IEEE Computer Graphics and Applications, Vol. 15, No. 5: September 1995, pp

Random Aside (can’t escape structure!) Alexei (Alyosha) Efros, (15-862): Computational Photography, Antonio Torralba & Aude Oliva (2002) Averages: Hundreds of images containing a person are averaged to reveal regularities in the intensity patterns across all the images.

Random Aside (can’t escape structure!) Tomasz Malisiewicz,

Random Aside (can’t escape structure!) “100 Special Moments” by Jason Salavon Jason Salavon,

Random Aside (can’t escape structure!) “Every Playboy Centerfold, The Decades (normalized)” by Jason Salavon 1960s1970s1980s Jason Salavon,

Back (sadly) to Texture Models raster scan Normalizations

PCA Galore Reduce Dimensions of shape vector Reduce Dimension of “texture” vector They are still correlated; repeat..

Object/Image to Parameters modeling ~80

Playing with the Parameters First two modes of shape variationFirst two modes of gray-level variation First four modes of appearance variation

Active Appearance Model Search  Given: Full training model set, new image to be interpreted, “reasonable” starting approximation  Goal: Find model with least approximation error  High Dimensional Search: Curse of the dimensions strikes again

Active Appearance Model Search  Trick: Each optimization is a similar problem, can be learnt  Assumption: Linearity  Perturb model parameters with known amount  Generate perturbed image and sample error  Learn multivariate regression for many such perterbuations

Active Appearance Model Search  Algorithm:  current estimate of model parameters:  normalized image sample at current estimate

Active Appearance Model Search  Slightly different modeling:  Error term:  Taylor expansion (with linear assumption)  Min (RMS sense) error:  Systematically perturb and estimate by numerical differentiation

Active Appearance Model Search (Results)