Facial Motion Cloning Using Global Shape Deformation Marco Fratarcangeli and Marco Schaerf University of Rome “La Sapienza”

University of Rome "La Sapienza" - Department of Computer and Systems Science Motivation Reduce artistic skill and time required for animation by re-using existing animation Solution: Facial Motion Cloning SourceTarget 3Target 2Target 1

University of Rome "La Sapienza" - Department of Computer and Systems Science Overview Motivation Related Work Our Approach –Input & Output –Shape fitting –Cloning process Results Conclusions

University of Rome "La Sapienza" - Department of Computer and Systems Science Related Work Expression Cloning [Noh et Neumann 2001]

University of Rome "La Sapienza" - Department of Computer and Systems Science Related Work Facial Motion Cloning [Pandzic 2003]

University of Rome "La Sapienza" - Department of Computer and Systems Science Related Work Difference from Noh: our output target faces are able to perform generic face animation; Difference from Pandzic: cloning time

University of Rome "La Sapienza" - Department of Computer and Systems Science Input & Output Manually picked feature points on the input meshes; Feature points are compliant with the MPEG-4 specification; (Facial Definition Points)

University of Rome "La Sapienza" - Department of Computer and Systems Science Input & Output 84 key movements defined by MPEG-4 (Facial Animation Parameters) … … Output: corresponding key positions for the target face

University of Rome "La Sapienza" - Department of Computer and Systems Science Source Shape Fitting Find the morphing function f(P) fitting the source into the target (Radial Basis Functions [Fang 96]) Volume Morphing

University of Rome "La Sapienza" - Department of Computer and Systems Science Radial Basis Functions (RBF) if we find a suitable set of correspondence points between source and target, then, we can calculate the interpolation function f(p) fitting the source into the target. [Fang 96] That is, given the known data u i = f (p i ), we can compute u k = f (p k ).

University of Rome "La Sapienza" - Department of Computer and Systems Science Iterative Fitting Enriching the correspondence set  precise fitting

University of Rome "La Sapienza" - Department of Computer and Systems Science Iterative Fitting – Visual Example Enriching the correspondence set  precise fitting

University of Rome "La Sapienza" - Department of Computer and Systems Science Iterative Fitting After the initial rough fitting… 1. Project target vertices towards source surface 2. Insert the target vertices having biggest distance into the correspondences set 3. Recompute the interpolation function f(P) and perform morphing 4. Is the source face fitted into the target? NOYES Map & clone…

University of Rome "La Sapienza" - Department of Computer and Systems Science Source Shape Fitting Find the morphing function f(P) fitting the source into the target (Radial Basis Functions [Fang 96]) Volume Morphing Map the target into the deformed source

University of Rome "La Sapienza" - Department of Computer and Systems Science Cloning Process Use f(P) to deform all the source key positions and map the displacements to the target faces.

University of Rome "La Sapienza" - Department of Computer and Systems Science

Results - Performance IterationsTotal Time (s) beta  data 88.8 data  beta 31.7 joakim  data data  joakim 42.1 beta  kevin kevin  beta 514.0

University of Rome "La Sapienza" - Department of Computer and Systems Science Results – Error Assessment source  source Beta0.028 % Data0.280 % Joakim0.084 % kevin0.141 % Average of the difference between the initial coordinates and the final coordinates

University of Rome "La Sapienza" - Department of Computer and Systems Science Conclusions PROs: –Pre-existing facial animation becomes reusable; –Fast cloning process: in seconds we have a brand new talking head (1-20 secs on a AMD 2,14 GHz); –Animation through linear interpolation of the key positions leads real-time performance; –MPEG-4 parametrization permits very low bit-rate transmission [Model-based coding - Forchheimer 83]; CONs: –Target models can not have higher resolution than source models; –Projection method (Target  Source) is not perfect...

University of Rome "La Sapienza" - Department of Computer and Systems Science The End Further material soon here:

University of Rome "La Sapienza" - Department of Computer and Systems Science

MPEG-4 FBA based animation MPEG-4 Facial and Body Animation (FBA) standard encodes the basic actions of a virtual face through Facial Animation Parameters (FAPs) Morph Targets are the key positions corresponding to the FAPs

University of Rome "La Sapienza" - Department of Computer and Systems Science MPEG-4 FBA based animation Linear interpolation between morph targets leads to a wide range of facial expressions Goal: synthesize automatically the whole set of morph targets for a static target face