MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 1 Converting 3D Facial Animation with Gouraud shaded SVG A method to extract 3D animated files and to convert it in SVG allowing Gouraud shading Michel Gaudry, Thomas Di Giacomo, Nadia Magnenat-Thalmann

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 2 Outline Introduction –Context –Purpose Related work –3D on mobile devices –Conversion of 3D to other media 3D to SVG converter –Context and requirements –Process –Structure –2D data extraction –Generation of SVG SVG Optimization Gouraud shading Experiments and evaluation –The pyramid –The talking head Conclusion

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 3 Introduction Context and motivation –Developement of wearable devices –User's growing interest for web graphics and 3D –Hardware limitations –Increasing availability of SVG for web and mobiles devices –Possibility to convert 3D animated faces to SVG

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 4 Introduction Purpose –Extend the authoring of SVG Enable new SVG authoring by creating scenes in a 3D environment and converting it in SVG Extend SVG rendering with motion captured sequence or highly realistic 3D rendering Extends availlability of 3D graphics with other medias –Generate animated and Gouraud shaded SVG to simulate 3D scenes Extend 3D possibilities on mobile devices Method to convert generic 3D to SVG Specific for large 3D scenes and focus on face animation and deformation Simulation of light shading with Gouraud

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 5 Related work 3D on mobile devices –Adaptations Hybrid approaches VRML Simplification of 3D –Conversion of 3D to other media To image and video –Current growing technics Java 2 Micro Edition and JSR-184 OpenGL ES X3D 3D with SVG –Advantages –Conversion from other languages

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 6 3D to SVG Converter Purpose –Convert 3D animations –Animation data from external file –Allow animation optimization and Gouraud shading Main structure –Open a 3D animation –Identification and extraction of 2D data –Generation of SVG Optimized animation Gouraud shaded image

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 7 Context and requirements Formats –VRML chosen as input –FAP for face deformation Animation inputs –User actions –Predefined Not additional software or scripts Preserve –Pixel spatial information –Pixel color information Gouraud shading –Non animated objects Real time conversion –Ease authoring of SVG animation with 3D

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 8 Structure 3 main modules –Extraction –Animation –Generation User interface 2 optional generation modules 3D rendering with Open GL GUI manager User menu Generation Optimization Gouraud VRML FAP Animation & record Extraction SVG

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 9 Process 3D to SVG –Extraction from VRML file –Get dynamic data from user or file input 2D data –2D mesh initialized with a list of path and colors per triangles –Conversion of triangles animation with gradient, position and visibility SVG generator –Pre-generation of SVG file –Generation

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva D data extraction Algorithm –Initialization of the 2D "copy": topology, index, colors static data: camera parameters –Update and Extraction 3D vertices dynamic positions Culling of 3D faces Projection of 3D to viewpoint plane Storage of computed 2D coordinates

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva D data extraction Gouraud shading –Method –In SVG Projection Visibility

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 12 Generation of SVG From application cache Animation...

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 13 Generation of SVG To represent triangles – Coordinates Visibility Filling –Single color –Gradients <animate attributeName="d" begin="abs" dur="ass" fill="freeze" vallues=" M x00 y00 L x10 y10 L x20 y20 z;... M x0n y0n L x1n y1n L x2n y2n z "/>

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 14 Generation of SVG Optimization –Problem Generated files can be big –Module Can be integrated in an application or can be used independently to optimize any SVG animation file Processed before or after generation Complementary to gzip compression –Methods Reduce length of static sequences Remove invisibles sequences

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 15 Generation of SVG Optimization methods –Reduce static sequences Remove repeated coordinates when the polygon is not moving –Remove invisible sequences Remove the tags and cut main animation in parts –Others possibilities Use relative coordinates Simplify animation path

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 16 Generation of SVG Gouraud shading –Purpose Improve realism, from flash-shaded triangles to interpolated colors vertex –Method Adaptation of the method presented in « Interactive 3D Viewer Written in SVG » (SVG Open 2003) In large scale For a 3D converted mesh –Gouraud with SVG Use of gradients and filters Superposition of triangles –Gradients 3 gradients per triangle, one from each vertex Blending of the gradients with attribute Filters – Gradient area Gradient’s path Alpha value : 100% Alpha value : 0% Triangle area

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 17 Generation of SVG Gouraud shading –Structure of a triangle T1 T4 T1 T3 T2 g path Sub-triangle feColorMatrix path Element canvas

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 18 File structure Header –SVG –Viewport Definitions –Filters –Triangles –Gradients <feColorMatrix type="matrix" values=" "/>

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 19 File structure Construction of the triangles –Subtriangle –Merge of the 3 gradient's triangles –Adjust opacity

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 20 Experiments and evaluation Files used for test –The pyramid A simple rotating polygon with user interaction –The face Simple face with 484 triangles Mesh deformation using MPEG-4 3D facial animation as input

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 21 The pyramid Results –Visual Reproduction of 3D scene Faces visibility –Size Acceptable for playback with standard SVG viewers on light devices

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 22 The talking head Results –Animation Reproduction of the original animation –Gouraud shading Results Drawbacks

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 23 Results table PyramidFace Face with Gouraud VRML size in kb231 Number of polygons8484 Duration in s Raw size in kb Optimized size in kb Optimized gzipped size in kb

MIRALab Where Research means Creativity SVG Open 2005 University of Geneva 24 Conclusion Conversion –Improvements Optimization –Interpolation of animation's path Gouraud shading