1. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 1 Converting 3D Facial Animation with Gouraud shaded SVG A method to extract 3D animated files and to convert it in SVG allowing Gouraud shading Michel Gaudry, Thomas Di Giacomo, Nadia Magnenat-Thalmann

2. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 2 Outline Introduction –Context –Purpose Related work –3D on mobile devices –Conversion of 3D to other media 3D to SVG converter –Context and requirements –Process –Structure –2D data extraction –Generation of SVG SVG Optimization Gouraud shading Experiments and evaluation –The pyramid –The talking head Conclusion

3. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 3 Introduction Context and motivation –Developement of wearable devices –User's growing interest for web graphics and 3D –Hardware limitations –Increasing availability of SVG for web and mobiles devices –Possibility to convert 3D animated faces to SVG

4. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 4 Introduction Purpose –Extend the authoring of SVG Enable new SVG authoring by creating scenes in a 3D environment and converting it in SVG Extend SVG rendering with motion captured sequence or highly realistic 3D rendering Extends availlability of 3D graphics with other medias –Generate animated and Gouraud shaded SVG to simulate 3D scenes Extend 3D possibilities on mobile devices Method to convert generic 3D to SVG Specific for large 3D scenes and focus on face animation and deformation Simulation of light shading with Gouraud

5. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 5 Related work 3D on mobile devices –Adaptations Hybrid approaches VRML Simplification of 3D –Conversion of 3D to other media To image and video –Current growing technics Java 2 Micro Edition and JSR-184 OpenGL ES X3D 3D with SVG –Advantages –Conversion from other languages

6. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 6 3D to SVG Converter Purpose –Convert 3D animations –Animation data from external file –Allow animation optimization and Gouraud shading Main structure –Open a 3D animation –Identification and extraction of 2D data –Generation of SVG Optimized animation Gouraud shaded image

7. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 7 Context and requirements Formats –VRML chosen as input –FAP for face deformation Animation inputs –User actions –Predefined Not additional software or scripts Preserve –Pixel spatial information –Pixel color information Gouraud shading –Non animated objects Real time conversion –Ease authoring of SVG animation with 3D

8. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 8 Structure 3 main modules –Extraction –Animation –Generation User interface 2 optional generation modules 3D rendering with Open GL GUI manager User menu Generation Optimization Gouraud VRML FAP Animation & record Extraction SVG

9. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 9 Process 3D to SVG –Extraction from VRML file –Get dynamic data from user or file input 2D data –2D mesh initialized with a list of path and colors per triangles –Conversion of triangles animation with gradient, position and visibility SVG generator –Pre-generation of SVG file –Generation

10. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 10 2D data extraction Algorithm –Initialization of the 2D "copy": topology, index, colors static data: camera parameters –Update and Extraction 3D vertices dynamic positions Culling of 3D faces Projection of 3D to viewpoint plane Storage of computed 2D coordinates

11. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 11 2D data extraction Gouraud shading –Method –In SVG Projection Visibility

12. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 12 Generation of SVG From application cache Animation...

13. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 13 Generation of SVG To represent triangles – Coordinates Visibility Filling –Single color –Gradients <animate attributeName="d" begin="abs" dur="ass" fill="freeze" vallues=" M x00 y00 L x10 y10 L x20 y20 z;... M x0n y0n L x1n y1n L x2n y2n z "/>

14. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 14 Generation of SVG Optimization –Problem Generated files can be big –Module Can be integrated in an application or can be used independently to optimize any SVG animation file Processed before or after generation Complementary to gzip compression –Methods Reduce length of static sequences Remove invisibles sequences

15. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 15 Generation of SVG Optimization methods –Reduce static sequences Remove repeated coordinates when the polygon is not moving –Remove invisible sequences Remove the tags and cut main animation in parts –Others possibilities Use relative coordinates Simplify animation path

16. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 16 Generation of SVG Gouraud shading –Purpose Improve realism, from flash-shaded triangles to interpolated colors vertex –Method Adaptation of the method presented in « Interactive 3D Viewer Written in SVG » (SVG Open 2003) In large scale For a 3D converted mesh –Gouraud with SVG Use of gradients and filters Superposition of triangles –Gradients 3 gradients per triangle, one from each vertex Blending of the gradients with attribute Filters – Gradient area Gradient’s path Alpha value : 100% Alpha value : 0% Triangle area

17. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 17 Generation of SVG Gouraud shading –Structure of a triangle T1 T4 T1 T3 T2 g path Sub-triangle feColorMatrix path Element canvas

18. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 18 File structure Header –SVG –Viewport Definitions –Filters –Triangles –Gradients <feColorMatrix type="matrix" values=" 1 0 0 0 0 0 1 0 0 0 0 0 1 0 0 1 1 1 1 0 0 0 0 0 1 "/>

19. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 19 File structure Construction of the triangles –Subtriangle –Merge of the 3 gradient's triangles –Adjust opacity

20. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 20 Experiments and evaluation Files used for test –The pyramid A simple rotating polygon with user interaction –The face Simple face with 484 triangles Mesh deformation using MPEG-4 3D facial animation as input

21. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 21 The pyramid Results –Visual Reproduction of 3D scene Faces visibility –Size Acceptable for playback with standard SVG viewers on light devices

22. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 22 The talking head Results –Animation Reproduction of the original animation –Gouraud shading Results Drawbacks

23. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 23 Results table PyramidFace Face with Gouraud VRML size in kb231 Number of polygons8484 Duration in s12.088.28- Raw size in kb5295684610 Optimized size in kb914798- Optimized gzipped size in kb 27139663

24. MIRALab Where Research means Creativity SVG Open 2005 University of Geneva www.miralab.ch 24 Conclusion Conversion –Improvements Optimization –Interpolation of animation's path Gouraud shading