Kumar, Roger Sepiashvili, David Xie, Dan Professor Chen April 19, 1999 Progressive 3D Mesh Coding

Synthetic/Natural Hybrid Coding (SNHC) in MPEG-4 –A/V plane with 2D overlays, text, 3D objects, faces, etc. MPEG-4 System and Description Languages (MSDL) –MSDL will provide flexible run-time environment for invoking decompression tools, algorithms, etc. –MSDL will include class libraries for A/V object types along with their bitstream syntax. –Client will download A/V object types for a specific session from the server at start up. MSDL A/V Objects… –VRML –ActiveX Animation –Java Media 3D –etc. 3D Models - MPEG-4 Overview

MPEG-4 V2 will provide tools for coding 3-D Objects –Coding of generic 3-D Meshes –Level of Detail (LOD) Scalability Decoder can render simplified version of the mesh Useful for viewing distant objects & for less-powerful rendering engines. –Spatial Scalability Decoder can render the mesh at reduced spatial resolution Used in combination with LOD Scalability –Progressive Transmission to receive progressively better approximations to the model possible approaches are: –transmission of successive LOD approximations –progressive mesh coding (discussed later) 3D Models - MPEG-4 Overview

Texture Coding and Compression –Static textures can be compressed by JPEG –Moving textures can be compressed by MPEG-1 or H.263 Geometry Coding and Compression –There is no standard for geometry coding and compression. –Most often geometry is represented by a triangular 3D mesh. –Algorithms that quantize vertex positions and normals, and then apply Huffman or entropy encoding can achieve compression in the range of 15:1 to 65:1, depending on the nature of the model. –Mechanical models are easier to compress than anatomical. Need for Geometry Compression: –Each vertex represented by three floating pt. numbers. –If each vertex shared by six polygons, and max number of vertices per model is 2^20, then 76 bits/triangle needed. –Several hundred Kilobytes for average model (just geometry) Compression of 3D Models

Progressive Mesh (PM) addresses these issues: Progressive Transmission –show progressively better approximations to the model Smooth Visual Transition (geomorphs) –transit smoothly between different levels of approximation Selective Refinement –spatial refinement –level of detail refinement Mesh Optimization –constructing approximation meshes from non-optimal scanned meshes with smaller number of vertices Mesh Compression –mesh simplification algorithm allows very efficient coding Can Be Lossless (if no optimization was done) Why Progressive Mesh Coding ?

Original mesh M ^ is stored as a coarse mesh M 0 and n detail records that indicate how to refine M 0 into the original mesh M ^. Mesh Optimization –reduction of number of faces Mesh Simplification –construction of a coarse mesh M 0 Mesh Compression –storage space reduction Progressive Mesh (PM)

Find mesh M that accurately fits the model and has a small number of vertices. Goal is to optimize for rendering efficiency Often requires significant user intervention Done by minimizing the energy function E dist (M) = total squared distance of points of original model from the mesh –measures accuracy E rep (M) is proportional to the number of vertices in the mesh –measures conciseness E spring (M) is energy of a spring placed on each edge of a mesh –penalizes vertices that are too far away from adjacent vertices PM - Mesh Optimization

Original mesh M ^ is stored as a coarse mesh M 0 and n detail records that indicate how to refine M 0 into the original mesh M ^. Can simplify mesh using edge collapse, edge split, edge swap. Only edge collapse is needed. Edge collapse is fully invertible, so inverse transformation reconstructs the original mesh. PM - Mesh Simplification

Done by minimizing the energy function E dist (M), E spring (M) are same as before E scalar (M) measures accuracy of scalar attributes are properties of corners (i.e. [vertex,face] tuples) examples are: diffuse color, texture, etc. E disc (M) measures geometric accuracy of discontinuity curves preserving sharp edges is important E disc (M) is equal to the total squared distance of a set of points sampled from sharp edges (discontinuity curves) of M ^ to the discontinuity curve they belong. Algorithm: –mark all candidates for edge collapse in terms of priority, where priority of each transformation is estimated by energy cost  E. –starting with top at priority queue, perform edge collapse and recompute priorities of edges in neighborhood of this edge collapse. –Keep doing until desired number of faces is met. PM - Mesh Simplification

Progressive Meshes (Microsoft Research) – Java 3D API - 3D Geometry Compression – /forDevelopers/j3dguide/AppendixCompress.doc.html References All movies are made by Hugues Hoppe

Did you mention that "Java3D also does compression"? How? –Our first slide stated that the MPEG-4 System and Description Languages (MSDL) will allow 3D geometry to be compressed with different algorithms, one of which could be Java Media 3D (Java3D). MPEG-4 will also specify an built-in algorithm for geometry compression (most likely Progressive Mesh). If a better compression scheme is developed a few years later, MSDL would allow MPEG-4 to use this algorithm. –Java3D specifies an algorithm for geometry compression. This algorithm does not allow for progressive transmission of meshes, as the Progressive Mesh (PM) algorithm does. –In the Java3D algorithm, a 3D geometry is converted into a generalized triangular mesh. In Java3D, the vertex points are quantized, and then difference between two quantized vertex points is Huffman encoded. The vertex information is stored along with values for diffuse color and a normal vector. –Color information is also quantized and Huffman encoded, whereas Normal information is encoded using a more complicated scheme (mapping normals to points on the unit sphere, please see section B.8). –If no color or normal information is sent for a vertex, the color or normal information for the previously-sent vertex is used. Question #1

What is the speed for Mesh Optimization? –In our presentation, we stated that Mesh Optimization is quite slow. Below are exact numbers and examples. Question #2 Mesh Optimization: 3152 faces to 334 faces 17.0 minutes Mesh Optimization: 18,274 faces to 1348 faces 47.0 minutes Mesh Optimization: 8073 faces to 515 faces 44.5 minutes Mesh Optimization: 3832 faces to 432 faces 10.2 minutes Timings preformed by Hugues Hoppe on a DEC Workstation in