Shape Compression using Spherical Geometry Images Surface parametrizations 2001/05/18 Shape Compression using Spherical Geometry Images Hugues Hoppe, Microsoft Research Emil Praun, University of Utah IMA Workshop on Computer Graphics

Mesh representation irregular semi-regular completely regular

What if images were represented with irregular meshes? Drawbacks: storage of connectivity no random lookup rendering compositing filtering compression demo

Simple 2D grid Advantages: implicit connectivity 2D lookup raster-scan alpha blending DSP JPEG 2000

Representations for media Audio: uniform 1D grid Images: uniform 2D grid Video: uniform 3D grid Geometry: irregular mesh historical artifact?

Surface parametrizations 2001/05/18 Geometry image 2D grid sampling 3D geometry Our approach is to represent an arbitrary surface using a completely regular grid of samples on a square domain, which we call a geometry image. <click> Here is an example. As you can see, it looks just like an ordinary image. It has 257-by-257 pixels. The only difference is that it has 12 bits/channel instead of the usual 8. Why do we call this a geometry image? Because its RGB colors encode XYZ positions, and in fact <click> entirely describe this 3D shape. As we will see later in the talk, the regularity of this representation can permit image processing techniques, such as compression, to be applied to geometry directly. geometry image 257 x 257; 12 bits/channel IMA Workshop on Computer Graphics

Surface parametrizations 2001/05/18 Geometry image Here is an outline of the basic steps involved in creating a geometry image. First, we slice open the mesh along an appropriate set of cut paths, creating a surface with the topology of a disk. <click> Second, we parametrize this cut surface onto the square domain of the geometry image. To illustrate this parametrization, we’ll show a demo. <demo> Here we see a morph between the cut bunny and its embedding in the square. You can see how the cut in blue is being morphed into the boundary of the square. Because this is so much fun, I’ll show the morph one more time. IMA Workshop on Computer Graphics

Surface parametrizations 2001/05/18 Geometry image Here is an outline of the basic steps involved in creating a geometry image. First, we slice open the mesh along an appropriate set of cut paths, creating a surface with the topology of a disk. <click> Second, we parametrize this cut surface onto the square domain of the geometry image. To illustrate this parametrization, we’ll show a demo. <demo> Here we see a morph between the cut bunny and its embedding in the square. You can see how the cut in blue is being morphed into the boundary of the square. Because this is so much fun, I’ll show the morph one more time. render [r,g,b] = [x,y,z] IMA Workshop on Computer Graphics

Advantages for hardware rendering Surface parametrizations 2001/05/18 Advantages for hardware rendering Regular sampling  no vertex indices. Sequential traversal of source data Unified parametrization  no texture coordinates. IMA Workshop on Computer Graphics

Main questions cut? parametrize?

Construction approaches General cut Spherical Multi-chart [Gu et al. SIGGRAPH 2002] [Praun & Hoppe. SIGGRAPH 2003] [Sander et al. SGP 2003] arbitrary surface genus-zero surface  cut symmetries >1 chart  zippering

Construction approaches General cut [Gu et al. SIGGRAPH 2002] arbitrary surface genus 6

Construction approaches General cut Spherical Multi-chart [Gu et al. SIGGRAPH 2002] [Praun & Hoppe. SIGGRAPH 2003] [Sander et al. SGP 2003] arbitrary surface genus-zero surface  cut symmetries >1 chart  zippering 400x160 piecewise regular

Construction approaches General cut Spherical Multi-chart [Gu et al. SIGGRAPH 2002] [Praun & Hoppe. SIGGRAPH 2003] [Sander et al. SGP 2003] arbitrary surface genus-zero surface  cut symmetries >1 chart  zippering

Spherical parameterization and remeshing Surface parametrizations 2001/05/18 Spherical parameterization and remeshing [Praun, Hoppe 2003] IMA Workshop on Computer Graphics

Spherical parameterization and remeshing Surface parametrizations 2001/05/18 Spherical parameterization and remeshing [Praun, Hoppe 2003] IMA Workshop on Computer Graphics

Spherical geometry images Surface parametrizations 2001/05/18 Spherical geometry images IMA Workshop on Computer Graphics

Steps mesh M sphere S domain D image I demo

Spherical parametrization Surface parametrizations 2001/05/18 Spherical parametrization mesh M sphere S [Kent et al. 1992] [Haker et al. 2000] [Alexa 2002] [Grimm 2002] [Sheffer et al. 2003] [Gotsman et al. 2003] Two challenges: robustness good sampling  coarse-to-fine  stretch metric To parameterize a model onto the sphere we want to assign to each vertex a coordinate on the sphere. Edges will correspond to great circle arcs, and mesh triangles to spherical triangles. There are two principal goals for such a map: robustness and good sampling rate. Robustness implies that the resulting map is guaranteed to be one-to-one, meaning that no triangles overlap on the sphere. By good sampling rate we mean that interesting features of the model receive enough real estate on the sphere in order to be accurately sampled. We achieve these goals by using a coarse-to-fine strategy, and minimizing map distortion using Sander et al.’s stretch metric. [Hormann et al. 1999] [Sander et al. 2001] [Sander et al. 2002] IMA Workshop on Computer Graphics

Coarse-to-fine algorithm Surface parametrizations 2001/05/18 Coarse-to-fine algorithm Convert to progressive mesh We start by converting the model to a progressive mesh sequence, successively simplifying it using half-edge collapses, until we reach a tetrahedron. We embed the tetrahedron on the sphere and then we refine the model back to its original resolution. For each vertex split operation we assign a spherical coordinate to the newly introduced vertex, and also optimize the location of its neighbors in order to minimize map distortion. Parametrize coarse-to-fine (maintain embedding & minimize stretch) IMA Workshop on Computer Graphics

Traditional conformal metric Surface parametrizations 2001/05/18 Traditional conformal metric Preserve angles but “area compression” Bad for sampling using regular grids As I mentioned, to evaluate map distortion we use Sander’s stretch metric. The most popular metric for parametrization is conformality. This metric is not well suited for regular resampling since it suffers from area compression. In this example, the horse’s legs and head receive a very small area in the map, and therefore the remesh does a poor job at sampling those features. IMA Workshop on Computer Graphics

Surface parametrizations 2001/05/18 [Sander et al. 2001] Stretch metric [Sander et al. 2002] Penalizes undersampling Better samples the surface By contrast, the stretch metric penalizes undersampling, by trying to map large distances on the sphere (and therefore many samples) to short distances on the mesh, so it doesn’t miss features. As you can see, this metric does a better job of capturing the horse geometry. IMA Workshop on Computer Graphics

Applications of spherical remeshing Surface parametrizations 2001/05/18 Applications of spherical remeshing Level-of-detail control Morphing Geometry amplification Shape compression IMA Workshop on Computer Graphics

Level-of-detail control

Morphing Align meshes on the sphere. Interpolate the resulting geometry images.

Geometry amplification Surface parametrizations 2001/05/18 Geometry amplification [Losasso et al. SGP 2003] “smooth geometry images” simulation CPU GPU 33x33 65x65 129x129 floating-point geometry image 257x257 + 257x257 scalar displacements demo IMA Workshop on Computer Graphics

Shape compression (Genus-zero shapes) Spherical image topology Infinite 2D tiling Wavelets on regular 2D grid

Spherical image topology

Spherical image topology Surface parametrizations 2001/05/18 Spherical image topology Not only do samples along the boundary correspond to each other, but there is a simple way to extend neighborhood queries outside the geometry image. For example, if we are at the blue dot on the left of the image and we want the neighbor located a few samples above it, that coresponds on the uncut octahedron to walking across the orange edge that was cut into the adjacent face. In the domain, we leave the image on the left going up and enter it on the right going down. IMA Workshop on Computer Graphics

Spherical image topology Surface parametrizations 2001/05/18 Spherical image topology Another way to look at this is that we keep going up into another copy of the geometry image, obtained by rotating the original about the midpoint of the top edge. IMA Workshop on Computer Graphics

Surface parametrizations 2001/05/18 Infinite 2D tiling We can apply this rotation process to all the edge midpoints to obtain an infinite lattice of geometry images. This lattice is continuous everywhere and derivativative continuous everywhere except at the repeated instances of the four edge midpoints. Derivative continuity is very important for many processing applications, and we make use of it in our compression results. IMA Workshop on Computer Graphics

Wavelets on regular 2D grid spherical wavelets image wavelets [Schröder & Sweldens 1995] [Davis 1995] [Antonini et al 1992]

Test models

Compression results

Compression results

Compression results

Compression results

Compression results

Summary Geometry image Applications Simplicity of 2D grid Rendering LOD Morphing Geometry amplification Shape compression

Future work Visual error metrics [Touma & Gotsman 1998] [Sorkine et al 2003] Attenuation of rippling artifacts Surface boundaries Animated meshes “geometry videos” [Briceño et al 2003]