Texture Synthesis on Surfaces Paper by Greg Turk Presentation by Jon Super

Texture Synthesis on Surfaces +

Goals of texture placement Take a 2D texture and wrap it onto a surface Avoid noticeable seams between patches Avoid noticeable seams between patches Minimize stretching and distortion Minimize stretching and distortion Produce a natural-looking result Produce a natural-looking result

The next best thing Lapped Textures – Praun et al. – SIGGRAPH00 Uses many overlapped texture patches Aligns patches with underlying vector field Generally minimizes distortion

The next best thing Lapped texture’s shortcomings Low-frequency components Low-frequency components Boundary mismatch Boundary mismatch Direction field singularities Direction field singularities

Texture synthesis to the rescue!

Process Create a mesh hierarchy over the model Create a mesh hierarchy over the model Set up a vector field over the mesh Set up a vector field over the mesh Calculate sweep distance for each vertex Calculate sweep distance for each vertex Assign colors based on neighbors Assign colors based on neighbors

Process Create a mesh hierarchy over the model Create a mesh hierarchy over the model Set up a vector field over the mesh Set up a vector field over the mesh Calculate sweep distance for each vertex Calculate sweep distance for each vertex Assign colors based on neighbors Assign colors based on neighbors

Mesh Hierarchy Equivalent of a Gaussian pyramid to a surface Multiple layers of different resolutions Multiple layers of different resolutions Each layer has 4 times the number of vertices of the layer above it Each layer has 4 times the number of vertices of the layer above it Each layer contains every vertex of the layer above it Each layer contains every vertex of the layer above it

Point placement n points randomly placed on the surface n points randomly placed on the surface Repulsion between the points spaces them evenly Repulsion between the points spaces them evenly These points are now fixed in place, representing level m of an m-level mesh These points are now fixed in place, representing level m of an m-level mesh Repeated for each layer, adding 3 times more points each time (3n, 12n, etc) Repeated for each layer, adding 3 times more points each time (3n, 12n, etc)

Connectivity Connectivity between vertices of each layer Connectivity between vertices of each layer Delaunay triangulation applied Delaunay triangulation applied Mesh is connected in a web of triangles Mesh is connected in a web of triangles

Mesh hierarchy operations Interpolation Interpolation Low-pass filtering Low-pass filtering Downsampling Downsampling Upsampling Upsampling

Process Create a mesh hierarchy over the model Create a mesh hierarchy over the model Set up a vector field over the mesh Set up a vector field over the mesh Calculate sweep distance for each vertex Calculate sweep distance for each vertex Assign colors based on neighbors Assign colors based on neighbors

Vector field Provides order to the placement of the texture on the model Provides order to the placement of the texture on the model Each mesh vertex is given a vector Each mesh vertex is given a vector User initiates the process by defining a few key vectors (~12) User initiates the process by defining a few key vectors (~12)

Vector field creation All vertices not user-defined set to 0 All vertices not user-defined set to 0 User-defined vectors are downsampled to the coarsest mesh User-defined vectors are downsampled to the coarsest mesh Low-pass filtering to fill out top mesh Low-pass filtering to fill out top mesh Upsample for each layer to fill to bottom Upsample for each layer to fill to bottom

Process Create a mesh hierarchy over the model Create a mesh hierarchy over the model Set up a vector field over the mesh Set up a vector field over the mesh Calculate sweep distance for each vertex Calculate sweep distance for each vertex Assign colors based on neighbors Assign colors based on neighbors

Surface sweeping Uses the directional flow of the vector field to assign ordering to the vertices Uses the directional flow of the vector field to assign ordering to the vertices Ordering will allow a sort of raster scan over the 3D model Ordering will allow a sort of raster scan over the 3D model The color in the image The color in the image cycles as the sweep cycles as the sweep distance increases distance increases

Sweep distance calculation Select an anchor point to base the distance Select an anchor point to base the distance Each vertex takes the value that is the weighted sum of what its neighbors think Each vertex takes the value that is the weighted sum of what its neighbors think Distance along the flow direction added to the neighbor’s sweep distance Distance along the flow direction added to the neighbor’s sweep distance Sweep distances will propagate out from the anchor until every vertex has one Sweep distances will propagate out from the anchor until every vertex has one

Process Create a mesh hierarchy over the model Create a mesh hierarchy over the model Set up a vector field over the mesh Set up a vector field over the mesh Calculate sweep distance for each vertex Calculate sweep distance for each vertex Assign colors based on neighbors Assign colors based on neighbors

Pixel Neighborhoods Similar technique to 2D texture synthesis Similar technique to 2D texture synthesis Examine nearby colors and look for a similar pattern in the texture image Examine nearby colors and look for a similar pattern in the texture image Different kinds of pixel neighborhoods Different kinds of pixel neighborhoods

Mesh coordinate space Pixel neighborhoods don’t translate directly to the unorganized vertices in a mesh Pixel neighborhoods don’t translate directly to the unorganized vertices in a mesh Vertex vector is down Vertex vector is down Rotate 90 degrees CCW for right Rotate 90 degrees CCW for right

Multi-level synthesis Start at top, work down Start at top, work down Initialize the vertices of mesh layer k with color values from Gaussian pyramid layer k Initialize the vertices of mesh layer k with color values from Gaussian pyramid layer k Make a pass with the half square Make a pass with the half square Top layer should have a crude representation of the texture Top layer should have a crude representation of the texture

Multi-level synthesis Extrapolate and refine each lower level Extrapolate and refine each lower level Extrapolate initializes the colors using a large full square neighborhood of the level above Extrapolate initializes the colors using a large full square neighborhood of the level above Refine improves on that by combining a large full square on the current level with a small full square on the level above Refine improves on that by combining a large full square on the current level with a small full square on the level above

Algorithm

Results