2. Texture Synthesis on [Arbitrary Manifold] Surfaces Presented by: Sam Z. Glassenberg* * Several slides borrowed from Wei/Levoy presentation

3. Topics Investigated Thus Far 2D Texture Mapping (CS318) 3D Textures (Hypertexture) 2D Texture Synthesis

4. + Synthesize a texture on a surface by coloring mesh vertices Input TextureInput MeshResult Today’s Goal

5. Vertex Painting Advantages –Speed No additional transformation required –Simplicity –Uniformly distributed across geometry Disadvantages –Texture resolution dependent on mesh complexity

6. 2 SIGGRAPH Papers: 1.Texture Synthesis on Surfaces, by Greg TurkTexture Synthesis on Surfaces 2.Texture Synthesis over Arbitrary Manifold Surfaces, by Li-Yi Wei and Marc LevoyTexture Synthesis over Arbitrary Manifold Surfaces

8. Desirable Properties Share advantages of 2D algorithm –Quality –Efficient –General –Easy to use Minimum distortion Minimum discontinuity

9. Differences between Images and Meshes image mesh Synthesis Order Pixels/Vertices Local Orientation (Vector field) ? ? scanline u v grid Neighborhood normal ? ? tangent bitangent

10. Wei/Levoy Solution image mesh Synthesis Order Pixels/Vertices Local Orientation (Vector field) ? ? scanline u v grid Neighborhood normal ? ? tangent bitangent Wei and Levoy mesh re-tiling [Turk’92] user-specified relaxation random flattening/resampling

11. Turk Solution Turk repulsion [Turk’91] User specified/ interpolated sweeping Surface marching image mesh Synthesis Order Pixels/Vertices Local Orientation (Vector field) ? ? scanline u v grid Neighborhood normal ? ? tangent bitangent

12. Both Papers Extend Fast Texture Synthesis using Tree-structured Vector Quantization Specifically by generalizing their definition of “search neighborhoods” to apply to meshes

13. Texture Synthesis by Neighborhood Search noise Input pyramid noise Output pyramid Search Copy

14. Surface Texture Synthesis by Neighborhood Search (Wei/Levoy) Input pyramid Output pyramid Search

15. Process 1. Build image/mesh pyramids 2. Assign texture orientation/Computation order 3. Generate texture

16. What aspects of image pyramids must we maintain in mesh pyramids? Uniform density Power-of-two complexity differences between levels

17. Image & Mesh Pyramids Mesh Retiling [Turk’92]

18. 2 Turk-ish Methods for Mesh Retiling Turk ’92, used by Wei/Levoy –Uniformly distributes mesh vertices –Requires “shooting normals” to move between levels Turk ’91, used by Turk –Uniformly distributes mesh vertices –Maintains parent/child relationship between levels

19. Retiling Density 24576 vertices73728 vertices

20. Turk ‘91 Create a mesh hierarchy in which mesh M k = (V k, T k ) is defined by its Vertices and Triangles –For the lowest mesh in the hierarchy, place n points on the surface –Use repulsion to distribute points easily –Add 3n points to make the next level –Repeat Connect points by projecting nearby points onto a tangent plane Perform Delaunay triangulation to reduce triangles

21. Now, we need to determine orientation… In Turk’s method, we use this orientation to determine the computation order (Surface Sweeping) In Wei/Levoy’s method, orientation is needed to “flatten” the neighborhoods.

22. Texture Orientation Generate a coordinate frame Three orthogonal axes – s (texture right) – t (texture up) – n (surface normal)

23. Texture Orientation Methods for orienting textures –user-specified (Turk) –random (Wei/Levoy) –smooth or symmetric (Wei/Levoy)

24. Texture Orientation (User-specified)

25. 4-way symmetric texture 4-way symmetric vector field Texture Orientation (Symmetry)

26. Texture Orientation random2-way symmetry4-way symmetry

27. Texture Orientation (Relaxation) Minimize an error function

28. Results (Wei/Levoy): Random Orientation

29. Results (Wei/Levoy): Other Orientations RandomUser-specified Relaxation 2-way symmetry 4-way symmetry

30. Texture Synthesis on Surfaces: Wei/Levoy Style

31. Synthesis

32. Synthesis : 2 Lowest Levels

33. Synthesis : Lowest Level Random copy

34. Synthesis Pass1 : Extrapolation Search Copy Shooting normal

35. Mesh Neighborhood Resample Compare Resampled Grid 2D Patch 3D Patch Flatten (Maillot’93)

36. Neighborhood Flattening Project the triangles adjacent to p onto p’s local texture coordinate system Add triangles one-at-a-time until neighborhood template is covered

37. Neighborhood Comparison compare ?

38. Synthesis Pass 2 : Full Neighborhood Search Copy

39. Multiresolution Synthesis

40. Texture Synthesis on Surfaces: According to Turk

41. Turk’s Surface Sweeping Select an anchor vertex A Assign s(v) = sweep distance to vertex v along the vector field from A (for all v) Consensus orientation b/w 2 vertices v and w: O vw = (O(v) + O(w))/2 Calculate a new s(v) as a weighted average that its neighboring values dictate it should have. Visit vertices in order by sweep distance

42. Turk’s Texture Synthesis (Pseudocode) I = Input texture N(v) = Neighborhood around v M(a,b) = Neighborhood around (a,b) D(M,N) = Match Value (sum of squared differences)

43. Turk’s Mesh Neighborhoods r = average distance between mesh vertices O(v) = Surface tangent vector P(v) = O(v) rotated 90 o about surface normal Together, O and P make a coordinate frame ! u v normal P O Now, we can traverse the surface by point repelling

44. Point Repelling Use color interpolation to determine color at current surface point Move r in the direction of O or P When an edge is reached, fold the path over the next polygon

45. Results smooth4-way symmetryrandom

46. Results Surface displacement smooth2-way symmetry

47. Results

48. Summary of Differences Turk’s approachOur approach Vector field Traversal order Neighborhood Mesh hierarchy smoothrandom, symmetric sweepingrandom surface marchingflattening/resampling explicit parent/childshooting normal