1. Pedro V. Sander Xianfeng Gu Steven J. Gortler Harvard University
Silhouette Clipping
Pedro V. Sander
Xianfeng Gu
Steven J. Gortler
Harvard University
Hugues H. Hoppe
John M. Snyder
Microsoft Research
Good evening
I’m Pedro Sander
Today I will talk about Silhouette Clipping
This is a joint project with Xianfeng Gu and Steven Gortler from Harvard University,
And Hugues Hoppe and John Snyder from Microsoft Research

2. Mesh simplification + texture/bump map original mesh coarse mesh
poor silhouette
Meshes can be very complex, causing the rendering system to be transform bound.
A common solution to this problem is to use a simplified mesh with fewer triangle faces.
However, mesh detail is lost during simplification.
This detail can be captured in the form of a texture map or bump map.
While this approach improves the rendering speed significantly, it doesn’t preserve the quality of the original model’s silhouette.
original mesh
coarse mesh

3. Silhouette clipping original mesh coarse mesh + texture/bump map
We introduce silhouette clipping, a method to efficiently clip the coarse geometry using the external silhouette of the original model.
original mesh
coarse mesh
+ texture/bump map
silhouette clipped

4. Silhouette clipping original mesh coarse mesh + texture/bump map
This silhouette clipped model has only 500 triangle faces, while the original has over 69,000 faces.
original mesh
coarse mesh
+ texture/bump map
silhouette clipped

5. Silhouette clipping demo
Now I will show a demo of the Stanford Bunny that highlights the advantage of using silhouette clipping.
Here you can see the model smooth shaded
<,>
Now we add texture maps
<m…> Now we toggle silhouette clipping on and off
<k>
Now we toggle between the silhouette clipped model and the original model

6. Overview [Soucy et al 96] [Cignoni et al 98] [Cohen et al 98]
coarse mesh
rendered
original mesh
normal map
Several schemes address the issue of texture-mapping a simplified mesh in order to preserve its appearance.
In most cases, the mesh is first simplified and then texture-mapped using attributes from the original mesh.
At runtime, the texture-mapped coarse mesh is rendered.
[Soucy et al 96]
[Cignoni et al 98]
[Cohen et al 98]

7. Overview coarse hull rendered original mesh normal map stencil mask
In silhouette clipping, we start with the original model.
During preprocess:
We first simplify the model creating a coarse hull.
The coarse hull is a mesh that contains the original mesh. This containment property is necessary for silhouette clipping.
We also create texture and/or normal maps for this coarse hull.
Then we create an edge hierarchy in order to perform efficient silhouette extraction at runtime.
At runtime, for a given view-point:
We extract the silhouette from the original model creating a stencil mask.
2D silhouette
edge hierarchy
Preprocess
Runtime

8. Overview coarse hull rendered silhouette clipped original mesh
normal map
Finally, we render the texture-mapped coarse mesh clipped subject to the stencil mask.
2D silhouette
edge hierarchy
Preprocess
Runtime

9. Major contributions Progressive hull Texture creation
Silhouette extraction
Stencil setting
Now I will highlight the major contributions of our work.

10. Major contributions Progressive hull Texture creation
Silhouette extraction
Stencil setting
To perform silhouette clipping, the coarse mesh must enclose the original one.
The Progressive Hull is a level-of-detail representation that achieves that goal.

11. Major contributions Progressive hull Texture creation
Silhouette extraction
Stencil setting
We introduce a new method for texture mapping the coarse mesh.
This method faithfully captures the detail of the original mesh.

12. Major contributions Progressive hull Texture creation
Silhouette extraction
Stencil setting
We introduce a scheme that allows efficient extraction of silhouette edges at runtime using a precomputed data structure.

13. Major contributions Progressive hull Texture creation
Silhouette extraction
Stencil setting
Finally, we introduce a stencil buffer algorithm to clip the coarse mesh subject to the extracted silhouette.

14. Given mesh, create coarse mesh that encloses it.
Coarse hull
The Progressive Hull allows us to create a coarse mesh that encloses the original model.
Original mesh
Coarse hull
Given mesh, create coarse mesh that encloses it.

15. Progressive hull Progressive mesh sequence
The progressive hull is equivalent to a progressive mesh sequence with the property that every mesh encloses all finer meshes in the sequence.
Although this method constructs a sequence of approximating meshes, we only use one coarse hull.
Progressive mesh sequence
Every mesh encloses all finer meshes.

16. Edge collapse Perform greedy sequence of collapses.
M i
M i-1
We perform a greedy sequence of edge collapses until we reach the desired number of faces.
During simplification, we use the edge collapse, which removes two faces from the mesh.
In order to ensure the new mesh encloses the previous mesh, the placement of the new vertex can be expressed as a linear programming problem.
Perform greedy sequence of collapses.
Ensure new mesh encloses old mesh: Linear programming problem

17. Linear programming (2D)
Lets see how this works in 2D.
The yellow edge is the edge to be collapsed.
Each edge constrains the unified vertex to lie in a half-space.
Thus, it can be anywhere inside the orange region.
Within this feasible region, we pick the point that minimizes mesh volume.
Which is this.

18. Linear programming (2D)

19. Linear programming (2D)

20. Linear programming (2D)

21. Linear programming (2D)

22. Linear programming (2D)

23. Linear programming (2D)
new vertex position

24. Progressive hull demo Here is a demo of creating a progressive hull
Coarse mesh
Back to fine mesh

25. Texture creation coarse hull rendered [Cignoni et al 98]
Another contribution of our work is a new scheme for texture mapping simplified meshes.
Like Cignoni et al and Soucy et al, we create a separate texture image for each face of the coarse mesh, and pack these triangles in a rectangular texture.
rendered
[Cignoni et al 98]
[Soucy et al 96]
original mesh
normal map

26. Texture creation coarse original vertex normals closest-point
So, we must map points on the coarse mesh with points on the fine mesh.
Like Cignoni et al, we only use the coarse and fine geometries to perform the parametrization, as opposed to using the entire progressive mesh sequence.
For each texel in the coarse mesh, Cignoni et al obtain the texel value by finding the closest point on the original mesh.
Note the discontinuity near concave regions <point>
We instead, find this texel value, by shooting a ray from the coarse mesh in the direction of the interpolated surface normal.
Even though normal-shooting is not guaranteed to be a one-to-one mapping, it has far fewer discontinuities.
closest-point
[Cignoni et al 98]
normal-shooting
Ours

27. Silhouette extraction
Now I will talk about our silhouette extraction algorithm.
Given the original mesh and the current viewpoint, it returns the edges in the silhouette
Given mesh and viewpoint, return edges on silhouette.

28. What is a silhouette edge?
Edge is on silhouette iff one adjacent triangle faces towards viewpoint:
What do we mean by a silhouette edge?
It is an edge s.t. one of its adjacent triangles is facing towards the viewpoint, and the other one is facing away from it.
not silhouette
silhouette

29. Related schemes
Brute force: O(edges) but O(edges on silhouette)  O(edges) !
[Markosian et al 97] probabilistic
[Gooch et al 99] assumes orthographic view
[Barequet et al 99] [Hertzmann & Zorin 00] dual-space formulation
[Kumar et al 96] [Johannsen & Carter 98] backface culling Ours: optimized hierarchy of anchored cones extracts 3,500 sil edges of 10,000 edges at 250 fps
One could extract the silhouette edges using a linear time brute-force algorithm.
This is very expensive, considering the number of silhouette edges is on the order of squareroot the number of edges
Markosian et al introduced a probabilistic method that doesn’t guarantee to extract all silhouette edges, but it extracts the most significant ones
Gooch et al introduce a hierarchical culling approach that assumed an orthographic projection
More recently, Barequet et al and Hertzmann & Zorin introduced schemes based on a dual space formulation
Our scheme is inspired by previous methods for backface culling by Kumar et al and Johannsen & Carter.
It constructs a tree using an anchored cone primitive with the objective of minimizing an expected extraction cost metric, therefore being very efficient.

30. Hierarchical culling Each node contains: cluster of mesh edges
spatial culling primitive
We construct a tree in order to perform hierarchical culling.
Each node in the tree represents a cluster of mesh edges.
Each node also has a spatial culling primitive.
The spatial culling primitive allows for a simple test that can quickly determine if, for a given viewpoint, all edges in the cluster are not in the silhouette.
TODO: little graph of culling entire subtree (and text)

31. Hierarchical culling Each node contains: cluster of mesh edges
spatial culling primitive
We construct a tree in order to perform hierarchical culling.
Each node in the tree represents a cluster of mesh edges.
Each node also has a spatial culling primitive.
The spatial culling primitive allows for a simple test that can quickly determine if, for a given viewpoint, all edges in the cluster are not in the silhouette.
<> If the test passes, than an entire sub-tree can be culled.

32. Spatial culling primitive
Given cluster of edges, check if there are no silhouette edges.
True if adjacent triangles are all front-facing or all back-facing.
Given a cluster of edges, there will be no silhouette edges,
if all faces adjacent to all edges in the cluster are all front-facing <> or all back-facing <>.
In other words, if the view-point lies within the green or the pink region, then there are no silhouette edges in the cluster.

33. Spatial culling primitive
Given cluster of edges, check if there are no silhouette edges.
True if adjacent triangles are all front-facing or all back-facing.

34. Spatial culling primitive
Given cluster of edges, check if there are no silhouette edges.
True if adjacent triangles are all front-facing or all back-facing.

35. Anchored cone primitive
front-facing
cone
Our approach is to approximate each region by an anchored cone.
So, if the viewpoint is inside one of the cones, then the cluster has no silhouette edges.
back-facing
cone

36. Anchored cone primitive
Calculate central axis Gilbert’s algorithm
Calculate cone angle
Calculate anchor positions Linear programming
To construct a cone of maximum angle,
we use Gilbert’s algorithm to compute the optimal normal.

37. Anchored cone primitive
Calculate central axis Gilbert’s algorithm
Calculate cone angle
Calculate anchor positions Linear programming
Then we calculate the maximum possible cone angle.

38. Anchored cone primitive
Calculate central axis Gilbert’s algorithm
Calculate cone angle
Calculate anchor positions Linear programming
Finally, we calculate the anchor positions using linear programming, minimizing in normal direction.

39. Hierarchy construction
Bottom-up greedy joining
two clusters
parent
merge
adopt
We use a bottom-up greedy approach for constructing the tree.
We allow these three operations to join any two root clusters in the forest.
Either parent, merge or adopt.

40. Hierarchy construction
Each root has probability of culling.
We compute an expected extraction cost for each root, and the forest.
Choose next join operation to maximally decrease expected extraction cost.
Our goal is the pick the tree with the minimum average extraction cost possible.
We use a metric that measures the expect extraction cost given a distribution of viewpoints.
Each node has a probability of culling.
So, we can compute an expected extraction cost for the forest.
We choose the next join operation in order to maximally decrease the extraction cost.

41. Silhouette extraction video
Here you can see the silhouette edges highlighted in red
Now we will freeze the silhouette and look at it from a different point of view
The edges in blue are the edges that had to be explicitly tested
Now we show the same sequence with the dragon model

42. Performance Model Bunny Dragon Parasaur Knot Holes3 Total edges
104,511
600,000
65,799
278,784
282,624
Sil edges
3,461
23,493
3,227
3,291
1,737
Tested edges
10,256
67,934
10,938
13,134
5,976
Time (ms)
4.1
28.2
4.3
7.9
3.3
This table highlights the performance of the silhouette extraction algorithm.
All tests were performed on a Pentium III 550Mhz machine.
The first row shows the number of edges in the model.

43. Performance Model Bunny Dragon Parasaur Knot Holes3 Total edges
104,511
600,000
65,799
278,784
282,624
Sil edges
3,461
23,493
3,227
3,291
1,737
Tested edges
10,256
67,934
10,938
13,134
5,976
Time (ms)
4.1
28.2
4.3
7.9
3.3
The second row shows the average number of silhouette edges from a distribution of viewpoints.

44. Performance Model Bunny Dragon Parasaur Knot Holes3 Total edges
104,511
600,000
65,799
278,784
282,624
Sil edges
3,461
23,493
3,227
3,291
1,737
Tested edges
10,256
67,934
10,938
13,134
5,976
Time (ms)
4.1
28.2
4.3
7.9
3.3
The third row shows the average number of edges that were not pruned by the search tree.

45. Performance Model Bunny Dragon Parasaur Knot Holes3 Total edges
104,511
600,000
65,799
278,784
282,624
Sil edges
3,461
23,493
3,227
3,291
1,737
Tested edges
10,256
67,934
10,938
13,134
5,976
Time (ms)
4.1
28.2
4.3
7.9
3.3
Finally, the fourth row shows the average total extraction time in milliseconds.

46. Performance Model Bunny Dragon Parasaur Knot Holes3 Total edges
104,511
600,000
65,799
278,784
282,624
Sil edges
3,461
23,493
3,227
3,291
1,737
Tested edges
10,256
67,934
10,938
13,134
5,976
Time (ms)
4.1
28.2
4.3
7.9
3.3
Note that, for instance on the Stanford bunny, about one tenth of the edges are tested, and one third of those are in the silhouette.
The total silhouette extraction time was, on average, 4.1 milliseconds.

47. Stencil setting
Now I will describe our stencil algorithm for clipping the coarse model using the silhouette of the original model.

48. Stencil setting algorithm
Algorithm inspired by concave polygon-filling algorithm.
Silhouette edges organized in hash table.
Each contour drawn as fan of triangles:
Triangle front-facing  stencil += 1
Triangle back-facing  stencil –= 1
The algorithm is inspired by the stencil algorithm for filling concave polygons.
We first organize the silhouette edges in a hash table.
Then we draw a fan of triangles for each silhouette contour.
For all front-facing triangles, we increment the stencil buffer.
And for all back-facing triangles, we decrement the stencil buffer.

49. Stencil setting algorithm
Here is how it works…
We pick any starting point, and then render triangles to the stencil buffer connecting that point to every edge in the silhouette.

50. Stencil setting algorithm
Increment

51. Stencil setting algorithm
Increment

52. Stencil setting algorithm
Increment

53. Stencil setting algorithm
Increment all those

54. Stencil setting algorithm

55. Stencil setting algorithm
decrement

56. Stencil setting algorithm
decrement

57. Stencil setting algorithm
Increment (note stencil buffer value becomes 2)

58. Stencil setting algorithm
increment

59. Stencil setting algorithm
decrement

60. Stencil setting algorithm
The values in the resulting stencil buffer represent the winding number of each pixel with respect to the silhouette curve.
We can then just render the coarse model subject to the stencil buffer.

61. Stencil setting video
This is an animation of the process of filling the stencil buffer
The red regions indicate negative stencil buffer values, while the green regions indicate positive ones.
The resulting stencil mask is used to clip the coarse model.

62. Silhouette antialiasing
Transfer stencil to alpha buffer: stencil = 0  alpha = stencil = 1  alpha = 1.0
Render silhouette edges in alpha buffer as antialiased line segments.
Render coarse hull subject to alpha buffer.
For little additional cost, we can also antialias the model.
After stencil setting, we transfer the stencil buffer to the alpha buffer.
Then we render the silhouette edges to the alpha buffer as antialiased lines.
This gives us fractional alpha values near the silhouette.
Then we render the coarse hull subject to the alpha buffer.

63. Silhouette antialiasing video
Here we compare an aliased silhouette on the left with an antialiased silhouette on the right.

64. Timing analysis All times in milliseconds Model Bunny Dragon Parasaur
Knot
Holes3
Faces (original)
69,674
400,000
43,866
185,856
188,416
Rendering time
34.7
204.7
20.6
81.1
90.3
Faces (coarse)
500
4,000
1,020
928
4.8
5.2
4.9
4.4
SC Rendering time
7.8
50.3
6.9
10.3
5.5
Speedup factor
4.1
3.0
7.9
16.4
This table shows the overall timings for silhouette clipping.
The first two rows highlight the number of faces on the original model and how long it takes to render it.
All times in milliseconds

65. Timing analysis All times in milliseconds Model Bunny Dragon Parasaur
Knot
Holes3
Faces (original)
69,674
400,000
43,866
185,856
188,416
Rendering time
34.7
204.7
20.6
81.1
90.3
Faces (coarse)
500
4,000
1,020
928
4.8
5.2
4.9
4.4
SC Rendering time
7.8
50.3
6.9
10.3
5.5
Speedup factor
4.1
3.0
7.9
16.4
The next two rows show the number of faces on the coarse model, and how long it takes to render it.
All times in milliseconds

66. Timing analysis All times in milliseconds Model Bunny Dragon Parasaur
Knot
Holes3
Faces (original)
69,674
400,000
43,866
185,856
188,416
Rendering time
34.7
204.7
20.6
81.1
90.3
Faces (coarse)
500
4,000
1,020
928
4.8
5.2
4.9
4.4
SC Rendering time
7.8
50.3
6.9
10.3
5.5
Speedup factor
4.1
3.0
7.9
16.4
The next row shows how long it takes to render the silhouette clipped coarse model.
All times in milliseconds

67. Timing analysis All times in milliseconds Model Bunny Dragon Parasaur
Knot
Holes3
Faces (original)
69,674
400,000
43,866
185,856
188,416
Rendering time
34.7
204.7
20.6
81.1
90.3
Faces (coarse)
500
4,000
1,020
928
4.8
5.2
4.9
4.4
SC Rendering time
7.8
50.3
6.9
10.3
5.5
Speedup factor
4.1
3.0
7.9
16.4
Finally, the last row shows the speedup factors obtained comparing silhouette clipping with rendering the original model.
All times in milliseconds

68. Summary Silhouette clipping & antialiasing Fast silhouette extraction
Progressive hull
Normal-shooting parametrization
In this paper we introduced silhouette clipping, a method for efficiently rendering a coarse model clipped to the original model’s silhouette.
We introduced a fast silhouette extraction algorithm.
We introduced the progressive hull data structure, which is a progressive mesh sequence with the containment property.
And we introduced the normal shooting parametrization for texture-mapping a simplified model.
We hope that the last 3 contributions will also be useful in other areas of Computer Graphics.

69. Future work Handle internal silhouettes Visual hull simplification
Silhouette level-of-detail
A significant limitation of this approach is the fact that it doesn’t handle internal silhouettes. We have thought of a possible solution, but we haven’t yet tried it.
When performing silhouette extraction, we don’t have to use the original model. We can simplify the model as long as we preserve its visual hull (its opaque appearance). Initial experiments show that we can reduce the complexity of a complex mesh by ¼ or more without affecting its visual hull.
Another interesting approach is to do silhouette extraction with LOD. If the model is very far away a coarser silhouette can be returned, while if the model is near by, a finer one is used.