Realtime NPR Toon and Pencil Shading Joel Jorgensen May 4, 2010

Toon Shading Fairly easy to go from Phong shading to Toon shading coloring Just quantize the color based on some thresholds of the light intensity

Outlines Silhouette edges are tricky Image volume-based method, 2 passes First, render normals (compressed into RGB values) and depth (as alpha) into texture

Outlines (cont) Then render scene normally, and at the end overlay with rectangle that takes up full screen and is textured with the normal/depth texture from first pass Use special shader when drawing rectangle to detect discontinuities between normals and depths (Also use pencil texture instead of black when switching to pencil outline)

Pencil Texture 3D texture, increasing in darkness along depth Draw a bunch of slightly curved (with cos) lines by subtracting small, slightly varying amounts from each pixel in the line The more overlapping lines, the darker it gets Randomly scatter starting points around image to get decent coverage Each image is copy of image above it with more lines drawn on, gives continuity as light shifts Also wrap around horizontally and vertically so adjacent textures are continuous

Direction of Curvature Pencil lines follow direction of min curvature Our brains are good at using them as shape cues Mathematically challenging to calculate well; beyond my abilities Simple approximation: test neighboring points’ distance from tangent plane – Farthest point from tangent plane lies in direction of max curvature – Closest point to tangent plane lies in direction of min curvature Project onto tangent plane to get curvature at that vertex

Layering/Blending Humans aren’t perfect – Lines aren’t all parallel – Overlap a bit to create more visual interest Model is too “perfect” – An approximation with flat faces, but it makes those faces really flat – If we just put one texture on each face, it would be obviously faceted Blend multiple lines in different directions to hide the faceted nature of the model

Blending/Layering (cont) For each face, draw texture 3 times, rotated in each of the directions of minimum curvature of its corresponding vertices Pass directions of min curvature to vertex shader along with texture coordinates For each of the three directions of curvature – Project direction of min curvature onto screen – Determine angle of rotation – Rotate texture coordinates by that amount – Pass rotated texture coordinates to fragment shader

Fragment Shader Calculates intensity as in Phong model, dot product of light vector and surface normal Looks up 3 texture values – Uses the rotated texture coordinates from vertex shader for s, t – Uses intensity as depth coordinate Blends them together and uses that as the color for the fragment

Billboarding Will o’ the Wisp – Wandering spirit – Light source – Particle system (sparkles) Sparkles are textured squares Rotated to face the camera each frame Uses a method similar to the camera’s lookAt method (with some tweaks because they’re not the camera, so some things are kind of backwards)

But wait, there’s more!

Colored Pencils! Blend the texture with the color of the material Tricky, since just adding makes things lighter, and just multiplying would result in a totally colored object where it should have been white (since the “empty” pencil texture layer is pure white) Have to subtract both the pencil texture and the diffuse material color from 1, then multiply the result, then subtract that result from 1 again to get back to the right colors – Actually subtracted from the sepia tone to give it a more “papery” look, but same idea

Difficulties Math is hard! – Tried to wrap my head around some of the techniques for approximating the principal directions of curvature for several hours. – Determined that the amount of time I would need to actually get something like that working would far exceed the amount available for the project. – Switched to crude approximation

Problems Sliding/swimming textures – Unable to figure out why this is happening. Probably something to do with the projection and rotation code, but I haven’t been able to determine what exactly is causing it – Might be able to precompute these instead of trying to do them on the fly

Problems (cont) Triangles very obvious in areas where they’re close together and narrow (e.g. bottoms of the bowls) – Probably caused by the texture being scaled down in the mipmaps and pressed together so they kind of blur into a uniform grey color – Might be able to fix it by explicitly generating mipmaps that try to scale things more cleverly, similar to the TAMs described in the book, rather than using hardware mipmap generation

Problems (cont) Sparkles from the Will o’ the Wisp slide behind the outline because it’s decaled on top of the whole scene – Could fix by rendering the Will o’ the Wisp the very last

Problems (cont) Sparkles sometimes chop bits out of each other (transparent background of sparkles that are in front of other sparkles but rendered before those sparkles will be grey even if there’s another sparkle behind it) – Could be fixed by doing depth sort and then rendering from back to front – Not very obvious, though, so probably not worth the effort at this point; would be a bigger deal if the sparkles were larger