1. Queensland University of Technology CRICOS No. 000213J INB382/INN382 Real-Time Rendering Techniques Lecture 11: Non- Photorealistic Rendering, Particle Systems and Translucency Ross Brown

2. CRICOS No. 000213J a university for the world real R Lecture Contents Non-Photorealistic Rendering Particle Systems Translucency

3. CRICOS No. 000213J a university for the world real R Nothing in particular to say "Electricity is actually made up of extremely tiny particles called electrons, that you cannot see with the naked eye unless you have been drinking” - Dave Barry

4. CRICOS No. 000213J a university for the world real R Lecture Context This week will be the last set of actual rendering techniques presented Thus it is a grab bag of particles, non- photorealism, translucency etc. To tie up some loose ends technique wise Thus the slight decorrelation between the sections of the lecture

5. CRICOS No. 000213J a university for the world real R Non-photo Realistic Rendering Learned so far the process of simulating the transport of light energy in a room Intention of creating scenes of immense realism Photo Realism - just like a photograph of the scene Some requirements in rendering are biased towards the creation of a Mood A parallel of the development of modern art – moving from realism to impressionism and beyond Picasso - man playing a guitar?!?

6. CRICOS No. 000213J a university for the world real R Non-Photorealistic Rendering Some forms are not as extreme Such as the simulation of painterly techniques Such as pen and ink and impressionistic brush strokes We will look at Technical and Non- technical applications

7. CRICOS No. 000213J a university for the world real R Technical Applications of NPR Technical drawings often use particular colour palettes with gradients that highlight the shape of the object – from work by Gooch 1998[2]

8. CRICOS No. 000213J a university for the world real R Technical Drawing General Principles Edge lines – the set containing surface boundaries, silhouettes and discontinuities are drawn with black curves Matte objects with intensities far from black or white with warmth or coolness of color indicative of surface normal – single light source and white highlights No shadowing Very anisotropic shading of metal objects

9. CRICOS No. 000213J a university for the world real R Technical Drawing General Principles Edge highlighting comes from perceptual studies indicating assistance in shape recognition – due to edge detectors in the Human Visual System Need colours that are distinct from black and white to add information Shape information more important than lighting information in technical scenarios

10. CRICOS No. 000213J a university for the world real R Tone-Based Shading In colored medium, artists often use both hue and luminance –Adding white to colors gives tints –Adding black to colors gives shades –Adding grey give tones –Complementary colors are used to create tones as well

11. CRICOS No. 000213J a university for the world real R Colour Temperature Warm – red, orange, yellow Cool – blue, violet, green Temperate – red-violets and yellow-greens Depth cue comes from the perception that cool colors recede while warm colors advance

12. CRICOS No. 000213J a university for the world real R Mixing Warm and Cool Tones Can add to classic shading algorithm by using the cosine term to blend between two RGB colours – k cool and k warm Where l is the light vector perpendicular to the gaze direction Light is up and to the right for technical scenario

13. CRICOS No. 000213J a university for the world real R Transitions Use blue to yellow tones to insure a cool to warm color transition regardless of diffuse color of the object –k blue = (0,0,b), b  [0,1] –k yellow = (y,y,0), y  [0,1] –k cool = k blue + αk d –k warm = k yellow + βk d Where k d is the diffuse lighting strength

14. CRICOS No. 000213J a university for the world real R Transitions The values for b and y will determine the strength of the overall temperature shift The values of α and β will determine the prominence of the object color and the strength of the luminance shift

15. CRICOS No. 000213J a university for the world real R Shading Comparison The model is appropriate for a range of object colors Note that with the new shading method objects retain their “color name” so colors can still be used to differentiate objects like countries on a political map, but the intensities used do not interfere with the clear perception of black edge lines and white highlights Traditional shading (top row) and the new tone-based shading (bottom row)

16. CRICOS No. 000213J a university for the world real R Skeleton Example

17. CRICOS No. 000213J a university for the world real R Tank Example in War Thunder Game Shading shows internal structure in a cluttered environment. They should use Gooch!

18. CRICOS No. 000213J a university for the world real R Metal objects Illustrators use a different technique to communicate whether an object is metal Represent a metallic surface by alternating dark and light bands Artistic representation of real effects that can be seen on milled metal parts, such as those found on cars or appliances Milling creates what is known as “anisotropic reflection” Lines are streaked in the direction of the axis of minimum curvature, parallel to the milling axis

19. CRICOS No. 000213J a university for the world real R Metal Objects To simulate a milled object, map a set of twenty stripes of varying intensity along the parametric axis of maximum curvature The stripes are random intensities between 0.0 and 0.5 with the stripe closest to the light source direction overwritten with white Between the stripe centres, the colours are linearly interpolated

20. CRICOS No. 000213J a university for the world real R Metal Objects Metal-shaded object is more obviously metal than the Phong-shaded image Cool-warm hue metal-shaded object is not quite as convincing as the achromatic image But it is more visually consistent with the cool- warm matte shaded model so it is useful when both metal and matte objects are shown together

21. CRICOS No. 000213J a university for the world real R Metal Tone Mapping

22. CRICOS No. 000213J a university for the world real R Gooch Coloured Metal Examples

23. CRICOS No. 000213J a university for the world real R Cg Gooch Vertex Shader

24. CRICOS No. 000213J a university for the world real R Cg Gooch Pixel Shader

25. CRICOS No. 000213J a university for the world real R Unity Demonstration

26. CRICOS No. 000213J a university for the world real R Toon Shading Artistic technique consists of solid heavily quantised colour, surrounded by black edges A case of amplification by simplification (McCloud 1994) Again, it is an example of using a rendering technique to evoke a particular emotional feel to the scene being rendered

27. CRICOS No. 000213J a university for the world real R Cartoon Shading Used because kids buy games The rise of adult animation This approach is used in the game Cel Damage© – EA Games Also used in films and music videos – A Scanner Darkly And every second INB380 Unity project

28. CRICOS No. 000213J a university for the world real R Cartoon Shading Can be performed by quantising textures applied to geometry from truecolor to lower bit size samples Or, can quantise the values generated by normal lighting techniques to another palette

29. CRICOS No. 000213J a university for the world real R Cartoon Shading Cartoon shading uses a palette of two or three tones For both specular and diffuse components Edges are often rendered in black Can be implemented easily in a vertex and fragment shaders

30. CRICOS No. 000213J a university for the world real R Toon Tone Quantisation Quantisation is performed by using a step function (1D Texture) to convert the smooth (ish) float to one of two values Is a form of texture correspondence function Performed for the specular, diffuse and edge factors 1.0 0.2 NLNL 1.0 1D Texture 0.0 1.0

31. CRICOS No. 000213J a university for the world real R Silhouette Rendering The shader uses a dot product to determine the slope of the geometry – edge = max(dot(N, V), 0); The quantised black value then shades edges on edge to the viewer Does not work for a cube edge. Why?

32. CRICOS No. 000213J a university for the world real R Cg Vertex Shader

33. CRICOS No. 000213J a university for the world real R Cg Pixel Shader – Note 1D Texture Samples

34. CRICOS No. 000213J a university for the world real R Unity Cartoon Shading Demonstration

35. CRICOS No. 000213J a university for the world real R Other Silhouette Rendering Techniques Move the back faces forward in z and then render in black Move the back faces out along vertex normals and then render in black – Unity technique Perform image processing in a fragment shader (later) (Moller and Haines 2003)‏ front face back face Render translate d back face in black black edge passes z buffer Camera render front faces expand back face s along normals and render

36. CRICOS No. 000213J a university for the world real R Vertex Extrusion Shader

37. CRICOS No. 000213J a university for the world real R Unity Edge Silhouette Demonstration

38. CRICOS No. 000213J a university for the world real R Edge Possibilities – how in Unity? http://blenderartists.org/forum/showthread?t=25458

39. CRICOS No. 000213J a university for the world real R Particle Systems Developed in 1983 by W.T. Reeves [1] Suitable for natural phenomenon –clouds, fire, water spray, rain, etc Large population of individual particles –all with geometric extents of very small or zero size –millions – with GPU methods Number of particles –number of particles at time t –average number + random number * allowable variance at that time

40. CRICOS No. 000213J a university for the world real R Particle Systems Behaviour is governed by a general script –random behaviour is added to give individual characteristics –eg, for sparks, script could be parabolic giving parabolic change in shape, colour, etc –size of particle cloud can remain constant, increase or decrease over time –overall extent can remain constant, explode or implode over time

41. CRICOS No. 000213J a university for the world real R Particle Systems - Basic Assumptions Modern particle systems can collide with each other using GPU support Only cast shadows on the rest of the environment –not on each other – probably not such an issue now… Don’t normally reflect light using lighting model –can be now modelled as emitting point light sources –can be shaded using GPU shaders

42. CRICOS No. 000213J a university for the world real R Particle Systems - Basic Assumptions React to linear forces –don’t react to angular forces – torque doesn’t apply More assumptions depending on the application Often use textured quads as the base primitive Powerful GPUs offer greater possibilities for mesh particles

43. CRICOS No. 000213J a university for the world real R Initial State/Attribute Requirements Shape Lifetime value Initial values for –position –velocity and direction –size –colour –transparency NOTE - most of these can be randomised

44. CRICOS No. 000213J a university for the world real R Steps Involved in Particle Generation at Each Time Frame 1.New particles are generated and injected into the current system 2.Each new particle is assigned its individual attributes 3.Any particles that have exceeded their lifetime are extinguished 4.The current particles are moved according their scripts 5.The current particles are rendered

45. CRICOS No. 000213J a university for the world real R Particle Attribute Control Lifetime attribute is initially set (randomised) and decremented at each frame until zero indicates termination Particle animation –colour, shape, size, etc, are functions of global time and change according a script which is then randomised –acceleration is based on active forces standard particle forces also forces which act between particle pairs velocity and displacement updated accordingly

46. CRICOS No. 000213J a university for the world real R What can be Modelled? Fire Fireworks/Sparks Plants Smoke Rain Anything else?

47. CRICOS No. 000213J a university for the world real R Fire Emits from source shape Weak gravity effect - plasma Swirling component (where from?!?) added to velocity Flame components change colour and die out Leave blended trail

48. CRICOS No. 000213J a university for the world real R Fireworks/Sparks Emits from point source Source moves with fireworks Velocity like projectile - particle Leaves a shorter tail http://www.gamasutra.com/features/20000623/vanderburg_01.htm

49. CRICOS No. 000213J a university for the world real R Plants [2] Emits from small source area Velocity like projectile - particle Render temporal path as polyline Add flowers etc as decorations

50. CRICOS No. 000213J a university for the world real R Smoke Shaped emitter Use textured quads as particles, blended over each other Similar to flame – gaseous physics Particles become less visible over time http://www.exhibitresearch.com/kevin/ioe/smoke.pdf

51. CRICOS No. 000213J a university for the world real R Rain Large area emitter Elongated quads with raindrop texture Collision with ground plane Wind effects on velocity of particles http://www.gamasutra.com/features/20000623/vanderburg_pfv.htm

52. CRICOS No. 000213J a university for the world real R Rendering Primitives Points – for sparks, fireworks etc. Colour and visibility attenuates over time Curves – for plants, render entire temporal existence of particle as stems Textured Quads – for smoke, flames, blood plant leaves etc. Pixel Shaders – for final appearance modifications http://www.2ld.de/gdc2004/ http://www.gamasutra.com/features/20000623/vanderburg_02.htm

53. CRICOS No. 000213J a university for the world real R Particle Data Structure Each particle –position (3D vector) –velocity (3D vector) –combined forces (3D vector) –mass (float) Particle system –array of particles –number of particles –current time w.r.t. life span of particle system class particle { Vector position; Vector velocity; Vector forces; int lifeTime; float mass; }; class particles { Particle *pList; int numParticles; Time time; };

54. CRICOS No. 000213J a university for the world real R Simple Particle Implementation Tips Based on Angel 1 page 468-469 Ideal Newton points –x(t) = x 0 + v x t + f x t 2 /2m –y(t) = y 0 + v y t + f y t 2 /2m –z(t) = z 0 + v z t + f z t 2 /2m Where: –(x 0,y 0,z 0 ) is initial position –v is initial velocity –f is force acting on particle – gravity in all dirs. –t is time –m is mass 1. Angel, E., Interactive Computer Graphics, A top-down approach with OpenGL (Fifth Edition)

55. CRICOS No. 000213J a university for the world real R Particle Shaders Transformation completely performed within the vertex shader Pass the vertices and time to the vertex shader for animation Pixel shader in our case can be a simple sample of a texture

56. CRICOS No. 000213J a university for the world real R Particle Shaders Pixel shader can easily be modified to create animated textures for more sophisticated particle systems – noise textures etc. Can animate the appearance over time, as well as the location of the particle

57. CRICOS No. 000213J a university for the world real R Unity3D Interface Provides Nearly All Needed Functionality http://docs.unity3d.com/Manual/ParticleSystems.html

58. CRICOS No. 000213J a university for the world real R Translucent Objects So far we have considered the rendering of many objects with a number of surface properties Our lighting model so far has dealt with the interaction of light with objects in three ways: –Reflection – diffuse, specular –Absorption – giving different colours –Transmission – to mimic transparency Now we deal theoretically with the fourth factor, Scattering

59. CRICOS No. 000213J a university for the world real R Subsurface Scattering [6] Translucent materials are frequently encountered in the natural world Examples include snow, plants, milk, cheese, meat, human skin, cloth, marble, and jade Degree of translucency may vary, but the characteristic appearance is distinctly smooth and soft as a result of light scattering inside the objects This process is known as subsurface scattering http://graphics.ucsd.edu/papers/layered /

60. CRICOS No. 000213J a university for the world real R Subsurface Scattering Traditionally subsurface scattering has been approximated as Lambertian diffuse reflection – which is only part of the answer There have also been proposed methods for simulating subsurface scattering by tracing photons through the material via photon mapping A BRDF only accounts for scattering at a single point, and it cannot be used to simulate light transport through the medium of an object - refer to Lecture notes on BRDFs Subsurface scattering requires treating the material as a participating medium with a surface That is you have to model the internal structure of the object, not just the surface

61. CRICOS No. 000213J a university for the world real R BRDF vs BSSRDF BRDF is an approximation of the Bidirectional Surface Scattering Distribution Function (BSSRDF) for which it is assumed that light enters and leaves at the same point (i.e., x o = x i ) Given a BSSRDF, the outgoing radiance is computed by integrating the incident radiance over incoming directions and area, A BSSRDFBRDF

62. CRICOS No. 000213J a university for the world real R Bidirectional Surface Scattering Distribution Function The BSSRDF, S, relates the outgoing radiance L o, at the point x o in direction ω o, to the incident flux, Ф at the point x i from direction ω i :

63. CRICOS No. 000213J a university for the world real R Full BSSRDF Complete BSSRDF model is a sum of the diffusion approximation S d and a single scattering term S (1) Possibly a phase function - without a phase function the scattering can be modelled as isotropic which is tends to be in most cases Model accounts for light transport between different locations on the surface, and it simulates both the directional component (due to single scattering) as well as the diffuse component (due to multiple scattering) Fresnel term is computed to account for reflection on the medium boundaries – see previous lecture notes

64. CRICOS No. 000213J a university for the world real R Diffuse Component – Multiple Scattering Diffusion approximation is based on the observation that the light distribution in highly scattering media tends to become isotropic Each scattering event blurs the light distribution, and as a result the light distribution tends toward uniformity as the number of scattering events increases – more collisions, the more uniform Dipole method consists of positioning two point sources near the surface One point source, the positive real light source, is located at the distance z r beneath the surface A negative virtual light source, is located above the surface at a distance z v

65. CRICOS No. 000213J a university for the world real R Directional Component – Single Scattering Single scattering occurs only when the refracted incoming and outgoing rays intersect Is computed as an integral over path length s along the refracted outgoing ray

66. CRICOS No. 000213J a university for the world real R Marble Examples BRDFBSSRDFMonte Carlo Simulation

67. CRICOS No. 000213J a university for the world real R BSSRDF Components Multiple Scatter S d Single Scatter S (1) Fresnel

68. CRICOS No. 000213J a university for the world real R Milk Example BRDFSkimFull Cream

69. CRICOS No. 000213J a university for the world real R Skin Example BRDF - hardBSSRDF - soft

70. CRICOS No. 000213J a university for the world real R GPU Gems 3 Approach [5] http://http.developer.nvidia.com/GPUGems3/gpugems3_ch14.html https://www.youtube.com/watch?v=MqkICU4kKHY

71. CRICOS No. 000213J a university for the world real R First Model the Skin Kelemen and Szirmay- Kalos BRDF with Fresnel for top oily layer Cannot just use dipole approach detailed before due to multiple layers Ergo, NO single scattering modelling needed

72. CRICOS No. 000213J a university for the world real R Light Diffusion Modelling “Fricken Laser Beam” diffusion profile through membrane – note red is strongly spread Three layers approximated by sum of six 2D Gaussian blurs for each (r,g,b) Remember glow from last week (read the notes)?

73. CRICOS No. 000213J a university for the world real R Overall Algorithm Render shadow maps Render (u, v) stretch correction map – to deal with texture distortions on surface Render diffuse irradiance into off-screen texture – normal rendering N.L For (6 x rgb) Gaussian(s) used in diffusion profile approximation: –Perform a separable blur pass in U (temporary buffer). –Perform a separable blur pass in V (keep all of these for final pass). Render mesh in 3D: –Access each Gaussian convolution texture and combine linearly. –Add specular BRDF for each light source – oily surface

74. CRICOS No. 000213J a university for the world real R In Conclusion In this subject we have covered nearly every possible visual appearance phenomena that can be modelled From basic methods of modelling appearance we have progressed to quite visually accurate models of the interaction of light with the material of an object We have reached the point of creating, in the most part, real-time systems that fool our consciousness into believing it is a photograph of a real scene Next week we wrap up with rendering efficiency issues for scene graphs and the future of computer graphics

75. CRICOS No. 000213J a university for the world real R References 1.Amy Gooch, Bruce Gooch, Peter Shirly, Elaine Cohen. A Non-Photorealistic Lighting Model For Automatic Technical Illustration, Proceedings of ACM Siggraph, pp. 447-452, 1998 2.John D. Owens, David Luebke, Naga Govindaraju, Mark Harris, Jens Kr¨uger, Aaron E. Lefohn, and Timothy J. Purcell. A survey of general-purpose computation on graphics hardware. In Eurographics 2005, State of the Art Reports, pages 21–51. Eurographics Association, August, 2005. 3.Reeves, W. Particle Systems—a Technique for Modelling a Class of Fuzzy Objects ACM Transactions on Graphics (TOG) Volume 2 Issue 2 1983 4.Henrik Wann Jensen, Stephen R. Marschner, Marc Levoy and Pat Hanrahan: "A Practical Model for Subsurface Light Transport". Proceedings of SIGGRAPH'2001 5.Eugene d'Eon, David Luebke, Advanced Techniques for Realistic Real- Time Skin Rendering, GPU Gems 3, http://http.developer.nvidia.com/GPUGems3/gpugems3_ch14.html, Accessed 19/05/2012 http://http.developer.nvidia.com/GPUGems3/gpugems3_ch14.html