NonPhotoRealism and an Evaluation of Rendering Lecture 4

Painterly Rendering: Scene Reconstruction lScene Data Capture (from ‘What Dreams May Come’):  Scan in film stock (analog to digital). Soon to be outmoded.  Used orange balls to help with registration between frames.  Scene also filmed with Lidar (Laser Radar) scanning.  Computer vision used to reconstruct a 3D model. Before VFX Post-Production After VFX Post-Production

Painterly Rendering: Process lCombines object and image space algorithms. lObject Space:  Particles on the object surface ensure consistency between animation frames. lImage Space:  Reference images of the scene are used to determine the orientation, colour and size brush attributes.  Reference images of brush strokes are altered according to the brush attributes and composited onto the image.

Painterly Rendering: Algorithm Create particles to represent geometry FOR each frame of animation Create reference pictures using geometry, surface attributes and lighting Sort particles by distance from viewpoint FOR each particle (starting with furthest from viewpoint) [1] Transform particles to screen space [2] Determine brush stroke attributes from particles and/or reference pictures [3] Randomly perturb attributes based on user selected attributes [4] Composite brush stroke into paint buffer ENDFOR

Painterly Rendering: Algorithm

Painterly Rendering: Conclusions lJittering:  Some random variation in brush attributes is needed to ensure a painterly appearance.  Problem: this damages temporal coherence.  Solution: associate a random seed with each particle. lPainterly rendering is an intensive process which requires considerable user intervention (parameter tweaking). lMay need several passes to composite different effects: shadows, highlights, etc. lTip of the Iceberg. There has been much research in this area recently, producing a variety of techniques in both painterly and line-art rendering.

Painterly Rendering: Examples lAltering brush stroke and painting parameters to create different effects: pastel, pointilist, marker-pen, woodcut.

Other NonPhotoRealistic Styles lPen-and-Ink Illustration: Difference Image Algorithm: 1.Draw a hatching stroke in the darkest part of a greyscale image. 2.Subtract a blurred version of the stroke from the image and repeat. lCartoon Illustration:  Place Graftal (procedural) textures along silhouettes.  Evoke the style of Dr. Suess.  Effectively indicates complexity, e.g. grass, fur, trees.

NonPhotoRealism: References lSurvey Article:  Landsdown, J. and Schofield, S. “Expressive Rendering: A Review of Nonphotorealistic techniques”, IEEE Computer Graphics and Applications, Vol. 15, No. 3, 1994, pp lPainterly Rendering:  Meier, B. “Painterly Rendering for Animation”, SIGGRAPH ’96, pp  Haeberly, P. “Paint by Numbers: Abstract image representations”, SIGGRAPH ’90, pp lLine-Art Rendering (pen-and-ink lines and cross-hatching effects which emulate sketching):  Wikenbach, G. and Salesin, D. “Computer Generated Pen-and-Ink Illustration”, SIGGRAPH ’94, pp lCartoon Illustration:  Kowalski, M. et al. “Art-based Rendering of Fur, Grass and Trees”, SIGGRAPH ’99, pp

Rendering in Computer Graphics lPrincipal Rendering Techniques: 1.Polygon Scan Conversion 2.Ray Tracing (Basic, Distributed, Recursive) lAdjunct Rendering Techniques: 1.Radiosity 2.Non-PhotoRealistic Rendering 3.Surface Detail lEvaluation: 1.Realism 2.Speed 3.Memory Consumption l‘Real World’ choices

Realism: Direct Illumination lLight may reflect off many surfaces on its way from the light to the camera. lStandard ray tracing and polygon scan conversion can handle a single diffuse or specular bounce. lDirect Diffuse Reflection  Assumes that the object is a perfect Lambertian (diffuse) reflector. lDirect Specular Reflection  Use Phong approximation to true specular (highlight) reflection.

Realism: Indirect Illumination lDistributed recursive ray tracing can handle multiple specular bounces. Recursion halts when a diffuse surface is encountered. lRadiosity can handle multiple diffuse bounces. lThe general case:  Some research into adapting and combining radiosity and ray tracing.  Results tend to be very inefficient. [1] Diffuse  Specular [2] Specular  Specular [3] Diffuse  Diffuse [4] Specular  Diffuse

Realism: Effects 1.Motion Blur:  [RT]Temporally distributed ray tracing OR  [PSC]Blending between frames with Accumulation Buffer. 2.Refraction:  [RT]Recursive ray tracing of refracted rays.  [PSC]Not supported but can do non-refractive transparency. 3.Depth of Field (focus):  [RT] Distributed ray tracing over the camera lense OR  [PSC] Use z-values to affect focus lOther Effects: lense flare, antialiasing.  Achievable by both RT and PSC.

Realism: Reflection in PSC l[RT] Recursive ray tracing of reflected rays. l[PSC] Environment Mapping  Also known as reflection mapping.  Render the scene in all directions from a single viewpoint. Can render spherically or onto a box. Use these rendered images as a texture map.  Prone to distortion. A single central reflection point stands in for multiple reflection points on the object’s surface. Particularly poor for planar surfaces. l[PSC] Scene Duplication  Create a darker duplicate of the scene behind the reflective object. Only really works for reflections off a plane.

Realism: Shadows lShadows depend on visibility from the light source. Surfaces not visible are in shadow. l[RT] Basic ray tracing, with rays shot to light sources. l[PSC] Methods: 1.Scan-line: intersect current scan-line with potential shadowing edges – very expensive. 2.Pre-processed surface detail: place surface detail polygons over surfaces visible from the light source. Done once per scene per light source. 3.Shadow volumes: The volume in shadow behind a polygon is stored. During rendering viewing rays are intersected with the shadow volumes. 4.Two-pass z-buffer: Combine z-buffers rendered from the light source (shadow mask) and viewpoint. Efficient but can have aliasing problems.

Rendering Evaluation lPSC is capable of emulating many of the effects but not illumination of RT. PSCRT Illumination: diffuse  specular  specular  specular  diffuse  diffuse (+ Radiosity) specular  diffuse  Effects: motion blur refraction  reflection shadows

Costs: Computation lGiven: lRecursive and Distributed Ray Tracing:   ray-object intersections lPolygon Scan Conversion:  polygon scan-line renderings  Seemingly independent of, but affects screen size of polygons.  As increases the size of polygons and their scan-line rendering time tend to decrease.

Costs: Memory lBoth algorithms require:  screen buffer (assuming 16 bits per colour component)  scene storage (assuming exclusively naïve triangle meshes ) lRecursive and Distributed Ray Tracing:  Negligible storage of bounding objects or cell occupancy lists. Otherwise no extra costs. lPolygon Scan Conversion:  Z-buffer: (assuming 32 bit depth value)

What is Actually Used? lVisualization:  Clarity over realism = PSC  Sometimes volumetric rendering is required. lSimulation, 3D Games:  Speed over realism = PSC + (Sprites, Ray Traced backdrops, Radiosity)  Extensive pre-processing is acceptable. lSpecial Effects:  Realism over speed BUT high resolution (6000x4000) = Mostly PSC + some RT for ‘beauty’ shots.  Example: Toy Story (proprietary PSC required 1-3 hours per frame).  Not all SFX are computer generated. Use Models, mattes (hand- painted backgrounds) and pyrotechnics.