Multisample Antialiasing Kurt Akeley CS248 Lecture 6 11 October

CS248 Lecture 6Kurt Akeley, Fall 2007 Overview Two approaches to antialiasing n Pre-filter n Over-sample: increase sample rate Pre-filter n Works well for points and lines, not for triangles n Can’t reasonably represent pre-filtered triangles n Can’t handle occlusion n Can effectively eliminate aliasing with a moderately-large filter window Over-sample n Works well for all primitives n Though less well than pre-filtering for points and lines n Cannot eliminate aliasing! n Let’s see why …

CS248 Lecture 6Kurt Akeley, Fall 2007 Aliasing of fundamentals and harmonics Imperfect analogy, but useful pedagogical tool: n Fundamental: spatial density of edges n Harmonics: spectrum introduced by a single edge Over-sampling cannot overcome increased fundamental frequency n Demo But over-sampling, followed by a resampling at a lower rate, can overcome harmonics due to a single edge

CS248 Lecture 6Kurt Akeley, Fall 2007 Practical solution Over-sample to reduce jaggies due to edges n Optimize for the case of a single edge Use other approaches to limit edge rates n Band-limit fundamental geometric frequencies n E.g., level-of-detail (LOD) model selection n Band-limit image fundamentals and harmonics n E.g., texture MIPmapping

CS248 Lecture 6Kurt Akeley, Fall 2007 LOD model selection Low LODMedium LODHigh LOD Demo

CS248 Lecture 6Kurt Akeley, Fall 2007 Supersampling

CS248 Lecture 6Kurt Akeley, Fall 2007 Supersampling Supersampling algorithm: 1. Over-sample, e.g., at 4x the pixel rate 2. Reconstruct at the over-sampled rate 3. Band-limit to match the pixel rate 4. Resample at the pixel rate to yield pixel values 5. Reconstruct to display

CS248 Lecture 6Kurt Akeley, Fall 2007 Sample at 4x pixel rate x = * = F(s)f(x) Pixel-rate fundamental

CS248 Lecture 6Kurt Akeley, Fall 2007 Reconstruct 4x samples x = * = F(s)f(x) Cutoff at ½ the over-sample rate Aliased !

CS248 Lecture 6Kurt Akeley, Fall 2007 Band-limit 4x reconstruction to pixel-rate x = * = F(s)f(x)

CS248 Lecture 6Kurt Akeley, Fall 2007 Resample at pixel rate x = * = F(s)f(x)

CS248 Lecture 6Kurt Akeley, Fall 2007 Reconstruct pixel samples x = * = F(s)f(x) Rect filter approximates LCD display

CS248 Lecture 6Kurt Akeley, Fall 2007 Optimizations The over-sample reconstruction convolution and the band-limiting convolution steps can be combined: n Convolution is associative (f * g) * h = f * (g * h) And g and h are constant n f * (g * h) = f * filter The filter convolution can be reduced to a simple weighted sum of sample values: n The result is sampled at pixel rate n So only values at pixel centers are needed n These are weighted sums of the 4x samples

CS248 Lecture 6Kurt Akeley, Fall 2007 Optimized supersampling Do once 1. Compute filter by convolving the over-sample reconstruction and band-limiting filters 2. Compute filter weights for a single pixel sample by sampling the filter waveform During rendering: 1. Over-sample 2. Filter to compute pixel values from sample values using the filter weights During display 1. Reconstruct

CS248 Lecture 6Kurt Akeley, Fall 2007 Sample Patterns

CS248 Lecture 6Kurt Akeley, Fall 2007 Sample patterns Pattern categories n Regular grid n Sparse regular grid n Pseudo-random (stochastic) Filter extent ( filter combines reconstruction and band-limiting) n Within the unit square of a pixel n Beyond the unit square Pattern variation n Temporal n Spatial Shading rate vs. geometric sampling rate n Same n Different

CS248 Lecture 6Kurt Akeley, Fall 2007 Regular-grid sample patterns Implementation n Regular sample spacing simplifies attribute evaluation n Mask, color, and depth n Large sample count is expensive Edge optimization n Poor, especially for screen-aligned edges Image quality n Good for large numbers of samples Single Pixel

CS248 Lecture 6Kurt Akeley, Fall 2007 Sparse regular-grid sample patterns Implementation n Regular sample spacing simplifies attribute evaluation n Mask, color, and depth n Moderate sample count is cost-effective Edge optimization n Good, especially for screen-aligned edges Image quality n Good for moderate numbers of samples Single Pixel 8-rooks pattern 8-queens pattern

CS248 Lecture 6Kurt Akeley, Fall 2007 NVIDIA Quincunx Single Pixel Quincunx pattern Quincunx is a special example of a sparse regular grid n Only two samples per pixel (center and one corner) n Filter accesses five samples n Center n Four corners Implementation n Regular sample spacing simplifies attribute evaluation n Mask, color, and depth n Minimal sample count (2 per pixel) is very cost-effective Edge optimization n OK, though less optimal for screen-aligned edges Image quality n Better than without antialiasing This information is from memory, I did not confirm it with NVIDIA.

CS248 Lecture 6Kurt Akeley, Fall 2007 Stochastic sample patterns Implementation n Irregular sample spacing complicates attribute evaluation n Mask, color, and depth n Moderate sample count is cost-effective Edge optimization n Good, though less optimal for screen- aligned edges Image quality n Excellent for moderate numbers of samples Single Pixel random pattern

CS248 Lecture 6Kurt Akeley, Fall 2007 Filter extent Within pixel unit square n Near optimal for small sample counts (<=8) n Equal sample weighting is typical n Easily parallelizable n Simplifies immediate filtering Beyond the pixel unit square n Required for large sample counts (>=16) n Sinc/Gaussian sample weights n Implementation is more difficult n May filter only during display “The SAGE Graphics Architecture”, Michael Deering and David Naegle, Proceedings of SIGGRAPH 2002 Single Pixel

CS248 Lecture 6Kurt Akeley, Fall 2007 Variation in sample patterns Vary patterns n Temporally: frame to frame n Spatially: pixel to pixel Improves image quality by replacing regular aliasing patterns with noise Complicates implementation Interferes with repeatability n Violated for temporal variation n Requires spatial pattern to be window-aligned

CS248 Lecture 6Kurt Akeley, Fall 2007 The accumulation buffer OpenGL mechanism to support supersampling Additional color buffer n Same spatial dimensions as the frame buffer n Greater color resolution and range (-1,1) Used to accumulate (sum) the results of multiple rendering passes Many applications n Supersample antialiasing n Motion blur n Depth of field blur n Combinations of these effects and more …

CS248 Lecture 6Kurt Akeley, Fall 2007 Accumulation buffer supersampling glutInitDisplay(… | GLUT_ACCUM); // setup desired rendering modes glAccum(GL_LOAD, 0.0); for (int i=0; i<n; ++i) { // jitter scene to sample location i // render the entire scene glAccum(GL_ACCUM, sampleweight[i]); } glAccum(GL_RETURN, 1.0);

CS248 Lecture 6Kurt Akeley, Fall 2007 Supersampling features and limitations Features n Unconstrained sample locations (stochastic) n Unconstrained filter shapes and extents n Adaptive quality / performance trade-off Utilitizing glAccum(GL_MULT, factor ); Limitations n No per-pixel variation in sample pattern n Low performance

CS248 Lecture 6Kurt Akeley, Fall 2007 Multisampling

CS248 Lecture 6Kurt Akeley, Fall 2007 Shading rate (another pattern variation) Shading - the computation of color for a fragment Supersampling n Shades each sample n E.g., accumulation buffer n Or shades at an independently specified rate n E.g., REYES Multisampling is an optimized form of supersampling n Optimized n To eliminate jaggies due to edge aliasing n For efficiency and performance n Shades each fragment once (potentially huge savings!) n There are no geometric edges in a fragment n Use pre-filtering to eliminate edges in shading source data n Stores color with each sample n There may be edges within a pixel

CS248 Lecture 6Kurt Akeley, Fall 2007 Multisample implementation ( n samples) Vertex assembly Primitive assembly Rasterization Fragment operations Display Vertex operations Application Primitive operations struct { float x,y,z,w; float r,g,b,a; } vertex; struct { vertex v0,v1,v2 } triangle; struct { short x,y; bool mask[n]; float depth[n]; float r,g,b,a; } fragment; struct { short R,G,B; int depth[n]; byte r[n],g[n],b[n]; } pixel; Frame buffer Point sampling at multisample locations sets mask and computes depths. Color is taken from center-most sample. Fragment samples are depth-merged into the multisample buffer, then filtered to the color buffer.

CS248 Lecture 6Kurt Akeley, Fall 2007 Multisample rasterization operations Fragment selection n Identify pixels for which fragments are to be generated n New: generate fragment if any sample is within the primitive n Requires tiled sampling, rather than point sampling n Generates more fragments Attribute assignment n Assign attribute values to each fragment n Sample color at the center of the pixel (as before) n New: compute the Boolean per-sample coverage mask n True if-and-only-if the sample is within the primitive n New: compute depth values for each sample location

CS248 Lecture 6Kurt Akeley, Fall 2007 Point-sampled fragment selection Generate fragment if pixel center is inside triangle Implements point-sampled aliased rasterization 9 fragments generated

CS248 Lecture 6Kurt Akeley, Fall 2007 Tiled fragment selection Generate fragment if unit square intersects triangle Implements multisample rasterizations n 4x4 sample pattern with unit-square filter extent 21 fragments generated

CS248 Lecture 6Kurt Akeley, Fall 2007 Multisampling in OpenGL glutInitDisplay(… | GLUT_MULTISAMPLE); glEnable(GL_MULTISAMPLE); // render as usual, almost everything works “Multisampling is trivial to add to an application.” ― OpenGL Programming Guide

CS248 Lecture 6Kurt Akeley, Fall 2007 Occlusion revisited The conundrum: n Pre-filtering blurs geometry n Accurate occlusion requires sampling of exact (sharply defined) geometry n Can’t have both  A partial solution: n Pre-filtered geometry cannot occlude accurately n occluded objects bleed through the seems n But pre-filtered geometry can be occluded by exact geometry

CS248 Lecture 6Kurt Akeley, Fall 2007 Getting the best of both approaches glutInitDisplay(…, GLUT_MULTISAMPLE); // setup occlusion modes (e.g. depth buffering) glEnable(GL_MULTISAMPLE); // render triangles (must happen first) glDisable(GL_MULTISAMPLE); glEnable(GL_LINE_SMOOTH); glEnable(GL_BLEND); glBlendFunc(GL_SRC_ALPHA, GL_ONE); // render lines and/or points (must happen last) glDisable(GL_LINE_SMOOTH); glDisable(GL_BLEND);

CS248 Lecture 6Kurt Akeley, Fall 2007 Summary Supersampling does not eliminate aliasing n It can reduce jaggies n It must be used with other pre-filtering to eliminate aliasing Multisampling n Is supersampling with pixel-rate shading n Optimized for efficiency and performance n Is a full-scene antialiasing technique n Works for all rendering primitives n Interacts well with other OpenGL mechanisms –Especially fragment/frame buffer operations n Handles occlusions correctly n Pre-filtering does not n Can be used in conjunction with pre-filter antialiasing n To optimize point and line quality

CS248 Lecture 6Kurt Akeley, Fall 2007 Assignments Before next Tuesday’s class, read n FvD 5 n OpenGL multisample specification(s) n n Project 2: n Will be assigned next Tuesday

CS248 Lecture 6Kurt Akeley, Fall 2007 End