Anti-Aliasing Jian Huang, CS594, Fall 2008 This set of slides references our text book and the slides used at Ohio State by Prof. Roger Crawfis.

Aliasing?

Aliasing Aliasing comes from in-adequate sampling rates of the continuous signal The theoretical foundation of anti- aliasing has to do with frequency analysis It’s always easier to look at 1D cases, so let’s first look at a few of those.

Example of Sampling

Examples of Inadequate Sampling

Cosine Integrations Consider these formulas

Improper Cosine Integrals Evaluating from (- ,  )

Fourier Transform The Fourier Transform represents a periodic function as a continuous summation of sin’s and cos’s.

Fourier Transform Note: This is also called the direct current or DC component.

Inverse Fourier Transform We can get back our original function f(x) from F(u) using the inverse transform:

OK – Why do this? Fourier space is a very good space for analyzing and understanding our signals. Rarely ever want to transform to Fourier space. There are some great theories developed in terms of sampling and convolution.

Fourier Analysis By looking at F(u), we get a feel for the “frequencies” of the signal. We also call this frequency space. Intuitively, you can envision, the sharper an edge, the higher the frequencies. From a numerical analysis standpoint, the sharper the edge the greater the tangent magnitude, and hence the interpolation errors.

Fourier Analysis Bandlimited –We say a function is bandlimited, if F(u)=0 for all frequencies u>c and u<-c. Amplitude Spectrum –The magnitude, |F(u)|, is called the amplitude spectrum or simply the spectrum. Phase Spectrum or Phase

Fourier Properties Linearity Scaling

Convolution Definition:

Convolution Pictorially f(x) h(x)

Convolution f(t) x h(t-x)

Convolution Consider the function (box filter):

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This function windows our function f(x). f(t)

Convolution This particular convolution smooths out some of the high frequencies in f(x). f(x)  g(x) f(t)

Another Look At Convolution

Filtering and Convolution Different functions achieve different Results.

Impulse Function Consider the special function (called impulse): such that,

Impulse and Convolution Then, if we take the convolution of f(x) with  (x), we get: f(x)  (x) = f(x)

Sampling Function A Sampling Function or Impulse Train is defined by: where T is the sample spacing. T

Sampling Function The Fourier Transform of the Sampling Function is itself a sampling function. The sample spacing is the inverse.

Convolution Theorem The convolution theorem states that convolution in the spatial domain is equivalent to multiplication in the frequency domain, and vica versa.

Convolution Theorem This powerful theorem can illustrate the problems with our point sampling and provide guidance on avoiding aliasing. Consider: f(x) S T (x) f(t) T

Convolution Theorem What does this look like in the Fourier domain? F(u) S(u)

Convolution Theorem In Fourier domain we would convolve F(u) S(u) S(u)  F(u)

Aliasing What this says, is that any frequencies greater than a certain amount will appear intermixed with other frequencies. In particular, the higher frequencies for the copy at 1/T intermix with the low frequencies centered at the origin.

Aliasing and Sampling Note, that the sampling process introduces frequencies out to infinity. We have also lost the function f(x), and now have only the discrete samples. This brings us to our next powerful theory.

Sampling Theorem The Shannon Sampling Theorem A band-limited signal f(x), with a cutoff frequency of, that is sampled with a sampling spacing of T may be perfectly reconstructed from the discrete values f[nT] by convolution with the sinc(x) function, provided: is called the Nyquist limit.

Sampling Theory Why is this? The Nyquist limit will ensure that the copies of F(u) do not overlap in the frequency domain. I can completely reconstruct or determine f(x) from F(u) using the Inverse Fourier Transform.

Sampling Theory In order to do this, I need to remove all of the shifted copies of F(u) first. This is done by simply multiplying F(u) by a box function of width 2. F(u) S(u) 

Sampling Theory In order to do this, I need to remove all of the shifted copies of F(u) first. This is done by simply multiplying F(u) by a box function of width 2. F(u) S(u) 

General Process Original function Sampled function Reconstructed Function Acquisition Reconstruction Re-sampled function Resampling

How? - Convolution Spatial Domain: Mathematically: f(x)*h(x) = Frequency Domain: Evaluated at discrete points (sum) Multiplication: Convolution:

Reconstruction Mathematically: f(x)*h(x) = (  f[i])*h(x)  f[i] h(x)

General Process - Frequency Domain Acquisition Reconstruction Resampling Original function Sampled function Reconstructed Function Re-sampled function

Sampling What we have in computer graphics is a point sampling of our scene, or: –I(x) = f(x)S T (x) What we would like is more of an integration across the pixel (or larger area): –I(x) = f(x)  h(x) What should h(x) be?

Pre-Filtering Acquisition Reconstruction Original function Band-limited function Sampled Function Reconstructed function

Ideal Reconstruction with Sinc function Spatial Domain: convolution is exact Frequency Domain: cut off freq. replica

Reconstructing Derivatives Spatial Domain: convolution is exact Frequency Domain: cut off freq. replica

Possible Errors Post-aliasing –reconstruction filter passes frequencies beyond the Nyquist frequency (of duplicated frequency spectrum) => frequency components of the original signal appear in the reconstructed signal at different frequencies Smoothing –frequencies below the Nyquist frequency are attenuated Ringing (overshoot) –occurs when trying to sample/reconstruct discontinuity Anisotropy –caused by not spherically symmetric filters

How Good? = Error Spatial Domain: local error asymptotic error numerical error Frequency Domain: global error visual appearance blurring aliasing smoothing Approximation Theory/Analysis Signal Processing

Sources of Aliasing Non-bandlimited signal Low sampling rate (below Nyquist) Non perfect reconstruction

Reconstruction Kernels stop bandpass band Smoothing error Postaliasing error Ideal filter filter The spatial extent of reconstruction kernels, or interpolation basis functions, depend on the cut-off frequency as well.

Reconstruction Kernels Nearest Neighbor (Box) Triangular func Sinc Gaussian + many others Spatial d.Frequency d.

Higher Dimensions An-isotropic Filters (radially symmetric) separable filters

Interpolation (an example) Very important; regardless of algorithm expensive => done very often for one image Requirements for good reconstruction –performance –stability of the numerical algorithm –accuracy Nearest neighbor Linear

Put Things in Perspective In graphics, need to use continuous space functions. But can only work with discrete data. So, let’s reconstruct from discrete data to continuous space (convolution) and resample Interpolation is doing the same thing. Computing one data point in the resulting function, say, at x 1. So, which reconstruction kernel (basis function) does linear/bilinear/tri-linear interpolations use?

A Cosine Example Consider the function f(x)=cos(2  x). x u F(u) f(x)

Sampling Theory So, given f[nT] and an assumption that f(x) does not have frequencies greater than 1/2T, we can write the formula: f[nT] = f(x) S T (x)  F(u)  S T (u) F(u) = (F(u)  S T (u)) Box 1/2T (u) therefore, f(x) = f[nT]  sinc(x)

A Cosine Example Now sample it at T=½ x u f(x) 1 F(u)

A Cosine Example Problem: –The amplitude is now wrong or undefined. Note however, that there is one and only one cosine with a frequency less than or equal to 1 that goes through the sample pts. u 1 F(u)

A Cosine Example What if we sample at T=¾? x u f(x) 1 F(u)

Supersampling Supersampling increases the sampling rate, and then integrates or convolves with a box filter, which is finally followed by the output sampling function. x f(x)

Sampling and Anti-aliasing If you can not get rid of it, convert it to noise. Basic checkerboard Checkerboard with noise

Sampling and Anti-aliasing The images were calculated as follows: –A 2Kx2K image was constructed and smoothly rotated into 3D. –For Uniform Sampling, it was downsampled to a 512x512 image. –Noise was added to the image, sharpened and then downsampled for the other one. –Both were converted to B&W.

Sampling and Anti-aliasing The problem: –The signal is not band-limited. –Uniform sampling can pick-up higher frequency patterns and represent them as low-frequency patterns. F(u) S(u)

Sampling and Anti-aliasing Turning the high-frequencies into noise. Recall, multiplication by the sampling function is equivalent to convolution in the Fourier domain. Fourier transform of noise.

Non-uniform Sampling

Quality considerations So far we just mapped one point results in bad aliasing (resampling problems) we really need to integrate over polygon super-sampling is not a very good solution (slow!) most popular (easiest) - mipmaps

Quality considerations Pixel area maps to “ weird ” (warped) shape in texture space pixel u v xsxs ysys

Quality considerations We need to: –Calculate (or approximate) the integral of the texture function under this area –Approximate: Convolve with a wide filter around the center of this area Calculate the integral for a similar (but simpler) area.

Quality considerations the area is typically approxiated by a rectangular region (found to be good enough for most applications) filter is typically a box/averaging filter - other possibilities how can we pre-compute this?

Mip-maps An image-pyramid is built. 256 pixels

Mip-maps Find level of the mip-map where the area of each mip-map pixel is closest to the area of the mapped pixel. pixel u v xsxs ysys 2x2 pixels level selected

Mip-maps Pros –Easy to calculate: Calculate pixels area in texture space Determine mip-map level Sample or interpolate to get color Cons –Area not very close – restricted to square shapes (64x64 is far away from 128x128). –Location of area is not very tight.

Summed Area Table (SAT) Use an axis aligned rectangle, rather than a square Precompute the sum of all texels to the left and below for each texel location –For texel (u,v), replace it with: sum (texels(i=0…u,j=0…v))

Summed Area Table (SAT) Determining the rectangle: –Find bounding box and calculate its aspect ratio pixel u v xsxs ysys

Summed Area Table (SAT) Determine the rectangle with the same aspect ratio as the bounding box and the same area as the pixel mapping. pixel u v xsxs ysys

Summed Area Table (SAT) Center this rectangle around the bounding box center. Formula: Area = aspect_ratio*x*x Solve for x – the width of the rectangle Other derivations are also possible using the aspects of the diagonals, …

Summed Area Table (SAT) Calculating the color –We want the average of the texel colors within this rectangle u v (u3,v3) (u2,v2) (u1,v1) (u4,v4)

Summed Area Table (SAT) To get the average, we need to divide by the number of texels falling in the rectangle. –Color = SAT(u3,v3)-SAT(u4,v4)-SAT(u2,v2)+SAT(u1,v1) –Color = Color / ( (u3-u1)*(v3-v1) ) This implies that the values for each texel may be very large: –For 8-bit colors, we could have a maximum SAT value of 255*nx*ny –32-bit pixels would handle a 4kx4k texture with 8-bit values. –RGB images imply 12-bytes per pixel.

Summed Area Table (SAT) Pros –Still relatively simple Calculate four corners of rectangle 4 look-ups, 5 additions, 1 mult and 1 divide. –Better fit to area shape –Better overlap Cons –Large texel SAT values needed –Still not a perfect fit to the mapped pixel.

Elliptical Weighted Average (EWA) Filter Treat each pixel as circular, rather than square. Mapping of a circle is elliptical in texel space. pixel u v xsxs ysys

EWA Filter Precompute? Can use a better filter than a box filter. Heckbert chooses a Gaussian filter.

EWA Filter Calculating the Ellipse Scan converting the Ellipse Determining the final color (normalizing the value or dividing by the weighted area).

EWA Filter Calculating the ellipse –We have a circular function defined in (x,y). –Filtering that in texture space h(u,v). –(u,v) = T(x,y) –Filter: h(T(x,y))

EWA Filter Ellipse: –  (u,v) = Au 2 + Buv + Cv 2 = F –(u,v) = (0,0) at center of the ellipse A = v x 2 +v y 2 B = -2(u x v y + u y v x ) C = u x 2 +u y 2 F = u x v y + u y v x

EWA Filter Scan converting the ellipse: –Determine the bounding box –Scan convert the pixels within it, calculating  (u,v). –If  (u,v) < F, weight the underlying texture value by the filter kernel and add to the sum. –Also, sum up the filter kernel values within the ellipse.

EWA Filter Determining the final color –Divide the weighted sum of texture values by the sum of the filter weights.

EWA Filter What about large areas? –If m pixels fall within the bounding box of the ellipse, then we have O(n 2 m) algorithm for an nxn image. –m maybe rather large. We can apply this on a mip-map pyramid,rather than the full detailed image. –Tighter-fit of the mapped pixel –Cross between a box filter and gaussian filter. –Constant complexity - O(n 2 )