1. Sampling and Reconstruction
The sampling and reconstruction process
Real world: continuous
Digital world: discrete
Basic signal processing
Fourier transforms
The convolution theorem
The sampling theorem
Aliasing and antialiasing
Uniform supersampling
Nonuniform supersampling
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

2. Camera Simulation Sensor response Lens Shutter Scene radiance
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

3. Imagers = Signal Sampling
All imagers convert a continuous image to a discrete sampled image by integrating over the active “area” of a sensor.
Examples:
Retina: photoreceptors
CCD array
Virtual CG cameras do not integrate, they simply sample radiance along rays …
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

4. Displays = Signal Reconstruction
All physical displays recreate a continuous image from a discrete sampled image by using a finite sized source of light for each pixel.
Examples:
DACs: sample and hold
Cathode ray tube: phosphor spot and grid
DAC
CRT
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

5. Sampling in Computer Graphics
Artifacts due to sampling - Aliasing
Jaggies
Moire
Flickering small objects
Sparkling highlights
Temporal strobing
Preventing these artifacts - Antialiasing
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

6. Jaggies Retort sequence by Don Mitchell Staircase pattern or jaggies
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

7. Basic Signal Processing

8. Fourier Transforms
Spectral representation treats the function as a weighted sum of sines and cosines
Each function has two representations
Spatial domain - normal representation
Frequency domain - spectral representation
The Fourier transform converts between the spatial and frequency domain
Spatial
Domain
Frequency
Domain
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

9. Spatial and Frequency Domain
Spatial Domain
Frequency Domain
Pictures of Fourier transforms
Square
Rotated square
Smaller and larger squares
Get coordinates of frequency space right
Check out optical transforms book
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

10. Convolution Definition
Convolution Theorem: Multiplication in the frequency domain is equivalent to convolution in the space domain.
Symmetric Theorem: Multiplication in the space domain is equivalent to convolution in the frequency domain.
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

11. The Sampling Theorem

12. Sampling: Spatial Domain
What is the spacing? Is it unit spacing or is it T
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

13. Sampling: Frequency Domain
Magic fact 1: Fourier transform of a sequence of pulses is a sequence of pulses
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

14. Reconstruction: Frequency Domain
Overlay box
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

15. Reconstruction: Spatial Domain
Magic fact 2: The Fourier transform of a box is a sinc
Step by step
Long discussion about interpolating functions and properties of the sinc
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

16. Sampling and Reconstruction
If the reconstruction is not perfect, can get artifacts. Thus, even though no aliasing, this is often referred to as post-aliasing.
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

17. Sampling Theorem
This result if known as the Sampling Theorem and is due to Claude Shannon who first discovered it in 1949
A signal can be reconstructed from its samples
without loss of information, if the original
signal has no frequencies above 1/2 the
Sampling frequency
For a given bandlimited function, the rate at which it must be sampled is called the Nyquist Frequency
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

18. Aliasing

19. Undersampling: Aliasing
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

20. Sampling a “Zone Plate”y
Zone plate:
Sampled at 128x128
Reconstructed to 512x512
Using a 30-wide
Kaiser windowed sinc x
Interesting question that confused me.
Does the zone pattern plate repeat? I was confused and thought I was in the Fourier
domain. And hence I thought the zone plate would repeat in the spatial domain. Does it
In fact repeat? Can you prove it?
Someone answered that the reason why was that the frequency was coupled to position.
Left rings: part of signal
Right rings: prealiasing
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

21. Ideal Reconstruction
Ideally, use a perfect low-pass filter - the sinc function - to bandlimit the sampled signal and thus remove all copies of the spectra introduced by sampling
Unfortunately,
The sinc has infinite extent and we must use simpler filters with finite extents. Physical processes in particular do not reconstruct with sincs
The sinc may introduce ringing which are perceptually objectionable
Properties of the display and visual system
ClearType example of designing the filter assuming properties of the display
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

22. Sampling a “Zone Plate”y
Zone plate:
Sampled at 128x128
Reconstructed to 512x512
Using optimal cubic x
Reconstructed using Mitchell’s cubic filter (1/3,1/3)
Left rings: part of signal
Right rings: prealiasing
Middle rings: postaliasing
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

23. Mitchell Cubic Filter Properties: From Mitchell and Netravali
(1,0) cubic b-spline
(0,C) cardinal cubics; (0,.5) catmull-rom
(B,0) duff’s tensioned b-splines
From Mitchell and Netravali
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

24. Antialiasing

25. Antialiasing by Prefiltering
Need to explain why this is a good idea. Aliases are annoying:
They move around depending on the sampling rate
The are very noticable because they show up at all frequencies
The beat against themselves to form patterns like Moires
Boxes in frequency space
Boxes in the spatial domain
And vice versa
Asymmetry is confusing.
Color code diagrams showing whether they are in the spatial domain or frequency space
Frequency Space
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

26. Antialiasing Antialiasing = Preventing aliasing
Analytically prefilter the signal
Solvable for points, lines and polygons
Not solvable in general
e.g. procedurally defined images
Uniform supersampling and resample
Nonuniform or stochastic sampling
Only for polygons with constant or intensity varying as a polynomial
Not solvable for perspective-correct polygons, or texture-mapped polygons, …
Complex when geometry intersects within a pixel
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

27. Uniform Supersampling
Increasing the sampling rate moves each copy of the spectra further apart, potentially reducing the overlap and thus aliasing
Resulting samples must be resampled (filtered) to image sampling rate
NVIDIA shows in 2D frequency space
Samples
Pixel
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

28. Point vs. Supersampled Point 4x4 Supersampled
Checkerboard sequence by Tom Duff
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

29. Analytic vs. Supersampled
How to compute the exact area. Compute the intersection of the polygon with the area corresponding.
Exact Area
4x4 Supersampled
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

30. Distribution of Extrafoveal Cones
Two properties
The energy should be evenly distributed and not concentrated into spikes
The spectrum should be deficient in low energt
This is called blue noise.
The maximal response to noise is about 4.5 cycles per degree [Mitchell 87]
Monkey eye
cone distribution
Fourier transform
Yellot theory
Aliases replaced by noise
Visual system less sensitive to high freq noise
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

31. Non-uniform Sampling Intuition Uniform sampling Non-uniform sampling
The spectrum of uniformly spaced samples is also a set of uniformly spaced spikes
Multiplying the signal by the sampling pattern corresponds to placing a copy of the spectrum at each spike (in freq. space)
Aliases are coherent, and very noticable
Non-uniform sampling
Samples at non-uniform locations have a different spectrum; a single spike plus noise
Sampling a signal in this way converts aliases into broadband noise
Noise is incoherent, and much less objectionable
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

32. Jittered Sampling Add uniform random jitter to each sample
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

33. Jittered vs. Uniform Supersampling
4x4 Jittered Sampling
4x4 Uniform
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

34. Analysis of Jitter Non-uniform sampling Jittered sampling
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell

35. Poisson Disk Sampling Dart throwing algorithm
Very expensive
Mitchell87 and lrt describe a better algorithm.
Poisson disk version of the checkerboard
Also, the reason this gets pushed out is that there is more white space in the jittered pattern. This means that the effective sampling rate is actually less because of this extra empty space.
The Fourier transform of an irregularly space set of pulses is not necessarily a set of pulses.
Can efficiently compute Fourier transform of a set of pulses.
Have not described explicitly. That we are multiplying the samples times the image and this corresponds
To convolving the FT of the image with the spectral.
Dart throwing algorithm
University of Texas at Austin CS395T - Advanced Image Synthesis Fall Don Fussell