1. © University of Wisconsin, CS559 Spring 2004
Last Time
Filtering
Box filter
Bartlett filter
Gaussian Filter
Edge-detect (high-pass) filter
Enhancement filters
Resampling
Map the point from the new image back into the old image
Locally reconstruct the function using a filter
Today we’ll see why this is the right thing to do
2/12/04
© University of Wisconsin, CS559 Spring 2004

2. © University of Wisconsin, CS559 Spring 2004
Today
Ideal reconstruction and aliasing
Compositing
2/12/04
© University of Wisconsin, CS559 Spring 2004

3. © University of Wisconsin, CS559 Spring 2004
Ideal Reconstruction
When you display an image, ideally you would like to reconstruct the original (ideal) picture
When you resample, you would like to draw new samples from the perfect original function
Last time we saw you generally can’t do this because you need infinitely dense samples to reconstruct sharp edges
What’s the math?
2/12/04
© University of Wisconsin, CS559 Spring 2004

4. Sampling in Spatial Domain
Sampling in the spatial domain is like multiplying by a spike function
You take some ideal function and get data for a regular grid of points 
2/12/04
© University of Wisconsin, CS559 Spring 2004

5. Sampling in Frequency Domain
Sampling in the frequency domain is like convolving with a spike function
Follows from the convolution theory: multiplication in spatial equals convolution in frequency
Spatial spike function in the frequency domain is also the spike function 
2/12/04
© University of Wisconsin, CS559 Spring 2004

6. Reconstruction (Frequency Domain)
To reconstruct, we must restore the original spectrum
That can be done by multiplying by a square pulse 
2/12/04
© University of Wisconsin, CS559 Spring 2004

7. Reconstruction (Spatial Domain)
Multiplying by a square pulse in the frequency domain is the same as convolving with a sinc function in the spatial domain 
2/12/04
© University of Wisconsin, CS559 Spring 2004

8. Aliasing Due to Under-sampling
If the sampling rate is too low, high frequencies get reconstructed as lower frequencies
High frequencies from one copy get added to low frequencies from another  
2/12/04
© University of Wisconsin, CS559 Spring 2004

9. © University of Wisconsin, CS559 Spring 2004
More Aliasing
Poor reconstruction also results in aliasing
Consider a signal reconstructed with a box filter in the spatial domain (which means using a sinc in the frequency domain):  
2/12/04
© University of Wisconsin, CS559 Spring 2004

10. © University of Wisconsin, CS559 Spring 2004
Aliasing in Practice
We have two types of aliasing:
Aliasing due to insufficient sampling frequency
Aliasing due to poor reconstruction
You have some control over reconstruction
If resizing, for instance, use an approximation to the sinc function to reconstruct (instead of Bartlett, as we used last time)
Gaussian is closer to sinc than Bartlett
But note that sinc function goes on forever (infinite support), which is inefficient to evaluate
You have some control over sampling if creating images using a computer
Remove all sharp edges (high frequencies) from the scene before drawing it
That is, blur character and line edges before drawing
2/12/04
© University of Wisconsin, CS559 Spring 2004

11. © University of Wisconsin, CS559 Spring 2004
Compositing
Compositing combines components from two or more images to make a new image
The basis for film special effects (even before computers)
Create digital imagery and composite it into live action
Important part of animation – even hand animation
Background change more slowly than foregrounds, so composite foreground elements onto constant background
2/12/04
© University of Wisconsin, CS559 Spring 2004

12. © University of Wisconsin, CS559 Spring 2004
Very Simple Example =
over
2/12/04
© University of Wisconsin, CS559 Spring 2004

13. © University of Wisconsin, CS559 Spring 2004
Mattes
A matte is an image that shows which parts of another image are foreground objects
Term dates from film editing and cartoon production
How would I use a matte to insert an object into a background?
How are mattes usually generated for television?
2/12/04
© University of Wisconsin, CS559 Spring 2004

14. © University of Wisconsin, CS559 Spring 2004
Working with Mattes
To insert an object into a background
Call the image of the object the source
Put the background into the destination
For all the source pixels, if the matte is white, copy the pixel, otherwise leave it unchanged
To generate mattes:
Use smart selection tools in Photoshop or similar
They outline the object and convert the outline to a matte
Blue Screen: Photograph/film the object in front of a blue background, then consider all the blue pixels in the image to be the background
2/12/04
© University of Wisconsin, CS559 Spring 2004

15. © University of Wisconsin, CS559 Spring 2004
Alpha
Basic idea: Encode opacity information in the image
Add an extra channel, the alpha channel, to each image
For each pixel, store R, G, B and Alpha
alpha = 1 implies full opacity at a pixel
alpha = 0 implies completely clear pixels
There are many interpretations of alpha
Is there anything in the image at that point (web graphics)
Transparency (real-time OpenGL)
Images are now in RGBA format, and typically 32 bits per pixel (8 bits for alpha)
All images in the project are in this format
2/12/04
© University of Wisconsin, CS559 Spring 2004

16. © University of Wisconsin, CS559 Spring 2004
Pre-Multiplied Alpha
Instead of storing (R,G,B,), store (R,G,B,)
The compositing operations in the next several slides are easier with pre-multiplied alpha
To display and do color conversions, must extract RGB by dividing out 
=0 is always black
Some loss of precision as  gets small, but generally not a big problem
2/12/04
© University of Wisconsin, CS559 Spring 2004

17. Compositing Assumptions
We will combine two images, f and g, to get a third composite image
Not necessary that one be foreground and background
Background can remain unspecified
Both images are the same size and use the same color representation
Multiple images can be combined in stages, operating on two at a time
2/12/04
© University of Wisconsin, CS559 Spring 2004

18. © University of Wisconsin, CS559 Spring 2004
Image Decomposition
The composite image can be broken into regions
Parts covered by f only
Parts covered by g only
Parts covered by f and g
Parts covered by neither f nor g
Compositing operations define what should happen in each region: who (f or g) owns each region
2/12/04
© University of Wisconsin, CS559 Spring 2004

19. Basic Compositing Operation
At each pixel, combine the pixel data from f and the pixel data from g with the equation:
F and G describe how much of each input image survives, and cf and cg are pre-multiplied pixels, and all four channels are calculated
To define a compositing operation, define F and G
2/12/04
© University of Wisconsin, CS559 Spring 2004

20. © University of Wisconsin, CS559 Spring 2004
Sample Images
Image
Alpha
2/12/04
© University of Wisconsin, CS559 Spring 2004

21. © University of Wisconsin, CS559 Spring 2004
“Over” Operator
2/12/04
© University of Wisconsin, CS559 Spring 2004

22. © University of Wisconsin, CS559 Spring 2004
“Over” Operator
Computes composite with the rule that f covers g
2/12/04
© University of Wisconsin, CS559 Spring 2004

23. © University of Wisconsin, CS559 Spring 2004
“Inside” Operator
2/12/04
© University of Wisconsin, CS559 Spring 2004

24. © University of Wisconsin, CS559 Spring 2004
“Inside” Operator
Computes composite with the rule that only parts of f that are inside g contribute
2/12/04
© University of Wisconsin, CS559 Spring 2004

25. © University of Wisconsin, CS559 Spring 2004
“Outside” Operator
2/12/04
© University of Wisconsin, CS559 Spring 2004

26. © University of Wisconsin, CS559 Spring 2004
“Outside” Operator
Computes composite with the rule that only parts of f that are outside g contribute
2/12/04
© University of Wisconsin, CS559 Spring 2004

27. © University of Wisconsin, CS559 Spring 2004
“Atop” Operator
2/12/04
© University of Wisconsin, CS559 Spring 2004

28. © University of Wisconsin, CS559 Spring 2004
“Atop” Operator
Computes composite with the over rule but restricted to places where there is some g
2/12/04
© University of Wisconsin, CS559 Spring 2004

29. © University of Wisconsin, CS559 Spring 2004
“Xor” Operator
Computes composite with the rule that f contributes where there is no g, and g contributes where there is no f
2/12/04
© University of Wisconsin, CS559 Spring 2004

30. © University of Wisconsin, CS559 Spring 2004
“Xor” Operator
2/12/04
© University of Wisconsin, CS559 Spring 2004

31. © University of Wisconsin, CS559 Spring 2004
“Clear” Operator
Computes a clear composite
Note that (0,0,0,>0) is a partially opaque black pixel, whereas (0,0,0,0) is fully transparent, and hence has no color
2/12/04
© University of Wisconsin, CS559 Spring 2004

32. © University of Wisconsin, CS559 Spring 2004
“Set” Operator
Computes composite by setting it to equal f
Copies f into the composite
2/12/04
© University of Wisconsin, CS559 Spring 2004

33. Compositing Operations
F and G describe how much of each input image survives, and cf and cg are pre-multiplied pixels, and all four channels are calculated
Operation F G
Over
Inside
Outside
Atop
Xor
Clear
Set
2/12/04
© University of Wisconsin, CS559 Spring 2004

34. © University of Wisconsin, CS559 Spring 2004
Unary Operators
Darken: Makes an image darker (or lighter) without affecting its opacity
Dissolve: Makes an image transparent without affecting its color
2/12/04
© University of Wisconsin, CS559 Spring 2004

35. © University of Wisconsin, CS559 Spring 2004
“PLUS” Operator
Computes composite by simply adding f and g, with no overlap rules
Useful for defining cross-dissolve in terms of compositing:
2/12/04
© University of Wisconsin, CS559 Spring 2004

36. © University of Wisconsin, CS559 Spring 2004
Obtaining  Values
Hand generate (paint a grayscale image)
Automatically create by segmenting an image into foreground background:
Blue-screening is the analog method
Remarkably complex to get right
“Lasso” is the Photoshop operation
With synthetic imagery, use a special background color that does not occur in the foreground
Brightest blue or green is common
2/12/04
© University of Wisconsin, CS559 Spring 2004

37. Compositing With Depth
Can store pixel “depth” instead of alpha
Then, compositing can truly take into account foreground and background
Generally only possible with synthetic imagery
Image Based Rendering is an area of graphics that, in part, tries to composite photographs taking into account depth
2/12/04
© University of Wisconsin, CS559 Spring 2004

38. © University of Wisconsin, CS559 Spring 2004
Where to now…
We are now almost done with images
Still have to take a quick look at painterly rendering
We will spend several weeks on the mechanics of 3D graphics
Coordinate systems and Viewing
Clipping
Drawing lines and polygons
Lighting and shading
We will finish the semester with modeling and some additional topics
2/12/04
© University of Wisconsin, CS559 Spring 2004