© University of Wisconsin, CS559 Spring 2004 Last Time Filtering Box filter Bartlett filter Gaussian Filter Edge-detect (high-pass) filter Enhancement filters Resampling Map the point from the new image back into the old image Locally reconstruct the function using a filter Today we’ll see why this is the right thing to do 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Today Ideal reconstruction and aliasing Compositing 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Ideal Reconstruction When you display an image, ideally you would like to reconstruct the original (ideal) picture When you resample, you would like to draw new samples from the perfect original function Last time we saw you generally can’t do this because you need infinitely dense samples to reconstruct sharp edges What’s the math? 2/12/04 © University of Wisconsin, CS559 Spring 2004

Sampling in Spatial Domain Sampling in the spatial domain is like multiplying by a spike function You take some ideal function and get data for a regular grid of points  2/12/04 © University of Wisconsin, CS559 Spring 2004

Sampling in Frequency Domain Sampling in the frequency domain is like convolving with a spike function Follows from the convolution theory: multiplication in spatial equals convolution in frequency Spatial spike function in the frequency domain is also the spike function  2/12/04 © University of Wisconsin, CS559 Spring 2004

Reconstruction (Frequency Domain) To reconstruct, we must restore the original spectrum That can be done by multiplying by a square pulse  2/12/04 © University of Wisconsin, CS559 Spring 2004

Reconstruction (Spatial Domain) Multiplying by a square pulse in the frequency domain is the same as convolving with a sinc function in the spatial domain  2/12/04 © University of Wisconsin, CS559 Spring 2004

Aliasing Due to Under-sampling If the sampling rate is too low, high frequencies get reconstructed as lower frequencies High frequencies from one copy get added to low frequencies from another   2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 More Aliasing Poor reconstruction also results in aliasing Consider a signal reconstructed with a box filter in the spatial domain (which means using a sinc in the frequency domain):   2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Aliasing in Practice We have two types of aliasing: Aliasing due to insufficient sampling frequency Aliasing due to poor reconstruction You have some control over reconstruction If resizing, for instance, use an approximation to the sinc function to reconstruct (instead of Bartlett, as we used last time) Gaussian is closer to sinc than Bartlett But note that sinc function goes on forever (infinite support), which is inefficient to evaluate You have some control over sampling if creating images using a computer Remove all sharp edges (high frequencies) from the scene before drawing it That is, blur character and line edges before drawing 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Compositing Compositing combines components from two or more images to make a new image The basis for film special effects (even before computers) Create digital imagery and composite it into live action Important part of animation – even hand animation Background change more slowly than foregrounds, so composite foreground elements onto constant background 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Very Simple Example = over 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Mattes A matte is an image that shows which parts of another image are foreground objects Term dates from film editing and cartoon production How would I use a matte to insert an object into a background? How are mattes usually generated for television? 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Working with Mattes To insert an object into a background Call the image of the object the source Put the background into the destination For all the source pixels, if the matte is white, copy the pixel, otherwise leave it unchanged To generate mattes: Use smart selection tools in Photoshop or similar They outline the object and convert the outline to a matte Blue Screen: Photograph/film the object in front of a blue background, then consider all the blue pixels in the image to be the background 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Alpha Basic idea: Encode opacity information in the image Add an extra channel, the alpha channel, to each image For each pixel, store R, G, B and Alpha alpha = 1 implies full opacity at a pixel alpha = 0 implies completely clear pixels There are many interpretations of alpha Is there anything in the image at that point (web graphics) Transparency (real-time OpenGL) Images are now in RGBA format, and typically 32 bits per pixel (8 bits for alpha) All images in the project are in this format 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Pre-Multiplied Alpha Instead of storing (R,G,B,), store (R,G,B,) The compositing operations in the next several slides are easier with pre-multiplied alpha To display and do color conversions, must extract RGB by dividing out  =0 is always black Some loss of precision as  gets small, but generally not a big problem 2/12/04 © University of Wisconsin, CS559 Spring 2004

Compositing Assumptions We will combine two images, f and g, to get a third composite image Not necessary that one be foreground and background Background can remain unspecified Both images are the same size and use the same color representation Multiple images can be combined in stages, operating on two at a time 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Image Decomposition The composite image can be broken into regions Parts covered by f only Parts covered by g only Parts covered by f and g Parts covered by neither f nor g Compositing operations define what should happen in each region: who (f or g) owns each region 2/12/04 © University of Wisconsin, CS559 Spring 2004

Basic Compositing Operation At each pixel, combine the pixel data from f and the pixel data from g with the equation: F and G describe how much of each input image survives, and cf and cg are pre-multiplied pixels, and all four channels are calculated To define a compositing operation, define F and G 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Sample Images Image Alpha 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Over” Operator 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Over” Operator Computes composite with the rule that f covers g 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Inside” Operator 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Inside” Operator Computes composite with the rule that only parts of f that are inside g contribute 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Outside” Operator 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Outside” Operator Computes composite with the rule that only parts of f that are outside g contribute 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Atop” Operator 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Atop” Operator Computes composite with the over rule but restricted to places where there is some g 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Xor” Operator Computes composite with the rule that f contributes where there is no g, and g contributes where there is no f 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Xor” Operator 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Clear” Operator Computes a clear composite Note that (0,0,0,>0) is a partially opaque black pixel, whereas (0,0,0,0) is fully transparent, and hence has no color 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “Set” Operator Computes composite by setting it to equal f Copies f into the composite 2/12/04 © University of Wisconsin, CS559 Spring 2004

Compositing Operations F and G describe how much of each input image survives, and cf and cg are pre-multiplied pixels, and all four channels are calculated Operation F G Over Inside Outside Atop Xor Clear Set 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Unary Operators Darken: Makes an image darker (or lighter) without affecting its opacity Dissolve: Makes an image transparent without affecting its color 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 “PLUS” Operator Computes composite by simply adding f and g, with no overlap rules Useful for defining cross-dissolve in terms of compositing: 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Obtaining  Values Hand generate (paint a grayscale image) Automatically create by segmenting an image into foreground background: Blue-screening is the analog method Remarkably complex to get right “Lasso” is the Photoshop operation With synthetic imagery, use a special background color that does not occur in the foreground Brightest blue or green is common 2/12/04 © University of Wisconsin, CS559 Spring 2004

Compositing With Depth Can store pixel “depth” instead of alpha Then, compositing can truly take into account foreground and background Generally only possible with synthetic imagery Image Based Rendering is an area of graphics that, in part, tries to composite photographs taking into account depth 2/12/04 © University of Wisconsin, CS559 Spring 2004

© University of Wisconsin, CS559 Spring 2004 Where to now… We are now almost done with images Still have to take a quick look at painterly rendering We will spend several weeks on the mechanics of 3D graphics Coordinate systems and Viewing Clipping Drawing lines and polygons Lighting and shading We will finish the semester with modeling and some additional topics 2/12/04 © University of Wisconsin, CS559 Spring 2004