Environment Matting and Compositing CS 395 – Advanced GraphicsPresented by Jay Hetler Douglas E. Zongker ~ Dawn M. Werner ~ Brian Curless ~ David H. Salsin
What is Matting? Extracting the foreground image from the background Describing the opacity of the foreground at every point What is Compositing? Holding back the part of the new background that the foreground blocks. Placing the foreground element in a new background
Original Uses Weather ReportsCinema Special Effects
Matting Equation C = F + (1- )B C ~ Color F ~ Foreground color B ~ Background color ~ Amount of light that passes through the foreground
How difficult is Matting?
Matte Image
Good Results?
Problems with Matting Images do not look realistic Lack of Refracted Light Lack of Reflected Light Solution: Modify the Matting Equation
Environment Matting Equation C = F + (1- )B + C ~ Color F ~ Foreground color B ~ Background color ~ Amount of light that passes through the foreground ~ Contribution of light from Environment that travels through the object
Explanation of = R M(T, A ) R ~ Reflectance coefficient M ~ Texture mapping operator for axis-aligned rectangle (A) of texture (T) i = 1 i i i m
Environment Mattes
Performance Calibration Matting: minutes extraction time for each texture map (Pentium II 400Mhz) Compositing: 4-40 frames per second Real-Time?
How much better is Environment Matting? Alpha Matte Environment Matte Photograph
How much better is Environment Matting? Alpha Matte Environment Matte Photograph
Movies!
Importance of Sidedrops Light comes from other parts of the environment
Sidedrop Example
Sidedrop x2
Failure Cases Box-filter Gaussian-weighted Photograph
Failure Cases Composite Photograph Failure of Engineering Failure of Theory
Next Steps Depth Accuracy Real-Time Capture
Progress? Depth Accuracy No Real-Time