1. Environment Matting and Compositing CS 395 – Advanced GraphicsPresented by Jay Hetler Douglas E. Zongker ~ Dawn M. Werner ~ Brian Curless ~ David H. Salsin

2. What is Matting? Extracting the foreground image from the background Describing the opacity of the foreground at every point What is Compositing? Holding back the part of the new background that the foreground blocks. Placing the foreground element in a new background

3. Original Uses Weather ReportsCinema Special Effects

4. Matting Equation C = F + (1-  )B C ~ Color F ~ Foreground color B ~ Background color  ~ Amount of light that passes through the foreground

5. How difficult is Matting?

6. Matte Image

7. Good Results?

8. Problems with Matting Images do not look realistic Lack of Refracted Light Lack of Reflected Light Solution: Modify the Matting Equation

9. Environment Matting Equation C = F + (1-  )B +  C ~ Color F ~ Foreground color B ~ Background color  ~ Amount of light that passes through the foreground  ~ Contribution of light from Environment that travels through the object

10. Explanation of   = R M(T, A )  R ~ Reflectance coefficient M ~ Texture mapping operator for axis-aligned rectangle (A) of texture (T) i = 1 i i i m

11. Environment Mattes

12. Performance Calibration Matting: 10-20 minutes extraction time for each texture map (Pentium II 400Mhz) Compositing: 4-40 frames per second Real-Time?

13. How much better is Environment Matting? Alpha Matte Environment Matte Photograph

14. How much better is Environment Matting? Alpha Matte Environment Matte Photograph

15. Movies!

16. Importance of Sidedrops Light comes from other parts of the environment

17. Sidedrop Example

18. Sidedrop x2

19. Failure Cases Box-filter Gaussian-weighted Photograph

20. Failure Cases Composite Photograph Failure of Engineering Failure of Theory

21. Next Steps Depth Accuracy Real-Time Capture

22. Progress? Depth Accuracy No Real-Time