1. Rendering Synthetic Objects into Real Scenes: Bridging Traditional and Image-based Graphics with Global Illumination and High Dynamic Range Photography Paul Debevec, University of California at Berkley

2. Motivation Visual Effects Visual Effects –Adding synthetic actors/props Architectural simulation Architectural simulation –Adding buildings to landscapes

3. Related Work Previous methods used to insert synthetic objects into real scenes Previous methods used to insert synthetic objects into real scenes –Drawing –Environment mapping –Approximate geometric model and lighting Global illumination algorithms Global illumination algorithms Recovery of HDR radiance maps from photographs Recovery of HDR radiance maps from photographs

4. Related Work High Dynamic Range refers to the range of brightness values present in a scene High Dynamic Range refers to the range of brightness values present in a scene Eyes, photos, displays, all capture a limited, non-linear range Eyes, photos, displays, all capture a limited, non-linear range Everyday brightness range (about 10 12 :1) 1000:1100:1 Source: Microsoft Developers Network

5. Related Work  Why is this a problem?  Limited brightness range results in lost scene detail from saturation and/or underexposure  OK for traditional image use Photos from OpenEXR by LucasFilm, LTD. http://www.openexr.com/samples.html

6. Related Work What about image-based lighting? What about image-based lighting? –Photo pixel values are not representative of relative Radiance in the scene Images by Paul Debevec, 98

7. Related Work Solution: Take multiple photos with varying shutter speeds and ‘combine’ somehow to create accurate relative radiance maps Solution: Take multiple photos with varying shutter speeds and ‘combine’ somehow to create accurate relative radiance maps Images by Paul Debevec, 98

8. Related Work Is HDR really important? Is HDR really important?

9. Method Divide the scene into three parts Divide the scene into three parts

10. Method Distant Scene Distant Scene –Light-based model –Required to provide correct incident illumination to local/synthetic objects from desired viewpoint –Assumed that no light from the model will affect the distant scene –Any level of geometric detail wanted

11. Method Local Scene Local Scene –Material-based model (has BRDF) –Must approximate the real scene local to the position of the synthetic objects –‘Local’ means photometrically local –‘Approximate’ means geometrically AND photometrically representative of the real scene

12. Method Synthetic Objects Synthetic Objects –Complete materials-based models of elements that are to be inserted into the scene –Geometric and lighting information –Any shape and material supported by the global illumination method that you plan to use

13. Method Compositing using a light probe Compositing using a light probe –One method to create the distant scene from a real location –Uses a metal sphere and a camera to gather light from a point

14. Method Compositing using a light probe Compositing using a light probe –This light is mapped onto the distant scene geometry and used to calculate the lighting –Lighting is calculated with global illumination

15. Method Results Results

16. Method Problem with the current definition of the local scene Problem with the current definition of the local scene –We said that it must approximate the material of the real scene –It can be really hard to get the BRDF/material of a real object! –Can we modify the local scene so it doesn’t require this?

17. Method Local Scene 2 (Differential Rendering) Local Scene 2 (Differential Rendering) –Approximate the material of the scene to within reason (not exact) –Render with and without your synthetic objects and calculate the difference –Add that difference back into the original image

18. Method Difference between local scene render with and without synthetic objects Difference between local scene render with and without synthetic objects - =

19. Results

20. Conclusion This paper presented a framework for rendering synthetic objects into real world environments with real lighting This paper presented a framework for rendering synthetic objects into real world environments with real lighting –High Dynamic Range radiance maps –Global Illumination –Distant and local scenes, synthetic objects –Differential rendering

21. Future Work Method to generate boundary between distant and local scene Method to generate boundary between distant and local scene More efficient global illumination More efficient global illumination

22. Comments Pros Pros –Open framework –Good results Cons Cons –Hard to decide what local scene should be! –Local scene can be very complicated