1. Image-Based Rendering from a Single Image Kim Sang Hoon Samuel Boivin – Andre Gagalowicz INRIA

2. Introduction To recover an approximation of BRDF of surface from a single image. (including specular, isotropic or anisotropic surfaces) Hierarchical, interactive technique using error between the rendered image and the real image.

3. Background Camera parameter (3D geometrical model, 2D image) [DeMenthon and Davis] Model-based object pose in 25 line of code – ECCV 92 BRDF Model: [Ward] Measuring and modeling anisotropic reflection – SIGGRAPH 92 Radiance map: [Tumblin and Rushmeier] Tone reproduction for realistic image – IEEE Computer Graphics and Applications November 1993. How to find light pose from a single image

4. Note: Images record color, Radiance maps record brightness

5. Background and Related Work 1. Reflectance Recovery using a Specific Device. - Estimate the five parameters of anisotropic BRDF model. [Ward] Measuring and modeling anisotropic reflectance – SIGGRAPH 92 2. Reflectance Recovery from Several Images. - Method without Global Illumination. - Method with Global Illumination. 3. Reflectance Recovery from a Single Image. - Method without Global Illumination. - Method with Global Illumination. - Radiosity-based Algorithm

6. Elements of Reflectance Recovery Notion of Group - Input : 3D geometrical model, a single image - Extraction of the object reflectance from the pixel. (by the projection of these objects in the image) - Problems (using a single image) -A lot of surfaces are not visible. - Notion of Group - The object and the surface have a same reflectance property - Manual operation (Geometrical modeling process)

7. Elements of Reflectance Recovery Reflectance Model and Data Description -Image-Based Modeling (Alias | Wavefront’s Maya modeler) -Camera parameters [DeMenthon and Davis] Model-based object pose in 25 line of code – ECCV 92 -Photometric recovery method [Ward] Measuring and modeling anisotropic reflection – SIGGRAPH 92 Five parameters for a complex BRDF: Diffuse ρ d, Specular ρ d, anisoptropic direction ( x ), anisotropic roughness ( a x, a y )

8. Elements of Reflectance Recovery 3D Geometrical Modeling Geometrical model - object, camera, light sources poses Phothmetric model - reflectance, light source intensity  Synthetic image using a classical rendering Software (Phoenix – global illumination software)

9. Overview of the Algorithm

10. Inverse Rendering from a Single Image Case of perfect diffuse surface -Diffuse reflectance : the average of radiances covered by the projection of the group in the original image. -Textured surface (using a pure diffuse) : Create a good visual approximation.

11. Inverse Rendering from a Single Image Case of perfect diffuse surface Error between the original and the rendered image Where, B : average radiance P : pixels covered by the projection of object j in the original image. T( ) is camera transfer function ( - correction function) Camera transfer function : To convert light input into electrical (analog or digital) signals.

12. Inverse Rendering from a Single Image Diffuse reflectance ρ d of object j is proportional to the average radiance B The function f () eliminates problems by smaller object. Textures are not take into account – only consider a diffuse reflectance parameter ρ d. The radiances, the emittances and the full geometry (form factors)  Solve radiosity equation for the reflectance.

13. Inverse Rendering from a Single Image Case of perfect and non-perfect specular surface Diffuse hypothesis failed  considered as a perfect mirror. Perfect specular surface -The easiest case to solve ( ρ d =0, ρ s = 1 ), Need not iteration Non-perfect specular surface - Require iteration to obtain an optimum ρ s

14. Inverse Rendering from a Single Image Case of both diffuse and specular surfaces with no roughness

15. Inverse Rendering from a Single Image Case of isotropic surfaces Recover Diffuse (ρ d ), Specular (ρ d ) and roughness (a) using Ward’s BRDF model. Case of anisotropic surfaces -Most complicate case -Anisotropic model of Ward requires to minimize a function of 5 parameters. (Diffuse ρd, Specular ρd, anisoptropic direction (x), anisotropic roughness (ax, ay)

16. Inverse Rendering from a Single Image Case of textured surfaces - Extracting the texture from image is an easy task [Wolberg] Digital Image Warping – IEEE Computer Society Press - Consider that it already has received the energy from the light source.  Otherwise, over-illuminated. - Radiosity texture : balances the extracted texture with an intermediate texture in order to minimize the error (the real and synthetic image) - Case of perfect diffuse surface Texture is computed by an iterative method. - At the first, extract from the real image. - Synthetic image - Multiplied by the ratio (newly texture of synthetic / texture of real image)

17. Inverse Rendering from a Single Image Case of textured surfaces -The problems - A texture including the shadows, the specular reflection and the highlight. - It is extremely hard to solve using a single image.

18. Results

20. Future Works [Debevec] Efficient View-Dependent Image-Based Rendering with Projective Texture-Mapping – EGRW 98 [Debevec] Rendering Synthetic Objects into Real Scenes - SIGGRAPH 98 [Debevec] Modeling and Rendering Architecture from Photographs –SIGGRAPH96 [Ward] Measuring and modeling anisotropic reflection – SIGGRAPH 92 Fixed Camera – Obtain multiple images (Different exposure, light poses) 1.Assume perfect diffuse surface – extract texture (iterative method) –Render image. 2.Find radiance map – Estimate BRDF – Render image.