1. A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Color and Reflectance in Imaging and Computer Vision Workshop 2009 October 4, 2009 Christian Riess Johannes Jordan Prof. Elli Angelopoulou Pattern Recognition Lab (CS 5) University of Erlangen-Nuremberg

2. Seite 2 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 2 Natural Images: Shadows, Interreflections Typical assumption in Color Constancy and Illuminant Color Estimation: One single illuminant. Consider a challenging image: Multiple light sources, shadows, inter- reflections. Maybe we need a more complex model?

3. Seite 3 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 3 Lambertian Reflectance: Ideal Diffuse Reflectance Dichromatic Reflectance Model [1]: Specular and Diffuse Common Reflectance Models [1] S. Shafer. Using Color to Separate Reflection Components. Color Research Application, pp. 210-218, 1985. Wave length Directions of incidence on surface and exitance towards viewer Surface radiance Geometric terms Surface reflectance (albedo) Direct illuminant

4. Seite 4 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 4 Bi-Illuminant Dichromatic Reflection Model [1] [1] B. Maxwell, R. Friedhoff, and C. Smith. A Bi-Illuminant Dichromatic Reflection Model for Understanding Images. Computer Vision and Pattern Recognition, pp. 1-8, 2008. Wave length Directions of incidence on surface and exitance towards viewer Surface radiance Geometric terms Surface reflectance (albedo) Direct illuminant Diffuse illuminant Direct light, specular refl.Direct light, diffuse refl. Ambient light, specular refl. Ambient light, diffuse refl.

5. Seite 5 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 5 The Ambient Term The BIDR contains a very flexible catch all ambient term. For practical exploitation, further constraints have to be made These constraints can form a taxonomy of illumination-related techniques

6. Seite 6 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 6 BIDR as a General Framework (I) [1] K. Barnard and G. Finlayson. Shadow Identification using Colour Ratios. IS&T/SID 8 th Colour Imaging Conference: Colour Science, Systems and Applications, pp. 97-101, 2000. Shadow Boundaries: Learning illuminant ratios [1] Channelwise relative attenuation of the direct illuminant Isotropic ambient light Weighting factor between direct and ambient light Exclude specularities Make ambient light isotropic Split illuminant into chromatic and energy part Attenuate direct illuminant

7. Seite 7 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 7 BIDR as a General Framework (II) [1] C. Lu and M. Drew. Practical Scene Illuminant Estimation via Flash/No-Flash Pairs. IS&T/SID 14 th Colour Imaging Conference: Colour Science, Systems and Applications, 2006. Scene Illuminant Estimation with Flash/No-Flash Images [1] Consider normal illumination as ambient light, flash is direct light Knowledge of the flash light characteristics resolves the scene parameters Flash image: Non-Flash image: Difference of log-chromaticities recovers average ambient light Exclude specularities Assume Planckian illuminant, flash light is known

8. Seite 8 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 8 Experiments: Setup Is the consideration of such a model justified at all? -> only if ambient illumination is complicated enough to disturb our analysis Chalk (sort-of Lambertian), direct light, ambient light and a combination of direct and ambient light. Ambient illuminationDirect and ambient illumination Direct illumination

9. Seite 9 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 9 Dichromatic Reflectance under laboratory conditions With additional (more or less uniform) ambient lighting, we would expect the line to be rotated and/or shifted: Experiments: Expectations Diffuse Line Specular triangle RGB Cube Diffuse Line Specular triangle RGB Cube

10. Seite 10 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 10 Experiments: Results Diffuse and ambient pixels almost linearly distributed in RGB space:

11. Seite 11 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 11 Experiments: Results Red Green Blue Chalk: regions of interest Characteristic spots on chalk pixels Per-pixel contribution of each illuminant

12. Seite 12 C. Riess October 4, 2009A Common Framework for Ambient Illumination in the Dichromatic Reflectance Model Seite 12 Conclusion and Outlook Dichromatic Reflection Model is limited to a single light source, making several real-world images difficult to analyze Outdoor scenes (Partially) shadowed areas Natural extension: Bi-Illuminant Dichromatic Reflectance Model A common framework for modelling shadows, interreflections or a second direct illuminant Experiments on real-world images demonstrate the need for more complex assumptions Outlook: Develop new illumination estimation algorithms based on less constraining assumptions