1. Retinex Theory Psych221 Final Project Mike Jahr March 16, 2000

2. Color Constancy Color depends on wavelength But, objects reflect different wavelengths under different lighting conditions. –Banana in daylight, fluorescent light, no light... To us, they seem to retain their color.

3. How is this possible? There is more to color than wavelength. The visual system must somehow “discount the illuminant”

4. A Juicy Burger

5. A Closer Look...

6. What’s going on? It’s not very saturated, but the red burger has browns, greens, tans… How can we see these colors in only red and white light?

7. Enter Edwin H. Land Land was the founder of Polaroid; interested in color While running Maxwell’s experiments (3 color projectors), he noticed this It spawned decades of experiments

8. The Mondrian Apparatus Land set up 3 filtered light sources (LMS) Can calibrate each one; precisely control light Telescopic photometer Actually closer to a Van Doesburg...

9. Mondrian Experiments Measure reflectance from a green patch Calibrate lights so that a blue patch reflects an identical spectrum It still looks blue!

10. More Mondrian Calibrate lights for even reflectance from the green patch Cover all other patches; looks gray Uncover all patches; looks green

11. Land’s Conclusions Perceived color depends on reflected spectrum, but also on surroundings Relative reflectance is more important than absolute reflectance

12. Discount the Illuminant: Retinex “A framework for computing perceived colors on the basis of the relative intensities of three wavelengths and their spectral interactions.” Processed in retina or cortex? Retinex!

13. Principles of Retinex Process each receptor class independently Objective is to calculate illuminant- independent “lightness” values Lightness values represent perceived color

14. The Algorithm Pick a starting pixel x 1, then form a path by randomly selecting neighboring pixels Update an accumulator at each pixel: Threshold step: if difference is small, use previous sensor response

15. The Algorithm II Keep a counter N(x) for each pixel After a number of paths, normalize A(x) by N(x) for each pixel Result is L(x), the lightness value Algorithm has two parameters: –number of paths, length of each path

16. What is Lightness? Should not depend on viewing conditions Should only depend on surface properties Results in a triplet that is tough to interpret –The retinex color space Issue: what to do with it?

17. My Implementation 1Convert image from RGB to LMS via phosphor spectra and cone sensitivities 2Run algorithm to get lightness values 3Do something with lightness values?? B&W implementation

18. Retinex Variants McCann et al. –Retinex with reset Horn –Determining lightness from an image Marini –Retinex with Brownian motion

19. Illusions under Retinex Original imageProcessed image

20. More Illusions Original imageRetinex image

21. Biological Basis Some monkey neurons respond to colors, not wavelengths –Cortical area V4 in prestriate cortex Even goldfish can discount the illuminant

22. Problems with Retinex Too dependent on composition of surfaces in image Higher-order processes influence color

23. Conclusion Retinex is a long-lived theory, has sparked much debate and many imitators Although not a generally accurate model of human vision, it does perform well in some situations

24. Appendix Source files, sample images, sample output, etc. can be found in src/ along with brief explanations of each.src/

25. References E. H. Land, “Recent advances in retinex theory and some implications for cortical applications: Color vision and the natural image,” Proc. Nat. Acad. Sci. USA 80, 5163–5169 (1983). E. H. Land, “Recent advances in retinex theory,” Vision Res. 26, 7–22 (1986). B. K. P. Horn, “Determining lightness from an image,” Comp. Graphics Image Process. 3, 277–299 (1974). D. H. Brainard and B. A. Wandell, “Analysis of the retinex theory of color vision,” J. Opt. Soc. Am. 3, 1651–1661 (1986). J. J. McCann, “Lessons learned from Mondrians applied to real images and color gamuts,” IS&T Rep. 14, 6 (1999). http://www.imaging.org/pubs/reporter/articles/14_6_mccann/index.html http://www.imaging.org/pubs/reporter/articles/14_6_mccann/index.html

26. References E. H. Adelsen, “Lightness perception and lightness illusions,” in M. Gazzaniga, M.S., Ed., The Cognitive Neurosciences, Cambridge, MA: MIT Press, pp. 339-351 (1999). http://www-bcs.mit.edu/people/adelson/publications/gazzan.dir/gazzan.htm http://www-bcs.mit.edu/people/adelson/publications/gazzan.dir/gazzan.htm F.W. Campbell, F.R.S., “Dr. Edwin H. Land,” Biographical Memoirs of Fellows of the Royal Society, 40, 195-219 (1994). http://www.rowland.org/land/land.html http://www.rowland.org/land/land.html D. Marini and L. Marini, “Measuring the colours we receive,” Science Tribune, October (1997). http://www.tribunes.com/tribune/art97/mari.htmhttp://www.tribunes.com/tribune/art97/mari.htm