1. Computer Vision CS 776 Spring 2014 Color Prof. Alex Berg (Slide credits to many folks on individual slides)

4. Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth

5. What is color? Color is the result of interaction between physical light in the environment and our visual system. Color is a psychological property of our visual experiences when we look at objects and lights, not a physical property of those objects or lights. -- S. Palmer, Vision Science: Photons to Phenomenology

6. The Physics of Light Any source of light can be completely described physically by its spectrum: the amount of energy emitted (per time unit) at each wavelength 400 - 700 nm. © Stephen E. Palmer, 2002 Relative spectral power

7. Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Causes of color The sensation of color is caused by the brain. Some ways to get this sensation include: Pressure on the eyelids Dreaming, hallucinations, etc. Main way to get it is the response of the visual system to the presence/absence of light at various wavelengths. Light could be produced in different amounts at different wavelengths (compare the sun and a fluorescent light bulb). Light could be differentially reflected (e.g. some pigments). It could be differentially refracted - (e.g. Newton ’ s prism) Wavelength dependent specular reflection - e.g. shiny copper penny (actually most metals). Flourescence - light at invisible wavelengths is absorbed and reemitted at visible wavelengths.

8. Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Radiometry for colour All definitions are now “ per unit wavelength ” All units are now “ per unit wavelength ” All terms are now “ spectral ” Radiance becomes spectral radiance watts per square meter per steradian per unit wavelength Radiosity --- spectral radiosity

9. Electromagnetic spectrum Human Luminance Sensitivity Function Slide by Svetlana Lazebnik

10. Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Measurements of relative spectral power of sunlight, made by J. Parkkinen and P. Silfsten. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm. The color names on the horizontal axis give the color names used for monochromatic light of the corresponding wavelength --- the “colors of the rainbow”. Mnemonic is “Richard of York got blisters in Venice”. Violet Indigo Blue Green Yellow Orange Red

11. Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Relative spectral power of two standard illuminant models --- D65 models sunlight,and illuminant A models incandescent lamps. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm. The color names on the horizontal axis give the color names used for monochromatic light of the corresponding wavelength --- the “colors of the rainbow”. Violet Indigo Blue Green Yellow Orange Red

12. Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Measurements of relative spectral power of four different artificial illuminants, made by H.Sugiura. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm.

13. Spectra of Light Sources Some examples of the spectra of light sources © Stephen E. Palmer, 2002 Rel. power

14. Reflectance Spectra of Surfaces Some examples of the reflectance spectra of surfaces Wavelength (nm) % Light Reflected Red 400 700 Yellow 400 700 Blue 400 700 Purple 400 700 © Stephen E. Palmer, 2002

15. Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Spectral albedoes for several different leaves, with color names attached. Notice that different colours typically have different spectral albedo, but that different spectral albedoes may result in the same perceived color (compare the two whites). Spectral albedoes are typically quite smooth functions. Measurements by E.Koivisto.

16. Interaction of light and surfaces Reflected color is the result of interaction of light source spectrum with surface reflectance Slide by Svetlana Lazebnik

17. Interaction of light and surfaces What is the observed color of any surface under monochromatic light? Olafur Eliasson, Room for one color

18. The Eye The human eye is a camera! Lens - changes shape by using ciliary muscles (to focus on objects at different distances) Pupil - the hole (aperture) whose size is controlled by the iris Iris - colored annulus with radial muscles Retina - photoreceptor cells Slide by Steve Seitz

19. Rods and cones, fovea Rods are responsible for intensity, cones for color perception Rods and cones are non-uniformly distributed on the retina Fovea - Small region (1 or 2°) at the center of the visual field containing the highest density of cones – and no rods Slide by Steve Seitz pigment molecules

20. Demonstration of visual acuity With one eye shut, at the right distance, all of these letters should appear equally legible (Glassner, 1.7). Slide by Steve Seitz

21. Blind spot With left eye shut, look at the cross on the left. At the right distance, the circle on the right should disappear (Glassner, 1.8). Slide by Steve Seitz

22. Rod / Cone sensitivity Why can’t we read in the dark? Slide by A. Efros

23. © Stephen E. Palmer, 2002 Three kinds of cones: Physiology of Color Vision Ratio of L to M to S cones: approx. 10:5:1 Almost no S cones in the center of the fovea

24. Physiology of Color Vision: Fun facts “M” and “L” pigments are encoded on the X-chromosome That’s why men are more likely to be color blind http://www.vischeck.com/vischeck/vischeckURL.php http://www.vischeck.com/vischeck/vischeckURL.php “L” gene has high variation, so some women may be tetrachromatic Some animals have one (night animals), two (e.g., dogs), four (fish, birds), five (pigeons, some reptiles/amphibians), or even 12 (mantis shrimp) types of cones http://www.mezzmer.com/blog/how-animals-see-the-world/ http://en.wikipedia.org/wiki/Color_vision Slide by D. Hoiem

25. Color perception Rods and cones act as filters on the spectrum To get the output of a filter, multiply its response curve by the spectrum, integrate over all wavelengths –Each cone yields one number S ML Wavelength Power How can we represent an entire spectrum with 3 numbers? We can’t! Most of the information is lost –As a result, two different spectra may appear indistinguishable »such spectra are known as metamers Slide by Steve Seitz

26. Metamers

27. Standardizing color experience We would like to understand which spectra produce the same color sensation in people under similar viewing conditions Color matching experiments Wandell, Foundations of Vision, 1995

28. Color matching experiment 1 Source: W. Freeman

29. Color matching experiment 1 p 1 p 2 p 3 Source: W. Freeman

30. Color matching experiment 1 p 1 p 2 p 3 Source: W. Freeman

31. Color matching experiment 1 p 1 p 2 p 3 The primary color amounts needed for a match Source: W. Freeman

32. Color matching experiment 2 Source: W. Freeman

33. Color matching experiment 2 p 1 p 2 p 3 Source: W. Freeman

34. Color matching experiment 2 p 1 p 2 p 3 Source: W. Freeman

35. Color matching experiment 2 p 1 p 2 p 3 We say a “negative” amount of p 2 was needed to make the match, because we added it to the test color’s side. The primary color amounts needed for a match: p 1 p 2 p 3 Source: W. Freeman

36. Trichromacy In color matching experiments, most people can match any given light with three primaries Primaries must be independent For the same light and same primaries, most people select the same weights Exception: color blindness Trichromatic color theory Three numbers seem to be sufficient for encoding color Dates back to 18 th century (Thomas Young) Slide by Svetlana Lazebnik

37. Grassman’s Laws (~linearity) Color matching appears to be linear If two test lights can be matched with the same set of weights, then they match each other: Suppose A = u 1 P 1 + u 2 P 2 + u 3 P 3 and B = u 1 P 1 + u 2 P 2 + u 3 P 3. Then A = B. If we mix two test lights, then mixing the matches will match the result: Suppose A = u 1 P 1 + u 2 P 2 + u 3 P 3 and B = v 1 P 1 + v 2 P 2 + v 3 P 3. Then A + B = (u 1 +v 1 ) P 1 + (u 2 +v 2 ) P 2 + (u 3 +v 3 ) P 3. If we scale the test light, then the matches get scaled by the same amount: Suppose A = u 1 P 1 + u 2 P 2 + u 3 P 3. Then kA = (ku 1 ) P 1 + (ku 2 ) P 2 + (ku 3 ) P 3. Slide by Svetlana Lazebnik

38. Linear color spaces Defined by a choice of three primaries The coordinates of a color are given by the weights of the primaries used to match it mixing two lights produces colors that lie along a straight line in color space mixing three lights produces colors that lie within the triangle they define in color space Slide by Svetlana Lazebnik

39. Linear color spaces How to compute the weights of the primaries to match any spectral signal? p 1 p 2 p 3 ? Given: a choice of three primaries and a target color signal Find: weights of the primaries needed to match the color signal p 1 p 2 p 3 Slide by Svetlana Lazebnik

40. Linear color spaces In addition to primaries, need to specify matching functions: the amount of each primary needed to match a monochromatic light source at each wavelength RGB matching functions RGB primaries Slide by Svetlana Lazebnik

41. Linear color spaces How to compute the weights of the primaries to match any spectral signal? Let c(λ) be one of the matching functions, and let t(λ) be the spectrum of the signal. Then the weight of the corresponding primary needed to match t is λ Matching functions, c(λ) Signal to be matched, t(λ) Slide by Svetlana Lazebnik

42. RGB space Primaries are monochromatic lights (for monitors, they correspond to the three types of phosphors) Subtractive matching required for some wavelengths RGB matching functions RGB primaries Slide by Svetlana Lazebnik

43. Comparison of RGB matching functions with best 3x3 transformation of cone responses Wandell, Foundations of Vision, 1995 Slide by Svetlana Lazebnik

44. Linear color spaces: CIE XYZ Primaries are imaginary, but matching functions are everywhere positive The Y parameter corresponds to brightness or luminance of a color 2D visualization: draw (x,y), where x = X/(X+Y+Z), y = Y/(X+Y+Z) Matching functions http://en.wikipedia.org/wiki/CIE_1931_color_space Slide by Svetlana Lazebnik

45. Uniform color spaces Unfortunately, differences in x,y coordinates do not reflect perceptual color differences CIE u’v’ is a projective transform of x,y to make the ellipses more uniform McAdam ellipses: Just noticeable differences in color Slide by Svetlana Lazebnik

46. Nonlinear color spaces: HSV Perceptually meaningful dimensions: Hue, Saturation, Value (Intensity) RGB cube on its vertex Slide by Svetlana Lazebnik

47. Color perception Color/lightness constancy The ability of the human visual system to perceive the intrinsic reflectance properties of the surfaces despite changes in illumination conditions Instantaneous effects Simultaneous contrast Mach bands Gradual effects Light/dark adaptation Chromatic adaptation Afterimages J. S. Sargent, The Daughters of Edward D. Boit, 1882 Slide by Svetlana Lazebnik

48. Checker shadow illusion http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html

49. Checker shadow illusion Possible explanations Simultaneous contrast Reflectance edges vs. illumination edges http://web.mit.edu/persci/people/adelson/checkershadow_illusion.html

50. Color Constancy by “Gamut” Approach… Limited range of material reflectance properties Limited range of possible illuminants Given observed colors we can work out possible range of illuminants, combine those with some prior over known illuminants and obtain

51. More reading & thought problems David Forsyth’s “Gamut” based reasoning for color constancy (IJCV 1990) http://luthuli.cs.uiuc.edu/~daf/papers/colorconst.pdf I am very interested in pursing the line of reasoning I mentioned in class, where the above ideas for color constancy are extended to texture recognition.