Computer Vision CS 776 Spring 2014 Color Prof. Alex Berg (Slide credits to many folks on individual slides)

Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth

What is color? Color is the result of interaction between physical light in the environment and our visual system. Color is a psychological property of our visual experiences when we look at objects and lights, not a physical property of those objects or lights. -- S. Palmer, Vision Science: Photons to Phenomenology

The Physics of Light Any source of light can be completely described physically by its spectrum: the amount of energy emitted (per time unit) at each wavelength nm. © Stephen E. Palmer, 2002 Relative spectral power

Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Causes of color The sensation of color is caused by the brain. Some ways to get this sensation include: Pressure on the eyelids Dreaming, hallucinations, etc. Main way to get it is the response of the visual system to the presence/absence of light at various wavelengths. Light could be produced in different amounts at different wavelengths (compare the sun and a fluorescent light bulb). Light could be differentially reflected (e.g. some pigments). It could be differentially refracted - (e.g. Newton ’ s prism) Wavelength dependent specular reflection - e.g. shiny copper penny (actually most metals). Flourescence - light at invisible wavelengths is absorbed and reemitted at visible wavelengths.

Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Radiometry for colour All definitions are now “ per unit wavelength ” All units are now “ per unit wavelength ” All terms are now “ spectral ” Radiance becomes spectral radiance watts per square meter per steradian per unit wavelength Radiosity --- spectral radiosity

Electromagnetic spectrum Human Luminance Sensitivity Function Slide by Svetlana Lazebnik

Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Measurements of relative spectral power of sunlight, made by J. Parkkinen and P. Silfsten. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm. The color names on the horizontal axis give the color names used for monochromatic light of the corresponding wavelength --- the “colors of the rainbow”. Mnemonic is “Richard of York got blisters in Venice”. Violet Indigo Blue Green Yellow Orange Red

Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Relative spectral power of two standard illuminant models --- D65 models sunlight,and illuminant A models incandescent lamps. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm. The color names on the horizontal axis give the color names used for monochromatic light of the corresponding wavelength --- the “colors of the rainbow”. Violet Indigo Blue Green Yellow Orange Red

Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Measurements of relative spectral power of four different artificial illuminants, made by H.Sugiura. Relative spectral power is plotted against wavelength in nm. The visible range is about 400nm to 700nm.

Spectra of Light Sources Some examples of the spectra of light sources © Stephen E. Palmer, 2002 Rel. power

Reflectance Spectra of Surfaces Some examples of the reflectance spectra of surfaces Wavelength (nm) % Light Reflected Red Yellow Blue Purple © Stephen E. Palmer, 2002

Computer Vision - A Modern Approach Set: Color Slides by D.A. Forsyth Spectral albedoes for several different leaves, with color names attached. Notice that different colours typically have different spectral albedo, but that different spectral albedoes may result in the same perceived color (compare the two whites). Spectral albedoes are typically quite smooth functions. Measurements by E.Koivisto.

Interaction of light and surfaces Reflected color is the result of interaction of light source spectrum with surface reflectance Slide by Svetlana Lazebnik

Interaction of light and surfaces What is the observed color of any surface under monochromatic light? Olafur Eliasson, Room for one color

The Eye The human eye is a camera! Lens - changes shape by using ciliary muscles (to focus on objects at different distances) Pupil - the hole (aperture) whose size is controlled by the iris Iris - colored annulus with radial muscles Retina - photoreceptor cells Slide by Steve Seitz

Rods and cones, fovea Rods are responsible for intensity, cones for color perception Rods and cones are non-uniformly distributed on the retina Fovea - Small region (1 or 2°) at the center of the visual field containing the highest density of cones – and no rods Slide by Steve Seitz pigment molecules

Demonstration of visual acuity With one eye shut, at the right distance, all of these letters should appear equally legible (Glassner, 1.7). Slide by Steve Seitz

Blind spot With left eye shut, look at the cross on the left. At the right distance, the circle on the right should disappear (Glassner, 1.8). Slide by Steve Seitz

Rod / Cone sensitivity Why can’t we read in the dark? Slide by A. Efros

© Stephen E. Palmer, 2002 Three kinds of cones: Physiology of Color Vision Ratio of L to M to S cones: approx. 10:5:1 Almost no S cones in the center of the fovea

Physiology of Color Vision: Fun facts “M” and “L” pigments are encoded on the X-chromosome That’s why men are more likely to be color blind “L” gene has high variation, so some women may be tetrachromatic Some animals have one (night animals), two (e.g., dogs), four (fish, birds), five (pigeons, some reptiles/amphibians), or even 12 (mantis shrimp) types of cones Slide by D. Hoiem

Color perception Rods and cones act as filters on the spectrum To get the output of a filter, multiply its response curve by the spectrum, integrate over all wavelengths –Each cone yields one number S ML Wavelength Power How can we represent an entire spectrum with 3 numbers? We can’t! Most of the information is lost –As a result, two different spectra may appear indistinguishable »such spectra are known as metamers Slide by Steve Seitz

Metamers

Standardizing color experience We would like to understand which spectra produce the same color sensation in people under similar viewing conditions Color matching experiments Wandell, Foundations of Vision, 1995

Color matching experiment 1 Source: W. Freeman

Color matching experiment 1 p 1 p 2 p 3 Source: W. Freeman

Color matching experiment 1 p 1 p 2 p 3 Source: W. Freeman

Color matching experiment 1 p 1 p 2 p 3 The primary color amounts needed for a match Source: W. Freeman

Color matching experiment 2 Source: W. Freeman

Color matching experiment 2 p 1 p 2 p 3 Source: W. Freeman

Color matching experiment 2 p 1 p 2 p 3 Source: W. Freeman

Color matching experiment 2 p 1 p 2 p 3 We say a “negative” amount of p 2 was needed to make the match, because we added it to the test color’s side. The primary color amounts needed for a match: p 1 p 2 p 3 Source: W. Freeman

Trichromacy In color matching experiments, most people can match any given light with three primaries Primaries must be independent For the same light and same primaries, most people select the same weights Exception: color blindness Trichromatic color theory Three numbers seem to be sufficient for encoding color Dates back to 18 th century (Thomas Young) Slide by Svetlana Lazebnik

Grassman’s Laws (~linearity) Color matching appears to be linear If two test lights can be matched with the same set of weights, then they match each other: Suppose A = u 1 P 1 + u 2 P 2 + u 3 P 3 and B = u 1 P 1 + u 2 P 2 + u 3 P 3. Then A = B. If we mix two test lights, then mixing the matches will match the result: Suppose A = u 1 P 1 + u 2 P 2 + u 3 P 3 and B = v 1 P 1 + v 2 P 2 + v 3 P 3. Then A + B = (u 1 +v 1 ) P 1 + (u 2 +v 2 ) P 2 + (u 3 +v 3 ) P 3. If we scale the test light, then the matches get scaled by the same amount: Suppose A = u 1 P 1 + u 2 P 2 + u 3 P 3. Then kA = (ku 1 ) P 1 + (ku 2 ) P 2 + (ku 3 ) P 3. Slide by Svetlana Lazebnik

Linear color spaces Defined by a choice of three primaries The coordinates of a color are given by the weights of the primaries used to match it mixing two lights produces colors that lie along a straight line in color space mixing three lights produces colors that lie within the triangle they define in color space Slide by Svetlana Lazebnik

Linear color spaces How to compute the weights of the primaries to match any spectral signal? p 1 p 2 p 3 ? Given: a choice of three primaries and a target color signal Find: weights of the primaries needed to match the color signal p 1 p 2 p 3 Slide by Svetlana Lazebnik

Linear color spaces In addition to primaries, need to specify matching functions: the amount of each primary needed to match a monochromatic light source at each wavelength RGB matching functions RGB primaries Slide by Svetlana Lazebnik

Linear color spaces How to compute the weights of the primaries to match any spectral signal? Let c(λ) be one of the matching functions, and let t(λ) be the spectrum of the signal. Then the weight of the corresponding primary needed to match t is λ Matching functions, c(λ) Signal to be matched, t(λ) Slide by Svetlana Lazebnik

RGB space Primaries are monochromatic lights (for monitors, they correspond to the three types of phosphors) Subtractive matching required for some wavelengths RGB matching functions RGB primaries Slide by Svetlana Lazebnik

Comparison of RGB matching functions with best 3x3 transformation of cone responses Wandell, Foundations of Vision, 1995 Slide by Svetlana Lazebnik

Linear color spaces: CIE XYZ Primaries are imaginary, but matching functions are everywhere positive The Y parameter corresponds to brightness or luminance of a color 2D visualization: draw (x,y), where x = X/(X+Y+Z), y = Y/(X+Y+Z) Matching functions Slide by Svetlana Lazebnik

Uniform color spaces Unfortunately, differences in x,y coordinates do not reflect perceptual color differences CIE u’v’ is a projective transform of x,y to make the ellipses more uniform McAdam ellipses: Just noticeable differences in color Slide by Svetlana Lazebnik

Nonlinear color spaces: HSV Perceptually meaningful dimensions: Hue, Saturation, Value (Intensity) RGB cube on its vertex Slide by Svetlana Lazebnik

Color perception Color/lightness constancy The ability of the human visual system to perceive the intrinsic reflectance properties of the surfaces despite changes in illumination conditions Instantaneous effects Simultaneous contrast Mach bands Gradual effects Light/dark adaptation Chromatic adaptation Afterimages J. S. Sargent, The Daughters of Edward D. Boit, 1882 Slide by Svetlana Lazebnik

Checker shadow illusion

Checker shadow illusion Possible explanations Simultaneous contrast Reflectance edges vs. illumination edges

Color Constancy by “Gamut” Approach… Limited range of material reflectance properties Limited range of possible illuminants Given observed colors we can work out possible range of illuminants, combine those with some prior over known illuminants and obtain

More reading & thought problems David Forsyth’s “Gamut” based reasoning for color constancy (IJCV 1990) I am very interested in pursing the line of reasoning I mentioned in class, where the above ideas for color constancy are extended to texture recognition.