Computer Vision Chapter 6 Color

Visible light Color Wavelength violet 380–450 nm blue 450–495 nm green 495–570 nm yellow 570–590 nm orange 590–620 nm red 620–750 nm (high frequency/shorter wavelength)

White light So what is “white” light? White light is composed of approx. equal energy in all wavelengths of the visible spectrum.

Perception of color A blue surface appears blue when illuminated by white light. The same surface appears violet when illuminated by red light.

Perception of color Dependent on: Spectrum of light source Characteristics of illuminated object Spectral sensitivity of the sensor

Perception of color Additional factors: Surface is specular or matte (lambertian) Distance of surface from light Orientation of surface and light

Surface normals Say we have 3 points, p1, p2, and p3 on our surface element. Form two vectors, U=p2-p1 V=p3-p1. Then N=UxV is the surface normal (cross product). Nx = UyVz – UzVy Ny = UzVx – UxVz Nz = UxVy – UyVx Analog in 2D is UxVy – UyVx Note regarding length of N; note regarding direction of N.

Sensitivity of receptors

Types of images Binary image = range of I is in {0,1} Gray-scale image = range of I is in {0,1,…,max} Multispectral image = vector-valued Ex. Color image = range is in {<r,g,b> | (r,g,b) in {0,1,…,max} }, for 8 bits, max=255 (28-1) <0,0,255> = ? <255,0,0> = ? <100,100,100> = ?

Representing color RGB (red green blue) CMY (cyan magenta yellow) Additive (to black: <0,0,0>) CMY (cyan magenta yellow) Subtractive (from white: <0,0,0>); based on printing inks on white paper Often CMYK (K=key=black) because black from CMY is poor. HSI (hue saturation intensity) aka HSV (hue saturation value)

Additive (left) vs. subtractive (right)

Color cube (note RGB values in [0.0..1.0])

color cube vs. color wheel

Interactive: http://www.had2know.com/technology/hsi-rgb-color-converter-equations.html Hue angle in [0..2Pi]. Saturation in [0..1]. Intensity in [0..255].

Other systems Luminance + 2 chromaticity values YIQ luminance red-cyan Y = 0.3 R + 0.59 G + 0.11 B red-cyan I = 0.6 R – 0.28 G – 0.32 B magenta-green Q = 0.21 R – 0.52 G + 0.31 B Used by color and black & white broadcast TV (before digital).

Other systems Luminance + 2 chromaticity values YUV Y = 0.3 R + 0.59 G + 0.11 B U = 0.493 (B – Y) V = 0.877 (R – Y) Used by JPEG and MPEG (compresses better than RGB)

Color histograms

Color histograms How can one construct a color histogram?

Color histograms How can one construct a color histogram? as 3D color clouds

Color histograms How can one construct a color histogram? as histograms of each individual component

Color histograms Another method for constructing a color histogram: Given an RGB value, construct an integer i = R2G2B2 where R2 = the 2 high order bits of the R component (and similarly for G and B) 222222 = 26 = 64 bins in histogram Maps 3D RGB values into 1D.

64 bin color histogram example

64 bin color histogram example

Color histograms Another method of constructing a color histogram: Quantize RGB into 16 levels (4 bits for each component, 24 = 16) Create a histogram of red values Create a histogram of green values Create a histogram of blue values Concatenate these 3 histograms into one single 16x3 = 48 bin histogram Also maps 3D RGB values into 1D.

Comparing histograms

Comparing color histograms Say we have a database of images and their color histograms. Given an image that is not in our database, we can calculate its histogram. And see what histogram it best matches. But how can we compare histograms?

Method 1: Comparing two (1D) histograms hI and hM Note: match is not symmetric. I.e., match(a,b) <> match(b,a) for all a and b.

Method 2: Comparing two (1D) histograms hI and hM Normalize (divide) each hI and hM by total number of pixels (to approximate probabilities). Calculate distance d( hI, hM ) as 2 64- (or 48-) dimensional points. How do we calculate the distance between 2 64- or 48-) dimensional points?

Method 2: Comparing two (1D) histograms hI and hM cont’d. Normalize (divide) each hI and hM by total number of pixels (to approximate probabilities). Calculate distance d( hI, hM ) as 2 64- (or 48-) dimensional points. How do we calculate the distance between 2 64- or 48-) dimensional points?

Method 3: Relative entropy (or the Kullback Leibler distance) from “Elements of Information Theory,” 2nd ed., (Wiley Series in Telecommunications and Signal Processing) by Thomas M. Cover and Joy A. Thomas (Hardcover - Jul 18, 2006), p. 18. (all logs are base 2)

Method 4: Mutual information from “Elements of Information Theory,” 2nd ed., (Wiley Series in Telecommunications and Signal Processing) by Thomas M. Cover and Joy A. Thomas (Hardcover - Jul 18, 2006) (all logs are base 2)

Method 4: Mutual information cont’d. How to calculate the joint distribution or joint p.m.f., p(x,y):

Color segmentation example

Color segmentation example Problem: Find (segment) face in workstation camera image. Approach: Training phase: Determine clusters from pixels from known faces using red-green values. Classification phase: Given a pixel that we need to classify (as either face or not), produce a labeled image which is then filtered according to region size.

non face pixel colors cluster of face pixel colors

End color segmentation example

Color histograms Yet another method is to map 3D RGB values into a 3D histogram. Note that K-means can also be easily applied to any of these 3 color histogram methods.

Homework Exercises 6.6, 6.8, and 6.9.

Homework Exercise 6.6. Using Algorithm 6.1 (procedure RGB_to_HSI), convert the RGB value (100,150,200) into an HSI value and, convert the RGB value (0.0, 1.0, 0.0) to HSI.

Homework Exercise 6.8. Suppose a color camera encodes a given pixel in RGB as (200, 50, 100), where 255 is the highest (most energy) value. What should be the equivalent triple in the HSI system? What should be the equivalent triple in the YIQ system?

Homework Exercise 6.9. Is the transformation from RGB to YIQ invertible? Is so, compute the inverse.

Color is a key feature in identifying the real tiger.

original increased saturation decreased saturation HSI is a natural representation for increased or decreasing the amount of color in an image. Hue angle in [0..2Pi]. Saturation in [0..1]. Intensity in [0..255].

image after segmentation and labeling (coloring)

Storing images in file

{ P2 = ASCII gray PGM header ASCII max data value data comments height (rows) width (cols)

P3 = ASCII color PPM Each pixel’s color is represented as <red,green,blue>. P3 w h maxval v1r v1g v1b v2r v2g v2b . . . . . . . . . vw*hr vw*hg vw*hb

2D Images Stored as 1D Arrays Say our 2D color (RGB) image is stored as a 1D array, I. How do we calculate the offset to a particular element given a particular row and column? First, do you recall how it’s done for a 2D gray image?