1. Lecture 6 Chapters 6 and 7 Color and Texture
Slides by:
David A. Forsyth
Clark F. Olson
Linda G. Shapiro

2. Color Used heavily in human vision
Color is a pixel property, making some recognition problems easy
Visible spectrum for humans is 400nm (blue) to 700 nm (red)
Machines can “see” much more; ex. X-rays, infrared, radio waves

3. Causes of color The sensation of color is caused by the brain.
Some ways to get this sensation include:
Pressure on the eyelids
Dreaming, hallucinations, etc.
Main way to get it is the response of the visual system to the presence/absence of light at various wavelengths.
Issues that affect perception of color:
Light sources with different spectrums (compare the sun and a fluorescent light bulb)
Differential reflection (e.g. some pigments) and absorption
Differential refraction - (e.g. Newton’s prism)
Different distance and angle of reflection
Sensitivity of sensor

4. Some physics
White light is composed of all visible frequencies ( )
Ultraviolet and X-rays are of much smaller wavelength
Infrared and radio waves are of much longer wavelength

5. Electromagnetic spectrum
At least 3 spectral bands required (e.g. R,G,B)
Human Luminance Sensitivity Function

6. The physics of light Some examples of the spectra of light sources
© Stephen E. Palmer, 2002

7. The physics of light
Some examples of the reflectance spectra of surfaces
Red
Yellow
Blue
Purple
% Photons Reflected
Wavelength (nm)
© Stephen E. Palmer, 2002

8. Albedos Color varies along a linear scale (wavelength).
Spectral albedos for different leaves, with color names attached.
Color varies along a linear scale (wavelength).
Different colors typically have different spectral albedo.
Measurements by E.Koivisto.
Violet Indigo Blue Green Yellow Orange Red

9. The appearance of colors
Color appearance is strongly affected by (at least):
other nearby colors,
adaptation to previous views
“state of mind”
Image from:

10. The appearance of colors
Color appearance is strongly affected by (at least):
other nearby colors,
adaptation to previous views
“state of mind”
Image from:

11. What color is this dress?

12. Color spaces
RGB: primaries are monochromatic (formally 645.2nm, 526.3nm, 444.4nm)
CIE XYZ: Primaries are imaginary (negative spectral radiance), but have other convenient properties
Also:
CMY: subtractive color space used for printing
HSV: perceptually salient space for several applications
YIQ: used for TV (NTSC) – good for compression
A choice of three primaries yields a linear color space - the coordinates of a color are given by the weights of the primaries used to match it.
Choice of primaries is equivalent to choice of color space.

13. Color spaces: HSV Intuitive color space H S V (S=1,V=1) (H=1,V=1)

14. Comparing color spaces

15. Color cube R, G, B values normalized to (0, 1) interval
humans perceive gray for triples on the diagonal
“Pure colors” on corners

16. Color receptors and color deficiency
Trichromacy is justified - in most people, there are three types of color receptor, called cones, which vary in their sensitivity to light at different wavelengths (shown by molecular biologists).
Some people have fewer than three types of receptor; most common deficiency is red-green color blindness in men.

17. Texture
Texture is a description of the spatial arrangement of color or intensities in an image or a selected region of an image.
Regular or stochastic patterns caused by bumps, grooves, and/or markings
Structural approach: Texture is a set of primitive texels in some regular or repeated relationship.

18. Texture
Finding texels is difficult in most images:

19. Texture and Material

20. Texture and Orientation

21. Texture and Scale

22. Statistical texture
Most common approach in computer vision is to compute statistics in the image to represent texture.
- Computationally efficient
- Can be used for classification and segmentation
Simplistic approach: apply edge detection
- Number of edge pixels is one measure of texture
- Orientation is another (average or histogram)

23. Co-occurrence matrix
A co-occurrence matrix is a 2D array N (or C) in which:
Both the rows and columns represent a set of possible image values.
Nd(i,j) indicates how many times value i co-occurs with value j in a particular spatial relationship d.
The spatial relationship is specified by a vector d = (dr,dc).
This is essentially a 2D histogram storing a particular spatial relationship between intensity values.

24. Co-occurrence matrix 1 0 1 2 1 1 0 0 0 0 2 2 i j 1 2 6 0 4 2 2 0 0 0 4 i j 1 2
6 0 4
2 2 0
0 0 4
d = (0,1) C d
co-occurrence
matrix
grayscale
image

25. Co-occurrence features
Numeric features computed from the co-occurrence matrix can be used to represent and compare textures.

26. Co-occurrence matrix How do you choose d?
Are the textures small, medium, large?
One suggestion (Zucker and Terzopoulos): use a statistical test to select value(s) that have the most “structure”.

27. Texture representation
Another method to represent image texture is by convolving the image with a set of filters.
Each pixel is represented by a vector of filter responses, the “texture signature”
Strong response when image is similar to filter
Weak response when not similar
The filters that are typically used look like:
Spots
Bars

28. Filters are templates
Applying a filter at some point can be seen as taking a dot-product between the image and the filter.
Both are viewed as 1D vectors rather than 2D images
Filtering the image is a set of dot products.
Insight:
filters look like the effects they are intended to find
filters find effects they look like
why?

29. Filters are templates Positive responses
The filter is the little block in the top left hand corner. Notice this is a fair spot-detector.
Positive responses

30. Filters are templates Positive responses
The filter is the little block in the top left hand corner. Notice this is a fair bar-detector.
Positive responses

31. Scaled representations
Big bars and little bars (elongated features like limbs or stripes) are both interesting features to detect in an image.
- Also could be dots or other shapes
It is inefficient to detect big bars with big filters.
- And there is superfluous detail in the filter kernel
Alternative:
Apply filters of fixed size to images of different sizes
Typically, a collection of images whose edge length changes by a factor of 2 (or the square root of 2)
This is a pyramid by visual analogy (sometimes called a Gaussian pyramid)

32. Image Pyramids Known as a Gaussian Pyramid [Burt and Adelson, 1983]
In computer graphics, a mip map [Williams, 1983]
Slide by Steve Seitz

33. Scaled representations
A bar in the biggest image is a hair on the zebra’s nose; in middle images, a stripe; in the smallest, the animal’s nose

34. Representing textures
Real textures are made up of patterns of irregular subelements.
What are the subelements?
not well defined, in general
usually reduced to most basic shapes: spots and bars at various sizes and orientations
How do we find them?
by applying filters
After applying bar and spot filters apply statistics locally:
mean
standard deviation
histograms

35. Representing textures

36. Overcomplete representation: filter banks
LM Filter Bank
Code for filter banks:

37. Representing textures
Filters (not to scale):
Original image:
Filter responses:
Spots and bars at a fine scale for the butterfly; images show squared response for
corresponding filter

38. Template matching Goal: find in image
Main challenge: What is a good similarity or distance measure between two patches?
Correlation
Zero-mean correlation
Sum of Squared Differences
Normalized Cross Correlation
Slide: Hoiem

39. Matching with filters Goal: find in image
Method 0: filter the image with eye patch
f = image
g = filter
Problem: response is stronger for higher intensity
What went wrong?
Input
Filtered Image
Slide: Hoiem

40. Matching with filters Goal: find in image
Method 1: filter the image with zero-mean eye
mean of f
True detections
False detections
Likes bright pixels where filters are above average, dark pixels where filters are below average. Problems: response is sensitive to gain/contrast, pixels in filter that are near the mean have little effect, does not require pixel values in image to be near or proportional to values in filter.
Input
Filtered Image (scaled)
Thresholded Image
Slide: Hoiem

41. Matching with filters Goal: find in image Method 2: SSD
True detections
Problem: SSD sensitive to average intensity
Input
1- sqrt(SSD)
Thresholded Image
Slide: Hoiem

42. What’s the potential downside of SSD?
Matching with filters
Goal: find in image
Method 2: SSD
What’s the potential downside of SSD?
Problem: SSD sensitive to average intensity
Input
1- sqrt(SSD)
Slide: Hoiem

43. Matching with filters Goal: find in image
Method 3: Normalized cross-correlation
mean template
mean image patch
Invariant to mean and scale of intensity
Slide: Hoiem

44. Matching with filters Goal: find in image
Method 3: Normalized cross-correlation
True detections
Input
Normalized X-Correlation
Thresholded Image
Slide: Hoiem

45. Matching with filters Goal: find in image
Method 3: Normalized cross-correlation
True detections
Input
Normalized X-Correlation
Thresholded Image
Slide: Hoiem