1. Recap from Friday
Pinhole camera model Perspective projections Lenses and their flaws Focus Depth of field Focal length and field of view

2. What is wrong with this picture?

3. What is wrong with this picture?

4. Capturing Light… in man and machine
Many slides by Alexei A. Efros
CS 195g: Computational Photography
James Hays, Brown, Spring 2010

5. Image Formation
Digital Camera
Film
The Eye

6. Digital camera A digital camera replaces film with a sensor array
Each cell in the array is light-sensitive diode that converts photons to electrons
Two common types
Charge Coupled Device (CCD)
CMOS
Slide by Steve Seitz

7. Sensor Array
CMOS sensor

8. Sampling and Quantization

9. Interlace vs. progressive scan
Slide by Steve Seitz

10. Progressive scan
Slide by Steve Seitz

11. Interlace
Slide by Steve Seitz

12. The Eye The human eye is a camera!
Iris - colored annulus with radial muscles
Pupil - the hole (aperture) whose size is controlled by the iris
What’s the “film”?
photoreceptor cells (rods and cones) in the retina
Slide by Steve Seitz

13. The Retina

14. Retina up-close
Tapetum lucidum
Light

15. Two types of light-sensitive receptors
Cones
cone-shaped
less sensitive
operate in high light
color vision
Rods
rod-shaped
highly sensitive
operate at night
gray-scale vision
© Stephen E. Palmer, 2002

16. Rod / Cone sensitivity
The famous sock-matching problem…

17. Distribution of Rods and Cones
Night Sky: why are there more stars off-center?
© Stephen E. Palmer, 2002

18. Eye Movements
Saccades Microsaccades Ocular microtremor (OMT)

19. Electromagnetic Spectrum
At least 3 spectral bands required (e.g. R,G,B)
Human Luminance Sensitivity Function

20. …because that’s where the
Visible Light
Why do we see light of these wavelengths?
…because that’s where the
Sun radiates EM energy
© Stephen E. Palmer, 2002

21. The Physics of Light Any patch of light can be completely described
physically by its spectrum: the number of photons
(per time unit) at each wavelength nm.
© Stephen E. Palmer, 2002

22. The Physics of Light Some examples of the spectra of light sources
© Stephen E. Palmer, 2002

23. The Physics of Light
Some examples of the reflectance spectra of surfaces
Red
Yellow
Blue
Purple
% Photons Reflected
Wavelength (nm)
© Stephen E. Palmer, 2002

24. The Psychophysical Correspondence
There is no simple functional description for the perceived
color of all lights under all viewing conditions, but …...
A helpful constraint:
Consider only physical spectra with normal distributions
mean
area
variance
© Stephen E. Palmer, 2002

25. The Psychophysical Correspondence
Mean
Hue
# Photons
Wavelength
© Stephen E. Palmer, 2002

26. The Psychophysical Correspondence
Variance
Saturation
Wavelength
# Photons
© Stephen E. Palmer, 2002

27. The Psychophysical Correspondence
Area
Brightness
# Photons
Wavelength
© Stephen E. Palmer, 2002

28. Physiology of Color Vision
Three kinds of cones:
Why are M and L cones so close?
Why are there 3?
© Stephen E. Palmer, 2002

29. More Spectra
metamers

30. Color Sensing in Camera (RGB)
3-chip vs. 1-chip: quality vs. cost
Why more green?
Why 3 colors?
Slide by Steve Seitz

31. Practical Color Sensing: Bayer Grid
Estimate RGB at ‘G’ cells from neighboring values
words/Bayer-filter.wikipedia
Slide by Steve Seitz

32. RGB color space RGB cube Easy for devices But not perceptual
Where do the grays live?
Where is hue and saturation?
Slide by Steve Seitz

33. HSV Hue, Saturation, Value (Intensity)
RGB cube on its vertex
Decouples the three components (a bit)
Use rgb2hsv() and hsv2rgb() in Matlab
Slide by Steve Seitz

34. Project #1 How to compare R,G,B channels? No right answer
Sum of Squared Differences (SSD):
Normalized Correlation (NCC):

35. Image half-sizing This image is too big to fit on the screen. How
can we reduce it?
How to generate a half-
sized version?

36. Image sub-sampling
1/8
1/4
Throw away every other row and column to create a 1/2 size image
- called image sub-sampling
Slide by Steve Seitz

37. Image sub-sampling 1/2 1/4 (2x zoom) 1/8 (4x zoom)
Aliasing! What do we do?
Slide by Steve Seitz

38. Gaussian (lowpass) pre-filtering
Solution: filter the image, then subsample
Filter size should double for each ½ size reduction. Why?
Slide by Steve Seitz

39. Subsampling with Gaussian pre-filtering
Slide by Steve Seitz

40. Compare with...
1/2
1/4 (2x zoom)
1/8 (4x zoom)
Slide by Steve Seitz

41. Gaussian (lowpass) pre-filtering
Solution: filter the image, then subsample
Filter size should double for each ½ size reduction. Why?
How can we speed this up?
Slide by Steve Seitz

42. Image Pyramids In computer graphics, a mip map [Williams, 1983]
Known as a Gaussian Pyramid [Burt and Adelson, 1983]
In computer graphics, a mip map [Williams, 1983]
A precursor to wavelet transform
Slide by Steve Seitz

43. A bar in the big images is a hair on the zebra’s nose; in smaller images, a stripe; in the smallest, the animal’s nose
Figure from David Forsyth

44. What are they good for? Improve Search Pre-computation
Search over translations
Like project 1
Classic coarse-to-fine strategy
Search over scale
Template matching
E.g. find a face at different scales
Pre-computation
Need to access image at different blur levels
Useful for texture mapping at different resolutions (called mip-mapping)

45. Gaussian pyramid construction
filter mask
Repeat
Filter
Subsample
Until minimum resolution reached
can specify desired number of levels (e.g., 3-level pyramid)
The whole pyramid is only 4/3 the size of the original image!
Slide by Steve Seitz