1. Computational Photography: Color perception, light spectra, contrast Connelly Barnes

2. Color Perception, etc ●Previously: ○Camera obscura / pinhole camera ○Cameras with lenses ○Modeling camera projections

3. Color Perception, etc ●Today: ○Human / electronic eyes ○Electromagnetic spectrum ○Color spaces Various slides by Alexei Efros, Fredo Durand, James Hays

4. Image Formation Digital Camera The Eye Film Slide by Efros

5. Sensor Array CMOS sensor Slide by Efros CCD sensor

6. Sampling and Quantization

7. Interlace vs. progressive scan http://www.axis.com/products/video/camera/progressive_scan.htmSlide by Steve Seitz

8. Progressive scan http://www.axis.com/products/video/camera/progressive_scan.htmSlide by Steve Seitz

9. Interlace http://www.axis.com/products/video/camera/progressive_scan.htmSlide by Steve Seitz

10. Rolling Shutter SLR cameras at high shutter speed, most CMOS cameras

11. The Eye The human eye is a camera! Iris - colored annulus with radial muscles Pupil - the hole (aperture) whose size is controlled by the iris What’s the “film”? photoreceptor cells (rods and cones) in the retina Slide by Steve Seitz

12. The Retina

13. © Stephen E. Palmer, 2002 Cones cone-shaped less sensitive operate in high light color vision Two types of light-sensitive receptors Rods rod-shaped highly sensitive operate at night gray-scale vision

14. Rod / Cone sensitivity

15. Some Goals of Human Eye Recognize food Recognize friends, mates Detect predators Navigation -- identify 3D structure Limited memory, computation budget Highest resolution in fovea - (2 degrees, 50% of visual cortex) Absolute luminance discarded Edges, corners retained Store only a tiny fraction of what is observed

16. Visual Clutter - Bandwidth Overload

17. Eye Movements Motion Magnification -- Eye Movements

18. Electromagnetic Spectrum http://www.yorku.ca/eye/photopik.htm Human Luminance Sensitivity Function

19. Why do we see light of these wavelengths? © Stephen E. Palmer, 2002 …because that’s where the Sun radiates EM energy Visible Light

20. The Physics of Light Any patch of light can be completely described physically by its spectrum: the number of photons (per time unit) at each wavelength 400 - 700 nm. © Stephen E. Palmer, 2002

21. The Physics of Light Some examples of the spectra of light sources © Stephen E. Palmer, 2002

22. The Physics of Light Some examples of the reflectance spectra of surfaces Wavelength (nm) % Photons Reflected Red 400 700 Yellow 400 700 Blue 400 700 Purple 400 700 © Stephen E. Palmer, 2002

23. Ordinary Human Vision (Trichromatism)

24. Perceptual Sensitivity ITU Recommendation for HDTV: Y = 0.21 R + 0.72 G + 0.07 B Evolved to detect vegetation, berries?

25. Tetrachromatism Most birds, and many other animals, have cones for ultraviolet light. Some humans, mostly female, seem to have slight tetrachromatism. Bird cone responses

26. Color Spectra metamers

27. Slide by Fredo Durand

31. Color Image R G B

32. Images in Python/MATLAB 0.920.930.940.970.620.370.850.970.930.920.99 0.950.890.820.890.560.310.750.920.810.950.91 0.890.720.510.550.510.420.570.410.490.910.92 0.960.950.880.940.560.460.910.870.900.970.95 0.710.81 0.870.570.370.800.880.890.790.85 0.490.620.600.580.500.600.580.500.610.450.33 0.860.840.740.580.510.390.730.920.910.490.74 0.960.670.540.850.480.370.880.900.940.820.93 0.690.490.560.660.430.420.770.730.710.900.99 0.790.730.900.670.330.610.690.790.730.930.97 0.910.940.890.490.410.78 0.770.890.990.93 0.920.930.940.970.620.370.850.970.930.920.99 0.950.890.820.890.560.310.750.920.810.950.91 0.890.720.510.550.510.420.570.410.490.910.92 0.960.950.880.940.560.460.910.870.900.970.95 0.710.81 0.870.570.370.800.880.890.790.85 0.490.620.600.580.500.600.580.500.610.450.33 0.860.840.740.580.510.390.730.920.910.490.74 0.960.670.540.850.480.370.880.900.940.820.93 0.690.490.560.660.430.420.770.730.710.900.99 0.790.730.900.670.330.610.690.790.730.930.97 0.910.940.890.490.410.78 0.770.890.990.93 0.920.930.940.970.620.370.850.970.930.920.99 0.950.890.820.890.560.310.750.920.810.950.91 0.890.720.510.550.510.420.570.410.490.910.92 0.960.950.880.940.560.460.910.870.900.970.95 0.710.81 0.870.570.370.800.880.890.790.85 0.490.620.600.580.500.600.580.500.610.450.33 0.860.840.740.580.510.390.730.920.910.490.74 0.960.670.540.850.480.370.880.900.940.820.93 0.690.490.560.660.430.420.770.730.710.900.99 0.790.730.900.670.330.610.690.790.730.930.97 0.910.940.890.490.410.78 0.770.890.990.93 R G B row column Image as array: h x w x channels I(y,x,channel) Red channel, upper left corner: MATLAB: I(1,1,1), Python: I[0,0,0]

33. Color spaces: RGB 0,1,0 0,0,1 1,0,0 Image from: http://en.wikipedia.org/wiki/File:RGB_color_solid_cube.png Some drawbacks Strongly correlated channels Non-perceptual Default color space R (G=0,B=0) G (R=0,B=0) B (R=0,G=0)

34. Color spaces: HSV Intuitive color space H (S=1,V=1) S (H=1,V=1) V (H=1,S=0)

35. Color spaces: L*a*b* “Perceptually uniform” color space L (a=0,b=0) a (L=65,b=0) b (L=65,a=0)

36. White Balance Slide by Alexei Efros

37. Problem: Dynamic Range 1500 1 1 25,000 400,000 2,000,000,000 The real world is High dynamic range Slide by Alexei Efros

38. pixel (312, 284) = 42 Image 42 photons? Is Camera a photometer? Slide by Alexei Efros

39. Long Exposure 10 -6 10 6 10 -6 10 6 Real world Picture 0 to 255 High dynamic range Slide by Alexei Efros

40. Short Exposure 10 -6 10 6 10 -6 10 6 Real world Picture 0 to 255 High dynamic range Slide by Alexei Efros

41. sceneradiance (W/sr/m ) sceneradiance  sensorirradiancesensorirradiancesensorexposuresensorexposure LensLensShutterShutter 22 tttt analog voltages analog voltages digital values digital values pixel values pixel values CCD ADC Remapping Image Acquisition Pipeline Camera is NOT a photometer!

42. Varying Exposure

43. What does the eye sees? The eye has a huge dynamic range Do we see a true radiance map?

44. Eye is Not a Photo-meter