1. Computational Photography
Course 15:
Computational Photography
A.3: Understanding Film-like Photography Tumblin

2. Northwestern University
Computational Photography
A3: Understanding Film-Like Photography or ‘from 2D Pixels to 4D Rays’ (10 minutes)
Jack Tumblin
Northwestern University

3. Naïve, Ideal Film-like Photography
Sensor: a film emulsion, : or a grid of light meters (pixels)
Ray
‘Center of Projection’
Position (x,y)
Angle(,)
As you can see, the ideal of film-like photography is very simple when stated in rays: the goal is to measure the radiance along all rays through a single point, the ‘center of projection’ defined by some system of lenses.
2D Sensor:
Pixel Grid,Film,…
Well-Lit 3D Scene:

4. Rays and the ‘Thin Lens Law’
Focal length f: where parallel rays converge
Focus at infinity: Adjust for S2=f
Closer Focus ? Larger S2 f
Sensor S2
Thin Lens
Try it Live! Physlets…

5. Rays and the ‘Thin Lens Law’
Focal length f: where parallel rays converge
Focus at infinity: Adjust for S2=f
Closer Focus ? Larger S2 f f
Sensor
Scene S2 S2
Thin Lens
Try it Live! Physlets…

6. Not One Ray, but a Bundle of Rays
Lens
Scene
Sensor
Aperture
BUT Ray model isn’t perfect: ignores diffraction
Lens, aperture set the point-spread-function (PSF) (How? See: Goodman,J.W. ‘An Introduction to Fourier Optics’ 1968)

7. Basic Ray Optics: Lens Aperture
For the same focal length:
Larger lens
Gathers a wider ray bundle:
More light: brighter image
Narrower depth-of-focus
Smaller lens
dimmer image
focus becomes less critical
more depth of focus
The degree to which we can approximate a thin lens STRONGLY limits image quality: in general, no lens is simple to make, and cheap lenses cost you dearly in image quality. Here are a few of the reasons how REAL lenses fall short of ideal ‘thin’ lenses:

8. Film-like Optics: Thin Lens Flaws
Aberrations: Real lenses don’t converge rays perfectly
Spherical: edge rays  center rays
Coma: diagonal rays focus deeper at edge

9. Lens Flaws: Chromatic Aberration
Dispersion: wavelength-dependent refractive index
(enables prism to spread white light beam into rainbow)
Modifies ray-bending and lens focal length: f()
color fringes near edges of image
Corrections: add ‘doublet’ lens of flint glass, etc.

10. Chromatic Aberration
Lens Design Fix: Multi-element lenses Complex, expensive, many tradeoffs!
Computed Fix: Geometric warp for R,G,B.
Near Lens Center
Near Lens Outer Edge

11. Radial Distortion (e.g. ‘Barrel’ and ‘pin-cushion’)
straight lines curve around the image center

12. Vignette Effects
Bright at center, dark at edges. Several causes compounded:
Edge pixels span smaller angle and center pixels
Ray path length is longer off-axis
Internal shadowing
Compensation:
Use anti-vignetting filters, (darkest at center)
OR Position-dependent pixel-detector sensitivity.

13. Film-like Color Sensing
Visible Light: narrow band of e’mag. spectrum
  nm (nm = 10-9 meter wavelength)
(humans:<1 octave honey bees: 3-4 ‘octaves do honey bees sense harmonics, see color ‘chords’ ?
At least 3 spectral bands required (e.g. R,G,B)
Equiluminant Curve defines ‘luminance’ vs. wavelength

14. Film-like Color Sensing
Visible Light: narrow band of emag spectrum
  nm (nm = 10-9 meter wavelength)
At least 3 spectral bands required (e.g. R,G,B)
At least 3 spectral bands required (e.g. R,G,B)
RGB spectral curves Vaytek CCD camera with Bayer grid

15. Color Sensing 3-chip: vs. 1-chip: quality vs. cost

16. 1-Chip Color Sensing: Bayer Grid
Estimate RGB at ‘G’ cels from neighboring values
words/Bayer-filter.wikipedia

17. Polarization Sunlit haze is often strongly polarized.
Polarization filter yields much richer sky colors
Ramesh: which one is polarized, which is non-polarized? Source for this?

18. RAYS and PROCESSING ONE Ray carries doubly infinitesimal power:
Ray bundles with finite, measurable power will:
Span a non-zero area
Fill a non-zero solid angle
Everything is Linear: (HUGE win!)
Ray reflectance, transmission, absorption, scatter*…
Rays are REVERSIBLE. Helmholtz reciprocity
Ray bundles? Not so much: falls quickly with angle,area growth…

19. Film-like Photography: Many Limitations
Optics:
Single focus distance, limited depth-of-field, limited field-of-view, internal reflections/flare/glare
Lighting:
Camera has no knowledge of ray source strength, position, direction; little control (e.g. flash)
Sensor:
Exposure setting, motion blur, noise, response time,
Processing:
Quantization/color depth, camera shake, scene movement…

20. Conclusions
Film-like photography methods limit digital photography to film-like results or less.
Broaden, unlock our views of photography:
4-D, 8-D, even 10-D Ray Space holds the photographic signal. Look for new solutions by creating, gathering, processing RAYS, not focal-plane intensities.
Choose the best, most expressive sets of rays, THEN find the best way to measure them.

21. Useful links:
Interactive Thin Lens Demo (or search ‘physlet optical bench’)
For more about color:
Prev. SIGGRAPH courses (Stone et al.)
Good:
Good:
Good:

22. Computational Photography
Course 15:
Computational Photography