Basic Principles of Imaging and Lenses

Light

Light Photons ElectromagneticRadiation

These three are the same… Light *pure energy Electromagnetic Waves *energy-carrying waves emitted by vibrating electrons Photons *particles of light

EM Radiation Travels as a Wave c = 3 x 10 8 m/s

EM Radiation Carries Energy Quantum mechanics tells us that for photons E = hf where E is energy and h is Planck’s constant. But f = c/ Putting these equations together, we see that E = hc/

Electromagnetic Wave Velocity The speed of light is the same for all seven forms of light. It is 300,000,000 meters per second or 186,000 miles per second.

The Electromagnetic Spectrum Radio Waves - communication Microwaves - used to cook Infrared - “heat waves” Visible Light - detected by your eyes Ultraviolet - causes sunburns X-rays - penetrates tissue Gamma Rays - most energetic

The Multi-Wavelength Sun X-Ray UV Visible Infrared Radio Composite

EM Spectrum Relative Sizes

The Visible Spectrum Light waves extend in wavelength from about 400 to 700 nanometers.

Camera Obscura, Gemma Frisius, A Brief History of Images

(Russell Naughton) Camera Obscura "When images of illuminated objects... penetrate through a small hole into a very dark room... you will see [on the opposite wall] these objects in their proper form and color, reduced in size... in a reversed position, owing to the intersection of the rays". Leonardo da Vinci Slide credit: David Jacobs

Lens Based Camera Obscura, A Brief History of Images

(Jack and Beverly Wilgus) Jetty at Margate England, Slide credit: David Jacobs

Still Life, Louis Jaques Mande Daguerre, A Brief History of Images

Abraham Lincoln? ? 1568 A Brief History of Images

Silicon Image Detector, A Brief History of Images

A Brief History of Images Digital Cameras

A Brief History of Images Hasselblad HD

Geometric Optics and Image Formation

Pinhole Cameras Pinhole camera - box with a small hole in it Image is upside down, but not mirrored left-to-right Question: Why does a mirror reverse left-to-right but not top-to-bottom?

Pinhole and the Perspective Projection (x,y) screenscene Is an image being formed on the screen? YES! But, not a “clear” one. image plane effective focal length, f’ optical axis y x z pinhole

Problems with Pinholes Pinhole size (aperture) must be “very small” to obtain a clear image. However, as pinhole size is made smaller, less light is received by image plane. If pinhole is comparable to wavelength of incoming light, DIFFRACTION effects blur the image! Sharpest image is obtained when: pinhole diameter Example: If f’ = 50mm, = 600nm (red), d = 0.36mm

The Reason for Lenses

Image Formation using (Thin) Lenses Lenses are used to avoid problems with pinholes. Ideal Lens: Same projection as pinhole but gathers more light! i o Gaussian Lens Formula: f is the focal length of the lens – determines the lens’s ability to bend (refract) light f different from the effective focal length f’ discussed before! P P’ f

Focus and Defocus Depth of Field: Range of object distances over which image is sufficiently well focused, i.e., range for which blur circle is less than the resolution of the imaging sensor. d aperture diameter aperture Gaussian Law: Blur Circle, b Blur Circle Diameter :

Problems with Lenses Compound (Thick) Lens Vignetting Chromatic AbberationRadial and Tangential Distortion thickness principal planes nodal points B A more light from A than B ! Lens has different refractive indices for different wavelengths. image plane ideal actual ideal actual

Spherical Aberration Spherical lenses are the only easy shape to manufacture, but are not correct for perfect focus.

Two Lens System Rule : Image formed by first lens is the object for the second lens. Main Rays : Ray passing through focus emerges parallel to optical axis. Ray through optical center passes un-deviated. image plane lens 2lens 1 object intermediate virtual image final image Magnification: Exercises: What is the combined focal length of the system? What is the combined focal length if d = 0?

Lens systems A good camera lens may contain 15 elements and cost a many thousand dollars The best modern lenses may contain aspherical elements

Human Eye The eye has an iris like a camera Focusing is done by changing shape of lens Retina contains cones (mostly used) and rods (for low light) The fovea is small region of high resolution containing mostly cones Optic nerve: 1 million flexible fibers Slide credit: David Jacobs

The Eye The human eye is a camera! –Iris - colored annulus with radial muscles –Pupil - the hole (aperture) whose size is controlled by the iris –What’s the “film”? photoreceptor cells (rods and cones) in the retina

Human Eye vs. the Camera We make cameras that act “similar” to the human eye

Image Formation Digital Camera The Eye Film

Insect Eye We make cameras that act “similar” to the human eye Fly Mosquito

The Retina

Retina up-close Light

© Stephen E. Palmer, 2002 Cones cone-shaped less sensitive operate in high light color vision Two types of light-sensitive receptors Rods rod-shaped highly sensitive operate at night gray-scale vision

Rod / Cone sensitivity The famous sock-matching problem…

Human Eye Rods –Intensity only –Essentially night vision and peripheral vision only –Since we are trying to fool the center of field of view of human eye (under well lit conditions) we ignore rods

Human Eye Cones –Three types perceive different portions of the visible light spectrum

Human Eye Because there are only 3 types of cones in human eyes, we only need 3 stimulus values to fool the human eye –Note: Chickens have 4 types of cones

© Stephen E. Palmer, 2002 Distribution of Rods and Cones Night Sky: why are there more stars off-center?

The Physics of Light Some examples of the spectra of light sources © Stephen E. Palmer, 2002

More Spectra metamers

The Physics of Light Some examples of the reflectance spectra of surfaces Wavelength (nm) % Photons Reflected Red Yellow Blue Purple © Stephen E. Palmer, 2002

The Psychophysical Correspondence There is no simple functional description for the perceived color of all lights under all viewing conditions, but …... A helpful constraint: Consider only physical spectra with normal distributions area mean variance © Stephen E. Palmer, 2002

The Psychophysical Correspondence MeanHue # Photons Wavelength © Stephen E. Palmer, 2002

The Psychophysical Correspondence VarianceSaturation Wavelength # Photons © Stephen E. Palmer, 2002

The Psychophysical Correspondence AreaBrightness # Photons Wavelength © Stephen E. Palmer, 2002

Digital camera A digital camera replaces retina with a sensor array –Each cell in the array is light-sensitive diode that converts photons to electrons –Two common types Charge Coupled Device (CCD) CMOS –

CCD Cameras Slide credit: David Jacobs

Sensor Array CMOS sensor

Sampling and Quantization

Interlace vs. progressive scan

Progressive scan

Interlace

Color Sensing in Camera (RGB) 3-chip vs. 1-chip: quality vs. cost Why more green? Why 3 colors?

Practical Color Sensing: Bayer Grid Estimate RGB at ‘G’ cels from neighboring values words/Bayer-filter.wikipedia

Image Formation f(x,y) = reflectance(x,y) * illumination(x,y) Reflectance in [0,1], illumination in [0,inf]

White Balance White World / Gray World assumptions

Problem: Dynamic Range , ,000 2,000,000,000 The real world has High dynamic range

pixel (312, 284) = 42 Image 42 photos? Is Camera a photometer?

Long Exposure Real world Picture 0 to 255 High dynamic range

Short Exposure Real world Picture 0 to 255 High dynamic range

sceneradiance (W/sr/m )  sensorirradiancesensorexposure LensShutter 2 tttt analog voltages digital values pixel values CCDADCRemapping Image Acquisition Pipeline Camera is NOT a photometer!

Varying Exposure

What does the eye sees? The eye has a huge dynamic range Do we see a true radiance map?