Professor Sebastian Thrun CAs: Dan Maynes-Aminzade and Mitul Saha Stanford CS223B Computer Vision, Winter 2006 Lecture 2 Lenses, Filters, Features Professor Sebastian Thrun CAs: Dan Maynes-Aminzade and Mitul Saha [with slides by D Forsyth, D. Lowe, M. Polleyfeys, C. Rasmussen, G. Loy, D. Jacobs, J. Rehg, A, Hanson, G. Bradski,…]

Today’s Goals Thin Lens Aberrations Features 101 Linear Filters and Edge Detection

Pinhole Camera (last Wednesday) “shards of diamond” – those little pieces of beautiful truth scattered about. -- Brunelleschi, XVth Century Marc Pollefeys comp256, Lect 2

Snell’s Law Snell’s law n1 sin a1 = n2 sin a2

Thin Lens: Definition f optical axis focus Spherical lense surface: Parallel rays are refracted to single point

Thin Lens: Projection f z optical axis Image plane Spherical lense surface: Parallel rays are refracted to single point

Thin Lens: Projection f f z optical axis Image plane Spherical lense surface: Parallel rays are refracted to single point

Thin Lens: Properties Any ray entering a thin lens parallel to the optical axis must go through the focus on other side Any ray entering through the focus on one side will be parallel to the optical axis on the other side

Thin Lens: Model Q P O Fr Fl p R Z f f z

Transformation

A Transformation

The Thin Lens Law Q P O Fr Fl p R Z f f z

The Thin Lens Law

Limits of the Thin Lens Model 3 assumptions : all rays from a point are focused onto 1 image point Remember thin lens small angle assumption 2. all image points in a single plane 3. magnification is constant Deviations from this ideal are aberrations

Today’s Goals Thin Lens Aberrations Features 101 Linear Filters and Edge Detection

Aberrations 2 types : geometrical : geometry of the lense, small for paraxial rays chromatic : refractive index function of wavelength Marc Pollefeys

Geometrical Aberrations spherical aberration astigmatism distortion coma aberrations are reduced by combining lenses

Astigmatism Different focal length for inclined rays Marc Pollefeys

Astigmatism Different focal length for inclined rays Marc Pollefeys

Spherical Aberration rays parallel to the axis do not converge outer portions of the lens yield smaller focal lenghts

Can be corrected! (if parameters are know) Distortion magnification/focal length different for different angles of inclination pincushion (tele-photo) barrel (wide-angle) Can be corrected! (if parameters are know) Marc Pollefeys

Coma point off the axis depicted as comet shaped blob Marc Pollefeys

Chromatic Aberration rays of different wavelengths focused in different planes cannot be removed completely Marc Pollefeys

Vignetting Effect: Darkens pixels near the image boundary

CCD vs. CMOS Mature technology Specific technology Recent technology High production cost High power consumption Higher fill rate Blooming Sequential readout Recent technology Standard IC technology Cheap Low power Less sensitive Per pixel amplification Random pixel access Smart pixels On chip integration with other components Marc Pollefeys

Today’s Goals Thin Lens Aberrations Features 101 Linear Filters and Edge Detection

Today’s Question What is a feature? What is an image filter? How can we find corners? How can we find edges? (How can we find cars in images?)

What is a Feature? Local, meaningful, detectable parts of the image

Features in Computer Vision What is a feature? Location of sudden change Why use features? Information content high Invariant to change of view point, illumination Reduces computational burden

(One Type of) Computer Vision Feature 1 Feature 2 : Feature N Image 2 Image 1 Feature 1 Feature 2 : Feature N Computer Vision Algorithm

Where Features Are Used Calibration Image Segmentation Correspondence in multiple images (stereo, structure from motion) Object detection, classification

What Makes For Good Features? Invariance View point (scale, orientation, translation) Lighting condition Object deformations Partial occlusion Other Characteristics Uniqueness Sufficiently many Tuned to the task

Today’s Goals Features 101 Linear Filters and Edge Detection Canny Edge Detector

What Causes an Edge? Depth discontinuity Surface orientation discontinuity Reflectance discontinuity (i.e., change in surface material properties) Illumination discontinuity (e.g., shadow) Slide credit: Christopher Rasmussen

Quiz: How Can We Find Edges?

Edge Finding 101 im = imread('bridge.jpg'); image(im); figure(2); bw = double(rgb2gray(im)); image(bw); gradkernel = [-1 1]; dx = abs(conv2(bw, gradkernel, 'same')); image(dx); colorbar; colormap gray [dx,dy] = gradient(bw); gradmag = sqrt(dx.^2 + dy.^2); image(gradmag); matlab colorbar colormap(gray(255)) colormap(default)

Edge Finding 101 Example of a linear Filter

Today’s Goals Thin Lens Aberrations Features 101 Linear Filters and Edge Detection

What is Image Filtering? Modify the pixels in an image based on some function of a local neighborhood of the pixels 10 5 3 4 1 7 7 Some function

Linear Filtering Linear case is simplest and most useful Replace each pixel with a linear combination of its neighbors. The prescription for the linear combination is called the convolution kernel. 10 5 3 4 1 7 0.5 1.0 7 kernel

Linear Filter = Convolution g11 g11 I(i-1,j-1) I(.) g12 + g12 I(i-1,j) I(.) g13 + g13 I(i-1,j+1) + I(.) g21 g21 I(i,j-1) g22 + g22 I(i,j) I(.) I(.) g23 + g23 I(i,j+1) + I(.) g31 g31 I(i+1,j-1) I(.) g32 + g32 I(i+1,j) I(.) g33 + g33 I(i+1,j+1) f (i,j) =

Linear Filter = Convolution

Filtering Examples

Filtering Examples

Filtering Examples

Image Smoothing With Gaussian figure(3); sigma = 3; width = 3 * sigma; support = -width : width; gauss2D = exp( - (support / sigma).^2 / 2); gauss2D = gauss2D / sum(gauss2D); smooth = conv2(conv2(bw, gauss2D, 'same'), gauss2D', 'same'); image(smooth); colormap(gray(255)); gauss3D = gauss2D' * gauss2D; tic ; smooth = conv2(bw,gauss3D, 'same'); toc

Smoothing With Gaussian Averaging Gaussian Slide credit: Marc Pollefeys

Smoothing Reduces Noise The effects of smoothing Each row shows smoothing with gaussians of different width; each column shows different realizations of an image of gaussian noise. Slide credit: Marc Pollefeys

Example of Blurring Image Blurred Image - =

Edge Detection With Smoothed Images figure(4); [dx,dy] = gradient(smooth); gradmag = sqrt(dx.^2 + dy.^2); gmax = max(max(gradmag)); imshow(gradmag); colormap(gray(gmax));

Scale Increased smoothing: Eliminates noise edges. Makes edges smoother and thicker. Removes fine detail.

The Edge Normal

Displaying the Edge Normal figure(5); hold on; image(smooth); colormap(gray(255)); [m,n] = size(gradmag); edges = (gradmag > 0.3 * gmax); inds = find(edges); [posx,posy] = meshgrid(1:n,1:m); posx2=posx(inds); posy2=posy(inds); gm2= gradmag(inds); sintheta = dx(inds) ./ gm2; costheta = - dy(inds) ./ gm2; quiver(posx2,posy2, gm2 .* sintheta / 10, -gm2 .* costheta / 10,0); hold off;

Separable Kernels

Combining Kernels / Convolutions 0.0030 0.0133 0.0219 0.0133 0.0030 0.0133 0.0596 0.0983 0.0596 0.0133 0.0219 0.0983 0.1621 0.0983 0.0219

Effect of Smoothing Radius 1 pixel 3 pixels 7 pixels

Robert’s Cross Operator 1 0 0 -1 0 1 -1 0 + S = [ I(x, y) - I(x+1, y+1) ]2 + [ I(x, y+1) - I(x+1, y) ]2 or S = | I(x, y) - I(x+1, y+1) | + | I(x, y+1) - I(x+1, y) |

Sobel Operator -1 -2 -1 0 0 0 1 2 1 -1 0 1 -2 0 2 S1= S2 = -1 -2 -1 0 0 0 1 2 1 -1 0 1 -2 0 2 S1= S2 = Edge Magnitude = Edge Direction = S1 + S1 2 tan-1 S1 S2

The Sobel Kernel, Explained Sobel kernel is separable! Averaging done parallel to edge

Sobel Edge Detector figure(6) edge(bw, 'sobel')

Robinson Compass Masks -1 0 1 -2 0 2 0 1 2 -1 0 1 -2 -1 0 1 2 1 0 0 0 -1 -2 -1 2 1 0 1 0 -1 0 -1 -2 2 0 -2 1 1 -1 -1 0 -1

Claim Your Own Kernel! Frei & Chen

Comparison (by Allan Hanson) Analysis based on a step edge inclined at an angle q (relative to y-axis) through center of window. Robinson/Sobel: true edge contrast less than 1.6% different from that computed by the operator. Error in edge direction Robinson/Sobel: less than 1.5 degrees error Prewitt: less than 7.5 degrees error Summary Typically, 3 x 3 gradient operators perform better than 2 x 2. Prewitt2 and Sobel perform better than any of the other 3x3 gradient estimation operators. In low signal to noise ratio situations, gradient estimation operators of size larger than 3 x 3 have improved performance. In large masks, weighting by distance from the central pixel is beneficial.