Midterm Review

World is practically continuous in time space color brightness dynamic range brightness

Digital World Quantized in: –time –space –color –brightness how many frames/second? how many pixels? how many primaries? how many bits/pixel?

How TV Worked (in the old days) What is a cathode? What is a cathode ray? What is a cathode ray tube? What is a phosphor? What are "phosphorescence" and "fluorescence"? How does a "cathode ray tube" CRT work? How was the TV image acquired? What is a "raster" scan? What are horizontal and vertical "sync"? What is "interlace"? Why do that? What determines the "resolution"? What is "gamma" and why should it be corrected? How did they get color? How do decisions made in the 40's affect CG, CV, and IP today?

Impulse Response Function Point Spread Function What is the image of a point? –Shape of pinhole for points at infinity –Typically a little blob for a good lens –Could have aberrations and are distance, color, or position dependent. What happens as we enlarge the pinhole?

Blurring as convolution: IRF’s and apertures Blurring by convolution with impulse response function: I blurred (x)=  I input (y) h(x-y) dy –Replace each point y by y’s intensity times IRF h centered at y h(x-y) is effect of a fixed y on x over various image points x –Sum up over all such points Aperture in image space x: –effect of y on x over various y: h(-[y-x]) – weighting of various input image points in producing image at a fixed point x

Linear Systems Favorite model because we have great tools F(a+b) = F(a) + F(b), F(k a) = k F(a) Shift Invariant Time Invariant Is camera projection linear?

Properties of convolution: I out (x) =  I in (y) h(x-y) dy h(x) is called the convolution kernel Linear in both inputs, I in and h Symmetric in its inputs, I in and h Cascading convolutions is convolution with the convolution of the two kernels: (I * h 1 ) * h 2 = I * (h 1 * h 2 ) –Thus cascading of convolution of two Gaussians produces Gaussian with  = (   2 2 ) ½ Any linear, shift invariant operator can be written as a convolution or a limit of one

Linear Shift-Invariant Operators Blurring with IRF that is constant over the scene Viewing scene through any fixed aperture All derivatives D –So D(I * h 1 ) = DI * h 1 = I * Dh 1 Designed operations, e.g., for smoothing, noise removal, sharpening, etc. Can be applied to parametrized functions of u –E.g., smoothing surfaces

Example Impulse Responses

Properties of Convolution Commutative: a*b = b*a Associative: (a*b)*c = a*(b*c) Distributive: a*b + a*c = a*(b+c) Central Limit Theorem: convolve a pulse with itself enough times you get a Gaussian

Properties of convolution, continued For any convolution kernel h(x), if the input I in (x) is a sinusoid with wavelength (level of detail) 1/, i.e., I in (x) = A cos(2  x) + B sin(2  x), then the output of the convolution is a sinusoid with the same wavelength (level of detail), i.e., I in * h = C cos(2  x) + D sin(2  x), for some C and D dependent on A, B, and h(x)

Sampling and integration (digital images) Model –Within-pixel integration at all points Has its own IRF, typically rectangular –Then sampling Sampling = multiplication by pixel area  brush function –Brush function is sum of impulses at pixel centers –Sampling = aliasing: in sinusoidal decomposition higher frequency components masquerading as and thus polluting lower frequency components –Nyquist frequency: how finely to sample to have adequately low effect of aliasing

The eye and retina

Retina 120e6 rod cells (scotopic vision) 6e6 cone cells (photopic vision) Sensors operate by polarization of proteins by photons Produce a “pulse train” with rate proportional to log of intensity

Rods and cones density in the retina

Dynamic Range 11 orders of magnitude! Single photons when dark adapted! Scotopic in the dark Photopic in the light Maybe only 20 levels at one point Maybe only 1000 levels at one average brightness

Visual Performance 20:20 vision corresponds to 1 arc minute Fovea: 20 minutes max density uniform, 2 degrees “rod-free” area 400 to 700 nanometer wavelength 510nm maximum rod sensitivity (green) 560nm maximum cone sensitivity (orange)

Contrast Sensitivity

A Model of Human Vision with Limited Feedback

Receptive Fields

Perception of Brightness Affected strongly by boundariness signals of form system Determined by relative intensity changes –Weber’s law: just noticeable difference constant  I/I –Averaging within boundaries –Enhancement/sharpening at boundaries Mach effect

Weber’s Law Just noticeable differences

“Simultaneous Contrast” -- Brightness Determined by Relative Luminance

Fourier Transform Pairs and how to use them in reasoning Interpretation of DFT as a matrix Computing FT in N dimensions Lowest and highest frequencies present Convolution Effects of non-linearity Resizing images Sampling and Reconstruction Physical origins of bandlimits

More FT Apply what we learned to some other problem –What is the FT of a Gabor function? –What is the impulse response of a low-pass filter? How about a band-pass filter? –Suppose you wanted to estimate which note on a piano keyboard best corresponds to a signal? –What is the likely effect of those on-center/off- surround receptive fields in the visual field on our visual sensitivity to varying frequencies?

The exam Wednesday 25 October Open books, notes, etc. You can use a calculator or computer but no communication with other people. Bring your own paper and pencil. Write legibly! Pledge your paper. Don’t Panic!