What are Good Apertures for Defocus Deblurring? Columbia University ICCP 2009, San Francisco Changyin Zhou Shree K. Nayar

Geometry of Defocus Lens Object Sensor Focus Plane Aperture Pattern PSF

Formulation of Defocus Focused Image PSFImage Noise Captured Image In the Spatial Domain

Formulation of Defocus Focused Image PSFImage Noise Captured Image In the Fourier Domain

Various Aperture Coding Techniques [Veeraraghavan et al. 2007] [Levin et al. 2007] [Welford 1960] Annular Pattern [Gottesman 1989] MURA Pattern [Dowski 1993] Wave-front Coding

What are Good Apertures for Defocus Deblurring?

Outline - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Background - Experiments

How to Evaluate Aperture Patterns? Focused Image Captured Image An Optimal Deblurring Algorithm Deblurred Image Natural image Gaussian white noise Evaluate K using expected quality of deblurred image Basic idea:

How to Evaluate Aperture Patterns? Focused Image Captured Image An Optimal Deblurring Algorithm Deblurred Image Natural image Gaussian white noise Linear system

The Optimal Linear Deblurring Algorithm For white noise and L 2 distance, Weiner filter is optimal Optimize C by minimizing the expected recovery error

The Optimal Linear Deblurring Algorithm A variant of Weiner deconvolution 1/f law : Expected power spectrum of natural images : Gaussian noise level

How to Evaluate Aperture Patterns? Focused Image Captured Image An Optimal Deblurring Algorithm Deblurred Image Natural image Gaussian white noise Linear system Evaluation Function:

Evaluation Criterion for Aperture Pattern Expected deblurring quality at frequency R(K | ) : Expected deblurring quality at noise level. Noise Level Frequency Prior Power Spectrum

Noise Level Evaluate Patterns Using the Criterion R Curve of Circular Pattern Noise Level R

Evaluate Patterns Using the Criterion Coded apertures help more when noise level is low. Circular aperture is better when noise level is high. Curves of Relative R: R(K)/R(K 0 ) Noise Level Veeraraghavan Levin Annular Image(binary) Image(gray) Random

Outline - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Background - Experiments

Pattern Optimization Difficult to solve analytically Difficult to solve analytically Patterns evaluated in the Fourier domain, but strictly constrained in the spatial domain. Patterns evaluated in the Fourier domain, but strictly constrained in the spatial domain. Difficult to do brute force search Difficult to do brute force search For binary patterns of resolution N x N, the number of possible solutions is huge, For binary patterns of resolution N x N, the number of possible solutions is huge, when N = 13, if evaluating one pattern takes 1 millisecond, the brute force search requires yrs.

Pattern Optimization Pattern Evolution in Genetic Algorithm: 13 x 13 binary patterns; 8 different noise levels 2 nd Run: 3 nd Run: σ = σ =0.001 σ =0.002 σ =0.005 σ =0.008 σ =0.01 σ =0.02 σ =0.03

Evaluate the Optimized Patterns

Relative R curves of the Optimized Patterns CircularVeeraraghavan σ= Noise Level Image

Outline - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Background - Experiments

Implementation Precision Laser Photoplot (1 micron)

Implementation Canon EF 50mm f/1.8 Lens

Implementation Image Pattern Levin et al.’s Pattern Our Optimized Pattern Circular Pattern (wide open) Veeraraghavan et al.’s Pattern

Comparison Experiments on a CZP Chart Focused Image Captured Images Circular Pattern Levin et al.’s Pattern Veeraraghavan et al’s PatternImage Pattern Our Optimized Pattern

Comparison Experiments on a CZP Chart Deblurred Images Focused ImageCircular PatternLevin et al.’s Pattern Veeraraghavan et al’s PatternImage Pattern Our Optimized Pattern

Shrek Captured Image using the Optimized Pattern

Shrek Captured Image using the Optimized Pattern Deblurring Result

On the Street Captured Image Using the Optimized Pattern

On the Street Captured Image Using the Optimized Pattern Deblurring Result

Traffic Scene Captured Image Using the Optimized Pattern

Traffic Scene Captured Image Using the Optimized Pattern Deblurring Result

Summary Main Contributions Aperture Evaluation Criterion for Defocus Deblurring Aperture Evaluation Criterion for Defocus Deblurring Aperture Pattern Optimization for Defocus Deblurring Aperture Pattern Optimization for Defocus Deblurring Future Work Optimize gray-level patterns Optimize gray-level patterns Apply Criterion to other PSF Engineering Problems Apply Criterion to other PSF Engineering Problems Extend Analysis to Account for Diffraction Extend Analysis to Account for Diffraction

Thank You!

Genetic Algorithm for Pattern Optimization

Noise level = st Run: G = 1G = 10G = 20G = 40G = 60G = 80 2 nd Run: 3 nd Run: G = 1G = 10G = 20G = 40G = 60G = 80 G = 1G = 10G = 20G = 40G = 60G = 80