1. What are Good Apertures for Defocus Deblurring? Columbia University ICCP 2009, San Francisco Changyin Zhou Shree K. Nayar

2. Geometry of Defocus Lens Object Sensor Focus Plane Aperture Pattern PSF

3. Formulation of Defocus Focused Image PSFImage Noise Captured Image In the Spatial Domain

4. Formulation of Defocus Focused Image PSFImage Noise Captured Image In the Fourier Domain

5. Various Aperture Coding Techniques [Veeraraghavan et al. 2007] [Levin et al. 2007] [Welford 1960] Annular Pattern [Gottesman 1989] MURA Pattern [Dowski 1993] Wave-front Coding

6. What are Good Apertures for Defocus Deblurring?

7. Outline - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Background - Experiments

8. How to Evaluate Aperture Patterns? Focused Image Captured Image An Optimal Deblurring Algorithm Deblurred Image Natural image Gaussian white noise Evaluate K using expected quality of deblurred image Basic idea:

9. How to Evaluate Aperture Patterns? Focused Image Captured Image An Optimal Deblurring Algorithm Deblurred Image Natural image Gaussian white noise Linear system

10. The Optimal Linear Deblurring Algorithm For white noise and L 2 distance, Weiner filter is optimal Optimize C by minimizing the expected recovery error

11. The Optimal Linear Deblurring Algorithm A variant of Weiner deconvolution 1/f law : Expected power spectrum of natural images : Gaussian noise level

12. How to Evaluate Aperture Patterns? Focused Image Captured Image An Optimal Deblurring Algorithm Deblurred Image Natural image Gaussian white noise Linear system Evaluation Function:

13. Evaluation Criterion for Aperture Pattern Expected deblurring quality at frequency R(K | ) : Expected deblurring quality at noise level. Noise Level Frequency Prior Power Spectrum

14. Noise Level Evaluate Patterns Using the Criterion R Curve of Circular Pattern Noise Level R

15. Evaluate Patterns Using the Criterion Coded apertures help more when noise level is low. Circular aperture is better when noise level is high. Curves of Relative R: R(K)/R(K 0 ) Noise Level Veeraraghavan Levin Annular Image(binary) Image(gray) Random

16. Outline - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Background - Experiments

17. Pattern Optimization Difficult to solve analytically Difficult to solve analytically Patterns evaluated in the Fourier domain, but strictly constrained in the spatial domain. Patterns evaluated in the Fourier domain, but strictly constrained in the spatial domain. Difficult to do brute force search Difficult to do brute force search For binary patterns of resolution N x N, the number of possible solutions is huge, For binary patterns of resolution N x N, the number of possible solutions is huge, when N = 13, if evaluating one pattern takes 1 millisecond, the brute force search requires 10 45 yrs.

18. Pattern Optimization Pattern Evolution in Genetic Algorithm: 13 x 13 binary patterns; 8 different noise levels 2 nd Run: 3 nd Run: σ =0.0001 σ =0.001 σ =0.002 σ =0.005 σ =0.008 σ =0.01 σ =0.02 σ =0.03

19. Evaluate the Optimized Patterns

20. Relative R curves of the Optimized Patterns CircularVeeraraghavan σ= 0.001 Noise Level Image

21. Outline - Evaluation Criterion for Aperture Patterns - Pattern Optimization - Background - Experiments

22. Implementation Precision Laser Photoplot (1 micron)

23. Implementation Canon EF 50mm f/1.8 Lens

24. Implementation Image Pattern Levin et al.’s Pattern Our Optimized Pattern Circular Pattern (wide open) Veeraraghavan et al.’s Pattern

25. Comparison Experiments on a CZP Chart Focused Image Captured Images Circular Pattern Levin et al.’s Pattern Veeraraghavan et al’s PatternImage Pattern Our Optimized Pattern

26. Comparison Experiments on a CZP Chart Deblurred Images Focused ImageCircular PatternLevin et al.’s Pattern Veeraraghavan et al’s PatternImage Pattern Our Optimized Pattern

27. Shrek Captured Image using the Optimized Pattern

28. Shrek Captured Image using the Optimized Pattern Deblurring Result

29. On the Street Captured Image Using the Optimized Pattern

30. On the Street Captured Image Using the Optimized Pattern Deblurring Result

31. Traffic Scene Captured Image Using the Optimized Pattern

32. Traffic Scene Captured Image Using the Optimized Pattern Deblurring Result

33. Summary Main Contributions Aperture Evaluation Criterion for Defocus Deblurring Aperture Evaluation Criterion for Defocus Deblurring Aperture Pattern Optimization for Defocus Deblurring Aperture Pattern Optimization for Defocus Deblurring Future Work Optimize gray-level patterns Optimize gray-level patterns Apply Criterion to other PSF Engineering Problems Apply Criterion to other PSF Engineering Problems Extend Analysis to Account for Diffraction Extend Analysis to Account for Diffraction

34. Thank You!

35. Genetic Algorithm for Pattern Optimization

36. Noise level = 0.005 1 st Run: G = 1G = 10G = 20G = 40G = 60G = 80 2 nd Run: 3 nd Run: G = 1G = 10G = 20G = 40G = 60G = 80 G = 1G = 10G = 20G = 40G = 60G = 80