Download presentation

Presentation is loading. Please wait.

Published byLillian Gomez Modified over 3 years ago

1
Removing blur due to camera shake from images. William T. Freeman Joint work with Rob Fergus, Anat Levin, Yair Weiss, Fredo Durand, Aaron Hertzman, Sam Roweis, Barun Singh Massachusetts Institute of Technology

2
Overview Original Our algorithm

3
Close-up Original Naïve SharpeningOur algorithm

4
Lets take a photo Blurry result

5
Slow-motion replay

6
Motion of camera

7
Image formation process = Blurry image Sharp image Blur kernel Input to algorithm Desired output Convolution operator Model is approximation

8
Why is this hard? Simple analogy: 11 is the product of two numbers. What are they? No unique solution: 11 = 1 x 11 11 = 2 x 5.5 11 = 3 x 3.667 etc….. Need more information !!!!

9
Multiple possible solutions = Blurry image Sharp image Blur kernel = =

10
Is each of the images that follow sharp or blurred?

14
Another blurry one

17
Natural image statistics Histogram of image gradients Characteristic distribution with heavy tails

18
Blury images have different statistics Histogram of image gradients

19
Parametric distribution Histogram of image gradients Use parametric model of sharp image statistics

20
Uses of natural image statistics Denoising [Roth and Black 2005] Superresolution [Tappen et al. 2005] Intrinsic images [Weiss 2001] Inpainting [Levin et al. 2003] Reflections [Levin and Weiss 2004] Video matting [Apostoloff & Fitzgibbon 2005] Corruption process assumed known

21
Existing work on image deblurring Software algorithms: – Extensive literature in signal processing community – Mainly Fourier and/or Wavelet based – Strong assumptions about blur not true for camera shake – Image constraints are frequency-domain power-laws Assumed forms of blur kernels

23
A focus on image constraints, not image priors

24
Some image constraints/priors

25
Toy example: observed image: 1.00.0

26
1.00.0 Toy example: observed image:

27
1.00.0

28
Three sources of information 1. Reconstruction constraint: = Input blurry image Estimated sharp image Estimated blur kernel 3. Blur prior: Positive & Sparse 2. Image prior: Distribution of gradients

29
Prior on image gradients (mixture of Gaussians giving a Laplacian-like distribution) Distribution of gradients (log-scale) Green curve is our mixture of gaussians fit.

30
Prior on blur kernel pixels (mixture of exponentials) b P(b)

31
How do we use this information? Obvious thing to do: – Combine 3 terms into an objective function – Run conjugate gradient descent – This is Maximum a-Posteriori (MAP)

32
Maximum A-Posteriori y – observed blurry image x – unobserved sharp image b – blur kernel i – image patch index f – derivative filter Likelihood Latent image prior Blur prior Assumption: all pixels independent of one another Sparse and

33
Results from MAP estimation Maximum a-Posteriori (MAP) Our method: Variational Bayes Input blurry image

34
Variational Bayes http://citeseer.ist.psu.edu/cache/papers/cs/16537/http:zSzzSzwol.ra.phy.cam.ac.ukzSzjwm1003zSzspringer_chapter8.pdf/miskin00ensemble.pdf

35
Miskin and Mackay, 2000

36
Setup of variational approach Need likelihood and prior in same space, so use gradients: Likelihood Prior on latent image gradients – mixture of Gaussians Prior on blur elements – mixture of Exponentials i – image pixel j – blur pixel We use C=4, D=4 Also have Gamma hyperpriors on

37
Variational inference Approximate posterior with Cost function Assume Use gradient descent, alternating between updating while marginalizing out over and vice versa Adapted code from Miskin & Mackay 2000 is Gaussian on each pixel is rectified Gaussian on each pixel

38
Variational Bayesian method Based on work of Miskin & Mackay 2000 Keeps track of uncertainty in estimates of image and blur by using a distribution instead of a single estimate Helps avoid local maxima and over-fitting

39
Variational Bayes Variational Bayesian method Maximum a-Posteriori (MAP) Pixel intensity Score Objective function for a single variable

40
MAP vs Variational MAP Variational MAP using variational initialization

41
Blurry synthetic image

42
Inference – initial scale

43
Inference – scale 2

44
Inference – scale 3

45
Inference – scale 4

46
Inference – scale 5

47
Inference – scale 6

48
Inference – final scale

49
Our output

50
Ground truth

51
Matlabs deconvblind

52
True kernel Estimated kernel

53
Tried the same algorithm on an image of real camera blur, with very similar blur kernel Failure!

54
Whiteboard scene: Does camera shake give a stationary blur kernel? 8 different people, handholding camera, using 1 second exposure

55
View of the dots at each corner in photos taken by 4 people Top left Bot. left Top right Bot. right Person 1 Person 3 Person 4 Person 2

56
Tonescale: output from camera

57
Linear response to light intensities looks like this

58
Overview of algorithm Input image 1. Pre-processing 2. Kernel estimation - Multi-scale approach 3. Image reconstruction -Standard non-blind deconvolution routine

59
Preprocessing Convert to grayscale Input image Remove gamma correction User selects patch from image Bayesian inference too slow to run on whole image Infer kernel from this patch

60
Initialization Input image Initialize 3x3 blur kernel Initial blur kernel Blurry patch Initial image estimate Convert to grayscale Remove gamma correction User selects patch from image

Similar presentations

© 2017 SlidePlayer.com Inc.

All rights reserved.

Ads by Google