1. A Comparative Study for Single Image Blind Deblurring
Wei-Sheng Lai
UC Merced
Jia-Bin Huang
UIUC
Zhe Hu
UC Merced
Narendra Ahuja
UIUC
Ming-Hsuan Yang
UC Merced
Hi everyone. My name is Wei-Sheng Lai. Today I am going to talk about a comparative study for single image blind deblurring.

2. Single Image Blind Deblurring
Algorithms:
Datasets:
Real images:
Fergus et al. 2006
Shan et al. 2008
Cho & Lee 2009
Krishnan et al. 2011
Whyte et al. 2011
Hirsch et al. 2011
Xu et al. 2013
Zhong et al. 2013
Sun et al. 2013
Michaeli et al. 2014
Pan et al. 2014
Perrone et al. 2014
Levin et al. 2009
However, real-world blurred images are much more complicate. For example, the scene depth variation, non-linear camera response functions, saturation and compression artifacts are common factors in real images but not considered in previous datasets.
Kohler et al. 2012
Sun et al. 2013
Depth variation
Camera response functions
Saturation
Compression artifacts

3. Our Goal Performance evaluation on real-world blurred images
a dataset of real images
large scale comparative study
In this work, we provide the first large scale performance evaluation of blind motion deblurring algorithms on real-world blurred images.
We first collect a dataset of 100 real blurred images from different sources.// such as Google, Flickr, and photos captured by ourselves.
Then, we conduct a user study and comparative analysis to understand the relative strengths of state-of-the-art algorithms.

4. User-Study Evaluate on 14 methods, 100 images
14 2 =910 comparisons per image
collect about 100k paired comparisons from 2000 subjects
In our user study, we show two images side by side, and ask users to select which one looks better.
We evaluate totally 14 methods on 100 images.
We collect user votes from around 2000 human subjects (on Amazon Mechanical Turk).

5. From Paired Comparisons to Full Ranking
Fit votes to the Bradley-Terry Model (B-T Model)
𝑀 𝑖𝑗 =#times that users choose method i over method j
𝑆 𝑖 = the B-T score of method i
Cumulative Frequency
𝑀 𝑀 ⋯ 𝑀
B-T Model
To obtain the full ranking of each algorithm, we need convert the pairwise comparisons to real-value scores.
We use the Bradley-Terry model to convert the user votes to the B-T scores, and plot the cumulative frequency of the B-T scores for each method.
//In this figure, the curve on the right has better performance.
𝑆 𝑆 ⋯ 𝑆

6. Comparing Real and Synthetic Datasets
We also conduct the baseline user-study on synthetic datasets.// for both uniform and non-uniform blurred images.
We show the scatter plots of the BT scores between real and synthetic datasets to understand their performance difference.
//Third, we show the scatter plot the BT scores on real and synthetic datasets.
//In these figures, the methods on the lower-right triangle have better performance on the synthetic dataset. But their performance is not as competitive in the real dataset.
//As a result, we say that the performance of those methods on the lower-right are over-estimated on the synthetic dataset.

7. Comparing Image Quality Metrics
We also compute the correlation between user subject scores and several image quality metrics, including full-reference and no-reference metrics.
Full-reference metrics
No-reference metrics

8. Observations 𝛻𝑥 1 𝛻𝑥 2 , 𝑥 0 >
Image priors: sparse priors are more robust than edge prediction methods
Image formations:
Datasets: performance on synthetic datasets does not reflect the performance on real images
Quality metrics: IFC/VIF > PSNR/SSIM; none of no-reference metrics are applicable
𝛻𝑥 𝛻𝑥 2 , 𝑥 0 >
Finally, if you need a quality metric to evaluate the performance on synthetic images, IFC and VIF are better than PSNR and SSIM.
For real blurred images, none of existing no-reference metrics are suitable to evaluate the quality of deblurred results.

9. Conclusions First large scale comparative study on real-world images
quantitatively evaluate the progress of the field
identify potential research directions
Code, datasets and results are available: bit.ly/deblur_study
Poster #22
In conclusion, we provide the first comparative study of blind motion deblurring on real-world blurred images.
We evaluate the progress of this field, and identify potential directions for future research.
Our code and datasets are available on our project website.
If you are interested in more details, please come to our poster at section 22.