recent advances in image deblurring
TRANSCRIPT
![Page 1: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/1.jpg)
Recent Advances in Image Deblurring
Seungyong Lee(Collaboration w/ Sunghyun Cho)
![Page 2: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/2.jpg)
Disclaimer
• Many images and figures in this course note have been copied from the papers and presentation materials of previous deblurring and deconvolution methods.
• In those cases, the original papers are cited in the slides.
![Page 3: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/3.jpg)
IntroductionBlind Deconvolution
![Page 4: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/4.jpg)
blur [bl3:(r)]• Long exposure• Moving objects• Camera motion
– panning shot
![Page 5: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/5.jpg)
blur [bl3:(r)]• Often degrades image/video
quality severely• Unavoidable under dim light
circumstances
![Page 6: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/6.jpg)
Various Kinds of Blurs
Camera shake (Camera motion blur)
Out of focus (Defocus blur) Combinations (vibration & motion, …)
Object movement (Object motion blur)
![Page 7: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/7.jpg)
Camera Motion Blur• Caused by camera shakes during
exposure time– Motion can be represented as a
camera trajectory
![Page 8: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/8.jpg)
Object Motion Blur• Caused by object motions during exposure time
![Page 9: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/9.jpg)
Defocus Blur• Caused by the limited depth of field of a camera
![Page 10: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/10.jpg)
Optical Lens Blur • Caused by lens aberration
![Page 11: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/11.jpg)
Deblurring?• Remove blur and restore a latent sharp image
from a given blurred image find its latent sharp image
![Page 12: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/12.jpg)
Deblurring: Old Problem!• Trott, T., “The Effect of Motion of Resolution”,
Photogrammetric Engineering, Vol. 26, pp. 819-827, 1960.• Slepian, D., “Restoration of Photographs Blurred by Image Motion”,
Bell System Tech., Vol. 46, No. 10, pp. 2353-2362, 1967.
![Page 13: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/13.jpg)
Why is it important?• Image/video in our daily lives
– Sometimes a retake is difficult!
![Page 14: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/14.jpg)
Why is it important?• Strong demand for high quality deblurring
CCTV, car black box Medical imaging Aerial/satellite photography
Robot vision
![Page 15: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/15.jpg)
Deblurring
from a given blurred image find its latent sharp image
![Page 16: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/16.jpg)
Commonly Used Blur Model
= *
Blurred image Latent sharp imageBlur kernel
or Point Spread Function (PSF)
Convolution operator
![Page 17: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/17.jpg)
Blind Deconvolution
= *
Blurred image Latent sharp imageBlur kernel
or Point Spread Function (PSF)
Convolution operator
![Page 18: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/18.jpg)
Non-blind Deconvolution
= *
Blurred image Latent sharp imageBlur kernel
or Point Spread Function (PSF)
Convolution operator
![Page 19: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/19.jpg)
Uniform vs. Non-uniform BlurUniform blur• Every pixel is blurred in the
same way• Convolution based blur model
![Page 20: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/20.jpg)
Uniform vs. Non-uniform BlurNon-uniform blur• Spatially-varying blur• Pixels are blurred differently• More faithful to real camera
shakes
![Page 21: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/21.jpg)
Most Blurs Are Non-Uniform
Camera shake (Camera motion blur)
Out of focus (Defocus blur) Combinations (vibration & motion, …)
Object movement (Object motion blur)
![Page 22: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/22.jpg)
Introduction
Blind Deconvolution
![Page 23: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/23.jpg)
Introduction
Blind Deconvolution
• Introduction• Recent popular
approaches• Summary
![Page 24: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/24.jpg)
Blind Deconvolution (Uniform Blur)
= *
Blurred image Latent sharp imageBlur kernel
or Point Spread Function (PSF)
Convolution operator
![Page 25: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/25.jpg)
Key challenge: Ill-posedness!
Blurred image
Possible solutions• Infinite number of solutions
satisfy the blur model
• Analogous to
1002 504 25
3 33.333…
*
*
*
=
![Page 26: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/26.jpg)
• Parametric blur kernels– [Yitzhakey et al. 1998], [Rav-Acha and Peleg 2005], …– Directional blur kernels defined by (length, angle)
In The Past…
*
![Page 27: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/27.jpg)
In The Past…• But real camera shakes are much more complex
![Page 28: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/28.jpg)
In The Past…• Parametric blur kernels
– Very limited assumption– Often failed, poor quality
Blurred image Latent sharp image* Images from [Yitzhaky et al. 1998]
![Page 29: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/29.jpg)
Nowadays…• Some successful approaches have been introduced…
– [Fergus et al. SIGGRAPH 2006], [Shan et al. SIGGRAPH 2008],[Cho and Lee, SIGGRAPH Asia 2009], …
– More realistic blur kernels– Better quality– More robust
• Commercial software– Photoshop CC Shake reduction
![Page 30: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/30.jpg)
Introduction
Blind Deconvolution
• Introduction
• Recent popular approaches
• Summary
![Page 31: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/31.jpg)
Recent Popular ApproachesMaximum Posterior (MAP) based
Variational Bayesian based
Edge Prediction based
![Page 32: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/32.jpg)
Recent Popular ApproachesMaximum Posterior (MAP) based
Variational Bayesian based
Edge Prediction based
• [Shan et al. SIGGRAPH 2008],[Krishnan et al. CVPR 2011],[Xu et al. CVPR 2013], …
• Seek the most probable solution, which maximizes a posterior distribution
• Easy to understand• Convergence problem
![Page 33: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/33.jpg)
Recent Popular ApproachesMaximum Posterior (MAP) based
Variational Bayesian based
Edge Prediction based
• [Fergus et al. SIGGRAPH 2006],[Levin et al. CVPR 2009],[Levin et al. CVPR 2011], …
• Not seek for one most probable solution, but consider all possible solutions
• Theoretically more robust• Slow
![Page 34: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/34.jpg)
Recent Popular ApproachesMaximum Posterior (MAP) based
Variational Bayesian based
Edge Prediction based
• [Cho et al. SIGGRAPH Asia 2009],[Xu et al. ECCV 2010],[Hirsch et al. ICCV 2011], …
• Explicitly try to recover sharp edges using heuristic image filters
• Fast• Proven to be effective in practice,
but hard to analyze because of heuristic steps
![Page 35: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/35.jpg)
Recent Popular ApproachesMaximum Posterior (MAP) based
Variational Bayesian based
Edge Prediction based
• [Shan et al. SIGGRAPH 2008],[Krishnan et al. CVPR 2011],[Xu et al. CVPR 2013], …
• Seek the most probable solution, which maximizes a posterior distribution
• Easy to understand• Convergence problem
![Page 36: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/36.jpg)
Maximize joint posterior probability with respect to and
MAP based Approaches
Blur kernel Latent image Blurred image
Posterior distribution
![Page 37: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/37.jpg)
Bayes rule:
MAP based Approaches
Posterior distribution Likelihood Prior on Prior on
Blur kernel Latent image Blurred image
![Page 38: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/38.jpg)
Negative log-posterior:
MAP based Approaches
Regularization on blur kernel Data fitting term Regularization on
latent image
![Page 39: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/39.jpg)
Negative log-posterior:
Alternatingly minimize the energy function w.r.t. and
MAP based Approaches
Regularization on blur kernel Data fitting term Regularization on
latent image
![Page 40: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/40.jpg)
Negative log-posterior:
Alternatingly minimize the energy function w.r.t. and Ill-posedness:• Data fitting term has several solutions• Thus, and are very important for resolving the ill-posedness!
MAP based Approaches
Regularization on blur kernel Data fitting term Regularization on
latent image
![Page 41: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/41.jpg)
MAP based Approaches
Input blurred image
Latent image estimation
- maximizes posterior w.r.t.
Blur kernel estimation
- maximizes posterior w.r.t.
Output
![Page 42: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/42.jpg)
MAP based Approaches• Chan and Wong, TIP 1998
– Total variation based prior for estimating a parametric blur kernel• Shan et al. SIGGRAPH 2008
– First MAP based method to estimate a nonparametric blur kernel• Krishnan et al. CVPR 2011
– Normalized sparsity measure, a novel prior on latent images• Xu et al. CVPR 2013
– L0 norm based prior on latent images
![Page 43: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/43.jpg)
Shan et al. SIGGRAPH 2008• Carefully designed likelihood, priors & optimization methods
Natural image statistics based
prior on
Likelihood based on intensities & derivatives
Kernel statistics based prior on
![Page 44: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/44.jpg)
Shan et al. SIGGRAPH 2008• A few minutes for a small image• High-quality results
![Page 45: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/45.jpg)
Shan et al. SIGGRAPH 2008• Convergence problem
– Often converge to the no-blur solution [Levin et al. CVPR 2009]– Natural image priors prefer blurry images
Shan et al. SIGGRAPH 2008 Fergus et al. SIGGRAPH 2006(variational Bayesian based)
Ground truth
![Page 46: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/46.jpg)
Xu et al. CVPR 2013• norm based sparse prior for latent image
, ∝ ,norm based sparse prior on
• No natural prior, i.e., does not seek for naturally-looking latent images
• But, unnatural images with a few sharp edges
• Better for resolving the ill-posedness
Latent image estimated by prior
Blurred image
![Page 47: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/47.jpg)
Xu et al. CVPR 2013• Better prior & sophisticated optimization methods better convergence & better quality
![Page 48: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/48.jpg)
Recent Popular ApproachesMaximum Posterior (MAP) based
Variational Bayesian based
Edge Prediction based
• [Fergus et al. SIGGRAPH 2006],[Levin et al. CVPR 2009],[Levin et al. CVPR 2011], …
• Not seek for one most probable solution, but consider all possible solutions
• Theoretically more robust• Slow
![Page 49: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/49.jpg)
Variational Bayesian• MAP
– Find the most probable solution
– May converge to a wrong solution
• Variational Bayesian– Approximate the
underlying distribution and find the mean
– More stable– Slower
VariationalBayes
Maximum a-Posteriori (MAP)
Pixel intensity
Scor
e
MAP v.s. Variational Bayes
![Page 50: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/50.jpg)
Variational Bayesian• Fergus et al. SIGGRAPH 2006
– First approach to handle non-parametric blur kernels• Levin et al. CVPR 2009
– Show that variational Bayesian approaches can perform more robustly than MAP based approaches
• Levin et al. CVPR 2010– EM based efficient approximation to variational Bayesian approach
![Page 51: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/51.jpg)
Fergus et al. SIGGRAPH 2006• Posterior distribution
![Page 52: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/52.jpg)
Fergus et al. SIGGRAPH 2006– find an approximate distribution by minimizing Kullback-Leibler (KL)
divergence
– cf) MAP based approach:
approximate distributions for blur kernel , latent image , and noise variance
, ,
,
![Page 53: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/53.jpg)
Fergus et al. SIGGRAPH 2006• First method to estimate a nonparametric blur kernel• Complex optimization• Slow: more than an hour for a small image
![Page 54: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/54.jpg)
Levin et al. CVPR 2010• Efficient optimization based on EM
Marginalizing over
![Page 55: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/55.jpg)
Levin et al. CVPR 2010
Inputblurred image
E-stepmean &
covariance of
M-stepupdate using
mean & covariance of
Outputmean of
Similar to MAP, but also considers covariance of
![Page 56: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/56.jpg)
Levin et al. CVPR 2010State-of-the-art results
Speed:- 255x255- 2-4 minutes- MATLAB
![Page 57: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/57.jpg)
Recent Popular ApproachesMaximum Posterior (MAP) based
Variational Bayesian based
Edge Prediction based
• [Cho et al. SIGGRAPH Asia 2009],[Xu et al. ECCV 2010],[Hirsch et al. ICCV 2011], …
• Explicitly try to recover sharp edges using heuristic image filters
• Fast• Proven to be effective in practice,
but hard to analyze because of heuristic steps
![Page 58: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/58.jpg)
Edge Prediction based Approaches• Joshi et al. CVPR 2008
– Proposed sharp edge prediction to estimate blur kernels– No iterative estimation– Limited to small scale blur kernels
• Cho & Lee, SIGGRAPH Asia 2009– Proposed sharp edge prediction to estimate large blur kernels– Iterative framework– State-of-the-art results & very fast
• Cho et al. CVPR 2010– Applied Radon transform to estimate a blur kernel from blurry edge profiles– Small scale blur kernels
• Xu et al. ECCV 2010– Proposed a prediction scheme based on structure scales as well as gradient magnitudes
• Hirsch et al. ICCV 2011– Applied a prediction scheme to estimate spatially-varying camera shakes
![Page 59: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/59.jpg)
Cho & Lee, SIGGRAPH Asia 2009• Key idea: blur can be estimated from a few edges No need to restore every detail for kernel estimation
Blurred image Latent image with only a few edges and no texture
![Page 60: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/60.jpg)
Cho & Lee, SIGGRAPH Asia 2009
Input Simpledeconvolution
FastKernel
Estimation
OutputPrediction
Quickly restore important edgesusing simple image filters
![Page 61: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/61.jpg)
Cho & Lee, SIGGRAPH Asia 2009
Fast but low quality deconvolution Prediction
Updated kernelPrevious kernel
![Page 62: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/62.jpg)
Cho & Lee, SIGGRAPH Asia 2009
PredictionSimple & fast image filtering operations
Thresholdinggradients
Bilateral filtering &Shock filtering
Fast but low-quality deconvolution
Visualized by Poisson image reconstruction
![Page 63: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/63.jpg)
Cho & Lee, SIGGRAPH Asia 2009
Blurry input Deblurring result Blur kernel
• State of the art results• A few seconds• 1Mpix image• in C++
![Page 64: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/64.jpg)
Xu & Jia, ECCV 2010• Extended edge prediction to handle blur larger than image structures
Blurred image Fergus et al.SIGGRAPH 2006
Shan et al.SIGGRAPH 2008
For this complex scene, most methods fail to estimate a correct blur kernel.Why?
![Page 65: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/65.jpg)
Xu & Jia, ECCV 2010
Blur > structures• Hard to tell which blur
is caused by which edge
• Most method fails
Blur < structures• Each blurry pixel is
caused by one edge• Easy to estimate a
blur kernel
![Page 66: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/66.jpg)
Xu & Jia, ECCV 2010
Structure scale aware gradient
thresholding
Smoothing &Shock filtering
Deconvolution
Visualized by Poisson image reconstruction
![Page 67: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/67.jpg)
Xu & Jia, ECCV 2010
Blurred image Fergus et al.SIGGRAPH 2006
Shan et al.SIGGRAPH 2008
Xu & Jia, ECCV 2010
![Page 68: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/68.jpg)
Recent Popular ApproachesMaximum Posterior (MAP) based
Variational Bayesian based
Edge Prediction based
Which one is better?
![Page 69: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/69.jpg)
Benchmarks• Many different methods…• Which one is the best?
– Quality– Speed
• Different works report different benchmark results– Depending on test data– Levin et al. CVPR 2009, 2010– Köhler et al. ECCV 2012
![Page 70: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/70.jpg)
Benchmarks• Levin et al. CVPR 2009
– Provide a dataset• 32 test images• 4 clear images (255x255)• 8 blur kernels (10x10 ~ 25x25)• One of the most popular datasets
– Evaluate blind deconvolutionmethods using the dataset
![Page 71: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/71.jpg)
Benchmarks• Levin et al. CVPR 2009
– Cumulative histogram of error ratios– Higher ⇔ better– Results
• Fergus et al. SIGGRAPH 2006• Shan et al. SIGGRAPH 2008
1.5 2 2.5 3 3.5 40
0.2
0.4
0.6
0.8
1
1.2
1.4
1.6
FergusShanNaiv MAPxkGaussian prior
Succ
esse
s pe
rcen
t
100
80
60
40
20
Error ratios
MAPk, Gaussian prior
Shan et al. SIGGRAPH08Fergus, variational MAPk
MAPx,k sparse prior
![Page 72: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/72.jpg)
• Levin et al. CVPR 2010– Cumulative histogram of error ratios– Higher ⇔ better– Results
• Levin et al. CVPR 2010• Fergus et al. SIGGRAPH 2006• Cho & Lee SIGGRAPH Asia 2009
Benchmarks
Proposed methods by [Levin et al. CVPR 2010]
![Page 73: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/73.jpg)
Benchmarks• Köhler et al. ECCV 2012
– Record and analyze real camera motions• Recorded 6D camera shakes in the 3D
space using markers• Played back camera shakes using a robot
arm– Provide a benchmark dataset based on
real camera shakes– Provide benchmark results for recent
state-of-the-art methods
![Page 74: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/74.jpg)
Benchmarks• Köhler et al. ECCV 2012
– Dataset• 48 test images• 4 sharp images• 12 non-uniform blur kernels
![Page 75: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/75.jpg)
Benchmarks• Köhler et al. ECCV 2012
– Benchmark based on PSNR– Results
• Xu & Jia, ECCV 2010• Cho & Lee, SIGGRAPH Asia 2009• Shan et al. SIGGRAPH 2008• Krishnan et al. CVPR 2011• Fergus et al. SIGGRAPH 2006
PSN
R (d
B)MAP Edge prediction
20
22
24
26
28
30
Blurred Shan et al.SIGGRAPH
2008
Krishnanet al. CVPR
2011
Fergus etal.
SIGGRAPH2006
Cho & Lee,SIGGRAPHAsia 2009
Xu & Jia,ECCV 2010
VariationalBayesian
![Page 76: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/76.jpg)
Benchmarks• All three approaches show similar performance
– Implementation details & tricks– Benchmark datasets– Parameters used in benchmarks
• But, in general, more recent one shows better quality
• Speed?– Edge prediction > MAP >> Variational Bayesian
![Page 77: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/77.jpg)
Introduction
Blind Deconvolution
• Introduction• Recent popular
approaches
• Summary
![Page 78: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/78.jpg)
Summary• Blind deconvolution
– Severely ill-posed problem• Different approaches
– MAP, Variational Bayesian, Edge prediction• Performance
– Quality: similar– Speed: Edge prediction > MAP >> Variational Bayesian
• Still challenging
![Page 79: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/79.jpg)
Remaining Challenges• All methods still fail quite often• Noise• Outliers• Non-uniform blur• Limited amount of edges• Speed…• Etc…
Failure example of Photoshop Shake Reduction
![Page 80: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/80.jpg)
Photoshop Shake Reduction• Based on [Cho and Lee, SIGGRAPH ASIA 2009]• Improved noise handling• Automatic kernel size estimation• Automatic region suggestion
for blur kernel estimation
• DEMO
![Page 81: Recent Advances in Image Deblurring](https://reader030.vdocuments.us/reader030/viewer/2022012803/61bd20a961276e740b0fa2b6/html5/thumbnails/81.jpg)
Q & Ahttp://cg.postech.ac.kr