image deblurring with optimizations
DESCRIPTION
Image Deblurring with Optimizations. University of Washington The Chinese University of Hong Kong Adobe Systems, Inc. Qi Shan Leo Jiaya Jia Aseem Agarwala. The Problem. 2. An Example. Previous Work (1). Hardware solutions:. [Ben-Ezra and Nayar 2004]. [Levin et al. 2008]. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/1.jpg)
Image Deblurring with Optimizations
Qi ShanQi ShanLeo Jiaya Jia Leo Jiaya Jia Aseem AgarwalaAseem Agarwala
University of WashingtonThe Chinese University of Hong KongAdobe Systems, Inc.
![Page 2: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/2.jpg)
2
The Problem
![Page 3: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/3.jpg)
An Example
![Page 4: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/4.jpg)
4
Previous Work (1)
Hardware solutions:
[Raskar et al. 2006]
[Ben-Ezra and Nayar 2004]
[Levin et al. 2008]
![Page 5: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/5.jpg)
5
Previous Work (2)
Multi-frame solutions:
[Petschnigg et al. 2004]
[Jia et al. 2004] [Rav-Acha and Peleg 2005]
[Yuan et al. 2007]
![Page 6: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/6.jpg)
6
Previous Work (3)
Single image solutions:
[Jia 2007][Fergus et al. 2006]
[Levin et al. 2007]
![Page 7: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/7.jpg)
Most recent work on Single Image Deblurring
Qi Shan, Jiaya Jia, and Aseem AgarwalaHigh-Quality Motion Deblurring From a Single Image. SIGGRAPH 2008
Lu Yuan, Jian Sun, Long Quan and Heung-Yeung ShumProgressive Inter-scale and intra-scale Non-blind Image Deconvolution. SIGGRAPH 2008.
Joshi, N., Szeliski, R. and Kriegman, D. PSF Estimation using Sharp Edge Prediction, CVPR 2008.
A. Levin, Y. Weiss, F. Durand, W. T. Freeman Understanding and evaluating blind deconvolution algorithms. CVPR 2009
Sunghyun Cho and Seungyong Lee, Fast Motion Deblurring.SIGGRAPH ASIA 2009
And many more...
![Page 8: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/8.jpg)
Some take home ideas
1. Using hierarchical approaches to estimate kernel in different scales
2. Realize the importance of strong edges
3. Bilateral filtering to suppress ringing artifacts
4. RL deconvolution is good, but we've got better chioces
5. Stronger prior does a better job
6. Deblurring by assuming spatially variant kernel is a good way to go
![Page 9: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/9.jpg)
Today's topic
How to apply natural image statistics, image local smoothness constraints, and kernel sparsity prior in a MAP process
Short discussion on
1. the stability of a non-blind deconvolution process
2. noise resistant non-blind deconvolution and denoising
![Page 10: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/10.jpg)
10
Image Global Statistics
…
![Page 11: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/11.jpg)
11
…
Image Global Statistics
![Page 12: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/12.jpg)
12
Image Global Statistics
![Page 13: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/13.jpg)
13
LI
Image Local Constraint
![Page 14: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/14.jpg)
14
LI
Image Local Constraint
![Page 15: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/15.jpg)
15
LI
Image Local Constraint
![Page 16: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/16.jpg)
16
LI
12 )( |) ( 0,i ii N dL dIp L
p2
Image Local Constraint
![Page 17: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/17.jpg)
17
exponentially distributed
) ( jfj ep f
Kernel Statistics
![Page 18: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/18.jpg)
18
Combining All constraints
1 2min ( , ) min log[ ( ) ( ) ( ) ( )]E L f p n p dL p L p f
L f n
Two-step iterative optimization• Optimize L• Optimize f
![Page 19: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/19.jpg)
19
Idea: separate convolution
22'( ) \ (|| * || )E L Sum L f I
log ( )p n
Optimize L
1log ( )p dL 2log ( )p L
22 2(\ ( || || ))i i iSum m dL dI
Optimization Process
idLreplace with i
1 1 1|| log ( ) ||p dL
![Page 20: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/20.jpg)
20
22'( ) \ (|| * || )E L Sum L f I
log ( )p n
Idea: separate convolution
Optimize L
1log ( )p dL 2log ( )p L
1 1 1|| lo g ( ) ||p 22 2(\ ( || || ))i iiSum m dI
Optimization Process
idLreplace with i
![Page 21: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/21.jpg)
21
22'( ) \ (|| * || )E L Sum L f I
1 1 1|| log ( ) ||p 22(|| || )dL 2
2 2(\ (|| || ))i iSum dI
22arg min \ (|| * || )opt LL Sum dL f dI 2
2(|| || )dL
Adding a new constraint to makeRemoving terms that are not relevant to
~ dLL
Updating L
An easy quadratic optimization problem with a closed form solution in the frequency domain
![Page 22: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/22.jpg)
22
Updating
Removing terms that are not relevant to
22'( ) \ (|| * || )E Sum L f I
1 1 1|| lo g ( ) ||p 22(|| || )dL 2
2 2(\ (|| || ))i iSum dI
21 1 1 2 2arg min || log ( ) || (\ (|| || ))opt i ip Sum dI
22(|| || )dL
![Page 23: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/23.jpg)
23
each only contains a single variable Ψi'i
E
21 1 1 2 2arg m in || lo g ( ) || (\ ( || || ))op t i ip Sum dI
22(|| || )dL
arg min(\ ( ' ))i
Sum E
It is then a set of easy single variable optimization problems
![Page 24: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/24.jpg)
24
Iteration 0 (initialization)
![Page 25: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/25.jpg)
25
Time: about 30 seconds for an 800x600 image
Iteration 8 (converge)
![Page 26: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/26.jpg)
26
A comparison
RL deconvolution
![Page 27: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/27.jpg)
27
A comparison
Our deconvolution
![Page 28: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/28.jpg)
28
Two-step iterative optimization• Optimize L• Optimize f
22 1( ) || * || || ||E f L f I f
1 2min ( , ) min log[ ( ) ( ) ( ) ( )]E L f p n p dL p L p f
Optimization with a total variation regularization
![Page 29: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/29.jpg)
29
Results
![Page 30: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/30.jpg)
30
Results
![Page 31: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/31.jpg)
31
![Page 32: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/32.jpg)
32
![Page 33: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/33.jpg)
33
More results
![Page 34: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/34.jpg)
34
More results
![Page 35: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/35.jpg)
Today's topic
How to apply natural image statistics, image local smoothness constraints, and kernel sparsity prior in a MAP process
Short discussion on
1. the stability of a non-blind deconvolution process
2. noise resistant non-blind deconvolution and denoising
![Page 36: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/36.jpg)
Stability
Considering the simplest case: Wiener Filtering
T
T
FX B
F F I
How about if *B B n
**
T
T T
F XX X n
F F I F F I
* *T
T
FX B
F FAnd
![Page 37: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/37.jpg)
Stability
Thus
* 2 2 22 2|| || || ||
PX X C
PP
Pwhere is the frequency domain representation of
is the variance of the noise
Observation: the noise in the blur image is magnified in
the deconvolved image. And the Noise Magnification
Factor (NMF) is solely determined by the filter
F2
F
![Page 38: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/38.jpg)
Some examples
![Page 39: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/39.jpg)
Some examples
Dense kernels are less stable for deconvolution than sparse ones
![Page 40: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/40.jpg)
40
Noise resistant deconvolution and denoising
With Jiaya Jia, Singbing Kang and Zenlu QinIn CVPR 2010
Blind and non-blind image deconvolution softwareis available online and will be updated soon!
See you in San Francisco!
![Page 41: Image Deblurring with Optimizations](https://reader035.vdocuments.us/reader035/viewer/2022081501/56813e78550346895da8993c/html5/thumbnails/41.jpg)