blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... ·...
TRANSCRIPT
![Page 1: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/1.jpg)
[email protected] www.utia.cas.cz
Institute of Information Theory and Automation
Academy of Sciences of the Czech Republic Prague
Digital Image Restoration Blur as a chance and not a nuisance
Filip Šroubek
1 CMP 2017
![Page 2: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/2.jpg)
2 CMP 2017
![Page 3: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/3.jpg)
original image
u
Image Degradation Model
3
channel
convolution
acquired image
= z + n
noise
CMP 2017
![Page 4: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/4.jpg)
Energy Minimization
CMP 2017 4
![Page 5: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/5.jpg)
Data term
Energy Minimization
5 CMP 2017
![Page 6: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/6.jpg)
Data term
Blur regularization
• Blur has different shapes • Compact support • Non-negative • Preserve energy
Energy Minimization
6 CMP 2017
Motion blur
Out-of-focus & Abberations
![Page 7: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/7.jpg)
Image regularization
Energy Minimization
7 CMP 2017
Data term
![Page 8: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/8.jpg)
0 0.2 0.4 0.6 0.8 1
-0.5 0 0.5
Statistics of sharp images
CMP 2017 8
Intensity
Gradient
![Page 9: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/9.jpg)
-0.5 0 0.5
0 0.2 0.4 0.6 0.8 1
Statistics of blurred images
CMP 2017 9
Intensity
Gradient
![Page 10: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/10.jpg)
Regularization
CMP 2017 10
![Page 11: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/11.jpg)
• “NO-BLUR” solution: • WHY? • Regularization favors blurred images
Energy Minimization
11 CMP 2017
![Page 12: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/12.jpg)
Regularization favors blur
CMP 2017 12
5 10 15 200.5
1
1.5
![Page 13: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/13.jpg)
Regularization favors blur
CMP 2017 13
5 10 15 200.5
1
1.5
Artificially sparsify images
![Page 14: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/14.jpg)
Ad-hoc steps!
• To avoid “no-blur” solution:
• Artificial sharpening • Remove spikes • Adjusting priors on the fly • Hierarchical approach
CMP 2017 14
Chan TIP98 Shan SigGraph08 Cho SigGraph09 Xu ECCV09, CVPR13 Almeida TIP10 Krishnan CVPR11 Zhong CVPR13 Sun ICCP13 Michaeli ECCV14 Perrone PAMI15 Pan CVPR16
![Page 15: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/15.jpg)
Artificial Sharpening
15
Shock filter Blur prediction
h-step
Deconvolution u-step
Blurred image
Cho et al., SIGGRAPH 2009
CMP 2017
![Page 16: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/16.jpg)
Remove Spiky Objects
16
Xu et al., ECCV 2010
CMP 2017
![Page 17: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/17.jpg)
Remove Spiky Objects
CMP 2017 17
Mask out small objects
Xu et al., ECCV 2010
![Page 18: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/18.jpg)
Remove Spiky Objects
CMP 2017 18
Reconstructed image with small objects removed
Xu et al., ECCV 2010
![Page 19: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/19.jpg)
Hierarchical Deconvolution
19
Scale N-1
Scale N-2
Scale N
image blur
CMP 2017
![Page 20: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/20.jpg)
Bayesian Paradigm
a posteriori distribution unknown
likelihood given by our problem
a priori distribution our prior knowledge
20 CMP 2017
![Page 21: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/21.jpg)
Blind deconvolution with MAP
21 CMP 2017
![Page 22: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/22.jpg)
Bayesian Paradigm revisited
• Marginalize the posterior
• Maximize the marginalized prob.
• Then maximize the posterior
CMP 2017 22
![Page 23: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/23.jpg)
CMP 2017 23
-0.4-0.6
-0.8-1
-1.21
0.5
0
-0.5
-1
0.5
0.6
0.7
0.8
0.9
1
no-blur solution
correct solution
![Page 24: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/24.jpg)
How to marginalize?
• If Gaussian distributions analytic solution exists in the form of Gaussian distribution
• If not (our case) approximation • Laplace approximation • Factorization with Variational Bayes
CMP 2017 24
![Page 25: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/25.jpg)
Variational Bayes
• Factorization of the posterior
and then marginalization is trivial. • Every factor q depends on moments of other
variables => must be solved iteratively.
CMP 2017 25
Miskin AICA00 Fergus SIGGRAPH06 Tzikas TIP09 Whyte IJCV12 Levin PAMI11 Babacan ECCV12 Wipf JMLR14
![Page 26: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/26.jpg)
Marginalization in blind deconvolution
26 CMP 2017
![Page 27: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/27.jpg)
Limitations of BD
• Gaussian noise
CMP 2017 27
original
SNR=20dB SNR=50dB
![Page 28: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/28.jpg)
Limitations of BD
• Model discrepancies
CMP 2017 28
original
7% discrepancy
![Page 29: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/29.jpg)
Automatic Relevance Determination (Students’ t-distribution) • Standard data term
replaced by
CMP 2017 29
![Page 30: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/30.jpg)
CMP 2017 30
![Page 31: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/31.jpg)
Space-variant Blur
CMP 2017 31
Camera motion
Optical aberrations
Object motion
Scene depth
![Page 32: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/32.jpg)
Approximation of SV Blur
• Patch-wise convolution • General SV PSF • Locally convolution may not hold
• Parametric model (Blur Basis) • More accurate • Model may not hold
• Conversion to space-invariant • HW design
• Local object motion • Segmentation • Line blur
CMP 2017 32
Joshi CVPR08, Sorel ICIP09, Ji CVPR12,Sun CVPR15
Levin NIPS06, Shan ICCV07, Dai CVPR08, Chakrabarti CVPR10, Kim CVPR14
Whyte CVPR10, Gupta ECCV10, Hirsch ICCV11, Zhang NIPS13
Levin SIGGRAPH08, Ben-Ezra PAMI04, Tai PAMI10, Wavefront coding
![Page 33: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/33.jpg)
Fast Moving Objects
• FMO moves over a distance exceeding its size within exposure time
CMP 2017 33
D.R.,J.K.,F.S.,L.N.,J.M. arXiv:1611.07889
![Page 34: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/34.jpg)
Formation model
CMP 2017 34
Alpha matting
![Page 35: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/35.jpg)
FMO Appearance Estimation
CMP 2017 35
![Page 36: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/36.jpg)
FMO Localization
• The pipeline consists of three stages:
CMP 2017 36
1) Explorer 2) Detector 3) Tracker
![Page 37: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/37.jpg)
Input FMO Video
CMP 2017 37
![Page 38: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/38.jpg)
FMO Tracking + Reconstruction
CMP 2017 38
![Page 39: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/39.jpg)
Temporal SR - Table tennis
CMP 2017 39
![Page 40: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/40.jpg)
Temporal SR - Darts
CMP 2017 40
![Page 41: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/41.jpg)
Rotating FMOs
• Simple convolution
replaced by space-variant convolution is a function of trajectory and angular velocity is a 3D object (sphere) • Gradient descent w.r.t • Exhaustive search w.r.t angular velocity
CMP 2017 41
![Page 42: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/42.jpg)
Temporal SR with Rotation
42 CMP 2017
Input video frame Estimated high-speed video
![Page 43: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/43.jpg)
Limitations • Poor collaboration between tracking and restoration
• Estimated blur and FMO appearance improve tracking
• Unstable FMO appearance estimation • Combine multiple video frames
• Track multiple FMOs
CMP 2017 43
![Page 44: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/44.jpg)
Thank You
For Your Attention
44 CMP 2017
![Page 45: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/45.jpg)
![Page 46: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/46.jpg)
original image
u
acquired images
= zk
channel K
channel 2
channel 1
[u * hk]
Multichannel Model
+ nk
noise
46 CMP 2017
![Page 47: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/47.jpg)
Image regularization
term
Blur Regularization
term
Data term
Gaussian noise
L2 norm
Multichannel Model
• Acquisition model:
• Optimization problem
47 CMP 2017
![Page 48: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/48.jpg)
0
Multichannel Blur Regularization
u
h2 * u u h1 * = = z1 z2
48
z1 * * u h1 = * z2 * h2 u * = *
CMP 2017
![Page 49: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/49.jpg)
Camera motion
49 CMP 2017
![Page 50: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/50.jpg)
Degraded images
Reconstructed image
Astronomical images
PSF estimation
50 CMP 2017
![Page 51: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/51.jpg)
original image
u
acquired images
= zk
channel K
channel 2
channel 1
[u * hk]
MC Model with Decimation
+ nk
noise
D
51 CMP 2017
![Page 52: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/52.jpg)
Robustness to misalignment
52
original image PSF degraded image
CMP 2017
![Page 53: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/53.jpg)
Super-resolution
53
Superresolved image (2x) Optical zoom (ground truth)
rough registration
CMP 2017
![Page 54: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/54.jpg)
8 images
SR 3x
SR 2x
original
CMP 2017 54
![Page 55: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/55.jpg)
Input video Super-resolution
Video Super-resolution
55 CMP 2017
![Page 56: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/56.jpg)
Embedded Systems
56 CMP 2017
![Page 57: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/57.jpg)
Acquired blurred image
57 CMP 2017
![Page 58: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/58.jpg)
Blur estimation
58 CMP 2017
![Page 59: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/59.jpg)
Patch-wise Deconvolution
59 CMP 2017
![Page 60: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/60.jpg)
Super-resolution
CMP 2017 60
JPEG SR
![Page 61: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/61.jpg)
Embedded Super-Resolution
Input image
SR from 5 images
SR from 10 images
61 CMP 2017
![Page 62: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/62.jpg)
Patch-wise restoration
62 CMP 2017
![Page 63: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/63.jpg)
63 CMP 2017
![Page 64: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/64.jpg)
Regularization
CMP 2017 64
![Page 65: Blur as a chance and not a nuisancecmp.felk.cvut.cz/cmp/events/colloquium-2017.01.19/... · 2017-01-19 · sroubekf@utia.cas.cz . . Institute of Information Theo. ry and Automation](https://reader034.vdocuments.us/reader034/viewer/2022050222/5f67766d4802e812092d5e01/html5/thumbnails/65.jpg)
Adjusting priors
65
• Start with an overestimated noise level and slowly decrease it to the correct level.
• Start with p<<1 and slowly increase it to p=1. CMP 2017