bm3d-amp: a new image recovery algorithm based on bm3d ...cam6.web.rice.edu/talks/icip_v15.pdf ·...
TRANSCRIPT
![Page 1: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/1.jpg)
BM3D-AMP: A New Image
Recovery Algorithm Based on
BM3D Denoising
Chris Metzler, Richard Baraniuk
Rice University
Arian Maleki
Columbia University
![Page 2: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/2.jpg)
Compressive Sensing Problem
![Page 3: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/3.jpg)
Solution: Assume Structure
• Sparse: OMP [Tropp 04], IST [Figueiredo et al 07], AMP [Donoho et al 09]
• Minimal Total Variation: TVAL3 [Li et al 09]
• Tree-Sparse: Model-CoSaMP [Baraniuk et al 10],Turbo-AMP [Som and Schniter 12]
• Group-Sparse: NLR-CS [Dong et al 14]
![Page 4: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/4.jpg)
Using Structure is Hard
• Write as penalty or constraint
• How to efficiently solve for non-convex R(x)?
• What is R(x) for natural images?
• What is R(x) for RF, microscopy, and other applications?
![Page 5: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/5.jpg)
This Talk
• Develop algorithm that can easily use almost any structure
• Predict performance with accurate state evolution
• Derive theoretical guarantees• Measurements required
• Robust to noise
• Demonstrate state-of-the-art performance• 10dB better than wavelet sparsity
• 50x faster than group-sparsity
![Page 6: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/6.jpg)
Insight: Denoisers Use Structure
• Gaussian Kernel• Smooth
• Soft Wavelet Thresholding [Donoho and Johnstone 94]
• Sparse wavelet representation
• BLS-GSM [Portilla et al 03]
• Coefficients follow Gaussian Mixture Model
• NLM [Baudes et al 05]
• Correlated structures
• BM3D [Dabov et al 07]
• Group-sparse in DCT/Wavelet representation
• BM3D-SAPCA [Dabov et al 09]
• Group-sparse in adaptive basis
![Page 7: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/7.jpg)
Denoisers as Black Boxes
Denoiser
![Page 8: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/8.jpg)
Denoisers as Projections
C
![Page 9: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/9.jpg)
Naïve Algorithm: Denoising-based
Iterative Thresholding (D-IT)
![Page 10: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/10.jpg)
Naïve Algorithm: D-IT
![Page 11: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/11.jpg)
Naïve Algorithm: D-IT
![Page 12: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/12.jpg)
Naïve Algorithm: D-IT
![Page 13: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/13.jpg)
Naïve Algorithm: D-IT
![Page 14: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/14.jpg)
Naïve Algorithm: D-IT
![Page 15: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/15.jpg)
Failure of D-IT: Systematic Errors
![Page 16: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/16.jpg)
Systematic Errors: Overshooting
![Page 17: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/17.jpg)
Systematic Errors: Overshooting
Too High
![Page 18: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/18.jpg)
Systematic Errors: Overshooting
Too High Too Low
Too High Too Low
![Page 19: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/19.jpg)
Systematic Errors: Overshooting
Too High Too Low
Too High Too Low
![Page 20: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/20.jpg)
Systematic Errors: Overshooting
Too High Too Low
Too High Too Low
![Page 21: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/21.jpg)
Systematic Errors: Overshooting
Too High Too Low
Too High Too Low
Use residuals from
previous iterations to
avoid overshooting
![Page 22: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/22.jpg)
New Algorithm: D-AMP
Onsager Correction
![Page 23: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/23.jpg)
D-AMP Benefits
D-IT
• Updates proportional to residual (P controller).
• 5dB improvement over L1
D-AMP
• Updates use rate residuals change (PD controller).
• 10dB improvement over L1
(state-of-the-art)
• Onsager Correction
![Page 24: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/24.jpg)
Onsager Correction:
• Where did it come from?• Approximation of message passing algorithm
![Page 25: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/25.jpg)
Onsager Correction:
• Where did it come from?• Approximation of message passing algorithm
• Why does it help?• zt stores residuals over many iterations (momentum)
• Corrects for bias in denoiser solutions
• Makes errors uncorrelated (Gaussian) and thus easy to remove
![Page 26: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/26.jpg)
Onsager Correction:
• Where did it come from?• Approximation of message passing algorithm
• Why does it help?• zt stores residuals over many iterations (momentum)
• Corrects for bias in denoiser solutions
• Makes errors uncorrelated (Gaussian) and thus easy to remove
• How is it calculated?• Approximation from Monte Carlo SURE [Ramani et al 08]
![Page 27: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/27.jpg)
D-AMP Avoids Systematic Errors
![Page 28: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/28.jpg)
D-AMP Theoretical Properties
• Predictable performance
• Denoiser determines phase transition
• Robust to noise
• Easy to tune
![Page 29: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/29.jpg)
State Evolution
• Errors are uncorrelated → Gaussian noise at each iteration
• Variance of noise determines MSE
• State evolution predicts noise variance
![Page 30: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/30.jpg)
State Evolution of D-IT and D-AMP
![Page 31: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/31.jpg)
Main Theoretical Results
• Denoiser Performance
• Phase Transition: Determined by denoiser
• Noise Sensitivity: Graceful failure
![Page 32: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/32.jpg)
Parameter Tuning
• Denoiser parameters
• Problem: Tune denoiser parameters over multiple iterations
• Result: Greedy parameter selection is optimal
![Page 33: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/33.jpg)
3x Under-Sampling
![Page 34: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/34.jpg)
20x Under-Sampling
Wavelet Sparse (L1) BM3D-AMP (our algorithm)
![Page 35: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/35.jpg)
10x Under-Sampling with Noise
NLR-CS BM3D-AMP
![Page 36: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/36.jpg)
Performance without Noise
![Page 37: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/37.jpg)
Computation Time
30x Faster
70x Faster
![Page 38: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/38.jpg)
Performance with Noise
![Page 39: BM3D-AMP: A New Image Recovery Algorithm Based on BM3D ...cam6.web.rice.edu/talks/ICIP_v15.pdf · Recovery Algorithm Based on BM3D Denoising Chris Metzler, Richard ... 10x Under-Sampling](https://reader034.vdocuments.us/reader034/viewer/2022042120/5e9a220b37977700aa4bdbf4/html5/thumbnails/39.jpg)
D-AMP Summary
• Arbitrary denoiser• NLM
• BM3D
• Useful state evolution
• State-of-the-art performance
• Resilient to noise
• >97% reduction in average computation time