school of electrical & computer engineering image denoising using gaussian scale mixtures in the...
TRANSCRIPT
School of Electrical & Computer Engineering
Image Denoising Using Gaussian Scale Mixtures in the Wavelet domain
Alex CunninghamBen Clarke
Dy narath Eang
ECE6258
20 November 2008
Image Denoising Using Gaussian Scale Mixtures in the Wavelet domain
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
2School of Electrical & Computer Engineering
Agenda
Algorithm Overview Background
Wavelets Steerable Pyramid Decomposition Gaussian Scale Mixture Bayesian Least Square Estimation
Image Denoising Details Implementation Details
Detailed Algorithm Detailed Matlab Implementation
Results Comparison to other denoising techniques Improvement schemes
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
3School of Electrical & Computer Engineering
Algorithm Overview
General Denoising Algorithm: Decompose Image into Steerable Wavelet Pyramid Subbands For each subband:
Perform local Wiener Estimation Denoise coefficients using Bayesian Least Squares Estimation Reconstruct Denoised Subband
Reconstruct image from denoised wavelet subbands
Paper Source J. Portilla, V. Strela, M. Wainwright, and E. P. Simoncelli, “Image denoising using a scale
mixture of Gaussians in the wavelet domain,” IEEE Trans. Image Processing, vol. 12, no. 11, pp. 1338–1351, November 2003.
Additional Tools Matlab Pyramid Tools (Computational Vision Lab at NYU)
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
4School of Electrical & Computer Engineering
Wavelet Overview
Continuous Wavelet Transform (CWT)
Discrete Wavelet Transform (DWT)
CWT Decomposition:
Images Source: Matlab Help
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
5School of Electrical & Computer Engineering
Steerable Wavelet Pyramids (Spyr) Overview
Spyrs Combine: Multiple Levels of
DWT Multiple
Orientations of Differential Filters
“Steerability” comes from the set of oriented filters
Can reconstruct image without aliasing
(Circle) Image Source: E. Simoncelli, W. Freeman, “The Steerable Pyramid: A Flexible Architecture for Multi-Scale Derivative Computation,” 2nd Ann. IEEE Intl. Conf. on Image Processing. Washington, DC. October, 1995.
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
6School of Electrical & Computer Engineering
Gaussian Scale Mixture (GSM) Overview
GSM = A useful statistical model
Problem: Images are NOT spatially homogeneous or scale invariant and their statistics do NOT follow Gaussian distribution
Solution: Use GSM to model wavelet coefficients
Usefulness: Characterize the coefficients’ marginal distribution shape and strong correlation between the neighbors’ amplitudes
Form: ux z
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
7School of Electrical & Computer Engineering
Bayesian Least Squares (BLS) Estimation Overview
BLS = A useful estimation method
Problem: Two-step empirical Bayes estimation is suboptimal
Solution: Use BLS to estimate the real coefficients.
Usefulness: Improves the empirical Bayes Estimator and it’s now a single step process
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
8School of Electrical & Computer Engineering
GSM and Covariance Matrices
There are five main variables: w: noise subband y: observed noisy image subband x: model GSM subband u: estimated real signal subband z: scalar RV used as a multiplier
The observation is the signal plus noise – these 3 RV’s are independent
Similarly, the covariance matrices follow the same additive rule
Taking E{z} = 1, then Eq(6a) can be rearranged
Eq (5)
Eq (6a)
Eq (7)
wuwxy z
wuy z CCC
wyu CCC
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
9School of Electrical & Computer Engineering
Bayes Least Squares and Local Wiener Estimation
BLS estimates the denoised coefficients by estimating at the center coefficients of all neighborhoods
It needs two main things: E[Xc|y,z] p(z|y)
Local Wiener Estimation helps compute the 1st main item
The 2nd main item can be computed sequentially using Equations 13 and 14.
Equations come directly from the author’s paper
Eq (8)
Eq (9)
Eq (10)
Eq (11)
Eq (12)
Eq (14)
Eq (13)
0
,|)|(| zxzpx cc yyy
vIMyx 1)(,, zzz
yCCCyx 1)(,| wuu zzz
TTTuwu zzz SQISQSSCCC )(
0)()|(
)()|()|(
dpp
zpzpzp
z
z
y
yy
N
n n
nncnc z
zmzx
1 1,|
y
Nn nw
N
N
n n
n
z
zzp
1
1
2
1)2(
121
exp
)|(
Cy
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
10School of Electrical & Computer Engineering
Image Denoising Algorithm
Full algorithm:*
1) Decompose the image into subbands. 2) For each subband (except the lowpass residual):
a) Compute neighborhood noise covariance, Cw, from the image-domain noise covariance.
b) Estimate noisy neighborhood covariance, Cy.
c) Estimate Cu from Cw and Cy using (7). d) Compute A and M (Section III-B: Local Wiener Estimate). e) For each neighborhood:
i) For each value z in the integration range: A) Compute E{xc|y,z} using (12). B) Compute p(y|z) using (14).
ii) Compute p(z|y) using (13) and(4). iii) Compute E{xc|y} numerically using (8).
3) Reconstruct the denoised image from the processed subbands and the lowpass residual. (p. 1343)
* The algorithm is copied directly from the journal paper, p. 1343.
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
11School of Electrical & Computer Engineering
Matlab Algorithm Tree
denoise_demo.m Load image of interest Make copy of image and add Gaussian noise of known variance Define number of levels/bands for steerable pyramid (SPyr) decomposition Calculate bound extensions and PSDs Decompose noise-image and noisy image into SPyr structure Perform denoising (denoise_ctrl.m) Reconstruct image from denoised SPyr samples Calculate signal-to-noise ratios, display images
denoise_demo.m
pyr_decomp.m denoise_ctrl.m pyr_reconstruct.m
pyr_extract.m local_wiener_estimator.m expect_estimator.mcovariance_calculator.m
bound_extension.m
pyr_insert.m
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
12School of Electrical & Computer Engineering
Denoising Functions
denoise_ctrl.m Extract particular subband (both noise image and noisy image) Calculate covariance matrices (covariance_calculator.m) Perform (local_weiner_estimator.m) from covariance matrices to get eigenvalues and
eigenvectors Use eigenvalues/eigenvectors to estimate the real coefficients by find expectation values for
each pixel in the neighborhood (expect_estimator.m) Reinsert denoised subband into pyramid structure
covariance_calculator.m Create neighborhood matrices (for noise image and noisy images) assuming zero-mean
Gaussian distributed Compute signal covariance (Cu) as difference between observed covariance (Cy) and noise
covariance (Cw)
local_wiener_estimator.m With assumption of AWGN, perform wiener calculations
Requires computation of eigenvalue/eigenvectors for covariance matrices Result is that for various covariance matrices, we get wiener matrices
expect_estimator.m Perform equations (12) and (14) From equation (14), perform equation (13) Use results of equations (12) and (13) to compute E{xc|y} numerically.
0
,|)|(| zxzpx cc yyy
N
n n
nncnc z
zmzx
1 1,|
y
0)()|(
)()|()|(
dpp
zpzpzp
z
z
y
yy
Nn nw
N
N
n n
n
z
zzp
1
1
2
1)2(
121
exp
)|(
Cy
Eq (8)*
Eq (12)
Eq (13)
Eq (14)
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
13School of Electrical & Computer Engineering
Statistical Results
Portilla Basic 512x512 or 256x256, 8-bit
grayscale images Scan σ across a range of standard
deviations Compute Peak SNR (PSNR) for
each of six images Last column σ(PSNR) is the
estimated standard deviation of the results.
Ours Compute same PSNR Compute Improvement SNR
(ISNR) from noisy image to denoised image
σ / PSNR Lena Barb Boats Fgrpt House Peprs σ(PSNR)1 / 48.13 48.46 48.37 48.44 48.46 48.85 48.38 0.0092 / 42.11 43.23 43.29 42.99 43.05 44.07 43.00 0.0125 / 34.15 38.49 37.79 36.97 36.68 38.65 37.31 0.01410/ 28.13 35.61 34.03 33.58 32.45 35.35 33.77 0.01715 / 24.61 33.90 31.86 31.70 30.14 33.64 31.74 0.02420 / 22.11 32.66 30.32 30.38 28.60 32.39 30.31 0.03125 / 20.17 31.69 29.13 29.37 27.45 31.40 29.21 0.03750 / 14.15 28.61 25.48 26.38 24.16 28.26 25.90 0.04975 /10.63 26.84 23.65 24.79 22.40 26.41 24.00 0.061100 / 8.13 25.64 22.61 23.75 21.22 25.11 22.66 0.070
+σ is the input standard deviation for the AWGN
σ PSNR ISNR PSNR ISNR PSNR ISNR1 48.11 -0.02 47.91 -0.22 48.46 0.332 43.13 1.02 42.43 0.32 43.99 1.885 38.45 4.30 35.66 1.51 38.63 4.48
10 35.58 7.45 31.83 3.70 35.30 7.1715 33.88 9.28 29.96 5.35 33.57 8.9620 32.66 10.55 28.65 6.54 32.33 10.2225 31.68 11.51 27.61 7.43 31.35 11.1850 28.60 14.45 24.15 10.00 28.25 14.1075 26.87 16.25 22.05 11.42 26.52 15.89
100 25.70 17.57 20.58 12.45 25.32 17.19
Lena Flintstones House
Portilla PSNR table
Our SNR table
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
14School of Electrical & Computer Engineering
Pictorial Results
Extreme case: σ (sigma) = 100
Moderate case: σ (sigma) = 10
SNR Improvement of 12.45dB
SNR Improvement of 5.87dB
Original Image Original with Additive White Gaussian NoiseOriginal Image
Range: [1, 255] Dims: [512, 512] / 2
Denoised Image, PSNR = 20.5813dB, ISNR = 12.4505dB
Range: [1, 255] Dims: [512, 512] / 2
Original Image Original with Additive White Gaussian Noise
Original Image Original with Additive White Gaussian NoiseOriginal Image Original with Additive White Gaussian NoiseOriginal Image
Range: [12, 246] Dims: [512, 512] / 2
Denoised Image, PSNR = 34.0015dB, ISNR = 5.8707dB
Range: [12, 246] Dims: [512, 512] / 2
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
15School of Electrical & Computer Engineering
Comparison
Extreme case: σ (sigma) = 100
Subjective ranking: GSM-BLS (Portilla) Wiener 5x5 Discrete wavelet transform
(DWT) Wavelet packet (WP)
Original Image
Range: [1, 255] Dims: [512, 512] / 2
Denoised Image, PSNR = 20.5813dB, ISNR = 12.4505dB
Range: [1, 255] Dims: [512, 512] / 2
Denoised Image (GSM) Denoised Image (Wiener 5x5)
Denoised Image (DWT Denoiser)
Denoised Image (GSM) Denoised Image (Wiener 5x5)
Denoised Image (DWT Denoiser)
Denoised Image (GSM) Denoised Image (Wiener 5x5)
Denoised Image (DWT Denoiser) Denoised Image (WP Denoiser)
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
16School of Electrical & Computer Engineering
Improvement Scheme #1
‘‘Image Denoising using shiftable directional pyramid and CGSM’’
Authors: An Vo, Truong Nguyen, & Soontron Oraintara
Publication date: May 2007 Synopsis: Improvement in PSNR
values for certain images (Barbara) was achieveable by incorporating CGSM and PDTDFB.
PDTDFB – Pyramidal Dual Tree Directional Filter Bank
CGSM – Complex Gaussian Scale mixture.
*SSIM = Structural Similarity
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
17School of Electrical & Computer Engineering
Improvement Scheme #2
‘‘Image Denoising using GSM model and BLS estimation over wider range of observation’’
Author:Hyng Il Koo & Nam Ik Cho Publication date: April 2008 Synopsis: Subjective improvement in the quality of the images by
considering a larger range of coefficients and using a new statistical model for the p(y|z).
CRF = Conditional random field Sum-product algorithm = inference algorithm
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
EC
E62
58 D
igita
l Im
age
Pro
cess
ing,
Gro
up3
18School of Electrical & Computer Engineering
References
Original reference J. Portilla, V. Strela, M. Wainwright, and E. P. Simoncelli, “Image
denoising using a scale mixture of Gaussians in the wavelet domain,” IEEE Trans. Image Processing, vol. 12, no. 11, pp. 1338–1351, November 2003.
Other denoising papers A. Vo, T. Nguyen, S. Oraintara, “Image Denoising using Shiftable
Directional Pyramid and Scale Mixtures of Complex Gaussians” ISCAS 2007, pp. 4000-4003, May 2007
H. I. Koo and N. I. Cho, “Image Denoising Based on a Statistical Model for Wavelet Coefficients,” ICASSP 2008, pp. 1269-1272, April 2008.
E. Simoncelli, W. Freeman, “The Steerable Pyramid: A Flexible Architecture for Multi-Scale Derivative Computation,” 2nd Ann. IEEE Intl. Conf. on Image Processing. Washington, DC. October, 1995.