Deconvolution, Deblurring andRestoration
T-61.182, Biomedical Image Analysis
Seminar Presentation 14.4.2005
Seppo Mattila & Mika Pollari
Overview (1/2)
• Linear space-invariant (LSI) restoration filters - Inverse filtering - Power spectrum equalization - Wiener filter - Constrained least-squares restoration - Metz filter• Blind Deblurring
Overview (2/2)
• Homomorphic Deconvolution• Space-variant restoration
– Sectioned image restoration– Adaptive-neighbourhood deblurring– The Kalman filter
• Applications - Medical - Astronomical
Introduction
• Find the best possible estimate of the original unknown image from the degraded image.
• One typical degradation process has a form:g x ,y h x ,y f x , y n x, y
g(x,y) measured imagef(x,y) true (ideal) imageh(x,y) point spread function (PSF) (impuse response function)n(x,y) additive random noise
Image Restoration General
• One has to have some a priori knowledge about the degragation process.
• Usually one needs 1) model for degragation, some information from 2) original image and 3) noise.
• Note! Eventhough one doesn’t know the original image some information such as power spectral density (PSD) and autocorreletion function (ACF) are easy to model.
Linear-Space Invariant (LSI) Restoration Filters
• Assume: linear and shift-invariant degrading process
• Random noise statistically indep. of image-generating process
• Possible to design LSI filters to restore the image
g x ,y h x ,y f x , y n x, y
G u, v H u, v F u ,v n u, v
Inverse Filtering
• Consider degrading process in matrix form:
• Given g and h, estimate f by minimising the squared error between observed image (g) and :
where and are approximations of f and g
• Set derivative of є2 to zero:
2 T gTg f ThTg gThf f ThThf
g hf
g g g h f
F u,vG u,vH u, v
(see Sect. 3.5.3 for details)
F u,vG u,vH u, v
n u,vH u, v
(if no noise)
(if noise present)
f g
g
Inverse Filtering Examples
• Works fine if no noise but...
• H(u,v) usually low-pass function.
• N(u,v) uniform over whole spectrum.
• High-freq. Noise amplified!! 0.4x
0.2x
f' G
Power Spectrum Equalization (PSE)• Want to find linear transform L such that:
• Power spectral density (PSD) = FT(Autocorrelation function)
f x ,y L g x ,y
PSD( (u,v)) = PSD(f(u,v))
L u, v1
H u, v 2 n u ,v f u, v
1 2
F u,vH u ,v 2
H u ,v 2 n u, v f u, v
1 2G u, vH u ,v
i.e. F u,v L u ,v G x, y
f
. . .
The Wiener Filter (1/2)
• Degradation model:
• Assumtions: Image and noise are second-order-stationary random processes and they are statistically independent
• Optimal mean-square error (MSE) criterion Find Wiener filter (L) which minimize MSE
hfg
LgfffE ~
where~2
The Wiener Filter (2/2)
• Minimizing the criterion we end up to optimal Wiener filter.
• The Wiener filter depends on the autocorrelation function (ACF) of the image and noise (This is no problem).
• In general ACFs are easy to estimate.
1)( Tf
Tfw hhhL
Constrained Least-squares Restoration
• Minimise: with constraint: where L is a linear filter operator
● Similar to Wiener filter but does not require the PSDs of the image and noise to be known
● The mean and variance of the noise needed to set optimally. If = 0 inverse filter
Lf2
g h f2
n 2
F u,vH u, v 2
H u, v 2 L u, v 2
G u, vH u,v
. . .
The Metz Filter
• Modification to inverse filter.
• Supress the high frequency noise instead of amplyfying it.
• Select factor so that mean-square error (MSE) between ideal and filtered image is minimized.
),(
),(11 2
vuH
vuHLM
Motion Deblurring – Simple Model
• Assume simple in plane movement during the exposure
• Either PSF or MTF is needed for restoration T
dttbytaxfyxg0
))(),((),(
T
T
dttvbtuajvuH
dttvbtuajvuFvuG
0
0
))()((2exp(),(
))()((2exp(),(),(
Blind Deblurring
• Definition of deblurring.
• Blind deblurring: models of PSF and noise are not known – cannot be estimated separately.
• Degragated image (in spectral domain) consist some information of PSF and noise but in combined form.
),(),(),( yxyxfPSFyxg
Method 1 – Extension to PSE
• Broke image to M x M size segment where M is larger than dimensions of PSF then
• Average of PSD of these segments tend toward the true signal and noise PSD
• This is combined information of blur function and noise which is needed in PSE
• Finaly, only PSD of image is needed
Extension to PSE Cont...
2/1
2
1
22
2
2
),(),(|),(|
),(),(
: ofr denominato is
),(~
),(~
|),(|1
:Average
),(),(|),(|),(
,...,1 where),(),(),(),(
2
vuvuvuH
vuvuL
L
vuvuvuHQ
vuvuvuHvu
Qlnmnmfnmpsfnmg
f
fPSE
PSE
Q
lf
lflgl
lll
Method 2 – Iterative Blind Deblurring
• Assumptation: MTF of PSF has zero phase.
• Idea: blur function affects in PSD but phase information preserves original information from edges.
),(),( and v)(u,v)M(u,M ),(M
FT of mag./phase look the and zero tonoiseset
),(),(),(),(
fPSFg vuvuvu
yxyxfyxpsfyxg
fg
Iterative Blind Deblurring Cont...
• Fourier transform of restored image is
• Note that smoothing operator S[] has small effect to smooth functions (PSF). This leads to iterative update rule
)),(v)exp(j(u,M~
),(F~
f vuvu g
][ M
~or 1 M
~
where][
]~
[M~
0f
0f
1lf
g
ggg
g
lf
g
MS
MMM
MS
MSM
Homomorphic deconvolution
• Start from:• Convert convolution operation to addition:
● Complex cepstrum:
Complex cepstra related: ● Practical application, however, not simple...
G u, v H u, v F u ,v
log G u,v log H u, v log F u, v
g x ,y FT 1 log G u, v
g x ,y h x ,y f x ,y
Space-variant Image Restoration
• So far we have assumed that images are spatially (and temporaly) stationary
• This is (generally) not true – at the best images are locally stationary
• Techniques to overcome this problem:– Sectioned image restoration– Adaptive neighbourhood deblurring– The Kalman filter (the most elegant approach)
Sectioned Image Restoration
• Divide image into small [P x P] rectangular, presumably stationary segments.
• Centre each segment in a region, and pad the surrounding with the mean value.
• For each segment apply separately image restoration (e.g. PSE or wiener).
),(),(f),(),(g lll nmnmnmhnm
Adaptive-neighborhood deblurring (AND)
• Grow adaptive neighborhood regions:
• Apply 2D Hamming window to each region:
• Estimate the noise spectrum:
gm,n p,q h p,q fm,n p,q nm,n p,q
Centered on (m,n) Pixel locationswithin the region
gm,n p,q wH p,q h p,q fm,n p,q wH p,q nm,n p,q wH p,q
nm,n u, v Am,n u ,v Gm,n u ,v
A is a freq. domain scale factor that depends on the spectral characterisics of the region grown etc.
Adaptive-neighborhood deblurring (AND) Cont…
• Frequency-domain estimate of the uncorrupted adaptive-neighborhood region:
• Obtain estimate for deblurred adaptive neigborhood region
m,n(p,q) by FT-1
• Run for every pixel in the input image g(x,y)
Fm,n u,v 1 n u,v
gm,nu, v
1 2 Gm,n u, v
H u, v
Deblurred image
f
Kalman Filter
• Kalman filter is a set of mathematical equations.
• Filter provides recursive way to estimate the state of the process (in non-stationary environment), so that mean of squared errors is minimized (MMSE).
• Kalman filter enables prediction, filtering, and smoothing.
Kalman Filter State-Space
• Process Eq.
• Observation Eq.
• Innovation process:
dn)f(n)1,a(n1)f(n
Oh(n)f(n)g(n)
)|(~)()( 1 nGngngn
Kalman Filter in a Nutshell (1/2)
• Data observations are available • System parameters are known
– a(n+1,n), h(n), and the ACF of driving and observation noise
• Initial conditions
• Recursion
2
0
0
10order of elementsmatrix Diagonal )0,1(
0)1()G|(1f~
D
fE
p
Kalman Filter in a Nutshell (2/2)
1) Compute the Kalman gain K(n)
2) Obtain the innovation process
3) Update
4) Compute the ACF of filtered state error
5) Compute the ACF of predicted state error
10 )]()()1,()([)()1,(n)1,a(nK(n) nnhnnnhnhnn T
pT
p
)|(~
)()()( 1 nGnfnhngn
)()()|(~
),1()|1(~
1 nnKGnfnnaGnf nn
)1,()()()1,()1,()( nnnhnKnnannn ppp
)(),1()(),1(),1( nnnannnann ndT
pp
Astronomical applications
• Images blurred by atmospheric turbulence• Observing above the atmosphere very expensive (HST)• Improve the ground-based resolution by
– Suitable sites for the observatory (@ 4 km height)– Real time Adaptive optics correction– Deconvolution
Point Spread Function (PSF) in Astronomy
Ideal PSF if no atmosphereFWHM ~ 1.22x/D
< 0.1" (8m telescope) Atmospheric turbulence broadens the PSF Gaussian PSF with FWHM ~ 1"
Iobserved
= Ireal
⊗PSF
Easy to measure and model from several stars usually present in astro-images Determines the spatial resolution of an image Commonly used for image matching and deconvolution
Richardson-Lucy deconvolution
• Used in both fields: astronomy & medical imaging• Start from Bayes's theorem, end up with:
• Takes into account statistical fluctuations in the signal, therefore can reconstruct noisy images!
• In astronomy the PSF is known accurately
• From an initial guess f0(x) iterate until converge
f i 1 xg x
f i x h xh x f i x