Digital Image Processing

IMAGE FILTERING AND RESTORATION

Hamid R. Rabiee

Fall 2015

In the Name of Allah, the Most Compassionate, the Most Merciful

Example 22

From Matlab ImageToolbox

Documentation pp12-4

Imperfectness in Image Capturing

Blurring ~ linear spatial-invariant filter model w/ additive noise

Impulse response h(n1, n2) & H(1, 2)

Point Spread Function (PSF) ~ positive I/O

[No blur] h(n1, n2) = (n1, n2)

[Linear translational motion blur]

local average along motion direction

[Uniform out-of-focus blur]

local average in a circular neighborhood

Atomspheric turbulence blur, etc.

23

H

u(n1, n2 ) v(n1, n2 )

N(n1, n2 )

otherwise 0

tan and

x if L

1

),;,( yx

222

Ly

Lyxh

otherwise 0

x if R

1

);,(

222

2Ry

Ryxh

Fourier Transform of PSF for Common Distortions

24

From Bovik’s Handbook

Sec.3.5 Fig.2&3

Undo Linear Spatial-Invariant Distortion

Assume noiseless and PSF of distortion h(n1, n2) is known

Restoration by Deconvolution / Inverse-Filtering Often used for deblurring

Want to find g(n1, n2) satisfies h(n1, n2) g(n1, n2) = (n1, n2)

h(k1, k2) g(n1 -k1, n2-k2) = (n1, n2) for all n1, n2

Easy to solve in spectrum domain

Convolution Multiplication

H(1, 2) G(1, 2) = 1

Interpretation: choose G to compensate distortions from H

25

H

u(n1, n2) v(n1, n2)

(n1, n2)=0

G

u’(n1, n2)w(n1, n2)

),(

1),(

21

21

H

G

Problems With Inverse Filtering Under Noise

Zeros in H(1, 2) Interpretation: distortion by H removes all info. in those freq. Inverse filter tries to “compensate” by assigning infinite gains

Amplifies noise:W(1, 2) = H (1, 2) U (1, 2) U’ (1, 2) = (W+N) / H N/H if W=0

Solutions?

26

Examples of Inverse & Pseudo-inverse Filtering

27

From Jain Fig.8.10

Problems With Inverse Filtering Under Noise

Zeros in H(1, 2) Interpretation: distortion by H removes all info. in those freq.

Inverse filter tries to “compensate” by assigning infinite gains

Amplifies noise:

W(1, 2) = H (1, 2) U (1, 2)

U’ (1, 2) = (W+N) / H N/H if W=0

Solutions ~ Pseudo-inverse Filtering Assign zero gain for G at spectrum nulls of H

Interpretation: not bother to make impossible compensations

28

|),(H| if ,0

|),(H| if ,),(

1

),(

21

21

2121HG

Handling Spectrum Nulls Via High-FreqCut-off

Limit the restoration to lower frequency components to avoid amplifying noise at spectrum nulls

29

Problems With Inverse Filtering Under Noise

Inverse filtering is sensitive to noise Does not explicitly modeling and handling noise

Try to balance between deblurring vs. noise suppression Minimize MSE between the original and restored

e = E{ [ u(n1, n2) – u’(n1, n2) ] 2 } where u’(n1, n2) is a func. of {v(m1, m2) }

Best estimate is conditional mean E[ u(n1 , n2) | all v(m1 , m2) ]

usually difficult to solve for general restoration (need conditional probability

distribution, and estimation is nonlinear in general)

Get the best linear estimate instead Wiener filtering Consider the (desired) image and noise as random fields

Produce a linear estimate from the observed image to minimize MSE

30

Hu(n1, n2) v(n1, n2)

(n1, n2)

Gu’(n1, n2)w(n1, n2)

Wiener Filtering

Get the best linear estimate minimizing MSE Assume spatial-invariant filter u’(n1, n2) = g (n1, n2) v(n1, n2) Assume wide-sense stationarity for original signal and noise Assume noise is zero-mean and uncorrelated with original signal

Solutions Bring into orthogonal condition

E{ [ u(n1, n2) – u’(n1, n2) ] v(m1, m2) }=0 Represent in correlation functions: Ruv(k,l) = g(k,l) Rvv(k,l) Take DFT to get representation in power spectrum density

31

),(

),(),(

21

2121

vv

uv

S

SG

),(),(),(

),(),(),(),( :Note

2121

*

21

2121

2

2121

uuuv

uuvv

SHS

SSHS

),(),(),(

),(),(

2121

2

21

2121

*

SSH

SH

uu

uu

More on Wiener Filtering

Balancing between two jobs for deblurring noisy image

HPF filter for de-blurring(undo H distortion)

LPF for suppressing noise

Noiseless case ~ S = 0 (inverse filter)

Wiener filter becomes pseudo-inverse filter for S 0

No-blur case ~ H = 1 (Wiener Smoothing Filter)

Zero-phase filter to attenuate noise according to SNR at each freq.

32

0|),(H| if ,0

0|),(H| if ,),(

1

/||),(

21

21

2102

*

021

HSSH

HG S

uv

S

1),(

),(

),(),(

),(),(

21

21

2121

211wiener21

SNR

SNR

uu

uuH

S

S

SS

SG

uuSH

SH

G

*

wiener21

1),(

Comparisons 33

From Jain Fig.8.11

Example: Wiener Filtering vs. Inverse Filtering

34

Figure is from slides at Gonzalez/ Woods DIP book

website (Chapter 5)

Example (2):Wiener Filtering vs. Inverse Filtering

35

Figure is from slides at Gonzalez/ Woods

DIP book website (Chapter 5)

Review: Handling Noise in Deconvolution

Inverse filtering is sensitive to noise Does not explicitly modeling and handling noise

Try to balance between deblurring vs. noise suppression Consider the (desired) image and noise as random fields Minimize MSE between the original and restored

e = E{ [ u(n1, n2) – u’(n1, n2) ] 2 } where u’(n1, n2) is a func. of {v(m1, m2) }

Best estimate is conditional mean E[ u(n1 , n2) | all v(m1 , m2) ]

Get the best linear estimate Wiener filtering Produce a linear estimate from the observed image to minimize MSE

36

Hu(n1, n2) v(n1, n2)

(n1, n2)

Gu’(n1, n2)w(n1, n2)

Wiener Filter: From Theory to Practice

Recall: assumed p.s.d. of image & noise random fields and freq. response of distortion filter are known

Why make the assumptions?

Are these reasonable assumptions?

What do they imply in our implementation of Wiener filter?

37

Wiener Filter: Issues to be Addressed

Wiener filter’s size

Theoretically has infinite impulse response ~ require large-size DFTs

Impose filter size constraint: find the best FIR that minimizes MSE

Need to estimate power spectrum density of orig. signal

Estimate p.s.d. of blurred image v and compensate variance due to noise

Estimate p.s.d. from a set of representative images similar to the images to be restored

Or use statistical model for the orig. image and estimate parameters

Constrained least square filter ~ see Jain’s Sec.8.8 & Gonzalez Sec.5.9

Optimize smoothness in restored image (least-square of the rough transitions)

Constrain differences between blurred image and blurred version of reconstructed image

Estimate the restoration filter w/o the need of estimating p.s.d.

Unknown distortion H ~ Blind Deconvolution

38

Basic Ideas of Blind Deconvolution

Three ways to estimate H: observation, experimentation, math. modeling

Estimate H via spectrum’s zero patterns

Two major classes of blur (motion blur and out-of-focus)

H has nulls related to the type and the parameters of the blur

Maximum-Likelihood blur estimation

Each set of image model and blur parameters gives a “typical” blurred output; Probability

comes into picture because of the existence of noise

Given the observation of blurred image, try to find the set of parameters that is most likely to

produce that blurred output

Iteration ~ Expectation-Maximization approach (EM)

Given estimated parameters, restore image via Wiener filtering

Examine restored image and refine parameter estimation

Get local optimums

To explore more: Bovik’s Handbook Sec.3.5 (subsection-4, pp136)

“Blind Image Deconvolution” by Kundur et al, IEEE Sig. Proc. Magazine, vol.13, 1996

39

Filtering Through Transform Domain Operation

E.g.1 Realize Wiener in DFT domain

E.g.2 Use zonal mask in transform domain

Realize “ideal” LPF/BPF/HPF

Computation complexity for transform could be high for large image

40

End of Lecture

Thank You!Slides adapted from ENEE631, by Min Wu