© 2002-2003 by Yu Hen Hu1ECE533 Digital Image Processing

Image Restoration

© 2002-2003 by Yu Hen Hu2ECE533 Digital Image Processing

What is Image Restoration The purpose of image restoration is to restore a

degraded/distorted image to its original content and quality.

Distinctions to Image Enhancement» Image restoration assumes a degradation model that is known

or can be estimated.» Original content and quality ≠ Good looking

© 2002-2003 by Yu Hen Hu3ECE533 Digital Image Processing

Image Degradation Model

Spatial variant degradation model

Spatial-invariant degradation model

» Frequency domain representation

( , ) ( , , , ) ( , ) ( , )g x y h x y m n f m n x y

( , ) ( , ) ( , ) ( , )g x y h x m y n f m n x y

( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v

© 2002-2003 by Yu Hen Hu4ECE533 Digital Image Processing

Noise Models Most types of noise are

modeled as known probability density functions

Noise model is decided based on understanding of the physics of the sources of noise. » Gaussian: poor illumination» Rayleigh: range image» Gamma, exp: laser imaging» Impulse: faulty switch during

imaging, » Uniform is least used.

Parameters can be estimated based on histogram on small flat area of an image

© 2002-2003 by Yu Hen Hu5ECE533 Digital Image Processing

Noise Removal Restoration Method

Mean filters» Arithmetic mean filter» Geometric mean filter» Harmonic mean filter» Contra-harmonic mean

filter Order statistics filters

» Median filter» Max and min filters» Mid-point filter» alpha-trimmed filters

Adaptive filters» Adaptive local noise

reduction filter» Adaptive median filter

© 2002-2003 by Yu Hen Hu6ECE533 Digital Image Processing

Mean Filters

,( , )

1ˆ( , ) ( , )x ys t S

f x y g s tmn

,

1

( , )

ˆ ( , ) ( , )x y

mn

s t S

f x y g s t

© 2002-2003 by Yu Hen Hu7ECE533 Digital Image Processing

Contra-Harmonic Filters

,

,

1

( , )

( , )

( , )ˆ ( , )

( , )

x y

x y

Q

s t S

Q

s t S

g s t

f x yg s t

© 2002-2003 by Yu Hen Hu8ECE533 Digital Image Processing

Median Filter

,( , )

ˆ ( , ) ( , )x ys t S

f x y median g s t

Effective for removing salt-and-paper (impulsive) noise.

© 2002-2003 by Yu Hen Hu9ECE533 Digital Image Processing

LSI Degradation Models

Motion Blur» Due to camera panning or

fast motion

Atmospheric turbulence blur» Due to long exposure time

through atmosphere

» Hufnagel and Stanley

Uniform out-of-focus blur:

Uniform 2D Blur

min max1 0,( , )

0 .

ai bj i i ih i j

otherwise

2 2

2( , ) exp

2

i jh i j K

2 2 22

1( , )

0 .

i j Rh i j R

otherwise

2

1/ 2 , / 2

( , )0 .

L i j Lh i j L

otherwise

5/62 2( , ) exph i j k i j

© 2002-2003 by Yu Hen Hu10ECE533 Digital Image Processing

Turbulence Blur Examples

5/ 62 2( , ) exph i j k i j

© 2002-2003 by Yu Hen Hu11ECE533 Digital Image Processing

Motion Blur

Often due to camera panning or fast object motion.

Linear along a specific direction.

blurring filter

20 40 60

20

40

60

blurring filter mask

2 4 6 8

2

4

6

8

original image

20 40 60

10

20

30

40

50

60

blurred image

20 40 60

10

20

30

40

50

60

Blurdemo.m

© 2002-2003 by Yu Hen Hu12ECE533 Digital Image Processing

Inverse Filter

Recall the degradation model:

Given H(u,v), one may directly estimate the original image by

At (u,v) where H(u,v) 0, the noise N(u,v) term will be amplified!

( , ) ( , ) ( , ) ( , )G u v H u v F u v N u v

ˆ ( , ) ( , ) / ( , )

( , )( , )

( , )

F u v G u v H u v

N u vF u v

H u v

original, f

20 40 60

20

40

60

degraded: g

20 40 60

20

40

60

inverse filter

20 40 60

20

40

60Invfildemo.m

© 2002-2003 by Yu Hen Hu13ECE533 Digital Image Processing

Wiener Filtering

Minimum mean-square error filter» Assume f and are both 2D

random sequences, uncorrelated to each other.

» Goal: to minimize

» Solution: Frequency selective scaling of inverse filter solution!

» White noise, unknown Sf(u,v):

2ˆE f f

2

2

( , ) ( , )ˆ ( , )( , )( , ) ( , ) / ( , )f

H u v G u vF u v

H u vH u v S u v S u v

2

2

( , ) ( , )ˆ ( , )( , )( , )

H u v G u vF u v

H u vH u v K

original, f

20 40 60

20

40

60

degraded: g

20 40 60

20

40

60

Wiener filter, K=0.2

20 40 60

20

40

60

inverse filter

20 40 60

20

40

60

© 2002-2003 by Yu Hen Hu14ECE533 Digital Image Processing

Derivation of Wiener Filters

Given the degraded image g, the Wiener filter is an optimal filter hwin such that E{|| f – hwing||2} is minimized.

Assume that f and are uncorrelated zero mean stationary 2D random sequences with known power spectrum Sf and Sn. Thus,

2 2

2

2 2

2 2

( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , ) ( , )

( , ) ( , ) ( , ) ( , ) (

win win

Hwin

H Hwin win

f win f n

H Hwin f win

C E f h g E F u v H u v G u v

E F u v H u v E F u v G u v

H u v E F u v G u v H u v E G u v

S u v H u v H u v S u v S u v

H u v H u v S u v H u v H u

, ) ( , )fv S u v

2

2

( , ) ( , )

( , ) ( , )

( , ) ( , )

( , ) ( , ) 0

f

n

H

H

E F u v S u v

E N u v S u v

E F u v N u v

E F u v N u v

*

2

( , ) 0

( , ) ( , )( , )

( , ) ( , ) ( , )

Set C/ win

fwin

f n

H u v

H u v S u vH u v

H u v S u v S u v

© 2002-2003 by Yu Hen Hu15ECE533 Digital Image Processing

Constrained Least Square (CLS) Filter

For each pixel, assume the noise has a Gaussian distribution. This leads to a likelihood function:

A constraint representing prior distribution of f will be imposed:

the exponential form of pdf of f is known as the Gibbs’ distribution.

Since L(f) p(g|f), use Bayes rule,

since g is given, to maximize the posterior probability, one should minimize

q is an operator based on prior knowledge about f. For example, it may be the Laplacian operator!

2

2

1( ) exp **

2L f g h f

2( ) exp **p f q f

( | ) ( | ) ( ) / ( )p f g p g f p f p g

2 2** **g h f q f

© 2002-2003 by Yu Hen Hu16ECE533 Digital Image Processing

Intuitive Interpretation of CLS

Prior knowledge: Most images are smooth ||q**f|| should be minimized

However, the restored image , after going through the same degradation process h, should be close to the given degraded image g.

The difference between g and is bounded by the amount of the additive noise:

In practice, |||| is unknown and needs to be estimated with the variance of the noise

ˆ ( , )f x y

ˆ( , )** ( , )h x y f x y

2 2**g h f

© 2002-2003 by Yu Hen Hu17ECE533 Digital Image Processing

Solution and Iterative Algorithm

To minimize CCLS, Set

CCLS/ F = 0. This yields

The value of however, has to be determined iteratively! It should be chosen such that

Iterative algorithm (Hunt)

1. Set initial value of ,

2. Find , and compute R(u,v).

3. If ||R||2 - ||N||2 < - a, set = BL, increase , else if

||R||2 - ||N||2 > a, set = Bu, decrease , else stop iteration.

4. new = (Bu+BL)/2, go to step 2.

2

2 2

( , ) ( , ) ( , )

( , ) ( , ) ( , )

CLSC G u v H u v F u v

N u v Q u v F u v

*

2 2

( , )ˆ ( , ) ( , )( , ) ( , )

H u vF u v G u v

H u v Q u v

2 2

2 2

ˆ( , ) ( , ) ( , ) ( , )

( , ) ( , )

G u v H u v F u v N u v

R u v N u v a

ˆ ( , )F u v

© 2002-2003 by Yu Hen Hu18ECE533 Digital Image Processing

CLS Demonstrationiteration 1, gamma=5.0005

20 40 60

20

40

60

iteration 4, gamma=0.62594

20 40 60

20

40

60

iteration 7, gamma=0.079117

20 40 60

20

40

60

iteration 10, gamma=0.010765

20 40 60

20

40

60

iteration 13, gamma=0.0022206

20 40 60

20

40

60

iteration 15, gamma=0.0028309

20 40 60

20

40

60