Image Restoration 1) Image restoration concerns the removal or reduction

of degradations which have occurred during the

acquisition of the image.

2) Such degradations may include noise, which are errors

in the pixel values, or optical effects such as out of

focus blurring, or blurring due to camera motion.

3) Image restoration is to restore a degraded image back3) Image restoration is to restore a degraded image back

to the original image while image enhancement is to

manipulate the image so that it is suitable for a specific

application.

4) Restoration can be performed successfully;

a) Using neighborhood operations

b) Using frequency domain processes1

DKVishwakarma

A model of Image

Degradation/Restoration Process

Image Degraded Image

22

Degraded image can be expressed by;

),(),(),(),( yxyxhyxfyxg η+∗=

where h(x,y) is a system that causes image distortion , it is also called as point spread

function (psf) and η(x,y) is noise (Random Error).DKVishwakarma

•We can perform some operation frequency domain it

is given by;

Optical transfer function (OTF)

),(),().,(),( vuNvuFvuHvuG +=

),( vuH

• If we knew the value of H, & N then we can recover

the F by equation;

3

),(/)],(),([),( vuHvuNvuGvuF +=

DKVishwakarma

Some Important Noise Models

• Noise cannot be predicted but can be approximately

described in statistical way using the probability

density function (PDF).

• Gaussian noise: The probability density function of

Gaussian Noise is given by;Gaussian Noise is given by;

• 70% of its value lies in the range of and

about 95% lies in the range of .4

22 2/)(

2

1)( σµ

πσ−−= zezp

)](),[( σµσµ +−

)]2(),2[( σµσµ +−

DKVishwakarma

5DKVishwakarma

Rayleigh Noise:

• The PDF of Rayleigh Noise is given by;

• Where the mean and variance is given by

<

≥−=−−

azfor 0

for )(2

)(/)( 2

azeazbzp

baz

6

4/ba πµ +=4

)4(2 πσ

−=b

DKVishwakarma

Erlang (Gamma) noise:

• The pdf of Erlang Noise is given by;

• Where a>0, b is a positive integer and the mean and

variance are given by;

<

≥−−=

−−

0zfor 0

0for )()!1()(

1

zeazb

za

zpaz

bb

variance are given by;

7

a

b=µ

2

2

a

b=σ

DKVishwakarma

Exponential Noise:

• The PDF of exponential noise is given by;

• Where a>0. The mean and Variance is given by;

<

≥=

0z 0

0for ae)(

-az zzp

• This PDF is special case of Erlang PDF with b=1.

8

a

1=µ

2

2 1

a=σ

DKVishwakarma

Uniform Noise:

• The PDF of uniform noise is given by

≤≤=

otherwise 0

afor a-b

1

)(bz

zp

12

)( 22 ab −=σ

2

ba +=µ

9DKVishwakarma

Impulse Noise (Salt and pepper Noise):

• The PDF of (bipolar) impulse noise is given by;

• If b>a, intensity b will appear as light dot in the

=

=

=

otherwise 0

for

for

)( bzP

azP

zp b

a

image. If Pa or Pb is zero, the impulse noise is called

unipolar.

10DKVishwakarma

Image Degradation with Additive Noise:

Original image

Degraded images

),(),(),( yxyxfyxg η+=

11

Histogram

DKVishwakarma

Original image

Degraded images

Image Degradation with Additive Noise (cont.)

),(),(),( yxyxfyxg η+=

12

Histogram

DKVishwakarma

Periodic Noise:

• If the image signal is subject to a periodic, rather

than a random disturbance, we might obtain an

image corrupted by periodic noise. The effect is of

bars over the image.

• This noise is generated from electrical or

electromechanical interference during imageelectromechanical interference during image

acquisition.

13

The

twins

image

corrupt

ed by

periodic

noise

In

freque

ncy

domain

DKVishwakarma

MATLAB CODE ‘salt & pepper’

• ‘salt & pepper’ noise

• Also called as impulse

noise, shot noise, or

binary noise.

x=imread('cameraman.jpg');

y=imnoise(x, 'salt & pepper'

,0.2); % 20% pixels get

corrupted.

subplot(2,1,1),imshow(x);

subplot(2,1,2),imshow(y);

14DKVishwakarma

Gaussian Noise

• Gaussian noise is an

idealized form of white

noise, which is caused by

random fluctuations in the

signal. We can observe

white noise by watching awhite noise by watching a

television which is slightly

mistuned to a particular

channel.

• y=imnoise(x, ‘Gaussian' , μ,

σ2);

15DKVishwakarma

Speckle noise

• Gaussian noise can bemodelled by random valuesadded to an image; specklenoise (or more simply justspeckle) can be modelledby random valuesmultiplied by pixel values,hence it is also calledmultiplied by pixel values,hence it is also calledmultiplicative noise.

• Speckle noise is a majorproblem in some radarapplications.

• y=imnoise(x,'speckle’ V);

• Zero mean and V(Variance)is by default 0.04

16DKVishwakarma

Periodic Noise

• t=imread('cameraman.jpg')

• s=size(t);

• [x,y]=meshgrid(1:s(1),1:s(2));

• p=sin(x/3+y/5)+1;

• z=(im2double(t)+p/2)/2;

• subplot(2,1,1),imshow(t);• subplot(2,1,1),imshow(t);

• subplot(2,1,2),Imshow(z);

17DKVishwakarma

Recovery of corrupted noise

• ‘salt and pepper’ noise, ‘Gaussian’ noise and

‘speckle’ noise can all be cleaned by using spatial

filtering techniques.

• ‘periodic’ noise, however, requires the use of

frequency domain filtering.frequency domain filtering.

• This is because whereas the other forms of noise can

be modelled as local degradations, ‘periodic’ noise is

a global effect.

18DKVishwakarma

Low pass filtering [Mean]

• Low pass filtering can be applied in spatial domain forremoval of noise;

• Average filter : It can be obtained by using fspecialcommand in MATLAB of any size(mxn).

>>fspecial(‘average’, [7 ,7]);

• Arithmetic mean filter(Moving Average filter):

• It removes local variation in the image and reduce theeffect of blurring.

19

∑∈

=xySts

tsgmn

yxf),(

),(1

),(ˆ

DKVishwakarma

• Geometric mean filter:

• Each restored pixel is product of the pixels in sub image

window, raised to the power 1/mn.

• This filter loses less details as compared to mean filter.

mn

Sts xy

tsgyxf

1

),(

),(),(ˆ

= ∏

∈

• This filter loses less details as compared to mean filter.

• Harmonic filter:

20

∑∈

=

xySts tsg

mnyxf

),( ),(

1),(ˆ

Works well for salt noise but fails for pepper noiseDKVishwakarma

Geometric Mean Filter: Example

Original

image

Image

corrupted

by AWGN

21

Image

obtained

using a 3x3

geometric

mean filter

Image

obtained

using a 3x3

arithmetic

mean filter

AWGN: Additive White Gaussian NoiseDKVishwakarma

Contraharmonic mean filter:

• Contraharmonic filter restored the image based on the

expression;

• Positive Q is suitable for eliminating pepper noise.

∑

∑

∈

∈

+

=

xy

xy

Sts

Q

Sts

Q

tsg

tsg

yxf

),(

),(

1

),(

),(

),(ˆQ = the filter order

• Positive Q is suitable for eliminating pepper noise.

• Negative Q is suitable for eliminating salt noise.

• For Q = 0, the filter reduces to an arithmetic mean filter.

• For Q = -1, the filter reduces to a harmonic mean filter.

22DKVishwakarma

Contraharmonic Filters: Example

Image

corrupted

by pepper

noise with

prob. = 0.1

Image

corrupted

by salt

noise with

prob. = 0.1

23

Image

obtained

using a 3x3

contra-

harmonic

mean filter

With Q = 1.5

Image

obtained

using a 3x3

contra-

harmonic

mean filter

With Q=-1.5

DKVishwakarma

Order-Statistic Filters:

subimage

Original image

Statistic parameters

Mean, Median, Mode,

Min, Max, Etc.

24

Moving

window

Output imageDKVishwakarma

Order-Statistics Filters:[contd..]

Median filter

{ }),(median),(ˆ),(

tsgyxfxySts ∈

=

Max filter

{ }),(max),(ˆ),(

tsgyxfxySts ∈

=Useful for finding brightest point in

the image.

Reduce “dark” noise (pepper noise)

Provide excellent noise Filtering with

considerable less Blurring than linear

smoothing Filter of same size. Effective

for both unipolar and bipolar impulse

noise.

25

),( xySts ∈

Min filter

{ }),(min),(ˆ),(

tsgyxfxySts ∈

=

Midpoint filter

{ } { }

+=

∈∈),(min),(max

2

1),(ˆ

),(),(

tsgtsgyxfxyxy StsSts

Reduce “dark” noise (pepper noise)

Useful for finding the darkest point

in the image.

Reduce “bright” noise (salt noise)

Best for removing

randomly distributed

Gaussian noise.

DKVishwakarma

Median Filter : How it works

A median filter is good for removing impulse, isolated noise

Salt noisePepper noise

Median

Median filtering seems almost tailor-made for removal of ‘salt and pepper noise’.

Median is a middle value of set when they are sorted. If the numbers are even then

mean of middle two values will be median.

Degraded image

Salt noise

Pepper noise

Moving

window

Sorted

array

Filter output

Normally, impulse noise has high magnitude and is isolated. When we sort pixels in

the moving window, noise pixels are usually at the ends of the array. it’s rare that

the noise pixel will be a median value.

DKVishwakarma

Median Filter : Example

Image

corrupted

by salt-and-

pepper

noise with

pa=pb= 0.1

1 2

27Images obtained using a 3x3 median filter

43

DKVishwakarma

Max and Min Filters: Example

Image

corrupted

by pepper

noise with

prob. = 0.1

Image

corrupted

by salt

noise with

prob. = 0.1

28

Image

obtained

using a 3x3

max filter

Image

obtained

using a 3x3

min filter

DKVishwakarma

Median Filter [Example]

• noisy values will end up at the top or bottom of thesorted list. Thus the median will in general replace anoisy value with one closer to its surroundings.

• In Matlab, median filtering is implemented by the medfilt2 function:

>>x=imread(‘cameraman’);

>>y=imnoise(x, ‘salt & pepper’, 0.02);

>> z=medfilt2(y,[5 5]);

29DKVishwakarma

Median filtering [Types of Median filter] cont…

• Median filtering is a special case of a more general

process called rank-order filtering. Rather than take

the median of a set, we order the set and take the nth

value, for some predetermined value of n.

3x3 matrix then ‘Rank order filter’ with 5.1 2 3

4 5 6 3x3 matrix then ‘Rank order filter’ with 5.

5x5 matrix then ‘Rank order filter’

with 13.

30

7 8 9

1 2 3 4 5

6 7 8 9 10

11 12 13 14 15

16 17 18 19 20

21 22 23 24 25

DKVishwakarma

Median filtering [Types of Median filter] cont…

Median would be 3rd of these values.

Median would be 5th of these

values.

0 1 0

2 3 4

0 5 0

0 0 1 0 0

0 0 1 0 0

1 1 1 1 1 values.

In MATLAB it can be implemented using ordfilt2

command. ordfilt2(y,3,[0 1 0; 1 1 1;0 1 0])

31

1 1 1 1 1

0 0 1 0 0

0 0 1 0 0

DKVishwakarma

Question

32DKVishwakarma

Alpha-trimmed Mean Filter

∑∈−

=xySts

r tsgdmn

yxf),(

),(1

),(ˆ

Where gr(s,t) represent the remaining mn-d pixels after removing the d/2

highest and d/2 lowest values of g(s,t).

Formula:

33

highest and d/2 lowest values of g(s,t).

This filter is useful in situations involving multiple types of noise such as a

combination of salt-and-pepper and Gaussian noise.

DKVishwakarma

Alpha-trimmed Mean Filter: Example

Image

corrupted

by additive

uniform

noise

Image

additionally

corrupted

by additive

salt-and-

pepper

noise

1 2

34

Image

obtained

using a 5x5

arithmetic

mean filter

2 Image

obtained

using a 5x5

geometric

mean filter

2

DKVishwakarma

Alpha-trimmed Mean Filter: Example (cont.)

Image

obtained

using a 5x5

arithmetic

mean filter

Image

obtained

using a 5x5

geometric

mean filter

35

Image

obtained

using a 5x5

median filter

Image

obtained

using a 5x5

alpha-

trimmed

mean filter

with d = 5

DKVishwakarma

Adaptive Filter:

- Filter behavior depends on statistical characteristics of

local areas inside mxn moving window.

- More complex but superior performance compared with

“fixed” filters

General concept:

- Adaptive filters are a class of filters which change their

36

- Adaptive filters are a class of filters which change their

characteristics according to the values of the grayscales

under the mask; they may act more like median filters, or

more like average filters, depending on their position

within the image.

- These filters can be used to clean ‘Gaussian noise’ by

using local statistical properties of the values under the

mask.DKVishwakarma

Adaptive Filter[CONTD..]

Statistical characteristics:

Local mean:

2

ησ

Noise variance:

37

∑∈

=xySts

L tsgmn

m),(

),(1

Local mean:

Local variance:

∑∈

−=xySts

LL mtsgmn ),(

22 )),((1

σ

DKVishwakarma

Adaptive, Local Noise Reduction Filter:

Purpose: want to preserve edges

1. If σσσσηηηη2 is zero, ���� No noise

the filter should return g(x,y) because g(x,y) = f(x,y)

2. If σσσσL2 is high relative to σσσσηηηη2, ���� Edges (should be preserved),

the filter should return the value close to g(x,y)

Concept:

38

3. If σσσσL2 = σσσσηηηη2, ���� Areas inside objects

the filter should return the arithmetic mean value mL

( )LL

myxgyxgyxf −−= ),(),(),(ˆ2

2

σση

Formula:

The filter returns a value close to either g(x,y) or mL depending upon the local

variance is high or low.DKVishwakarma

Adaptive Noise Reduction Filter: Example

Image

corrupted

by additive

Gaussian

noise with

zero mean

and σσσσ2=1000

Image

obtained

using a 7x7

arithmetic

mean filter

Image

39

Image

obtained

using a 7x7

geometric

mean filter

Image

obtained

using a 7x7

adaptive

noise

reduction

filter

DKVishwakarma

MATLAB Implementation:

• Adaptive filter can be implemented in MATLAB by

using wiener2 function.

• x=imread('cameraman.jpg');

• y=imnoise(x,'gaussian',0,0.005);

• t2=wiener2(y,[5,5]);

• t3=wiener2(y,[7,7]);• t3=wiener2(y,[7,7]);

• t4=wiener2(y,[9,9]);

• subplot(2,2,1),imshow(y);

• subplot(2,2,2),imshow(t2);

• subplot(2,2,3),imshow(t3);

• subplot(2,2,4),imshow(t4);

40DKVishwakarma

Estimation of Degradation Model

Degradation model:

),(),(),(),( yxyxhyxfyxg η+∗=

Purpose: to estimate h(x,y) or H(u,v)

),(),(),(),( vuNvuHvuFvuG +=or

Why? If we know exactly h(x,y), regardless of noise, we can do

DKVishwakarma41

Methods:

1. Estimation by Image Observation

2. Estimation by Experiment

3. Estimation by Modeling

Why? If we know exactly h(x,y), regardless of noise, we can do

deconvolution to get f(x,y) back from g(x,y).

Estimation by Image Observation:

f(x,y) f(x,y)*h(x,y) g(x,y)

Original image (unknown) Degraded image

Observation

DKVishwakarma42

Subimage

Reconstructed

Subimage

),( vuGs ),( yxgs

),(ˆ yxfs

DFT

DFT

),(ˆ vuFs

Restorationprocess byestimation

),(ˆ

),(),(),(

vuF

vuGvuHvuH

s

ss =≈

Estimated Transfer function

This case is used when we

know only g(x,y) and cannot

repeat the experiment!

Estimation by Experiment

Used when we have the same equipment set up and can repeat

the experiment.

Input impulse image

System

H( )

Response image fromthe system

DKVishwakarma43

H( )

),( vuG

),( yxg),( yxAδ

{ } AyxADFT =),(δ

A

vuGvuH

),(),( =

DFTDFT

Estimation by Modeling:

Used when we know physical mechanism underlying the image

formation process that can be expressed mathematically.

Atmospheric

Turbulence model

6/522 )(),( vukevuH +−=

Example:Original image Severe turbulence

DKVishwakarma44

k = 0.00025k = 0.001

k = 0.0025

Low turbulenceMild turbulence

Estimation by Modeling: Motion Blurring

Assume that camera velocity is ))(),(( 00 tytxThe blurred image is obtained by

dttyytxxfyxg

T

))(),((),( 00

0

++= ∫where T = exposure time.

dxdyeyxgvuG vyuxj∫ ∫=∞ ∞

+− )(2),(),( π

DKVishwakarma45

dtdxdyetyytxxf

dxdyedttyytxxf

dxdyeyxgvuG

T

vyuxj

vyuxj

T

vyuxj

∫ ∫ ∫

∫ ∫ ∫

∫ ∫

++=

++=

=

∞

∞−

∞

∞−

+−

∞

∞−

∞

∞−

+−

∞− ∞−

+−

0

)(2

00

)(2

0

00

)(2

))(),((

))(),((

),(),(

π

π

π

Estimation by Modeling: Motion Blurring (cont.)

[ ]

dtevuF

dtevuF

dtdxdyetyytxxfvuG

T

tvytuxj

T

tvytuxj

T

vyuxj

∫

∫

∫ ∫ ∫

+−

+−

∞

∞−

∞

∞−

+−

=

=

++=

0

))()((2

0

))()((2

0

)(2

00

00

00

),(

),(

))(),(( ),(

π

π

π

DKVishwakarma46

0

Then we get, the motion blurring transfer function:

dtevuH

T

tvytuxj∫ +−=0

))()((2 00),(π

For constant motion ),())(),(( 00 btattytx =

)(

0

)(2 ))(sin()(

),( vbuaj

T

vbuaj evbuavbua

TdtevuH +−+− +

+== ∫ ππ ππ

Motion Blurring Example

For constant motion

)())(sin()(

),( vbuajevbuavbua

TvuH +−+

+= πππ

DKVishwakarma47

Original image Motion blurred image

a = b = 0.1, T = 1

Removal of Periodic Noise: using frequency domain filtering

• Periodic noise may occur if the imaging equipment

(the acquisition or networking hardware) is subject to

electronic disturbance of a repeating nature, such as

may be caused by an electric motor.

• We can easily create periodic noise by overlaying an

image with a trigonometric function.image with a trigonometric function.

• Periodic noise appears as concentrated bursts of

energy in the Fourier transform, at locations

corresponding to the frequencies of the periodic

interference.

• The approach is use to filter these periodic noise is

selective filtering.

48DKVishwakarma

Selective Filtering

• Selective filtering can be classified as;

1) Band Reject

2) Band Pass

3) Notch Filter

DKVishwakarma49

Periodic Noise Reduction by Freq. Domain Filtering Degraded image

Periodic noise

can be reduced by

setting frequency

components

corresponding to

noise to zero.

DKVishwakarma50

Band reject filter Restored image

MATLAB Implementation:

DKVishwakarma51