Digital Image Processing L t 10

p. 1

Lecture 10 Noise reduction Noise reduction

Noise models and noise estimation Using linear filters, ex) average filters Using non linear filters ex) the median filter Using non-linear filters, ex) the median filter Using adaptive filters (spatial variant) Periodic Noise Reduction by Frequency domain filtering

Image restoration Image restoration The degradation model Inverse filtering Least square filtering Least square filtering Wiener filtering Estimating the degradation function An often cited standard method: Richardson–Lucy deconvolution An often cited standard method: Richardson Lucy deconvolution

(not in Gonzalez & Woods) (Extra, optional) Gonzales & Woods:

Chapter 5, pp. 311-362 Paperback 2013: Add 22 to page numbers Chapter 5, pp. 311 362 Paperback 2013: Add 22 to page numbers Chapter 4, pp. 294-298 Paperback 2013: Add 22 to page numbers

Maria Magnusson, CVL, Institutionen för Systemteknik, Tekniska Högskolan, Linköpings Universitet

Some important noise modelsd th i i ti

p. 2

and their use in practice Gaussian noise Gaussian noise

Easy to use mathematically. Therefore often used in practice even if the model is not perfect.

Electronic circuit noise and sensor noise due to poor *

pillumination* or high temperature. *= rather poisson noise

Rayleigh noise Occurs in range imaging. Can model skewed histograms.

Gamma and Exponential noise Occurs in laser imaging. Can be used for approximating

skewed histograms.U if i Uniform noise Not so practical, but can be useful in random number

generation in simulations. Salt-and-pepper (impulse) noise Salt-and-pepper (impulse) noise

Quick transients due to as faulty switching during imaging Periodic noise

Electrical or electromechanical interference during image Electrical or electromechanical interference during image acquisition

Newspaper printing

S i t t i d l

p. 3

Some important noise models

Fig. 5.2Eq. (5.2-1)-(5.2-14)

A simple imageith diff t i

Fig. 5.4

p. 4

with different noise

A simple imageith diff t i

Fig. 5.4

p. 5

with different noise P i i

p. 6

Poisson noise probability mass function: probability mass function:

,...1,0för,!

Pr kekakN a

k

Mean and variance:!k

aa 2and Is used to statistically characterize the distribution of

photons per unit of area

aa NN and

photons per unit of area. Important for normal camera images, and medical image

systems using X-rays and gamma-rays.systems using X rays and gamma rays. Also called Poisson noise, Photon noise or Quantum noise.

P i i

p. 7

Poisson noise Previous slide gave that Poisson noise has equal mean

and variance:It b i t d ith G i i ith

aNN 2 It can be approximated with Gaussian noise with

aNN 2

P i i

p. 8

Poisson noise The standard

deviation

i ith th

aN

increase with the signal value

B t the signal to

aN

But the signal to noise ratio

improve! aaa

M di filt t h i

p. 9

Median filter, techniqueTh di filt The median filter moves over the in-image, similarly as during convolution.du g co o ut o

The values in the in-image, under the filter, are noted. E l 1 1 9 2 3 3 3 3 3Example: 1 1 9 2 3 3 3 3 3

The values are sorted to: 1 1 2 3 3 3 3 3 91 1 2 3 3 3 3 3 9

The median value becomes 3 in the example. It is put in Image with movingp pthe out-image.

g gmedian filter

Sorting is gcomputationally

heavy!

Median and average filter i

p. 10

comparison

Original WithOriginal image

With salt- andpepper

inoise

AfterAfter medianfilter3x3

averagefilter 3x3 3x33x3

P ti f th di filtp. 11

Properties of the median filter Edges are

preserved.N i i d Noise is suppressed (especially salt-and-pepper noise)and pepper noise).

Thin lines are destroyed.destroyed.

Smooth surfaces arise.

The median filter b li d l ti

p. 12

can be applied several times

Image AfterImage with salt-and-pep-

i

Aftermedianfilter 3x3,1 tiper noise 1 time

AfterAftermedianfilter 3x3,3 times

medianfilter 3x3,2 times

Fig. 5.10

3 times2 times

Median and average filter i

p. 13

comparison Lab 7

Image Salt andImage with additive niform

Salt- and pepper noisedd duniform

noiseadded

After AfterAfteraveragefilter 5x5

Aftermedianfilter 5x5

Fig. 5.12

Adaptive filtering p. 14

(wiener2 according to MATLAB)C t d i f i N2 i th Compute mean and variance of size N2 in the neighborhood :

2f f 2

,

2

2

,

N

yxfyx

2

,,

N

yxfyx ≈(5.2-15) ≈(5.2-16)

The adaptive filtering expression:

NN

p g p

yxfyxg n ,, 2

22(5.3-12)

where is the variance of the noise.

yfyg ,, 2

2n

Adaptive and average filter i

p. 15

comparison Lab 7

ImageImage with additive Ga ssian

Afteraveragefilter 7x7Gaussian

noisefilter 7x7

AfterAfteradaptivefilter 7x7

Fig. 5.13

Periodic noise reduction by f d i filt i

p. 16

frequency domain filtering Fig. 5.16

Image Fou-Image with sinus-oidal

Fou-rier trans-formoidal

noiseform

Re-Butter-sult afterfilte-

worthband-reject te

ringeject

filter

B d j t filt ith ti

p. 17

Bandreject filters with equations

Fi 5 15Fig. 5.15

Tab. 4.6

OBS!

Notch (reject) filtersill t t d

p. 18

illustrated

Fig. 5.18

Notch (reject) filtersith ti

p. 19

with equations OBS!

( ) T b 4 4

,,,Q

kkNR vuHvuHvuH

(4.10-2):be shouldh Butterwort Lowpass

Tab. 4.4

,and,where

,,,1

kk

kkkNR

vuHvuH

vuHvuHvuH

,),(1

1,2

vuDvuH

n

lyrespectiveandat centered filters highpass are

kkkk vuvu radialtheiswhere

),(10

vuD

D

lyrespective, and , kkkk vuvu

Tab. 4.5

center thefrom distance

radial theis , where vuD

0 +101

Ex) Notch filt

p. 20

filtersFourier

S t llit

Fouriertransform

NotchSatelliteimage of Florida

Notch pass filter

Result after notchnotch reject filter Result

afterafter notch pass filter

Fig. 5.19

filter

Ex) Notchfilt

p. 21

filters

Sampled newsnews-paperimage

Fouriertransform

Result after

Notch reject after

notch reject filt

reject filter applied on the filter

Fig. 4.64

on the Fourier transform

Image restoration and I h t

p. 22

Image enhancement Image restoration

To recover a degraded image An objective process An objective process Uses a priori knowledge of the degradation phenomenon Example: Removal of image blurp g

Image enhancement To make an image easier to study A heuristic process Take advantage of the psychophysical aspects of the

human visual systemhuman visual system Example: Contrast stretching

Th i t ti bl

p. 23

The image restoration problem An image has often been exposed to a known

degradation.

vuNvuFvuHvuGyxnyxfyxhyxg ,,,, (5.1-1)

(5 1-2) vuNvuFvuHvuG ,,,,

Noise:

(5.1-2)

g(x,y)G( )+

n(x,y), N(u,v)

Degradation:h(x y) H(u v)

f(x,y)F( ) G(u,v)h(x,y), H(u,v)F(u,v)

We wish to find out as much as possible about f(x,y).

Th i t ti d l

p. 24

The image restoration model We know h(x,y), H(u,v) and statistical properties

for n(x,y), N(u,v).Noise:

g(x,y)+

Noise:n(x,y), N(u,v)

Degradation:f(x,y) g(x,y)G(u,v)

+h(x,y), H(u,v)f(x,y)F(u,v)

Restaurationfilter

vuF

yxf,ˆ,ˆ

Fig. 5.1

We look for that are as close as vuFyxf ,ˆ,,ˆ

vuF , g

possible to f(x,y), F(u,v). yf ,,,

I t ti l

p. 25

Image restoration, example( )h( )

Lab 7

n(x,y)h(x,y)

+*

( )f(x,y)

g(x,y)

Restorationfilter

yxf ,ˆ

T ti l bl

p. 26

Two partial problems Find the model (i.e. h(x,y) and n(x,y)) that best

describes the degradationFi d th t ti filt th t b t t th Find the restoration filter that best restores the original image.

N i

g(x y)+

Noise:n(x,y), N(u,v)

Degradation:f(x y) g(x,y)G(u,v)

+Degradation:h(x,y), H(u,v)

f(x,y)F(u,v)

Restaurationfilter

vuF

yxfˆ

,ˆ

vuF ,Fig. 5.1

Th t ti th d

p. 27

Three restoration methods 1) Inverse filtering 1) Inverse filtering 2) Constrained least squares filtering 3) Wiener filtering 3) Wiener filtering

(minimum mean square error filtering)

ExamplesExamples of images that can be restoredof images that can be restored Images with out-of-focus blur Images with motion blur Images with motion blur Images distorted by with atmospheric blur Some microscopy images, especially the z-py g p y

direction for 3D confocal microscopy

T t i

p. 28

Test imageh d d d h The test image is an aerial image degraded with

atmospheric turbulence. (5.6-3)

Degradation model: Fig. 5.25

Test image Degraded image, k=0.0025 Degraded image, k=0.001

1) Inverse filtering anddifi d i filt i

p. 29

-modified inverse filtering

vuGvuF ,,ˆ (5 7-1) vuHvuF

,, (5.7 1)

vuGF ,ˆ

vuH

vuF,

,,

yxnyxfyxhyxg ,,,,Compare

vuNvuFvuHvuG ,,,,with:

R lt ith I filt i

p. 30

Result with Inverse filteringI fil i li d h d d d i k 0 0025 Inverse filtering applied to the degraded image, k=0.0025.

For high frequencies, the noise dominates over the signal. Therefore the inverse filter is preferably limited to a radius e e o e e e se e s p e e ab y ed o a ad usless than W.Here: W=40, W=70 and W=“the band limit”.

Fig. 5.27

Full inverseCut off outside 40 Cut off outside 70

2) Constrainedl t filt i

p. 31

least squares filtering

22

* ,,,ˆ vuGvuHvuF (5 9-4)

22 ,,,

vuPvuH (5.9 4)

Where |P(u,v)| is the absolute value of the Laplacian filter, i.e. 2224P Laplacian filter, i.e.

or P(u,v) is the Fourier

2224, vuvuP

010 or P(u,v) is the Fouriertransform of

141010

, yxp (5.9-5)

010

, yp

Proof for Constrainedl t filt i

p. 32

least squares filteringTh b diff F h i i i I( ) b l

vuFHGvuI ˆ 2

There can be many different F that minimizes I(u,v) below:

vuFHGvuI ,,

The negative Laplace filter enhances high frequencies:

2224 vuvuP

The F that minimize the equation below cannot

4, vuvuP

ˆˆ 22

The F that minimize the equation below cannot contain too much high frequencies:

vuFPFHGvuI ,,

Proof for Constrainedl t filt i

p. 33

least squares filtering cont.Mi i i hi

Equivalent

vuFPFHGvuI ˆˆ 22

Minimize this eq:q

equations. Willbe shown on the

white board if vuFPFHGvuI ,,

Rewrite it to:

white board if we have time.

vuGHGHFvuI 1ˆ 2

22*

vuGFvuI ,1,

This gives the constrained

l t filt i22 PH least squares filtering

equation directly!

Example) frequency response for p. 34

constrained least squares filtering

1

H 0,1uH

ku

vuHsinc

,

0,uH kvsinc

22

* 0,uH

,

22 0,0, uPuH

It ti d t i ti f

p. 35

Iterative determination of f f 1) Specify an initial value of

2) Compute2 2

F̂HG ≈(5.9-6) and (5.9-7)

3) Stop if is sufficiently small. Otherwise chose a new and go to 2.

Result with constrainedl t filt i

p. 36

least squares filtering

Fig. 5.31Fig. 5.25

Not optimal Iteratively determined Degraded test image

3) Wi filt i

p. 37

3) Wiener filtering

där,,,ˆ vuWvuGvuF

vuHvuW ,, 2

*

(5.8-2)

vuSvuSvuH FN ,,, 2

whereis the power spectrum of the un degraded image

vuSvuS NF ,and,is the power spectrum of the un-degraded image and noise, respectively.

Note that vuSvuS NF ,,is proportional to the SNR (signal-to-noise ratio).

NF ,,

Wh Wi filt ?

p. 38

Why Wiener filter?Th Wi filt i li Norbert Wiener The Wiener filter is a linear filter that is optimal in the sense that it minimizes the

Norbert Wiener, USA,1894-1964se se t at t es t e

mean square error between the undistorted image f(x,y) and the restored imageand the restored image yxf ,ˆ

i.e. it minimizes

22 ˆ

2 ffEe

Autocorrelation and t

p. 39

power spectrum

dbdabyaxfbafyxrf

,,,

The autocorrelation function in the Fourier domain:in the Fourier domain:

vuSvuFvuFvuFyxr Ff ,,,,, 2* y Ff ,,,,,

The autocorrelation function and theThe autocorrelation function and the power spectrum is a Fourier pair:

j 2

dydxeyxrvuS yvxujfF

2,,

Autocorrelation, power spectrum p. 40

and stochastic signals (ex) noise)Thi i lid f t h ti i l

byaxnbanEyxrn ,,,This is valid for stochastic signals:

yyn ,,,

2 xn xrnWhite noise:

2uNuSN

By using the autocorrelation function and the power

x x u

By using the autocorrelation function and the power spectrum, we can make calculations with noise!

(Example of a li ti i )

p. 41

more realistic noise)

Bandlimited white noise:

xn xrn 2uNuSN

f

x x u

1

1f

xn efxr 1

21

21

2

11

fuuNuSN

1f

Example) frequency response f t Wi filt

p. 42

for two Wiener filters

vuH , kusinc,

kvsinc

dB10,0, SNRuW

dB200 SNRW dB20,0, SNRuW

Result with approximate Wiener filt i i t ti l h K

p. 43

filtering, interactively chosen K H *

KvuH

vuHvuW

2

*

,,, (5.8-6)In Lab 7, we will

use the non-approximate

Fig. 5.28

KvuH ,Wiener filter

Inverse, cut off outside 70 Wiener Filter, (5.8-6)Degraded test image

What do we need to know to be bl t f Wi filt i ?

p. 44

able to perform Wiener filtering?Th t l ti f ti f th i ( ) The autocorrelation function of the noise, rn(x,y) Ex) rn(x,y)=n2 (x,y) for white noise with variance n2

The autocorrelation function of the image, rf(x,y)Ex) 22 Ex) 222, yx

ff yxr

The degradation function h(x,y) Ex) the Gauss-like function that models atmospheric

turbulenceturbulence

The autocorrelation functionf th i ( )

p. 45

of the noise, rn(x,y) The noise can be measured here. Suppose that

the noise is white so that rn(x,y)=n2(x,y). The variance 2 can be calculated here in the square variance n2 can be calculated here in the square of size N2:

22

, yxfyx

2,2

Nyx

n ≈(5.2-16)

,

, yxfyx

where

2,

Nyx ≈(5.2-15)

The autocorrelation function of th i ( )

p. 46

the image, rf(x,y)

byaxfbafyxrf ,,,a b

f f bafyxfyxfyxr ,,,, 2

Th t l ti f ti i l i l i

a b

f

The autocorrelation function is always maximal in the point (x,y)=(0,0). This is valid for all images.

The autocorrelation function is always The autocorrelation function is always symmetrical, i.e. rf(x,y)=rf(-x,-y)

The autocorrelation function f th i ( )

p. 47

of the image, rf(x,y) Lab 7

This model is good for natural images:

222 yxff yxr

2f , ff yxr

yf

x It is supposed that It is supposed that

images of the same type, for yp ,example faces or license plates, h th have the same

The autocorrelation function f i ( )

p. 48

of one image, rf(x,y) Lab 7

Circular convolution effects in

222, yxff yxr

Circular convolution effects in the outer area of the image because the computation was

f d i th F iff

performed in the Fourier domain.

3 ways to estimate the degradation p. 49

function H(u,v) 1) Estimation by image observation

If the image is blurred, we can look at a small rectangular section of the image containing sample rectangular section of the image containing sample structures, like part of the object and the background

Look at areas with high contrast to diminish the effect of inoise

Compute

vuG vuF

vuGvuHs

ss ,ˆ

,, (5.6-1)

where gs(x,y) is the observed sub-image and fs(x,y) is the estimated original image in the actual sub-area

3 ways to estimate the degradation p. 50

function H(u,v) Estimation by experimentation

Show an impulse image to the system and observe an image g(x y)image g(x,y).

Compute

vuGvuH ,, (5.6-2)

where G(u,v) is the Fourier transform of the observed image and A is the strength of the impulse

A

vuH , (5.6 2)

image and A is the strength of the impulse. Estimation by modeling

Ex1) Use the model for atmospheric turbulence Ex1) Use the model for atmospheric turbulence Ex2) This is valid for linear motion

vbuajbTH i vbuajevbuavbua

vuH

sin, (5.6-11)

System identification & image restoration p. 51

with constrained least squares filteringa) Image restoration b) System identification

n(x,y)a) Image restoration

? n(x,y)b) System identification

?g(x,y)

+h(x,y)f(x,y) g(x,y)

+h(x,y)f(x,y)

?

+h(x,y)

Mi i i

+h(x,y)

Mi i iMinimize:

22 ˆˆ FLFHG

Minimize:

22 ˆˆ HLFHG FLFHG HLFHG *

ˆ GHF *ˆ GFH

22 LHF

22 LFH

An often cited standard method:p. 52

Extra,

Richardson–Lucy deconvolutionTh Ri h d L l i h l k L

optio-nal

The Richardson–Lucy algorithm, also known as Lucy–Richardson deconvolution, is an iterative procedure for recovering a latent image f(x,y) that has been blurred by a known point spread function, h(x,y).

The observed image is denoted g(x,y). The statistics are performed under the assumption that are The statistics are performed under the assumption that are

Poisson distributed, which is appropriate for photon noise in the data:

yxh

hfyxgyxfyxf jj ,ˆ

,,ˆ,ˆ1

The start image f^0 can be a constant image.

yxhyxf j

jj ,,

g 0 g Matlab experiments on next page: original, blurred and

noisy, restorated with 5 iterations, and 15 iterations.

Matlab experiment p. 53