1. Introduction1. Introduction

DegradationsDegradations

• Noises (Dot/Pattern)Noises (Dot/Pattern)

• Illumination Imperfections Illumination Imperfections (Brightness /Contrast)(Brightness /Contrast)

• Color ImperfectionsColor Imperfections

• BlurringBlurring

Image BlurImage Blur

• Out-of-Focus BlurOut-of-Focus Blur

• Aberrations in the optical systemsAberrations in the optical systems

• Motion BlurMotion Blur

• Atmospheric Turbulence BlurAtmospheric Turbulence Blur

In Addition to these blurring effects, In Addition to these blurring effects, noisenoise always corrupts any recorded always corrupts any recorded image.image.

Image Restoration Image Restoration

= Image Deblurring = Image Deblurring

= Image Deconvolution= Image Deconvolution

= is concerned with the = is concerned with the reconstruction or estimation of the reconstruction or estimation of the uncorrupted image from a blurred uncorrupted image from a blurred and noisy oneand noisy one

g(x,y) ˆ ( , )f x y

Blind Image DeconvolutionBlind Image Deconvolution

• Step #1 : Blur identificationStep #1 : Blur identification

• Step #2 : Image restoration Step #2 : Image restoration

Image Restoration :- Image Restoration :-

needsneeds

• Characteristics of the degrading Characteristics of the degrading systemssystems

• Characteristics of noiseCharacteristics of noise

(prior knowledge) (prior knowledge)

ทำ��ไมภ�พจึงเสี ยไป ทำ��ไมภ�พจึงเสี ยไป ((ต้�นต้�นเหต้�เหต้�) :- ) :-

f(x, y )

ภ�พในธรรมช�ต้�

d(x, y )

สี�เหต้�สี�เหต้�η(x, y )

noise

g(x, y )

ภ�พทำ �เสี ยไปแล้�ว

g(x, y ) = d (x, y ) * f (x, y ) + η (x, y ) Spatial Domain

1)

Blur ModelBlur Model

Spatial Domain

ทำ��ไมภ�พจึงเสี ยไป ทำ��ไมภ�พจึงเสี ยไป ((ต้�นต้�นเหต้�เหต้�) :- ) :-

F(u,v)

ภ�พในธรรมช�ต้�

D(u, v)

สี�เหต้�สี�เหต้�χ(u, v)

G(u, v)

ภ�พทำ �เสี ยไปแล้�ว

G(u, v) = D(u, v)F(u, v) + χ(u, v) Spectral Domain

2)

Blur ModelBlur Model

Frequency Domain

กระทำ�� กระทำ�� Image RestorationImage Restoration เพ!�อเพ!�อ

G(u,v)

ภ�พทำ �เสี ยไปแล้�ว

H(u, v)

ออกแบบออกแบบFiltFilterer

χ(u, v)ภ�พทำ �ได้�คื!นม�

ˆ ( , )f x yifft

ˆ ( , )F x y

2. Blur Models2. Blur Models

เพ!�อศึกษ�ธรรมช�ต้�ของ เพ!�อศึกษ�ธรรมช�ต้�ของ

d (xd (x, y ) or D(u,v) ) or D(u,v)

ซึ่�งเร ยกว*� ซึ่�งเร ยกว*� Point-spread Point-spread FunctionFunction (PSF) (PSF) หร!อ หร!อ Degradation function Degradation function หร!อ หร!อ Blurring functionBlurring function

The blurring of images is modeled in (1) The blurring of images is modeled in (1) as the as the convolutionconvolution of an ideal image (f of an ideal image (f or F) with a 2-D point-spread function or F) with a 2-D point-spread function (PSF), d or D.(PSF), d or D.

คื�ณสีมบ,ต้�ทำ �สี��คื,ญของ คื�ณสีมบ,ต้�ทำ �สี��คื,ญของ PSF PSF ของของสี�เหต้� สี�เหต้�

• Spatially invariant – image is blurred Spatially invariant – image is blurred in exactly the same way at every in exactly the same way at every locationlocation

• D or d takes on non-negative valuesD or d takes on non-negative values

• D or d is real valuesD or d is real values

• D or d is modeled as passive operation D or d is modeled as passive operation – no energy is absorbed or generated– no energy is absorbed or generated

2.1 No Blur2.1 No Blur

In case the recorded image is imaged In case the recorded image is imaged perfectly, no blur will be apparent in perfectly, no blur will be apparent in the discrete image.the discrete image.

d(x,y) = (x,y) (delta)d(x,y) = (x,y) (delta)

elsewhere 0

0 y x if 1),( yx กล้

�ง6)

2.2 Linear Motion Blur2.2 Linear Motion Blur

Motion blurMotion blur

• Translation ***** Translation ***** ระยะทำ�ง ระยะทำ�ง (L)(L)

• Rotation ****Rotation **** ม�ม ม�ม ((ว,ด้เทำ ยบก,บแกนว,ด้เทำ ยบก,บแกนนอนนอน))

• Sudden change of scale (Sudden change of scale (ย*อย*อ//ขย�ยขย�ย))

• Combinations of theseCombinations of these

elsewhere 0

tan y

x and

2

L x if

1

),:,(22

y

LLyxd

7a)

L = 50, phi = 45 degree

2.3 Uniform Out-of-Focus 2.3 Uniform Out-of-Focus BlurBlur

D/d D/d เป.นแผ่*นวงกล้มเป.นแผ่*นวงกล้ม-disk-disk

elsewhere 0

R x if 1

):,(222

2y

RRyxd

8a)

R = 10

2.4 Atmospheric 2.4 Atmospheric Turbulence BlurTurbulence Blur

D/d = Gaussian FunctionD/d = Gaussian Function

2

22

2exp):,(

GG

yxCyxd

9a)

3.3.Image Restoration Image Restoration AlgorithmsAlgorithms

ว�ธ แก�ไขคืว�ม ว�ธ แก�ไขคืว�ม blurblur

Let Let h(nh(n11,n,n22)) be PSF of the linear filter. be PSF of the linear filter.

),(*),(),(ˆ212121 nngnnhnnf

ภ�พทำ �ได้�คื!นม�

PSF ของ filter ภ�พ blur ทำ �ม อย0*ก�รกระทำ��

convolution

),(),(),(ˆ vuGvuHvuF

ObjectiveObjective

...is to design appropriate restoration ...is to design appropriate restoration filters (h, H)filters (h, H) for use in Eq. 10 for use in Eq. 10

Measurement of restoration qualityMeasurement of restoration quality

Signal-to-noise-ratio (SNR)Signal-to-noise-ratio (SNR)

10

of blurred image

variance of the original image, f10log

variance of the difference image, g-f

g

g

SNR

SNR

dB

f-f̂ image, difference theof variance

f image, original theof variancelog10

image restored of

10ˆ

ˆ

f

f

SNR

SNR

dB

ˆ

10

variance of the difference image, g -f10log

ˆvariance of the difference image, f-f

gfSNR SNR SNR

SNR

dB

3.1 Inverse Filters3.1 Inverse Filters

An inverse filter is a linear filter whose An inverse filter is a linear filter whose point-spread function, point-spread function, hhinvinv(n(n11,n,n22)) is the is the

inverse of the blurring function, inverse of the blurring function, d(nd(n11,n,n22).).

),(

1),(

vuDvuH inv 13)

น��คื*� น��คื*� HH ทำ �ออกแบบแล้�วน 1แทำนคื*�ล้งทำ �ออกแบบแล้�วน 1แทำนคื*�ล้งในสีมก�ร ในสีมก�ร 10 (10 (กรณ ไม*คื��นงถึง กรณ ไม*คื��นงถึง noisenoise))

),(),(),(ˆ vuGvuHvuF

),(),(

1),(ˆ vuG

vuDvuF

),(),(

),(),(),(ˆ vuF

vuD

vuFvuDvuF

จากสมการ 10

จากสมการ 2

น��คื*�ใน น��คื*�ใน ม�กระทำ�� ม�กระทำ�� inverse inverse Fourier transform Fourier transform จึะได้�จึะได้�

),(ˆ vuF

)),(ˆ(2),(ˆ vuFifftyxf

กรณ คื��นงถึง กรณ คื��นงถึง noisenoise ด้�วยด้�วย

),(),(),(ˆ vuGvuHvuF

1ˆ ( , ) ( , ) ( , ) ( , )( , )

F u v D u v F u v W u vD u v

),(

),(),(),(ˆ

vuD

vuWvuFvuF *14**

χ

χ

เม!�อน��คื*�ใน เม!�อน��คื*�ใน ม�กระทำ�� ม�กระทำ�� inverse Fourier transform inverse Fourier transform จึะได้�ภ�พจึะได้�ภ�พกล้,บม� แต้* กล้,บม� แต้* noise noise ทำ �ม อย0*ในภ�พก3จึะถึ0กทำ �ม อย0*ในภ�พก3จึะถึ0กขย�ยจึนเห3นได้�อย*�งช,ด้เจึน เพร�ะเทำอมทำ � ขย�ยจึนเห3นได้�อย*�งช,ด้เจึน เพร�ะเทำอมทำ � 22 ของสีมก�รของสีมก�ร 14) 14) กล้*�วคื!อ กล้*�วคื!อ

1 )1 )ผ่ล้ห�รไม*สี�ม�รถึน�ย�ม ถึ�� ผ่ล้ห�รไม*สี�ม�รถึน�ย�ม ถึ�� D(u,v)D(u,v) ม คื*�ม คื*�เทำ*�ก,บศึ0นย4เทำ*�ก,บศึ0นย4

2)2) ผ่ล้ห�รจึะม คื*�ม�กม�ย ถึ�� ผ่ล้ห�รจึะม คื*�ม�กม�ย ถึ�� D(u,v)D(u,v) ม คื*�ม คื*�น�อยเข��ใกล้�ศึ0นย4น�อยเข��ใกล้�ศึ0นย4

),(ˆ vuF

3.2 Least-Squares Filters3.2 Least-Squares Filters

3.2.1 The Wiener Filter3.2.1 The Wiener Filter

3.2.2 The Constrained 3.2.2 The Constrained Least-squared FilterLeast-squared Filter

3.2.1 The Wiener Filter3.2.1 The Wiener Filter

The Wiener filter is a linear spatially The Wiener filter is a linear spatially invariant filter of the forminvariant filter of the form

),(*),(),(ˆ212121 nngnnhnnf

in which the point-spread function in which the point-spread function h(nh(n11,n,n22)) is chosen such that it is chosen such that it

minimizes the mean-squared error minimizes the mean-squared error (MSE) between the ideal and restored (MSE) between the ideal and restored image.image.

22121 )),(ˆ),(( nnfnnfEMSE

1

0

1

0

2

2121

1 2

),(ˆ),(N

n

M

n

nnfnnfMSE

Expectation = Mean

0)(

2

2

h

MSE

The minimization problem,The minimization problem,

has solution (in spectral domain)has solution (in spectral domain)

),(

),(),(),(

),(),(

*

*

vuS

vuSvuDvuD

vuDvuH

f

ww

16)

DD** (u,v) (u,v) = = complex conjugate ofcomplex conjugate of D(u,v)D(u,v)

SSww (u,v) (u,v) = the power spectrum of the noise = the power spectrum of the noise

SSff (u,v) (u,v) = the power spectrum of the ideal = the power spectrum of the ideal

imageimage

Estimation of Estimation of SSww (u,v) (u,v)

1) In the case 1) In the case SSww (u,v) (u,v) = 0 = 0, ,

noiselessnoiseless, we have, we have

),(),(

),(),(

*

*

vuDvuD

vuDvuHw

0 v)D(u,for 0

0 v)D(u,for ),(

1

),( vuDvuHw

17)

2) In the case 2) In the case SSww (u,v) (u,v) << << SSff (u,v) (u,v) , ,

the Wiener filter approaches the the Wiener filter approaches the inverse filter.inverse filter.

0 v)D(u,for 0

0 v)D(u,for ),(

1

),( vuDvuHw

3) In the case 3) In the case SSww (u,v) (u,v) >> >> SSff (u,v) (u,v) , the , the

Wiener filter acts as a frequency Wiener filter acts as a frequency rejection filter, rejection filter, HHww(u,v)(u,v) -> 0 -> 0..

4) In the case 4) In the case the noise is white noisethe noise is white noise, ,

18)18)

The estimation of noise variance can be The estimation of noise variance can be left to the user as if it were a tunable left to the user as if it were a tunable parameter.parameter.

Small values of will yield a result Small values of will yield a result close to the inverse filter, while large close to the inverse filter, while large values will over-smooth the restored values will over-smooth the restored image.image.

2( , )w wS u v

2

Estimation of Estimation of SSff (u,v) (u,v)

1) Replace 1) Replace SSff (u,v) (u,v) by an by an

estimate of the power estimate of the power spectrum of spectrum of the blurred the blurred imageimage and and variance of noisevariance of noise,,

2 * 21( , ) ( , ) ( , ) ( , )f g w wS u v S u v G u v G u v

MN

19)

2) Replace 2) Replace SSff (u,v) (u,v) by an by an

estimate of the power estimate of the power spectrum of spectrum of the the representative images.representative images.

3) Estimate 3) Estimate SSff (u,v) (u,v) by using by using

statistical model (Eq. 20a)-statistical model (Eq. 20a)-b)).b)).

3.2.2 The Constrained 3.2.2 The Constrained Least-Squares FilterLeast-Squares Filter

g(x,y)

ˆ( , ) * ( , ) ( , )d x y f x y g x y

h(x,y)

d(x,y)ˆ ( , )f x y

แท้จร�ง

สรางขึ้��น

21)

ˆ( , ) ( , ) * ( , ) 0g x y d x y f x y

1 2

2

21 1

1 2 1 2 1 20 0

2

ˆ( , ) ( , ) * ( , )

ˆ= ( , ) ( , ) * ( , )N M

k k

w

g x y d x y f x y

g k k d k k f k k

Introduce c() PSF of high-pass filter, then we have the solution as the following Eq.

*

* *

( , )( , )

( , ) ( , ) ( , ) ( , )cls

D u vH u v

D u v D u v C u v C u v

Tunable parameter

3.3 Iterative Filters3.3 Iterative Filters

4. Blur Identification 4. Blur Identification AlgorithmsAlgorithms

1. ITU1. ITU

International Telecommunications UnionInternational Telecommunications Union