16404857 digital image processing 07 image restoration noise removal
TRANSCRIPT
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
1/53
1
TDI2131 Digital Image Processing
Image Restoration & NoiseRemoval
Lecture 7
John See
Faculty of Information TechnologyMultimedia University
Some portions of content adapted CYPee's notes, MMU. Most figures from Gonzalez/Woods
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
2/53
2
Lecture Outline
Degradation & Restoration Process Models
Noise Models
Restoration in the presence ofnoise only (spatial
filtering) Restoration of Periodic Noise
Restoration in the presence ofdegradation & noise(frequency domain filtering)
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
3/53
3
Some Announcements
Reminder: Assignment 1 is due this Friday.
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
4/53
4
Image Restoration
Image Restoration To improve the appearance of an image
by application of a restoration process that uses a
mathematical model for image degradation
Types of degradation:
Blurring caused by motion or atmospheric disturbance
Geometric distortion caused by imperfect lenses
Superimposed interference pattern caused by mechanical systems
Noise from electronic sources
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
5/53
5
How are these images restored?
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
6/53
6
Image Restoration: The Idea
Example
degraded
images
Knowledge of
image creation
process
Developdegradation
model
Developinverse degradation
process
Apply
inverse degradationprocess
Input
imageg(x,y)
Output
image ( , ) f x y
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
7/53
7
Degradation/Restoration Process Model
Consists of 2 parts Degradation function & Noise function
General model in spatial domain:
g(x,y): degraded image, h(x,y): degradation function,
f(x,y): original image, n(x,y): additive noise function
),(),(*),(),( yxnyxfyxhyxg +=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
8/53
8
Degradation Model in Freq. Domain
General model in frequency domain:
G(x,y): Fourier transform of degraded image,
H(x,y): Fourier transform of degradation function,F(x,y): Fourier transform of original image,
N(x,y): Fourier transfor of additive noise function
What needs to be done??
FINDDegradation function and Noise model
),(),(),(),( vuNvuFvuHvuG +=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
9/53
9
Noise Models
Noise Any undesired information that contaminates
an image Variety of sources:
Digital image acquisition process, e.g. For CCD (chargedcoupled device) camera, electronics signal fluctuations in
detector, caused by thermal energy and light levels
During image transmission, e.g. Wireless network, may becorrupted by lightning or other atmospheric disturbance
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
10/53
10
Noise Models
Assumptions of Noise models in this course:
Noise is independent of spatial coordinates (except forspatially periodic noise)
Noise is uncorrelated w.r.t the image (pixel values)
Spatial Noise Descriptor concern of the statistical behavior
of the intensity values in the noise component of the model
Characterized by probability density function (PDF)
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
11/53
11
Noise Models
Typical image noise models that can be modeled:
Gaussian Rayleigh
Erlang (Gamma)
Exponential
Uniform
Impulse (salt-and-pepper)
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
12/53
12
Noise Models - PDFs
Gaussian Noise
where
z: gray scale
= mean (average)
= standard deviation
2 2( ) / 2
21( )
2
z p z e
=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
13/53
13
Noise Models - PDFs
Rayleigh Noise
where
mean =
variance =
22
( )exp( ( ) / for( )
0 for
z a z a b z ap z b
z a
=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
14/53
14
Noise Models - PDFs
Uniform Noise
where
mean =
variance =
1for
( )( )
0 elsewhere
a z bb ap z
=
( ) / 2a b+
2( ) /12b a
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
15/53
15
Noise Models - PDFs
Uniform Noise
where
==
=
otherwise0
(salt)for(pepper)for
)( bzPazP
zp b
a
b a>
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
16/53
16
Example: Noise Models
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
17/53
17
Example: Noise Models
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
18/53
18
Restoration in Presence of Noise Only
Spatial Filtering Consider an image degraded with only additive noise.
The degradation model is further simplified as
in spatial domain, and
in frequency domain
( , ) ( , ) ( , )g x y f x y n x y= +
( , ) ( , ) ( , )G u v F u v N u v= +
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
19/53
19
Noise Removal with Spatial Filters
Spatial filters can effectively remove various types of
noise in digital images Typically operate on small neighborhoods, from 3x3
to 11x11.
Some can be implemented as convolution masks
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
20/53
20
Mean Filters
Arithmetic Mean Filter
Computes the average value of the corrupted image g(x,y)in area defined by Sxy. Sxy represents the set of
coordinates in a rectangular subimage window of size
mxn, centred at point (x,y)
The operation can be implemented using convolutionmask
For random noise
( , )
1( , ) ( , )xys t S
f x y g s t mn
=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
21/53
21
Mean Filters
Geometric Mean Filter
The image restored using a geometric mean filter:
For random noise: lose less detail than Arithmetic Mean
Harmonic Mean Filter
The image restored using a harmonic mean filter:
For salt noise
1
( , )
( , ) ( , )xy
mn
s t S
f x y g s t
=
( , )
( , )1
( , )xys t S
mn f x y
g s t
=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
22/53
22
Mean Filters
Contraharmonic Mean Filter
The image restored with a contraharmonic mean filter:
where Q is called the order of filter
Positive Q: pepper
Negative Q: salt
Q=0: Arithmetic mean
Q=-1: Harmonic mean
1
( , )
( , )
( , )
( , )( , )
xy
xy
Q
s t S
Q
s t S
g s t
f x yg s t
+
=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
23/53
23
Example: Mean Filters
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
24/53
24
Example: Mean Filters (Cont'd)
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
25/53
25
Example: Mean Filters (Cont'd)
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
26/53
26
Order-Statistics Filters
Order Filter based on a specific type of image statistics
called order statistics, sometimes known as Order-Statistics
Filter
Order Statistics: Arranges all the pixels in sequential order
(smallest to largest), based on gray-level value
The selection of the value to be replaced in the center pixel,
is determined by the statistical function used (min, max,
median, etc.)
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
27/53
27
Max & Min Filters
Minimum Filter select the smallest value within an ordered
window of pixel values, denoted as
Works best when the noise is primarily of the salt-type
(high value) Maximum Filter select the largest value within an ordered
window of pixel values, denoted as
Works best for pepper-type noise (low value)
( , )
( , ) min { ( , )}xys t S
f x y g s t
=
( , )
( , ) max { ( , )}xys t S
f x y g s t
=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
28/53
28
Median Filters
Median Filter select the middle pixel value within an
ordered window of pixel values, denoted as
Works best with salt-and-pepper noise (both high and low
values) Better noise remover than Averaging Filter, which causes
blurry edges and details in image, thus not effective against
impulse (salt-and-pepper) noise
Preserve line structures
{ }),(),(),(
tsgmedianyxfxySts
=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
29/53
29
Example: Median Filter
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
30/53
30
Example: Min & Max Filters
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
31/53
31
Midpoint Filter
Midpoint Filter select the average of the maximum and
minimum pixel values within the window, denoted as
Useful for Gaussian and uniform noise
( , )( , )
1( , ) max { ( , )} min { ( , )}2 xyxy s t Ss t S
f x y g s t g s t
= +
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
32/53
32
Alpha-trimmed Mean Filter
Alpha-trimmed Mean Filter select the average of the values
within the window, but with some of the endpoint-ranked
values excluded
Suppose we delete d/2 lowest and d/2 highest gray level
values of g(s,t) in the neighborhood Sxy
. Let gr
(s,t) represent
the remaining mn-d pixels, the remaining pixels are
averaged:
Useful for combination noise such as salt-and-pepper with
Gaussian noise
( , )
1( , ) ( , )XY
r
s t S
f x y g s t
mn d
=
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
33/53
33
Example: Mean Filters
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
34/53
34
Adaptive Filters
Filters that consider behavior changes based on statistical
characteristics of the image inside the filter region.
Capable of superior performance, but causes increase in filter
complexity
Textbook Extra Reading: Adaptive, local noise reduction filter
Adaptive median filter
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
35/53
35
Adaptive Filters
Filters that consider behavior changes based on statistical
characteristics of the image inside the filter region.
Capable of superior performance, but causes increase in filter
complexity
Textbook Extra Reading: Adaptive, local noise reduction filter
Adaptive median filter
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
36/53
36
Periodic Noise
Periodic Noise can be effectively filtered using frequency
domain techniques
Periodic Noise: Concentrated bursts of energy in the Fourier
transform, at locations corresponding to the frequencies of
the periodic interference
Approach: Use selective filters to isolate noise Bandreject,
Bandpass, Notch filters
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
37/53
37
Bandreject Filters
Bandreject Filters Remove noise from a certain location (or band) in
the frequency domain
Image corrupted with additive periodic noise can be easily removed
with a bandreject filter
Bandpass Filter the complement function of the Bandreject filter,
performs the opposite operation.
Useful for isolating noise pattern for analysis
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
38/53
38
Example: Bandreject Filter
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
39/53
39
Notch Filters
Notch Filter Rejects or passes frequencies in predefined
neighborhoods about a center frequency
Due to symmetry of the Fourier Transform, notch filters must
appear in symmetric pairs about the origin in order to obtain
meaningful results Available as Notch Pass and Notch Reject (one a complement
of the other)
Useful for removing periodic noise (horizontal, vertical,
diagonal periodic lines in images) which are concentrated on
one small spot in the frequency spectrum
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
40/53
40
Example: Notch (Reject) Filters
l h l
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
41/53
41
Example: Notch Filter
i d l
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
42/53
42
Restoration Process Model
Degraded image
g(x,y)
Degradation function
h(x, y)
Noise modeln(x, y)
Fourier
Transform
Frequency
Domain
Filter
R(u,v)
G(u,v)
H(u,v)
N(u,v)
Inverse
FourierTransform
Restored Image ( , ) f x y
R i P
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
43/53
43
Restoration Process
Mathematical model:
G(u,v) = H(u,v)F(u,v) + N(u,v)
where:
G(u,v) = Fourier transform of degraded image
H(u,v) = Fourier transform of degradation function
F(u,v) = Fourier transform of original imageN(u,v) = Fourier transform of additive noise function
To obtain the restored image:
where:
= the restored image, an approximation of
= the inverse Fourier transform
R(u,v) = the restoration (frequency domain) filter
( , ) f x y
1 1 ( , ) [ ( , )] [ ( , ) ( , )]f x y F u v F R u v G u v = =
1[]
D d ti F ti ?
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
44/53
44
Degradation Function?
Question: How do we estimate the degradationfunction?
Image Observation
Experimentation
Mathematical Modeling
I Filt i
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
45/53
45
Inverse Filtering
Uses the same model, with assumption of no noise.Fourier transform of degraded image:
Fourier transform of the original image will be:
To find the original image, take the inverse Fourier
transform of F(u,v):
( , ) ( , ) ( , ) 0G u v H u v F u v= +
( , ) 1
( , ) ( , )( , ) ( , )
G u v
F u v G u vH u v H u v= =
1 1
1
( , )( , ) [ ( , )]
( , )
1( , )
( , )
G u vf x y F F u v F
H u v
F G u v
H u v
= =
=
E l I Filt i
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
46/53
46
Example: Inverse Filtering
50 50 25
( , ) 20 20 20
20 35 22
H u v
=
=
221
351
201
201
201
201
251
501
501
),(1
vuH
( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , )( , ) ( , ) ( , ) ( , )
G u v H u v F u v N u v N u vF u v F u v
H u v H u v H u v H u v= = + = +
In erse Filtering Some Problems
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
47/53
47
Inverse Filtering: Some Problems
If any points in H(u,v) are zero division by zero
Solution: Do not take zero-points of H(u,v) into account
In presence of noise:
As the value of H(u,v) becomes very small, the second term
becomes very large, and it overshadows the F(u,v)
Limit the restoration to a specific radius about the origin inthe spectrum the restoration cutoff frequency
( , ) ( , ) ( , ) ( , ) ( , ) ( , ) ( , )( , ) ( , ) ( , ) ( , )
G u v H u v F u v N u v N u vF u v F u v
H u v H u v H u v H u v= = + = +
Example: Inverse Filter
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
48/53
48
Example: Inverse Filter
original image
Image blurred withan 11 x 11 gaussian
convolution mask
Inverse filter, with cutofffrequency = 40,
histogram stretched with 3%
low and high clipping to show
detail
Inverse filter, with cutoff
frequency = 60, histogram
stretched
Wiener Filter
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
49/53
49
Wiener Filter
Also known as minimum mean-square error (MMSE) filter
Attempt to model the error in the restored image through
the use of statistical characteristics of noise
The average error is mathematically minimized, resulting in
the equation for Wiener filter:
where
2*
2 2( , ) ( , )
( , ) ( , )
( , )( , ) 1( , )
( , )( , ) ( , )n nl l
w S u v S u v
S u v S u v
H u vH u vR u v
H u v H u v H u v= =
+ +
*
2
2
( , ) complex conjugate of ( , )
( , ) ( , ) power spectrum of the noise
( , ) ( , ) power spectrum of the original image
n
l
H u v H u v
S u v N u v
S u v F u v
=
= =
= =
Example: Wiener Filter
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
50/53
50
Example: Wiener Filter
Image blurred with
an 11 x 11 gaussian
convolution mask
Image with gaussian noise
variance = 5; mean = 0
Inverse filter, with cutoff
frequency = 80, histogram
stretched with 3 % low and
high clipping to show detail
Wiener filter, with cutoff
frequency = 80, histogram
stretched
Example: Motion Blur + Additive Noise
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
51/53
51
Example: Motion Blur + Additive Noise
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
52/53
Recommended Readings
-
8/14/2019 16404857 Digital Image Processing 07 Image Restoration Noise Removal
53/53
53
Recommended Readings
Digital Image Processing (3rd Edition), Gonzalez &
Woods,
Chapter 4: Image Restoration
5.1 5.4, 5.6 5.10 (Week 7)
Chapter 9: Morphological Image Processing
9.1 9.4 (Week 8)