chapter 3: filtering & enhancing images
DESCRIPTION
Chapter 3: Filtering & Enhancing Images. This chapter is about image processing , since the methods take an input image and create another image as output. Other appropriate terms often used are filtering, enhancement , or conditioning . - PowerPoint PPT PresentationTRANSCRIPT
1
Chapter 3: Filtering & Enhancing ImagesChapter 3: Filtering & Enhancing ImagesChapter 3: Filtering & Enhancing ImagesChapter 3: Filtering & Enhancing Images
This chapter is about image processing, since the methods take an input image and create another image as output. Other appropriate terms often used are filtering, enhancement, or conditioning.
The major motion is that the image contains some signal or structure, which we want to extract, along with uninteresting or unwanted variation, which we want to suppress.
If decisions are made about the image, they are made at the level of a single pixel or its local neighborhood.
2
(1) What Needs Enhancement?
(2) Image Enhancement by Point Operation Contrast StretchingDigital NegativeIntensity Level SlicingHistogram Modeling
(3) Image Enhancement by Spatial (mask) Operation Noise SmoothingMedian FilteringSharping MaskingZooming
ContentsContentsContentsContents
3
Before launching into the methods of this chapter, it is useful to review some of the problems that need them. Two general categories of problems follow.
An image needs improvement Low‑level features must be detected
What Needs Enhancement? What Needs Enhancement?
Example 1: Scratches from original photo of San Juan are removed
ScratchesScratches
4
Example 2: Intensity of photo of Alaskan Pipeline rescaled to show much better detail
Example 3: Image of airplane part has edges enhanced to support automatic recognition and measurement
5
This chapter deals mostly with traditional methods of image enhancement. Before moving on, it is important to define and distinguish the terms.
Image enhancement operators improve the detectability of important image details or objects by man or machine.
Example operations include noise reduction (smoothing), contrast stretching, and sharpening (edge enhancement).
6
Image EnhancementImage Enhancement
• Sharpening of image features ( such as edges or boundaries) to make a graphic display more useful for analysis.
• Do not increase information content but increase dynamic range of chosen feature to be detected more easily.
7
Left - Original sensed fingerprint;
Center - Image enhanced by detection and thinning of ridges;
Right - Identification of special features called minutia, which can be used for matching to millions of fingerprint representations in a database.
Example
8
Image Enhancement - OperationsImage Enhancement - Operations
(1) Point operations– Contrast stretching– Noise clipping– Window slicing– Histogram modeling
9
Image Enhancement -OperationsImage Enhancement -Operations
• (2) Spatial (mask) operation– Noise smoothing– Median filtering– Sharping masking– Zooming
10
Two ways to enhance images:
Changing the intensity values of pixels Transforming the pixels via a single function that maps an input gray value into a new output value
Remapping the gray values is often called stretching 伸缩
Points OperationPoints Operation
f(x)
Mapping functionInput image Output image
gin gout=f(gin)
11
Definition - A point operator applied to an image is an operator in which the output pixel is determined only by the input pixel,
Out[x, y] = f (In[x, y]);
possibly function f depends upon some global parameters.
• A contrast stretching 对比度伸缩 operator is a point operator that uses a piecewise smooth function f (In[x, y]) of the input gray level to enhance important details of the image.
12
Contrast StretchingContrast Stretching
• Improve contrast due to poor /nonuniform lighting conditions or nonlinearity or small dynamic range of image sensor.
• Typical contrast stretching transformation:
13
Contrast Stretching
Lub
bua
au
vbu
vau
u
v
b
a
0
Where u is the gray-level values of input image 0-255, v is the transformed gray-level values of output image 0-255, L=255
14
Output image
Mapping function
f(x)
gin
gout
0
255
255
Input image
Output image
15
Contrast Stretching
(a) Original (b) Enhanced
16
Contrast Stretching
(b) Enhanced(a) Original
17
Contrast Stretching
• Example:
(a) Original (b) Enhanced
18
Contrast Stretching
• Clipping: when – Useful for noise reduction when input signal
is known to lie in the range [a,b].
• Thresholding: when a=b=t (threshold)– Output becomes binary.– Useful for binary or other images that have
bimodal distance of gray levels.
• Can define other similar operations.
0
19
Contrast Stretching
Cliping and thresholding
20
Contrast Stretching
Cliping and thresholding
21
Digital Negative
• Create digital negative of images• Display of medical images
uLv
22
Output image
Input image
Mapping function (Reverse)
f(x)=x -1
gin
gout
0
255
255
23
Digital Negative
Digital negatives
24
Digital Negative
Digital negatives
25
Digital Negative
Digital negatives
26
Intensity Level Slicing
Without background:
With background:
otherwise
buaLv
0
otherwiseu
buaLv
27
Intensity Level Slicing
Fully illuminates pixels lying in the interval [a,b] and removes the background.
• Segmentation of certain gray level region
• e.g. Image from remote sensing.
28
Intensity Level Slicing
Level slicing of intensity window[175,250]
(a) Visual and infrared images
(b) Segment images
29
Various Mapping FunctionsVarious Mapping Functions
gin
gout
gin
gout
gin
gout
gin
gout
gin
gout
gin
gout
30
Histogram ModificationHistogram Modification
The histogram of the “bad” image is very narrow (Fig. a), while the histogram of the “good” image is more spread (Fig. b).
To change a “bad” image to the “good” image, we need modify the histogram.
Fig.a - bad image
Gray level r
Number of pixels
Gray level s
Number of pixels
Fig.b - good image
When is it necessary to modify the Histogram of an image
31
Histogram Modification Histogram Modification
The histograms of four basic types of images :
Gray level z
H(z)
Dark imageThe gray levels are concentrated toward the dark end of the grayscale range. Thus this histogram corresponds to an image with overall dark characteristics.
Dark image
32
Histogram Modification Histogram Modification
Just is the opposite of the above.
Gray level z
H(z)
Bright image
Bright image
33
Gray level z
H(z)
Low-contrast image
The histogram has a narrow shape, which indicates little dynamic range and thus corresponds to an image having low contrast.
The histogram has significant spread, corresponding to an image with high contrast.
Gray level z
H(z)High-contrast image
Low-contrast image
High-contrast image
34
Suppose that the grey levels in the original image are given by the values, which are obtained from a variable r.
The grey levels in the new image are given by the values , which are obtained from a variable s.
We would like to find a transformation s = T (r) such that the probability density function pr(r) in original image (Fig. a) is transformed into a probability density function ps(s) in the new image, which looks like that in Fig. b.
How can we modify the Histogram
Gray level r
pr(r)
Gray level s
ps(s)
s=T(r)
35
Since pr(r) is the probability density function of random variable r.
The number of pixels with grey level values in the range r to r +dr is Pr(r)dr.
The transformation we seek will transform this range to [s, s+ds]: [r, r+dr] [s, s+ds]
Gray level r
Number of pixels
r+drr[r, r+dr]
Gray level s
Number of pixels
s[s, s+ds]
s+ds
T
36
The total number of pixels in this range will remain the same but in the enhanced image this number will be ps(s)ds:
ps(s)ds = pr(r)dr
This equation can be used to define the transformation T that must be applied to variable r to obtain variable s, provided we define function ps(s).
37
Histogram equalization is an important method in histogram modification.
Histogram equalization is often tried to enhance an image.
The two requirements on the operator are that
(a) the output image should use all available gray levels
This requirement means that the target output image uses all gray values z = z1, z = z2, … z = zn
(b) the output image has approximately the same number of pixels of each gray level.
Histogram EqualizationHistogram Equalization
38
Ps(s) is uniform, i.e. a flat histogram to obtain an image with a more balanced contrast
i.e. the transformation required is just the cumulative distribution function (c.d.f.) of u
drrPusduuPdssPu
uus 0
)()()(1)(
duuPdssP us )()(
39
• Discrete case
i.e. use the histogram in place of the p.d.f. in the continuous case (the discrete equivalent to the p.d.f.)
similarly, the continuous c.d.f. is now replaced by
⇒ a monotonically increasing sequence
pixels ofnumber total
levelgray with pixels ofnumber )( kk
ku
u
n
nuP
))(..( 0
u
u
k
j
jk uduPfc
n
nS
11210 Lssss
40
if the original digital image has L gray levels, it may be necessary to re-quantize sk to the same set of L discrete levels also
Method1:
Method 2:
1 if ,1
/)1(int
ofindex 0
0*
k
k
k
sLLs
sss
*index of index of the nearest reconstruction level}ks
41
• method 1 is more efficient but it pre-supposes that uniform quantization is appropriate
Example 1 (for 3 bit image)
42
Note:
(1) re-quantization method is arbitrary
(2) may not get a 1-1 mapping inverse problem⇒(3) may not get a flat histogram unless L is large
43
Example of Histogram EqualizationExample of Histogram Equalization
Histogram after equalization
Original image Original histogram
Modified image
44
Original image Original histogram
Modified image Histogram after equalization
45
Histogram Equalization
(a) Input image (b) Processed image
46
Histogram Equalization
(c) Input image (d) Processed image
47
Histogram Equalization
(e) Input image (f) Processed image
48
Spatial (mask) OperationsSpatial (mask) Operations
• Operations performed on local neighborhood of input pixels.
• Usually image is convolved withconvolved with a finite impulse response (FIR) filter-spatial maskmask.
49
Mask operation:Mask operation:
1. Centre the mask at (x, y).
2. Multiply every pixel that is contained within the mask area by the corresponding mask coefficient.
3. Add the products together.
4. Either replace f (x, y) by the sum or compare the sum with a threshold and replace f (x, y) according to the comparison.
5. Repeat 1 to 4 for the whole image.
50
51
52
Image Smoothing Image Smoothing
Often, an image is composed of some underlying ideal structure, which we want to detect and describe, together with some random noise or artifact, which we would like to remove.
Image contains both Gaussian noise and bright ring artifacts
Image with random noise
ScratchesScratches
Image contains artifacts
53
Box FilterDefinition : Smoothing an image by equally weighting a rectangular neighborhood of pixels is called using a box filter.
Output-Image [r,c] = Average of some neighborhood of Input-Image [r,c]
Example: 55 Neighborhood Filter - averages 25 pixel values in a 55 neighborhood of the input image pixel in order to create a smoothed output image.
25],[],[2
2
2
2
i j
jcirIncrOut
54
ExampleExample
80
9
12
30
80
8
13
3
18
0
80
30
8
20
80
30
40
3
40
40
50
5
0
20
40
00 + 03 + 08 + 12 + 03 + 05 + 40 + 30 + 09 + 13 + 40 + 40 + 80 + 80 + 00 + 50 + 30 + 80 + 80 + 00 + 20 + 40 + 20 + 30 + 18
25 = 30
80
9
12
30
80
8
13
3
18
0
80
30
8
20
80
30
40
3
40
40
50
5
0
20
40 30
55
Neighborhood Averaging
• Original Image • Noisy Image• NAF (3-by-3)• NAF (5-by-5)• NAF (7-by-7)
56
Neighborhood Averaging
• Use for noise smoothing LP filtering and subsampling of images.
• Assuming white noise ηwith zero mean and variance
• Then the spatial average:
2
nmnmunmy ,,,
57
assuming equal weight where is the spatial average of . Note that has zero mean and
i.e. Noise power is reduced by a factor of
•Remark: Neighborhood averaging introduces a distortion in the form of blurring.
,
1, , ,
W k l W
v m n u m k n l m nN
nm,
nm,
WN/22
WN
nm,
58
Neighborhood Averaging
(a) Original (b) noisy
59
Neighborhood Averaging
(c) 3×3 filter (d) 5×5 filter
Spatial averaging filters for smoothing images containing Gaussian noise.
60
Definition : When a Gaussian filter is used, pixel [x,y] is weighted according to
2
2
2
21
),(
d
eyxg
22 )()( cc yyxxd
x
Gaussian Filter
d is the distance of the neighborhood pixel [x, y] from the center pixel [xc, yc] of the output image where the filter is being applied. [xc]
g(x)
[x]
d
[x]
Rather than weight all input pixels equally, it is better to reduce the weight of the input pixels with increasing distance from the center pixel I[xc, yc]. The Gaussian filter does this and is perhaps the most commonly used of all filters.
61
[xc]x
g(x)Gaussian Function
One-Dimensional Gaussian Function
Two-Dimensional Gaussian Function
62
Example
80
9
12
30
80
8
13
3
18
0
80
30
8
20
80
30
40
3
40
40
50
5
0
20
40
001 + 031 + 081 + 121 + 031 + 051 + 402 + 302 + 092 + 131 + 401 + 402 + 803 + 802 + 001 + 501 + 302 + 802 + 802 + 001 + 201 + 401 + 201 + 301 + 181
25 = 52
80
9
12
30
80
8
13
3
18
0
80
30
8
20
80
30
40
3
40
40
50
5
0
20
40 52
63
Examples Examples
Noisy image
Ideal image
Pixel values in row 100 of the noisy image
Pixel values in row 100 of the smoothed image
Noise averaged using a 55 neighborhood
64
Examples Examples
Noisy image
M = 32M = 16
M = 8M = 2
M = 128
65
Often, it is useful to remove small regions from an image.
A small region
might just be the result of noise, or
it might represent low level detail that should be suppressed from the image description being constructed.
Small regions can be removed
by changing single pixels, or
by removing components after connected components are extracted.
Removal of Small Image Regions Removal of Small Image Regions
66
The presence of single dark pixels in bright regions, or single bright pixels in dark regions, is called salt‑and‑pepper noise.
Often, salt‑and‑pepper noise is the natural result of creating a binary image via thresholding.
Salt corresponds to pixels in a dark region that somehow passed the threshold for bright.
Pepper corresponds to pixels in a bright region that were below threshold for dark.
Removal of Salt-and-Pepper Noise (Binary Image) Removal of Salt-and-Pepper Noise (Binary Image)
Gray level z
H(z) Salt
Pepper
67
1
11
0
1
1
1
1 1
0
00
1
0
0
0
0 0
Salt noise in 8‑neighbors
Pepper noise in 8‑neighbors
68
Binary image of bacteria
Salt‑and‑pepper noise removed using 8‑neighbors
Pepper noise removed using 8‑neighbors
Salt noise removed using 8‑neighbors
1
11
0
1
1
1
1 1
1
11
1
1
1
1
1 1
0
00
1
0
0
0
0 0
0
00
0
0
0
0
0 0
Pepper noise
Salt noise
Removal of Salt-and-Pepper Noise by Mask Removal of Salt-and-Pepper Noise by Mask
69
Binary image of bacteria
Salt‑and‑pepper noise removed using 4‑neighbors
Pepper noise removed using 4‑neighbors
Salt noise removed using 4‑neighbors
1
1
0
1
1 1
1
1
1
1
0
0
1
0
0 0
0
0
0
0
Pepper noise
Salt noise
Removal of Salt-and-Pepper Noise by Mask Removal of Salt-and-Pepper Noise by Mask
70
Binary image of bacteria
Salt‑and‑pepper noise removed using 8‑neighbors
Salt‑and‑pepper noise removed using 4‑neighbors
Better !Better !
71
Median Filtering ( Gray-level Image)Median Filtering ( Gray-level Image)
Definition --- Let A[i] i = 0, 1, …(n-1) be a sorted array of n real
numbers. The median of the set of numbers in A is A[(n ‑ 1)/2]
For odd n, the array has a unique middle as defined above.
If n is even, then we can define that there are two medians, at A[n/2] and A[n/2 ‑ 1], or one median, which is the average of these two values, namely:
A[n/2] = Median value g1
A[n/2 ‑ 1] = Median value g2
Median value (g1 + g2)/2
In Median Filter, a pixel value is replaced with the median value of the neighborhood.
72
ExampleExample
80
9
12
30
80
8
13
3
18
0
80
30
8
20
80
30
40
3
40
40
50
5
0
20
40
80
9
12
30
80
8
13
3
18
0
80
30
8
20
80
30
40
3
40
40
50
5
0
20
40 13
A[(n-1)/2] = A[(13-1)/2] = A[6] = 13
A[i] = 0, 3, 5, 8, 9, 12, 13, 18, 20, 30, 40, 50, 80
i = 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12
73
Examples Examples
Noisy image
Ideal image
Pixel values in row 100 of the noisy image
Median filtering using a 55 neighborhood
Pixel values in row 100 of the smoothed image
74
Examples Examples
Input image contains both Gaussian noise and bright ring artifacts added to four previously uniform regions
Result of applying a 7 7 median filter
75
(b) 3×3 median filtered
(a) Gray-level Image with salt & pepper noise
76
(c) Gray-level Image with Gaussian noise
(d) 3×3 median filtered
77
(a) Original Gray-level Image (b) With salt & pepper noise
78
(c) Five nearest neighbors spatial average
(d) 3×3 median filtered
Spatial averaging versus median filtering
79
Properties of Median Filter
• However, performance poor when noise pixel in a window>1/2 no. of pixels in the windows.
80
Computing the median requires more computation time than computing a neighborhood average, since the neighborhood values must be partially sorted.
Moreover, median filtering is not so easily implemented in special hardware that might be necessary for real‑time processing, such as in a video pipeline.
However, in many image analysis tasks, its value in image enhancement is worth the time spent.
81
Image SharpeningImage Sharpening
• Crisping the edges
• A signal proportional to the unsharp, or LPF (low pass frequency) version of the image is subtracted from the image.
[or adding the gradient, or HPF (high pass frequency) signal to the image]
82
• Idea is to boost the HF (high frequency) components (edges) while the LF remain intact.
• High-Frequency Filtering– To correct blurs– Emphasis depends on frequency, chosen such
as to just compensate the blurring.– Frequency with strong noise should not be
emphasized– May require transform operations.
83
Image SharpeningImage Sharpening
)1,(),1()1,(,14
1,, nmunmunmunmunmunmg
• LPF: spatial averaging operation• HPF:
by subtracting the LPF image from the original image.
, ( , ) ( , )HP LPh m n u m n h m n
84
Image Sharpening
Sharping mask operation
85
Image Sharpening
(a) Original (b) enhanced
Figure 7.24
86
(a) Original image (b) enhanced
87
(a) Original (b) enhanced
88
Magnification of Image (Zoom)
(a) Replication• Zero Order Hold: each pixel along a scan line id
repeated once, and each scan line is repeated.• For an image, interlace it by rows and
columns of zero to get a matrix, then convolve with
NM NM 22
11
11H
89
Magnification of Image (Zoom)
Zooming by replication from 128×128 to 256 ×256 and 512× 512 images
90
Zooming by replication from 128×128 to 256 ×256 and 512× 512 images
91
• (b) Linear Interpolation– First Order Hold-straight line fitted between pixels
along row, then column.
– Can also be obtained by interlacing the image by rows and columns of zero, and then convolve with
4
1
2
1
4
12
11
2
14
1
2
1
4
1
H
92
Zooming by linear interpolation from 128 ×128 to 256 × 256 and 512 × 512
images
93
Zooming by linear interpolation from 128 ×128 to 256 × 256 and 512 × 512 images
94
Magnification of Image (Zoom)
• Higher-order interpolation: (order p):– Padding each row and column by p rows and p
columns of zeros.– Convolve it p times with H.– e.g. p=3-cubic spline interpolation between
pixels.
95
Magnification of Image (Zoom)