syde 575: introduction to image processing image enhancement by smoothing textbook sections 3.4 –...
TRANSCRIPT
![Page 1: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/1.jpg)
SYDE 575: Introduction to Image Processing
Image Enhancement by Smoothing
Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2
![Page 2: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/2.jpg)
Noise Reduction by Image Averaging
We want to be able to reduce the amount of noise in an image
Assume that the noise process has a mean of zero
One approach is to take many images f(x,y) of the same scene and use these to generate a noise-free image g(x,y)
![Page 3: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/3.jpg)
Noise Reduction by Image Averaging
Assume noise is additive and Gaussian distributed with zero mean
~ (0, )n N s Suppose we take the average of q number
of noise samples at a point in the image
1 q
kk i
m nq =
= å
1 2, , ...., qn n n
![Page 4: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/4.jpg)
Noise Reduction by Image Averaging
Now, apply concept to each pixel in the whole image
Given an infinite number of noise samples, the average approaches the mean of the distribution, which in this case is 0
1 1
1 1( , ) ( , ) ( , )
q q
kk k
g x y f x y n x yq q= =
= +å å
1
1A s , ( , ) 0 , ( , ) ( , )
q
kk
q n x y g x y f x yq =
® ¥ ® ®å
![Page 5: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/5.jpg)
Example
Source: Gonzalez and Woods
![Page 6: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/6.jpg)
Noise Reduction by Spatial Filtering
Image averaging takes advantage of information redundancy from the individual images to reduce noise
Not always possible to acquire so many images!
Images may not be perfectly registered so errors due to spatial misalignment
Alternative option: Take advantage of information redundancy from different pixels within the same image to reduce noise
![Page 7: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/7.jpg)
Spatial Filtering - Convolution
2D discrete convolution
output input point spread function (2d) or impulse response (1d)
[ , ] [ , ] [ , ] [ , ] [ , ]j i
g m n f m n h m n f i j h m i n j¥ ¥
=- ¥ =- ¥
= * = × - -å å
Impulse response h[m,n] can be viewed as a spatial filter for an input image f[m,n] to produce output image g[m,n]
![Page 8: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/8.jpg)
Spatial Filtering
Source: Gonzalez and Woods
Let us represent w(x,y) as a 2D convolution mask
![Page 9: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/9.jpg)
( 1) / 2 ( 1) / 2
( 1) / 2 ( 1) / 2
( , ) ( , ) ( , )m n
s m t n
g x y w s t f x s y t- -
=- - =- -
= + +å å
Spatial Filtering
Spatial filtering of an image f with 2D convolution using symmetrical mask w of size m x n can be expressed as
output inputconvolution mask; the “system”
![Page 10: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/10.jpg)
Spatial Filtering
Source: Gonzalez and Woods
![Page 11: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/11.jpg)
Local Average Filter for Smoothing
Instead of averaging between images, we can average neighboring pixels using the following point spread function (PSF)
Recall similar 1-d mask [1/3 1/3 1/3] Why is the 1/9 required?
![Page 12: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/12.jpg)
Averaging Filter
Source: Gonzalez and Woods
Removes noise but also blurs
![Page 13: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/13.jpg)
Properties of Averaging Filter
• Linear?• Shift Invariant?• Memory?• Causal?• Stable?• Invertible?
![Page 14: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/14.jpg)
Weighted Average Filter
Problem: Simple averaging of neighboring pixels lead to over-smoothing
Possible solution: Instead of weighting all neighboring pixels equally, assign higher weights to pixels that are closer to the input pixel
![Page 15: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/15.jpg)
Weighted Averaging Filter: Example
Source: Gonzalez and Woods
![Page 16: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/16.jpg)
Weighted Averaging Filter: Example
Noisy Average WeightedAverage
![Page 17: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/17.jpg)
Gaussian Smoothing
• A weighted average can be produced using a Gaussian as a weight
• In 1d
• In 2d
![Page 18: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/18.jpg)
1D Gaussian
Source: Wikipedia
![Page 19: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/19.jpg)
2d Gaussian
Source: Wikipedia
![Page 20: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/20.jpg)
Characteristics of Local Averaging
a) h(m,n) >= 0 for all (m,n)
b) S h(m,n) = 1 (DC gain is 1)
c) Non-causal
d) Typically odd-dimensions (practical)
e) Even symmetry
![Page 21: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/21.jpg)
Order-Statistic Filters
Nonlinear spatial filters, i.e.,
Best known example: median filter
a1f1 (m,n) + a2f2 (m,n) ≠ a1g1 (m,n) + a2g2 (m,n)
![Page 22: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/22.jpg)
Median Filter
Provides good noise reduction for certain types of noise such as impulse noise
Considerably less blurring than weighted averaging filter
Forces a pixel to be like its neighbors
Steps Order pixels within an area Replace value of center pixel with median
value (half of all pixels have intensities greater than or equal to the median value)
![Page 23: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/23.jpg)
Median Filter: Example
255 10 9
10 255 10
8 10 10
255109 10 10108 10 255
255 10 9
10 10
8 10 10
10
median=10
![Page 24: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/24.jpg)
Median Filter: Example
Source: Gonzalez and Woods
![Page 25: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/25.jpg)
S-statistic Filter
• Another method to avoid impact of outliers on smoothing is the S-statistic filter
• Basically, only use those values within a region that are within a certain range relative to the mean
• Typically, this is done by calculating the m and s of the local region and determining a local average based only on pixels within a certain number of s
![Page 26: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/26.jpg)
Recap: Spatial smoothing filters
All of the filters discussed so far are spatial smoothing filters
Weight of each pixel in the neighborhood covered by the filter depends on the proximity of the pixel to the center pixel being filtered Typically, the closer the pixel is to the
center pixel, the higher the weight
![Page 27: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/27.jpg)
Range Smoothing Filters
Problem: oversmoothing of edges and other fine image detail using ideal and Gaussian filters
Alternative solution: range smoothing filters Weight of each pixel in the neighborhood covered by the
filter depends on the similarity of the pixel's intensity value to that of the center pixel being filtered The closer the pixel's intensity value is to that of the
center pixel, the higher the weight
![Page 28: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/28.jpg)
Bilateral Filtering
Problem: range smoothing filter simply remaps intensity values No notion of space Poor noise reduction performance (in fact, not
really useful by itself) Idea: Combine spatial smoothing filters with range
smoothing filters! Good noise reduction (Spatial smoothing) Good edge and detail preservation (Range
smoothing)
![Page 29: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/29.jpg)
Bilateral Filtering
Resulting filter is non-linear
( ( , ) ( , ) ) ( , )bf c c c ch h f x y f x y h x x y y= - - -
Range smoothing filter Spatial smoothing filter
![Page 30: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/30.jpg)
Example
Suppose we use Gaussian models for the range and spatial smoothing filters
Source: Tomasi et al. 1998
![Page 31: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/31.jpg)
Uses: Noise Suppression
noisy Gaussian spatial filter Bilateral filter
![Page 32: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/32.jpg)
Uses: Special Effects
Bilateral filter reduces small image details while preserving large edge details
What would happen if we apply bilateral filtering multiple times? More and more smaller details get smoothed out Large edges remain well-preserved
Result: cartoon-like image
![Page 33: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/33.jpg)
Uses: Special Effects
original 5 iterations
![Page 34: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/34.jpg)
Implementing Symmetrical 2-D Filters
• Large images in use today (e.g., 1k x 1k, 5k x 5k, and even larger) require increased computational requirements
• Symmetrical filters are commonly used in image processing systems
• There is a faster way to implement a 2-d symmetrical mask using 2 1-d arrays
![Page 35: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/35.jpg)
Example: 2d as 2 x 1d
1 1 1
1 1 1
1 1 1
* 1/9
1 2 3 4
1 1 1 1
1 2 2 3
1 1 1 1
“Image” Smoothing Mask – Note Symmetry
![Page 36: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/36.jpg)
Representing 2d Mask as 2 x 1d
1 1 1
1 1 1
1 1 1
1 1 1
1/9 * 1/3
1
1
1
= 1/3
![Page 37: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/37.jpg)
*
Top row and bottom row are equivalent
1 2 3 4
1 1 1 1
1 2 2 3
1 1 1 1
1 2 3 4
1 1 1 1
1 2 2 3
1 1 1 1
*
1 1 1
1 1 1
1 1 1
1 1 1 *1/3
1
1
1
1/3
- - - -
- 14 19 -
- 11 15 -
- - - -
= same
= 1/91/9
![Page 38: SYDE 575: Introduction to Image Processing Image Enhancement by Smoothing Textbook Sections 3.4 – 3.5, 5.3.1-5.3.2](https://reader036.vdocuments.us/reader036/viewer/2022062516/56649dce5503460f94ac290b/html5/thumbnails/38.jpg)
Computational Complexity
• Assume M by N image and m by n filter mask• For 2d mask, number of multiplications is:
M x N x m x n
• For 2 1d masks, number of multiplication is
M x N x (m + n)
• So, for a 15 by 15 mask, 2d implementation is 225MN and 1d implementation is 30MN
• Huge Savings! But note that 1d implementation requires storing an additional image in memory