lecture 4: linear filters
TRANSCRIPT
![Page 1: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/1.jpg)
Lecture 4: Linear filters
Tuesday, Sept 11
Many slides by (or adapted from) D. Forsyth, Y. Boykov, L. Davis, W. Freeman, M. Hebert, D. Kreigman, P. Duygulu
![Page 2: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/2.jpg)
Image neighborhoods
• Q: What happens if we reshuffle all pixels within the image?
• A: Its histogram won’t change. Point-wise processing unaffected.
• Filters reflect spatial information
![Page 3: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/3.jpg)
Image filtering
Modify the pixels in an image based on somefunction of a local neighborhood of the pixels
10 5 3
4 5 1
1 1 77
Some function
![Page 4: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/4.jpg)
Linear filtering
• Replace each pixel with a linear combination of its neighbors.
• Convolution kernel: prescription for the linear combination
10 5 3
4 5 1
1 1 77*
0 0 0
0 0.5 0
0 1.0 0.5
=kernel
![Page 5: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/5.jpg)
Why filter images?
• Noise reduction• Image enhancement• Feature extraction
![Page 6: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/6.jpg)
Convolutionsignalkernel
![Page 7: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/7.jpg)
Convolution
Shapiro & Stockman
![Page 8: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/8.jpg)
Convolution example
![Page 9: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/9.jpg)
Convolution example
![Page 10: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/10.jpg)
(in both dims)
![Page 11: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/11.jpg)
Filtering examples
![Page 12: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/12.jpg)
Filtering examples: Identity
![Page 13: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/13.jpg)
Filtering examples
![Page 14: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/14.jpg)
Filtering examples: Blur
![Page 15: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/15.jpg)
Filtering examples: Blur
![Page 16: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/16.jpg)
Filtering examples: Blur
![Page 17: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/17.jpg)
Filtering examples
![Page 18: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/18.jpg)
Filtering examples: Shift
![Page 19: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/19.jpg)
Filtering examples
![Page 20: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/20.jpg)
Filtering examples: sharpening
![Page 21: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/21.jpg)
Filtering examples: sharpening
![Page 22: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/22.jpg)
Properties
• Shift invariant – G(Shift(f(x))=Shift(G(f(x)))
• Linear– G(k f(x))=kG(f(x))– G(f+g) = G(f) + G(g)
![Page 23: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/23.jpg)
Properties
• Associative: (f * g) * h = f * (g * h)
• Differentiation rule:
![Page 24: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/24.jpg)
Filters as templates
• Applying filter = taking a dot-product between image and some vector
• Filtering the image is a set of dot products
• Insight – filters look like the
effects they are intended to find
– filters find effects they look like
![Page 25: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/25.jpg)
The University of
Ontario
Original
Gaussian noise
Salt and pepper noise
Impulse noise
Noise
Common types of noise:• Salt and pepper noise:
random occurrences of black and white pixels
• Impulse noise: random occurrences of white pixels
• Gaussian noise: variations in intensity drawn from a Gaussian normal distribution
Filtering is useful for noise reduction...
![Page 26: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/26.jpg)
Gaussian noise
![Page 27: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/27.jpg)
Effect of sigma on Gaussian noise
![Page 28: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/28.jpg)
Effect of sigma on Gaussian noise
![Page 29: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/29.jpg)
Effect of sigma on Gaussian noise
![Page 30: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/30.jpg)
sigma=1
Effect of sigma on Gaussian noise
![Page 31: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/31.jpg)
sigma=16
Effect of sigma on Gaussian noise
![Page 32: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/32.jpg)
Gaussian noise
• Issues– allows noise values greater than maximum or
less than zero– good model for small standard deviations– independence may not be justified– does not model other sources of “noise”
![Page 33: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/33.jpg)
Smoothing and noise
• Expect pixels to “be like” their neighbors• Expect noise processes to be independent from
pixel to pixel
• Smoothing suppresses noise, for appropriate noise models
• Impact of scale: more pixels involved in the image, more noise suppressed, but also more blurring
![Page 34: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/34.jpg)
Mean filtering0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 00 0 0 90 90 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 90 0 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 0 0 0 0 0 0 00 0 90 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0
![Page 35: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/35.jpg)
0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 00 0 0 90 90 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 90 0 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 0 0 0 0 0 0 00 0 90 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0
Mean filtering
0 10 20 30 30 30 20 100 20 40 60 60 60 40 200 30 60 90 90 90 60 300 30 50 80 80 90 60 300 30 50 80 80 90 60 300 20 30 50 50 60 40 20
10 20 30 30 30 30 20 1010 10 10 0 0 0 0 0
![Page 36: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/36.jpg)
Effect of mean filters
Gaussiannoise
Salt and peppernoise
3x3
5x5
7x7
![Page 37: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/37.jpg)
Mean kernel
• What’s the kernel for a 3x3 mean filter?0 0 0 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 00 0 0 90 90 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 90 0 90 90 90 0 00 0 0 90 90 90 90 90 0 00 0 0 0 0 0 0 0 0 00 0 90 0 0 0 0 0 0 00 0 0 0 0 0 0 0 0 0
],[ yxF
],[ vuH
![Page 38: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/38.jpg)
Smoothing by Averaging
![Page 39: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/39.jpg)
Smoothing with a Gaussian
• Averaging does not model defocused lens well
• Gaussian kernel weights pixels at its center much more strongly than its boundaries
![Page 40: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/40.jpg)
Isotropic Gaussian
Reasonable model of a circularly symmetric blob
![Page 41: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/41.jpg)
Smoothing with a Gaussian
![Page 42: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/42.jpg)
![Page 43: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/43.jpg)
Gaussian filters
• Gaussian function has infinite support, but discrete filters use finite kernels
1 2 12 4 21 2 1
],[ vuH
⋅161
![Page 44: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/44.jpg)
Gaussian filters
![Page 45: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/45.jpg)
More noise -
Wider kernel
Smoothing and noise
![Page 46: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/46.jpg)
Gaussian filters
• Remove “high-frequency” components from the image “low pass” filter
• Convolution with self is another Gaussian– So can smooth with small-width kernel,
repeat, and get same result as larger-width kernel would have
– 2x with σ 1x with √2σ• Separable kernel
![Page 47: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/47.jpg)
Separability
• Isotropic Gaussians factorable into product of two 1D Gaussians
• Useful: can convolve all rows, then all columns
• Linear vs. quadratic time in mask size
![Page 48: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/48.jpg)
Separability
![Page 49: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/49.jpg)
Correlation of filter responses
• Filter responses are correlated over scales similar to scale of filter
• Filtered noise is sometimes useful– looks like some natural textures
![Page 50: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/50.jpg)
![Page 51: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/51.jpg)
![Page 52: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/52.jpg)
![Page 53: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/53.jpg)
Edges and derivatives
• Edges correspond to fast changes
![Page 54: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/54.jpg)
Edges and derivatives
![Page 55: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/55.jpg)
Finite difference filters
0 0 01 0 -10 0 0
],[ vuH
Image derivatives can be approximated with convolution.
![Page 56: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/56.jpg)
Finite differences
• M = [-1 0 1]
![Page 57: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/57.jpg)
Finite difference filters
![Page 58: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/58.jpg)
Finite differences
Which is derivative in the x direction?
![Page 59: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/59.jpg)
Finite differences
Increasing noise ->(zero mean additive Gaussian noise)
Strong response to fast change sensitive to noise
![Page 60: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/60.jpg)
Smoothed derivatives
• Smooth before differentiation: assume that “meaningful” changes won’t be suppressed by smoothing, but noise will
• Two convolutions: smooth, then differentiate?
![Page 61: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/61.jpg)
Smoothed derivatives
![Page 62: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/62.jpg)
Derivative of Gaussian filter
[ ]11 −⊗[ ]0.0030 0.0133 0.0219 0.0133 0.00300.0133 0.0596 0.0983 0.0596 0.01330.0219 0.0983 0.1621 0.0983 0.02190.0133 0.0596 0.0983 0.0596 0.01330.0030 0.0133 0.0219 0.0133 0.0030
)()( hgIhgI ⊗⊗=⊗⊗
![Page 63: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/63.jpg)
Derivative of Gaussian filter
σ = 1 pixel σ = 3 pixels σ = 7 pixels
Derivatives in the x direction
![Page 64: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/64.jpg)
Derivative of Gaussian filter
The apparent structures differ depending on Gaussian’s scale parameter.
σ = 1 pixel σ = 2 pixels
![Page 65: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/65.jpg)
Smoothed derivatives: caveat• Tradeoff between localization and smoothing
![Page 66: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/66.jpg)
![Page 67: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/67.jpg)
Typical mask properties• Derivatives
– Opposite signs used to get high response in regions of high contrast
– Sum to 0 no response in constant regions– High absolute value at points of high contrast
• Smoothing– Values positive – Sum to 1 constant regions same as input– Amount of smoothing proportional to mask size
![Page 68: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/68.jpg)
Median filter
• Non-linear
• No new pixel values
• Removes spikes
![Page 69: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/69.jpg)
Median filter
Salt and pepper noise
Median filtered
![Page 70: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/70.jpg)
Median filter
![Page 71: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/71.jpg)
Median filter
![Page 72: Lecture 4: Linear filters](https://reader036.vdocuments.us/reader036/viewer/2022071602/613d6ab8736caf36b75d14b4/html5/thumbnails/72.jpg)
Next
• More on edges, pyramids, and texture• Pset 1 out tomorrow• Reading: chapters 8 and 9