today: image filters
DESCRIPTION
Today: Image Filters. Smooth/Sharpen Images... Find edges... Find waldo …. Motivation: noise reduction. We can measure noise in multiple images of the same static scene. How could we reduce the noise, i.e., give an estimate of the true intensities?. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/1.jpg)
Grauman
Today: Image Filters
Smooth/Sharpen Images... Find edges... Find waldo…
![Page 2: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/2.jpg)
Grauman
Motivation: noise reduction
• We can measure noise in multiple images of the same static scene.
• How could we reduce the noise, i.e., give an estimate of the true intensities?
![Page 3: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/3.jpg)
Grauman
Motivation: noise reduction
• How could we reduce the noise, i.e., give an estimate of the true intensities?
• What if there’s only one image?
![Page 4: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/4.jpg)
Grauman
First attempt at a solution
• Let’s replace each pixel with an average of all the values in its neighborhood
• Assumptions: – Expect pixels to be like their neighbors– Expect noise processes to be independent from
pixel to pixel
![Page 5: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/5.jpg)
Grauman
First attempt at a solution
• Let’s replace each pixel with an average of all the values in its neighborhood
• Moving average in 1D:
Source: S. Marschner
![Page 6: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/6.jpg)
Grauman
Weighted Moving Average
• Can add weights to our moving average• Weights [1, 1, 1, 1, 1] / 5
Source: S. Marschner
![Page 7: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/7.jpg)
Grauman
Weighted Moving Average
• Non-uniform weights [1, 4, 6, 4, 1] / 16
Source: S. Marschner
![Page 8: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/8.jpg)
Grauman
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. Seitz
![Page 9: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/9.jpg)
Grauman
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. Seitz
![Page 10: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/10.jpg)
Grauman
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. Seitz
![Page 11: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/11.jpg)
Grauman
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. Seitz
![Page 12: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/12.jpg)
Grauman
Moving Average In 2D
0 10 20 30 30
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
Source: S. Seitz
![Page 13: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/13.jpg)
Grauman
Moving Average In 2D
0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 90 0 90 90 90 0 0
0 0 0 90 90 90 90 90 0 0
0 0 0 0 0 0 0 0 0 0
0 0 90 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0
0 10 20 30 30 30 20 10
0 20 40 60 60 60 40 20
0 30 60 90 90 90 60 30
0 30 50 80 80 90 60 30
0 30 50 80 80 90 60 30
0 20 30 50 50 60 40 20
10 20 30 30 30 30 20 10
10 10 10 0 0 0 0 0
Source: S. Seitz
![Page 14: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/14.jpg)
Grauman
Convolution
• Convolution: – Flip the filter in both dimensions (bottom to top, right to left)– Then apply cross-correlation
F
H
![Page 15: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/15.jpg)
Grauman
Median filter
• No new pixel values introduced
• Removes spikes: good for impulse, salt & pepper noise
![Page 16: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/16.jpg)
Grauman
Median filter
Salt and pepper noise Median
filtered
Source: M. Hebert
Plots of a row of the image
![Page 17: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/17.jpg)
Grauman
Smoothing with a Gaussian
for sigma=1:3:10 h = fspecial('gaussian‘, fsize, sigma);out = imfilter(im, h); imshow(out);pause;
end
…
Parameter σ is the “scale” / “width” / “spread” of the Gaussian kernel, and controls the amount of smoothing.
![Page 18: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/18.jpg)
Grauman
Partial derivatives of an image
Which shows changes with respect to x?
-1 1
1 -1
or?
-1 1
xyxf
),(
yyxf
),(
(showing flipped filters)
![Page 19: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/19.jpg)
Grauman
Effects of noiseConsider a single row or column of the image
– Plotting intensity as a function of position gives a signal
Where is the edge?
![Page 20: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/20.jpg)
GraumanWhere is the edge?
Solution: smooth first
Look for peaks in
![Page 21: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/21.jpg)
Grauman
Derivative property of convolution
Differentiation property of convolution.
![Page 22: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/22.jpg)
Grauman
Derivative of Gaussian filters
x-direction y-direction
Source: L. Lazebnik
![Page 23: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/23.jpg)
Grauman
Effect of σ on derivatives
The apparent structures differ depending on Gaussian’s scale parameter.
Larger values: larger scale edges detectedSmaller values: finer features detected
σ = 1 pixel σ = 3 pixels
![Page 24: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/24.jpg)
Grauman
Template matching• Filters as templates:
Note that filters look like the effects they are intended to find --- “matched filters”
• Use normalized cross-correlation score to find a given pattern (template) in the image.– Szeliski Eq. 8.11
• Normalization needed to control for relative brightnesses.
![Page 25: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/25.jpg)
Grauman
Template matching
Scene
Template (mask)
A toy example
![Page 26: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/26.jpg)
Grauman
Template matching
Template
Detected template
![Page 27: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/27.jpg)
Grauman
Template matching
Detected template Correlation map
![Page 28: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/28.jpg)
Grauman
Where’s Waldo?
Scene
Template
![Page 29: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/29.jpg)
Grauman
Where’s Waldo?
Scene
Template
![Page 30: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/30.jpg)
Grauman
Where’s Waldo?
Detected template Correlation map
![Page 31: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/31.jpg)
Grauman
Template matching
Scene
Template
What if the template is not identical to some subimage in the scene?
![Page 32: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/32.jpg)
Grauman
![Page 33: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/33.jpg)
Grauman
So, what scale to choose?It depends what we’re looking for.
Too fine of a scale…can’t see the forest for the trees.Too coarse of a scale…can’t tell the maple grain from the cherry.
![Page 34: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/34.jpg)
Grauman
Predict the filtered outputs
000010000
* = ?000100000
* = ?
111111111
000020000 -* = ?
![Page 35: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/35.jpg)
Grauman
Practice with linear filters
000010000
Original
?
Source: D. Lowe
![Page 36: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/36.jpg)
Grauman
Practice with linear filters
000010000
Original Filtered (no change)
Source: D. Lowe
![Page 37: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/37.jpg)
Grauman
Practice with linear filters
000100000
Original
?
Source: D. Lowe
![Page 38: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/38.jpg)
Grauman
Practice with linear filters
000100000
Original Shifted leftby 1 pixel with correlation
Source: D. Lowe
![Page 39: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/39.jpg)
Grauman
Practice with linear filters
Original
?111111111
Source: D. Lowe
![Page 40: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/40.jpg)
Grauman
Practice with linear filters
Original
111111111
Blur (with abox filter)
Source: D. Lowe
![Page 41: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/41.jpg)
Grauman
Practice with linear filters
Original
111111111
000020000 - ?
Source: D. Lowe
![Page 42: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/42.jpg)
Grauman
Practice with linear filters
Original
111111111
000020000 -
Sharpening filter- Accentuates differences with
local average
Source: D. Lowe
![Page 43: Today: Image Filters](https://reader036.vdocuments.us/reader036/viewer/2022081511/568163dd550346895dd53542/html5/thumbnails/43.jpg)
Grauman
Filtering examples: sharpening