image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• frequency domain. • fourier...
TRANSCRIPT
![Page 1: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/1.jpg)
Image pyramids and frequency domain
16-385 Computer VisionSpring 2019, Lecture 3http://www.cs.cmu.edu/~16385/
![Page 2: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/2.jpg)
Course announcements
• Homework 1 will be posted tonight.- Homework 1 is due on February 6th at midnight.
• Office hours for the rest of the semester:Monday, 5-7 pm, NSH 1109, SharvaniTuesday, 5-7 pm, NSH 4119, AnshumanWednesday 4-6 pm, NSH 4119, NeerajThursday 4-6 pm, NSH 4119, AbhayFriday 3-5 pm, Smith Hall graphics lounge, YannisFriday 5-7 pm, Smith Hall graphics lounge, Bruce
![Page 3: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/3.jpg)
Overview of today’s lecture• Image downsampling.
• Aliasing.
• Gaussian image pyramid.
• Laplacian image pyramid.
• Fourier series.
• Frequency domain.
• Fourier transform.
• Frequency-domain filtering.
• Revisiting sampling.
![Page 4: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/4.jpg)
Slide credits
Most of these slides were adapted directly from:
• Kris Kitani (15-463, Fall 2016).
Some slides were inspired or taken from:
• Fredo Durand (MIT).• Bernd Girod (Stanford University).• James Hays (Georgia Tech).• Steve Marschner (Cornell University).• Steve Seitz (University of Washington).
![Page 5: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/5.jpg)
Image downsampling
![Page 6: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/6.jpg)
This image is too big to fit on the screen. How would you reduce it to half its size?
![Page 7: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/7.jpg)
delete even rowsdelete even columns
1/2
1/4
1/8
Throw away half the rows and columns
What is the problem with this approach?
Naïve image downsampling
delete even rowsdelete even columns
![Page 8: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/8.jpg)
1/2 1/4 (2x zoom) 1/8 (4x zoom)
Naïve image downsampling
What is the 1/8 image so pixelated (and do you know what this effect is called)?
![Page 9: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/9.jpg)
Aliasing
![Page 10: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/10.jpg)
Reminder
Images are a discrete, or sampled, representation of a continuous world
![Page 11: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/11.jpg)
Sampling
Very simple example: a sine wave
How would you discretize this signal?
![Page 12: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/12.jpg)
Sampling
Very simple example: a sine wave
![Page 13: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/13.jpg)
Sampling
Very simple example: a sine wave
Can I take as many samples as I want?
How many samples should I take?
![Page 14: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/14.jpg)
Sampling
Very simple example: a sine wave
Can I take as few samples as I want?
How many samples should I take?
![Page 15: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/15.jpg)
Undersampling
Very simple example: a sine wave
Unsurprising effect: information is lost.
![Page 16: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/16.jpg)
Undersampling
Very simple example: a sine wave
Unsurprising effect: information is lost.
Surprising effect: can confuse the signal with one of lower frequency.
![Page 17: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/17.jpg)
Undersampling
Very simple example: a sine wave
Unsurprising effect: information is lost.
Surprising effect: can confuse the signal with one of lower frequency.
Note: we could always confuse the signal with one of higher frequency.
![Page 18: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/18.jpg)
Aliasing
Fancy term for: Undersampling can disguise a signal as one of a lower frequency
Unsurprising effect: information is lost.
Surprising effect: can confuse the signal with one of lower frequency.
Note: we could always confuse the signal with one of higher frequency.
![Page 19: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/19.jpg)
Aliasing in textures
![Page 20: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/20.jpg)
Aliasing in photographs
This is also known as “moire”
![Page 21: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/21.jpg)
Temporal aliasing
![Page 22: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/22.jpg)
Temporal aliasing
![Page 23: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/23.jpg)
Temporal aliasing
![Page 24: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/24.jpg)
Anti-aliasing
How would you deal with aliasing?
![Page 25: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/25.jpg)
Anti-aliasing
How would you deal with aliasing?
Approach 1: Oversample the signal
![Page 26: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/26.jpg)
Anti-aliasing in textures
aliasing artifacts anti-aliasing by oversampling
![Page 27: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/27.jpg)
Anti-aliasing
How would you deal with aliasing?
Approach 1: Oversample the signal
Approach 2: Smooth the signal• Remove some of the detail effects that cause aliasing.• Lose information, but better than aliasing artifacts.
How would you smooth a signal?
![Page 28: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/28.jpg)
Gaussian filterdelete even rows
delete even columns
1/2
1/4
1/8
Apply a smoothing filter first, then throw away half therows and columns
Better image downsampling
Gaussian filterdelete even rows
delete even columns
![Page 29: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/29.jpg)
1/2 1/4 (2x zoom) 1/8 (4x zoom)
Better image downsampling
![Page 30: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/30.jpg)
1/2 1/4 (2x zoom) 1/8 (4x zoom)
Naïve image downsampling
![Page 31: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/31.jpg)
Anti-aliasing
Question 1: How much smoothing do I need to do to avoid aliasing?
Question 2: How many samples do I need to take to avoid aliasing?
We’ll see what this means soon.
Answer to both: Enough to reach the Nyquist limit.
![Page 32: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/32.jpg)
Gaussian image pyramid
![Page 33: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/33.jpg)
The name of this sequence of subsampled images
Gaussian image pyramid
![Page 34: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/34.jpg)
filter
sample
filter
sample
Constructing a Gaussian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
sample
Question: How much bigger than the original image is the whole pyramid?
![Page 35: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/35.jpg)
filter
sample
filter
sample
Constructing a Gaussian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
sample
Question: How much bigger than the original image is the whole pyramid?
Answer: Just 4/3 times the size of the original image! (How did I come up with this number?)
![Page 36: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/36.jpg)
What happens to the details of the image?
Some properties of the Gaussian pyramid
![Page 37: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/37.jpg)
What happens to the details of the image?• They get smoothed out as we move to
higher levels.
What is preserved at the higher levels?
Some properties of the Gaussian pyramid
![Page 38: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/38.jpg)
What happens to the details of the image?• They get smoothed out as we move to
higher levels.
What is preserved at the higher levels?• Mostly large uniform regions in the
original image.
How would you reconstruct the original image from the image at the upper level?
Some properties of the Gaussian pyramid
![Page 39: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/39.jpg)
What happens to the details of the image?• They get smoothed out as we move to
higher levels.
What is preserved at the higher levels?• Mostly large uniform regions in the
original image.
How would you reconstruct the original image from the image at the upper level?• That’s not possible.
Some properties of the Gaussian pyramid
![Page 40: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/40.jpg)
Blurring is lossy
- =
level 0 residual
What does the residual look like?
level 1 (before downsampling)
![Page 41: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/41.jpg)
Blurring is lossy
- =
level 0 level 1 (before downsampling) residual
Can we make a pyramid that is lossless?
![Page 42: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/42.jpg)
Laplacian image pyramid
![Page 43: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/43.jpg)
At each level, retain the residuals instead of the blurred images themselves.
Can we reconstruct the original image using the pyramid?
Laplacian image pyramid
![Page 44: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/44.jpg)
At each level, retain the residuals instead of the blurred images themselves.
Can we reconstruct the original image using the pyramid?• Yes we can!
What do we need to store to be able to reconstruct the original image?
Laplacian image pyramid
![Page 45: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/45.jpg)
Let’s start by looking at just one level
= +
level 0 residual
Does this mean we need to store both residuals and the blurred copies of the original?
level 1 (upsampled)
![Page 46: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/46.jpg)
Constructing a Laplacian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
compute residual
![Page 47: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/47.jpg)
Constructing a Laplacian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
compute residual
What is this part?
![Page 48: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/48.jpg)
Constructing a Laplacian pyramid
repeat:
filter
subsample
until min resolution reached
Algorithm
compute residual
It’s a Gaussian pyramid.
![Page 49: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/49.jpg)
What do we need to construct the original image?
![Page 50: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/50.jpg)
(1) residuals
What do we need to construct the original image?
![Page 51: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/51.jpg)
(2) smallest image
(1) residuals
What do we need to construct the original image?
![Page 52: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/52.jpg)
repeat:
upsample
sum with residual
until orig resolution reached
Algorithm
Reconstructing the original image
![Page 53: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/53.jpg)
Gaussian vs Laplacian Pyramid
Shown in opposite order for space.
Which one takes more space to store?
![Page 54: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/54.jpg)
Why is it called a Laplacian pyramid?
![Page 55: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/55.jpg)
As with derivative, we can combine Laplace filtering with Gaussian filtering
Reminder: Laplacian of Gaussian (LoG) filter
Laplacian of Gaussian
output
input
“zero crossings” at edges
![Page 56: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/56.jpg)
- =
-
unit Gaussian Laplacian
Difference of Gaussians approximates the Laplacian
Why is it called a Laplacian pyramid?
![Page 57: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/57.jpg)
Why Reagan?
![Page 58: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/58.jpg)
Why Reagan?
The Laplacian Pyramid as a Compact Image Code (1983)
Peter J. Burt , Edward H. Adelson
Ronald Reagan was President when the Laplacian pyramid
was invented
![Page 59: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/59.jpg)
Still used extensively
![Page 60: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/60.jpg)
input image
user-provided maskforeground details enhanced, background details reduced
Still used extensively
![Page 61: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/61.jpg)
Other types of pyramids
Steerable pyramid: At each level keep multiple versions, one for each direction.
Wavelets: Huge area in image processing (see 18-793).
![Page 62: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/62.jpg)
What are image pyramids used for?image blendingmulti-scale
texture mapping
focal stack compositing denoising
multi-scale detection multi-scale registration
image compression
![Page 63: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/63.jpg)
Some history
![Page 64: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/64.jpg)
Who is this guy?
![Page 65: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/65.jpg)
What is he famous for?
Jean Baptiste Joseph Fourier (1768-1830)
![Page 66: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/66.jpg)
What is he famous for?
Jean Baptiste Joseph Fourier (1768-1830)
‘Any univariate function can be rewritten as a weighted sum of sines and cosines of different frequencies.’
The Fourier series claim (1807):
… and apparently also for the discovery of the greenhouse effect
![Page 67: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/67.jpg)
Is this claim true?
Jean Baptiste Joseph Fourier (1768-1830)
‘Any univariate function can be rewritten as a weighted sum of sines and cosines of different frequencies.’
The Fourier series claim (1807):
![Page 68: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/68.jpg)
Is this claim true?
Jean Baptiste Joseph Fourier (1768-1830)
‘Any univariate function can be rewritten as a weighted sum of sines and cosines of different frequencies.’
The Fourier series claim (1807):
Well, almost.• The theorem requires additional conditions.• Close enough to be named after him.• Very surprising result at the time.
![Page 69: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/69.jpg)
Is this claim true?
Jean Baptiste Joseph Fourier (1768-1830)
‘Any univariate function can be rewritten as a weighted sum of sines and cosines of different frequencies.’
The Fourier series claim (1807):
Well, almost.• The theorem requires additional conditions.• Close enough to be named after him.• Very surprising result at the time.
Malus Lagrange Legendre Laplace
The committee examining his paper
had expressed skepticism, in part due
to not so rigorous proofs
![Page 70: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/70.jpg)
Amusing aside
1820 watercolor caricatures of French mathematicians Adrien-Marie Legendre (left) and Joseph Fourier (right) by French artist Julien-Leopold Boilly
Only known portrait of Adrien-Marie Legendre
For two hundred years, people were misidentifying this
portrait as him
Louis Legendre(same last name, different person)
![Page 71: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/71.jpg)
Fourier series
![Page 72: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/72.jpg)
Fourier’s claim: Add enough of these to get any periodic signal you want!
Basic building block
![Page 73: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/73.jpg)
Fourier’s claim: Add enough of these to get any periodic signal you want!
amplitude
angularfrequency
variablephase
sinusoid
Basic building block
![Page 74: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/74.jpg)
How would you generate this function?
Examples
= +? ?
![Page 75: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/75.jpg)
How would you generate this function?
Examples
= +? ?
![Page 76: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/76.jpg)
How would you generate this function?
Examples
= +? ?
![Page 77: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/77.jpg)
How would you generate this function?
Examples
+? ?
square wave
=
![Page 78: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/78.jpg)
How would you generate this function?
Examples
=
+? ?
square wave
≈
![Page 79: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/79.jpg)
How would you generate this function?
Examples
=
+? ?
square wave
≈
![Page 80: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/80.jpg)
How would you generate this function?
Examples
=
+? ?
square wave
≈
![Page 81: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/81.jpg)
How would you generate this function?
Examples
=
+? ?
square wave
≈
![Page 82: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/82.jpg)
How would you generate this function?
Examples
=
+? ?
square wave
≈
How would you express this mathematically?
![Page 83: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/83.jpg)
Examples
square wave
=
infinite sum of sine waves
How would could you visualize this in the frequency domain?
![Page 84: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/84.jpg)
Examples
square wave
=
infinite sum of sine waves
magnitude
frequency
![Page 85: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/85.jpg)
Frequency domain
![Page 86: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/86.jpg)
frequency
amplitude
Visualizing the frequency spectrum
![Page 87: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/87.jpg)
frequency
amplitude
Visualizing the frequency spectrumRecall the temporal domain visualization
+=
![Page 88: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/88.jpg)
frequency
amplitude
Visualizing the frequency spectrumRecall the temporal domain visualization
+=
How do we plot ...
![Page 89: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/89.jpg)
frequency
amplitude
Visualizing the frequency spectrumRecall the temporal domain visualization
+=
![Page 90: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/90.jpg)
frequency
amplitude
Visualizing the frequency spectrumRecall the temporal domain visualization
+=
![Page 91: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/91.jpg)
not visualizing the symmetric negative part
Need to understand this to understand the 2D version!
frequency
amplitude
Visualizing the frequency spectrum
What is at zero frequency?
Recall the temporal domain visualization
+=
![Page 92: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/92.jpg)
not visualizing the symmetric negative part
Need to understand this to understand the 2D version!
frequency
amplitude
Visualizing the frequency spectrum
signal average (zero for a sine wave with
no offset)
Recall the temporal domain visualization
+=
![Page 93: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/93.jpg)
Spatial domain visualization Frequency domain visualization
1D
2D
Examples
?
![Page 94: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/94.jpg)
Spatial domain visualization Frequency domain visualization
1D
2D
Examples
What do the three dots correspond to?
![Page 95: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/95.jpg)
Spatial domain visualization Frequency domain visualization
Examples
?
![Page 96: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/96.jpg)
Spatial domain visualization Frequency domain visualization
Examples
![Page 97: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/97.jpg)
How would you generate this image with sine waves?
Examples
![Page 98: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/98.jpg)
How would you generate this image with sine waves?
Examples
Has both an x and y components
![Page 99: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/99.jpg)
Examples
=+ ?
![Page 100: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/100.jpg)
Examples
=+
?
![Page 101: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/101.jpg)
Examples
=+
![Page 102: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/102.jpg)
Fourier’s claim: Add enough of these to get any periodic signal you want!
amplitude
angularfrequency
variablephase
sinusoid
Basic building block
What about non-periodic signals?
![Page 103: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/103.jpg)
Fourier transform
![Page 104: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/104.jpg)
Recalling some basics
rectangularcoordinates
Complex numbers have two parts:
what‘s this? what‘s this?
![Page 105: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/105.jpg)
Recalling some basics
rectangularcoordinates
Complex numbers have two parts:
real imaginary
![Page 106: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/106.jpg)
Recalling some basics
rectangularcoordinates
polar coordinates
Complex numbers have two parts:
real imaginary
Alternative reparameterization:
how do we compute these?
polar transform
![Page 107: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/107.jpg)
Recalling some basics
rectangularcoordinates
polar coordinates
Complex numbers have two parts:
real imaginary
Alternative reparameterization:
polar transform
polar transform
![Page 108: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/108.jpg)
Recalling some basics
rectangularcoordinates
polar coordinates
Complex numbers have two parts:
real imaginary
Alternative reparameterization:
polar transform
polar transform
How do you write these in exponential
form?
![Page 109: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/109.jpg)
Recalling some basics
rectangularcoordinates
polar coordinates
Complex numbers have two parts:
real imaginary
Alternative reparameterization:
or equivalentlypolar transform
exponential form
how did we get this?
![Page 110: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/110.jpg)
This will help us understand the Fourier transform equations
Recalling some basics
rectangularcoordinates
polar coordinates
Complex numbers have two parts:
real imaginary
Alternative reparameterization:
or equivalentlypolar transform
exponential form
Euler’s formula
![Page 111: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/111.jpg)
Fourier transform
Fourier transform
dis
cret
eco
nti
nu
ou
s
inverse Fourier transform
Where is the connection to the ‘summation of sine waves’ idea?
![Page 112: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/112.jpg)
Fourier transform
Fourier transform
dis
cret
eco
nti
nu
ou
s
inverse Fourier transform
Where is the connection to the ‘summation of sine waves’ idea?
![Page 113: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/113.jpg)
Where is the connection to the ‘summation of sine waves’ idea?
wave componentsscaling parameter
sum over frequencies
Euler’s formula
Fourier transform
![Page 114: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/114.jpg)
Fourier transform pairsspatial domain frequency domain
Note the symmetry: duality property of Fourier transform
![Page 115: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/115.jpg)
Computing the discrete Fourier transform (DFT)
![Page 116: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/116.jpg)
Computing the discrete Fourier transform (DFT)
is just a matrix multiplication:
In practice this is implemented using the fast Fourier transform (FFT) algorithm.
![Page 117: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/117.jpg)
Fourier transforms of natural images
original amplitude phase
![Page 118: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/118.jpg)
Fourier transforms of natural images
cheetah phase with zebra amplitude
Image phase matters!
zebra phase with cheetah amplitude
![Page 119: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/119.jpg)
Frequency-domain filtering
![Page 120: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/120.jpg)
Why do we care about all this?
![Page 121: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/121.jpg)
Convolution in spatial domain is equivalent to multiplication in frequency domain!
The convolution theorem
The Fourier transform of the convolution of two functions is the product of their Fourier transforms:
The inverse Fourier transform of the product of two Fourier transforms is the convolution of the two inverse Fourier transforms:
![Page 122: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/122.jpg)
What do we use convolution for?
![Page 123: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/123.jpg)
Convolution for 1D continuous signals
Definition of linear shift-invariant filtering as convolution:
filtered signal input signalfilter
Using the convolution theorem, we can interpret and implement all types of linear shift-invariant filtering as multiplication in frequency domain.
Why implement convolution in frequency domain?
![Page 124: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/124.jpg)
Filtering with fft:
im = double(imread(‘…'))/255;
im = rgb2gray(im); % “im” should be a gray-scale floating point image
[imh, imw] = size(im);
hs = 50; % filter half-size
fil = fspecial('gaussian', hs*2+1, 10);
fftsize = 1024; % should be order of 2 (for speed) and include padding
im_fft = fft2(im, fftsize, fftsize); % 1) fft im with
padding
fil_fft = fft2(fil, fftsize, fftsize); % 2) fft fil, pad to
same size as image
im_fil_fft = im_fft .* fil_fft; % 3) multiply fft
images
im_fil = ifft2(im_fil_fft); % 4) inverse fft2
im_fil = im_fil(1+hs:size(im,1)+hs, 1+hs:size(im, 2)+hs); % 5) remove padding
figure(1), imagesc(log(abs(fftshift(im_fft)))), axis image, colormap jet
Frequency-domain filtering in Matlab
Displaying with fft:
![Page 125: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/125.jpg)
=filter kernel
=
Spatial domain filtering
Frequency domain filtering
Fourier transform inverse Fourier transform
![Page 126: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/126.jpg)
Why does the Gaussian give a nice smooth image, but the square filter give edgy artifacts?
Gaussian filter
Box filter
Revisiting blurring
![Page 127: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/127.jpg)
Gaussian blur
![Page 128: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/128.jpg)
Box blur
![Page 129: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/129.jpg)
More filtering examples
?
?
filters shown in frequency-
domain
![Page 130: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/130.jpg)
More filtering exampleslow-pass
band-passfilters shown in frequency-
domain
![Page 131: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/131.jpg)
high-pass
?
More filtering examples
![Page 132: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/132.jpg)
high-pass
More filtering examples
![Page 133: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/133.jpg)
More filtering examples
frequency magnitude
original image low-pass filter
?
![Page 134: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/134.jpg)
More filtering examples
frequency magnitude
original image low-pass filter
![Page 135: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/135.jpg)
More filtering examples
frequency magnitude
original image high-pass filter
?
![Page 136: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/136.jpg)
More filtering examples
frequency magnitude
original image high-pass filter
![Page 137: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/137.jpg)
More filtering examples
frequency magnitude
original image band-pass filter
![Page 138: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/138.jpg)
More filtering examples
frequency magnitude
original image band-pass filter
![Page 139: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/139.jpg)
More filtering examples
frequency magnitude
original image band-pass filter
![Page 140: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/140.jpg)
More filtering examples
frequency magnitude
original image band-pass filter
![Page 141: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/141.jpg)
Revisiting sampling
![Page 142: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/142.jpg)
The Nyquist-Shannon sampling theorem
A continuous signal can be perfectly reconstructed from its discrete version using linear interpolation, if sampling occurred with frequency:
Equivalent reformulation: When downsampling, aliasing does not occur if samples are taken at the Nyquist frequency or higher.
This is called the Nyquist frequency
![Page 143: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/143.jpg)
How does the Nyquist-Shannon theorem relate to the Gaussian pyramid?
Gaussian pyramid
![Page 144: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/144.jpg)
How does the Nyquist-Shannon theorem relate to the Gaussian pyramid?
Gaussian pyramid
• Gaussian blurring is low-pass filtering.• By blurring we try to sufficiently decrease
the Nyquist frequency to avoid aliasing.
How large should the Gauss blur we use be?
![Page 145: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/145.jpg)
Gala Contemplating the Mediterranean Sea Which at Twenty Meters Becomes the Portrait of Abraham Lincoln (Homage to Rothko)
Salvador Dali, 1976
Frequency-domain filtering in human vision
![Page 146: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/146.jpg)
Low-pass filtered version
Frequency-domain filtering in human vision
![Page 147: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/147.jpg)
High-pass filtered version
Frequency-domain filtering in human vision
![Page 148: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/148.jpg)
Variable frequency sensitivity
Experiment: Where do you see the stripes?
frequency
con
tras
t
![Page 149: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/149.jpg)
Campbell-Robson contrast sensitivity curve
frequency
con
tras
t
Our eyes are sensitive to mid-range frequencies
Variable frequency sensitivity
• Early processing in humans filters for various orientations and scales of frequency
• Perceptual cues in the mid frequencies dominate perception
![Page 150: Image pyramids and frequency domain16385/s19/lectures/lecture3.pdf• Frequency domain. • Fourier transform. • Frequency-domain filtering. • Revisiting sampling. Slide credits](https://reader036.vdocuments.us/reader036/viewer/2022071409/610200db9122a91e514ea91a/html5/thumbnails/150.jpg)
References
Basic reading:• Szeliski textbook, Sections 3.4, 3.5
Additional reading:• Burt and Adelson, “The Laplacian Pyramid as a Compact Image Code,” IEEE ToC 1983.
The original Laplacian pyramid paper.• Hubel and Wiesel, “Receptive fields, binocular interaction and functional architecture in the cat's visual
cortex,” The Journal of Physiology 1962A foundational paper describing information processing in the visual system, including the different types of filtering it performs; Hubel and Wiesel won the Nobel Prize in Medicine in 1981 for the discoveries described in this paper.