c280, computer vision - people
TRANSCRIPT
![Page 1: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/1.jpg)
C280 Computer VisionC280, Computer Vision
Prof. Trevor Darrell
[email protected]@eecs.berkeley.edu
L t 5 P id
Freeman
Lecture 5: Pyramids
![Page 2: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/2.jpg)
Last time: Image Filters• Filters allow local image neighborhood to influence our
description and features
– Smoothing to reduce noise
– Derivatives to locate contrast, gradient
• Filters have highest response on neighborhoods that “look g p glike” it; can be thought of as template matching.
• Convolution properties will influence the efficiency with which we can process images.
– Associative
Filter separability– Filter separability
• Edge detection processes the image gradient to find curves, or chains of edgels.
Freeman
![Page 3: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/3.jpg)
TodayToday
• Review of Fourier TransformReview of Fourier Transform
• Sampling and Aliasing
id• Image Pyramids
• Applications: Blending and noise removal
Freeman
![Page 4: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/4.jpg)
Background: Fourier Analysis
Note symmetry in magnitudeF( ) F( )
Freeman
F(w)=F(‐w)
![Page 5: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/5.jpg)
Background: Fourier Analysis
Freeman
![Page 6: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/6.jpg)
Freeman
![Page 7: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/7.jpg)
Freeman
![Page 8: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/8.jpg)
Background: high/low pass
Freeman Image credit: Sandberg, UC Boulder
![Page 9: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/9.jpg)
Background: 2D FT Example
Freeman Image credit: Sandberg, UC Boulder
![Page 10: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/10.jpg)
Background: high/low pass
FreemancImage credit: Sandberg, UC Boulder
![Page 11: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/11.jpg)
Background: more examplesBackground: more examples
• http://mathworld.wolfram.com/FourierTransform.hthttp://mathworld.wolfram.com/FourierTransform.html
• http://en.wikipedia.org/wiki/Discrete‐p // p g/ /time_Fourier_transform
• http://www.cs.unm.edu/~brayer/vision/fourier.htmlp y
• …
Freemanc
![Page 12: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/12.jpg)
Magnitude vs Phase…?Magnitude vs Phase…?
• Mostly considered Magnitude spectra so farMostly considered Magnitude spectra so far
• Sufficient for many vision methods:hi h /l h l di l t i– high‐pass/low‐pass channel coding later in lecture.
simple edge detection focus/defocus models– simple edge detection, focus/defocus models
– certain texture models
d d ll f !• May discard perceptually significant structure!
Freemanc
![Page 13: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/13.jpg)
Phase and MagnitudePhase and Magnitude
• Fourier transform of a real • Curious factfunction is complex– difficult to plot, visualize
– instead, we can think of the
– all natural images have about the same magnitude transform
– hence, phase seems to matter, phase and magnitude of the transform
• Phase is the phase of the complex f
but magnitude largely doesn’t
• Demonstration– Take two pictures, swap the
h f htransform
• Magnitude is the magnitude of the complex transform
phase transforms, compute the inverse ‐ what does the result look like?
FreemancD.A. Forsyth
![Page 14: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/14.jpg)
Freeman
Computer Vision ‐ A Modern ApproachSet: Pyramids and Texture
Slides by D.A. ForsythcD.A. Forsyth
![Page 15: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/15.jpg)
This is the magnitude transform of the cheetah piccheetah pic
Freeman
Computer Vision ‐ A Modern ApproachSet: Pyramids and Texture
Slides by D.A. ForsythcD.A. Forsyth
![Page 16: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/16.jpg)
This is the phase transform of the cheetah piccheetah pic
Freeman
Computer Vision ‐ A Modern ApproachSet: Pyramids and Texture
Slides by D.A. ForsythcD.A. Forsyth
![Page 17: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/17.jpg)
Freeman
Computer Vision ‐ A Modern ApproachSet: Pyramids and Texture
Slides by D.A. ForsythcD.A. Forsyth
![Page 18: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/18.jpg)
This is the magnitude transform of the zebra picpic
Freeman
Computer Vision ‐ A Modern ApproachSet: Pyramids and Texture
Slides by D.A. ForsythcD.A. Forsyth
![Page 19: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/19.jpg)
This is the phase transform of the zebra picpic
Freeman
Computer Vision ‐ A Modern ApproachSet: Pyramids and Texture
Slides by D.A. ForsythcD.A. Forsyth
![Page 20: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/20.jpg)
Reconstruction with zebra phase, cheetah magnitude
FreemancD.A. Forsyth
![Page 21: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/21.jpg)
Reconstruction with cheetah phase, zebra magnitudemagnitude
FreemancD.A. Forsyth
![Page 22: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/22.jpg)
1D D.O.G.1D D.O.G.
Freeman
![Page 23: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/23.jpg)
Sampling and aliasingSampling and aliasing
Freeman
![Page 24: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/24.jpg)
Sampling in 1D takes a continuous function and replaces it with a t f l i ti f th f ti ’ l t t fvector of values, consisting of the function’s values at a set of
sample points. We’ll assume that these sample points are on a regular grid, and can place one at each integer for convenience.
Freeman
![Page 25: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/25.jpg)
Sampling in 2D does the same thing, only in 2D. We’ll assume that th l i t l id d l t hthese sample points are on a regular grid, and can place one at each integer point for convenience.
Freeman
![Page 26: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/26.jpg)
The Fourier transform of a sampled signal
F Sample f ( ) F f ( ) ( i j)
F Sample2D f (x,y) F f (x, y) (x i,y j)
i
i
F f (x y) **F (x i y j)
** F f (x,y) **F (x i, y j)
i
i
F u i v j
*
F u i,v j j
i
Freeman
![Page 27: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/27.jpg)
Freeman
![Page 28: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/28.jpg)
Freeman
![Page 29: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/29.jpg)
AliasingAliasing
• Can’t shrink an image by taking every second pixelg y g y p• If we do, characteristic errors appear
– In the next few slides– Typically, small phenomena look bigger; fast phenomena can look slower
– Common phenomenonCommon phenomenon• Wagon wheels rolling the wrong way in movies• Checkerboards misrepresented in ray tracing
Freeman
![Page 30: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/30.jpg)
Space domain explanation of Nyquist lsampling
You need to have at least two samples per sinusoid cycle to represent that sinusoid.
Freeman
![Page 31: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/31.jpg)
R l thResample the checkerboard by taking one sample at each circle. In the case of the top leftIn the case of the top left board, new representation is reasonable. Top right also yields aTop right also yields a reasonable representation. Bottom left is all black (dubious) and bottom ( )right has checks that are too big.
Freeman
![Page 32: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/32.jpg)
Smoothing as low‐pass filteringSmoothing as low pass filtering
• The message of the FT is • A filter whose FT is a that high frequencies lead to trouble with sampling.
• Solution: suppress high
box is bad, because the filter kernel has infinite supportSolution: suppress high
frequencies before sampling
multiply the FT of the
support• Common solution: use a Gaussian– multiply the FT of the
signal with something that suppresses high frequencies
Gaussian– multiplying FT by Gaussian is equivalent to convolving image withfrequencies
– or convolve with a low‐pass filter
convolving image with Gaussian.
Freeman
![Page 33: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/33.jpg)
Sampling without smoothing. Top row shows the images,sampled at every second pixel to get the nextsampled at every second pixel to get the next.
Freeman
![Page 34: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/34.jpg)
Sampling with smoothing. Top row shows the images. Weget the next image by smoothing the image with a Gaussian with sigma 1 pixel,h li d i l hthen sampling at every second pixel to get the next.
Freeman
![Page 35: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/35.jpg)
Sampling with smoothing. Top row shows the images. Weget the next image by smoothing the image with a Gaussian with sigma 1.4 pixels,then sampling at every second pixel to get the nextthen sampling at every second pixel to get the next.
Freeman
![Page 36: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/36.jpg)
Sampling exampleAnalyze crossed
tigratings…
Freeman
![Page 37: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/37.jpg)
Sampling exampleAnalyze crossed
tigratings…
Freeman
![Page 38: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/38.jpg)
Sampling exampleAnalyze crossed
tigratings…
Freeman
![Page 39: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/39.jpg)
Sampling exampleAnalyze crossed
tigratings…
Where does perceived near horizontal grating come f ?from?
Freeman
![Page 40: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/40.jpg)
A F(A)
Freeman
![Page 41: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/41.jpg)
B F(B)
Freeman
![Page 42: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/42.jpg)
AB F(A) * F(B)
Freeman
(using Szeliski notation, ‘*’ is convolution)
![Page 43: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/43.jpg)
AB F(A) * F(B)
Freeman
![Page 44: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/44.jpg)
CC
AB Lowpass( )F(A) * F(B)
Freeman
p ( )~=F(C)
![Page 45: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/45.jpg)
Control testControl test
• If our analysis is correct if we add those twoIf our analysis is correct, if we add those two sinusoids (or square waves), and if there is no non‐linearity in the display of the sum thennon linearity in the display of the sum, then there should only be summing, not convolution in the frequency domainconvolution, in the frequency domain.
Freeman
![Page 46: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/46.jpg)
AB A+B
Freeman F(A) + F(B)F(A) * F(B)
![Page 47: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/47.jpg)
A*BLow‐pass filtered
A+B
Freeman F(A) + F(B)F(A) * F(B)
![Page 48: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/48.jpg)
Image information occurs at all spatial scales
Freeman
![Page 49: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/49.jpg)
Image pyramidsImage pyramids
• Gaussian pyramidGaussian pyramid
• Laplacian pyramid
l /Q id• Wavelet/QMF pyramid
• Steerable pyramid
Freeman
![Page 50: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/50.jpg)
Image pyramidsImage pyramids
• Gaussian pyramidGaussian pyramid
Freeman
![Page 51: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/51.jpg)
The Gaussian pyramidThe Gaussian pyramid
• Smooth with gaussians, becauseSmooth with gaussians, because– a gaussian*gaussian=another gaussian
• Gaussians are low pass filters soGaussians are low pass filters, so representation is redundant.
Freeman
![Page 52: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/52.jpg)
The computational advantage of pyramids
Freemanhttp://www‐bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf
![Page 53: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/53.jpg)
Freemanhttp://www‐bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf
![Page 54: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/54.jpg)
Freeman
![Page 55: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/55.jpg)
Convolution and subsampling as a matrix multiply (1‐d case)p g p y ( )
xGx 112 xGx
1G
1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0 0 0
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0
Freeman
0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 4 6 4 1 0
(Normalization constant of 1/16 omitted for visual clarity.)
![Page 56: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/56.jpg)
Next pyramid levelNext pyramid level223 xGx
2G
1 4 6 4 1 0 0 0
0 0 1 4 6 4 1 0
0 0 0 0 1 4 6 4
0 0 0 0 0 0 1 40 0 0 0 0 0 1 4
Freeman
![Page 57: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/57.jpg)
The combined effect of the two pyramid levels
GG
1123 xGGx
1 4 10 20 31 40 44 40 31 20 10 4 1 0 0 0 0 0 0 0
12GG
0 0 0 0 1 4 10 20 31 40 44 40 31 20 10 4 1 0 0 0
0 0 0 0 0 0 0 0 1 4 10 20 31 40 44 40 30 16 4 0
0 0 0 0 0 0 0 0 0 0 0 0 1 4 10 20 25 16 4 0
Freeman
![Page 58: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/58.jpg)
Freemanhttp://www‐bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf
![Page 59: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/59.jpg)
Gaussian pyramids used forGaussian pyramids used for
• up‐ or down‐ sampling imagesup or down sampling images.
• Multi‐resolution image analysisL k f bj t i ti l l– Look for an object over various spatial scales
– Coarse‐to‐fine image processing: form blur estimate or the motion analysis on very lowestimate or the motion analysis on very low‐resolution image, upsample and repeat. Often a successful strategy for avoiding local minima insuccessful strategy for avoiding local minima in complicated estimation tasks.
Freeman
![Page 60: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/60.jpg)
Image pyramidsImage pyramids
• GaussianGaussian
• Laplacian
l /Q• Wavelet/QMF
• Steerable pyramid
Freeman
![Page 61: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/61.jpg)
Image pyramidsImage pyramids
• Laplacian
Freeman
![Page 62: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/62.jpg)
The Laplacian PyramidThe Laplacian Pyramid
• SynthesisSynthesis– Compute the difference between upsampled Gaussian pyramid level and Gaussian pyramid level.
– band pass filter ‐ each level represents spatial frequencies (largely) unrepresented at other levelfrequencies (largely) unrepresented at other level.
Freeman
![Page 63: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/63.jpg)
Laplacian pyramid algorithmLaplacian pyramid algorithm
1x 11xG 2x 2x 3x
333 )( xGFI
GF222 )( xGFI
111 xGF
Freeman 111 )( xGFI
![Page 64: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/64.jpg)
Upsampling
332 xFy
3F 6 1 0 0
4 4 0 0
1 6 1 0
0 4 4 0
0 1 6 1
0 0 4 4
0 0 1 6
Freeman
0 0 1 6
0 0 0 4
![Page 65: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/65.jpg)
Showing, at full resolution, the information captured at each level of a Gaussian (top) and Laplacian (bottom) pyramid.( p) p ( ) py
Freemanhttp://www‐bcs.mit.edu/people/adelson/pub_pdfs/pyramid83.pdf
![Page 66: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/66.jpg)
Laplacian pyramid reconstruction algorithm: f L L L drecover x1 from L1, L2, L3 and x4
G# is the blur‐and‐downsample operator at pyramid level #G# is the blur‐and‐downsample operator at pyramid level #F# is the blur‐and‐upsample operator at pyramid level #
Laplacian pyramid elements:L1 = (I – F1 G1) x1L2 = (I – F2 G2) x2L3 = (I – F3 G3) x3x2 = G1 x1x2 G1 x1x3 = G2 x2x4 = G3 x3
Reconstruction of original image (x1) from Laplacian pyramid elements:x3 = L3 + F3 x4x2 = L2 + F2 x3
Freeman
x1 = L1 + F1 x2
![Page 67: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/67.jpg)
Laplacian pyramid reconstruction algorithm: f L L L drecover x1 from L1, L2, L3 and g3
1x 2x 3xg
+
3g
+3L
+ 2L
Freeman 1L
![Page 68: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/68.jpg)
Freeman
Gaussian pyramid
![Page 69: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/69.jpg)
Freeman
Laplacian pyramid
![Page 70: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/70.jpg)
Image pyramidsImage pyramids
l /Q• Wavelet/QMF
Freeman
![Page 71: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/71.jpg)
Wavelets/QMF’sWavelets/QMF s
fUF
V t i d i
transformed image
fUF Vectorized image
Fourier transform, orWavelet transform, or
Freeman
Steerable pyramid transform
![Page 72: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/72.jpg)
The simplest wavelet transform: h H fthe Haar transform
U =
1 1
1 ‐1
Freeman
![Page 73: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/73.jpg)
The inverse transform for the Haar waveletThe inverse transform for the Haar wavelet
( )>> inv(U)
ans =
0.5000 0.5000
0.5000 ‐0.50000.5000 0.5000
Freeman
![Page 74: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/74.jpg)
Apply this over multiple spatial positionsApply this over multiple spatial positions
U =
1 1 0 0 0 0 0 0
1 ‐1 0 0 0 0 0 0
0 0 1 1 0 0 0 0
0 0 1 ‐1 0 0 0 0
0 0 0 0 1 1 0 0
0 0 0 0 1 ‐1 0 0
0 0 0 0 0 0 1 1
Freeman
0 0 0 0 0 0 1 1
0 0 0 0 0 0 1 ‐1
![Page 75: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/75.jpg)
The high frequenciesThe high frequencies
U =
1 1 0 0 0 0 0 0
1 ‐1 0 0 0 0 0 0
0 0 1 1 0 0 0 0
0 0 1 ‐1 0 0 0 0
0 0 0 0 1 1 0 0
0 0 0 0 1 ‐1 0 0
0 0 0 0 0 0 1 1
Freeman
0 0 0 0 0 0 1 1
0 0 0 0 0 0 1 ‐1
![Page 76: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/76.jpg)
The low frequenciesThe low frequencies
U =
1 1 0 0 0 0 0 0
1 ‐1 0 0 0 0 0 0
0 0 1 1 0 0 0 0
0 0 1 ‐1 0 0 0 0
0 0 0 0 1 1 0 0
0 0 0 0 1 ‐1 0 0
0 0 0 0 0 0 1 1
Freeman
0 0 0 0 0 0 1 1
0 0 0 0 0 0 1 ‐1
![Page 77: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/77.jpg)
The inverse transformThe inverse transform
>> inv(U)
ans =
0.5000 0.5000 0 0 0 0 0 0
0 5000 ‐0 5000 0 0 0 0 0 0
L H L H L H L H
0.5000 0.5000 0 0 0 0 0 0
0 0 0.5000 0.5000 0 0 0 0
0 0 0.5000 ‐0.5000 0 0 0 0
0 0 0 0 0.5000 0.5000 0 0
0 0 0 0 0.5000 ‐0.5000 0 0
0 0 0 0 0 0 0 000 0 000
Freeman
0 0 0 0 0 0 0.5000 0.5000
0 0 0 0 0 0 0.5000 ‐0.5000
![Page 78: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/78.jpg)
Simoncelli and Adelson, in “Subband coding”, Kluwer, 1990.
Freeman
![Page 79: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/79.jpg)
Simoncelli and Adelson, in “Subband coding”, Kluwer, 1990.
Freeman
![Page 80: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/80.jpg)
Simoncelli and Adelson, in “Subband coding”, Kluwer, 1990.
Freeman
![Page 81: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/81.jpg)
Simoncelli and Adelson, in “Subband coding”, Kluwer, 1990.
Freeman
![Page 82: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/82.jpg)
Now, in 2 dimensions…
Horizontal high pass
Frequency domain
Freeman Horizontal low pass
![Page 83: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/83.jpg)
Apply the wavelet transform separable in both dimensions
Horizontal high pass, vertical high pass
Horizontal high pass, vertical low-pass
Freeman
Horizontal low pass, vertical high-pass
Horizontal low pass,Vertical low-pass
![Page 84: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/84.jpg)
Simoncelli and Adelson, in “Subband coding”, Kluwer, 1990.
To create 2‐d filters, applyTo create 2 d filters, apply the 1‐d filters separably in the two spatial dimensionsdimensions
Freeman
![Page 85: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/85.jpg)
Wavelet/QMF representationWavelet/QMF representation
Freeman
![Page 86: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/86.jpg)
What is a good representation for image analysis?
(Goldilocks and the three representations)(Goldilocks and the three representations)
• Fourier transform domain tells you “what” ( l i ) b “ h ” I(textural properties), but not “where”. In space, this representation is too spread out.
• Pixel domain representation tells you “where”• Pixel domain representation tells you where (pixel location), but not “what”. In space, this representation is too localized
• Want an image representation that gives you a local description of image events—what is happening where That representation might be
Freeman
happening where. That representation might be “just right”.
![Page 87: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/87.jpg)
Good and bad features of /wavelet/QMF filters
• Bad:Bad: – Aliased subbands
Non oriented diagonal subband– Non‐oriented diagonal subband
• Good:(– Not overcomplete (so same number of
coefficients as image pixels).
G d f i i (JPEG 2000)– Good for image compression (JPEG 2000).
– Separable computation, so it’s fast.
Freeman
![Page 88: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/88.jpg)
Freeman
![Page 89: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/89.jpg)
Image pyramidsImage pyramids
• Steerable pyramid
Freeman
![Page 90: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/90.jpg)
Steerable filters
Freemanhttp://people.csail.mit.edu/billf/freemanThesis.pdf
![Page 91: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/91.jpg)
But we need to get rid of the corner regions before starting the recursive circular filtering
Freemanhttp://www.cns.nyu.edu/ftp/eero/simoncelli95b.pdf Simoncelli and Freeman, ICIP 1995
![Page 92: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/92.jpg)
Freemanhttp://www.merl.com/reports/docs/TR95-15.pdf
![Page 93: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/93.jpg)
Reprinted from “Shiftable MultiScale Transforms,” by Simoncelli et al., IEEE Transactions
Freeman
Reprinted from Shiftable MultiScale Transforms, by Simoncelli et al., IEEE Transactionson Information Theory, 1992, copyright 1992, IEEE
![Page 94: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/94.jpg)
Non‐oriented steerable pyramidNon oriented steerable pyramid
Freemanhttp://www.merl.com/reports/docs/TR95-15.pdf
![Page 95: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/95.jpg)
3‐orientation steerable pyramid3 orientation steerable pyramid
Freemanhttp://www.merl.com/reports/docs/TR95-15.pdf
![Page 96: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/96.jpg)
Steerable pyramidsSteerable pyramids
• Good:Good:– Oriented subbands
– Non‐aliased subbands
– Steerable filters
– Used for: noise removal, texture analysis and synthesis, l i h di / i di i i isuper‐resolution, shading/paint discrimination.
• Bad:O l t– Overcomplete
– Have one high frequency residual subband, required in order to form a circular region of analysis in frequency
Freeman
g y q yfrom a square region of support in frequency.
![Page 97: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/97.jpg)
Freemanhttp://www.cns.nyu.edu/ftp/eero/simoncelli95b.pdf Simoncelli and Freeman, ICIP 1995
![Page 98: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/98.jpg)
f id i• Summary of pyramid representations
Freeman
![Page 99: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/99.jpg)
Image pyramidsProgressively blurred andProgressively blurred and subsampled versions of the image. Adds scale invariance to fixed‐size algorithms• Gaussian
Shows the information added d h
to fixed size algorithms.
in Gaussian pyramid at each spatial scale. Useful for noise reduction & coding.
• Laplacian
Shows components at each
Bandpassed representation, complete, but withaliasing and some non‐oriented subbands.• Wavelet/QMF
Shows components at each scale and orientation separately. Non‐aliased subbands Good for texture
Freeman
subbands. Good for texture and feature analysis. But overcomplete and with HF residual.
• Steerable pyramid
![Page 100: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/100.jpg)
Schematic pictures of each matrix transform
Shown for 1‐d imagesg
The matrices for 2‐d images are the same idea, but more complicated, to account for vertical, as well as horizontal,
i hb l ti hineighbor relationships.
transformed image
fUF
Vectorized image
Fourier transform, orW l f
Freeman
Wavelet transform, orSteerable pyramid transform
![Page 101: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/101.jpg)
Fourier transform
= *= *
pixel domain image
Fourier bases are global:
Fourier transform imageare global:
each transform coefficient d d ll
transform
Freeman
depends on all pixel locations.
![Page 102: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/102.jpg)
Gaussian pyramid
*= *G i pixel imageGaussian pyramid
Overcomplete representation
Freeman
Overcomplete representation. Low‐pass filters, sampled appropriately for their blur.
![Page 103: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/103.jpg)
Laplacian pyramid
= *= *i l iLaplacian pixel imageLaplacian
pyramid
Overcomplete representation
Freeman
Overcomplete representation. Transformed pixels represent bandpassed image information.
![Page 104: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/104.jpg)
Wavelet (QMF) transform
Wavelet= *
Wavelet pyramid
pixel imageOrtho‐normal transform (like F i f )Fourier transform), but with localized basis functions.
Freeman
![Page 105: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/105.jpg)
Steerable pyramid
= *Multiple
orientations at one scale= *
i l iSteerable
one scale
pixel imageSteerable pyramid
Multiple orientations at
Over‐complete representation, but non aliased
the next scale
Freeman
but non‐aliased subbands.
the next scale…
![Page 106: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/106.jpg)
Matlab resources for pyramids (with tutorial)
http://www cns nyu edu/~eero/software htmlhttp://www.cns.nyu.edu/ eero/software.html
Freeman
![Page 107: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/107.jpg)
Matlab resources for pyramids (with tutorial)
http://www cns nyu edu/~eero/software htmlhttp://www.cns.nyu.edu/ eero/software.html
Freeman
![Page 108: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/108.jpg)
Why use these representations?Why use these representations?
• Handle real‐world size variations with aHandle real world size variations with a constant‐size vision algorithm.
• Remove noise• Remove noise
• Analyze texture
• Recognize objects
• Label image featuresg
Freeman
![Page 109: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/109.jpg)
Freemanhttp://web.mit.edu/persci/people/adelson/pub_pdfs/RCA84.pdf
![Page 110: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/110.jpg)
Freemanhttp://web.mit.edu/persci/people/adelson/pub_pdfs/RCA84.pdf
![Page 111: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/111.jpg)
Freemanhttp://web.mit.edu/persci/people/adelson/pub_pdfs/RCA84.pdf
![Page 112: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/112.jpg)
An application of image pyramids:i lnoise removal
Freeman
![Page 113: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/113.jpg)
Image statistics (or, mathematically, how can you tell image from noise?)y g )
Noisy image
Freeman
![Page 114: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/114.jpg)
Clean image
Freeman
![Page 115: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/115.jpg)
Pixel representation,Pixel representation, image histogram
Freeman
![Page 116: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/116.jpg)
Pixel representation, noisyPixel representation, noisy image histogram
Freeman
![Page 117: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/117.jpg)
bandpass filtered imagep g
Freeman
![Page 118: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/118.jpg)
bandpassed representation imagebandpassed representation image histogram
Freeman
![Page 119: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/119.jpg)
Pixel domain noise image and histogram
Freeman
![Page 120: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/120.jpg)
Bandpass domain noise image and histogram
Freeman
![Page 121: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/121.jpg)
Noise‐corrupted full‐freq and bandpass images
But want thethe bandpass image histogram t l k likto look like this
Freeman
![Page 122: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/122.jpg)
Bayes theoremBayes theorem
P(x, y) = P(x|y) P(y)P(x, y) = P(x|y) P(y)P(x, y) = P(x|y) P(y) By definition of conditional probability
soP(x|y) P(y) = P(y|x) P(x)soP(x|y) P(y) = P(y|x) P(x)
Using that twice
P(x|y) P(y) P(y|x) P(x)P(x|y) P(y) P(y|x) P(x)andP( | ) P( | ) P( ) / P( )P(x|y) = P(y|x) P(x) / P(y)
The parameters you LikelihoodConstant w.r.t.
Freeman
p ywant to estimate
What you observe Prior probability
Likelihood function
parameters x.
![Page 123: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/123.jpg)
Bayesian MAP estimator for clean bandpass coefficient values
Let x = bandpassed image value before adding noise.Let y = noise‐corrupted observation.
By Bayes theorem
P(x|y) k P(y|x) P(x)y = 25
P(x)
P(x|y) = k P(y|x) P(x)
P( | )
y
P(y|x)
P(y|x)
P(x|y)P(x|y)
Freeman
![Page 124: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/124.jpg)
Bayesian MAP estimatorLet x = bandpassed image value before adding noise.Let y = noise‐corrupted observation.
By Bayes theorem
P(x|y) k P(y|x) P(x)y = 50
P(x|y) = k P(y|x) P(x) y
P( | )P(y|x)
P(x|y)
Freeman
![Page 125: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/125.jpg)
Bayesian MAP estimatorLet x = bandpassed image value before adding noise.Let y = noise‐corrupted observation.
By Bayes theorem
P(x|y) k P(y|x) P(x)y = 115
P(x|y) = k P(y|x) P(x) y
P( | )P(y|x)
P(x|y)
Freeman
P(x|y)
![Page 126: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/126.jpg)
MAP estimate, , as function of b d ffi i l
x̂observed coefficient value, y
x̂
y
Freemanhttp://www‐bcs.mit.edu/people/adelson/pub_pdfs/simoncelli_noise.pdfSimoncelli and Adelson, Noise Removal via Bayesian Wavelet Coring
![Page 127: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/127.jpg)
Noise removal results
Freemanhttp://www‐bcs.mit.edu/people/adelson/pub_pdfs/simoncelli_noise.pdfSimoncelli and Adelson, Noise Removal via Bayesian Wavelet Coring
![Page 128: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/128.jpg)
Slide CreditsSlide Credits
• Bill FreemanBill Freeman
• and others, as noted…
Freeman
![Page 129: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/129.jpg)
More on statistics of natural scenesMore on statistics of natural scenes
• Olshausen and Field:– Natural Image Statistics and Efficient Coding, https://redwood.berkeley.edu/bruno/papers/stirling.ps.Z
– Relations between the statistics of natural images and the response properties of cortical cells.
– http://redwood.psych.cornell.edu/papers/field_8df7.pdf
• Aude Olivia:
Freeman
– http://cvcl.mit.edu/SUNSlides/9.912‐CVC‐ImageAnalysis‐web.pdf
![Page 130: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/130.jpg)
TodayToday
• Review of Fourier TransformReview of Fourier Transform
• Sampling and Aliasing
id• Image Pyramids
• Applications: Blending and noise removal
Freeman
![Page 131: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/131.jpg)
Next time: Feature Detection and Matching
• Local features
• Pyramids for invariant feature detectionPyramids for invariant feature detection
• Local descriptors
M t hi• Matching
Freeman
![Page 132: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/132.jpg)
Appendix: SteeringAppendix: Steering
Freeman
![Page 133: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/133.jpg)
Simple example of steerable filter
G160o
(x) cos(60o)G10o
(x) sin(60o)G190o
(x)G1 (x) cos(60 )G1 (x) sin(60 )G1 (x)
Freeman
![Page 134: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/134.jpg)
Freeman
![Page 135: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/135.jpg)
Steering theorem
O i i llOriginally written as sines & cosines
Freeman
![Page 136: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/136.jpg)
Steering theorem for polynomials
For an Nth order polynomial with even symmetry N+1 basis f i ffi ifunctions are sufficient.
Freeman
![Page 137: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/137.jpg)
Freeman
![Page 138: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/138.jpg)
Steerable quadrature pairsSteerable quadrature pairs
G2 H2
G2 H 2 | FT(G ) | | FT(H ) |G2 H2 | FT(G2 ) |, | FT(H2 ) |
Freeman
![Page 139: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/139.jpg)
How quadrature pair filters workHow quadrature pair filters work
Freeman
![Page 140: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/140.jpg)
How quadrature pair filters workHow quadrature pair filters work
Freeman
![Page 141: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/141.jpg)
Freeman
![Page 142: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/142.jpg)
Freeman
![Page 143: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/143.jpg)
Orientation analysisOrientation analysis
Freeman
![Page 144: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/144.jpg)
Orientation analysis
Freeman
![Page 145: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/145.jpg)
Freeman
![Page 146: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/146.jpg)
Freeman
![Page 147: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/147.jpg)
Freeman
![Page 148: C280, Computer Vision - People](https://reader033.vdocuments.us/reader033/viewer/2022052821/628fdd61c6bfc35e9a532894/html5/thumbnails/148.jpg)
Freeman