lecture 5: edges, corners, sampling, pyramidslecture 5: edges, corners, sampling, pyramids thursday,...
TRANSCRIPT
![Page 1: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/1.jpg)
Lecture 5: Edges, Corners, Sampling, Pyramids
Thursday, Sept 13
![Page 2: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/2.jpg)
![Page 3: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/3.jpg)
Normalized cross correlation
• Normalized correlation: normalize for image region brightness
• Windowed correlation search: inexpensive way to find a fixed scale pattern
• (Convolution = correlation if filter is symmetric)
Best matchTemplate
![Page 4: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/4.jpg)
Filters and scenes
![Page 5: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/5.jpg)
Filters and scenes
• Scenes have holistic qualities• Can represent scene categories with
global texture• Use Steerable filters, windowed for some
limited spatial information• Model likelihood of filter responses given
scene category as mixture of Gaussians, (and incorporate some temporal info…)
[Torralba, Murphy, Freeman, and Rubin, ICCV 2003][Torralba & Oliva, 2003]
![Page 6: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/6.jpg)
Steerable filters
• Convolution linear -- synthesize a filter of arbitrary orientation as a linear combination of “basis filters”
• Interpolated filter responses more efficient than explicit filter at arbitrary orientation
[Freeman & Adelson, The Design and Use of Steerable Filters, PAMI 1991]
![Page 7: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/7.jpg)
Steerable filters
=
=
Freeman & Adelson, 1991
Basis filters for derivative of Gaussian
![Page 8: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/8.jpg)
[Torralba, Murphy, Freeman, and Rubin, ICCV 2003]
Probability of the scene given global features
![Page 9: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/9.jpg)
[Torralba, Murphy, Freeman, and Rubin, ICCV 2003]
![Page 10: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/10.jpg)
Contextual priors• Use scene recognition predict objects present• For object(s) likely to be present, predict locations
based on similarity to previous images with the same place and that object
![Page 11: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/11.jpg)
[Torralba, Murphy, Freeman, and Rubin, ICCV 2003]
Scene category
Specific place
(black=right, red=wrong)
![Page 12: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/12.jpg)
Blue solid circle: recognition with temporal info
Black hollow circle: instantaneous recognition using global feature only
Cross: true location
![Page 13: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/13.jpg)
Image gradientThe gradient of an image:
The gradient points in the direction of most rapid change in intensity
The gradient direction (orientation of edge normal) is given by:
The edge strength is given by the gradient magnitude
Slide credit S. Seitz
![Page 14: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/14.jpg)
Effects of noiseConsider a single row or column of the image
• Plotting intensity as a function of position gives a signal
Where is the edge?
Slide credit S. Seitz
![Page 15: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/15.jpg)
Where is the edge?
Solution: smooth first
Look for peaks in
![Page 16: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/16.jpg)
Derivative theorem of convolution
This saves us one operation:
Slide credit S. Seitz
![Page 17: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/17.jpg)
Laplacian of Gaussian
Consider
Laplacian of Gaussianoperator
Where is the edge? Zero-crossings of bottom graph
![Page 18: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/18.jpg)
2D edge detection filters
• is the Laplacian operator:
Laplacian of Gaussian
Gaussian derivative of Gaussian
Slide credit S. Seitz
![Page 19: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/19.jpg)
The Canny edge detector
original image (Lena)
![Page 20: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/20.jpg)
The Canny edge detector
norm of the gradient
![Page 21: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/21.jpg)
The Canny edge detector
thresholding
![Page 22: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/22.jpg)
Non-maximum suppression
Check if pixel is local maximum along gradient direction, select single max across width of the edge• requires checking interpolated pixels p and r
Slide credit S. Seitz
![Page 23: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/23.jpg)
The Canny edge detector
thinning(non-maximum suppression)
![Page 24: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/24.jpg)
Predicting the next edge point
Assume the marked point is an edge point. Then we construct the tangent to the edge curve (which is normal to the gradient at that point) and use this to predict the next points (here either r or s).
(Forsyth & Ponce)
![Page 25: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/25.jpg)
Hysteresis ThresholdingReduces the probability of false contours
and fragmented edges
Given result of non-maximum suppression:For all edge points that remain,
- locate next unvisited pixel where intensity > thigh
- start from that point, follow chains along edge and add points where intensity < tlow
![Page 26: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/26.jpg)
Edge detection by subtraction
original
![Page 27: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/27.jpg)
Edge detection by subtraction
smoothed (5x5 Gaussian)
![Page 28: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/28.jpg)
Edge detection by subtraction
smoothed – original(scaled by 4, offset +128)
Why doesthis work?
![Page 29: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/29.jpg)
Gaussian - image filter
Laplacian of Gaussian
Gaussian delta function
![Page 30: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/30.jpg)
Causes of edges
Adapted from C. Rasmussen
If the goal is image understanding, what do we want from an edge detector?
![Page 31: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/31.jpg)
Learning good boundaries
• Use ground truth (human-labeled) boundaries in natural images to learn good features
• Supervised learning to optimize cue integration, filter scales, select feature typesWork by D. Martin and C. Fowlkes and D. Tal and J. Malik, Berkeley Segmentation Benchmark, 2001
![Page 32: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/32.jpg)
[D. Martin et al. PAMI 2004]
Human-marked segment boundaries
![Page 33: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/33.jpg)
Feature profiles (oriented energy, brightness, color, and texture gradients) along the patch’s horizontal diameter
[D. Martin et al. PAMI 2004]
What features are responsible for perceived edges?
![Page 34: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/34.jpg)
What features are responsible for perceived edges?
![Page 35: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/35.jpg)
Learning good boundaries
[D. Martin et al. PAMI 2004]
![Page 36: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/36.jpg)
Original Boundary detection Human-labeled
Berkeley Segmentation Database, D. Martin and C. Fowlkes and D. Tal and J. Malik
![Page 37: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/37.jpg)
[D. Martin et al. PAMI 2004]
![Page 38: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/38.jpg)
Edge detection and corners• Partial derivative estimates in x and y fail to
capture corners
Why do we care about corners?
![Page 39: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/39.jpg)
Case study: panorama stitching
[Brown, Szeliski, and Winder, CVPR 2005]
![Page 40: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/40.jpg)
How do we build panorama?
• We need to match (align) images
[Slide credit: Darya Frolova and Denis Simakov]
![Page 41: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/41.jpg)
Matching with Features• Detect feature points in both images
![Page 42: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/42.jpg)
Matching with Features• Detect feature points in both images
• Find corresponding pairs
![Page 43: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/43.jpg)
Matching with Features• Detect feature points in both images
• Find corresponding pairs
• Use these pairs to align images
![Page 44: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/44.jpg)
Matching with Features
• Problem 1:– Detect the same point independently in both
images
no chance to match!
We need a repeatable detector
![Page 45: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/45.jpg)
Matching with Features
• (Problem 2:– For each point correctly recognize the
corresponding one)
?
We need a reliable and distinctive descriptor
More on this aspect later!
![Page 46: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/46.jpg)
Corner detection as an interest operator
• We should easily recognize the point by looking through a small window
• Shifting a window in any direction should give a large change in intensity
![Page 47: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/47.jpg)
“flat” region:no change in all directions
“edge”:no change along the edge direction
“corner”:significant change in all directions
C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988
Corner detection as an interest operator
![Page 48: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/48.jpg)
Corner Detection
2
2,
( , ) x x y
x y x y y
I I IM w x y
I I I⎡ ⎤
= ⎢ ⎥⎢ ⎥⎣ ⎦
∑
M is a 2×2 matrix computed from image derivatives:
Sum over image region – area we are checking for corner
Gradient with respect to x, times gradient with respect to y
![Page 49: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/49.jpg)
Corner Detection
Eigenvectors of M: encode edge directions
Eigenvalues of M: encode edge strength
λ1, λ2 – eigenvalues of M
direction of the slowest change
direction of the fastest change
λmax λmin
![Page 50: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/50.jpg)
Corner Detection
λ1
λ2
“Corner”λ1 and λ2 are large,λ1 ~ λ2;E increases in all directions
λ1 and λ2 are small;E is almost constant in all directions
“Edge”λ1 >> λ2
“Edge”λ2 >> λ1
“Flat”region
Classification of image points using eigenvalues of M:
![Page 51: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/51.jpg)
Harris Corner Detector
Measure of corner response:
( )2det traceR M k M= −
1 2
1 2
dettrace
MM
λ λλ λ
== +
(k – empirical constant, k = 0.04-0.06)
Avoid computing eigenvaluesthemselves.
![Page 52: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/52.jpg)
Harris Corner Detector
λ1
λ2 “Corner”
“Edge”
“Edge”
“Flat”
• R depends only on eigenvalues of M
• R is large for a corner
• R is negative with large magnitude for an edge
• |R| is small for a flatregion
R > 0
R < 0
R < 0|R| small
![Page 53: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/53.jpg)
Harris Corner Detector
• The Algorithm:– Find points with large corner response
function R (R > threshold)– Take the points of local maxima of R
![Page 54: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/54.jpg)
Harris Detector: Workflow
![Page 55: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/55.jpg)
Harris Detector: WorkflowCompute corner response R
![Page 56: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/56.jpg)
Harris Detector: WorkflowFind points with large corner response: R>threshold
![Page 57: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/57.jpg)
Harris Detector: WorkflowTake only the points of local maxima of R
![Page 58: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/58.jpg)
Harris Detector: Workflow
![Page 59: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/59.jpg)
Harris Detector: Some Properties
• Rotation invariance
Ellipse rotates but its shape (i.e. eigenvalues) remains the same
Corner response R is invariant to image rotation
![Page 60: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/60.jpg)
Harris Detector: Some Properties
• Not invariant to image scale!
All points will be classified as edges
Corner !
More on interest operators/descriptors with invariance properties later.
![Page 61: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/61.jpg)
![Page 62: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/62.jpg)
This image is too big tofit on the screen. Howcan we reduce it?
How to generate a half-sized version?
![Page 63: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/63.jpg)
Image sub-sampling
Throw away every other row and column to create a 1/2 size image
- called image sub-sampling
1/4
1/8
Slide credit: S. Seitz
![Page 64: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/64.jpg)
Image sub-sampling
1/4 (2x zoom) 1/8 (4x zoom)1/2
![Page 65: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/65.jpg)
Sampling
• Continuous function discrete set of values
![Page 66: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/66.jpg)
Undersampling
• Information lost
Figure credit: S. Marschner
![Page 67: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/67.jpg)
Undersampling• Looks just like lower frequency signal!
![Page 68: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/68.jpg)
Undersampling• Looks like higher frequency signal!
Aliasing: higher frequency information can appear as lower frequency information
![Page 69: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/69.jpg)
Undersampling
Good sampling
Bad sampling
![Page 70: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/70.jpg)
Aliasing
![Page 71: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/71.jpg)
AliasingInput signal:
x = 0:.05:5; imagesc(sin((2.^x).*x))
Matlab output:
Not enough samples
![Page 72: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/72.jpg)
Aliasing in video
Slide credit: S. Seitz
![Page 73: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/73.jpg)
Image sub-sampling
1/4 (2x zoom) 1/8 (4x zoom)1/2
![Page 74: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/74.jpg)
How to prevent aliasing?• Sample more …• Smooth – suppress high frequencies
before sampling
![Page 75: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/75.jpg)
Gaussian pre-filtering
G 1/4
G 1/8
Gaussian 1/2
Solution: smooth the image, then subsample
![Page 76: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/76.jpg)
Subsampling with Gaussian pre-filtering
G 1/4 G 1/8Gaussian 1/2
Solution: smooth the image, then subsample
![Page 77: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/77.jpg)
Compare with...
1/4 (2x zoom) 1/8 (4x zoom)1/2
![Page 78: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/78.jpg)
Image pyramids
• Big bars (resp. spots, hands, etc.) and little bars are both interesting
• Inefficient to detect big bars with big filters• Alternative:
– Apply filters of fixed size to images of different sizes
![Page 79: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/79.jpg)
Image pyramids
• Known as a Gaussian Pyramid [Burt and Adelson, 1983]
![Page 80: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/80.jpg)
Gaussian image pyramids
![Page 81: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/81.jpg)
Forsyth & Ponce
![Page 82: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/82.jpg)
Image pyramids
• Useful for– Coarse to fine matching, iterative
computation; e.g. optical flow– Feature association across scales to
find reliable features– Searching over scale
![Page 83: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/83.jpg)
Image pyramids: multi-scale search
[Adelson et al., 1984]
![Page 84: Lecture 5: Edges, Corners, Sampling, PyramidsLecture 5: Edges, Corners, Sampling, Pyramids Thursday, Sept 13 Normalized cross correlation • Normalized correlation: normalize for](https://reader033.vdocuments.us/reader033/viewer/2022060915/60a8ee032d5a632e77378e5c/html5/thumbnails/84.jpg)
Image pyramids: multi-scale search
Learning algorithm
Figure from Rowley et al. 1998