Interest Point Detection
Lecture-4
Local features: main components1) Detection: Identify the
interest points
2) Description :Extract feature vector descriptor surrounding each interest point.
3) Matching: Determine correspondence between descriptors in two views
],,[ )1()1(11 dxx x
],,[ )2()2(12 dxx x
Kristen Grauman
Where can we use it? Automate object tracking Point matching for computing disparity Stereo calibration
Estimation of fundamental matrix Motion based segmentation Recognition 3D object reconstruction Robot navigation Image retrieval and indexing
Goal: interest operator repeatability
• We want to detect (at least some of) the same points in both images.
• Yet we have to be able to run the detection procedure independently per image.
No chance to find true matches!
Kristen Grauman
Goal: descriptor distinctiveness
• We want to be able to reliably determine which point goes with which.
• Must provide some invariance to geometricand photometric differences between the two views.
?
Kristen Grauman
Some patches can be localizedor matched with higher accuracy than
others.
Local features: main components1) Detection: Identify the
interest points
2) Description:Extract vector feature descriptor surrounding each interest point.
3) Matching: Determine correspondence between descriptors in two views
Kristen Grauman
What is an interest point Expressive texture
The point at which the direction of the boundary of object changes abruptly
Intersection point between two or more edge segments
Synthetic & Real Interest Points
Corners are indicated in red
Properties of Interest Point Detectors Detect all (or most) true interest points No false interest points Well localized. Robust with respect to noise. Efficient detection
Possible Approaches to Corner Detection Based on brightness of images
Usually image derivatives Based on boundary extraction
First step edge detection Curvature analysis of edges
Harris Corner Detector Corner point can be recognized in a window Shifting a window in any direction should give
a large change in intensity
C.Harris, M.Stephens. “A Combined Corner and Edge Detector”. 1988
Basic Idea
“flat” region:no change in all directions
“edge”:no change along the edge direction
“corner”:significant change in all directions
Aperture Problem
Correlation
k lljkihlkfhf ,,
KernelImage
hf
h1 h2 h3
h4 h5 h6
h7 h8 h9
hf1 f2 f3
f4 f5 f6
f7 f8 f9
998877
665544
332211
*
hfhfhfhfhfhf
hfhfhfhf
f
Correlation
k lljkihlkfhf ,, Cross correlation
k l
ljkiflkfff ,, Auto correlation
Correlation Vs SSD 2,,
k l
ljkihlkfSSD Sum of Squares Difference
k l
ljkiflkfff ,,
k l
ljkihlkfljkihlkfSSD 22 ,,,2,
minimize
minimize
maximize
k l
lkfljkihSSD ,,2
k l
lkfljkihnCorrelatio ,,maximize
k l
lkfljkihSSD ,,2
Mathematics of Harris Detector Change of intensity for the shift (u,v)
yx
yxIvyuxIyxwvuE,
2
intensityintensity shiftedfunctionwindow
]),(),([),(),(
Window functions
Auto-correlation
UNIFORM GAUSSIAN
Auto-Correlation
Taylor Series
ƒ(x) Can be represented at point a in terms of its derivatives
Mathematics of Harris Detector
yx
yx vIuIyxwvuE,
2])[,(),(
yx y
x
II
vuyxwvuE,
2
),(),(
yxyx
y
x
vu
IIII
vuyxwvuE,
),(),(
v
uII
II
yxwvuvuEyx
yxy
x
,),(),(
yx
yxIvyuxIyxwvuE,
2
intensityintensity shiftedfunctionwindow
]),(),([),(),(
yx
yx yxIvIuIyxIyxwvuE,
2
intensityintensity shiftedfunction window
]),(),([),(),(
Taylor Series
vu
MvuvuE ),(
Mathematics of Harris Detector
E(u,v) is an equation of an ellipse, where M is the covariance
Let 1 and 2 be eigenvalues of M
vu
MvuvuE ),(
yx yyyx
yxxx
IIIIIIII
yxwM,
),(
direction of the slowest change
direction of the fastest change
(max)-1/2
(min)-1/2
Eigen Vectors and Eigen ValuesThe eigen vector, x, of a matrix A is a special vector, with the following property
Where is called eigen value
To find eigen values of a matrix A first find the roots of:
Then solve the following linear system for each eigen value to find corresponding eigen vector
Example
Eigen Values
Eigen Vectors
Eigen Values
Eigen Vectors
Mathematics of Harris Detector
1
2
“Corner”1 and 2 are large,1 ~ 2;E increases in all directions
1 and 2 are small;E is almost constant in all directions
“Edge” 1 >> 2
“Edge” 2 >> 1
“Flat” region
Classification of image points using eigenvalues of M:
Mathematics of Harris Detector Measure of cornerness in terms of 1, 2
2det traceMkMR 22121 kR
2
1
00
M
Mathematics of Harris Detector
1
2 “Corner”
“Edge”
“Edge”
“Flat”
• R depends only on eigenvalues of M
• R is large for a corner
• R is negative with large magnitude for an edge
• |R| is small for a flatregion
R > 0
R < 0
R < 0|R| small
Compute corner response
Find points with large corner response: R> thresholdFind points with large corner response: R> threshold
Take only the points of local maxima of R
If pixel value is greater than its neighbors then it is a local maxima.
Other Version of Harris Detectors
21 R
21
21
1)()det(
Mtrace
MR
Triggs
Szeliski (Harmonic mean)
1R Shi-Tomasi
1
2
Algorithm
Reading Material Section 4.1.1 Feature Detectors
Richard Szeliski, "Computer Vision: Algorithms and Application“, Springer.