locating and describing interest points computer vision cs 543 / ece 549 university of illinois...
TRANSCRIPT
![Page 1: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/1.jpg)
Locating and Describing Interest Points
Computer VisionCS 543 / ECE 549
University of Illinois
Derek Hoiem
02/10/15
Acknowledgment: Many keypoint slides from Grauman&Leibe 2008 AAAI Tutorial
![Page 2: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/2.jpg)
Laplacian Pyramid clarification
![Page 3: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/3.jpg)
This section: correspondence and alignment
• Correspondence: matching points, patches, edges, or regions across images
≈
![Page 4: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/4.jpg)
This section: correspondence and alignment
• Alignment: solving the transformation that makes two things match better
T
![Page 5: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/5.jpg)
A
Example: fitting an 2D shape template
Slide from Silvio Savarese
![Page 6: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/6.jpg)
Example: fitting a 3D object model
Slide from Silvio Savarese
![Page 7: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/7.jpg)
Example: estimating “fundamental matrix” that corresponds two views
Slide from Silvio Savarese
![Page 8: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/8.jpg)
Example: tracking points
frame 0 frame 22 frame 49
x xx
Your problem 1 for HW 2!
![Page 9: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/9.jpg)
HW 2• Interest point detection and tracking
– Detect trackable points– Track them across 50 frames– In HW 3, you will use these tracked points for
structure from motion
frame 0 frame 22 frame 49
![Page 10: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/10.jpg)
HW 2• Alignment of object edge
images– Compute a transformation that
aligns two edge maps
![Page 11: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/11.jpg)
HW 2• Initial steps of object alignment
– Derive basic equations for interest-point based alignment
![Page 12: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/12.jpg)
This class: interest points• Note: “interest points” = “keypoints”, also
sometimes called “features”
• Many applications– tracking: which points are good to track?– recognition: find patches likely to tell us
something about object category– 3D reconstruction: find correspondences
across different views
![Page 13: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/13.jpg)
Human eye movements
Yarbus eye tracking
![Page 14: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/14.jpg)
Human eye movements
Study by Yarbus
Change blindness: http://www.simonslab.com/videos.html
![Page 15: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/15.jpg)
This class: interest points• Suppose you have to
click on some point, go away and come back after I deform the image, and click on the same points again. – Which points would
you choose?
original
deformed
![Page 16: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/16.jpg)
Overview of Keypoint Matching
K. Grauman, B. Leibe
AfBf
B1
B2
B3A1
A2 A3
Tffd BA ),(
1. Find a set of distinctive key- points
3. Extract and normalize the region content
2. Define a region around each keypoint
4. Compute a local descriptor from the normalized region
5. Match local descriptors
![Page 17: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/17.jpg)
Goals for Keypoints
Detect points that are repeatable and distinctive
![Page 18: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/18.jpg)
Key trade-offs
More Repeatable More Points
B1
B2
B3A1
A2 A3
Detection
More Distinctive More Flexible
Description
Robust to occlusionWorks with less texture
Minimize wrong matches Robust to expected variationsMaximize correct matches
Robust detectionPrecise localization
![Page 19: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/19.jpg)
Choosing interest points
Where would you tell your friend to meet you?
![Page 20: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/20.jpg)
Choosing interest points
Where would you tell your friend to meet you?
![Page 21: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/21.jpg)
Many Existing Detectors Available
K. Grauman, B. Leibe
Hessian & Harris [Beaudet ‘78], [Harris ‘88]Laplacian, DoG [Lindeberg ‘98], [Lowe 1999]Harris-/Hessian-Laplace [Mikolajczyk & Schmid ‘01]Harris-/Hessian-Affine[Mikolajczyk & Schmid ‘04]EBR and IBR [Tuytelaars & Van Gool ‘04] MSER [Matas ‘02]Salient Regions [Kadir & Brady ‘01] Others…
![Page 22: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/22.jpg)
Harris Detector [Harris88]
• Second moment matrix
K. Grauman, B. Leibe
)()(
)()()(),( 2
2
DyDyx
DyxDxIDI III
IIIg
Intuition: Search for local neighborhoods where the image content has two main directions (eigenvectors).
![Page 23: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/23.jpg)
Harris Detector [Harris88]
• Second moment matrix
)()(
)()()(),( 2
2
DyDyx
DyxDxIDI III
IIIg
26
1. Image derivatives
2. Square of derivatives
3. Gaussian filter g(sI)
Ix Iy
Ix2 Iy2 IxIy
g(Ix2) g(Iy2)g(IxIy)
222222 )]()([)]([)()( yxyxyx IgIgIIgIgIg
])),([trace()],(det[ 2DIDIhar
4. Cornerness function – both eigenvalues are strong
har5. Non-maxima suppression
1 2
1 2
det
trace
M
M
(optionally, blur first)
![Page 24: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/24.jpg)
Harris Detector: Mathematics
1. Want large eigenvalues, and small ratio
2. We know
3. Leads to
1 2
1 2
det
trace
M
M
(k :empirical constant, k = 0.04-0.06)
)()(
)()()( 2
2
DyDyx
DyxDxI III
IIIgM
t2
1
tMkM )(tracedet 2
Nice brief derivation on wikipedia
![Page 25: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/25.jpg)
Explanation of Harris Criterion
From Grauman and Leibe
![Page 26: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/26.jpg)
Harris Detector – Responses [Harris88]
Effect: A very precise corner detector.
![Page 27: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/27.jpg)
Harris Detector - Responses [Harris88]
![Page 28: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/28.jpg)
Hessian Detector [Beaudet78]
• Hessian determinant
K. Grauman, B. Leibe
yyxy
xyxx
II
IIIHessian )(
Ixx
IyyIxy
Intuition: Search for strongcurvature in two orthogonal directions
![Page 29: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/29.jpg)
Hessian Detector [Beaudet78]
• Hessian determinant
K. Grauman, B. Leibe
Ixx
IyyIxy
)()()())(det( 2 xIxIxIxHessian xyyyxx
2)^(. xyyyxx III In Matlab:
),(),(
),(),(),(
xIxI
xIxIxHessian
yyxy
xyxx
1 2
1 2
det
trace
M
M
Find maxima of determinant
![Page 30: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/30.jpg)
Hessian Detector – Responses [Beaudet78]
Effect: Responses mainly on corners and strongly textured areas.
![Page 31: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/31.jpg)
Hessian Detector – Responses [Beaudet78]
![Page 32: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/32.jpg)
So far: can localize in x-y, but not scale
![Page 33: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/33.jpg)
Automatic Scale Selection
K. Grauman, B. Leibe
)),(( )),((11
xIfxIfmm iiii
How to find corresponding patch sizes?
![Page 34: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/34.jpg)
Automatic Scale Selection• Function responses for increasing scale (scale signature)
K. Grauman, B. Leibe)),((
1xIf
mii )),((1
xIfmii
![Page 35: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/35.jpg)
Automatic Scale Selection• Function responses for increasing scale (scale signature)
K. Grauman, B. Leibe)),((
1xIf
mii )),((1
xIfmii
![Page 36: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/36.jpg)
Automatic Scale Selection• Function responses for increasing scale (scale signature)
K. Grauman, B. Leibe)),((
1xIf
mii )),((1
xIfmii
![Page 37: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/37.jpg)
Automatic Scale Selection• Function responses for increasing scale (scale signature)
K. Grauman, B. Leibe)),((
1xIf
mii )),((1
xIfmii
![Page 38: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/38.jpg)
Automatic Scale Selection• Function responses for increasing scale (scale signature)
K. Grauman, B. Leibe)),((
1xIf
mii )),((1
xIfmii
![Page 39: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/39.jpg)
Automatic Scale Selection• Function responses for increasing scale (scale signature)
K. Grauman, B. Leibe)),((
1xIf
mii )),((1
xIfmii
![Page 40: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/40.jpg)
What Is A Useful Signature Function?• Difference-of-Gaussian = “blob” detector
K. Grauman, B. Leibe
![Page 41: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/41.jpg)
Difference-of-Gaussian (DoG)
K. Grauman, B. Leibe
- =
![Page 42: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/42.jpg)
DoG – Efficient Computation• Computation in Gaussian scale pyramid
K. Grauman, B. Leibe
s
Original image 4
1
2
Sampling withstep s4 =2
s
s
s
![Page 43: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/43.jpg)
Find local maxima in position-scale space of Difference-of-Gaussian
K. Grauman, B. Leibe
)()( yyxx LL
s
s2
s3
s4
s5
List of (x, y, s)
![Page 44: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/44.jpg)
Results: Difference-of-Gaussian
K. Grauman, B. Leibe
![Page 45: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/45.jpg)
T. Tuytelaars, B. Leibe
Orientation Normalization
• Compute orientation histogram• Select dominant orientation• Normalize: rotate to fixed orientation
0 2p
[Lowe, SIFT, 1999]
![Page 46: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/46.jpg)
Maximally Stable Extremal Regions [Matas ‘02]
• Based on Watershed segmentation algorithm• Select regions that stay stable over a large
parameter range
K. Grauman, B. Leibe
![Page 47: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/47.jpg)
Example Results: MSER
52 K. Grauman, B. Leibe
![Page 48: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/48.jpg)
Available at a web site near you…• For most local feature detectors, executables
are available online:– http://www.robots.ox.ac.uk/~vgg/research/affine– http://www.cs.ubc.ca/~lowe/keypoints/– http://www.vision.ee.ethz.ch/~surf
K. Grauman, B. Leibe
![Page 49: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/49.jpg)
Local Descriptors• The ideal descriptor should be
– Robust– Distinctive– Compact– Efficient
• Most available descriptors focus on edge/gradient information– Capture texture information– Color rarely used
K. Grauman, B. Leibe
![Page 50: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/50.jpg)
Local Descriptors: SIFT Descriptor
[Lowe, ICCV 1999]
Histogram of oriented gradients
• Captures important texture information
• Robust to small translations / affine deformationsK. Grauman, B. Leibe
![Page 51: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/51.jpg)
Details of Lowe’s SIFT algorithm• Run DoG detector
– Find maxima in location/scale space– Remove edge points
• Find all major orientations– Bin orientations into 36 bin histogram
• Weight by gradient magnitude• Weight by distance to center (Gaussian-weighted mean)
– Return orientations within 0.8 of peak• Use parabola for better orientation fit
• For each (x,y,scale,orientation), create descriptor:– Sample 16x16 gradient mag. and rel. orientation– Bin 4x4 samples into 4x4 histograms– Threshold values to max of 0.2, divide by L2 norm– Final descriptor: 4x4x8 normalized histograms
Lowe IJCV 2004
![Page 52: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/52.jpg)
Matching SIFT Descriptors• Nearest neighbor (Euclidean distance)• Threshold ratio of nearest to 2nd nearest descriptor
Lowe IJCV 2004
![Page 53: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/53.jpg)
SIFT Repeatability
Lowe IJCV 2004
![Page 54: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/54.jpg)
SIFT Repeatability
![Page 55: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/55.jpg)
SIFT Repeatability
Lowe IJCV 2004
![Page 56: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/56.jpg)
Local Descriptors: SURF
K. Grauman, B. Leibe
• Fast approximation of SIFT idea Efficient computation by 2D box filters &
integral images 6 times faster than SIFT
Equivalent quality for object identification
[Bay, ECCV’06], [Cornelis, CVGPU’08]
• GPU implementation available Feature extraction @ 200Hz
(detector + descriptor, 640×480 img) http://www.vision.ee.ethz.ch/~surf
Many other efficient descriptors are also available
![Page 57: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/57.jpg)
Local Descriptors: Shape Context
Count the number of points inside each bin, e.g.:
Count = 4
Count = 10...
Log-polar binning: more precision for nearby points, more flexibility for farther points.
Belongie & Malik, ICCV 2001K. Grauman, B. Leibe
![Page 58: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/58.jpg)
Local Descriptors: Geometric Blur
Example descriptor
~
Compute edges at four orientations
Extract a patchin each channel
Apply spatially varyingblur and sub-sample
(Idealized signal)
Berg & Malik, CVPR 2001K. Grauman, B. Leibe
![Page 59: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/59.jpg)
Choosing a detector
• What do you want it for?– Precise localization in x-y: Harris– Good localization in scale: Difference of Gaussian– Flexible region shape: MSER
• Best choice often application dependent– Harris-/Hessian-Laplace/DoG work well for many natural categories– MSER works well for buildings and printed things
• Why choose?– Get more points with more detectors
• There have been extensive evaluations/comparisons– [Mikolajczyk et al., IJCV’05, PAMI’05]– All detectors/descriptors shown here work well
![Page 60: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/60.jpg)
Comparison of Keypoint Detectors
Tuytelaars Mikolajczyk 2008
![Page 61: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/61.jpg)
Choosing a descriptor
• Again, need not stick to one
• For object instance recognition or stitching, SIFT or variant is a good choice
![Page 62: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/62.jpg)
Things to remember
• Keypoint detection: repeatable and distinctive– Corners, blobs, stable regions– Harris, DoG
• Descriptors: robust and selective– spatial histograms of orientation– SIFT
![Page 63: Locating and Describing Interest Points Computer Vision CS 543 / ECE 549 University of Illinois Derek Hoiem 02/10/15 Acknowledgment: Many keypoint slides](https://reader036.vdocuments.us/reader036/viewer/2022062518/56649d345503460f94a0a537/html5/thumbnails/63.jpg)
Next time
• Feature tracking