interest points
DESCRIPTION
Interest points. CSE P 576 Larry Zitnick ( [email protected] ) Many slides courtesy of Steve Seitz. How can we find corresponding points?. Not always easy. NASA Mars Rover images. Answer below (look for tiny colored squares…). NASA Mars Rover images - PowerPoint PPT PresentationTRANSCRIPT
![Page 2: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/2.jpg)
How can we find corresponding points?
![Page 3: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/3.jpg)
Not always easy
NASA Mars Rover images
![Page 4: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/4.jpg)
NASA Mars Rover imageswith SIFT feature matchesFigure by Noah Snavely
Answer below (look for tiny colored squares…)
![Page 5: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/5.jpg)
Where do humans fixate?
Called “fixation points”, and a “saccade” is the process of moving between fixation points.
Slowly trace the outline of the above object.
![Page 6: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/6.jpg)
Where do humans fixate?
Called “fixation points”, and a “saccade” is the process of moving between fixation points.
Slowly trace the outline of the above object.
![Page 7: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/7.jpg)
Where do humans fixate?
Result of one subject “me”.
![Page 8: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/8.jpg)
Where do humans fixate?
"Eye Movements and Vision" by A. L. Yarbus; Plenum Press, New York; 1967
Top down or bottom up?
![Page 9: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/9.jpg)
Want uniquenessLook for image regions that are unusual
• Lead to unambiguous matches in other images
How to define “unusual”?
![Page 10: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/10.jpg)
Local measures of uniquenessSuppose we only consider a small window of pixels
• What defines whether a feature is a good or bad candidate?
Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.
![Page 11: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/11.jpg)
Feature detection
“flat” region:no change in all directions
“edge”: no change along the edge direction
“corner”:significant change in all directions
Local measure of feature uniqueness• How does the window change when you shift by a small amount?
Slide adapted from Darya Frolova, Denis Simakov, Weizmann Institute.
![Page 12: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/12.jpg)
Let’s look at the gradient distributions
![Page 13: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/13.jpg)
Principle Component AnalysisPrincipal component is the direction of highest variance.
How to compute PCA components:
1. Subtract off the mean for each data point.2. Compute the covariance matrix.3. Compute eigenvectors and eigenvalues.4. The components are the eigenvectors
ranked by the eigenvalues.
Next, highest component is the direction with highest variance orthogonal to the previous components.
Both eigenvalues are large!
![Page 14: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/14.jpg)
Simple example
Detect peaks and threshold to find corners.
![Page 15: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/15.jpg)
The mathTo compute the eigenvalues:
1. Compute the covariance matrix.
2. Compute eigenvalues.
Typically Gaussian weights
![Page 16: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/16.jpg)
The Harris operator- is a variant of the “Harris operator” for feature detection
• Very similar to - but less expensive (no square root)• Called the “Harris Corner Detector” or “Harris Operator”• Lots of other detectors, this is one of the most popular
Actually used in original paper:
![Page 17: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/17.jpg)
The Harris operator
Harris operator
![Page 18: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/18.jpg)
Harris detector example
![Page 19: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/19.jpg)
f value (red high, blue low)
![Page 20: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/20.jpg)
Threshold (f > value)
![Page 21: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/21.jpg)
Find local maxima of f
![Page 22: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/22.jpg)
Harris features (in red)
![Page 23: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/23.jpg)
How can we find correspondences?
![Page 24: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/24.jpg)
How can we find correspondences?
Similarity transform
![Page 25: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/25.jpg)
Scale and rotation?
Let’s look at scale first:
What is the “best” scale?
![Page 26: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/26.jpg)
Scale
The Laplacian has a maximal response when the difference between the “inside” and “outside” of the filter is greatest.
![Page 27: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/27.jpg)
Scale
Why Gaussian?
It is invariant to scale change, i.e., and has several other nice properties. Lindeberg, 1994
In practice, the Laplacian is approximated using a Difference of Gaussian (DoG).
![Page 28: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/28.jpg)
DoG example
σ = 1
σ = 66
![Page 29: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/29.jpg)
Scale
In practice the image is downsampled for larger sigmas.
Lowe, 2004.
![Page 30: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/30.jpg)
RotationHow to compute the rotation?
Create edge orientation histogram and find peak.
![Page 31: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/31.jpg)
Other interest point detectorsHarris Laplace
Scale & Affine Invariant Interest Point DetectorsK. MIKOLAJCZYK and C. SCHMID, IJCV 2004
![Page 32: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/32.jpg)
Other interest point detectorsAffine invariant
Scale & Affine Invariant Interest Point DetectorsK. MIKOLAJCZYK and C. SCHMID, IJCV 2004
![Page 33: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/33.jpg)
Computationally efficient
Approximate Gaussian filters using box filters:
SURF: Speeded Up Robust FeaturesHerbert Bay, Tinne Tuytelaars, and Luc Van Gool, ECCV 2006
![Page 34: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/34.jpg)
Computationally efficient Corner detection by sampling pixels based on decision tree
Machine learning for high-speed corner detectionEdward Rosten and Tom Drummond, ECCV 2006
![Page 35: Interest points](https://reader035.vdocuments.us/reader035/viewer/2022062812/56816356550346895dd400b4/html5/thumbnails/35.jpg)
How well do they work in practice?Let’s go to the videos…