lecture: ransac and feature...
TRANSCRIPT
![Page 1: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/1.jpg)
Lecture 6 -Stanford University
Lecture:RANSACandfeaturedetectors
JuanCarlosNiebles andRanjayKrishnaStanfordVisionandLearningLab
12-Oct-171
![Page 2: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/2.jpg)
Lecture 6 -Stanford University
Whatwewilllearntoday?
• Amodelfittingmethodforedgedetection– RANSAC
• Localinvariantfeatures– Motivation– Requirements,invariances
• Keypoint localization– Harriscornerdetector
12-Oct-172
![Page 3: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/3.jpg)
Lecture 6 -Stanford University
Whatwewilllearntoday
• Amodelfittingmethodforlinedetection– RANSAC
• Localinvariantfeatures– Motivation– Requirements,invariances
• Keypoint localization– Harriscornerdetector
12-Oct-173
![Page 4: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/4.jpg)
Lecture 6 -Stanford University
FittingasSearchinParametricSpace
• Chooseaparametricmodeltorepresentasetoffeatures
• Membershipcriterionisnotlocal– Can’ttellwhetherapointbelongstoagivenmodeljustbylookingatthatpoint.
• Threemainquestions:– Whatmodelrepresentsthissetoffeaturesbest?– Whichofseveralmodelinstancesgetswhichfeature?– Howmanymodelinstancesarethere?
• Computationalcomplexityisimportant– Itisinfeasibletoexamineeverypossiblesetofparametersandeverypossiblecombinationoffeatures
Source:L.Lazebnik
12-Oct-174
![Page 5: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/5.jpg)
Lecture 6 -Stanford University
Example:LineFitting• Whyfitlines?Manyobjectscharacterizedbypresenceofstraightlines
• Wait,whyaren’twedonejustbyrunningedgedetection?Slide credit: Kristen Grauman
12-Oct-175
![Page 6: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/6.jpg)
Lecture 6 -Stanford University
DifficultyofLineFitting• Extraedgepoints(clutter),multiplemodels:– Whichpointsgowithwhichline,ifany?
• Onlysomepartsofeachlinedetected,andsomepartsaremissing:– Howtofindalinethatbridgesmissingevidence?
• Noiseinmeasurededgepoints,orientations:– Howtodetecttrueunderlyingparameters?
Slide credit: Kristen Grauman
12-Oct-176
![Page 7: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/7.jpg)
Lecture 6 -Stanford University
Voting
• It’snotfeasibletocheckallcombinationsoffeaturesbyfittingamodeltoeachpossiblesubset.
• Votingisageneraltechniquewhereweletthefeaturesvoteforallmodelsthatarecompatiblewithit.– Cyclethroughfeatures,castvotesformodelparameters.– Lookformodelparametersthatreceivealotofvotes.
• Noise&clutterfeatureswillcastvotestoo,but typicallytheirvotesshouldbeinconsistentwiththemajorityof“good”features.
• Okifsomefeaturesnotobserved,asmodelcanspanmultiplefragments.
Slide credit: Kristen Grauman
12-Oct-177
![Page 8: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/8.jpg)
Lecture 6 -Stanford University
RANSAC[Fischler &Bolles 1981]
• RANdom SAmple Consensus
• Approach:wewanttoavoidtheimpactofoutliers,solet’slookfor“inliers”,anduseonlythose.
• Intuition:ifanoutlierischosentocomputethecurrentfit,thentheresultinglinewon’thavemuchsupportfromrestofthepoints.
Slid
e cr
edit
: Kr
iste
n G
raum
an
12-Oct-178
![Page 9: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/9.jpg)
Lecture 6 -Stanford University
RANSACloop:1. Randomlyselectaseedgroup ofpointsonwhichtobase
transformationestimate(e.g.,agroupofmatches)2. Computetransformationfromseedgroup3. Findinlierstothistransformation4. Ifthenumberofinliersissufficientlylarge,re-compute
least-squaresestimateoftransformationonalloftheinliers
• Keepthetransformationwiththelargestnumberofinliers
Slid
e cr
edit
: Kr
iste
n G
raum
an
12-Oct-179
RANSAC[Fischler &Bolles 1981]
![Page 10: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/10.jpg)
Lecture 6 -Stanford University
RANSACLineFittingExample
• Task:Estimatethebestline– Howmanypointsdoweneedtoestimatetheline?
Slid
e cr
edit
: Ji
nxia
ngCh
ai
12-Oct-1710
![Page 11: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/11.jpg)
Lecture 6 -Stanford University
RANSACLineFittingExample
• Task:Estimatethebestline
Sampletwopoints
Slid
e cr
edit
: Ji
nxia
ngCh
ai
12-Oct-1711
![Page 12: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/12.jpg)
Lecture 6 -Stanford University
RANSACLineFittingExample
• Task:Estimatethebestline
Fitalinetothem
Slid
e cr
edit
: Ji
nxia
ngCh
ai
12-Oct-1712
![Page 13: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/13.jpg)
Lecture 6 -Stanford University
RANSACLineFittingExample
• Task:Estimatethebestline
Total number of points within a threshold of line. Sl
ide
cred
it:
Jinx
iang
Chai
12-Oct-1713
![Page 14: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/14.jpg)
Lecture 6 -Stanford University
RANSACLineFittingExample
• Task:Estimatethebestline
Total number of points within a threshold of line.
“7 inlier points”
Slid
e cr
edit
: Ji
nxia
ngCh
ai
12-Oct-1714
![Page 15: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/15.jpg)
Lecture 6 -Stanford University
RANSACLineFittingExample
• Task:Estimatethebestline
Repeat,untilwegetagoodresult.
Slid
e cr
edit
: Ji
nxia
ngCh
ai
12-Oct-1715
![Page 16: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/16.jpg)
Lecture 6 -Stanford University
RANSACLineFittingExample
• Task:Estimatethebestline
Repeat,untilwegetagoodresult.
“11 inlier points”
Slid
e cr
edit
: Ji
nxia
ngCh
ai
12-Oct-1716
![Page 17: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/17.jpg)
Lecture 6 -Stanford University 12-Oct-1717
![Page 18: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/18.jpg)
Lecture 6 -Stanford University
RANSAC:Howmanysamples?• Howmanysamplesareneeded?
– Supposew isfractionofinliers(pointsfromline).– n pointsneededtodefinehypothesis(2forlines)– k sampleschosen.
• Prob.thatasinglesampleofn pointsiscorrect:
• Prob.thatallk samplesfailis:
Þ Choosek highenoughtokeepthisbelowdesiredfailurerate.
nwknw )1( -
Slide credit: David Lowe
12-Oct-1718
![Page 19: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/19.jpg)
Lecture 6 -Stanford University
RANSAC:Computedk(p=0.99)Sample
size
n
Proportion of outliers
5% 10% 20% 25% 30% 40% 50%
2 2 3 5 6 7 11 173 3 4 7 9 11 19 354 3 5 9 13 17 34 725 4 6 12 17 26 57 1466 4 7 16 24 37 97 2937 4 8 20 33 54 163 5888 5 9 26 44 78 272 1177
Slide credit: David Lowe
12-Oct-1719
![Page 20: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/20.jpg)
Lecture 6 -Stanford University
AfterRANSAC• RANSACdividesdataintoinliersandoutliersandyieldsestimatecomputedfromminimalsetofinliers.
• Improvethisinitialestimatewithestimationoverallinliers(e.g.withstandardleast-squaresminimization).
• Butthismaychangeinliers,soalternatefittingwithre-classificationasinlier/outlier.
Slid
e cr
edit
: D
avid
Low
e
12-Oct-1720
![Page 21: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/21.jpg)
Lecture 6 -Stanford University
RANSAC:ProsandCons
• Pros:– Generalmethodsuitedforawiderangeofmodelfittingproblems
– Easytoimplementandeasytocalculateitsfailurerate• Cons:
– Onlyhandlesamoderatepercentageofoutlierswithoutcostblowingup
– Manyrealproblemshavehighrateofoutliers(butsometimesselectivechoiceofrandomsubsetscanhelp)
• Avotingstrategy,TheHoughtransform,canhandlehighpercentageofoutliers
12-Oct-1721
![Page 22: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/22.jpg)
Lecture 6 -Stanford University
Whatwewilllearntoday?
• Amodelfittingmethodforedgedetection– RANSAC
• Localinvariantfeatures– Motivation– Requirements,invariances
• Keypoint localization– Harriscornerdetector
12-Oct-1722
Somebackgroundreading:RickSzeliski,Chapter4.1.1;DavidLowe,IJCV2004
![Page 23: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/23.jpg)
Lecture 6 -Stanford University
Imagematching:achallengingproblem
12-Oct-1723
![Page 24: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/24.jpg)
Lecture 6 -Stanford University
by Diva Sian
by swashford
Slid
e cr
edit
: St
eve
Seit
z
12-Oct-1724
Imagematching:achallengingproblem
![Page 25: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/25.jpg)
Lecture 6 -Stanford University
HarderCase
by Diva Sian by scgbt
Slid
e cr
edit
: St
eve
Seit
z
12-Oct-1725
![Page 26: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/26.jpg)
Lecture 6 -Stanford University
HarderStill?
NASA Mars Rover images
Slid
e cr
edit
: St
eve
Seit
z
12-Oct-1726
![Page 27: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/27.jpg)
Lecture 6 -Stanford University
AnswerBelow(Lookfortinycoloredsquares)
NASAMarsRoverimageswithSIFTfeaturematches(FigurebyNoahSnavely) Sl
ide
cred
it:
Stev
e Se
itz
12-Oct-1727
![Page 28: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/28.jpg)
Lecture 6 -Stanford University
Motivationforusinglocalfeatures• Globalrepresentationshavemajorlimitations• Instead,describeandmatchonlylocalregions• Increasedrobustnessto
– Occlusions
– Articulation
– Intra-categoryvariations
θqφ
dq
φθ
d
12-Oct-1728
![Page 29: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/29.jpg)
Lecture 6 -Stanford University
GeneralApproachN
pixe
ls
N pixels
Similarity measureAf
e.g. color
Bf
e.g. color
A1
A2 A3
Tffd BA <),(
1. Find a set of distinctive key-points
3. Extract and normalize the region content
2. Define a region around each keypoint
4. Compute a local descriptor from the normalized region
5. Match local descriptors
Slid
e cr
edit
: Ba
stia
n Le
ibe
12-Oct-1729
![Page 30: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/30.jpg)
Lecture 6 -Stanford University
CommonRequirements• Problem1:
– Detectthesamepointindependently inbothimages
No chance to match!
We need a repeatable detector! Slid
e cr
edit
: D
arya
Fro
lova
, D
enis
Sim
akov
12-Oct-1730
![Page 31: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/31.jpg)
Lecture 6 -Stanford University
CommonRequirements• Problem1:
– Detectthesamepointindependently inbothimages
• Problem2:– Foreachpointcorrectlyrecognizethecorrespondingone
We need a reliable and distinctive descriptor!
?
Slid
e cr
edit
: D
arya
Fro
lova
, D
enis
Sim
akov
12-Oct-1731
![Page 32: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/32.jpg)
Lecture 6 -Stanford University
Invariance:GeometricTransformations
Slid
e cr
edit
: St
eve
Seit
z
12-Oct-1732
![Page 33: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/33.jpg)
Lecture 6 -Stanford University
LevelsofGeometricInvariance
Slid
e cr
edit
: Ba
stia
n Le
ibe
12-Oct-1733
CS131 CS231a
![Page 34: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/34.jpg)
Lecture 6 -Stanford University
Invariance:PhotometricTransformations
• Oftenmodeledasalineartransformation:– Scaling+Offset
Slid
e cr
edit
: Ti
nne
Tuyt
elaa
rs
12-Oct-1734
![Page 35: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/35.jpg)
Lecture 6 -Stanford University
Requirements• Regionextractionneedstoberepeatable andaccurate
– Invarianttotranslation,rotation,scalechanges– Robust or covarianttoout-of-plane(»affine)transformations– Robust tolightingvariations,noise,blur,quantization
• Locality:Featuresarelocal,thereforerobusttoocclusionandclutter.
• Quantity:Weneedasufficientnumberofregionstocovertheobject.
• Distinctiveness:Theregionsshouldcontain“interesting”structure.
• Efficiency:Closetoreal-timeperformance.
Slid
e cr
edit
: Ba
stia
n Le
ibe
12-Oct-1735
![Page 36: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/36.jpg)
Lecture 6 -Stanford University
ManyExistingDetectorsAvailable
• Hessian&Harris [Beaudet ‘78],[Harris‘88]
• Laplacian,DoG [Lindeberg ‘98],[Lowe‘99]
• Harris-/Hessian-Laplace [Mikolajczyk &Schmid ‘01]
• Harris-/Hessian-Affine [Mikolajczyk &Schmid ‘04]
• EBRandIBR [Tuytelaars &VanGool ‘04]
• MSER [Matas ‘02]
• SalientRegions [Kadir &Brady‘01]
• Others…
• ThosedetectorshavebecomeabasicbuildingblockformanyrecentapplicationsinComputerVision.
Slid
e cr
edit
: Ba
stia
n Le
ibe
12-Oct-1736
![Page 37: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/37.jpg)
Lecture 6 -Stanford University
Whatwewilllearntoday?
• Amodelfittingmethodforedgedetection– RANSAC
• Localinvariantfeatures– Motivation– Requirements,invariances
• Keypoint localization– Harriscornerdetector
12-Oct-1737
Somebackgroundreading:RickSzeliski,Chapter4.1.1;DavidLowe,IJCV2004
![Page 38: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/38.jpg)
Lecture 6 -Stanford University
Keypoint Localization
• Goals:– Repeatabledetection– Preciselocalization– InterestingcontentÞ Lookfortwo-dimensionalsignalchanges
Slid
e cr
edit
: Ba
stia
n Le
ibe
12-Oct-1738
![Page 39: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/39.jpg)
Lecture 6 -Stanford University
FindingCorners
• Keyproperty:– Intheregionaroundacorner,imagegradienthastwoormoredominantdirections
• Cornersarerepeatable anddistinctive
C.Harris and M.Stephens. "A Combined Corner and Edge Detector.“Proceedings of the 4th Alvey Vision Conference, 1988.
Slid
e cr
edit
: Sv
etla
na L
azeb
nik
12-Oct-1739
![Page 40: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/40.jpg)
Lecture 6 -Stanford University
CornersasDistinctiveInterestPoints• Designcriteria
– Weshouldeasilyrecognizethepointbylookingthroughasmallwindow(locality)
– Shiftingthewindowinany direction shouldgivealargechange inintensity(goodlocalization)
“edge”:no change along the edge direction
“corner”:significant change in all directions
“flat” region:no change in all directions Sl
ide
cred
it:
Alyo
sha
Efro
s
12-Oct-1740
![Page 41: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/41.jpg)
Lecture 6 -Stanford University
Cornersversusedges
12-Oct-1741
Large
Large
Small
Large
Small
Small
Corner
Edge
Nothing
![Page 42: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/42.jpg)
Lecture 6 -Stanford University
Cornersversusedges
12-Oct-1742
??
??Corner
![Page 43: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/43.jpg)
Lecture 6 -Stanford University
HarrisDetectorFormulation
• Changeofintensityfortheshift[u,v]:
E(u,v) = w(x, y) I (x +u, y + v)− I (x, y)"# $%2
x ,y∑
IntensityShifted intensity
Window function
orWindow function w(x,y) =
Gaussian1 in window, 0 outside
Slid
e cr
edit
: Ri
ck S
zelis
ki
12-Oct-1743
![Page 44: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/44.jpg)
Lecture 6 -Stanford University
HarrisDetectorFormulation• Thismeasureofchangecanbeapproximatedby:
whereM isa2´2matrixcomputedfromimagederivatives:
úû
ùêë
é»
vu
MvuvuE ][),(
M
Sumoverimageregion– theareawearecheckingforcorner
Gradient with respect to x, times gradient with respect to y
2
2,( , ) x x y
x y x y y
I I IM w x y
I I Ié ù
= ê úê úë û
å
Slid
e cr
edit
: Ri
ck S
zelis
ki
12-Oct-1744
![Page 45: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/45.jpg)
Lecture 6 -Stanford University
HarrisDetectorFormulation
IxImage I IxIyIy
whereM isa2´2matrixcomputedfromimagederivatives:
M
Sumoverimageregion– theareawearecheckingforcorner
Gradient with respect to x, times gradient with respect to y
2
2,( , ) x x y
x y x y y
I I IM w x y
I I Ié ù
= ê úê úë û
å
Slid
e cr
edit
: Ri
ck S
zelis
ki
12-Oct-1745
![Page 46: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/46.jpg)
Lecture 6 -Stanford University
WhatDoesThisMatrixReveal?• First,let’sconsideranaxis-alignedcorner:
• Thismeans:– Dominantgradientdirectionsalignwithx ory axis– Ifeitherλ iscloseto0,thenthisisnotacorner,solookforlocationswherebotharelarge.
• Whatifwehaveacornerthatisnotalignedwiththeimageaxes?
úû
ùêë
é=
úúû
ù
êêë
é=
åååå
2
12
2
00l
l
yyx
yxx
IIIIII
M
Slid
e cr
edit
: D
avid
Jac
obs
12-Oct-1746
![Page 47: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/47.jpg)
Lecture 6 -Stanford University
WhatDoesThisMatrixReveal?• First,let’sconsideranaxis-alignedcorner:
• Thismeans:– Dominantgradientdirectionsalignwithx ory axis– Ifeitherλ iscloseto0,thenthisisnotacorner,solookforlocationswherebotharelarge.
• Whatifwehaveacornerthatisnotalignedwiththeimageaxes?
úû
ùêë
é=
úúû
ù
êêë
é=
åååå
2
12
2
00l
l
yyx
yxx
IIIIII
M
Slid
e cr
edit
: D
avid
Jac
obs
12-Oct-1747
![Page 48: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/48.jpg)
Lecture 6 -Stanford University
GeneralCase
• SinceM issymmetric,wehave
• WecanvisualizeM asanellipsewithaxislengthsdeterminedbytheeigenvaluesandorientationdeterminedbyR
RRM úû
ùêë
é= -
2
11
00l
l
Direction of the slowest change
Direction of the fastest change
(lmax)-1/2
(lmin)-1/2
adap
ted
from
Dar
ya F
rolo
va,
Den
is S
imak
ov
(Eigenvalue decomposition)
12-Oct-1748
![Page 49: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/49.jpg)
Lecture 6 -Stanford University
InterpretingtheEigenvalues• ClassificationofimagepointsusingeigenvaluesofM:
l1
“Corner”l1 andl2 arelarge,l1 ~ l2;E increasesinalldirections
l1 andl2 aresmall;E isalmostconstantinalldirections “Edge”
l1 >> l2
“Edge”l2 >> l1
“Flat”region
Slid
e cr
edit
: Kr
iste
n G
raum
an
l2
12-Oct-1749
![Page 50: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/50.jpg)
Lecture 6 -Stanford University
CornerResponseFunction
• Fastapproximation– Avoidcomputingthe
eigenvalues– α:constant
(0.04to0.06)
l2
“Corner”θ > 0
“Edge”θ < 0
“Edge”θ < 0
“Flat”region
θ = det(M )−α trace(M )2 = λ1λ2 −α(λ1 +λ2 )2
l1Slid
e cr
edit
: Kr
iste
n G
raum
an
12-Oct-1750
![Page 51: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/51.jpg)
Lecture 6 -Stanford University
WindowFunctionw(x,y)
• Option1:uniformwindow– Sumoversquarewindow
– Problem:notrotationinvariant
• Option2:SmoothwithGaussian– Gaussianalreadyperformsweightedsum
– Resultisrotationinvariant
1 in window, 0 outside
2
2,( , ) x x y
x y x y y
I I IM w x y
I I Ié ù
= ê úê úë û
å
Gaussian
2
2,
x x y
x y x y y
I I IM
I I Ié ù
= ê úê úë û
å
2
2( ) x x y
x y y
I I IM g
I I Is
é ù= *ê ú
ê úë û
Slid
e cr
edit
: Ba
stia
n Le
ibe
12-Oct-1751
![Page 52: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/52.jpg)
Lecture 6 -Stanford University
Summary:HarrisDetector[Harris88]
• Computesecondmomentmatrix(autocorrelationmatrix)
1. Image derivatives
Ix Iy
2. Square of derivatives
Ix2 Iy2 IxIy
3. Gaussian filter g(sI) g(Ix2) g(Iy2) g(IxIy)
R
2
2
( ) ( )( , ) ( )
( ) ( )x D x y D
I D Ix y D y D
I I IM g
I I Is s
s s ss s
é ù= *ê ú
ê úë û
2 2 2 2 2 2( ) ( ) [ ( )] [ ( ) ( )]x y x y x yg I g I g I I g I g Ia= - - +
θ = det[M (σ I ,σ D )]−α[trace(M (σ I ,σ D ))]2
4. Cornerness function – two strong eigenvalues
5. Perform non-maximum suppression
Slid
e cr
edit
: Kr
ysti
anM
ikol
ajcz
yk
12-Oct-1752
![Page 53: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/53.jpg)
Lecture 6 -Stanford University
HarrisDetector:Workflow
Slid
e ad
apte
d fr
om D
arya
Fro
lova
, D
enis
Sim
akov
12-Oct-1753
![Page 54: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/54.jpg)
Lecture 6 -Stanford University
HarrisDetector:Workflow- computercornerresponsesθ
Slid
e ad
apte
d fr
om D
arya
Fro
lova
, D
enis
Sim
akov
12-Oct-1754
![Page 55: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/55.jpg)
Lecture 6 -Stanford University
Slid
e ad
apte
d fr
om D
arya
Fro
lova
, D
enis
Sim
akov
HarrisDetector:Workflow- Takeonlythelocalmaximaofθ,whereθ>threshold
12-Oct-1755
![Page 56: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/56.jpg)
Lecture 6 -Stanford University
HarrisDetector:Workflow- ResultingHarrispoints
Slid
e ad
apte
d fr
om D
arya
Fro
lova
, D
enis
Sim
akov
12-Oct-1756
![Page 57: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/57.jpg)
Lecture 6 -Stanford University
HarrisDetector– Responses[Harris88]
Effect: A very precise corner detector.
Slid
e cr
edit
: Kr
ysti
anM
ikol
ajcz
yk
12-Oct-1757
![Page 58: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/58.jpg)
Lecture 6 -Stanford University
HarrisDetector– Responses[Harris88]
Slid
e cr
edit
: Kr
ysti
anM
ikol
ajcz
yk
12-Oct-1758
![Page 59: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/59.jpg)
Lecture 6 -Stanford University
HarrisDetector– Responses[Harris88]
Slid
e cr
edit
: Kr
iste
n G
raum
an
• Resultsarewellsuitedforfindingstereocorrespondences
12-Oct-1759
![Page 60: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/60.jpg)
Lecture 6 -Stanford University
HarrisDetector:Properties
• Translationinvariance?
Slid
e cr
edit
: Kr
iste
n G
raum
an
12-Oct-1760
![Page 61: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/61.jpg)
Lecture 6 -Stanford University
HarrisDetector:Properties
• Translationinvariance• Rotationinvariance?
Ellipse rotates but its shape (i.e. eigenvalues) remains the same
Corner response θ is invariant to image rotation
Slid
e cr
edit
: Kr
iste
n G
raum
an
12-Oct-1761
![Page 62: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/62.jpg)
Lecture 6 -Stanford University
HarrisDetector:Properties
• Translationinvariance• Rotationinvariance• Scaleinvariance?
Not invariant to image scale!
All points will be classified as edges!
Corner
Slid
e cr
edit
: Kr
iste
n G
raum
an
12-Oct-1762
![Page 63: Lecture: RANSAC and feature detectorsvision.stanford.edu/teaching/cs131_fall1718/files/06_ransac.pdf · RANSAC Line Fitting Example • Task: Estimate the best line ... 5 4 6 12 17](https://reader033.vdocuments.us/reader033/viewer/2022042920/5f6737a193f10f15167b7ad3/html5/thumbnails/63.jpg)
Lecture 6 -Stanford University
Whatwearelearnedtoday?
• Amodelfittingmethodforedgedetection– RANSAC
• Localinvariantfeatures– Motivation– Requirements,invariances
• Keypoint localization– Harriscornerdetector
12-Oct-1763