![Page 1: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/1.jpg)
Lecture 8 -Silvio Savarese 5-Feb-18
• Problemformulation• Leastsquaremethods• RANSAC• Houghtransforms• Multi-modelfitting• Fittinghelpsmatching!
Lecture8FittingandMatching
Reading:[HZ] Chapter: 4 “Estimation – 2D projective transformation”
Chapter: 11 “Computation of the fundamental matrix F”[FP] Chapter:10 “Grouping and model fitting”
Some slides of this lectures are courtesy of profs. S. Lazebnik & K. Grauman
![Page 2: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/2.jpg)
FittingGoals:• Choose a parametric model to fit a certain
quantity from data• Estimate model parameters
- Lines - Curves- Homographic transformations- Fundamental matrices- Shape models
![Page 3: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/3.jpg)
Example: fitting lines(for computing vanishing points)
![Page 4: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/4.jpg)
H
Example: Estimating an homographic transformation
![Page 5: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/5.jpg)
Example: Estimating F
![Page 6: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/6.jpg)
AExample: fitting a 2D shape template
![Page 7: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/7.jpg)
Example: fitting a 3D object model
![Page 8: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/8.jpg)
Fitting,matchingandrecognitionareinterconnectedproblems
![Page 9: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/9.jpg)
Fitting
Critical issues:- noisy data- outliers- missing data- Intra-class variation
![Page 10: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/10.jpg)
Critical issues: noisy data
![Page 11: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/11.jpg)
What’s H?
Critical issues: outliers
![Page 12: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/12.jpg)
What’s H?
Critical issues: outliers
L
![Page 13: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/13.jpg)
Critical issues: missing data (occlusions)
![Page 14: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/14.jpg)
Critical issues: noisy data (intra-class variability)
A
![Page 15: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/15.jpg)
FittingGoal: Choose a parametric model to
fit a certain quantity from data
Techniques: •Least square methods•RANSAC•Hough transform•EM (Expectation Maximization) [not covered]
![Page 16: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/16.jpg)
Least squares methods- fitting a line -
• Data: (x1, y1), …, (xn, yn)
• Model of the line: yi – m xi – b = 0
• Parameters: m, b
• Find (m, b) to minimize fitting error (residual):
∑ =−−=
n
i ii bxmyE1
2)(
(xi, yi)
y=mx+b
[Eq. 2]
[Eq. 1]
![Page 17: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/17.jpg)
dEdh
= −2XTY + 2XTXh = 0
E = yi − xi 1"#
$%
mb
"
#&
$
%'
(
)**
+
,--
2
i=1
n∑ =
y1!yn
"
#
&&&&
$
%
''''
−
x1 1! !xn 1
"
#
&&&&
$
%
''''
mb
"
#&
$
%'
2
= Y − Xh 2
Normal equation
∑ =−−=
n
i ii bxmyE1
2)(
XTXh = XTY
Least squares methods- fitting a line -
h = XTX( )−1XTY
= (Y − Xh)T (Y − Xh) =Y TY − 2(Xh)TY + (Xh)T (Xh)
Find h = [m, b]T that minimizes E
[Eq. 2]
[Eq. 6]
[Eq. 3]
[Eq. 4]
[Eq. 5]
[Eq. 7]
![Page 18: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/18.jpg)
Least squares methods- fitting a line -
∑ =−−=
n
1i2
ii )bxmy(E
(xi, yi)
y=mx+b
h = XTX( )−1XTY h = m
b
!
"#
$
%&
• Fails completely for vertical lines
Issues?
[Eq. 6]
![Page 19: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/19.jpg)
• Distance between point (xi , yi ,1) and line (a, b, d)
• Find (a, b, d) to minimize the sum of squared perpendicular distances
E = (axi + byi + d)2
i=1
n∑
( xi , yi , 1 )
A h = 0data model parameters
Least squares methods- fitting a line -
[Eq. 8]
[Eq. 9]
![Page 20: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/20.jpg)
1||h||tosubject||hA||Minimize =
TUDVA =
V ofcolumn last h =
A h = 0
Least squares methods- fitting a line -
A is rank deficient
See [HZ], sec. A5.3 - page 593
![Page 21: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/21.jpg)
H
Least squares methods- fitting an homography -
x
y
x’
y’
A h = 0data model parameters
[Eq. 10]
See HZ - Sec 4.1 for details (DLT algorithm)- Sec 4.1.2 (or APPENDIX)
![Page 22: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/22.jpg)
Least squares: Robustness to noise
![Page 23: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/23.jpg)
Least squares: Robustness to noise
outlier!
![Page 24: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/24.jpg)
H
Critical issues: outliers
CONCLUSION: Least square is not robust w.r.t. outliers
![Page 25: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/25.jpg)
Least squares: Robust estimators
• ui = error (residual) of ith point w.r.t. model parameters h = (a,b,d)
( )σρ ;ii
uE ∑=
Robust function ρ:• When u is large, ρ saturates to 1• When u is small, ρ is a function of u2
∑ =−+=
n
i ii dybxaE1
2)(Instead of minimizing
dybxau iii −+=We minimize
• ρ = robust function of ui with scale parameter σ
u
ρ
[Eq. 11]
[Eq.
12]
In conclusion:• Favors a configuration with small residuals• Penalizes large residuals
(ρ=rho)
[Eq. 8]
![Page 26: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/26.jpg)
Least squares: Robust estimators
• ui = error (residual) of ith point w.r.t. model parameters h = (a,b,d)
( )σρ ;ii
uE ∑=∑ =
−+=n
i ii dybxaE1
2)(Instead of minimizing
dybxau iii −+=We minimize
• ρ = robust function of ui with scale parameter σ
[Eq. 11]
[Eq.
12]
•Small sigma àhighly penalize large residuals
•Large sigma àmildly penalize large residual (like LSQR)
u
ρ
[Eq. 8]
![Page 27: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/27.jpg)
The effect of the outlier is eliminated
Least squares: Robust estimators
Good scale parameter σ
![Page 28: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/28.jpg)
Least squares: Robust estimators
Bad scale parameter σ (too small!)Fits only locally
![Page 29: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/29.jpg)
Least squares: Robust estimators
Bad scale parameter σ (too large!)Same as standard LSQ
•Robust fitting is a nonlinear optimization problem (iterative solution)•Least squares solution provides good initial condition
•CONCLUSION: Robust estimator useful if prior info about the distribution of points is known
![Page 30: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/30.jpg)
FittingGoal: Choose a parametric model to
fit a certain quantity from data
Techniques: •Least square methods•RANSAC•Hough transform
![Page 31: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/31.jpg)
Basic philosophy(voting scheme)
• Data elements are used to vote for one (or multiple) models
• Robust to outliers and missing data
• Assumption 1: Noisy data points will not vote consistently for any single model (“few” outliers)
• Assumption 2: There are enough data points to agree on a good model (“few” missing data)
![Page 32: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/32.jpg)
Example: Line fitting
• Enough “good” data points supporting the line model in presence of noise• “Few” outliers compared to the “good” data points – these few outliers
won’t “consistently” vote for a line model
![Page 33: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/33.jpg)
r(I, h)< δ, ∀I ∈ I
Oπminπ :P→ I,O{ }
such that:
r(I, h) = residual
δ
Model parameters a,b,d
RANSAC
Fischler & Bolles in ‘81.
(RANdom SAmple Consensus) :
[Eq. 12]
= (axi + byi + d)2
i=1
n∑
![Page 34: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/34.jpg)
RANSAC
Algorithm:1. Select random sample of minimum required size to fit model 2. Compute a putative model from sample set3. Compute the set of inliers to this model from whole data setRepeat 1-3 until model with the most inliers over all samples is found
P = Sample set = set of points in 2D
![Page 35: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/35.jpg)
RANSAC
Algorithm:1. Select random sample of minimum required size to fit model [?]2. Compute a putative model from sample set3. Compute the set of inliers to this model from whole data setRepeat 1-3 until model with the most inliers over all samples is found
P = Sample set = set of points in 2D
![Page 36: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/36.jpg)
RANSAC
Algorithm:1. Select random sample of minimum required size to fit model [?]2. Compute a putative model from sample set3. Compute the set of inliers to this model from whole data setRepeat 1-3 until model with the most inliers over all samples is found
P = Sample set = set of points in 2D
![Page 37: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/37.jpg)
δ
RANSAC
Algorithm:1. Select random sample of minimum required size to fit model [?]2. Compute a putative model from sample set3. Compute the set of inliers to this model from whole data setRepeat 1-3 until model with the most inliers over all samples is found
O = ?P = Sample set = set of points in 2D
I = ?=14= 6
![Page 38: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/38.jpg)
δ
RANSAC
O = 6
Algorithm:1. Select random sample of minimum required size to fit model [?]2. Compute a putative model from sample set3. Compute the set of inliers to this model from whole data setRepeat 1-3 until model with the most inliers over all samples is found
I =14
![Page 39: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/39.jpg)
How many samples?
• Computationally unnecessary (and infeasible) to explore the entire sample space
• N samples are sufficient• N = number of samples required to ensure, with a probability p,
that at least one random sample produces an inlier set that is free from “real” outliers
• Function of s and e:– e = outlier ratio– s = minimum number of data points needed to fit the model
• Usually, p=0.99
![Page 40: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/40.jpg)
δ
• Here a random sample is given by two green points• The estimated inlier set is given by the green+blue points• How many “real” outliers we have here?
Example
2
e = outlier ratio is 6/20 = 30%s = 2
![Page 41: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/41.jpg)
• Random sample is given by two green points• The estimated inlier set is given by the green+blue points• How many “real” outliers we have here?
Example
0
δ
e = outlier ratio is 6/20 = 30%s = 2
![Page 42: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/42.jpg)
Example
δ
N is the number of times we need to sample my data (and thus repeat the steps 1-3 in the previous slides) before I find the configuration above with probability p. Again this is function of e and s as well.
e = outlier ratio is 6/20 = 30%s = 2
![Page 43: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/43.jpg)
How many samples?• Number N of samples required to ensure, with a probability p,
that at least one random sample produces an inlier set that is free from “real” outliers for a given s and e.
• E.g., p=0.99
proportion of outliers es 5% 10% 20% 25% 30% 40% 50%2 2 3 5 6 7 11 173 3 4 7 9 11 19 354 3 5 9 13 17 34 725 4 6 12 17 26 57 1466 4 7 16 24 37 97 2937 4 8 20 33 54 163 5888 5 9 26 44 78 272 1177
e = outlier ratios = minimum number needed to fit the model
( ) ( )( )se11log/p1logN −−−=
Note: this table assumes “negligible” measurement noise
[Eq. 13]
![Page 44: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/44.jpg)
Estimating H by RANSAC
Algorithm:1. Select a random sample of minimum required size [?]2. Compute a putative model from these3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model with the most inliers over all samples is found
P = Sample set = set of matches between 2 images
•H → 8 DOF•Need 4 correspondences
Outlier match
![Page 45: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/45.jpg)
Estimating F by RANSAC
Algorithm:1. Select a random sample of minimum required size [?]2. Compute a putative model from these3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model with the most inliers over all samples is found
P = Sample set = set of matches between 2 images
•F → 7 DOF•Need 7 (8) correspondences
Outlier matches
![Page 46: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/46.jpg)
• Simple and easily implementable• Successful in different contexts
RANSAC - conclusions
Good:
Bad:• Many parameters to tune• Trade-off accuracy-vs-time• Cannot be used if ratio inliers/outliers is too small
![Page 47: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/47.jpg)
FittingGoal: Choose a parametric model to
fit a certain quantity from data
Techniques: •Least square methods•RANSAC•Hough transform
![Page 48: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/48.jpg)
Hough transformP.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959
![Page 49: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/49.jpg)
x
y
Hough transform
Given a set of points, find the line parameterized by m,nthat explains the data points best: that is, m = m’ and n = n’
P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959
y = m’ x + n’
![Page 50: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/50.jpg)
x
y
n
m
Hough transform
Given a set of points, find the line parameterized by m,nthat explains the data points best: that is, m = m’ and n = n’
P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959
Hough space defined by the parameters of the model we want to fit (i.e., m, n)
y1 = m x1 + n
(x1, y1)
(x2, y2)
y2 = m x2 + n
m’
n’
y = m’ x + n’
Original space where the data points are
![Page 51: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/51.jpg)
Hough transform
Any Issue?
P.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959
The parameter space [m,n] is unbounded…
![Page 52: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/52.jpg)
x
y
Hough transformP.V.C. Hough, Machine Analysis of Bubble Chamber Pictures, Proc. Int. Conf. High Energy Accelerators and Instrumentation, 1959
Hough space
ρθθ =+ siny cosx
θρ
Use a polar representation for the parameter space
θ
ρ
[Eq. 13]
Any Issue?
Original space
The parameter space [m,n] is unbounded…
![Page 53: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/53.jpg)
Hough transform -experiments
Hough spaceOriginal space θ
ρ
![Page 54: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/54.jpg)
How to compute the intersection point? In presence of noise!
IDEA: introduce a grid a count intersection points in each cell
Hough transform -experiments
Noisy data
Hough spaceOriginal space θ
ρ
![Page 55: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/55.jpg)
Issue: spurious peaks due to uniform noise
Hough transform -experiments
Hough spaceOriginal space θ
ρ
![Page 56: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/56.jpg)
• All points are processed independently, so can cope with occlusion/outliers
• Some robustness to noise: noise points unlikely to contribute consistently to any single cell
Hough transform -conclusionsGood:
Bad:• Spurious peaks due to uniform noise• Trade-off noise-grid size (hard to find sweet point)• Doesn’t handle well high dimensional models
![Page 57: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/57.jpg)
Courtesy of Minchae Lee
Applications – lane detection
![Page 58: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/58.jpg)
Applications – computing vanishing points
![Page 59: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/59.jpg)
Generalized Hough transformD. Ballard, Generalizing the Hough Transform to Detect Arbitrary Shapes, Pattern Recognition 13(2), 1981
• Parameterize a shape by measuring the location of its parts and shape centroid
• Given a set of measurements, cast a vote in the Hough (parameter) space
[more on forthcoming lectures]
B. Leibe, A. Leonardis, and B. Schiele, Combined Object Categorization and Segmentation with an Implicit Shape Model, ECCV Workshop on Statistical Learning in Computer Vision 2004
• Used in object recognition! (the implicit shape model)
![Page 60: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/60.jpg)
Lecture 8 -Silvio Savarese 5-Feb-18
• Problemformulation• Leastsquaremethods• RANSAC• Houghtransforms• Multi-modelfitting• Fittinghelpsmatching!
Lecture9FittingandMatching
![Page 61: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/61.jpg)
Fitting multiple models
• Incremental fitting
• E.M. (probabilistic fitting)
• Hough transform
![Page 62: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/62.jpg)
Incremental line fittingScan data point sequentially (using locality constraints)
Perform following loop:
1. Select N point and fit line to N points2. Compute residual RN
3. Add a new point, re-fit line and re-compute RN+1
4. Continue while line fitting residual is small enough,
Ø When residual exceeds a threshold, start fitting new model (line)
![Page 63: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/63.jpg)
Hough transformC
ourt
esy
of u
nkno
wn
Same cons and pros as before…
![Page 64: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/64.jpg)
Lecture 8 -Silvio Savarese 5-Feb-18
• Problemformulation• Leastsquaremethods• RANSAC• Houghtransforms• Multi-modelfitting• Fittinghelpsmatching!
Lecture9FittingandMatching
![Page 65: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/65.jpg)
Features are matched (for instance, based on correlation)
Fitting helps matching!
windowwindow
Image 1 Image 2
![Page 66: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/66.jpg)
Idea: • Fitting an homography H (by RANSAC) mapping features from images 1 to 2 • Bad matches will be labeled as outliers (hence rejected)!
Matches based on appearance onlyGreen: good matchesRed: bad matches
Image 1 Image 2
Fitting helps matching!
![Page 67: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/67.jpg)
Fitting helps matching!
![Page 68: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/68.jpg)
M. Brown and D. G. Lowe. Recognising Panoramas. In Proceedings of the 9th International Conference on Computer Vision -- ICCV2003
Recognising Panoramas
![Page 69: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/69.jpg)
Next lecture:Feature detectors and descriptors
![Page 70: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/70.jpg)
![Page 71: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/71.jpg)
bAx =• More equations than unknowns
• Look for solution which minimizes ||Ax-b|| = (Ax-b)T(Ax-b)• Solve
• LS solution
0)()(=
∂
−−∂
i
T
xbAxbAx
bAAAx TT 1)( −=
Least squares methods- fitting a line -
![Page 72: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/72.jpg)
t1t A)AA(A −+ =
A−1 =V∑−1UT
with equal to for all nonzero singularvalues and zero otherwise
1−∑
= pseudo-inverse of A
Solving bAAAx tt 1)( −=
Least squares methods- fitting a line -
tVUA ∑=
A+ =V∑+UT
= SVD decomposition of A
+∑
![Page 73: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/73.jpg)
Least squares methods- fitting an homography -
A h = 0
0
h
hh
3,3
2,1
1,1
=
⎥⎥⎥⎥
⎦
⎤
⎢⎢⎢⎢
⎣
⎡
!
From n>=4 corresponding points:
![Page 74: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/74.jpg)
Issue: spurious peaks due to uniform noisefeatures votes
Hough transform -experiments
![Page 75: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/75.jpg)
x
y
n
m
x
y m3 5 3 3 2 2
3 7 11 10 4 3
2 3 1 4 5 22 1 0 1 3 3
n
Hough transform
![Page 76: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/76.jpg)
Fitting helps matching!
Images courtesy of Brandon Lloyd
![Page 77: Lecture 8 Fitting and Matching - Stanford University...2. Compute a putative model from these 3. Compute the set of inliers to this model from whole sample space Repeat 1-3 until model](https://reader033.vdocuments.us/reader033/viewer/2022050600/5fa7869c8c07e328911f0c0e/html5/thumbnails/77.jpg)