EE 7700
Geometric Transformations
Bahadir K. Gunturk 2
Geometric Transformation
translation
scale
rotation & scale
2
1
43
21
b
b
y
x
aa
aa
y
x
Rigid flow
11001
'
'
243
121
y
x
baa
baa
y
x
cossin
sincos
s
s
0
0
Rotation matrix
Scale matrix
Bahadir K. Gunturk 3
Affine Flow
2
1
43
21
b
b
y
x
aa
aa
yy
xx
v
u
Bahadir K. Gunturk 4
Perspective Flow
1 2 3
7 8
'1
a x a y ax
a x a y
4 5 6
7 8
'1
a x a y ay
a x a y
'
'
u x x
v y y
Bahadir K. Gunturk 5
Bahadir K. Gunturk 6
Bahadir K. Gunturk 7
EE 7730
RANSAC: RANdom SAmple Consensus
Bahadir K. Gunturk 9
Outliers
Consider the least squares fit for optical flow:
1 2 1
3 4 2
( , )
( , )
u x y a x a y b
v x y a x a y b
1 1 1 1
1 1 1 2
1
3
4
2
1 0 0 0
0 0 0 1
1 0 0 0
0 0 0 1n n n
n nn
u x y a
v x y a
b
a
u ax y
bx yv
If some of the values are wrong, it will degrade the estimation.
Bahadir K. Gunturk 10
Outliers
It is best not to include outliers in the estimation.
Outliers
Best fit is degraded due to the outliers
Line Fitting Problem: Given (x1,y1), …, (xN,yN); find the line y=ax+b
1 1 1
1N N
y xa
by x
Bahadir K. Gunturk 11
Robust Estimation
Two step process: Classify data points as outliers or inliers Use inliers only to fit a model
Bahadir K. Gunturk 12
RANSAC
Repeat for k times: Randomly choose n points (the smallest number of points
required) from the data. Estimate the parameters using these points. For each data point other than these n points:
Check if the data point is within a threshold, t, distance of current model; if it is, the data point is consistent with current model.
The total number of data points that are consistent is model’s support.
If the support is larger than a predetermined number, d, then there is a good fit. Re-estimate the parameters using these inliers.
Choose the best fit with the smallest fitting error.
Bahadir K. Gunturk 13
RANSAC
Two samplesand their supports
for line-fitting
Bahadir K. Gunturk 14
Example
Find the perspective parameters
from Hartley & Zisserman
1 2 3
7 8
'1
a x a y ax
a x a y
4 5 6
7 8
'1
a x a y ay
a x a y
Bahadir K. Gunturk 15
Example
Apply corner detectors to both images
from Hartley & Zisserman
Bahadir K. Gunturk 16
Example
Find the best match within a search window.
from Hartley & Zisserman
Bahadir K. Gunturk 17
Example
Initial match results
188 matched features in left image pointing to locations of corresponding right image features
from Hartley & Zisserman
Bahadir K. Gunturk 18
Example
Inliers and outliers after RANSAC
89 outliers 99 inliersfrom Hartley & Zisserman
Bahadir K. Gunturk 19
Panoramic Image Reconstruction
Application: Mosaic construction
Find featuresMatch featuresFit parametric model
EE7730
Stereo Vision
Bahadir K. Gunturk 21
Pinhole Camera
Bahadir K. Gunturk 22
Review: Perspective Projection
' ' 'x y f
x y z
Bahadir K. Gunturk 23
Stereoscene pointscene point
optical centeroptical center
image planeimage plane
p p’
p p’
Bahadir K. Gunturk 24
Stereo Constraints
X1
Y1
Z1
O1
Image plane
Focal plane
M
p p’
Y2
X2
Z2O2
Epipolar Line
Epipole
Bahadir K. Gunturk 25
From Geometry to Algebra
O O’
P
pp’
All vectors shown lie on the same plane.
Bahadir K. Gunturk 26
From Geometry to Algebra
O O’
P
pp’
( , ,1)[ ( ')] 0 with
' ( ', ',1)
T
T
p u vp t Rp
p u v
Bahadir K. Gunturk 27
Matrix form of cross product
20
0
0
y z z y z
z x x z z x
x y y z y x
a b a b a a
a b a b a b a a b a b
a b a b a a
( ) 0
( ) 0
a a b
b a b
a×b=|a||b|sin(η)u a=axi+ayj+azk
b=bxi+byj+bzk
Bahadir K. Gunturk 28
The Essential Matrix
( , ,1)[ ( ')] 0 with
' ( ', ',1)
T
T
p u vp t Rp
p u v
' 0Tp Ep
' 0 with Tp Ep E t R Essential matrix
Bahadir K. Gunturk 29
A Simple Stereo System
Zw=0
LEFT CAMERA
Left image:reference
Right image:target
RIGHT CAMERA
Elevation Zw
disparity
Depth Z
baseline
Bahadir K. Gunturk 30
Parallel Cameras
ZT
fZxxTlr
OOll OOrr
PP
ppll pprr
TT
ZZ
xxll xxrr
ff
T is the stereo baseline
rlxx
TfZ
rlxxd Disparity:
Bahadir K. Gunturk 31
Stereo View
Left View Right View
Disparity
Bahadir K. Gunturk 32
Correlation Approach
For Each point (xl, yl) in the left image, define a window centered at the point
(xl, yl)LEFT IMAGE
Bahadir K. Gunturk 33
Correlation Approach
… search its corresponding point within a search region in the right image
(xl, yl)RIGHT IMAGE
Bahadir K. Gunturk 34
Correlation Approach
… the disparity (dx, dy) is the displacement when the correlation is maximum
(xl, yl)dx(xr, yr)RIGHT IMAGE
Bahadir K. Gunturk 35
Stereo results
Ground truthScene
Data from University of Tsukuba
(Seitz)
Bahadir K. Gunturk 36
Results with window correlation
Estimated depth of field Ground truth
(Seitz)
Bahadir K. Gunturk 37
Results with better method
A state of the art methodBoykov et al., Fast Approximate Energy Minimization via Graph Cuts,
International Conference on Computer Vision, September 1999.
Ground truth
(Seitz)
Bahadir K. Gunturk 38
Applications
courtesy of Sportvision
First-down line
Bahadir K. Gunturk 39
ApplicationsVirtual advertising
courtesy of Princeton Video Image