camera calibration sebastian thrun, gary bradski, daniel russakoff stanford cs223b computer vision ...
TRANSCRIPT
Camera Calibration
Sebastian Thrun, Gary Bradski, Daniel RussakoffStanford CS223B Computer Vision
http://robots.stanford.edu/cs223b
(with material from David Forsyth, James Rehg and Allen Hanson)
Sebastian Thrun Stanford University CS223B Computer Vision
A Quiz
How Many Flat Calibration Targets are Needed for Calibration? 1: 2: 3: 4: 5: 10:
How Many Corner Points do we need in Total?
1: 2: 3: 4: 10: 20:
Sebastian Thrun Stanford University CS223B Computer Vision
Example Calibration Pattern
Sebastian Thrun Stanford University CS223B Computer Vision
Perspective Camera Model
WX
WY
WZ
CX
CY
Object Space
CZ
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration Model (extrinsic)
?),(
W
W
W
C
C
Z
Y
X
fY
X
Sebastian Thrun Stanford University CS223B Computer Vision
Experiment 1: Parallel Board
Sebastian Thrun Stanford University CS223B Computer Vision
30cm10cm 20cm
Projective Perspective of Parallel Board
Sebastian Thrun Stanford University CS223B Computer Vision
Experiment 2: Tilted Board
Sebastian Thrun Stanford University CS223B Computer Vision
30cm10cm 20cm
500cm50cm 100cm
Projective Perspective of Tilted Board
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration Model (extrinsic)
),,,,( T
Z
Y
X
fY
X
W
W
W
C
C
rotation
translation (3D)
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration Model (extrinsic)
Z
Y
X
W
W
W
C
C
C
T
T
T
Z
Y
X
Z
Y
X
cossin0
sincos0
001
cos0sin
01
sin0cos
100
0cossin
0sincos
~
~
~
C
C
CC
C
Y
X
Z
f
Y
X~
~
~
Homogeneous Coordinates
Sebastian Thrun Stanford University CS223B Computer Vision
Homogeneous Coordinates
Idea: Most Operations Become Linear!
Extract Image Coordinates by Z-normalization
C
C
CC
C
Y
X
ZY
X~
~
~1
C
C
C
Z
Y
X
~
~
~
1
12
10
000,1
200,1
000,10
5.1
18
15
1
12
10
Sebastian Thrun Stanford University CS223B Computer Vision
Advantage of Homogeneous C’s
Z
Y
X
W
W
W
C
C
T
T
T
iZ
iY
iX
iY
iXi
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0
0
cos
sin
sin
cos
][~
][~
][
i-th data point
parametersrotation in nonlinear but
][][Equation Quadratici ibXiAX T
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration Model (intrinsic)
xs
ysPixel size
C
C
CC
C
Y
X
Z
f
Y
X~
~
~
Focal length
Image center
yx oo ,
Sebastian Thrun Stanford University CS223B Computer Vision
5.~
~
5.~
~
yC
C
y
xC
C
x
im
im
oZ
Y
s
f
oZ
X
s
f
y
x
Intrinsic Transformation
Sebastian Thrun Stanford University CS223B Computer Vision
5.
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
100
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0cossin
5.
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
100
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0sincos
y
Z
Y
X
W
W
W
Z
Y
X
W
W
W
y
x
Z
Y
X
W
W
W
Z
Y
X
W
W
W
x
im
im
o
T
T
T
iZ
iY
iX
T
T
T
iZ
iY
iX
s
f
o
T
T
T
iZ
iY
iX
T
T
T
iZ
iY
iX
s
f
y
x
Plugging the Model Together!
Sebastian Thrun Stanford University CS223B Computer Vision
Summary Parameters
Extrinsic
– Rotation
– Translation
Intrinsic
– Focal length
– Pixel size
– Image center coordinates
– (Distortion coefficients - see JYB’s tutorial )
,.
T
f
),( yx oo
,...1k
),( yx ss
Sebastian Thrun Stanford University CS223B Computer Vision
Q: Can We recover all Intrinsic Params?
No
Sebastian Thrun Stanford University CS223B Computer Vision
Summary Parameters, Revisited
Extrinsic
– Rotation
– Translation
Intrinsic
– Focal length
– Pixel size
– Image center coordinates
– (Distortion coefficients - see JYB’s tutorial )
,,
T
f
),( yx oo
,...1k
),( yx ssFocal length, in pixel units
Aspect ratioy
x
s
s
xs
f
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Trucco
Substitute
Advantage: Equations are linear in params
If over-constrained, minimize Least Mean Square fct
One possible solution:
Enforce constraint that R is rotation matrix
Lots of considerations to recover individual params…
cossin0
sincos0
001
cos0sin
01
sin0cos
100
0cossin
0sincos
333231
232221
131211
rrr
rrr
rrr
bAX
bAAAX TT 1
min)()( bAXbAX T
},,,,{ yxx
oos
fTRX
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Bouguet
5.
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
100
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0cossin
5.
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
100
][
][
][
cossin0
sincos0
001
cos0sin
010
sin0cos
0sincos
y
Z
Y
X
W
W
W
Z
Y
X
W
W
W
y
x
Z
Y
X
W
W
W
Z
Y
X
W
W
W
x
im
im
o
T
T
T
iZ
iY
iX
T
T
T
iZ
iY
iX
s
f
o
T
T
T
iZ
iY
iX
T
T
T
iZ
iY
iX
s
f
y
x
),,,,,,,( yxxW
W
W
im
im oos
fT
Z
Y
X
gy
x
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Bouguet, cont’d
Calibration Examples:
),,,,,,,
]1[
]1[
]1[
(]1[
]1[yx
xW
W
W
im
im oos
fT
Z
Y
X
gy
x
),,,,,,,
][
][
][
(][
][yx
xW
W
W
im
im oos
fT
NZ
NY
NX
gNy
Nx
…),,,,,,,
]2[
]2[
]2[
(]2[
]2[yx
xW
W
W
im
im oos
fT
Z
Y
X
gy
x
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Bouguet, cont’d
Least Mean Square
min),,,,,,,
][
][
][
(][
][
2
iyx
xW
W
W
im
im oos
fT
iZ
iY
iX
giy
ixJ
Sebastian Thrun Stanford University CS223B Computer Vision
Calibration a la Bouguet, cont’d
Least Mean Square
Gradient descent:J
},,,,,,{ yxx
oos
fTX
0X
0X
J
)(1.0 11
kkk XX
JXX
min),,,,,,,
][
][
][
(][
][
2
iyx
xW
W
W
im
im oos
fT
iZ
iY
iX
giy
ixJ
Sebastian Thrun Stanford University CS223B Computer Vision
Trucco Versus Bouguet
Trucco: Mimization of Squared
distance in parameter space
Bouguet Minimization of Squared
distance in Image space
)}ˆ
ˆ
ˆ
ˆ()
ˆ
ˆ
ˆ
ˆ(
2
1exp{
|2|
1)
ˆ
ˆ|
ˆ
ˆ( 1
5.
edge
edge
edge
edgeT
edge
edge
edge
edge
edge
edge
edge
edge
y
x
y
x
y
x
y
x
y
x
y
xp
)ˆ
ˆ
ˆ
ˆ()
ˆ
ˆ
ˆ
ˆ(
2
1)
ˆ
ˆ|
ˆ
ˆ(log 1
edge
edge
edge
edgeT
edge
edge
edge
edge
edge
edge
edge
edge
y
x
y
x
y
x
y
xconst
y
x
y
xp
Sebastian Thrun Stanford University CS223B Computer Vision
Q: How Many Images Do We Need?
Assumption: K images with M corners each 4+6K parameters 2KM constraints 2KM 4+6K M>3 and K 2/(M-3) 2 images with 4 points, but will 1 images with 5
points work? No, since points cannot be co-planar!
Sebastian Thrun Stanford University CS223B Computer Vision
Nonlinear Distortions
Barrel and Pincushion Tangential
Sebastian Thrun Stanford University CS223B Computer Vision
Barrel and Pincushion Distortion
telewideangle
Sebastian Thrun Stanford University CS223B Computer Vision
Models of Radial Distortion
)1( 42
21 rkrk
y
x
y
x
d
d
distance from center
Sebastian Thrun Stanford University CS223B Computer Vision
Tangential Distortion
cheapglue
cheap CMOS chipcheap lense image
cheap camera
Sebastian Thrun Stanford University CS223B Computer Vision
Image Rectification