lecture 2 camera calibration
Post on 11-Feb-2017
585 Views
Preview:
TRANSCRIPT
1
Lecture 2: Camera Calibration
Lecture 2Camera Calibration
Joaquim SalviUniversitat de Girona
Visual Perception
2
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
3
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
4
Lecture 2: Camera Calibration
– Dense reconstruction – Visual inspection
– Object localization – Camera localization
2.1 Calibration Introduction
• Some applications of this capability include
5
Lecture 2: Camera Calibration
Image courtesy of C. Taylor
“The Scholar of Athens,” Raphael, 1518
2.1 Calibration Introduction – Perspective Imaging
6
Lecture 2: Camera Calibration
WZ
WY
WX
WO
wP
Image Plane
{ }W
uP
IY
IX
IO
{ }I
Focal Point
WZ
WY
WX
WO
wP
Image Plane
{ }W
uP
IY
IX
IO
{ }I
Focal Point
1
I
u
I I
u u
X
P Y
1
W
w
W
W w
w W
w
X
YP
Z
In pixels
In metrics?
wl
wl
0
W
w
W
W w
w W
w
X
Yl
Z
2.1 Calibration Introduction
7
Lecture 2: Camera Calibration
Modelling
G(X) X ?
Calibration
X !!!
Modelling:
• Determine the equation that approximates the camera behaviour.
• Define the set of unknowns in the equation (camera parameters).
• The camera model is an approximation of the physics & optics of the camera.
Calibration:
• Get the numeric value of every camera parameter.
G(X)
2.1 Calibration Introduction
8
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
9
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
10
Lecture 2: Camera Calibration
2.2 Pinhole Model
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
dP
IY
IXIO{ }I
RY
RX{ }R
RO
Image
coordinate
system
11
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 1: World to Camera)
Camera
coordinate
system
World
coordinate
system
CY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
C
WK
Step 1
12
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 2: Projection)
Camera
coordinate
system
World
coordinate
system
CY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uPf
Step 2
wX
wY
wZ
uX
uY
RY
RX{ }R
RO
13
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 3: Lens Distortion)
Camera
coordinate
system
World
coordinate
system
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
RY
RX{ }R
RO
Step 3
dP
14
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 3: Lens Distortion)
dP
uP dr
CY
CX
Observed position
Ideal
projection
dr: radial distortion
a
b
Radial distortion effect (a: negative, b: positive)
Radial Distortion
15
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 3: Lens Distortion)
Axis with
maximum
radial
distortion
Axis with
minimum
tangential
distortion
CY
CX
Ideal
projection
Observed
position
dr: radial distortion
dt: tangential distortion
dPuP
dr
CY
CX
dt
Radial and Tangential Distortion
Image with distortionImage without distortion
16
Lecture 2: Camera Calibration
2.2 Pinhole Model (Step 4: Camera to Image)
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
IY
IXIO{ }I
RY
RX{ }R
RO
Image
coordinate
system
Step 4
dP
17
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
dP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 4
2.2 Pinhole Model
18
Lecture 2: Camera Calibration
2.2 Calibration Methods (I)
• Method of Hall– Lineal method
– Transformation matrix
• Method of Faugeras-Toscani– Lineal method
– Obtaining camera parameters
• Method of Faugeras-Toscani with distortion– Iterative method
– Radial distortion
• Method of Tsai– Iterative method
– Radial distortion
– Focal distance estimation
• Method of Weng– Iterative method
– Radial and tangential distortion
• … and many more
19
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
20
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
21
Lecture 2: Camera Calibration
2.3 The Method of Hall
• Method of Hall– Lineal method
– Transformation matrix
• Method of Faugeras-Toscani– Lineal method
– Obtaining camera parameters
• Method of Faugeras-Toscani with distortion– Iterative method
– Radial distortion
• Method of Tsai– Iterative method
– Radial distortion
– Focal distance estimation
• Method of Weng– Iterative method
– Radial and tangential distortion
• … and many more
22
Lecture 2: Camera Calibration
World
coordinate
system
WZ
WYWX
WO
wP
Image plane
{ }W
uP
IY
IXIO{ }IImage
coordinate
system
2.3 The Method of Hall - Modelling
23
Lecture 2: Camera Calibration
11 12 13 14
21 22 23 24
31 32 33 341
W
I w
u W
I w
u W
w
Xs X A A A A
Ys Y A A A A
Zs A A A A
Assume light is captured on the image plane by a linear projection
The matrix is defined up to a scale factor Multiple Solutions
A component is fixed to the unity Unique Solution
11 12 13 14
21 22 23 24
31 32 33 11
W
I w
u W
I w
u W
w
Xs X A A A A
Ys Y A A A A
Zs A A A
2.3 The Method of Hall - Modelling
24
Lecture 2: Camera Calibration
11 12 13 14
31 32 33
21 22 23 24
31 32 33
1
1
W W W
I w w w
u W W W
w w w
W W W
I w w w
u W W W
w w w
A X A Y A Z AX
A X A Y A Z
A X A Y A Z AY
A X A Y A Z
11 12 13 14
21 22 23 24
31 32 33 11
W
I w
u W
I w
u W
w
Xs X A A A A
Ys Y A A A A
Zs A A A
11 31 12 32 13 33 14
21 31 22 32 23 33 24
W I W W I W W I W I
w u w w u w w u w u
W I W W I W W I W I
w u w w u w w u w u
A X A X X A Y A X Y A Z A X Z A X
A X A Y X A Y A Y Y A Z A Y Z A Y
2.3 The Method of Hall - Calibration
25
Lecture 2: Camera Calibration
2 1
2
1 0 0 0 0
0 0 0 0 1
W W W I W I W I W
i w w w u w u w u wi i i i i i i i i
W W W I W I W I W
i w w w u w u w u wi i i i i i i i i
Q X Y Z X X X Y X Z
Q X Y Z Y X Y Y Y Z
2 1
2
I
i u i
I
i u i
B X
B Y
T
11 12 13 14 21 22 23 24 31 32 33A A A A A A A A A A A A
1
t tA Q Q Q B
QA B
Pseudoinverse leads to a unique solution:
1A Q B
Obtaining 11 unknowns and each 2D point gives two equations
So, at least 6 points are needed. More points leads to a more accurate solution.
11 31 12 32 13 33 14
21 31 22 32 23 33 24
W I W W I W W I W I
w u w w u w w u w u
W I W W I W W I W I
w u w w u w w u w u
A X A X X A Y A X Y A Z A X Z A X
A X A Y X A Y A Y Y A Z A Y Z A Y
2.3 The Method of Hall - Calibration
26
Lecture 2: Camera Calibration
Camera
coordinate
system
CY
CX CZ
CO{ }C
IY
IXIO{ }I
RY
RX
RO
{ }R
World
coordinate
system
WZ
WYWX
WO{ }W
Reconstruction
Area
Image of the calibrating pattern
2 1
2
1 0 0 0 0
0 0 0 0 1
W W W I W I W I W
i w w w u w u w u wi i i i i i i i i
W W W I W I W I W
i w w w u w u w u wi i i i i i i i i
Q X Y Z X X X Y X Z
Q X Y Z Y X Y Y Y Z
2 1
2
I
i u i
I
i u i
B X
B Y
1t t
A Q Q Q B
2.3 The Method of Hall - Calibration
27
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
28
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
29
Lecture 2: Camera Calibration
2.4 The Method of Faugeras-Toscani
• Method of Hall– Lineal method
– Transformation matrix
• Method of Faugeras-Toscani– Lineal method
– Obtaining camera parameters
• Method of Faugeras-Toscani with distortion– Iterative method
– Radial distortion
• Method of Tsai– Iterative method
– Radial distortion
– Focal distance estimation
• Method of Weng– Iterative method
– Radial and tangential distortion
• … and many more
30
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 4
2.4 The Method of Faugeras-Toscani
31
Lecture 2: Camera Calibration
• Extrinsic parameters: Model the situation and orientation of the camera with
respect to a world co-ordinate system.
• Intrinsic parameters: Model the behaviour of the internal geometry and the optical
characteristics of the camera.
u
w
v
Yc
Xc
Zc
Oc
Oi
(u0, v0)
Pu
P
image
co-ordinate
system
(píxels)
retinal
co-ordinate
system
(mm.)
Image plane
Retinal plane
Yr
Xr
Zr
w
World
co-ordinate
system
WZ
WY
WX
WO { }W
Camera
co-ordinate system
2.4 The Method of Faugeras-Toscani
32
Lecture 2: Camera Calibration
Yc
Xc
Zc
Zw
YwOc
Ow
Pw
Camera
co-ordinate
system World
co-ordinate system
Retinal Plane
K
Xw
X
C
W Y
Z
t
T t
t
11 12 13
21 22 23
31 32 33
, , ,C
W
C
W
R Rot X Rot Y Rot Z
r r r
R r r r
r r r
C W
w w
C C W C
w W w W
C W
w w
X X
Y R Y T
Z Z
1 1
C W
Cw w
W
P PK
3 3 3 1
1 30 1
C C
C W Wx x
W
x
R TK
2.4 Extrinsic Parameters
33
Lecture 2: Camera Calibration
CPw
CPu
Yc
Xc
Zc
Oc C
f
PZc
Yu
PYc XuPXc
C
C w
u C
w
C
C w
u C
w
XX f
Z
YY f
Z
2.4 The Intrinsic Parameters: Ideal Projection
34
Lecture 2: Camera Calibration
pixel
Retinal
plane
(0, 0)
Yr
Xr (0, 0)
(Xd, Yd)
Image
Plane
(Xp, Yp)
R C
d u u
R C
d v u
X k X
Y k Y
2.4 The Intrinsic Parameters: Pixel Conversion
35
Lecture 2: Camera Calibration
Yr
Xr
V
U(0, 0)
Principal point
(u0,v0)
Computer image
co-ordinate
system
Camera
co-ordinate
system
0
0
I R
d d
I R
d d
X X u
Y Y v
2.4 The Intrinsic Parameters: Principal Point
36
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 4
2.4 The Method of Faugeras-Toscani
37
Lecture 2: Camera Calibration
Real projection on the image plane (Xi, Yi)
(Xw, Yw, Zw) 3D object point with respect to world co-ordinate system
Affine transformation.
Modelled parameters: R, T
(Xc, Yc, Zc) 3D object point with respect to camera co-ordinate system
Perspective transformation.
Modelled parameter: f
(Xu, Yu) Ideal projection on the retinal plane
Pixel adjustment
Modelled parameters: ku, kv
(Xp, Yp) Real projection on the image plane
Adaptation to the computer image buffer
Modelled parameters: u0, v0
2.4 The Method of Faugeras-Toscani
38
Lecture 2: Camera Calibration
C
C w
u C
w
C
C w
u C
w
XX f
Z
YY f
Z
R C
d u u
R C
d v u
X k X
Y k Y
0
0
I R
d d
I R
d d
X X u
Y Y v
0
0
C
I w
u u C
w
C
I w
u v C
w
XX k f u
Z
YY k f v
Z
0
0
0 0
0 0
0 0 1 01
C
I w
u u C
I w
u v C
w
Xs X u
Ys Y v
Zs
vv
uu
fk
fk
2.4 The Method of Faugeras-Toscani - Modelling
39
Lecture 2: Camera Calibration
11 12 13
0
21 22 23
0
31 32 33
0 0
0 0
0 0 1 00 0 0 1 1
W
xI w
u u W
yI w
u v W
z w
r r r t Xs X u
r r r t Ys Y v
r r r t Zs
1 0 3 0
2 0 3 0
3
u u x z
v v y z
z
r u r t u t
A r v r t v t
r t
Intrínsecs Extrínsecs
2.4 The Method of Faugeras-Toscani - Modelling
40
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
41
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
42
Lecture 2: Camera Calibration
11 12 13
0
21 22 23
0
31 32 33
0 0
0 0
0 0 1 00 0 0 1 1
W
xI w
u u W
yI w
u v W
z w
r r r t Xs X u
r r r t Ys Y v
r r r t Zs
1 0 3 0
2 0 3 0
3
u u x z
v v y z
z
r u r t u t
A r v r t v t
r t
Intrínsecs Extrínsecs
2.5 The Method of Faugeras-Toscani – Modelling
43
Lecture 2: Camera Calibration
1 14 3 34
2 24 3 34
0
0
W I W
w u w
W I W
w u w
A P A X A P A
A P A Y A P A
31 14
34 34 34
32 24
34 34 34
I W W I
u w w u
I W W I
u w w u
AA AX P P X
A A A
AA AY P P Y
A A A
1 1 2
3 2 2
I W W I
u w w u
I W W I
u w w u
X T P C T P X
Y T P C T P Y
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2.5 The Method of Faugeras-Toscani – Calibration
1 0 3 0
2 0 3 0
3
u u x z
v v y z
z
r u r t u t
A r v r t v t
r t
343
242
141
AA
AA
AA
1343
242
141
w
W
u
I
u
I
P
AA
AA
AA
s
Ys
Xs
44
Lecture 2: Camera Calibration
1
2
3
1
2
T
T
X T
C
C
B QX
1 3
1 3
0 1 0
0 0 1
t tW I W
w u w xi i i
t tI W W
x u w wi i i
P X PQ
Y P P
I
u i
I
u i
XB
Y
1
t tX Q Q Q B
2.5 The Method of Faugeras-Toscani – Calibration
1 1 2
3 2 2
I W W I
u w w u
I W W I
u w w u
X T P C T P X
Y T P C T P Y
11 unknowns
minimum 6 points
45
Lecture 2: Camera Calibration
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
3 1r 2
1zt
T
2.5 The Method of Faugeras-Toscani – tz
𝑅 =
𝑟1𝑟2𝑟3
46
Lecture 2: Camera Calibration
1 2 1 2
1 2 1 2
cos
sin
v v v v
v v v v
0
1
1
0
t
i j
t
i j
i j
i j
r r i j
r r i j
r r i j
r r i j
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2.5 The Method of Faugeras-Toscani – Intrinsics
02
33130
33310
32121
·· u
t
rr
t
r
t
ru
t
r
t
r
t
r
t
ru
t
rTTTT
z
u
zzzzz
u
zz
t
2
1zt
T
2
2
210
T
TTu
t
47
Lecture 2: Camera Calibration
1 2 1 2
1 2 1 2
cos
sin
v v v v
v v v v
0
1
1
0
t
i j
t
i j
i j
i j
r r i j
r r i j
r r i j
r r i j
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2 31 2
0 02 2
2 2
1 2 2 3
2 2
2 2
tt
t t t t
u v
T TT Tu v
T T
T T T T
T T
2.5 The Method of Faugeras-Toscani – Intrinsics
48
Lecture 2: Camera Calibration
1 2 1 2
1 2 1 2
cos
sin
v v v v
v v v v
0
1
1
0
t
i j
t
i j
i j
i j
r r i j
r r i j
r r i j
r r i j
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2.5 The Method of Faugeras-Toscani – Extrinsics
tt
t
tt
t
u
z
z
u
zz
TT
T
T
TTTTr
TTT
T
T
TTTTr
tu
t
rTr
t
ru
t
rT
21
2
2
2
21211
221
2
2
2
2
212110
311
10
31
1
49
Lecture 2: Camera Calibration
1 2 1 2
1 2 1 2
cos
sin
v v v v
v v v v
0
1
1
0
t
i j
t
i j
i j
i j
r r i j
r r i j
r r i j
r r i j
v
z
y
v
zz
z
u
z
xu
zz
t
tvC
t
rv
t
rT
t
rT
t
tuC
t
ru
t
rT
022
03
3
32
011
03
1
2 1 2
1 1 22
1 2 2
2 2 3
2 3 22
2 3 2
2
3
2
t
t t
t
t t
T T Tr T T
T T T
T T Tr T T
T T T
Tr
T
2 1 2
1 2
1 2 2
2 2 3
2 2
2 3 2
2
1
t
x t t
t
y t t
z
T T Tt C
T T T
T T Tt C
T T T
tT
2.5 The Method of Faugeras-Toscani – Extrinsics
50
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
51
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
52
Lecture 2: Camera Calibration
2.6 The Method of Faugeras-Toscani with distortion
• Method of Hall– Lineal method
– Transformation matrix
• Method of Faugeras-Toscani– Lineal method
– Obtaining camera parameters
• Method of Faugeras-Toscani with distortion– Iterative method
– Radial distortion
• Method of Tsai– Iterative method
– Radial distortion
– Focal distance estimation
• Method of Weng– Iterative method
– Radial and tangential distortion
• … and many more
53
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
dP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 5
Step 4
2.6 The Method of Faugeras-Toscani with distortion
54
Lecture 2: Camera Calibration
Ideal
projection
Observed
position
drdt
Xr
Yr
dr: radial distortion
dt: tangential distortion
PuPd
2.6 Lens Distortion
55
Lecture 2: Camera Calibration
a
b
Radial distorsion effect Tangential distorsion effect
Xr
Axe of a
maximum
tangential
distortion
Axe of a
minimum
tangential
distortion
Radial distorsion is the most important and usually the only considered in
calibration.
2.6 Lens Distortion
56
Lecture 2: Camera Calibration
X X Du d x Y Y Du d y
D X k rx d 1
2 D Y k ry d 1
2r X Yd d 2 2
2 4
1 2
2 4
1 2
2 2
C
x d
C
y d
C C
d d
D X k r k r
D Y k r k r
r X Y
k1 is the most important component
and usuallly sufficient in most
applications.
2.6 Lens Distortion
Model of Faugeras-Toscani with distortion:
57
Lecture 2: Camera Calibration
u
w
v
Yc
Xc
Zc
Oc
Oi
(u0, v0)
Pu
P
Camera
co-ordinate system
image
co-ordinate
system
f Pd
retinal
co-ordinate
system
Image plane
Retinal plane
Yr
Xr
Zr
X
f
P
P
u Xc
Zc
Y
f
P
P
u Yc
Zc
X X Du d x Y Y Du d y
D X k rx d 1
2 D Y k ry d 1
2r X Yd d 2 2
X k Xp u d Y k Yp v d
X X ui p 0 Y Y vi p 0
2.6 The Method of Faugeras-Toscani with distortion
58
Lecture 2: Camera Calibration
Camera
coordinate
system
World
coordinate
system
0 0,u v
fCY
CX CZ
CO
WZ
WYWX
WO
wP
Image plane
{ }W
{ }C
uP
dP
IY
IXIO{ }I
RY
RX { }R
RO
C
WK
Image
coordinate
system
Step 1
Step 2Step 3
Step 5
Step 4
2.6 The Method of Faugeras-Toscani with distortion
59
Lecture 2: Camera Calibration
(Xw, Yw, Zw) 3D object point with respect to world co-ordinate system
Affine transformation.
Modelled parameters: R, T
(Xc, Yc, Zc) 3D object point with respect to camera co-ordinate system
Perspective transformation.
Modelled parameter: f
(Xu, Yu) Ideal projection on the retinal plane
Radial lens distortion.
Modelled parameter: k1
(Xd, Yd) Real projection on the retinal plane
Pixel adjustment
Modelled parameters: ku, kv
(Xp, Yp ) Real projection on the image plane
Adaptation to the computer image buffer
Modelled parameters: u0, v0
(Xi, Yi) Real projection on the image plane
2.6 The Method of Faugeras-Toscani with distortion
60
Lecture 2: Camera Calibration
2
1
2
1
C
C Cw
d dC
w
C
C Cw
d dC
w
Xf X k r X
Z
Yf Y k r Y
Z
0
0
I
dC
d
u
I
dC
d
v
X uX
k
Y vY
k
1 1
C W
w w
C W
Cw w
WC W
w w
X X
Y YK
Z Z
r X Yd d 2 2
The model is NON-LINEARIterative minimisation:
• Newton-Raphson
• Levenberg-Marquardt
2.6 The Method of Faugeras-Toscani with distortion
61
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
62
Lecture 2: Camera Calibration
Contents
2. Camera Calibration2.1 Calibration introduction
2.2 The pinhole model
2.3 The method of Hall
2.4 The method of Faugeras-Toscani – Modelling
2.5 The method of Faugeras-Toscani – Calibration
2.6 The method of Faugeras-Toscani with distortion
2.7 Experimental comparison of methods
63
Lecture 2: Camera Calibration
Hall Faugeras Faugeras distorted
Tsai Weng
Transformation
matrix
Step 3Lens
Distortion
Step 2Projection
Step 1World2camera
Transformation
with , , , tx, ty and tz
Projection with f
Radial distortion with k1
UndistortedMultiple
distortion
k1, g1, g2, g3, g4
Transformation with
u0, v0 , ku and kv
Transformation
with u0, v0 and sx Transformation
with u0, v0,
ku and kv
Step 4Camera2image
C C W C
w W w WP P T R
,
C C
C Cw w
u uC C
w w
X YX f Y f
Z Z
C C
w uP P
C C
u dP P
C I
d dP P
C C
u dP P
0
0
I C
d u d
I C
d v d
X k X u
Y k Y v
2 2
1
2 2
1
C C C C C
u d u u u
C C C C C
u d u u u
X X k X X Y
Y Y k Y X Y
1'
0
1
0
I C
d x x d
I C
d y d
X s d X u
Y d Y v
=
W C
w wP P
I W
d wP P A
2.7 Experimental Comparison - Methods
64
Lecture 2: Camera Calibration
wP
Optical Ray
3Dd
2.7 Experimental Comparison - Accuracy Evaluation
• 3D Measurement
– Distance with respect to the optical ray
– Normalized Stereo Calibration Error
• 2D Measurement
– Accuracy of distorted image coordinates
– Accuracy of undistorted image
coordinates
1 22 2
2 2 21
ˆ ˆ1
NSCEˆ 12
C C C Cn
w w w wi i i i
Ci w u vi
X X Y Y
n Z
Camera
coordinate
system
World
coordinate
system
fCY
CX
WZ
WX
WO
wP
Image plane
{ }W
uP
dP
IX{ }I
RX
{ }R
RO
Image
coordinate
system
ˆuP
ˆdP
0 0,u v
WYCZ
CO { }C
IY
IO
RY
uP
ˆuP
0 0,u v
dd
Observed Point
Linear Projection
- distortion
+ distortion
ud
ˆdP
dP
65
Lecture 2: Camera Calibration
2.7 Experimental Comparison: Synthetic Images (I)
2D distorted image (pix.) 2D undistorted image (pix.)
Mean Standard
desviation
Max Min Mean Standard
desviation
Max Min
1 Hall 0.2676 0.1979 1.2701 0.0213 0.2676 0.1979 1.2701 0.0213
2 Faugeras 0.2689 0.1997 1.2377 0.0075 0.2689 0.1997 1.2377 0.0075
3 Faugeras with distortion 0.0840 0.0458 0.2603 0.0081 0.0834 0.0454 0.2561 0.0080
4 Tsai 0.0838 0.0457 0.2426 0.0035 0.0832 0.0453 0.2386 0.0035
5 Weng 0.0845 0.0455 0.2608 0.0019 0.0843 0.0443 0.2584 0.0129
2D distorted
0
0,05
0,1
0,15
0,2
0,25
0,3
1 2 3 4 5
pix
.
Mean Standard deviation
2D undistorted
0
0,05
0,1
0,15
0,2
0,25
0,3
1 2 3 4 5p
ix.
Mean Standard deviation
66
Lecture 2: Camera Calibration
2.7 Experimental Comparison: Synthetic Images (II)
3D position (mm) NSCE
Mean Standard
desviation
Max Min
1 Hall 0.1615 0.1028 0.5634 0.0113 n/a
2 Faugeras 0.1811 0.1357 0.8707 0.0147 0.6555
3 Faugeras NR with distortion 0.0566 0.0307 0.1694 0.0055 0.2042
4 Tsai optimized 0.0565 0.0306 0.1578 0.0087 0.2037
5 Weng 0.0570 0.0305 0.1696 0.0088 0.2064
Normalized Stereo Calibration Error
Normalized Stereo Calibration Error
0
0,1
0,2
0,3
0,4
0,5
0,6
0,7
1 2 3 4 5
NSCE
3D position
0
0,05
0,1
0,15
0,2
1 2 3 4 5
mm
.
Mean Standard deviation
67
Lecture 2: Camera Calibration
Computing Time
160 punts 1800 punts
• Hall 1 ms 70 ms
• Faugeras 1 ms 70 ms
• Faugeras with distortion 10 ms 380 ms
• Tsai 10 ms 530 ms
• Weng 51 ms 4216 ms
Pentium III at 1 GHz.
2.7 Experimental Comparison: Synthetic Images (III)
68
Lecture 2: Camera Calibration
2.7 Experimental Comparison: Real Images (I)
Camera
coordinate
system
CY
CX CZ
CO{ }C
IY
IXIO{ }I
RY
RX
RO
{ }R
World
coordinate
system
WZ
WYWX
WO{ }W
Reconstruction
Area
Image of the calibrating pattern
3D position (mm) NSCE
Mean Standard
desviation
Max Min
Hall 0.5219 0.2595 1.1370 0.0143 n/a
Faugeras 0.7782 0.4253 2.0210 0.0187 4.0649
Faugeras with distortion 0.4967 0.3367 1.5642 0.0094 2.5489
Tsai 0.4815 0.3023 1.4014 0.0093 2.4836
Weng 0.4740 0.2904 1.2669 0.0087 2.4556
69
Lecture 2: Camera Calibration
2.7 Experimental Comparison: Real Images (II)
3D position (mm) NSCE
Mean Standard
desviation
Max Min
Hall 1.5698 0.9842 8.9249 0.0247 n/a
Faugeras 1.6187 0.9856 8.8812 0.0302 2.0175
Faugeras with distortion 0.9930 0.5660 3.2386 0.0154 0.9909
Tsai 0.9927 0.5655 3.2311 0.0153 0.9908
Weng 0.9896 0.5724 3.3526 0.0149 0.9869
Image of the calibration patternStereo camera over a mobile robot
70
Lecture 2: Camera Calibration
2.7 Experimental Comparison - Conclusions
• Implementation of 5 of the most used camera calibration
methods
– Notation was unified
– The methods were compared in terms of model and
calibration
• The accuracy of non-linear methods is better than linear
methods
• Modelling of radial distortion is quite sufficient when high
accuracy is required
• Accuracy measuring methods obtain similar results if they are
relatively compared
Additional bibliography:
J. Salvi, X. Armangué and J. Batlle. A Comparative Review of Camera Calibrating Methods with Accuracy Evaluation. Pattern Recognition, PR, pp. 1617-1635, Vol. 35, Issue 7, July 2002.
top related