camera calibration to artoolkitkowon.dongseo.ac.kr/~lbg/web_lecture/imageprocessing/... ·...

Camera Calibration

to ARToolkit

2012.12.

[email protected]

Agenda

11/26/2012 [email protected] 2

Homogeneous Coordinates

Projective Transformation

Pinhole Camera Model

Camera Calibrations

Zhengyou Zhang

http://www.cs.unc.edu/~marc/

Homogeneous Coordinates

[email protected]

Homogeneous coordinates


0 cbyax Ta,b,c

0,0)()( kkcykbxka TTa,b,cka,b,c ~

Homogeneous representation of lines

equivalence class of vectors, any vector is representative

Set of all equivalence classes in R3(0,0,0)T forms P2

Homogeneous representation of points

0 cbyax Ta,b,cl x , ,1x yTon if and only if

0l 11 x,y,a,b,cx,y,T 0,1,,~1,, kyxkyx

TT

The point x lies on the line l if and only if xTl=lTx=0

Homogeneous coordinates

Inhomogeneous coordinates Tyx,

, ,x y zT

but only 2DOF

Points from lines and vice-versa


l'lx

Intersections of lines

The intersection of two lines and is l l'

Line joining two points

The line through two points and is x'xl x x'

Example

1x

1y

Ideal points and the line at infinity


T0,,l'l ab

Intersections of parallel lines

TTand ',,l' ,,l cbacba

Example

1x 2x

Ideal points T0,, 21 xx

Line at infinity T1,0,0l

l22RP

tangent vector

normal direction

ab ,

ba,

Note that in P2 there is no distinction

between ideal points and others

A model for the projective plane


exactly one line through two points

exactly one point at intersection of two lines

Duality


x l

0xl T0lx T

l'lx x'xl

Duality principle:

To any theorem of 2-dimensional projective geometry

there corresponds a dual theorem, which may be

derived by interchanging the role of points and lines in

the original theorem

Projective Transformations

[email protected]

Projective 2D Geometry


Projective transformations


A projectivity is an invertible mapping h from P2 to itself

such that three points x1,x2,x3 lie on the same line if and

only if h(x1),h(x2),h(x3) do.

Definition:

A mapping h:P2P2 is a projectivity if and only if there

exist a non-singular 3x3 matrix H such that for any point

in P2 reprented by a vector x it is true that h(x)=Hx

Theorem:

Definition: Projective transformation

3

2

1

333231

232221

131211

3

2

1

'

'

'

x

x

x

hhh

hhh

hhh

x

x

x

xx' Hor

8DOF

projectivity=collineation=projective transformation=homography

Mapping between planes


central projection may be expressed by x’=Hx

(application of theorem)

Removing projective distortion


333231

131211

3

1

'

''

hyhxh

hyhxh

x

xx

333231

232221

3

2

'

''

hyhxh

hyhxh

x

xy

131211333231' hyhxhhyhxhx

232221333231' hyhxhhyhxhy

select four points in a plane with know coordinates

(linear in hij)

(2 constraints/point, 8DOF 4 points needed)


http://www.youtube.com/watch?v=NOU1lXR4JKo

http://www.youtube.com/watch?v=-tuS3GbHtV8







http://www.youtube.com/watch?v=i7woG0pqFjs

http://www.youtube.com/watch?v=zFMOtzpkgQo





More Examples


Transformation for lines


Transformation for lines

ll' -TH

xx' HFor a point transformation

Isometries


1100

cossin

sincos

1

'

'

y

x

t

t

y

x

y

x

1

11

orientation preserving:

orientation reversing:

x0

xx'

1

tT

RHE IRR T

special cases: pure rotation, pure translation

3DOF (1 rotation, 2 translation)

Invariants: length, angle, area

Similarities


1100

cossin

sincos

1

'

'

y

x

tss

tss

y

x

y

x

x0

xx'

1

tT

RH

sS IRR T

also know as equi-form (shape preserving)

metric structure = structure up to similarity (in literature)

4DOF (1 scale, 1 rotation, 2 translation)

Invariants: ratios of length, angle, ratios of areas, parallel lines

Affine transformations


11001

'

'

2221

1211

y

x

taa

taa

y

x

y

x

x0

xx'

1

tT

AH A

non-isotropic scaling! (2DOF: scale ratio and orientation)

6DOF (2 scale, 2 rotation, 2 translation)

Invariants: parallel lines, ratios of parallel lengths, ratios of areas

DRRRA

2

1

0

0

D

Projective transformations


xv

xx'

vP T

tAH

Action non-homogeneous over the plane

8DOF (2 scale, 2 rotation, 2 translation, 2 line at infinity)

Invariants: cross-ratio of four points on a line (ratio of ratio)

T21,v vv

vv

sPAS TTTT v

t

v

0

10

0

10

t AIKRHHHH

Ttv RKA s

K 1det Kupper-triangular, decomposition unique (if chosen s>0)

Overview Transformations


100

2221

1211

y

x

taa

taa

100

2221

1211

y

x

tsrsr

tsrsr

333231

232221

131211

hhh

hhh

hhh

100

2221

1211

y

x

trr

trr

Projective

8dof

Affine

6dof

Similarity

4dof

Euclidean

3dof

Concurrency, collinearity,

order of contact (intersection,

tangency, inflection, etc.),

cross ratio

Parallellism, ratio of areas,

ratio of lengths on parallel

lines (e.g midpoints), linear

combinations of vectors

(centroids).

The line at infinity l∞

Ratios of lengths, angles.

The circular points I,J

lengths, areas.

Line at infinity


2211

2

1

2

1

0v

xvxv

x

x

x

x

v

AAT

t

000 2

1

2

1

x

x

x

x

v

AAT

t

Projective Transformation : Line at infinity becomes finite,

allows to observe vanishing points, horizon,

Affine Transformation : Line at infinity stays at infinity

The line at infinity


v1 v2

l1

l2 l4

l3

l∞

21 vvl

211 llv

432 llv

The line at infinity l is a fixed line under a projective transformation H if and only if H is an affinity

l

1

0

0

1t

0ll

A

AH

TT

A

Recovery of affine properties from images


Projective transformation Affine rectification

Affine transformation

Affine Rectification


v1 v2

l1

l2 l4

l3

l∞

21 vvl

211 llv

432 llv

Projective 3D geometry


vTv

tAProjective

15dof

Affine

12dof

Similarity

7dof

Euclidean

6dof

Intersection and tangency

Parallellism of planes,

Volume ratios, centroids,

The plane at infinity π∞

The absolute conic Ω∞

Volume

10

tAT

10

tRT

s

10

tRT


[email protected]



TT ZfYZfXZYX )/,/(),,(

101

0

0

1

Z

Y

X

f

f

Z

fY

fX

Z

Y

X

101

01

01

1Z

Y

X

f

f

Z

fY

fX



camX0|IKx

1

y

x

pf

pf

K

Principal point offset Calibration Matrix

Internal Camera Parameters


Camera rotation and translation


C~

-X~

RX~

cam

X10

RCR

1

10

C~

RRXcam

Z

Y

X camX0|IKx

XC~

|IKRx

t|RKP C~

Rt

PXx

Camera Parameter Matrix P


CCD Cameras


Correcting Radial Distortion of Cameras


Calibration Process


Correcting Radial Distortion of Camera with

Rotation Angle


2 4 6

1 2 3

2 4 6

1 2 3

( )(1 )cos

( )(1 )sin

u x d x d d d

d y d d d

x c x c k r k r k r

y c k r k r k r

2 4 6

1 2 3

2 4 6

1 2 3

( )(1 )sin

( )(1 )cos

u y d x d d d

d y d d d

y c x c k r k r k r

y c k r k r k r

0 1 2

6 7 1

u up

u u

m x m y mx

m x m y

3 4 5

6 7 1

u up

u u

m x m y my

m x m y

Camera Calibration

[email protected]

Camera Models


“An empirical evaluation of factors influencing camera calibration accuracy using three publicly

available techniques”,Wei Sun and Jeremy R. Cooperstock

Machine Vision and Applications Volume 17, Number 1 (2006), 51-67,

Roger Y. Tsai


"An Efficient and Accurate Camera Calibration Technique for 3D Machine Vision", Roger Y. Tsai,

Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, Miami Beach, FL,

1986, pages 364-374,

Zhengyou Zhang


Z. Zhang. Flexible Camera Calibration By Viewing a Plane From Unknown Orientations. International Conference on Computer Vision, Corfu, Greece, pages 666-673, September 1999.

Basic Equations


Solving Camera Calibration


Zhengyou Zhang


Zhengyou Zhang


http://research.microsoft.com/en-us/um/people/zhang/

Camera Calibration Toolbox


http://www.vision.caltech.edu/bouguetj/calib_doc/

Camera Calibration


Calibrating a Stereo System


Robust Multi-camera Calibration


http://graphics.stanford.edu/~vaibhav/projects/calib-

cs205/cs205.html

Multi-Camera Self Calibration


http://cmp.felk.cvut.cz/~Esvoboda/

Multi-Camera Self Calibration


ARToolKit Camera Calibration


http://www.hitl.washington.edu/artoolkit/

OpenCV


http://opencv.willowgarage.com/documentation/

camera_calibration_and_3d_reconstruction.html


http://www.youtube.com/watch?v=DrXIQfQHFv0




http://www.youtube.com/watch?v=O6PEFJhsCEo



OpenCV 2.1 Camera Calibration


http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html





http://mmlab.disi.unitn.it/wiki/index.php/Camera_Calibration_Tool_in_OpenCV

http://mmlab.disi.unitn.it/wiki/index.php/Camera_Calibration_Tool_in_OpenCV



xcorrected = x + [2P1xy + P2(r2 + 2x

2)]

ycorrected = y + [P1(r2 + 2y

2) + 2P2xy]

Using the Polynomial Distortion Model to Correct Tangential Distortion The polynomial distortion model uses two parameters, P1 and P2, to characterize tangential distortion.

The distortion model for tangential distortion can be represented as:

http://zone.ni.com/reference/en-XX/help/372916L-01/nivisionconcepts/spatial_calibration_indepth/

xcorrected = x(1+ K1r2+ K2r

4 + K3r

6 + Knr

(n × 2))

ycorrected = y(1+ K1r2+ K2r

4 + K3r

6 + Knr

(n × 2))

Brown, D. C. Decentric distortion of lenses.

Journal of Photogrammetric Engineering and Remote Sensing, 32(3), 444-462, 1966.









http://www.nowpublishers.com/product.aspx?product=CGV&doi=0600000001&section=x1-10r1




calibrateCamera


solvePnP


findChessboardCorners


drawChessboardCorners


rodrigues2



http://dprg.geomatics.ucalgary.ca/system/files/AKAM_ENGO667_CH_1Handouts.pdf



Camera Calibration

[email protected]

Capture chessboard image


Chessboard no of row and col

10 : no of calibration images

Auto : automatically add images after 3*150 milliseconds

Start : start calibration

Calibration parameters


Focal length f_x, f_y

Center c_x, c_y

Radial : k1, k2, k3(check box)

Tangential : p1, p2

Und : Undistorted live image

Debug images


Filenames


Capture072309511500

month day hour min sec index

Corner072309511500

Image0723095115.xml

Object0723095115.xml

Distortion0723095115.xml

Intrinsics0723095115.xml

Image & Object xml


<?xml version="1.0"?>

<opencv_storage>

<Image0723095127 type_id="opencv-matrix">

<rows>480</rows>

<cols>2</cols>

<dt>f</dt>

<data>

2.25150803e+002 1.92720139e+002 2.42047302e+002 1.88396301e+002

2.59520660e+002 1.83857040e+002 2.77741913e+002 1.79775314e+002

2.96598541e+002 1.75898483e+002 3.15514801e+002 1.72283661e+002


<opencv_storage>

<Object0723095127 type_id="opencv-matrix">

<rows>480</rows>

<cols>3</cols>

<dt>f</dt>

<data>

0. 1.37500000e+001 0. 2.75000000e+000 1.37500000e+001 0.

5.50000000e+000 1.37500000e+001 0. 8.25000000e+000 1.37500000e+001

0. 11. 1.37500000e+001 0. 1.37500000e+001 1.37500000e+001 0.

Distortion & Intrinsics xml



<opencv_storage>

<Distortion0723095127 type_id="opencv-matrix">

<rows>4</rows>

<cols>1</cols>

<dt>f</dt>

<data>

-4.46110874e-001 2.08227739e-001 9.08007473e-003 -3.17067979e-003</data></Distortion0723095127>

</opencv_storage>


<opencv_storage>

<Intrinsics0723095127 type_id="opencv-matrix">

<rows>3</rows>

<cols>3</cols>

<dt>f</dt>

<data>

2.66292694e+002 0. 3.24350769e+002 0. 2.58133820e+002

2.33235382e+002 0. 0. 1.</data></Intrinsics0723095127>

</opencv_storage>

camera calibration to artoolkitkowon.dongseo.ac.kr/~lbg/web_lecture/imageprocessing/... ·...

Documents