image acquisition and projective geometry

Ioannis Rekleitis

ImageAcquisitionand

ProjectiveGeometry

Review: Linear Algebra• MatrixAddition• MatrixMultiplicationA*B• Matrix-VectorMultiplicationA*v• MatrixTransposeAT,(AB)T=ATBT

• MatrixInverseA-1• IdentityMatrixI3=

CSCE-574 Robotics 2

1 0 00 1 00 0 1

Dot Product

CSCE 590: Introduction to Image Processing 3

f

v1

v2

V1V2= |V1|*|V2|*cos(f)

Cross Product


f

v1

v2

V1XV2= |V1|*|V2|*sin(f)*v

Review: Coordinate Systems and Orientation Representations


Position Representation

• Positionrepresentationis:

CSCE 574: Robotics 6

X

Y

Z

úúú

û

ù

êêê

ë

é=

z

y

xA

ppp

PPA

Orientation Representations

• Describestherotationofonecoordinatesystemwithrespecttoanother


XA

YA

ZA

XB

YB

ZB

A

B

Rotation Matrix• WritetheunitvectorsofB inthecoordinatesystemofA.

• RotationMatrix:


XA

YA

ZA

A

AXB

AYB

AZB

B[ ]

úúú

û

ù

êêê

ë

é

×××××××××

=

úúú

û

ù

êêê

ë

é==

ABABAB

ABABAB

ABABAB

BA

BA

BAA

B

ZZZYZXYZYYYXXZXYXX

rrrrrrrrr

ZYXR

ˆˆˆˆˆˆˆˆˆˆˆˆˆˆˆˆˆˆ

ˆˆˆ

333231

232221

131211

Properties of Rotation Matrix


TBA

BA

AB

AB

TAB

TAB

BA

RRRIRR

RR

==

=

=

-13

Coordinate System Transformation


úû

ùêë

é=

úúúú

û

ù

êêêê

ë

é

=´ 10

100013333231

232221

131211

TRprrrprrrprrr

Mz

y

x

whereR istherotationmatrixandT isthetranslationvector

Hierarchy of Transformations 2D


Transformation Matrix #DoF PreservesTranslation [I|t]2x3 2 orientationRigid [R|t]2x3 3 lengthsSimilarity [sR|t]2x3 4 anglesAffine [A]2x3 6 parallelismProjective [H]3x3 8 Straightlines

The above transformations apply to a vector x=[x,y,1]T

Hierarchy of Transformations 2D


Transformation Matrix T*x

Translation [I|t]2x3 x=x+txy=y+ty

Rigid [R|t]2x3 x=cos(f)x-sin(f)y+txy=sin(f)x+cos(f)y+ty

Similarity [sR|t]2x3 x=ax-by+txy=bx+ay+ty

Affine [A]2x3 A isarbitraryProjective [H]3x3 H is3by3

3D Transformations


Transformation Matrix #DoF PreservesTranslation [I|t]3x4 3 orientationRigid [R|t] 3x4 6 lengthsSimilarity [sR|t] 3x4 7 anglesAffine [A] 3x4 12 parallelismProjective [H]4x4 15 Straightlines

The above transformations apply to a vector x=[x,y,z,1]T

Rotation Matrix

• The rotation matrix consists of 9 variables, butthere are many constraints. The minimumnumber of variables needed to describe arotation is three.


Rotation Matrix-Single Axis


( ) ( ) ( )( ) ( )

( )( ) ( )

( ) ( )

( )( ) ( )( ) ( )

úúú

û

ù

êêê

ë

é -=

úúú

û

ù

êêê

ë

é

-=

úúú

û

ù

êêê

ë

é-=

1000cossin0sincos

cos0sin010

sin0cos

cossin0sincos0001

qqqq

q

qq

qqq

qqqqq

z

y

x

R

R

R

Fixed Angles

• Onesimplemethodistoperformthreerotationsabouttheaxisoftheoriginalcoordinateframe:– X-Y-Zfixedangles

• Thereare12differentcombinations


( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( ) úúú

û

ù

êêê

ë

é

-+++-

=

=

yqqffqyqfyqyqfyfyqyqfyqyqfyfy

qfyyfq

coscossincossinsincoscossinsincoscossinsinsincossinsinsincossincoscossinsinsincoscoscos

,, xyzAB RRRR

Inverse Problem

• FromaRotationmatrixfindthefixedanglerotations:


( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( ) ( )

( ) ( ) ( ) ( ) ( )

( )( )( ) ( )

( ) ( )÷øöç

èæ=

÷øöç

èæ=

+-=

úúú

û

ù

êêê

ë

é=

úúú

û

ù

êêê

ë

é

-+++-

Þ=

ffq

ffy

f

yqqffqyqfyqyqfyfyqyqfyqyqfyfy

yfq

cos,cos2tan

cos,cos2tan

,2tan

:thuscoscossincossin

sincoscossinsincoscossinsinsincossinsinsincossincoscossinsinsincoscoscos

,,

3332

1121

221

21131

333231

232221

131211

rrA

rrA

rrrA

rrrrrrrrr

RR AB

AB

Euler Angles

• ZYX:Startingwiththetwoframesaligned,firstrotateabouttheZB axis,thenbytheYB axisandthenbytheXB axis.TheresultsarethesameaswithusingXYZfixedanglerotation.

• Thereare12differentcombinationofEulerAnglerepresentations


Euler Angles

• TraditionallythethreeanglesalongtheaxisarecalledRoll,Pitch,andYaw


X

Y

Z

Euler Angles


Roll


X

Y

Z

Euler Angles


Pitch


X

Y

Z

Euler Angles


Yaw


X

Y

Z

Euler Angle concerns: Gimbal Lock


UsingtheZYZ convention•(90°,45°,−105°)≡(−270°,−315°,255°) multiplesof360°•(72°,0°,0°)≡(40°,0°,32°) singularalignment(Gimbal lock)• (45°,60°,−30°)≡(−135°,−60°,150°) bistable flip

Axis-Angle Representation

• Representanarbitraryrotationasacombinationofavectorandanangle


X

Y

Z

Vq

Quaternions• Aresimilartoaxis-anglerepresentation• Twoformulations– Classical– BasedonJPL’sstandardsW.G.Breckenridge,“Quaternions - ProposedStandardConventions,”JPL,Tech.Rep.INTEROFFICEMEMORANDUMIOM343-79-1199,1999.

• AvoidsGimbal lock

• Seealso:M.D.Shuster,“Asurveyofattituderepresentations,”JournaloftheAstronautical Sciences,vol.41,no.4,pp.439–517,Oct.–Dec.1993.


QuaternionsClassic notation JPL-based


kqjqiqqq 3214 +++=

jikkiikjjkkjiijkji

==-==-==--===

,,1222

jikkiikjjkkjiijijkkji

=-==-==-=-====

, ,1222

kqjqiqqq 3214 +++=

( )( )( )

( )2cos ,

2sin2sin2sin

, 44

q

q

q

q

=

úúúú

û

ù

êêêê

ë

é

=úû

ùêë

é= q

k

k

k

qq

z

y

x

qq

ë û´´Ä= qpq , , , ,1 Iqpqq

( )( ) ( )( ) ( )( ) ( )ú

úúú

û

ù

êêêê

ë

é

=úúú

û

ù

êêê

ë

é=

úúúú

û

ù

êêêê

ë

é

=

z

y

x

qqq

q

qqqq

q

bqbqbq

q

cos2sin

cos2sin

cos2sin

,2cos ,

3

2

1

0

3

2

1

0

Vect

orNo

tatio

n

See also: N. Trawny and S. I. Roumeliotis, “Indirect Kalman Filter for 3D Attitude Estimation,” University of Minnesota, Dept. of Comp. Sci. & Eng., Tech. Rep. 2005-002, March 2005.

Coordinate frames on PR2


Image Formation in the Eye


Image is upside down in the retina/imaging plane!

Adjust focus length• Camera • Human eye

Slides courtesy of Prof. Yan Tong

center of projection

focal length

object

image plane

Camera Geometry

3Dà2Dtransformation:perspectiveprojection


canonicalaxes

f

pixelcoordinates

opticalaxis

Addcoordinatesystemsinordertodescribefeaturepoints...

z

xy

uv

objectcoordinates

v(row)

u(col)

principalpoint

Z

x

y

attheC.O.P.

Coordinate Systems


f

pixelcoordinates

z

xy

uv

objectcoordinates

(X,Y,Z)incanonicalcoordinatessimagecan.coords:(x,y)

canonicalaxes

Coordinate Systems


x=f XZ y=

f YZ

a nonlinear transformation

f

pixelcoordinates

z

xy

uv

objectcoordinates

(X,Y,Z)incanonicalcoordsimagecan.coords:(x,y)

canonicalaxes

goal:torecoverinformationabout(X,Y,Z)from(x,y)

From 3d to 2d


Lens Parameters


Thin lens theory: •Increasing the distance from the object to the lens will reduce the size of image•Large focus length will give a small FOV

Image plane

P

p

OF

f

d

S1 S2

FOV

FOV


Depth of Field & Out of Focus


http://www.azuswebworks.com/photography/dof.html

P

pO

F

P1 P2

p1

p2

Image plane

• DOF is inversely proportional to the focus length

• DOF is proportional to S1


Camera Calibration

• CameraModel– [uv1]Pixelcoords– Worldcoords

• IntrinsicParameters– focallengthsinpixels– skewcoefficient– focalpoint

• ExtrinsicParameters– RotationandTranslation


[ ]úúúú

û

ù

êêêê

ë

é

=úúú

û

ù

êêê

ë

é

11 w

w

w

c zyx

TRAvu

z

[ ]Twww zyx 1

úúú

û

ù

êêê

ë

é=

1000

0

oy

x

vu

A agayyxx mfmf ×=×= aa ,

g

ovu ,0

[ ]TR

Camera Calibration


ExistingpackagesinMATLAB,OpenCV,etc

Rectified Image Sample


Unrectified Rectified

From Clearpath Husky Axis M1013 camera




From Parrot ARDrone 2.0 front camera




From GoPro HERO3+ at Barbados 2015 Field Trials

ReRectified Image Sample


Rectified ReRectified

From Aqua front camera at Barbados 2013 Field Trials

image acquisition and projective geometry

Documents