29 april 2008birkbeck college, u. london1 geometric model acquisition steve maybank school of...

29 April 2008 Birkbeck College, U. London 1

Geometric Model Acquisition

Steve MaybankSchool of Computer Science and Information

SystemsBirkbeck College

London, WC1E 7HX

Edited version of the slides for the VVG Summer School, held at the University of Bath

21 September 2007


Geometric Model Acquisition

Aim: make a 3D model of a scene from two or more images taken from different viewpoints.

Why is it possible: the image differences depend in part on the shapes of the objects in the scene.


Two Images of the Same Scene

http://vasc.ri.cmu.edu/idb/images/stereo/fruitSOURCE "University of Illinois, Bill Hoff“DESCRIPTION "Fruit on table, digitized from 35mm."


Two Images of a Point in R3

• p

c1 •

c2•

q1 •

q2 •

Epipolar plane: <c1,c2,x>

Image 1

Image 2

objectpoint

opticalcentre

opticalcentre


Corresponding Points Points in different images

correspond,qq ~,

qq ~ if they are projections of the same

scene point p. In projective coordinates, projection is a matrix application,

qpM

qMp~~


Method for Finding Corresponding Points

oods.neighbourh levelgrey similar have ,~ i.e. large, is

~

~.~.~.

~.~,

ncorrelatio theif ~ thatassumed isIt .~, of oodsneighbourh levelgrey be ~,Let

2/12/1

qq

uu

uu

uuuu

uuuu

qq

qquu


Example 1 of Correlation Based Matching

Points in lh image =(150,100), (250,150), (350,250), (450,350), (250,450)Correlations (ρ) = 0.750, 0.685, 0.912, 0.644, 0.691Search area = (2d+1)x(2d+1) box, d=20.


What Do We Need for GAM?

Description of image formation in the camera.

Description of the relative positions of the cameras.

Equations involving the measurements, the scene points and the relative positions of the cameras.

Statistical description of the errors in the measurements.


Pinhole Camera

Light tight box

Small hole (optical centre)

Viewingscreen(image)

Object

Light rays

Central perspective projection model for imageformation (Brunelleschi, 15th C.).


Camera Coordinate Frame

(X,Y,Z)(0,0,0)(0,0,-f)

Origin (0,0,0) at the pin hole.Focal length of the camera = f.Axes of image coordinate frame are parallel to X, Y axes

of the CCF.Image point = (-Xf/Z, -Yf/Z)

•

•

• Z

YXx

y


Mathematical Version of the Camera Coordinate Frame

(X,Y,Z)

(0,0,0)

(0,0,f)

Origin (0,0,0) at the pin hole.Focal length of the camera = f.The image is in front of the pin hole!Image point = (Xf/Z, Yf/Z). The minus signs have gone.

•

•

• Z

yxX

Y Image plane

•


Relative Position of the Cameras

Y X~

ZX Z~

Y~

The relative position of the cameras is describedby an orthogonal matrix R and a translation vector t.

R, t


Transformation of Coordinates

YX~

ZX

Z~

Y~

If a point p has coordinates (X,Y,Z)T in the first CCF,then in the second CCF the same point p has coordinates

R, t

● p

t

Z

Y

X

R


Properties of Orthogonal Matrices

angle.rotation the

for one and axis for the two:freedom of degrees threehave rotations The

100

0cossin

0sincos

:matrixrotation a of Example

.1det ifonly and ifrotation a represents matrix orthogonal The

.1det that followsIt

.

100

010

001

ifonly and if orthogonal is matrix 3x3 The

R

RR

R

IRRR T


Projection Ray

Tfyx ,,point Image

Y

X

Z

●

0,,, :ray Projection Tfyx

Any scene point projecting to (x, y, f)T is on the projection ray.

CCF


Projection Rays of Corresponding Points 1

●

The projection rays of corresponding points intersect ata scene point. Geometric model acquisition is based onthis single constraint.

For an extreme example, see http://www.wisdom.weizmann.ac.il/~vision/VideoAnalysis/Demos/Traj2Traj/hall.htm


Projection Rays of Corresponding Points 2

Tfyx ,, Tfyx ~

,~,~~ ●

The equations of the projectionrays are known, but they hold indifferent coordinate systems.

f

y

x

f

y

x

~~

~


Transformation of Coordinates

by CCF second in thegiven is ,,ray The Tfyx

t

f

y

x

R

such that ~, numbers realexist therethus

.~~

~

~

f

y

x

t

f

y

x

R

,~

,~,~ith equation w theofproduct scalar theby taking eliminated are ~, numbersunknown TheT

fyxt

.0.~~

~

f

y

x

R

f

y

x

t


The Essential Matrix

.0~ thatfollowsIt .by matrix essential theDefine

.~~0

thus

0.~ then~ If

.~~such that matrix 33 theDefine

Eqq

RTEE

RqTqRqTq

RqqT

qq

tqqT

T

T

t

tTT

tT

t

t

t


Model Acquisition

matrix essentialan find ,1,~ points matchingGiven Niiqiq

.1 ,0~such that NiiEqiqE T

such that points find and from ,Recover ixEtR

,1 ,~~NiiqitiRqiix

camera. second theof CCF in the


Naïve Estimates of E

9. parameters ofNumber .1 subject to

,~

minimised, is expression following thefor which an Find

1

2

E

iEqiq

EN

i

T

6. parameters ofNumber .1 and subject to

,~

minimised, is expression following thefor which , Find

1

2

tIRR

iRqTiq

tR

T

N

it

T


Better Way of Estimating E

minimiseThen small. are

~ ,

and tsmeasuremen ueunknown tr theare ~ , where

,1 ,~~~

,1 ,

Write

ii

iaia

Niiiaiq

Niiiaiq

.1 , and ,1 ,0~ subject to

,~

1

22

tIRRNiiRaTia

ii

Tt

T

N

i


Geometric Picture

i i~ iq iq~● ●

First image Second image lEll ,

~~

l l~


Camera Calibration

Ideal pixelcoordinates

Measured pixelcoordinates

Ideal CCF

Camera calibration is a transformation from measured pixelcoordinates to ideal pixel coordinates.


Calibration Matrix

plane. image theofation transformaffinean defines matrix The

1,, scoordinate pixel ideal

1,, scoordinate pixel measured

100

0

21

21

K

Krq

qqq

rrr

v

us

K

T

v

u


Fundamental Matrix

.0~therefore

~~~ ,

0~

~ :points Matching

EKrKr

rKqKrq

Eqq

qq

TT

T

The fundamental matrix F is defined by

EKKF T~


Properties of E and F det(E)=det(Tt)det(R)=0 The matrix E is essential iff

SingularValues(E) = (σ,σ,0)

det(F)=det(K~)det(E)det(K)=0 The matrix F is fundamental iff

det(F)=0.


Minimal Data

.)identified are 0 with , (Solutions

solutions. 10 have ,51 ,0~ equations thegeneral,In

.532 is of freedom of degrees ofNumber

EE

iiEqiq

ET

solutions. 3 have ,70~ equations thegeneral,In

7.2-9 is of freedom of degrees ofNumber

iiFqiq

FT


Books

D.A. Forsyth and J. Ponce. Computer Vision: a modern approach. Prentice Hall, 2003.

R.C. Gonzalez and R.E. Woods. Digital Image Processing. Second edition, Prentice Hall, 2002.

29 april 2008birkbeck college, u. london1 geometric model acquisition steve maybank school of...

Documents