Spring2016ENGN2502---3DPhotography

Lecture7

GabrielTaubinBrownUniversity

Desktop3DPhotography

•  http://www.vision.caltech.edu/bouguetj/ICCV98/.index.html

3Dtriangulation:ray-planeIntersection

ray

plane

intersectionpoint

projector/

coordinate

systemsworld

coordinate system

3DTriangulationbyRay-Planeintersection

object being

scanned

pq n

projected light plane

p

illuminated

point on object

camera ray

v

intersection

of light plane

with object

world coordinate

system

image plane

center of projection

ray

direction

vector

Ifcameraandprojectorarecalibrated

object being

scanned

pq n

projected light plane

p

illuminated

point on object

camera ray

vintersection

of light plane

with object

common

world

coordinate

systemcenter of projection

ray direction

vector from

detected pixel

Lq

3DTriangulationbyRay-RayIntersection

object being

scanned

camera ray2v

projected

light ray

p

lines may not intersect !

Implicitequationoftheplane

pq n

}0)(:{ =−= pt qpnpP

v

world coordinate

system Plane normal vector

(unit length)

A fixed point

on the plane

p

Parametricequationoftheray

pq n

L = {p = qL +λv :λ > 0}

camera ray

Lq

v

world coordinate

system

ray direction

vector

(unit length)

p

v

TriangulationbyLine-PlaneIntersection

world coordinate

system

}{ vqpL L λ+==

camera ray

Lqv

Compose implicit and

parametric equations

p

pq n

}0)(:{ =−= pt qpnpP

projected light plane

TriangulationbyLine-PlaneIntersection

world coordinate

system

}{ vqpL L λ+==

camera ray

Lqv

Replace computed λ

in parametric equation

p

pq n

}0)(:{ =−= pt qpnpP

projected light plane

TriangulationbyLine-PlaneIntersection

world coordinate

system

}{ vqpL L λ+==

camera ray

Lqv

Eliminate parameter λ

p

pq n

}0)(:{ =−= pt qpnpP

projected light plane

TriangulationbyLine-LineIntersection

object being

scanned

}{ 2222 vqpL λ+==camera ray

2q

2v

projected light ray

1q1v

}{ 1111 vqpL λ+==

p

lines may not intersect !

TriangulationbyLine-LineIntersection

L2 = {p2 = q2 +λ2v2}

2q2v1q

1v

L1 = {p1 = q1 +λ1v1}

p1 = q1 +λ1v1

p2 = q2 +λ2v2

p1 − p2

v1t (p1 − p2 ) = 0v2t (p2 − p1) = 0

E(λ1,λ2 ) = dist(p2 − p1)2

Minimize

Necessary conditions

p = (p1 + p2 ) / 2

p

Approximate Line-Line Intersection

1q1v

2q

2v

1111 vqp λ+=

2222 vqp λ+=

),( 2112 λλp

Midpoint of segment joining

arbitrary points in the two lines

1q1v

2q

2v

),( 2112 λλp

Least-squares approach

Find parameters which minimize

Approximate Line-Line Intersection

1111 vqp λ+=

2222 vqp λ+=

3DLaserScanner

Whatistheequationoftheplane

inthecameracoordinatesystem?

Cameraandlaserareattached:

usecameracoordinatesystem

Plane defined by image line and center of projection

center of projection

qn

image plane

}0)(:{ =−= qpnpP t

}0:{ == uluL tImplicit equation of line in image coordinates

Triangulation by Laser Striping

M. J. Leotta, A. Vandergon, and G. Taubin. 3D Slit Scanning With Planar Constraints. Computer Graphics Forum, 2008

§  Manuallyormechanicallytranslatedlaserstripe

§  Per-pixeldepthbyray-planetriangulation

§  Requiresaccuratecameraandlaserplanecalibration

§  PopularsolutionforcommercialandDIY3Dscanners

3D Photography on Your Desk: Bouguet and Perona [ICCV 1998]

J.-Y. Bouguet and P. Perona. 3D photography on your desk. Intl. Conf. Comp. Vision, 1998

§  DIYscannerusingonlyacamera,ahalogenlamp,andastick

§  Per-pixeldepthbyray-planetriangulation

§  Requiresaccuratecameraandshadowplanecalibration

3D Photography on Your Desk: Bouguet and Perona [ICCV 1998]

J.-Y. Bouguet and P. Perona. 3D photography on your desk. Intl. Conf. Comp. Vision, 1998

Assembling Your Own Scanner

§  Parts:camera(QuickCam9000),lamp,stick,twoplanarobjects[~$100]

§  Step1:Buildthecalibrationboards(includefiducialsandchessboard)

§  Step2:Buildthepointlightsource(removereflectorandplaceinscene)

§  Step3:Arrangethecamera,lightsource,andcalibrationboards

Assembling Your Own Scanner

§  Parts:camera(QuickCam9000),lamp,stick,twoplanarobjects[~$100]

§  Step1:Buildthecalibrationboards(includefiducialsandchessboard)

§  Step2:Buildthepointlightsource(removereflectorandplaceinscene)

§  Step3:Arrangethecamera,lightsource,andcalibrationboards

Assembling Your Own Scanner

§  Parts:camera(QuickCam9000),lamp,stick,twoplanarobjects[~$100]

§  Step1:Buildthecalibrationboards(includefiducialsandchessboard)

§  Step2:Buildthepointlightsource(removereflectorandplaceinscene)

§  Step3:Arrangethecamera,lightsource,andcalibrationboards

Assembling Your Own Scanner

§  Parts:camera(QuickCam9000),lamp,stick,twoplanarobjects[~$100]

§  Step1:Buildthecalibrationboards(includefiducialsandchessboard)

§  Step2:Buildthepointlightsource(removereflectorandplaceinscene)

§  Step3:Arrangethecamera,lightsource,andcalibrationboards

Swept-Plane Reconstruction Geometry

CΧ1

2

3

)(tlΠ

),( yxCΛ

)(),( tyxP lC Π∩Λ=

P

p

)(tlΠ),( yxCΛ

Demo: Data Capture

Imax (x, y) = maxtI(x, y, t)

§  ConvertfromRGBtograyscale(forluminance-domainprocessing)

§  Determineper-pixelminimumandmaximumvalueoversequence

Video Processing: Assigning Per-Pixel Shadow Thresholds

Imin(x, y) = mintI(x, y, t)

Ishadow(x, y) =I m a x ( x ;y ) + I m i n ( x ;y )

2

§  ConvertfromRGBtograyscale(forluminance-domainprocessing)

§  Determineper-pixelminimumandmaximumvalueoversequence

§  Evaluateper-pixel�shadowthreshold�asaverageofmin.andmax.

Video Processing: Assigning Per-Pixel Shadow Thresholds

400 600 800

-60

-40

-20

0

20

40

60

column index

I 60(x

,y) -

I sha

dow(x

,y)

Video Processing: Spatial Shadow Edge Localization

§  Selectregionofinterestoneachcalibrationplane(occlusion-free)

§  Estimatezero-crossingstofindleadingandtrailingshadowboundaries

§  Fitalinetothesetofpointsalongeachshadowboundary

èResult:Best-fit2Dlinesforeachshadowedge(inimagecoordinates)

540 550 560 570 580

-60

-40

-20

0

20

40

column index

I 60(x

,y) -

I sha

dow(x

,y)

Video Processing: Temporal Shadow Edge Localization

80 100 120 140-40

-20

0

20

40

frame index

I(x,y

,t) -

I sha

dow(x

,y)

§  Tabulateper-pixeltemporalsequence(minusshadowthreshold)

§  Estimatezero-crossingstofindshadow-crossingtimes

èResult:Useshadow-crossingtimetolookupcorresponding3Dplane

crossing frame index for leading shadow crossing frame index for trailing shadow

fram

e in

dex

80

100

120

140

Intrinsic Camera Calibration

1 12

23

3

4

Estimated Camera Lens Distortion Map Camera Calibration Input

§  Howtoestimateintrinsicparametersanddistortionmodel?(unknowns:

focallength,skew,scale,principalpoint,anddistortioncoeffs.)

§  Popularsolution:Observeaknowncalibrationobject(Zhang[2000])

§  Each2Dchessboardcorneryieldstwoconstraintsonthe6-11unknowns

§  But,mustalsofind6extrinsicparametersperimage(rotation/translation)

§  èResult:Twoormoreimagesofachessboardaresufficient

camera coordinate system

CΧ

12 3 u

world coordinate system

WΧ

1 2

3p

TRXWC +=Χ

intrinsic parameters extrinsic parameters

Demo: Camera Calibration in Matlab

J.-Y. Bouguet. Camera Calibration Toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/

Extrinsic Camera Calibration

CΧ1

2

3

WΧ 12

3

?},{ TRTRXWC +=Χ

hΠ

vΠvΧ 1

2

3

vvvC TXR +=Χ

hΧ1

23

hhhC TXR +=Χ

w h

Demo: Mapping Pixels to Optical Rays

CΧ 12

3

),( yxCΛ

p

)( TRPKp +=λ

P

n

§  Howtomapanimagepixeltoanopticalray?

§  Solution:Invertthecalibratedcameraprojectionmodel

§  But,alsorequiresinversionofdistortionmodel(whichisnon-linear)

§  MappingimplementedinCameraCalibrationToolboxwithnormalize.m

èResult:Aftercalibration,pixelscanbeconvertedtoopticalrays

Shadow Plane Calibration

CΧ1

2

3

)(tlΠ

hΠ

vΠ

)(tvΛ

)(thλ

)(tvλ

)(tn

)()()( tttn vh Λ⊗Λ=)(thλ

)(tvλ

)(thΛ

)(tlΠ

)()()( tttP vh Λ∩Λ=

)(tP

Alternatives for Shadow Plane Calibration

J.-Y. Bouguet and P. Perona. 3D photography on your desk. Intl. Conf. Comp. Vision, 1998

CΧ1

2

3b st

B

h

sT

TL