structured lighting - nyu tandon school of...
TRANSCRIPT
![Page 1: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/1.jpg)
Structured LightingGuido Gerig
CS 6643, Spring 2016(credits: slides S. Narasimhan CMU, Marc
Pollefeys UNC)
http://mesh.brown.edu/byo3dhttp://www.cs.cmu.edu/afs/cs/academic/class/15385-
s06/lectures/ppts/lec-17.ppt
![Page 2: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/2.jpg)
Passive triangulation: Stereo vision
• Correspondence problem
• Geometric constraints
search along epipolarlines
• 3D reconstruction of matched pairs by triangulation
![Page 3: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/3.jpg)
Real-Time 3D Model Acquisition
Link:
http://graphics.stanford.edu/papers/rt_model/
http://graphics.stanford.edu/pap
ers/rt_model/
The SIGGRAPH Paper:
Full paper as PDF.
One-page abstract and Figure 1
as PDF.
Two-page abstract and Figure 1
as PDF.
A 5-minute video describing the
system:
AVI file, 640 x 480 pixels
(19MB)
RealVideo stream, 640 x 480
pixels, 1536 kbs
RealVideo stream, 320 x 240,
56 - 904 kbs
SIGGRAPH 2002 talk:
Talk as PPT
Embedded video clip:
sig02_begin_m.avi
Embedded video clip:
sig02_recap.avi
Embedded video clip: turtle2.avi
![Page 4: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/4.jpg)
Structured Light
![Page 5: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/5.jpg)
Structured Light
![Page 6: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/6.jpg)
Basic Principle
![Page 7: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/7.jpg)
Special Case: Light Spot Stereo
Calibrated Cameras: Detection of spot in left and right
cameras fully determines its 3D location. Spot to be
scanned across scene.
![Page 8: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/8.jpg)
Line Projection
![Page 9: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/9.jpg)
But we can do better
![Page 10: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/10.jpg)
Calibrated Projection
![Page 11: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/11.jpg)
Concept: Active Vision
Active manipulation of scene: Project light
pattern on object. Observe geometry of
pattern via camera → 3D geometry
![Page 12: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/12.jpg)
3D Scanning
Courtesy S. Narasimhan, CMU
![Page 13: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/13.jpg)
Typical Application
![Page 14: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/14.jpg)
![Page 15: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/15.jpg)
Overview
• Background
• General Setup
• Light Point Projection 2D and 3D
• Light Stripe Projection
• Static Light Pattern Projection
– Binary Encoded Light Stripes
– Segmenting Stripes
• 3D Photography on Your Desk
![Page 16: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/16.jpg)
General Setup
• one camera
• one light source
– types
• slide projector
• laser
– projection
• spot
• stripe
• pattern
![Page 17: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/17.jpg)
Light Spot Projection 2D
image
plane
Assume point-wise illumination by laser beam, only 2D
Reinhard Klette, Computer Vision
![Page 18: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/18.jpg)
Light Spot Projection 2D
Reinhard Klette, Computer Vision
![Page 19: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/19.jpg)
Light Spot Projection 2D• Coordinates found by triangulation:
– b can be found by projection geometry
– d = b*sin(a)/sin(a + b)
– P = (X0, Z0)
– X0 = d*cos(b)
– Z0 = h = d*sin(b)
• Calculating Location of P:– known b and a
- b defined by projection geometry/calibrat.
- Given image coordinate u and focal length f -> calculate b
- Given b, a, b -> calculate d
![Page 20: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/20.jpg)
Light Spot Projection 3D
Z
![Page 21: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/21.jpg)
Light Spot Projection 3D
![Page 22: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/22.jpg)
Light Spot Projection 3D
– X0 = (tan(a)*b*x)/(f + x*tan(a))
– Y0 = (tan(a)*b*y)/(f+x*tan(a))
– Z0 = (tan(a)*b*f)/(f+x*tan(a))
OBSERVATION????
Angle gamma γ not used !!!!
![Page 23: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/23.jpg)
Light Stripe Scanning – Single Stripe
Camera
Source
Surface
Light plane
• Optical triangulation– Project a single stripe of laser light
– Scan it across the surface of the object
– This is a very precise version of structured light scanning
– Good for high resolution 3D, but needs many images and takes time
Courtesy S. Narasimhan, CMU
![Page 24: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/24.jpg)
Light Stripe Projection
![Page 25: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/25.jpg)
Triangulation
• Project laser stripe onto object
Object
Laser
Camera
Light Plane0 DCzByAx
Courtesy S. Narasimhan, CMU
![Page 26: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/26.jpg)
Camera
Triangulation
• Depth from ray-plane triangulation:
– Intersect camera ray with light plane
Laser
Object
Light Plane
(distance)
0
dD
DCZBYAX
-
)','( yxImage Point
'/'
'/'
fZyY
fZxX
'''
'
CfByAx
DfZ
-
Courtesy S. Narasimhan, CMU
Use perspective
projection eq’s,
plug X, Y into
plane equation to
get Z
'f
![Page 27: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/27.jpg)
Straightforward: Single light
stripe and rotating Object
Object on turntable:
• Create P(X,Y,Z) profile for each rotation and fixed light slit
• Rotate object in discrete intervals and repeat
• Reconstruct 3D object by cylindric assembly of profiles → 3D
mesh
![Page 28: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/28.jpg)
Example: Laser scanner
Cyberware® face and head scanner
+ very accurate < 0.01 mm
− more than 10sec per scan
![Page 29: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/29.jpg)
Portable 3D laser scanner (this one by Minolta)
![Page 30: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/30.jpg)
Digital Michelangelo Projecthttp://graphics.stanford.edu/projects/mich/
Example: Laser scanner
![Page 31: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/31.jpg)
Can we do it without
expensive equipment?
![Page 33: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/33.jpg)
3D Photography on Your Desk• “Cheap” method that uses very common
tools to do 3D photography
• Requirements: PC, camera, stick, lamp,
and a checker board
• Uses “weak structured light” approach
![Page 34: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/34.jpg)
![Page 35: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/35.jpg)
![Page 36: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/36.jpg)
Lamp Calibration
![Page 37: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/37.jpg)
![Page 38: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/38.jpg)
![Page 39: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/39.jpg)
![Page 40: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/40.jpg)
![Page 41: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/41.jpg)
Low-Cost 3D Scanner for Everyonehttp://www.david-laserscanner.com/
![Page 42: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/42.jpg)
Low-Cost 3D Scanner for Everyonehttp://www.david-laserscanner.com/wiki/user_manual/3d_laser_scanning
![Page 43: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/43.jpg)
Cheap Scanner
• http://www.igp.ethz.ch/photogrammetry/education/lehrveranstaltungen
/MachineVisionFS2011/coursematerial/MV-SS2011-structured.pdf
• Cheap Scanner: http://www.david-laserscanner.com/
![Page 44: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/44.jpg)
Build your own 3D scanner
• Course notes: http://mesh.brown.edu/byo3d/notes/byo3D.pdf
• Slides: http://mesh.brown.edu/byo3d/slides.html
• Source code: http://mesh.brown.edu/byo3d/source.html
![Page 45: Structured Lighting - NYU Tandon School of Engineeringengineering.nyu.edu/~gerig/CS-GY-6643-S2016/... · Surface Light plane • Optical triangulation –Project a single stripe of](https://reader030.vdocuments.us/reader030/viewer/2022041102/5edd2498ad6a402d66681fb8/html5/thumbnails/45.jpg)
Next Lecture: Encoded Patterns
• Any spatio-temporal pattern of light projected on a surface (or
volume).
• Cleverly illuminate the scene to extract scene properties (eg., 3D).
• Avoids problems of 3D estimation in scenes with complex
texture/BRDFs.
• Very popular in vision and successful in industrial applications
(parts assembly, inspection, etc).