3d reconstruction & scanning - dongseo

3D Reconstruction & Scanning

lbg@dongseo.ac.kr

http://graphics.stanford.edu/projects/mich/

The Digital Michelangelo Project 1997~9

Agenda

4/28/2015 lbg@dongseo.ac.kr 2

Multiple View Geometry

Depth Camera

Structured Light

Camera Calibration

Projector Calibration

http://www.ten24.info/

http://www.youtube.com/watch?v=j4waCVRgBWI

http://www.youtube.com/watch?v=Wgp0Cg3UFA4

4/28/2015 lbg@dongseo.ac.kr 3

4/28/2015 lbg@dongseo.ac.kr 4

Projective Transformations

Camera Calibration

Epipolar Geometry

Feature Points

Correspondence Search

RANSAC Algorithm

3D Reconstruction

SIFT&ASIFT

Scale-invariant feature transform (or

SIFT) is an algorithm in computer vision to

detect and describe local features in images.

The algorithm was published by David Lowe in

1999.

Applications include object recognition, robotic

mapping and navigation, image stitching, 3D

modeling, gesture recognition, video tracking,

and match moving.

The algorithm is patented in the US; the owner

is the University of British Columbia.

….

David Lowe

Computer Science Department

University of British Columbia

2015-04-28 5

Scale Invariant Feature Transform

Depth Camera

lbg@dongseo.ac.kr

Motion sensing input device by Microsoft

Depth camera tech. developed by PrimeSense Invented in 2005

Software tech. developed by Rare

First announced at E3 2009 as “Project Natal”

Windows SDK Releases

7

Microsoft Kinect

http://www.microsoft.com/en-us/kinectforwindows/discover/features.aspx

lbg@dongseo.ac.kr 4/28/2015

4/28/2015 lbg@dongseo.ac.kr 8

ToF 3D Camera

4/28/2015 lbg@dongseo.ac.kr 9

Structured Light + Range Imaging

lbg@dongseo.ac.kr

http://www.cs.cmu.edu/afs/cs/academic/class/15385-s06/lectures/ppts/lec-17.ppt

http://www.sci.utah.edu/~gerig/CS6320-S2012/CS6320_3D_Computer_Vision.html

Stereo Triangulation

4/28/2015 lbg@dongseo.ac.kr 11

I J

Correspondence is hard!

Structured Light Triangulation

4/28/2015 lbg@dongseo.ac.kr 12

I J

Correspondence becomes easier!

Example: Laser scanner

4/28/2015 lbg@dongseo.ac.kr 13

Digital Michelangelo Project http://graphics.stanford.edu/projects/mich/

+ very accurate < 0.01 mm

− more than 10sec per scan

Cyberware® face and head scanner

Portable 3D laser scanner

4/28/2015 lbg@dongseo.ac.kr 14

http://www.youtube.com/watch?v=IqOg77liryg

Leica ScanStation C10

4/28/2015 lbg@dongseo.ac.kr 15

Leica ScanStation C10

4/28/2015 lbg@dongseo.ac.kr 16

http://www.youtube.com/watch?v=cUI6HD_d89k

http://www.youtube.com/watch?v=1lDO1UevAJI

http://www.youtube.com/watch?v=cxTYYce_d_8

Leica ScanStation C10

4/28/2015 lbg@dongseo.ac.kr 17

LMS LIDAR scanner from SICK

4/28/2015 lbg@dongseo.ac.kr 18

https://www.youtube.com/watch?v=yfQhEiq48YA

https://www.youtube.com/watch?v=aL-Ihvu0ugo

Structured Light 3D Surface Imaging

lbg@dongseo.ac.kr

Continuum of Triangulation Methods

4/28/2015 lbg@dongseo.ac.kr 20

Slow, robust Fast, fragile

Multi-stripe

Multi-frame

Single-frame Single-stripe

Faster Acquisition?

4/28/2015 lbg@dongseo.ac.kr 21

Project multiple stripes simultaneously

Correspondence problem: which stripe is which?

Common types of patterns:

Binary coded light striping

Gray/color coded light striping

http://vimeo.com/37090394

http://www.youtube.com/watch?v=a6g17-4FmFo

Structured Light Projection Classification

4/28/2015 lbg@dongseo.ac.kr 22

Jason Geng, "Structured-light 3D surface imaging: a tutorial," Adv. Opt. Photon. 3, 128-160 (2011)

Binary Coding

4/28/2015 lbg@dongseo.ac.kr 23

Projected

over time

Pattern 1

Pattern 2

Pattern 3

Example: 7 binary

patterns proposed

by Posdamer &

Altschuler

…

Codeword of this píxel: 1010010

identifies the corresponding pattern stripe

J. L. Posdamer and M. D. Altschuler. Surface measurement by space-encoded projected beam systems. Computer Graphics and Image Processing, 18:1–17, 1982.

Space

Time

4/28/2015 lbg@dongseo.ac.kr 24

4/28/2015 lbg@dongseo.ac.kr 25

4/28/2015 lbg@dongseo.ac.kr 26

4/28/2015 lbg@dongseo.ac.kr 27

4/28/2015 lbg@dongseo.ac.kr 28

4/28/2015 lbg@dongseo.ac.kr 29

4/28/2015 lbg@dongseo.ac.kr 30

4/28/2015 lbg@dongseo.ac.kr 31

4/28/2015 lbg@dongseo.ac.kr 32

4/28/2015 lbg@dongseo.ac.kr 33

4/28/2015 lbg@dongseo.ac.kr 34

4/28/2015 lbg@dongseo.ac.kr 35

4/28/2015 lbg@dongseo.ac.kr 36

4/28/2015 lbg@dongseo.ac.kr 37

4/28/2015 lbg@dongseo.ac.kr 38

Fringe Projection 3D Scanning

lbg@dongseo.ac.kr

Agenda

4/28/2015 lbg@dongseo.ac.kr 40

Fringe Projection Techniques

Three step phase shifting algorithm

http://www.jove.com/video/50421/high-resolution-high-speed-three-dimensional-video-imaging-with

Three-step Phase-shifting algorithm

4/28/2015 lbg@dongseo.ac.kr 41

Wrapped Phase Image

4/28/2015 lbg@dongseo.ac.kr 42

Unwrapped Phase Image

4/28/2015 lbg@dongseo.ac.kr 43

Calibration Plane

4/28/2015 lbg@dongseo.ac.kr 44

3D Result

4/28/2015 lbg@dongseo.ac.kr 45

Phase shifting patterns

4/28/2015 lbg@dongseo.ac.kr 46

Sinusoid patterns

4/28/2015 lbg@dongseo.ac.kr 47

System Setup

4/28/2015 lbg@dongseo.ac.kr 48

Large angle setting

4/28/2015 lbg@dongseo.ac.kr 49

Small angle setting

4/28/2015 lbg@dongseo.ac.kr 50

Calibration I

4/28/2015 lbg@dongseo.ac.kr 51

Calibration II

4/28/2015 lbg@dongseo.ac.kr 52

4/28/2015 lbg@dongseo.ac.kr 53

4/28/2015 lbg@dongseo.ac.kr 54

4/28/2015 lbg@dongseo.ac.kr 55

http://www.jove.com/video/50421/high-resolution-high-speed-three-dimensional-video-imaging-with

Thanks you!!

lbg@dongseo.ac.kr

http://kowon.dongseo.ac.kr/~lbg/

General Stereoscopic System

lbg@dongseo.ac.kr

General Stereoscopic System

4/28/2015 lbg@dongseo.ac.kr 58

Simple Stereoscopic System

4/28/2015 lbg@dongseo.ac.kr 59

Simple Stereoscopic System

4/28/2015 lbg@dongseo.ac.kr 60

Simple Stereoscopic System

4/28/2015 lbg@dongseo.ac.kr 61

Camera Calibration

lbg@dongseo.ac.kr

Pinhole Camera Model

4/28/2015 lbg@dongseo.ac.kr 63

TT ZfYZfXZYX )/,/(),,(

101

0

0

1

Z

Y

X

f

f

Z

fY

fX

Z

Y

X

101

01

01

1Z

Y

X

f

f

Z

fY

fX

Internal Camera Parameters

4/28/2015 lbg@dongseo.ac.kr 64

Camera rotation and translation

4/28/2015 lbg@dongseo.ac.kr 65

C~

-X~

RX~

cam

X10

RCR

1

10

C~

RRXcam

Z

Y

X camX0|IKx

XC~

|IKRx

t|RKP C~

Rt

PXx

Camera Distortion

4/28/2015 lbg@dongseo.ac.kr 66

Correcting Radial Distortion of Cameras

4/28/2015 lbg@dongseo.ac.kr 67

Projector Calibration

lbg@dongseo.ac.kr

Daniel Moreno and Gabriel Taubin

Brown University School of Engineering

Projector-Camera Calibration / 3D Scanning Software

http://mesh.brown.edu/calibration/

4/28/2015 lbg@dongseo.ac.kr 69

4/28/2015 lbg@dongseo.ac.kr 70

4/28/2015 lbg@dongseo.ac.kr 71

4/28/2015 lbg@dongseo.ac.kr 72

4/28/2015 lbg@dongseo.ac.kr 73

4/28/2015 lbg@dongseo.ac.kr 74

Thanks you!!

lbg@dongseo.ac.kr

http://kowon.dongseo.ac.kr/~lbg/

4/28/2015 lbg@dongseo.ac.kr 76

4/28/2015 lbg@dongseo.ac.kr 77

4/28/2015 lbg@dongseo.ac.kr 78

4/28/2015 lbg@dongseo.ac.kr 79

4/28/2015 lbg@dongseo.ac.kr 80

4/28/2015 lbg@dongseo.ac.kr 81

4/28/2015 lbg@dongseo.ac.kr 82

4/28/2015 lbg@dongseo.ac.kr 83

4/28/2015 lbg@dongseo.ac.kr 84

4/28/2015 lbg@dongseo.ac.kr 85

Calibration – Perspective Transformation

4/28/2015 lbg@dongseo.ac.kr 86

4/28/2015 lbg@dongseo.ac.kr 87

4/28/2015 lbg@dongseo.ac.kr 88

4/28/2015 lbg@dongseo.ac.kr 89

3D Acquisition from Shadows

4/28/2015 lbg@dongseo.ac.kr 90

3D Acquisition from Shadows

4/28/2015 lbg@dongseo.ac.kr 91

3D Acquisition from Shadows

4/28/2015 lbg@dongseo.ac.kr 92

3D Acquisition from Shadows

4/28/2015 lbg@dongseo.ac.kr 93

3D Acquisition from Shadows

4/28/2015 lbg@dongseo.ac.kr 94

3D Acquisition from Shadows

4/28/2015 lbg@dongseo.ac.kr 95

3D Model Acquisition Pipeline

4/28/2015 lbg@dongseo.ac.kr 96

3D Scanner

3D Model Acquisition Pipeline

4/28/2015 lbg@dongseo.ac.kr 97

3D Scanner

View Planning

3D Model Acquisition Pipeline

4/28/2015 lbg@dongseo.ac.kr 98

3D Scanner

Alignment View Planning

3D Model Acquisition Pipeline

4/28/2015 lbg@dongseo.ac.kr 99

3D Scanner

Alignment

Merging

View Planning

3D Model Acquisition Pipeline

4/28/2015 lbg@dongseo.ac.kr 100

3D Scanner

Alignment

Merging Done?

View Planning

3D Model Acquisition Pipeline

4/28/2015 lbg@dongseo.ac.kr 101

3D Scanner

Alignment

Merging Done?

View Planning

Display

Light Field Camera

lbg@dongseo.ac.kr

Stanford Multi-Camera Array

4/28/2015 lbg@dongseo.ac.kr 103

http://graphics.stanford.edu/projects/array/

640 × 480 pixels × 30 fps × 128 cameras

synchronized timing

continuous streaming

flexible arrangement

Stanford Light Field Archive

4/28/2015 lbg@dongseo.ac.kr 104

http://lightfield.stanford.edu/lfs.html

Stanford Light Field Archive

4/28/2015 lbg@dongseo.ac.kr 105

http://lightfield.stanford.edu/lfs.html

Lytro Camera

4/28/2015 lbg@dongseo.ac.kr 106

https://www.lytro.com/

Adaptive Optics Microlens Array

4/28/2015 lbg@dongseo.ac.kr 107

Adaptive Optics microlens array

125μ square-sided microlenses

4000 × 4000 pixels ÷ 292 × 292 lenses = 14 × 14 pixels per lens

Adobe LightField Camera

4/28/2015 lbg@dongseo.ac.kr 108

http://lightfield-forum.com/light-field-camera-prototypes/adobe-lightfield-camera-protypes/

Thanks you!!

lbg@dongseo.ac.kr

http://kowon.dongseo.ac.kr/~lbg/