3D Laser Stripe Scanneror “A Really Poor Man’s DeltaSphere”

Chad HantakDecember 6, 2004

Overview

• Introduction• Acquisition Device• Calibration• Software Framework• Processing• Limitations• Future Work

Introductionor “What’s the project about?”

• 3D Scanning System• Similar to DeltaSphere

• Range Samples from fixed COP

• Acquire sparse depth with color• Acquisition Device

• Cheap• Simple to operate

DeltaSphere

• Dense Depth• Time of Flight Acquisition• Collects Range Samples over 360

degrees of Azimuth and 135 degrees of elevation

• Cons• Slow

• Scanning• Processing

• Expensive

Acquisition Device Components

• Laser Level• Available at hardware stores• Used for leveling (pictures,

shelves, …)• Emits a plane of laser light

(Vertical or Horizontal)

• Camcorder• Canon GL2 (3 CCD)• Little overkill for this

Acquisition Device Construction

• Camcorder rotates about fixed point

• Laser Level is fixed next to camcorder

Depth Range Acquisition“The Theory”

• Calibrated Rig• Know laser stripe’s plane equation in

camera’s coordinate system

• During Rotation• Determine amount of rotation• Find laser’s stripe points in image• Determine where image rays intersect

stripe plane• Yields depth samples• At same time acquire color

Calibration

• Need to know laser stripe plane in camera coordinate system

• Record calibration grid with laser stripe activated

Bouget’s Matlab Toolkit

• Camera Intrinsics• Standard Bouget’s Toolkit

• Extras from toolkit• Calibration plane in camera’s

coordinate system for each Image

Laser Stripe’s Plane

• For each image• Find stripe points (user clicks)• Determine world rays• Intersect with grid’s plane for

world points

• Take all the world points and fit them to a plane (least squares)

• Result is laser stripe plane

Software Framework

• Implemented in C++• Libraries

• OpenCV• Image Processing• http://

www.intel.com/research/mrl/research/opencv/

• DirectShow• Interface to Camcorder• http://msdn.microsoft.com/library/default.asp?url=

/library/en-us/directshow/htm/introductiontodirectshow.asp

Two Executables

• depthPanorama• Hook into camcorder• Acquires / processes frames• Saves into shared memory

• panoramaView• Reads from shared memory• Displays result to user

depthPanorama

• 3 Systems• RotationTracker

• Estimates the amount of rotation

• ColorProcessor• Creates the color panorama

• DepthProcessor• Creates the depth panorama

RotationTracker

• Estimates the amount of rotation between two frames

• Uses Lucas-Kanade Point Tracking• Estimate rotation if “enough”

points moved

• Uses RANSAC to estimate the amount of rotation

ColorProcessor

• Updates the Color Panorama• Process

• Only update after “enough” rotation

• Extract and resize left half of frame into panorama’s memory

• Right half of frame contains the laser stripe

Depth Processor

• Updates the Depth Panorama• Process

• Find the laser stripe points in the frame (samples every x scan lines)

• Turn image points into world rays• Intersect rays with laser stripe

plane• For valid intersections, place depth

value in depth panorama• For “some” invalid intersections,

attempt to interpolate

Finding Laser Stripe Image Points

or “The Hard Part”

• Know in the frame where points are (right half)• Extension to ModelCamera system

• Process scan line looking for candidate peaks with symmetry

• Exploit fact camcorder is just rotating• Determine homography from one frame to the next,

warp previous frame, subtract from current frame• Homography is based on amount of rotation between

frames• Since laser stripe moves differently it will stand out

better in resultant image• There is still noise in the image• Need some image processing

• Edge Detection, Erosion, Dilation

Turing Image Points into Depth Samples

• Know the camcorder intrinsics• Allows image points to be turned into rays

into world from camcorder’s COP• Intersect these rays with plane• Splat depth to neighboring depth samples• Associate a confidence level with each

depth sample• Perfect; confidence = 1.0• Interpolated; confidence = 0.75• Splatted; confidence = 0.6

• When updating if incoming confidence higher, use it

No Intersection

• Due to noise, image points of stripe may not be correct• World ray may not intersect stripe

plane• If neighboring samples are valid,

interpolate missing depth value• Lower confidence

panoramaView

• Simply reads values sent by depthPanorama

• Displays texture-mapped quads of depth values

• Updates continually from shared memory

Results

Current Problems

• Rotation tracking• Off by a few degrees at end of 360

degree pan (< 10 degrees)• Tweaking LK Tracking parameters• Smooth out the rotation (use LK &

correlation to determine when rotation complete)

• Stripe point determination• Adjusting image processing parameters

• Depth discontinuities• Take them into account when rendering

Future Work

• Fix current problems• Better laser light / Point

detection• Stronger at distance• Want longer range scans

• Combine different scans• Introduces new slew of problems

References

• R. Laganiere, “Programming computer vision applications: A step-by-step guide to the use of the Intel OpenCV library and the Microsoft DirectShow technology”, http://www.site.uottawa.ca/~laganier/tutorial/opencv+directshow/.

• V. Popescu, E. Sacks and G. Bahmutov, “Interactive Modeling from Dense Color and Sparse Depth” , 3DPTV, 2004.

• S. Sinha and M. Pollefeys, “Towards Calibrating a Pan-Tilt-Zoom Camera Network”, 5th Workshop on Omnidirectional Vision, Camera Networks, and Non-classical Cameras (OMNIVIS), May 16, 2004.