scanning and reconstruction for dynamic...

11
Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer Science Technical Report CS-2006-25 Kevin Dale, Ewen Cheslack-Postava, Greg Humphreys, and David P. Luebke Abstract We present a novel 3D scanning system with the potential for interactive acquisition and visualiza- tion of dynamic scenes. Our system uses a spatio- temporally adaptive sampling strategy, and can take advantage of multiple simultaneous scanning devices operating at different resolutions. We also employ a level set framework for reconstructing potentially dynamic scenes from multiple concurrent streams of range data. In our framework, implicit surfaces are reconstructed periodically from new samples on a coarse grid, creating a sequence of reconstructions from disjoint sample sets that is used to estimate mo- tion in the scene. A high-resolution reconstruction proceeds alongside, where the surface is evolved by a convective flow that guides it towards the sample set. We employ a spatially-varying distance metric based on our motion estimate that adaptively con- trols the contribution of older samples to the final reconstruction. 1 Introduction Our increasing ability to accurately measure real world objects and phenomena has led to significant recent advances in computer graphics. However, the acquisition and reconstruction process for 3D ge- ometry is still very time-consuming and computa- tionally intensive, often generating gigabytes of data and taking many hours to complete [16]. Most ac- quisition systems employ a uniform sampling strat- egy, which can lead to artifacts near silhouette or crease edges. This assumption also leads to redun- dant data in regions of the scene that are changing slowly, which needlessly increases the required pro- cessing time. Furthermore, if the scene being scanned changes during scanning, most existing systems will produce incorrect and unpredictable results. Laser scanning systems produce highly detailed models, but are far too slow to capture dynamic scenes, while structured light approaches are fast but the fidelity of their output is limited by the resolution of the camera and structured light source. In this paper, we present a novel 3D scanning sys- tem to address these issues. Throughout the sys- tem we stress adaptive allocation of scanning re- sources, including sample placement, CPU usage, and reconstruction effort. Our system consists of a prototype scanner, adaptive scan control algorithms, and spatio-temporally adaptive reconstruction soft- ware. The scanner is based on a time-of-flight laser rangefinder capable of sampling up to 50 KHz, cou- pled with a fast, low-resolution depth camera. We believe the combination of a low resolution camera, either color or depth, and a highly accu- rate laser rangefinder provides a good compromise within the space of scanners. Structured light scan- ners lie at one end of the spectrum, providing fast, dense sampling but are limited in precision by the camera and structured light source. Laser rangefind- ers lie on the other end of the spectrum and produce dense sampling along a given path but cannot cover the entire field of view as quickly as structured light scanners. Somewhere in the middle lie stereo vision cameras which will often not produce a sampling den- sity at the resolution of the capture device but are very fast and cheap, and fixed arrays of time of flight 1

Upload: others

Post on 14-Mar-2020

4 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

Scanning and Reconstruction for Dynamic SurfacesUniversity of Virginia Computer Science Technical Report CS-2006-25

Kevin Dale, Ewen Cheslack-Postava, Greg Humphreys,and David P. Luebke

Abstract

We present a novel 3D scanning system with thepotential for interactive acquisition and visualiza-tion of dynamic scenes. Our system uses a spatio-temporally adaptive sampling strategy, and can takeadvantage of multiple simultaneous scanning devicesoperating at di!erent resolutions. We also employa level set framework for reconstructing potentiallydynamic scenes from multiple concurrent streams ofrange data. In our framework, implicit surfaces arereconstructed periodically from new samples on acoarse grid, creating a sequence of reconstructionsfrom disjoint sample sets that is used to estimate mo-tion in the scene. A high-resolution reconstructionproceeds alongside, where the surface is evolved bya convective flow that guides it towards the sampleset. We employ a spatially-varying distance metricbased on our motion estimate that adaptively con-trols the contribution of older samples to the finalreconstruction.

1 Introduction

Our increasing ability to accurately measure realworld objects and phenomena has led to significantrecent advances in computer graphics. However, theacquisition and reconstruction process for 3D ge-ometry is still very time-consuming and computa-tionally intensive, often generating gigabytes of dataand taking many hours to complete [16]. Most ac-quisition systems employ a uniform sampling strat-egy, which can lead to artifacts near silhouette orcrease edges. This assumption also leads to redun-

dant data in regions of the scene that are changingslowly, which needlessly increases the required pro-cessing time. Furthermore, if the scene being scannedchanges during scanning, most existing systems willproduce incorrect and unpredictable results. Laserscanning systems produce highly detailed models, butare far too slow to capture dynamic scenes, whilestructured light approaches are fast but the fidelityof their output is limited by the resolution of thecamera and structured light source.

In this paper, we present a novel 3D scanning sys-tem to address these issues. Throughout the sys-tem we stress adaptive allocation of scanning re-sources, including sample placement, CPU usage,and reconstruction e!ort. Our system consists of aprototype scanner, adaptive scan control algorithms,and spatio-temporally adaptive reconstruction soft-ware. The scanner is based on a time-of-flight laserrangefinder capable of sampling up to 50 KHz, cou-pled with a fast, low-resolution depth camera.

We believe the combination of a low resolutioncamera, either color or depth, and a highly accu-rate laser rangefinder provides a good compromisewithin the space of scanners. Structured light scan-ners lie at one end of the spectrum, providing fast,dense sampling but are limited in precision by thecamera and structured light source. Laser rangefind-ers lie on the other end of the spectrum and producedense sampling along a given path but cannot coverthe entire field of view as quickly as structured lightscanners. Somewhere in the middle lie stereo visioncameras which will often not produce a sampling den-sity at the resolution of the capture device but arevery fast and cheap, and fixed arrays of time of flight

1

Page 2: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

rangefinders which produce low resolution depth im-ages that are consistently dense. While any su"-ciently fast and densly sampling device could be usedto augment the laser rangefinder in our system, wefind the fixed array of time of flight rangefinders to bean attractive choice because of the consistent sampledensity and accuracy of the device. The low resolu-tion depth camera is used both to guide sampling andto initialize the reconstruction and the rangefinder isused to generate very dense, accurate samples to useas the destination for the reconstructed surface evo-lution.

Our system is robust to noise and can cap-ture dynamic scenes in real time by making ef-ficient and adaptive use of available scanning re-sources. Rangefinder-based devices typically can cap-ture depth information at very fine resolution butmust compromise between uniform spatial and tem-poral coverage. However our device, driven by anadaptive scanning technique, is capable of capturinggeometry at rates much closer to that of structuredlight scanning. Furthermore, our scanning and recon-struction algorithms are easily parallelized, and evensupport multiple simultaneous scanning devices.

To e"ciently capture dynamic scenes, our algo-rithms detect and focus on regions of spatial and tem-poral change. In particular, we build on the framelesssampling and reconstruction techniques introducedby Dayal et al. [10]. We have extended their adaptivesample-point selection ideas to choose entire samplingpaths for the laser to follow in order to maximize newinformation to be gathered. In this way, scanningresources can quickly find small features and objectedges to yield extremely high fidelity object recon-structions, while simultaneously tracking moving ob-jects and providing a real-time reconstruction of adynamic environment.

To address the issue of computational complexityand interactivity, we exploit temporal coherence witha reconstruction algorithm that is inherently paral-lelizable and handles both rigid and deforming ob-jects. The basic idea is that reconstruction filters areshaped in three dimensions based on an estimate ofthe local gradient. In this way, old samples are givenless weight in areas with a large temporal gradient,since they are likely to be less representative of the

current surface. If the temporal gradient is low, oldsamples can be reused, so static portions of the scenequickly converge to a very detailed model. Theseadaptive reconstruction filters are combined with alevel-set approach which is robust to noise and out-liers, and produces very high quality 3D models.

The remainder of the paper is organized as follows.Section 2 reviews related work. Sections 3 and 4 de-scribe our sampling and reconstruction algorithms,respectively. Section 5 shows our results. We con-clude in Section 6 and and also describe areas of fu-ture research.

2 Related Work

Surface acquisition and reconstruction is a well-studied field; here we refer mainly to the most rel-evant work to our own.

2.1 3D Scanning Techniques

Rusinkeiwicz et al. [22] present a system for acquir-ing 3D models in real-time. The user holds an objectin front of a 60 Hz structured-light rangefinder andthe display is constantly updated showing the mostrecent version of the scanned model. Because of theimmediate feedback, the user is able to quickly iden-tify holes in the model and rotate the model accord-ingly to acquire missing regions.

Blais et al. describe a method to recursively refinea model obtained from sparse range data using theiterated closest points algorithm. [6, 7]. Their sys-tem also tracks the object to center scanning on theobject, resulting in an accurate, high resolution 3Dmodel of the moving object.

The DeltaSphere-3000 Scene Digitizer [FIXME:cite] is a commercial 3D scanner. It can scan a verywide field of view (360! horizontal, 290! vertical),but is relatively slow, typically spending 12 minutesto perform a high resolution 360! scan. The DeltaS-phere treats reconstruction as a post-process, gener-ating triangle meshes after the scan has completed.

Koninckx et al. describe an adaptively codedstructured light approach that is able to e"ciently

2

Page 3: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

scan deforming surfaces by exploiting temporal co-herence [13]. Our system has similar goals but canproduce much higher fidelity models due to the reso-lution of our laser rangefinder.

2.2 Surface Reconstruction

Reconstruction of static surfaces has been imple-mented in a number of fashions. Explicit schemeshave been used that triangulate data based onVoronoi criteria [3, 11]. Implicit shemes have alsoproven e!ective; a common formulation of the prob-lem is to find a scalar-valued function with an isocon-tour that approximates the surface, from either un-organized points [12] or aligned range images [9, 26].[12, 26] present strategies that compute the mostlikely surface given a set of range maps. Zhao etal. [28] present a similiar variational formulation ofsurface reconstruction from unorganized points inwhich they introduce a fast tagging algorithm to con-struct an initial approximation of a minimal surface,speeding reconstruction by an order of magnitude.

2.3 Level Set Methods

The approaches taken in [26, 28] are based on thelevel set methods of [18] for stable and e"cient de-formation of implicit surfaces. Fast marching meth-ods provide an e"cient, non-iterative means to trackpropagating fronts but are only applicable to mono-tonically advancing fronts [23]. General level set tech-niques are required for surfaces that do not meet thiscriteria, such as those in this work. For iterativelevel set methods, the narrow band [1] and sparsefield [26] approaches provide e"cient solutions bylimiting computation to only those cells close to thetarget level set, e!ectively reducing computationalcomplexity by an order of magnitude.

Techniques based on level set methods for segmen-tation and tracking in video are similar to the tech-niques described in this paper. The geodesic activecontour model iteratively refines a goal curve by min-imizing an energy function; this process is equivalentto finding the geodesic curve that best matches spec-ified image characteristics [20]. Methods in [5] tracksegmentations of interest across frames of video by

coupling their evolution to an image-based deforma-tion.

3 Adaptive Sampling

Most previous scanning systems, especially thosebased on lasers, o!er only uniform sampling strate-gies. Our system samples the target adaptively, fo-cusing e!ort on spatial edges such as silhouettes andgeometric detail, as well as temporal edges such asocclusion events and deformations. We find theseevents using a low resolution camera (either depth orcolor), and our sampling algorithm guides the scan-ner towards these regions when appropriate.

The scanner e!ectively restricts sample distribu-tions to fall along piecewise-smooth curves in thescanner’s field of view. Although it would be possibleto generate discrete sampling patterns, this would beine"cient because physical scanners scanner spendthe majority of their time repositioning the mirrorsfor the next sample and very little time actually sam-pling. For this reason, we cast the laser samplingproblem as a path planning problem. Specifically, weseek the path !(t) : [0,!) " #2 from the startingmirror orientation !(0) that minimizes the followingcost function:

C(!) =! "

0

D(!(t), t)I(!(t), t)

dt,

where D(!(t), t) is a sample density estimate andI(!(t), t) is an importance measure. The calculationand update of I and D are discussed in detail in Sec-tion 3.2. Note that this problem is ill-posed becauseeven with global information the optimal path mustminimize the cost for all times in the future. Ouralgorithm is a greedy approximation to the optimalpath.

3.1 Gradient Descent Path Planning

We use a multi-resolution gradient descent methodto greedily select the new direction to move the laser.The base importance map I and density estimate Dare both maintained as image pyramids [27] without

3

Page 4: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

Figure 1: The gradient descent algorithm operatesby examining the neighborhood surrounding the gridcell containing the current laser position at the finestlevel of detail. If the importance of that cell is above athreshold, the search stops and the laser is guided to-wards the neighbor cell with the highest importance.Otherwise, the process is repeated at the next coarserlevel. If no motion is suggested by the importancemaps, the laser falls back on a raster-scan pattern.

the topmost level (e.g., the top level in our represen-tation is a 2$2 image; see Section 3.2). Beginningat the bottommost cell containing the laser location,we ascend the pyramid looking for a neighboring gridcell that would yield a path ! with low cost C(!). Alow path cost indicates that the cell is insu"cientlysampled given its current importance estimate. If thecost is below a user specified minimum then we stopascending the pyramid and choose to move the lasertowards that grid cell. If no neighboring grid cell sat-isfies this condition at the current level, we continueto the next level of the pyramid. If the entire pyra-mid is traversed and no direction has been chosen,we fall back on a raster scan pattern.

To produce a smooth curve, we interpolate the cho-sen next direction with the current direction. This is

beneficial because the physical limitations of scannerhardware make paths with high curvature very ex-pensive, as the mirrors must decelerate to follow asharp turn. A smoother curve will therefore allowthe scanner to cover the field of view more quickly.

The primary benefit of using a multi-resolutionmethod is that the scanner will focus on spatiallycoherent, highly important regions, but will also per-form broad scans as distant regions become increas-ingly important. Furthermore, it guarantees thatbroad scans will be performed periodically. If a re-gion is neglected, its associated sampling density willdrop and the resulting low cost will draw the scanneraway from other regions. In addition, a multireso-lution technique allows the scanner to intelligentlyallocate available samples by spending more time lo-cally when other areas are relatively unimportant,resulting in increasingly refined reconstructions.

3.2 Updating Importance and Den-sity

A single user-specified parameter " % [0, 1] con-trols the relative importance of spatial and tempo-ral change. Specifically, the importance functionI(#, $, t) is defined as

I = "T + (1& ")S + %,

where S(#, $, t) and T (#, $, t) are measures of spatialand temporal incoherence, respectively. % is a smallconstant used to ensure the importance function isnon-zero everywhere.

For " = 0, the scanner is responsive solely to spa-tial incoherence; the scanner will focus only on edgesand will be slow to respond to moving objects. For" = 1, the scanner is responsive only to temporalchanges; the scanner would focus almost entirely onthe dynamic regions of the scene, or, in the case ofa static scene, would degrade to a uniform rastersampling pattern. In practice, we have found thatthe value " = .7 provides an adequate balance be-tween spatial and temporal adaptivity, although itwould likely be fruitful to dynamically update " asthe scanner learns about the varying ratios of spatialand temporal change in the scene being acquired.

4

Page 5: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

With each image captured from the camera we up-date S and T , and in turn I. The spatial importancemeasure S is a simple Canny edge detection filter [8].The temporal importance measure T is calculated us-ing a motion history computed over a predeterminedlength of time (we use 1 second). At each frame,pixels with significant temporal change are found byperforming binary thresholding on the absolute dif-ference of the current and previous video frame. Themotion history image contains either the most recenttime at which a significant color di!erence was de-tected at each pixel or zero if the most recent colordi!erence is not within the tracked interval. Thismotion history image is then scaled to fit the cor-rect range of values for importance so that pixels forwhich a change was detected will have the highesttemporal importance measure and pixels which havenot changed recently will have the smallest temporalimportance measure. Once new S and T images havebeen computed, the base importance image and thenthe entire importance pyramid are updated.

There are two steps to tracking the sample den-sity estimation D. The first step occurs every time anew goal position is selected. The path between thecurrent position and the goal position is traversed atthe base density image level. For each pixel inter-sected by the selected path, we visit each level of thepyramid and increment the density of the cell at thelevel which contains the pixel, adjusting the increasein density appropriately based on the current level.Although this is only approximate (because the laserpath might be updated before or after it reaches itsgoal), it avoids latency problems that would arise incomputing the true density. This way, the path plan-ner can avoid getting stuck in local regions that willnot be updated for a long time.

The second update to the density estimation occurseach time a camera frame is collected. This updatehandles static decay of samples. As samples growolder our confidence in them decreases, so older sam-ples contribute less weight than newer samples. Thestatic decay rate, measured in samples/pixel/second,accounts for this e!ect. We approximate this decayby reducing the density values in the density map bya constant decay rate R, scaled by the elapsed time

#t since the last update:

D = D &R#t.

Finally, we must account for regions of temporalchange. In regions where there was motion, the pre-vious samples are no longer valid. When we detectsuch a region, we set the corresponding density es-timates to zero, using T from the above importancemap calcuation as a binary mask. After the the den-sity is fully estimated, its associated mipmap pyramidis regenerated.

4 Reconstruction

Our reconstruction algorithm extends the level set-based reconstruction technique of [28] to dynamicsurfaces. Iterative level set methods are inherentlyparallelizable, as demonstrated by implementationsin [4, 15]. While we have not parallelized our tech-nique, this possibility motivated our selection of alevel set-based reconstructor over other methods.

The high level idea is to use fresh samples to recon-struct a coarse approximation of the current scene.Di!erencing successive coarse approximations pro-vides a motion estimation over the domain, whichis then used to determine the contribution of futuresamples to the high-fidelity reconstruction. To dothis, we employ an adaptive, non-euclidean distancemetric that allows old samples to contribute to staticregions of the scene, while pruning them from dy-namic regions. The remainder of this section de-scribes this algorithm in detail.

4.1 Level Set Methods and Static Re-construction

Here, we briefly review level set methods and the rel-evant details from [28], upon which our technique isbased. The level set method [18] provides a set oftools for deforming implicit surfaces. Specifically, adeformable surface $ is modeled as the k-level set ofa time-varying scalar function $(x, t) in 3D:

$(t) = {x|$(x, t) = k},

5

Page 6: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

for k % #, x % #3, and t % #+. In [28], for a gen-eral data set S, they compute the distance functionto S, d(x) = dist(x, S), over the domain. A non-iterative tagging algorithm finds an initial approxi-mation of the surface, at which point the embeddingis evolved via the convection equation

&$

&t= '(d(x)) ·'$.

To smooth the reconstructed surface, evolutionproceeds according to their minimal surface model,

&$

&t= |'$|

"'d · '$

|'$| + d' · '$

|'$|

#,

which contains an additional mean curvature termthat smooths the reconstruction in proportion to lo-cal sample density. They employ the narrow bandalgorithm [1] in their iterative solver, which restrictscomputation to a band of cells around the zero levelset.

The level set approach to surface reconstruc-tion and evolution automatically handles topologi-cal changes such as merging and pinching, which aredi"cult to handle with explicit surface representa-tions. Moreover, it handles the relatively sparse, non-uniform, and somewhat noisy sample sets generatedby our scanner with relative ease [28]. When evolvingthe surface, this method computes geometric infor-mation such as normals and curvature at each itera-tion, yet computation at a given cell depends only onthat cell’s local neighborhood. So the SIMD natureof these techniques can be exploited by parallel im-plementations [4,14]; this idea is discussed more fullyin Section 6. Additionally, the narrow band approachreduces both the storage and computational complex-ity of reconstruction over an N3 grid to O(N2), mak-ing it comparable to explicit surface representationsin this regard.

4.2 Dynamic Targets

For each set of incoming low-resolution depth maps,we perform static reconstruction on the samplepoints, similarly to the method discussed above, withtwo exceptions. Like [28], we use a signed distance

function for our embedding $, as it improves accu-racy and e"ciency of numerical computations. Theconvection model does not maintain this property,however, and requires periodic reinitialization to re-store $ to a signed distance function–i.e., |'$| = 1.An iterative approach to reinitialization solves thereinitialization equation [25]:

$t + S($)(|'$|& 1) = 0.

S($) is the numerically smeared sign functionfrom [21]:

S($) =$$

$2 + |'$|2(#x)2

To avoid periodically reinitializing, we add a reini-tialization term to the convection equation itself,yielding

&$

&t= '(d(x)) ·'$& S($)(|'$|& 1)

Numerical error during reinitialization generallycauses the zero level set to move [19]. This addi-tional term produces the same problem here, andwhile there are more sophisticated reinitializationtechniques to avoid this, we have found the impact tobe minimal. We additionally omit the final minimalsurface step, whose mean curvature term requires aprohibitively small timestep. We found that the con-vection model alone produces su"cient results in ourtests.

The low-resolution depth maps alone do not pro-vide a su"cient sampling of the target; however, theyprovide enough information to capture low-frequencygeometric features. Most importantly, we can ap-proximate motion of the target surface by di!erencingsuccessive low-resolution static reconstructions. Onthe zero level set of $ at world-time ' , this di!erenc-ing, #$#' , approximates the motion in the normaldirection, and is conveniently defined across all levelsets.

After the di!erencing step, we update the unsigneddistance function d(x) for the full reconstruction withboth the low-resolution depth maps and with newsamples from the range scanners. During this step,

6

Page 7: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

Figure 2: The sample static scene, the associated low resolution depth image, and the sample distributionusing our adaptive technique. Note that important regions, i.e. depth discontinuities, are sampled at ahigher rate than geometrically uninteresting areas.

we employ the following distance metric based onthe previous di!erencing step. For space-time pointsa, b % #4,

dist(a, b)2 = (ax & bx)2 + (ay & by)2 + (az & bz)2

+k|#$(a)#'

#$(b)#'

|(at & bt)2,

for a user-defined scale factor k. We store d forthe high-resolution sample distance field in a regu-lar grid, and additionally store the range sample thatis closest to the given cel In particular, we updated by placing each new sample into its correspondinggrid cell, placing these sample-cell pairs into a queue,and computing distance based on the above metric.For each updated cell-sample pair, the new sample istested against the cell’s six neighboring cells in a sim-ilar fashion. This process continues until the queueis exhausted.

The convection model assumes a euclidean metric,and additionally requires that samples are boundedby the initial surface in order to converge on a reason-able reconstruction [28]. We perform two additionalsteps so that we can use the convection model. First,we evolve $ in the normal direction for some distance(, so that the zero level set contains the previous in-terior region and the new samples.

Then, we perform an iterative reinitialization of d,iterating over only those cells in the current narrowband. This restores d to an unsigned distance field forthe purpose of reconstruction; however, each cell stillcontains the position and time of the closest samplefor use in the next d update. Once this is complete,we evolve $ via the convection model to obtain thecurrent reconstruction at world-time ' .

5 Results

Although we built a prototype scanner, it is necessaryto use multiple scanners to capture and reconstructdynamic scenes using our reconstruction technique.We chose to simulate multiple scanners in softwarewith objects represented by highly detailed meshes.The simulated scanner consists of a laser rangefinderwith performance roughly identical to that of thehardware as well as a low resolution depth camera,providing a compromise between spatial and tempo-ral resolution. We examine the two parts of our sys-tem with two scenarios: a static target and a dynamictarget.

7

Page 8: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

Figure 3: A frame from the original animation and two sample frame results sampled and reconstructedusing our system.

5.1 Static target

Here we show the e!ectiveness of our adaptive sam-pling technique by scanning a static object. We scana sample scene, a room from Monticello, using a stan-dard raster sample pattern and our adaptive sam-pling approach. The raster pattern produces uni-formly distributed samples as we would expect fromany standard laser rangefinder scanner. We run thescanners for the same amount of time to generate thesame number of samples. The results of a 3 secondscan using our adaptive method is shown in Figure 2.

Note that the raster pattern wastes samples in ar-eas with no geometric detail whereas our system in-telligently dedicates more samples to geometricallyimportant regions and therefore collects more usefulinformation using fewer samples.

5.2 Dynamic target

To test a dynamic target we use a Stanford bunnywith animated ears. This represents a di"cult case:a balance must be struck between static areas anddynamic areas, both with high geometric detail. Thestatic geometry must be scanned enough to improvereconstruction detail, but the dynamic region mustbe scanned often enough to maintain at least a roughapproximation of the surface.

We arranged four virtual scanners around the

bunny in physically plausible positions and orienta-tions. Each scanner roughly matches the specifica-tions of our prototype scanner.

The simulated scanners were arranged about thebunny such that no scanner was covering the bottomof the bunny. This is analogous to the object beingplaced on a surface for scanning, obscuring the baseof the object. This leaves a large region unsampledand thus a large hole in the set of samples. We add aconstraint such that the plane upon which the bunnyrests blocks the evolution of the surface in the areabeneath the bunny. Results from this simulation canbe seen in Figure 3.

Our sampling density must meet a number of crite-ria in order to resolve fine features of the surface. In asimple static reconstruction, sampling densities mustsatisfy conditions similar to those in [2,28]. However,we must extend these ideas into the temporal dimen-sion as well since we are capturing dynamic surfaces.Because we rely on a video signal from an on-boardcamera, temporal aliasing can occur if there are mo-tion features that cannot be resolved at the camera’srefresh rate.

8

Page 9: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

6 Conclusions and FutureWork

We have presented an spatially and temporally adap-tive 3D scanning system. The system has a combi-nation of the best aspects of previous scanning solu-tions: high resolution output characteristic of previ-ous laser rangefinder-based systems and the speed ofstructured light solutions.

Although the results presented in this paper usesimulated range data, we have built a prototype scan-ner that is capable of following the paths computedby our sampling algorithms (Figure 4). We have notyet integrated this device fully into our scanning sys-tem; this remains for future work.

Figure 4: Our prototype scanner is composed of aninfrared 780nm laser rangefinder capable of produc-ing range samples at 50 KHZ. A pair of galvonometer-mounted mirrors control sampling direction.

Our scanning system is inherently parallelizable.An arbitrary number of physical scanners can beused, each operating independently based on its ownimportance map. The reconstructor can also be par-allelized and accelerated using programmable graph-ics hardware; fast distance field computation and anarrow-band level-set solver have been implementedon the GPU [15,24]. Level set solvers have also beene"ciently parallelized on multiprocessor systems [4].Proper implementation of a scalable system will re-quire attention to load-balancing, routing samplesto appropriate solvers, and minimizing the need for

inter-block communication in the solvers. This paral-lel system would also benefit from the loose couplingbetween sampling and reconstruction in our system.

Although the narrow band level set method pro-vides an e"cient method for surface evolution, otherimplementations may prove more suitable for ouradaptive sampling approach. For example, the octreerepresentation used by Losasso et al. [17] aligns wellwith our spatially adaptive approach. Another pos-sibility would be to represent the surface embeddingwith a spatially adaptive tetrahedral mesh. However,extending these methods to parallel implementationsis an unsolved problem.

Our reconstruction technique could be enhancedby various computer vision techniques such as imagesegmentation and motion tracking. For example, cer-tain optimizations could be made if we could classifythe motion. If we could determine an object is under-going only rigid body transformations then it mightbe possible to find the transformation directly andarrive immediately at a converged solution.

Another possible extension of our techniques wouldbe defining importance as a function over the spacebeing scanned rather than the fields of view of thescanners. This requires collaboration between thescanners but would also allow us to evaluate the nextbest view and dynamically allocate scanners to thepositions and orientations they are most needed.

References

[1] Adalsteinsson, D., and Sethian, J. A. Afast level set method for propagating interfaces.J. Comput. Phys. 118, 2 (1995), 269–277.

[2] Amenta, N., Bern, M., and Eppstein, D.The crust and the )-skeleton: Combinatorialcurve reconstruction. Graphical models and im-age processing: GMIP 60, 2 (???? 1998), 125–??

[3] Amenta, N., Bern, M., and Kamvysselis,M. A new voronoi-based surface reconstructionalgorithm. In SIGGRAPH ’98: Proceedings ofthe 25th annual conference on Computer graph-ics and interactive techniques (New York, NY,USA, 1998), ACM Press, pp. 415–421.

9

Page 10: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

[4] Awate, S. P., and Whitaker, R. T. An in-teractive parallel multiprocessor level-set solverwith dynamic load balancing. Tech. rep., Uni-versity of Utah School of Computing, 2004.

[5] Bertalmío, M., Sapiro, G., andRandall, G. Morphing active contours. IEEETrans. Pattern Anal. Mach. Intell. 22, 7 (2000),733–737.

[6] Blais, F., Picard, M., and Godin, G. Re-cursive model optimization using icp and freemoving 3d data acqusisition. In Proceedings of3DIM (October 2003), pp. 251–258.

[7] Blais, F., Picard, M., and Godin, G. Ac-curate 3d acquisition of freely moving objects.In Proceedings of the 2nd International Sympo-sium on 3D Data Processing, Visualization, andTransmission (Sept. 2004), vol. 0, pp. 422–429.

[8] Canny, J. A computational approach to edgedetection. IEEE Transactions on Pattern Anal-ysis and Machine Intelligence 8, 6 (1986), 679–698.

[9] Curless, B., and Levoy, M. A volumetricmethod for building complex models from rangeimages. In Proceedigs of SIGGRAPH ’96 (NewYork, NY, USA, 1996), ACM Press, pp. 303–312.

[10] Dayal, A., Wooley, C., Watson, B., andLuebke, D. Adaptive frameless rendering. InProceedings of the 2005 Eurographics Sympo-sium on Rendering (June 2005), pp. 265–275.

[11] Edelsbrunner, H. Shape reconstruction withdelaunay complex. In LATIN ’98: Proceed-ings of the Third Latin American Symposiumon Theoretical Informatics (London, UK, 1998),Springer-Verlag, pp. 119–132.

[12] Hoppe, H., DeRose, T., Duchamp, T., Mc-Donald, J., and Stuetzle, W. Surface re-construction from unorganized points. In Pro-ceedings of SIGGRAPH ’92 (New York, NY,USA, 1992), ACM Press, pp. 71–78.

[13] Koninckx, T. P., Griesser, A., and Gool,L. V. Real-time range scanning of deformablesurfaces by adaptively coded structured light.In Fourth International Conference on 3-D Dig-ital Imaging and Modeling (3DIM03) (October2003), S. Kawada, Ed., IEEE Computer Society,pp. 293–302.

[14] Lefohn, A., Cates, J. E., and Whitaker,R. T. Interactive, gpu-based level sets for 3dsegmentation. In Proceedings of Medical ImageComputing and Computer Assisted Intervention(MICCAI) 2003 (2003), pp. 564–572.

[15] Lefohn, A., Kniss, J., Hansen, C., andWhitaker, R. A streaming narrow-band al-gorithm: Interactive deformation and visualiza-tion of level sets. IEEE Transactions on Visu-alization and Computer Graphics 10, 4 (2004),422–433.

[16] Levoy, M., Pulli, K., Curless, B.,Rusinkiewicz, S., Koller, D., Pereira,L., Ginzton, M., Anderson, S., Davis, J.,Ginsberg, J., Shade, J., and Fulk, D.The digital michelangelo project: 3d scanningof large statues. In Proceedings of SIGGRAPH2000 (New York, NY, USA, 2000), ACMPress/Addison-Wesley Publishing Co., pp. 131–144.

[17] Losasso, F., Fedkiw, R., and Osher, S.Spatially adaptive techniques for level set meth-ods and incompressible flow.

[18] Osher, S., and Fedkiw, R. Fronts propogat-ing with curvature dependent speed, algorithmsbased on a hamilton-jacobi formulation. Journalof Computational Physics 79 (1988), 12–49.

[19] Osher, S., and Fedkiw, R. Level Set Meth-ods and Dynamic Implicit Surfaces, first ed.Springer-Verlag, 2001.

[20] Paragios, N., and Deriche, R. Geodesic ac-tive contours and level sets for the detection andtracking of moving objects. IEEE Transactionson Pattern Analysis and Machine Intelligence22, 3 (Mar. 2000), 266–280.

10

Page 11: Scanning and Reconstruction for Dynamic Surfacesgraphics.stanford.edu/~ewencp/papers/arsenic_tr.pdf · Scanning and Reconstruction for Dynamic Surfaces University of Virginia Computer

[21] Peng, D., Merriman, B., Zhao, H., Osher,S., and Kang, M. A pde based fast local levelset method, 1999.

[22] Rusinkiewicz, S., Hall-Holt, O., andLevoy, M. Real-time 3d model acquisition. InACM Transactions on Graphics (Proceedings ofSIGGRAPH) (July 2002), vol. 21, pp. 438–446.

[23] Sethian, J. A. A fast marching level setmethod for monotonically advancing fronts.PNAS 93, 4 (1996), 1591–1595.

[24] Sud, A., Otaduy, M. A., and Manocha, D.Difi: Fast 3d distance field computation usinggraphics hardware. In Computer Graphics Fo-rum (Proceedings of EUROGRAPHICS) (2004),vol. 23, pp. 557–566.

[25] Sussman, M., Smereka, P., and Osher, S.A level set approach for computing solutions toincompressible two-phase flow. J. Comput. Phys.114, 1 (1994), 146–159.

[26] Whitaker, R. A level-set approach to 3dreconstruction from range data. InternationalJournal of Computer Vision 29, 3 (1998), 203–231.

[27] Williams, L. Pyramidal parametrics. In Pro-ceedings of SIGGRAPH ’83 (New York, NY,USA, 1983), ACM Press, pp. 1–11.

[28] Zhao, H.-K., Osher, S., and Fedkiw, R.Fast surface reconstruction using the level setmethod. In Proceedings of IEEE Worshop onVariational and Level Set Methods in ComputerVision (July 2001), pp. 194–201.

11