mixed-reality for intuitive photo-realistic 3d-model generation...ndi laser scanner shape optical...

MR for Intuitive Photo-Realistic 3D-Model Generation > Sepp et al. > 18.11.2009

Folie 1

Mixed-Reality for Intuitive Photo-Realistic 3D-Model Generation Wolfgang Sepp, Tim Bodenmueller, Michael Suppa, and Gerd Hirzinger DLR, Institut für Robotik und Mechatronik

@ GI-Workshop VR/AR 2009


Folie 2

virtual world

real world

3D-Modeling with hand-held devices Overview

scanning process cycle over

acquire datareconstructInspect

deferred problem-recognition

non-efficient user-interface

observe

inspect reconstruct

move

scanobject

scanner

object


Folie 3

virtual world

real world






object

scanner

object

?


Folie 4

virtual world

real world






object

scanner

object

?


Folie 5

3D-Modeling with hand-held devices Aspects

Level of concurrency “which results are immediately available”

View Planning Support “which views are necessary to scan the object“

accumulated viewing positions or scan resultsnext best viewing positions

Navigation Support „how to navigate the scanner to a desired view“

link virtual object to real object relative poses between object, scanner, operator


Folie 6

3D-Modeling with hand-held devices State of the Art

system modalities registration view highest level of concurrency

Metris Modelmaker shape mechanical arm

N shaded points

Perceptron/Romer/PolyWorks shape mechanical arm

N shaded points

Handyscan shape & texture activeego-motion

N points

NDI Laser Scanner shape optical tracker N points

David shape & texture software 1 points


Folie 7

3D-Modeling with hand-held devices Requirements

spatial domaindepth measurement unit (scanner) pose measurement unit (tracker) calibration

radiometric domainsurface texture measurement unit (camera)

temporal domainsynchronisation (time coherence) latencies (online generation of 3D model)

requirements on spatial synchronisation depend on velocities of movements (expected 0.5m/sec, 45deg/sec)


Folie 8

The Multi-sensory 3D-Modeler Hardware

depth sensing principleslaser triangulation (rotating laser-scanner, laser-stripe profiler)stereo triangulation

pose measurementsinternal optical tracker (image-based ego-motion) external optical tracker (active/passive markers) coordinate measurement (mechanical arm)


Folie 9

The Multi-sensory 3D-Modeler Hardware

depth sensing principleslaser triangulation (rotating laser-scanner, laser-stripe profiler)stereo triangulation

pose measurementsinternal optical tracker (image-based ego-motion) external optical tracker (active/passive markers) coordinate measurement (mechanical arm)


Folie 10

The Multi-sensory 3D-Modeler Synchronisation

hardware synchronisation

avoid interferences software synchronisation

un-triggerable devices

labeling of sensor data

consistent data fusion


Folie 11

The Multi-sensory 3D-Modeler Data-Flow


Folie 12

Streaming Generation of 3D Models Data Flow

Level of concurrencyimmediate triangulationimmediate view integration

estimation of surface normal

densitylimitation

update ofnormal

selection ofvertices

localtriangulation

meshlimitation

pointmodel

wire-framemodel

mesh generation

3D pointmeasurement

visualisation


Folie 13

Photo-Realistic Texture Mapping

Texture-mapping modessingle-view mappingsingle-view mapping with brightness correctionmulti-view mapping

Dependenciesaccuracy of 3D modelaccuracy of pose measurementsynchronisation

Level of concurrencysequential (manual) acquisition of texture imagesimmediate mapping to surface model


Folie 14

Navigation Support Linking of virtual and real world objects

seamless factor

view alignment display navigation support usability

high operator pose see-through HMD, AR

operator – scanner - virtual object – real object

requires high accuracy in pose measurement

scanner pose desktop, AR scanner – virtual object – real object

scanner pose desktop, VR scanner – virtual object

measurement desktop, VR triangulation – virtual object

separate manual inspection

low mouse desktop, VR operator manual inspection


Folie 15

Navigation Support via sensor-pose


Folie 16

View Planning Support Deciding next best view(s)

seamless factor

modality of visualisation view planning support

usability

high textured surface-model texture- oriented

virtual object and real object not distinguishable in AR

shaded surface- model shape- oriented

good

wire-frame model link-oriented weak 3D impression

low raw point model measurement -oriented

missing 3D impression


Folie 17

View Planning Support via shaded, textured surface


Folie 18

Concurrent Scanning & Reconstruction with AR-support


Folie 19

Thank you!


Folie 20

Further Examples 1/2


Folie 21

Further Examples 2/2

Mixed-Reality for Intuitive Photo-Realistic 3D-Model Generation�Wolfgang Sepp, Tim Bodenmueller, Michael Suppa, and Gerd Hirzinger�DLR, Institut für Robotik und Mechatronik��@ GI-Workshop VR/AR 20093D-Modeling with hand-held devices�Overview3D-Modeling with hand-held devices�Overview3D-Modeling with hand-held devices�Overview3D-Modeling with hand-held devices�Aspects3D-Modeling with hand-held devices�State of the Art3D-Modeling with hand-held devices�RequirementsThe Multi-sensory 3D-Modeler�HardwareThe Multi-sensory 3D-Modeler�HardwareThe Multi-sensory 3D-Modeler�SynchronisationThe Multi-sensory 3D-Modeler �Data-FlowStreaming Generation of 3D Models�Data FlowPhoto-Realistic Texture MappingNavigation Support�Linking of virtual and real world objectsNavigation Support�via sensor-poseView Planning Support�Deciding next best view(s)View Planning Support�via shaded, textured surfaceConcurrent Scanning & Reconstruction�with AR-supportFoliennummer 19Further Examples 1/2Further Examples 2/2

mixed-reality for intuitive photo-realistic 3d-model generation...ndi laser scanner shape optical...

Documents