2008 brokerage 03 scalable 3 d models [compatibility mode]
TRANSCRIPT
Scaleable 3D modelsScaleable 3D modelsLuc Van Gool
Large-scale 3D modeling
� Clear trend towards ever more massive 3D models, and
dynamic capture (4D)
� SecondLife, Google Earth, games like the Runaway, 3D
navigation systems, Digital 3D Cartography (AGIV),
digitalisation of museum content (EDL), …digitalisation of museum content (EDL), …
� Developers of acquisition, editing, transmission, rendering,
and presentation technologies are struggling to match
� Problem is huge, as massive, high-quality modelling also
requires bridging the infamous semantic gap
Large-scale 3D modeling
� ACQUISITIONACQUISITION
3D acquisition taxonomy
uni-directional multi-directional
passive
uni-directional multi-directional
active
3D acquisition methods
LIDARStereo
Str. from motion
Shape-from
- texture
- contour
- silhouettes
- defocus
- shading
Line scanning
Structured light
Photom. stereo
3D acquisition taxonomy
uni-directional multi-directional
passive
uni-directional multi-directional
active
3D acquisition methods
LIDARStereo
Str. from motion
Shape-from
- texture
- contour
- silhouettes
- defocus
- shading
Line scanning
Structured light
Photom. stereo
Underlying principles
Structure from motion
Structured light
TRIANGULATION
Structured light
LIDAR TIME-OF-FLIGHT
Stereo
The underlying principle is “triangulation” :
�
Triangulation � 3D measurements
Active triangulation
8
Projector
Camera
Active triangulation
Camera image
Active triangulation
Need for registration...
Has to be automatic for
large-scale work
Time-of-flight
Cyrax
Distance = C x T ÷2
Time-of-flight
Cyrax
Laser sweeps over surface
No correspondences needed
Large-scale 3D capture
� Speed becomes essential
� … fast calculations (e.g. helped by GPU)
� … but also easy setup and data capture
� Increasingly systems that are based on comsumer HW� Increasingly systems that are based on comsumer HW
� … and intelligent modeling tools
� … and automatic data capture
Tracking and
Calibration
Densedepth
estimation
3Dsurface
modeling
3D acquisition / structure-from-motion
Points and cameras 3D modelsDepth map
Shape
3D acquisition / structure-from-motion
Shapefrom Video(KUL)
3D acquisition / structure-from-motion
ShapeShapefrom Stills(KUL)
ARC – Free 3D webservice
ARCARCARCARCAutomatic
Reconstruction Reconstruction
Conduit
non-commercial
use is free
www.arc3d.be
Automatic photo mining
18
(kooaba)
Use in movies / e.g. King Arthur (2003)
Dublin: The Hollywood Valley (Eyetronics)
19
Area B
Use in movies / e.g. King Arthur (2003)
20
Combined areas
Use in movies / e.g. King Arthur (2003)
21
Use in movies / e.g. King Arthur (2003)
Use in movies / e.g. King Arthur (2003)
One-shot
ShapeCam
3D acquisition / structured light
3D acquisition / structured light
Jackie Chan
Latest Latest RushHour Movie
(Eyetronics)
3D acquisition / structured light
MLB
1200 players1200 playersfor gaming
(Eyetronics)
3D acquisition / structured light
MLB
1200 players1200 playersfor gaming
(Eyetronics)
3D acquisition / structured light
MLB
1200 players1200 playersfor gaming
(Eyetronics)
4D acquisition / structured light
3D snapshots in fast succession (KUL)
4D acquisition / structured light
30
(KUL &
ETH Zurich)
4D acquisition / structured light
Real-time 3Dextraction, incl.texture
(KUL &
ETH Zurich)
3D acquisition / structured light
LIDAR
Chevron hydrocarbon plant
LIDAR
Cyrax / Leica
LIDAR
Large-scale 3D modeling
� EDITING / REPRESENTATIONEDITING / REPRESENTATION
LIDAR
(Cyrax / Leica)
LIDAR
39
LIDAR
� 400’x500’ area
� 10 vessels; 5 pumps
� 6,000 objects
� 81 scans from 30 tripod locations
� Cyrax field time = 50 hrs
Measuring for cartographic surveying
Large-scale 3Dmeasurements
Efficiency of the interface !
Procedural modeling
Modeling using rules from a style-specific shape grammar
(ETH Zurich & Procedural)
Large-scale 3D modeling
� STREAMING / RENDERINGSTREAMING / RENDERING
QoS
• Bandwith of channel and type of platform!
• Local, over network, wireless, ...?
• PC cluster, PC, PDA, mobile phone, ...?
• Level-of-Detail mechanisms• Level-of-Detail mechanisms
• Compact representations
• Exploit differential sensitivity HVS
• 3D meshes or... point splashing, IBR, ...?
A4MC3
WIFI
connected
community community
with PDAs
(EDM, IMEC, VUB)
MobiComp
(Un. Kent &&
Bologna )
MeshGrid
A scalable coding scheme for 3D models that:• enables progressive transmission• offers quality, resolution scalability and region-of-interest coding• guarantees a minimum distortion at any bit-rate• provides resilience against transmission errors
MeshGrid
A MeshGrid model consists of a connectivity-wireframe (CW) and a reference-grid (RG) that acts as the reference-system (RS) of the model.
MeshGrid
G1
V
G2
G1
G2
V
1
1 2
, with [0,1)G V
offset offsetG G
= ∈
uuuur
uuuuur
Vertex offset is a relative value:
1 1 2G V G G offset=
uuuur uuuuur�
Update vertex position whengrid is deformed or animated:
MeshGrid
� Volumetric Morphing
� Volumetric Animation
� Scalability, Regions Of Interest Coding
(a) (b) (c)
� Volumetric Animation
Image-based Rendering
(KUL)
(Un. Bonn)
51(EDM)
Large-scale 3D modeling
� PRESENTATIONPRESENTATION
Shuttered Walls & Synchronized Cameras
Blue-C project
walls transparent –acquisition, no projection
walls opaque –no acquisition, projection
(ETH Zurich)
Blue-C project
CyberWalk project
Towards holistic vision – 3D & recognition
56