three dimensional model construction for visualization
DESCRIPTION
Three Dimensional Model Construction for Visualization. Avideh Zakhor. Video and Image Processing Lab University of California at Berkeley [email protected]. Outline. Goals and objectives Previous work by PI Directions for future work. Goals and Objectives. - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/1.jpg)
Three Dimensional Model Construction for Visualization
Avideh Zakhor
Video and Image Processing LabUniversity of California at
![Page 2: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/2.jpg)
Outline
Goals and objectivesPrevious work by PIDirections for future work
![Page 3: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/3.jpg)
Goals and Objectives
Develop a framework for fast, automatic and accurate 3D model construction for objects, scenes, rooms, buildings (interior and exterior), urban areas, and cities.
Models must be easy to compute, compact to represent and suitable for high quality view synthesis and visualization
Applications: Virtual or augmented reality fly-throughs.
![Page 4: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/4.jpg)
Previous Work on Scene Modeling
Full/Assisted 3-D ModelingKanade et al.; Koch et al.; Becker & Bove; Debevec et
al.; Faugeras et al.; Malik & Yu. Mosaics and Panoramas
Szeliski & Kang; McMillan & Bishop; Shum & Szeliski
Layered/LDI RepresentationsWang & Adelson; Sawhney & Ayer; Weiss; Baker et al.
View Interpolation/IBR/Light FieldsChen & Williams; Chang & Zakhor; Laveau & Faugeras; Seitz & Dyer; Levoy & Hanrahan
![Page 5: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/5.jpg)
Previous Work on Building Models
Nevatia (USC): multi-sensor integrationTeller (MIT): spherical mosaics on a
wheelchair sized rover, known 6DOFVan Gool (Belgium): roof detection from
aerial photographsPeter Allen (Columbia): images and laser
range finders; view/sensor planning.Faugeras (INRIA)
![Page 6: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/6.jpg)
Previous Work on City Modeling
Planet 9: Combines ground photographs with existing city
maps manually.UCLA Urban Simulation Team:
Uses mutligen to create models from aerial photographs, together with ground video for texture mapping.
Bath and London models by Univ. of Bath. Combines aerial photgraphs with existing maps.
All approaches are slow and labor intensive.
![Page 7: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/7.jpg)
Work at VIP lab at UCB
Scene modeling and reconstruction.
![Page 8: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/8.jpg)
Multi-Valued Representation: MVR
Level k has k occluding surfacesForm multivalued array of depth and
intensity
![Page 9: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/9.jpg)
Observations
![Page 10: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/10.jpg)
Imaging geometry (1)
Planar translation
![Page 11: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/11.jpg)
Imaging Geometry (2)
Circular/orbital motion
![Page 12: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/12.jpg)
Dense Depth Estimation
Estimate camera motionCompute depth maps to build MVRs
Low-contrast regions problematic for dense depth estimation.
Enforce spatial coherence to achieve realistic, high quality visualization.
![Page 13: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/13.jpg)
Block Diagram for Dense Depth Estimation
Planar approximation of depth for low contrast regions.
![Page 14: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/14.jpg)
Oroginal Sequences
“Mug” sequence(13 frames)
“Teabox” sequence(102 frames)
![Page 15: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/15.jpg)
Low-Contrast Regions
Mug sequence Tea-box sequence
Complete tracking
![Page 16: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/16.jpg)
Multiframe Depth Estimation
Apply iterative estimation algorithm to enforce piecewise smoothness, without smoothing over depth discontinuities.
![Page 17: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/17.jpg)
Multiframe Depth Estimation
Multiframe Stereo+ Low-Contrast Processing
+ Piecewise Smoothing
Multiframe Stereo+ Low-Contrast Processing
+ Piecewise Smoothing
Mug Tea-box
![Page 18: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/18.jpg)
Multivalued Representation
Project depths to reference coordinates
![Page 19: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/19.jpg)
Results (1)
Mug sequence
Multivalued representation for frame 4(Level 0)
![Page 20: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/20.jpg)
Results
Mug sequence
Multivalued representation for frame 4(Level 1)
![Page 21: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/21.jpg)
Results
Mug sequence
Multivalued representation for frame 4(Combining Levels 0 and 1)
![Page 22: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/22.jpg)
Results
Reconstructed sequence Arbitrary flythrough
Mug sequence
![Page 23: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/23.jpg)
Results (2)
Multivalued representation for frame 22(Intensity, Level 0)
Teabox sequence
![Page 24: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/24.jpg)
Results
Multivalued representation for frame 22(Depth, Level 0)
Teabox sequence
![Page 25: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/25.jpg)
Results
Multivalued representation for frame 22(Intensity, Level 1)
Teabox sequence
![Page 26: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/26.jpg)
Results
Multivalued representation for frame 22(Depth, Level 1)
Teabox sequence
![Page 27: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/27.jpg)
Results
Multivalued representation for frame 22(Intensity, combining Levels 0 and 1)
Teabox sequence
![Page 28: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/28.jpg)
Results
Multivalued representation for frame 22(Depth, combining Levels 0 and 1)
Teabox sequence
![Page 29: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/29.jpg)
Results
Multivalued representation for frame 86(Intensity, Level 0)
Teabox sequence
![Page 30: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/30.jpg)
Results
Multivalued representation for frame 86(Depth, Level 0)
Teabox sequence
![Page 31: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/31.jpg)
Results
Multivalued representation for frame 86(Intensity, Level 1)
Teabox sequence
![Page 32: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/32.jpg)
Results
Multivalued representation for frame 86(Depth, Level 1)
Teabox sequence
![Page 33: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/33.jpg)
Results
Multivalued representation for frame 86(Intensity, combining Levels 0 and 1)
Teabox sequence
![Page 34: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/34.jpg)
Results
Multivalued representation for frame 86(Depth, combining Levels 0 and 1)
Teabox sequence
![Page 35: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/35.jpg)
Multiple MVRs
Perform view interpolation w/many MVRs
![Page 36: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/36.jpg)
Results: multiple MVRs
Reconstructed sequencefrom MVR86
Reconstruct sequence from MVR22
Teabox sequence
![Page 37: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/37.jpg)
Results: Multiple MVRs
Reconstructed sequence Arbitrary flyaround
![Page 38: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/38.jpg)
Extensions
Complex scenes with many “levels” are difficult to model with MVR; e.g. trees, leaves, etc
Difficult to ensure realistic visualization from all angles; Need to plan capture process carefully.
Tradeoff between CG polygon modeling and IBR; Use both in real visualization databases. Build polygon models from MVR.
![Page 39: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/39.jpg)
Issues for model construction
Choice of geometry for obtaining data
Choice of imaging technology.Choice of representation.Choice of models.Dealing with time varying scenes.
![Page 40: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/40.jpg)
Extensions:
So far, addressed “outside in” problem: Camera looked inward to “scan” the
object.Future work will focus on the “Inside
out” problem: Modeling a room, office. Modeling exterior or interior of a building Modeling an urban environment e.g. a city
![Page 41: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/41.jpg)
Strategy
Use: Range sensors, position sensors (GPS),
Gyros(orientation), omni camera, video. Existing datasets: 3D CAD models,
digital elevation maps (DEM), DTED, city maps, architectural drawings: apriori information
![Page 42: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/42.jpg)
Modeling interior of buildings
Leverage existing work in the computer graphics group at UCB: 3D model of Soda hall available from the “soda
walkthrough” project. 3D model built out of architectural drawings Use additional video, and laser range finder input
toEnhance the details of the 3D model: furniture,
etcAdd texture maps for photo-realistic walk-
throughs.
![Page 43: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/43.jpg)
City Modeling
Develop a framework for modeling parts of city of San Francisco: Use aerial photograph as provided by Space
Imaging Corp; resolution 1 ft. Use digitized city maps Use ground data collection vehicle to collect range
and intensity video from a panoramic camera, annotated with 6 DOF parameters.
Derive data fusion algorithms to process the above in speedy, automated and accurate fashion.
![Page 44: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/44.jpg)
Requirements
Automation (little or no interaction needed from human operators)
Speed: must scale with large areas and large data sets.
AccuracyRobustness to location of data collection.Ease of data collection.Representation suitable to hierarchical
visualization databases.
![Page 45: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/45.jpg)
Relationship to others
USC: accurate tracking and registration algorithms needed for model construction.
Syracuse: uncertainty processing, and data fusion for model construction.
G. Tech: How to combine CG polygonal model building with IBR models in vis. database? How can vis. databases deal with photo-realistic rendering?
![Page 46: Three Dimensional Model Construction for Visualization](https://reader036.vdocuments.us/reader036/viewer/2022062500/56815932550346895dc66ab1/html5/thumbnails/46.jpg)
Conclusions
Fast, accurate and automatic model construction is essential to mobile augmented reality systems.
Our goal is to provide photo-realistic rendering of objects, scenes, buildings, and cities, to enable, visualization, navigation and interaction.