peter allen, ioannis stamos*,alejandro troccoli, ben smith, marius leordeanu*, y.c. hsu dept. of...
TRANSCRIPT
![Page 1: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/1.jpg)
Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu
Dept. of Computer Science, Columbia University Dept. of Computer Science, Hunter College, CUNY*
3D Modeling of Historic Sites Using Range and Image Data
![Page 2: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/2.jpg)
•Sites are subject to erosion, wear, vandalism•Moving targets: many phases of construction, damage and repair over many years
•Need to track changes, foresee structural problems
•Modeling allows a wider audience to ”virtually” see and tour these sites
•Our Focus: methods that can reduce modeling time using automatic methods
Preserving Cultural Heritage Sites
![Page 3: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/3.jpg)
Cathedral St. Pierre, Beauvais, France
![Page 4: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/4.jpg)
Modeling the CathedralGoals:
• Cathedral on the World Monuments Fund's Most Endangered List.
• Create 3-D model to examine weaknesses in the building and proposed remedies
• Establish baseline for condition of Cathedral
• Visualize the building in previous contexts
• Basis for a new collaborative way of teaching about historic sites, in the classroom and on the Internet.
![Page 5: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/5.jpg)
• Commissioned in 1225 by Bishop Milon de Nanteuil
• Only the choir and transepts were completed - choir in 1272
• In 1284 part of the central vault collapsed
• Area where the nave and façade would be is still occupied by the previous church constructed just before 1000.
• Completed in 16th century, the transept was crowned by an ambitious central spire that allowed the cathedral to rival its counterpart in Rome.
• The tower collapsed on Ascension Day in 1573.
History: 1200 - 1600
![Page 6: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/6.jpg)
Rendition of original central
spire
![Page 7: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/7.jpg)
• Cathedral survived intense incendiary bombing that destroyed much of Beauvais in WW II.
• Between 1950-80 many critical iron ties were removed from the choir buttresses in a damaging experiment.
• Temporary tie-and-brace system installed in the 1990s may have made the cathedral too rigid, increasing rather than decreasing stresses upon it.
• There continues to be a lack of consensus on how to conserve the essential visual and structural integrity of this Gothic wonder.
History: 20th Century
![Page 8: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/8.jpg)
Problems with the Structure
• Wind Oscillation from English Channel winds
• Strange inner and outer aisle construction – can cause rotational moments in the structure
• Leaking Roof, foundation is settling
• Built in 3 campaigns over hundreds of years with differing attention to detail
![Page 9: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/9.jpg)
Time-Lapse Image - Spire Movement Due to Wind
![Page 10: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/10.jpg)
Technical Challenges
• Create Global and coherent geometric models: handle full range of geometries
• Reducing data complexity
• Registration of MANY million point data sets
• Range and intensity image fusion
![Page 11: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/11.jpg)
Beauvais Cathedral: Exterior Scanning Session
![Page 12: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/12.jpg)
Beauvais Cathedral: Interior
![Page 13: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/13.jpg)
Beauvais Cathedral: Interior Scanning Session
![Page 14: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/14.jpg)
Exterior: Raw Range Scan
![Page 15: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/15.jpg)
Beauvais: Scan Detail
![Page 16: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/16.jpg)
Range Registration
3 Step Process:
1. Pairwise registration between overlapping scans. Match 3D lines in overlapping range images.
2. Global registration using graph search to align all scans together.
3. Multi-scan simultaneous ICP registration algorithm (Nishino et. al.)
Produces accurate registration.
![Page 17: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/17.jpg)
Segmentation Algorithm
• Creates reduced data sets (~80%).• Fit local plane to neighborhood of range points.• Classify range points: planar, non-planar, unknown.• Merge into connected clusters of co-planar points.• Identify boundaries of planes. • Used to find prominent linear features for matching.
![Page 18: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/18.jpg)
N N1 2
P P1 2
R12
Patches fit around points P1 and P2
P1 and P2 are coplanar if:
• a=cos (N1 . N2) < angle threshold• d=max(|R12 N1|, |R12 N2|) < distance threshold
-1
Local Planarity Comparison
![Page 19: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/19.jpg)
Segmentation and 3-D Registration Lines
![Page 20: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/20.jpg)
Pairwise Registration of Scans: Overview
• Segmenter creates planes and associated boundary lines
• Matching lines in 2 scans can be problematic
• Attributes of lines serve as a good filter for matching lines: length of lines, area of planar region that line bounds
• Use pairs of matched lines to compute transform
• Transform each line in one scan to the other. Results are graded by metric of number of matched lines after transform
• Choose best graded transform
![Page 21: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/21.jpg)
Pairwise Registration
![Page 22: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/22.jpg)
Registered Scans – Beauvais Cathedral
![Page 23: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/23.jpg)
Global Registration
![Page 24: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/24.jpg)
Graph Search Global Registration• Create weighted graph of scans. Edges of graph are confidence in finding correct registration between pairs of scans
• Confidence (cost) is number of correctly aligned lines after applying registration (R,T)
• Global Registration: find max-cost path from pivot scan to each scan
![Page 25: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/25.jpg)
Final ICP Registration
![Page 26: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/26.jpg)
![Page 27: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/27.jpg)
![Page 28: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/28.jpg)
Beauvais Cathedral Model: Fly-Thru
![Page 29: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/29.jpg)
Inside-Out: Beauvais Cathedral
![Page 30: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/30.jpg)
Texture Mapped Models
![Page 31: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/31.jpg)
Texture Mapped Models
![Page 32: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/32.jpg)
Texture Mapped Models
![Page 33: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/33.jpg)
Automating the Process:
•Robot can serve as a sensor platform containing ranging and imaging devices
•Onboard sensors can register sensor data with the environment: GPS, Odom., Vision
•Given a 2-D map of an environment, robot can augment 2-D map with 3-D models
•Planner can interact with robot navigator to move to new viewing sites
Build a Mobile Site Modeling Robot
![Page 34: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/34.jpg)
GPS
DGPSScanner
Network
Camera
PTUCompass
The AVENUE Mobile Platform
UTONOMOUS
EHICLE FOR
XPLORATION AND
AVIGATION IN
RBAN
NVIRONMENTS
PC
Sonars
![Page 35: Peter Allen, Ioannis Stamos*,Alejandro Troccoli, Ben Smith, Marius Leordeanu*, Y.C. Hsu Dept. of Computer Science, Columbia University Dept. of Computer](https://reader035.vdocuments.us/reader035/viewer/2022081519/56649ccd5503460f94997f29/html5/thumbnails/35.jpg)
Future Work
• Continue to automate the model building process
• Real-time texture mapping for realistic walk-throughs
• View planning: finding the right viewpoints and number of views
• Incorporate sensor planning with robot path planning to plan next view
• Modeling with non-planar segments.
• Merging multiple overlapping photos.