key laboratory of machine perception (moe), peking university applications of 3d digital...

Key Laboratory of Machine Perception (MOE), Peking UniversityKey Laboratory of Machine Perception (MOE), Peking University

Applications of Applications of 3D Digital Technologies 3D Digital Technologies in Archaeology in Archaeology and Digital Heritageand Digital Heritage

Hongbin Zha ( 査紅彬）3D Visual Computing and Robotics Gro

upKey Lab on Machine Perception (MOE)

Peking University, China


OutlineOutline

IntroductionIntroduction Task definition of 3D digitizationTask definition of 3D digitization ApplicationsApplications Process pipeline and challengesProcess pipeline and challenges

Applications in Digital HeritageApplications in Digital Heritage Digital Longmen Grottos Digital Longmen Grottos （竜門石窟）（竜門石窟） ProjecProjec

tt Computer-assisted line drawingComputer-assisted line drawing Digital restorationDigital restoration

Maijishan Grottos Maijishan Grottos （麦積山石窟）（麦積山石窟） Project and Project and OthersOthers


3D Modeling of Real 3D Modeling of Real Objects Objects

mini-statue

mesh rendered mesh

Copying machineCopying machine

3D digital copy3D digital copy


Objectives of 3D DigitizationObjectives of 3D Digitization

Geometry Digitization:Geometry Digitization: Cover the whole surfaces Cover the whole surfaces

of an objectof an object Describe accurate shape Describe accurate shape

and structure detailsand structure details Digitization of Other Digitization of Other

Attributes:Attributes: Textures and colorsTextures and colors Transparency and Transparency and

glossiness glossiness …… ……


Characteristics of 3D Modeling Technologies

OObjects:bjects: WWith complex geometrical and topological ith complex geometrical and topological

structures, with dynamic deformation or structures, with dynamic deformation or movements -- architectures, human bodies movements -- architectures, human bodies or outdoor scenes. or outdoor scenes.

GGeometry Processing: eometry Processing: MModel simplification, data compression odel simplification, data compression

and transmission for large-scale scenes.and transmission for large-scale scenes.

AAutomatic Modeling:utomatic Modeling: 3D laser scanning, hybrid imaging 3D laser scanning, hybrid imaging

systems, systems, ddata acquisition using mobile ata acquisition using mobile platplatformsforms..


Break down Wall between Real and Cyber Worlds


Digital Museum and Digital Digital Museum and Digital HeritageHeritage

Digitization for museum Digitization for museum and heritage objectsand heritage objects

Interactive walkthrough Interactive walkthrough of heritage sitesof heritage sites

�Digital recording and Digital recording and documentation documentation �Digital exhibitionDigital exhibition�Digital reconstruction, Digital reconstruction, measurement and analysismeasurement and analysis�Remote visiting and Remote visiting and educationeducation


Digital recording and Digital recording and duplication of heritage articlesduplication of heritage articles

Digital restoration of heritage Digital restoration of heritage relicsrelics

Behavior analysis of life forms Behavior analysis of life forms based on skeleton fossilsbased on skeleton fossils

Digital recording of Digital recording of archaeological excavation sitesarchaeological excavation sites

Virtual reconstruction of Virtual reconstruction of historic scenerieshistoric sceneries

3D Digitization Assisted 3D Digitization Assisted Archaeology and Heritage Archaeology and Heritage


NRC Tylosaurus sp.Mosasaur fossils

Sta

nfo

rd

David

IBM Pietà Tokyo Buddha

Heritage Related WorkHeritage Related Work


Pipeline for 3D Imaging and Modeling


3D Scanners3D Scanners


3D Photography or 3D TV

Develop devices that can capture 3D imageDevelop devices that can capture 3D images or videos as general camerass or videos as general cameras image-based modelingimage-based modeling camera arrays or domescamera arrays or domes hybrid imaging systems hybrid imaging systems


Complexity in 3D shapes Complexity in 3D shapes and topological and topological structuresstructures

Discontinuities in data Discontinuities in data parameterization and parameterization and irregularities in samplingirregularities in sampling

Well-established digital Well-established digital processing tools (FFT, processing tools (FFT, Wavelet analysis etc.) can Wavelet analysis etc.) can not be taken into good not be taken into good use.use.

Difficulties in 3D Digital Difficulties in 3D Digital Geometry ProcessingGeometry Processing

sound

image

video


Big Sizes of 3D Models

3D models become 3D models become larger and larger: larger and larger:

High requirements for High requirements for geometrical and other geometrical and other attribute detailsattribute details

Complexities of objects Complexities of objects to be modeled to be modeled

Advance in 3D data Advance in 3D data acquisition technologiesacquisition technologies


OutlineOutline

IntroductionIntroduction Task definition of 3D digitizationTask definition of 3D digitization ApplicationsApplications Process pipeline and challengesProcess pipeline and challenges

Applications in Digital HeritageApplications in Digital Heritage Digital Longmen Grottos Digital Longmen Grottos （竜門石窟）（竜門石窟） ProjecProjec

tt Computer-assisted line drawingComputer-assisted line drawing Digital restorationDigital restoration

Maijishan Grottos Maijishan Grottos （麦積山石窟）（麦積山石窟） Project and Project and OthersOthers


Digital HeritageDigital Heritage

19

Display Fundamental Fundamental ResearchResearch

An input imageReconstruction result

• Modeling targets with complex geometry• Computation on 3D models• ……

Transformation

Digitization Projects Virtual Tourism of Maiji Grottoes (Collaborated with Top

pan) Surveying and Mapping Caves (Collaborated with Longm

en Grottoes) ……

Data Acquisition

Laser Scanners

Laser ScannerGPS IMU

Video Camera

Laser Scanner

Geometric reconstruction & Texture generation


Projects

Beijing

XiangTang Shan Grottoes

LongMen GrottoesMaiJiShan Grottoes

Five-Layer Tower

ChongQing DiaoYu Town

YangCan Tomb

NanYueWang Museum


Joint Projects with Longmen Grottoes

Computer assisted line draComputer assisted line drawing for archaeological reswing for archaeological researchearch

Virtual restoration of heritVirtual restoration of heritage sites and relicsage sites and relics

Digitization standards for gDigitization standards for grotto scanning and modelirotto scanning and modelingng

““Digital Longmen GrottoeDigital Longmen Grottoes” projects” project


Surveying and Mapping in Surveying and Mapping in Archaeology ResearchArchaeology Research

The purpose of surveying and mapping is to measure The purpose of surveying and mapping is to measure and record accurate, readable and parallel-projection and record accurate, readable and parallel-projection feature curvesfeature curves of cultural heritage. of cultural heritage.

Archeological reports: Archeological reports: Photos: record current situations and information to Photos: record current situations and information to

infer knowledge such as its producing dateinfer knowledge such as its producing date Texts: describe all explicit Texts: describe all explicit

known informationknown information line drawings: reflect line drawings: reflect

accurate, vivid and feature accurate, vivid and feature emphasized figuresemphasized figures


Line Drawing in Archeology Traditional methods:Traditional methods:

�Set baseline and grid, then measure using rulersSet baseline and grid, then measure using rulers�Disadvantages:Disadvantages:

Low accuracyLow accuracyTime-consumingTime-consumingDifficult to draw lines on multiple sections and Difficult to draw lines on multiple sections and

large areaslarge areas


3D Digitizing 3D Digitizing PipelinePipeline

Pre-Processing

Post-Processing

Registration

Merging

Display

Measurin

g

Restoratio

n

Application

s

＋＝

……

3D Scanning


3D Scanners in Use3D Scanners in Use

Cyrax 2500

RIEGL LMS-Z210i

OKIO

VIVID 910


3D scanner

Data Acquisition

Get 3D range information of surface

s.

Scan an object from different view p

oints.


Pre-processing

Denoising

Decimation


Registration


Data Merging


Post-processing


Line Drawing


3D Model Aided Line Drawing

Photo

Rendered 3D ModelLine Drawing Based on

3D Model AdvantageAdvantage

�Precise measurementPrecise measurement�FastFast�Parallel projection, no distortionParallel projection, no distortion�Multiple sections from any positionMultiple sections from any position�Line drawing for large scale scenesLine drawing for large scale scenes


Sections from MultipleSections from Multiple PositionsPositions


LeiGuTai, LongMen Grottoes

Pho

to

8.5m

20m

Traditional Traditional methodmethod

480 hours480 hours

Our Our methodmethod

8 hours (for modeling) 8 hours (for modeling) ++

24 hours (for line 24 hours (for line drawing)drawing)

Saved Saved timetime

448 hours448 hours

Para

llel projection m

ap

Line drawing

Efficiency ofEfficiency of 3D 3D Model Aided Model Aided Line DrawingLine Drawing


Buddha statue in the north cave

Photo Parallel projection map Line drawing

2.55m

0.9m

1.3m

1.8m

TraditioTraditional nal methodmethod

360 hours360 hours


4 hours (for 4 hours (for modeling)+32 hours modeling)+32 hours (for line drawing)(for line drawing)


324 hours324 hours



Para

llel p

roje

ction

m

ap

Lin

e d

raw

ing

Ph

oto

Bodhisattva statue in the middle cave

0.65m

0.4m

0.5m

1.85m

TraditioTraditional nal methodmethod

96 hours96 hours


4 hours (for 4 hours (for modeling)+40 modeling)+40 hours (for line hours (for line drawing)drawing)


52 hours52 hours



0.65m

0.40m

0.5m

1.85m

Automatic Line DrawingAutomatic Line Drawing

image from 3D dataimage from 3D dataautomatically automatically

produced sketchproduced sketchmodel assisted drawingmodel assisted drawing


A large scene in Longmen Grottoes

Automatic Line DrawingAutomatic Line Drawing

A Buddha statue


Digital Restoration

Budda in GuYang Cave, LongMen Grottoes


Bodhisattva in HuoDing Cave, LongMen Grottoes

Digital Restoration


LongMen Grottoes


MaiJiShan Grottoes A joint project: Toppan Printing Co., Wased

a Univ., Peking Univ. Objective: to develop advanced 3D modelin

g, rendering, displaying and interaction methods for digital heritage

A case study: Maijishan Grottos – production of 3D video for the Cave #123


MaiJiShan Grottoes


Budda in South Cave, XiangTangShan Grottoes

Digital Restoration


Statues in South Cave, XiangTangShan Grottoes

Digital Restoration


XiangTangShan Grottoes


ConclusionsConclusions

3D digital technology is very useful for 3D digital technology is very useful for preservation, research, and knowledge preservation, research, and knowledge dissemination for cultural heritagesdissemination for cultural heritages

It requires collaboration among different It requires collaboration among different disciplines: archaeology, cultural heritage, disciplines: archaeology, cultural heritage, information science and technology, …information science and technology, … Well-defined objectivesWell-defined objectives High-quality resultsHigh-quality results Useful in some aspects for the applicationUseful in some aspects for the application

How to find research issues from the How to find research issues from the application work?application work?


Thank You!Thank You!


TombTomb� 400m400m22

ObjectsObjects� 2020

GuangZhou NanYueWang Museum


Digital Archeology Report

Photo

Database managementDatabase management�Photos, 3D models, and relevant textsPhotos, 3D models, and relevant texts�From large scenes to single statuesFrom large scenes to single statues�Basic operations Basic operations 3D measurement and analysis 3D measurement and analysis Section from any positionSection from any position

3D View Measurement


Digital Museum

3D scanning and picturing

3D modeling

Texture mapping Virtual display


Other Cases

key laboratory of machine perception (moe), peking university applications of 3d digital...

Documents