uav-based damage mapping and the fp7-project …

UAV-BASED DAMAGE MAPPING AND THE FP7-PROJECT RECONASS

Norman Kerle. ESA DepartmentITC-OOA-Group

(with materials by Markus Gerke, Jorge Fernandez and Anand

Vetrivel)

2

ITC/ University Twente

International Institute for Geoinformation science and Earth Observation – independent for 60 years

Faculty of University of Twente 3 years ago

Various degree courses and certificates in Disaster management Earth sciences Geoinformatics Governance Land administration Natural resources Urban planning Water resources

GLOSSY meeting - Wageningen - 9 May 2014

1 20 40 976Netherlands

1105Indonesia

6010

Student numbers

100 400300200

3

ITC/ University Twente

Houses the United Nations University- ITC Centre for Spatial Analysis and Disaster Risk Management Training, education and curriculum

development

Knowledge development and research collaboration

Advisory services

In collaboration with many partners


www.unu-drm.nl

4

Me @ ITC

Geographer with study background in Hamburg (Germany), Ohio State (US) and Cambridge (UK)

Since early 1990s work in the disaster field, with focus on remote sensing

PhD in volcano remote sensing (lahars)

Advanced image analysis, and focus on object-oriented analysis (OOA), increasingly with oblique data from UAV


http://www.itc.nl/ooa-grouphttp://www.itc.nl/about_itc/resumes/kerle.aspx

Following disasters, rapid and comprehensive damage information is critical

Assessment is challenging

RATIONALE & MOTIVATION

5GLOSSY meeting - Wageningen - 9 May 2014

RATIONALE

6

VERTICAL IMAGE-BASED ASSESSMENT HAS LIMITATIONS

Port-au-Prince, Haiti

Oblique data are promising, but traditionally have limitations

Early experiments with Pictometry data that may overcome those

Not a UAV, but a MAV, 5 views

Use of data ideally in an automated manner, using object-basedapproaches

RATIONALE

7

PROMISE OF OBLIQUE DATA


5 digital cameras (1 nadir, 4 oblique)

GSD 15cm (nadir images, flying height ca. 1000m)

overlap for stereo coverage

PICTOMETRY

8

BASICS OF THE SYSTEM


Which Pictometry features allow damage mapping? (elevation,geometrics features, textures, etc.)

Identification of planar (=intact) and spectrally homogenous facades androof sections

Supervised classification of image segments

Effect of training on the classification

OBJECTIVES


Orthorectification of oblique images and disparity images (depth maps)

Feature extraction from ortho images and DSM

Assumption: intact buildings have planar surfaces

Find evidence for planarity through

Disparity maps (geometric homogeneity)

Image information (radiometric homogeneity)

3D point cloud segmentation to find planar surfaces

DAMAGE MAPPING: METHODOLOGY


5 classes were defined:a. Intact roofb. Broken roof/ rubblec. Intact facaded. Bare grounde. Vegetation

Training data by two interpreters Classification using a total of 22 features

in machine learning approach Final step: combination per building

EARTHQUAKE DAMAGE ASSESSMENTCLASSIFICATION OF IMAGE SEGMENTS

11

Direct damage indicators

D1

D2

D3

D4

D5

EARTHQUAKE DAMAGE ASSESSMENTCLASSIFICATION PER BUILDING

12

Test on a 6 block area of Port au Prince, Haiti

Depth images from stereo pairs (for different views)

3D point cloud


PARALLAX DISPARITY FOR IMAGE PAIR – MERGE OF MULTIPLE VIEWS

13

Planar areas have limited disparity

Rubble and broken features appear noisy


DAMAGE MAPPING RESULTS: WESTERN VIEW EXAMPLE

14

Intact roofBroken roof/ rubbleIntact facadeBare groundVegetation

Ortho image

Detected damage


15

Results using trainingdata by 2 analysts

For details seePE&RS paper, Gerke & Kerle, 2011

ACCURACY ASSESSMENT & CONCLUSIONS

16

Overall accuracy of approx. 70% (63% per building) Errors at image top where heights were missing

Problem: damage indicators do not add up linearly


ONGOING RESEARCH: USING UAV DATA & ECOGNITIONFOR THE CLASSIFICATION INSTEAD OF TRAINING DATA

17

Limitations: Expensive Pictometry data Single view, limited spatial resolution, limited control Adding damage indicators Avoid use of training data

Solution: Get your own UAV and eCognition

Aibot X6 (Aibotix )GLOSSY meeting - Wageningen - 9 May 2014

OBJECTIVES

18

Obtain data with a pole-based camera and a UAV Identify damage indicators in multi-perspective images with OOA Create a rough 3D building model with images and damage features

draped Decision tree for semantic reasoning


DATA ACQUISITION


DATA COLLECTION IN ITALY, GERMANY, NETHERLANDS


METHODOLOGY

Buildinglevel

Collapsed(D5)

Non-collapsed(D1-D4)

D5

Building’ssurroundings

Rubblepiles

Roof Facade

Stan-dingCollapsed

D4 Non-inclined

No rubblepiles

Facade’sgeometry

Inclined

Point cloud

Segmentation

UAV images

Colors indicate orientation: red is horizontal, blue vertical.Here change in color signals sloping roof

GLOSSY meeting - Wageningen - 9 May 2014 21

METHODOLOGY

22

UAV images

Buildinglevel

Point cloud

Segmentation

Collapsed(D5)

Non-collapsed(D1-D4)

Building’ssurroundings

D5 D4

Rubblepiles

No rubblepiles

Facade’sgeometry

Roof Facade

StandingCollapsed

Non-inclinedInclined

Extracted features:

3D constructvisualization Random

scenarios

Per-view scoreD1-D2-D3-D4

Per-building decision tree

Uncertainty evaluation

D4 D3 D2 D1

Facadeassessment

Expertanalysis

OOA

View/facadelevel

D1

Standing

Non-inclined

23

DAMAGE DETECTION WITH OOA


24

PER-BUILDING ASSESSMENT

Challenges:• How to process damage

evidence?• How to deal with occlusion?


25

EXPERT-BASED ASSESSMENT

ID EMS-98 scoreFacade 1 D0Facade 2 D3Façade 3 OccludedFaçade 4 D4Roof 1 D1

EXPERT-BASED ASSESSMENT

26

Problem: simple building primitive Preferable: based on actual DSM


ITC, University of Twente 27

Multi-view images from UAVof complex structures

3D point clouds Segmentation for 3D damage assessment

Rich 3D model data from, e.g., from ItalyChallenge is how to use such complex information


ITC, University of Twente 28

Multi-view images from UAV

Demolished factory building. 300 images taken, combined into a 3D point cloud.


RECONASS

29

2 FP7 projects on damage mapping

RECONASS: Reconstruction and Recovery Planning: Rapid and Continuously Updated Construction Damage, and Related Needs Assessment (start: 1 Dec 2013; 3 ½ years)

INACHUS: Technological and Methodological Solutions for Integrated Wide Area Situation Awareness and Survivor Localisation to Support Search and Rescue Teams (start: late 2014; 4 years)


OBJECTIVES OF RECONASS

30

Provide rapid assessment on structural integrity of high-value buildings (ministries, hospitals), based on wireless sensor networks, finite element modelling, and external surveillance with UAV technology

3.5 years, 10 partners (German THW as disaster response organization, ITC for geoinformatics, and 8 engineering organizations)

ITC covers one work package (Synergistic Damage Assessment with Air- and Space-borne Remote Sensing), with full-time AIO researcher

4 objectives


OBJECTIVES

31

Detailed and element-specific damage assessment on per building level using multi-view oblique aerial images

Synergistic use of UAV and wireless sensor network for complete, reliable and accurate damage assessment.

Integration of satellite- and airborne data-based information for synoptic damage assessment

Extension of sensor network with chemical and biological sensors for critical hazard monitoring


TASKS

32

- Consider specific damage- Joint use of spectral/ color image information

(texture, minute features), and 3D point cloud spalling

Crack


TASKS

33

- Relate to information from accelerometers, strain sensors, position tags and thermometers

- Relate to data or derived deformation/instability information?- Semantic relationship between sensor reading and damage as

observed outside?- Use UAV to patch gaps/ failed nodes?

TASKS

34

- First damage data available from satellites

- Limitations: vertical view, lack of familiarity of analyst with disaster area

- Use UAV information to validate and calibrate the damage mapsDamage map of Port-au-Prince, Haiti

Source: SERTIT


TASKS

35

- If more than 1 building is equipped with sensors, extra insights can be generated

- piggy bag additional sensors, such as chemical or biological

- Combine with environmental/atmospheric data to assess such threats

FROM RECONASS TO INACHUS

36

4 year project with > 20 partners Provide search & rescue forces with current and increasingly-

detailed intelligence on the situation Nested integration initial satellite assessment Hotspot identification Scenario modelling using synthetic data UAV deployment (area-based monitoring) Dasymetric population modelling Sensors and robots



37

Legislation remains unclear (no-go in various countries)

Reliability is a major issue

FINAL NOTES


38

FINAL NOTES

SUMMARY

39

Good progress in image-based damage mapping (2D to 3D, single- to multi-view, cognitive and semantic analysis)

UAVs promising in term of image quality and sensor control Progress in photogrammetry and OOA has been critical FP7 projects give us space for further developments

Limitations Recognizing and understanding damage (physical vs functional) Legal issues with UAV deployment Maturity of UAVs


40

Questions? For ITC’s OOA work see www.itc.nl/ooa-group Same for full references Papers also on https://www.researchgate.net/profile/Norman_Kerle For RECONASS see www.reconass.eu

Or email: [email protected]


RECONASS team

ITC, University of Twente41

http://www.itc.nl/resumes/kerle https://www.researchgate.net/profile/Norman_Kerle Email: [email protected]

http://www.itc.nl/resumes/gerke https://www.researchgate.net/researc

her/73705446_Markus_Gerke/ Email: [email protected]

Email: [email protected]

Markus Gerke

Norman Kerle

Anand Vetrivel


uav-based damage mapping and the fp7-project …

Documents