workshop on earth observation for urban planning and management, 20 th november 2006, hk 1 zhilin li...

Workshop on Earth Observation for Urban Planning and Management, 20th November 2006,

HK

1

Zhilin Li & Kourosh Khoshelham

Dept of Land Surveying & Geo-Informatics

Hong Kong Polytechnic University

Integration of Multi-source Datafor

Automated Building Extraction


HK

2

Automated building extraction: Problems

Diversity of buildings: Buildings in a variety of shapes and sizes;

Loss of information: Buildings are three-dimensional objects but the third dimension is lost in 2D images.


HK

3

Automated building extraction: Solutions

To minimise information loss• High percentage of overlap between images

• Multi-source images

• Multi-viewpoint images To compensate for the loss

• Additional data sources, e.g. 2-D map data and height data

• Knowledge, e.g. building models


HK

4

Our methodology Solution 1: Building extraction based single image Solution 2: integration of image data and height

data (DSM) Solution 3: integration of 2D ground plans, image

and height data (DSM)

ModelImage DSM Ground plan


HK

5

Image + model Image + height

+ model Image +

height + 2D plan +model

Process


HK

6

Approach 1

From image to edge From edge to line From line to grouped lines From grouped lines to building (matched

with building model selected from library)


HK

7

The process

Locate the building on image (drawing a box);

Select a model (of the building type) from model library (parametric models)

Model matched with buildings on image (represented by lines which are extracted from image)

Parameters for the model is then computed


HK

8

Recursivesplit-and-mergeline fitting process

edge detection (e.g. by Canny operator)

From image to edge to line


HK

9

From image lines to grouped lines:Perceptual grouping of image lines

Perceptual grouping based on two relations that are viewpoint invariant: • proximity• parallelism

Significance of each relation defined using a number of linguistic fuzzy rules

Example of a fuzzy rule for proximity relation:• If DISTANCE is Close and LENGTH is Short then

ProximityDecision is ProbablyConnected. Example of a fuzzy rule for parallelism relation:

• If ANGLE is Small and LENGTH is Short then ParallelismDecision is ProbablyParallel.


HK

10

Construction of the library of building models


HK

11

Model selection and matching

Model selection is interactive. A human operator selects a model from the library.

Once the model is selected, it will be automatically matched with the grouped image lines.

For the matching, corresponding model and image lines must be selected. The model is searched for the model lines that carry the same relations as in the grouped image lines.


HK

12

Matching procedure

The model is parameterized. Initial model parameters are estimated. Model lines are transformed to image space. Distance between the endpoints of every

model line and the corresponding image lines are measured.

Corrections to model parameters are estimated that minimize the sum of squared distances.


HK

13


HK

14

fusion of image and height data for automated building extraction

By making use of height data these two steps can be automated.

In this work a split-and-merge method is developed to fuse image and height data, and automate the model selection.

Localization of the building in the image is also carried out automatically by applying a top hat transform to height data.


HK

15

Integration of image data and height data (DSM)

Image data used to extract building planes using segmentation

Height data used to find building locations

Height data used to assist segmentation Also automated model matching


HK

16

Building edges from image by segmentation

to partition an image into a finite set of homogenous regions that correspond to surfaces in object space


HK

17

Detection of building candidates from height data (DSM)

Opening: flattens sharp peak and height blobs

Such areas can be retrieved by subtracting the opened DSM from the original DSM

The process is called morphological top hat transform


HK

18

Problems with image segmentation

Over-segmentation occurs when the segmentation method detects more than one region on a single surface.

Under-segmentation occurs when the segmentation method fails to sufficiently separate two or more surfaces.


HK

19

A split-and-merge process for refining the initial segmentation by making use of height data

Height points are projected from the DSM to image regions.

An LMS (least median of squares) estimator is used to fit planar faces to the height points in every image region.

Initial segmentation results are refined using the computed planar faces.


HK

20

Split process

Original image Over-grown

region Two planes found

in height data Over-grown

region split


HK

21

Merge process

Every two neighboring regions whose planar faces are

coplanar are merged.


HK

22

An example of the performance of the

split-and-merge process

Roof planes are identified by checking the height of the region-planes

over the DTM.

Roof planes are attributed as flat-roof or slanted-roof according to their

parameters.


HK

23

Automated model selection

Number of roof planes are used as an index to select a class of models from the library.

Attributes of the roof planes are used to further limit the number of selected models.


HK

24

Model

selection is

automated.

Only a 2D roof

model is

matched with

the image.

Library of parametric models


HK

25

2D parameterization of the roof models

Since the parametric forms of the roof planes are known, only a 2D model of the roof is matched with the image lines to compute the boundary of the roof.


HK

26

Matching a 2D roof model with image lines


HK

27

Approach 3: integration of 2D plans with image and height data for automated building extraction

Integration of 2D ground plans leads to the reconstruction of generic polyhedral models that cover a wider range of buildings.

While building detection using top hat transform may include objects other than buildings, using 2D ground plans and height data leads to a more reliable detection.


HK

28

Overview of the approach


HK

29

Overview of the approach Buildings are localized in the image using ground

plans and DEM. Assuming that walls are vertical, their parametric

forms are derived from the ground plans. The parametric forms of roof planes are derived

from the integration of image and height data using the split-and-merge technique.

Model faces are intersected and assembled to form a generic polyhedral model using a plane patch reconstruction method


HK

30

Experiments and results

- Image data consists of an orthoimage in four channels with 0.5m ground resolution.

- Height data consists of DSM and DTM with 1m resolution obtained from a laser scanner.

- Ground plans are manually digitized from the orthoimage.


HK

31

Color (RGB) aerial orthoimage Color infrared (CIR) orthoimage

Image data and the computed NDVI map

Computed NDVI mapGray-scale image (for segmentation)


HK

32

Lidar DSM (first echo) Lidar DTM

and detected building candidates

Detected building candidates

Height data

Lidar DSM (last echo)


HK

33

10 successful

reconstruction

(parametric

models);

3 failure in the

split-and-merge

process;

1 failure in the

model matching

process

Resultsmodels reconstructed using split-and-merge process and model matching

Front view

Rear view


HK

34

Result

Integration of 2D ground plans

manually digitized from

orthoimages in the plane patch

reconstruction method

The use of ground plans for the

localization of buildings leads to

an improvement in the results of

the split-and-merge process

B12

Plane patch reconstruction method


HK

35Rear view

11 successful

reconstruction

(generic polyhedral

models);

2 failure in the split-

and-merge process;

1 failure in the plane

patch reconstruction

method

Front view

Completeness evaluationsplit-and-merge process and plane patch reconstruction


HK

36

Other tests

Accuracy efficiency


HK

37

Conclusions

The combination of

the proposed

approaches in one

system will provide

the user with a great

flexibility to choose

the most suitable

method based on the

available data.


HK

38

End of presentation

workshop on earth observation for urban planning and management, 20 th november 2006, hk 1 zhilin li...

Documents

earth observation

urban planning

integration of image

line slide

image parameters

library slide

d plan model process

single image solution