integration of laser-derived dsms and matched image edges for generating an accurate surface model

10
Integration of laser-derived DSMs and matched image edges for generating an accurate surface model Kerry McIntosh, Amnon Krupnik * Department of Civil Engineering, Technion-Israel Institute of Technology, Haifa 32000, Israel Received 20 August 2000; accepted 21 December 2001 Abstract Airborne laser altimetry is a highly efficient and accurate method of obtaining data for the determination of visible surface topography. With minimal processing, the laser data can provide coordinates of points on the visible surface with high spatial frequency and precision. Although this technology has benefits compared to photogrammetric techniques, there are limiting factors due to the laser data having no structural and textural information. These limitations are significant in low-density laser data and may be overcome by utilizing both laser altimetry and photogrammetrically derived data in the surface determination process. The research described in this paper has been undertaken to accurately determine the visible surface in urban areas using airborne laser scanner data and digital aerial images. Edges detected and matched in aerial images are used to refine the digital surface model (DSM) produced from airborne laser scanner data. The laser data and the edge information are merged to exploit the benefits of each dataset, facilitating the generation of an accurate surface model. This model provides a better representation of surface discontinuities, especially building walls. The paper presents the algorithms developed and shows that the surface accuracy is improved by 49% and 15% for the two tested areas, respectively. D 2002 Elsevier Science B.V. All rights reserved. Keywords: Data integration; Airborne laser scanning; Edge matching; DSM; Surface reconstruction; Surface discontinuity modelling 1. Introduction Digital surface models (DSMs) of urban areas are becoming widely used in an increasing number of applications, such as digital orthophoto production, 3D city modelling and 3D building reconstruction. Laser scanning is recognized as an accurate data source for DSM generation in urban areas (Haala et al., 1997). The spatial resolution of the data is depend- ent on several factors, such as flying height, flying speed and scanner frequency (Lemmens et al., 1997). Laser data provide accurate points with high spatial frequency, however, breaklines are not explicitly present in the data (Ackermann, 1999; Axelsson, 1999; Haala et al., 1997; Kraus and Pfeifer, 1998), and therefore, the position of surface discontinuities can only be estimated or calculated by methods such as segmentation of the range data (Haala et al., 1997; Schenk et al., 1999). To illustrate this point, Fig. 1 presents an elevation image generated from laser data with 2 m grid spacing showing high-rise buildings. The poor definition of the building walls (as noted also by Vosselman, 1999) is apparent in the ‘ragged’ nature of the building edges. The spatial density of the laser data affects the level of inaccuracy that is introduced due to the lack of 0924-2716/02/$ - see front matter D 2002 Elsevier Science B.V. All rights reserved. PII:S0924-2716(02)00042-4 * Corresponding author. Tel.: +972-4-829-2661; fax: +972-4- 823-4757. E-mail address: [email protected] (A. Kr upnik). www.elsevier.com/locate/isprsjprs ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167 – 176

Upload: kerry-mcintosh

Post on 04-Jul-2016

213 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

Integration of laser-derived DSMs and matched image edges for

generating an accurate surface model

Kerry McIntosh, Amnon Krupnik*

Department of Civil Engineering, Technion-Israel Institute of Technology, Haifa 32000, Israel

Received 20 August 2000; accepted 21 December 2001

Abstract

Airborne laser altimetry is a highly efficient and accurate method of obtaining data for the determination of visible surface

topography. With minimal processing, the laser data can provide coordinates of points on the visible surface with high spatial

frequency and precision. Although this technology has benefits compared to photogrammetric techniques, there are limiting

factors due to the laser data having no structural and textural information. These limitations are significant in low-density laser

data and may be overcome by utilizing both laser altimetry and photogrammetrically derived data in the surface determination

process. The research described in this paper has been undertaken to accurately determine the visible surface in urban areas using

airborne laser scanner data and digital aerial images. Edges detected and matched in aerial images are used to refine the digital

surface model (DSM) produced from airborne laser scanner data. The laser data and the edge information are merged to exploit

the benefits of each dataset, facilitating the generation of an accurate surface model. This model provides a better representation

of surface discontinuities, especially building walls. The paper presents the algorithms developed and shows that the surface

accuracy is improved by 49% and 15% for the two tested areas, respectively. D 2002 Elsevier Science B.V. All rights reserved.

Keywords: Data integration; Airborne laser scanning; Edge matching; DSM; Surface reconstruction; Surface discontinuity modelling

1. Introduction

Digital surface models (DSMs) of urban areas are

becoming widely used in an increasing number of

applications, such as digital orthophoto production,

3D city modelling and 3D building reconstruction.

Laser scanning is recognized as an accurate data

source for DSM generation in urban areas (Haala et

al., 1997). The spatial resolution of the data is depend-

ent on several factors, such as flying height, flying

speed and scanner frequency (Lemmens et al., 1997).

Laser data provide accurate points with high spatial

frequency, however, breaklines are not explicitly

present in the data (Ackermann, 1999; Axelsson,

1999; Haala et al., 1997; Kraus and Pfeifer, 1998),

and therefore, the position of surface discontinuities

can only be estimated or calculated by methods such

as segmentation of the range data (Haala et al., 1997;

Schenk et al., 1999). To illustrate this point, Fig. 1

presents an elevation image generated from laser data

with 2 m grid spacing showing high-rise buildings.

The poor definition of the building walls (as noted

also by Vosselman, 1999) is apparent in the ‘ragged’

nature of the building edges.

The spatial density of the laser data affects the level

of inaccuracy that is introduced due to the lack of

0924-2716/02/$ - see front matter D 2002 Elsevier Science B.V. All rights reserved.

PII: S0924 -2716 (02 )00042 -4

* Corresponding author. Tel.: +972-4-829-2661; fax: +972-4-

823-4757.

E-mail address: [email protected] (A. Krupnik).

www.elsevier.com/locate/isprsjprs

ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176

Page 2: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

terrain information. A high spatial density of data may

well provide enough information to define the surface

to the required accuracy, however, if the spatial den-

sity is low, extra information will be required to

accurately define the surface. The method described

in this research was aimed especially at such low-

density laser data.

Digital photogrammetric methods of automatic sur-

face reconstruction have become widely used due to

the efficiency and cost effectiveness of the production

process, especially in open and flat areas, and when

using small and medium scale imagery (Krzystek and

Ackermann, 1995). However, most software packages

perform poorly in areas with abrupt height differences,

such as urban regions (Haala, 1999). The decline in the

performance of softcopy photogrammetric packages

can be caused by failures of the image matching

process (Axelsson, 1998). Such failures may be due

to factors like lack of texture in the images (Haala,

1994), poor image quality, shadows in the images,

occlusions, surface discontinuities (Haala et al., 1997)

and foreshortening. These problems occur especially

when using digital photogrammetry in urban areas and

result in inaccuracies in the DSM, which can be seen in

the smoothed discontinuities of the reconstructed sur-

face, as well many blunders close to them (Baltsavias,

1999; Haala, 1999; Toth and Grejner-Brzezinska,

1999).

One of the benefits of photogrammetry is that the

imagery contains more information than just the

position of pixels in the images. Grey value changes

in the images allow the identification and classifica-

tion of objects, such as buildings or vegetation, and

can be used to detect edges in the images, which often

indicate the location of surface discontinuities (Balt-

savias, 1999; Fradkin and Ethrog, 1997; Haala and

Anders, 1997). Furthermore, utilizing imagery in con-

junction with surface reconstruction enables the pro-

duction of orthophotos and 3D views of the terrain.

Multispectral imagery can also help in the classifica-

tion of laser data.

Investigations and observations comparing DSMs

produced from laser data and those derived from digital

photogrammetric methods have been made in several

research studies. In areas of the imagery lacking texture

or contrast, the image matching might not provide

accurate results whereas the accuracy of the laser is

not affected (Baltsavias, 1999; Kraus and Pfeifer,

1998). However, photogrammetric data have a higher

planimetric accuracy than laser data (Baltsavias, 1999).

The complementary nature of the two data sources

has been widely recognized and their combination has

been suggested by researchers since several years

(Fritsch and Kilian, 1994; Haala, 1994). This sugges-

tion has been reiterated recently (Ackermann, 1999;

Axelsson, 1999; Baltsavias, 1999; Csatho et al., 1999;

Fritsch, 1999; Haala, 1999; Haala and Anders, 1997;

Toth and Grejner-Brzezinska, 1999; Vosselman, 1999).

The approach of using imagery to provide edge infor-

mation has been highlighted by several researchers

(Ackermann, 1999; Axelsson, 1998; Csatho et al.,

1999; Haala and Anders, 1997), and is the main

concept of the approach undertaken in this research.

It is proposed to merge the accurate edge informa-

tion from the imagery with the laser scanner data,

providing a more accurate dataset than either of the

two separate data sources. A preprocessing step is

required to generate surfaces from each data source

and accurately register the surfaces to the same coor-

dinate system. The main component of the proposed

approach combines information from each data source

to obtain the optimal topographic surface. The photo-

grammetric data are utilized by extracting edges from

the stereo imagery, and processing the edges using

feature-based matching techniques. Then, a 3D recon-

struction of the matched edges is performed. These

edges are used to obtain accurate locations of the

surface discontinuities in the urban scene. The laser

data is classified in ground and building points with

aim to detect 3D lines which correspond to building

rooflines. These lines are used as breaklines when

Fig. 1. Laser height data shown as grey level image.

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176168

Page 3: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

merged with the laser data. A new surface in trian-

gulated irregular network (TIN) representation is

generated using the merged data and constrained trian-

gulation.

Testing of the approach has been undertaken using

an urban site covering Ocean City, MD, USA. Laser

data and aerial images, acquired on the same day by

NASA and NGS, respectively, are used. Experiments

have been performed to test and refine the algorithm.

This paper presents the datasets, the preprocessing

step, the data fusion approach and detailed test results.

2. The data fusion approach

2.1. Data preprocessing: surface registration

The surface registration is undertaken to determine

the transformation parameters between the laser and the

photogrammetrically derived surfaces. Theoretically,

the two datasets should refer to the same coordinate

system. However, the systematic errors inherent in the

laser data may introduce a misalignment between the

two surfaces, which must be eliminated before data

fusion may be performed accurately (Kraus and Pfeifer,

1998).

Ideally, a seven parameter conformal transforma-

tion should have been used for converting the laser

data to the coordinate system of the photogrammetric

model. However, this requires surfaces with different

surface normal directions in order to avoid singularity.

Here, only three shift parameters were used and they

were sufficient for accurate co-registration of the two

datasets. The determined transformation parameters

must be as accurate as possible to ensure that no

unnecessary degradation occurs in the accuracy of the

surface generated from merging the datasets.

The surface registration algorithm was developed

specifically for matching visible surface models of

urban areas produced using different acquisition meth-

ods, and in particular, surfaces from airborne laser

scanner data and surfaces automatically generated

using photogrammetric data. Both the laser surface

and the photogrammetrically derived surface are rep-

resented by irregularly distributed elevation points.

Defining the laser elevation points as the source surface

that should be transformed into the other (target) sur-

face, an observation equation can be written for each

surface point. The actual observation is the elevation

difference between the two surfaces at the planimetric

location of each laser point. Since the elevation of the

target surface at this location is generally not known, a

mathematical surface is fitted to the photogrammetri-

cally derived points in the vicinity of the laser data

point. This surface provides both the elevation of the

target surface at the required location, and the surface

gradients, which are necessary for the linearized obser-

vation equations. A detailed description of the surface

matching algorithm is provided in Schenk et al. (2000).

Another approach for matching such surfaces is given

by Habib and Schenk (1999).

2.2. Edge extraction

The edge pixels are detected in each image of the

stereo pair. At present, the optimal zero-crossing

operator (Sarkar and Boyer, 1991) is used for the

detection of edge pixels. The edge pixels are analyzed

to find connected edge chains, and using these chains,

straight line-segments (termed edges in the following)

are determined. Straight edges are utilized as building

rooflines as they are more accurately located than

point features when using feature matching techniques

(Fradkin and Ethrog, 1997).

The automatic extraction of edges has certain

limitations that must be addressed. Using automatic

extraction methods, not all surface discontinuities are

detected due to factors such as lack of contrast in the

imagery. There may be a number of incomplete or

incorrect edges, and edges that do not represent sur-

face discontinuities (breaklines) may be detected.

These particular breaklines may refer to visual edges

in the images that are purely changes in gray value,

such as road marks. Including these edges as break-

lines in the DSM is not expected to be detrimental to

the data fusion process.

2.3. Edge matching

The edge matching approach currently being used

for this research is based on that of Schmid and

Zisserman (1997), and employs the fundamental

matrix (F matrix) to guide the matching process. Edges

arematched by averaging the correlation coefficients of

conjugate points along them. Only edges longer than

10 pixels were used, as shorter edges are more likely to

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176 169

Page 4: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

represent noise rather than actual features. Given an

edge in one image, the epipolar lines that correspond to

its endpoints in the other image are determined by the

fundamental matrix. Knowing the elevation range of

features on the ground (which translates to a disparity

range), a parallelogram is drawn (Fig. 2, top right),

which constitutes the search space for edge segments in

the second image (Medioni and Nevatia, 1985). Can-

didates for matching will be portions of edges that fall

into this search space, as shown in Fig. 2 (top right),

with the limitation that the edges in the two images

should have a similar direction (e.g., edge C in Fig. 2

top right in not accepted as a candidate). This constraint

is justified when a normal aerial model is used, i.e., the

images are nearly vertical and their x axes are almost

parallel. More than one edge in the second image can

correspond to an edge in the first image, providing that

they do not overlap in the epipolar line direction. Thus,

the pair of edges marked ‘‘A’’ in the right image in Fig.

2 (middle right) may correspond to two portions of the

edge in the left image, while the two edges marked ‘‘B’’

are not accepted as possible match candidates because

they overlap. Selecting candidates for matching is

performed in both directions, i.e., selecting an edge in

one image and defining a search space in the second

image, and vice versa.

Cross-correlation coefficients are calculated be-

tween each pixel of the edge in the first image and its

corresponding pixel on the candidate edge in the other

image. The nearest pixel to the intersection between the

epipolar line defined by each edge pixel of the first

image and the candidate edge is selected (Fig. 2, bottom

right). The correlation window size was set to 15� 15

pixels, as suggested by Schmid and Zisserman (1997).

The measure for selecting the best match is the max-

imum of the average correlation value along the edge.

The calculation of the cross-correlation takes place

only on the common overlapping part of the two edges,

i.e., the portion of the candidate edge in the second

image that falls within the search area.

It should be noted that the approach taken here for

edge matching is not optimal. Thus, for example,

edges along the epipolar lines cannot be matched.

Other edge matching techniques may be used at this

point, and further investigation will be undertaken to

determine the optimal technique.

2.4. 3D segment reconstruction

The 3D coordinates of the edges are calculated using

the interior and exterior orientation parameters of the

aerial imagery. The end points of the overlapping

section of the edges are required in the image coordi-

nate system for each image. As the edge information is

in pixel coordinates, an affine transformation is used to

convert the information appropriately. The interior

orientation of the imagery defines this affine trans-

formation, and the rotation matrix and camera position

are given by the known exterior orientation. The 3D

coordinates of each endpoint of the edges are calculated

using this information and least-squares forward inter-

section (see Kraus, 1993, p. 15 for equations). In order

to avoid blunders, the 3D coordinates of the segment

end points are compared to the laser data elevation,

interpolated at the same planimetric location. Only

Fig. 2. Edge matching procedure: definition of the search space—

relevant candidates are marked with dots at the ends (top); matching

an edge in one image to two edges in the other image (middle);

definition of correlation windows (bottom).

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176170

Page 5: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

segments with end points that are within a 1-m eleva-

tion range from the interpolated elevation are consid-

ered for further processing.

One should note that the reconstruction procedure

generates breaklines, as well as 3D lines which are the

result of intensity discontinuity only. Although the

latter are not necessary for surface reconstruction,

including them in the process hardly makes any

difference. Therefore, no attempt was made to differ-

entiate between them and real breaklines.

2.5. Surface discontinuities

In an urban area, most surface discontinuities are

associated with man-made objects in particular build-

ings. These discontinuities are used as constraints

during the triangulation process, as described in Sec-

tion 2.6. The surface is well described if both the roof

outlines and the building outlines at ground elevation

are available. Most of the former are included in the set

of 3D segments obtained during the reconstruction

step. However, using imagery cannot provide a com-

plete set of the breaklines at ground level.

In order to obtain as many breaklines as possible, the

following approach was used. In the first step, a

classification of the laser points was performed. An

ideal classification would differentiate points on top of

buildings from points on top of vegetation and points

on the ground. Here, only a rudimentary classification

was utilized, which separated building points (defined

as higher than h + 0.75r, where h is the mean and r is

the standard deviation of the laser point elevations) and

ground points. Themean and standard deviation should

be calculated for a sufficiently small neighbourhood of

each point (here, the whole test areas were used). The

threshold was obtained empirically, and is believed to

be suitable for urban areas with flat terrain, and one to

two storey buildings. Using the classification results,

he laser points in the vicinity of each 3D segment

(a buffer that contains 5–10 points at each side of the

segment) were analysed. A 3D segment was charac-

terized as a breakline if most laser points on one side

were classified as ground points, while most points on

the other side were classified as building points.

Further analysis characterized breaklines as rooftop

or ground breaklines. This was done by comparing the

average elevation of the 3D breakline with the neigh-

bouring ground and rooftop laser points.

In order to obtain additional ground breaklines, each

rooftop breakline was paired with a ground breakline

wherever available. This was done by searching for

ground breaklines which were parallel and within 0.5m

distance (in planimetry) from a rooftop breakline. For

rooftop breaklines that did not have a corresponding

ground breakline, an additional ground breakline was

added as follows. The rooftop breakline was projected

vertically onto the ground surface to produce a new,

ground breakline. The elevation values along the pro-

jected breakline were determined by interpolating a

bilinear surface, using the ground laser points. The

result is a representation of the vertical wall surface by

two 3D lines, one at roof height and one at ground level.

Since the implementation used for triangulation (as

well as most similar implementations) does not allow

multiple elevation values at the same planimetric

location, the bottom line was shifted slightly outwards

Fig. 3. Images of the test areas A (top) and B (bottom).

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176 171

Page 6: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

parallel to itself. As this shift is very small (it only needs

to introduce a numerical difference between the two

planimetric locations), the slope introduced in the

resulting surface is insignificant.

2.6. TIN generation

The input for the TIN generation process com-

prises of the laser data points and the breaklines. Laser

points that are located close to the breaklines are

eliminated in order to avoid inaccurate responses from

the wall itself, or when a laser beam falls exactly on

the edge of a roof. The breaklines (represented as a set

of straight 3D segments) are introduced as triangle

legs, and constrain the triangulation process in a way

that no new triangle leg can cross them. The inclusion

of the elevation information at the TIN nodes allows

the production of the digital surface model using the

combined laser data and photogrammetrically derived

3D segments.

3. Tests and results

Testing was carried out using a dataset over Ocean

City, MD, USA, which includes digital stereo imagery

Fig. 4. Test area A—extracted and matched line segments.

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176172

Page 7: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

and laser data. The B/W images were taken by a Wild

RC 10 wide-angle camera, at a scale of 1:3500 and a

forward overlap of 75%, and scanned by a photo-

grammetric scanner at 28 Am. The laser data was pro-

duced by NASA’s ATM conical scanner, with an

average ground spacing of 2 m, and a footprint of

1.5 m. Only one response of the laser pulse was re-

corded (see Csatho et al., 1998 for more details). The

dataset covers different types of areas, including

residential areas, flat terrain, beach front, dunes,

canals and high-rise buildings. Two sections of the

residential area have been used for the testing of the

algorithm. Fig. 3 shows images of these sections.

3.1. Data processing

The data were processed with the methods ex-

plained in Section 2. The transformation parameters

for the co-registration of the two datasets were deter-

mined as follows. Rooflines were measured manually

at a digital stereoplotter to produce planes. These

planes were used with the laser points in the registration

Fig. 5. Test area A: surface derived from laser points (top); surface

derived from laser data and manually measured breaklines (middle);

surface derived from laser data and automatically generated break-

lines and added ground breaklines (bottom).

Fig. 6. Test area B: surface derived from laser points (top); surface

derived from laser data and manually measured breaklines (middle);

surface derived from laser data and automatically generated break-

lines with added ground breaklines (bottom).

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176 173

Page 8: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

process described in Section 2.1 for calculating the

shifts in X, Y and Z directions. Once these were cal-

culated, the laser surface was transformed into the

coordinate system of the roof planes. Measuring the

roofs manually allowed a good registration, which is

independent of the 3D segment generation technique.

3D lines were generated for both test areas using the

procedure described in Sections 2.2–2.5. Fig. 4 (top)

shows the edges extracted from the left and right ima-

ges of test area A. The matched 3D segments, projected

on the left image are shown in Fig. 4 (bottom).

In order to evaluate the quality of the edge matching

procedure, breaklines were also measured manually. In

addition, ‘‘ground truth’’ measurements were made

over the two test areas to determine quantitatively the

improvement in the surface representation when using

the merged data. Dense grids (0.5 m spacing) were

measured over the entire surface (ground and roofs)

covered by the two areas. The breaklines and the grids

Table 1

Comparison of ground truth data to surfaces from laser data and

merged data

Ground truth Compared to RMS (m)

Area A Laser 1.14

Laser + reconstructed breaklines 0.59

Laser +manually measured breaklines 0.52

Area B Laser 0.72

Laser + reconstructed breaklines 0.61

Laser +manually measured breaklines 0.53

Fig. 7. Histogram of height differences (in m) between ground truth and: laser data (top), laser data and reconstructed breaklines (bottom).

Results shown for test area A (left) and test area B (right).

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176174

Page 9: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

were measured at a softcopy photogrammetric work-

station, and their accuracy is estimated to be 5 cm in

planimetry and 10 cm in height.

3.2. Results

Three types of surfaces were generated for the test

areas. The first surface was based only on the laser

points, and serves for comparing the results of the

approach presented here to a standard laser-based

DSM. The second surface was generated using the

laser data and the manually measured breaklines. This

is the representation of the visible surface that the

research attempts to achieve automatically. The third

surface is generated using the laser data and the

automatically extracted breaklines.

Fig. 5 shows the three surfaces for test area A. A

visual comparison of the top and bottom surfaces

shows the significant improvement achieved by includ-

ing the automatically derived breaklines with the laser

data when determining the visible surface model over

an urban area. The middle surface, which utilized the

manually measured breaklines, is only slightly better

than the bottom surface. The results of test area B are

shown in Fig. 6. Here, again, the improvement in the

representation of the surface is apparent when compar-

ing the top and bottom surfaces.

A quantitative analysis of the results was per-

formed by comparing the surfaces to the manually

measured grids points. These grid points were used as

check points for the surface created from the laser

data, the surface created from the laser data that were

merged with the automatically reconstructed break-

lines, and the surface created from the laser data

merged with the manually measured breaklines. The

RMS of the height differences for both areas are given

in Table 1.

The comparison of the datasets described in Table 1

has also been analyzed and depicted as histograms of

height differences. Fig. 7 shows the comparison of the

ground truth surface for test areas A and B to the laser

points (top) and the surface generated using the laser

points and reconstructed breaklines (bottom). The

reason for the major improvement when using break-

lines, observed when comparing the two left histo-

grams is rooted in the laser data themselves. The

number of the laser points that fall on the roof of the

right (dark) building in test area A is very small,

possibly due to poor reflection of the laser beam at

dark surfaces. Therefore, when reconstructing the sur-

face only with laser points, the outline of this building

is considerably smoothed, as can be clearly seen in Fig.

5 (top). Including the breaklines significantly improves

the building shape.

4. Conclusions

This paper describes the development of an algo-

rithm for the integration of laser height data and

photogrammetrically derived 3D segments. Tests of

the algorithm have been carried out using data over

Ocean City, MD, USA. Obviously, obtaining an

accurate and complete surface model automatically

for this urban area by utilizing images with only

pairwise overlap would be impossible with the current

technology. The tests have shown that the visible

surface is represented more accurately when using

the combined data than when using the laser data

alone. The improvement is noticeable both by com-

paring the reconstructed surfaces visually, and by a

quantitative comparison using manual ground truth

measurements. In the two test areas, the use of the

combined data led to an accuracy improvement of

49% (55 cm) and 15% (11 cm), respectively.

Further research is required with respect to several

aspects of the suggested algorithm. The edge match-

ing and the laser data classification used in the current

implementation are not optimal. It is expected that

improvement of these parts, especially a proper sep-

aration of buildings from vegetation and the ground

surface, will improve the selection of relevant 3D

segments for constraining the surface. Additional

investigation is also required for adapting the algo-

rithm to non-flat terrain. It should be also noted that

the laser data used here is rather sparse. Therefore, it

would be also interesting to check the effect of adding

breaklines to denser laser data.

Acknowledgements

This research project is supported by the United

States–Israel Binational Science Foundation, Grant

No. 97-00433, and by the Fund for the Promotion of

Research at the Technion.

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176 175

Page 10: Integration of laser-derived DSMs and matched image edges for generating an accurate surface model

References

Ackermann, F., 1999. Airborne laser scanning—present status and

future expectations. ISPRS Journal of Photogrammetry and Re-

mote Sensing 54 (1), 64–67.

Axelsson, P., 1998. Integrated sensors for platform orientation and

topographic data acquisition. Proceedings of the Symposium on

Digital Photogrammetry, Istanbul, Turkey, 1–11.

Axelsson, P., 1999. Processing of laser scanner data—algorithms

and applications. ISPRS Journal of Photogrammetry and Re-

mote Sensing 54 (1), 138–147.

Baltsavias, E., 1999. A comparison between photogrammetry and

laser scanning. ISPRS Journal of Photogrammetry and Remote

Sensing 54 (1), 83–94.

Csatho, B., Krabill, W., Lucas, J., Schenk, T., 1998. A multisensor

data set of an urban and coastal scene. International Archives of

Photogrammetry and Remote Sensing 32 (3-2), 588–592.

Csatho, B., Schenk, T., Lee, D.-C., Filin, S., 1999. Inclusion of

multispectral data into object recognition. International Archives

of Photogrammetry and Remote Sensing 32 (7-4-3W6), 53–60.

Fradkin, M., Ethrog, U., 1997. Feature matching for automatic

generation of distortionless digital orthophoto. In: McKeown Jr.,

D.M., McGlone, J.C., Jamet, O. (Eds.), Integrating Photogram-

metric Techniques with Scene Analysis and Machine Vision III.

Proceedings SPIE, vol. 3072, pp. 153–164.

Fritsch, D., 1999. Virtual cities and landscape models—what has

photogrammetry to offer? In: Fritsch, D., Spiller, R. (Eds.),

Photogrammetric Week ’99. Wichmann, Heidelberg, pp. 3–14.

Fritsch, D., Kilian, J., 1994. Filtering and calibration of laser scan-

ner measurements. International Archives of Photogrammetry

and Remote Sensing 30 (3), 227–234.

Haala, N., 1994. Detection of buildings by fusion of range and

image data. International Archives of Photogrammetry and Re-

mote Sensing 30 (3), 341–346.

Haala, N., 1999. Combining multiple data sources for urban data

acquisition. In: Fritsch, D., Spiller, R. (Eds.), Photogrammetric

Week ’99. Wichmann, Heidelberg, pp. 329–339.

Haala, N., Anders, K.-H., 1997. Acquisition of 3D urban models by

analysis of aerial images, digital surface models and existing 2D

building information. In: McKeown Jr., D.M., McGlone, J.C.,

Jamet, O. (Eds.), Integrating Photogrammetric Techniques with

Scene Analysis and Machine Vision III. Proceedings SPIE, vol.

3072, pp. 212–222.

Haala, N., Brenner, C., Anders, K.-H., 1997. Generation of 3D city

models from digital surface models and 2D GIS. International

Archives of Photogrammetry and Remote Sensing 32 (3-4W2),

68–75.

Habib, A., Schenk, T., 1999. A new approach for matching surfaces

from laser scanners and optical sensors. International Archives

of Photogrammetry and Remote Sensing 32 (3W14), 55–61.

Kraus, K., 1993. Photogrammetry. vol. 1, 4th edn., Dummler, Bonn.

Kraus, K., Pfeifer, N., 1998. Determination of terrain models in

wooded areas with airborne laser scanner data. ISPRS Journal

of Photogrammetry and Remote Sensing 53 (4), 193–203.

Krzystek, P., Ackermann, F., 1995. New investigations into the

practical performance of automatic DEM generation. Proc.

ACSM/ASPRS Annual Convention, Charlotte, NC, vol. 2, pp.

488–500.

Lemmens, M., Deijkers, H., Looman, P., 1997. Building detection

by fusing airborne laser-altimeter DEMs and 2D digital maps.

International Archives of Photogrammetry and Remote Sensing

32 (3-4W2), 42–49.

Medioni, G., Nevatia, R., 1985. Segment-based stereo matching.

Computer Vision, Graphics and Image Processing 31 (1), 2–18.

Sarkar, S., Boyer, K., 1991. Optimal infinite impulse response zero

crossing based edge detectors. Computer Vision, Graphics and

Image Processing 54 (2), 224–243.

Schenk, T., Csatho, B., Lee, D.-C., 1999. Quality control issues of

airborne laser ranging data and accuracy study in an urban area.

International Archives of Photogrammetry and Remote Sensing

32 (3W14), 101–108.

Schenk, T., Krupnik, A., Postolov, Y., 2000. Comparative study of

surface matching algorithms. International Archives of Photo-

grammetry and Remote Sensing 33 (B4), 518–524.

Schmid, C., Zisserman, A., 1997. Automatic line matching across

views. Proceedings Computer Vision and Pattern Recognition

(CVPR’97), 666–671.

Toth, C., Grejner-Brzezinska, D., 1999. Improved DEM extraction

techniques—combining LIDAR data with direct digital GPS/

INS orientated imagery. International Archives of Photogram-

metry and Remote Sensing 32 (2-5-3/W10), 6A-4, 1–7.

Vosselman, G., 1999. Building reconstruction using planar faces in

very high density height data. International Archives of Photo-

grammetry and Remote Sensing 32 (3-2W5), 87–92.

K. McIntosh, A. Krupnik / ISPRS Journal of Photogrammetry & Remote Sensing 56 (2002) 167–176176