gen

Ar

chnol

forme 28

tively. IKONOS can acquire two (stereo) or more nents. In recent years, considerable research efforts havebeen devoted to the efficient utilization of these images.Research has covered, for example, sensor modeling

ISPRS Journal of Photogrammetry & Remoteimages of the same region almost simultaneously1. Introduction

Linear array CCD sensors are widely used to acquirepanchromatic and multispectral images in pushbroommode for photogrammetric and remote sensing applica-tions. IKONOS, successfully launched on 24 September1999, was the first civilian satellite with high-resolutionsensors, acquiring panchromatic and multispectralimages with 1-m and 4-m ground pixel size, respec-

(along-track acquisition) by agile pointing of the sensorthrough rotation of the satellite body, thus reducing theradiometric differences between these images andfacilitating automated measurement processes. Thisfact, together with the high spatial resolution of theimages, the 11-bit quantization and the narrow field ofview, provides a challenge for algorithmic redesign, andthis opens up the possibility of reconsidering andimproving many photogrammetric processing compo-High-resolution satellite images at sub-5-m footprint, such as IKONOS and SPOT5 HRG/HRS images, are becomingincreasingly available to the earth observation community and their respective clients. The related cameras all use linear array CCDtechnology for image sensing. The processing of these kinds of images provides a challenge for algorithmic redesign and this offersthe possibility of reconsidering and improving many photogrammetric processing components. This contribution presents anadvanced matching approach for automatic DSM generation from high-resolution satellite images. It can provide dense, precise andreliable results. The method matches multiple (more than two) images simultaneously and it uses a coarse-to-fine hierarchicalsolution with an effective combination of several image matching algorithms and automatic quality control. The DSMs aregenerated by a combination of matching results of feature points, grid points and edges.

The proposed approach has been applied to IKONOS images over a testfield in Thun, Switzerland with accurate ground controlpoints, a 1600-m height range and variable land cover, but with sub-optimal imaging conditions (snow, long shadows). Theaccuracy tests are based on the comparison between the reference data from an airborne laser scanner and the automaticallyextracted DSMs. The RMS errors for the whole area, excluding trees and bushes, are 23 m, while for bare ground the accuracy isabout 1 m or even better. 2006 International Society for Photogrammetry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V. All rightsreserved.

Keywords: High-resolution satellite imagery; IKONOS; Image matching; DSM generationAbstractMulti-image matching for DSM

Li Zhang ,

Institute of Geodesy and Photogrammetry, Swiss Federal Institute of Te

Received 14 June 2005; received in revisedAvailable onlin Corresponding author. Tel.: +41 1 633 30 63; fax: +41 1 633 11 01.E-mail address: zhangl@geod.baug.ethz.ch (L. Zhang).

0924-2716/$ - see front matter 2006 International Society for PhotogrammAll rights reserved.doi:10.1016/j.isprsjprs.2006.01.001eration from IKONOS imagery

min Gruen

ogy (ETH) Zurich, ETH-Hoenggerberg, CH-8093 Zurich, Switzerland

11 August 2005; accepted 4 January 2006February 2006

Sensing 60 (2006) 195211www.elsevier.com/locate/isprsjprsand image orientation (Baltsavias et al., 2001; Jacobsen,2003; Grodecki and Dial, 2003; Fraser et al., 2002;

etry and Remote Sensing, Inc. (ISPRS). Published by Elsevier B.V.

gramFraser and Hanley, 2003; Poli, 2004; Eisenbeiss et al.,2004), automatic digital terrain model (DTM) anddigital surface model (DSM) generation (Toutin, 2004;Zhang and Gruen, 2004; Poon et al., 2005; Stolle et al.,2005; Krau et al., 2005) and feature extraction (Shan,2003; Hu and Tao, 2003; Di et al., 2003; Baltsavias etal., 2004).

In regard to automatic DSM/DTM generationthrough image matching, which has gained considerableresearch attention over recent years, a wide variety ofapproaches have been developed and automatic DTMgeneration packages are nowadays commercially avail-able for most digital photogrammetric workstations. Theaccuracy performance of these software systems and theproblems encountered are quite similar, and thus far theperformance of commercial image matchers has notlived up to the standards set by manual DSM/DTMmeasurement (Gruen et al., 2000). The main problems inautomated DSM/DTM generation are due to:

(a) absence of sufficient texture,(b) distinct object discontinuities,(c) local object patch not being planar,(d) repetitive objects,(e) occlusions,(f) moving objects including shadows,(g) multi-layered and transparent objects,(h) radiometric artifacts including specular reflec-

tions, and(i) reduction from DSM to DTM.

The key to successful and reliable DSM extraction isthe matching of a dense pattern of features with anappropriate matching strategy, making use of allavailable and explicit knowledge concerning the sensormodel, network structure and image content, epipolargeometry constraints and the generation of a piecewisesmooth surface model. For an appropriate matchingstrategy, a combination of both the area-based matching(ABM) and feature-based matching (FBM) approacheshas to be considered, with matching parameter self-tuning, generation of more redundant matches and acoarse-to-fine hierarchical matching strategy.

In particular, account must be taken of the fact thatIKONOS and other linear array imagery displayparticular characteristics and possibilities for automaticimage matching. Firstly, compared to the traditionalscanned 8-bit images, images from these sensors havebetter radiometric performance (i.e. higher dynamicrange and signal-to-noise ratio). Most of the linear arraysensors have the ability to provide more than 8-bit/pixel

196 L. Zhang, A. Gruen / ISPRS Journal of Photoimages. This results in a major improvement for imagematching in terms of reducing the number of mis-matches for homogeneous areas and especially fordark shadow areas. Secondly, there is the ability toprovide multiple-view terrain coverage in one flightmission or satellite orbit. This enables the multi-imagematching approach, which leads to a reduction in theproblems caused by occlusions, multiple solutions andsurface discontinuities. Consequently, higher measure-ment accuracy is achieved through the intersection ofmore than two image rays. Also, along-track stereoimages from the same orbit, recorded within a very shorttime interval, have a distinct advantage over across-track image pairs because they reduce radiometricdifferences and thus increase the correlation successrate.

In this paper, an advanced image matching approachis presented and the key algorithms used are discussed.We then give a detailed DSM accuracy evaluation usingIKONOS Geo-level images of a testfield in Thun,Switzerland, which displayed a large height range andvariable terrain relief and land cover, and whichemployed a laser DSM as reference data. The imageswere acquired at different times and preprocessed withdifferent calibration parameters. The accuracy tests arebased on the comparison between the reference datafrom the airborne laser scanner and the automaticallyextracted DSMs. The RMS errors for the whole area,excluding trees and other vegetation, are 23 m, whilefor bare ground the accuracy is about 1 m or better. Wedemonstrate through these experiments that our ap-proach leads to accurate and reliable results.

2. Matching approach

Our approach uses a coarse-to-fine hierarchicalsolution with a combination of several image matchingalgorithms and automatic quality control. The newcharacteristics provided by the IKONOS imagingsystems, namely the multiple-view terrain coverageand the high quality image data, are also efficientlyutilized in this approach. The approach was originallydeveloped for multi-image processing of the very high-resolution three-line scanner (TLS) aerial images (Gruenand Zhang, 2003) and it has been extended to processother linear array images as well.

The approach essentially consists of three mutuallyconnected components: an image preprocessing, multi-ple primitive multi-image (MPM) matching and arefined matching procedure. The overall data flow isshown schematically in Fig. 1. The images and the givenor previously estimated orientation elements are used as

metry & Remote Sensing 60 (2006) 195211input. After preprocessing of the original images and

mated

gramproduction of the image pyramids, the matches of three

Fig. 1. Workflow of the auto

L. Zhang, A. Gruen / ISPRS Journal of Photofeature types (feature points, grid points and edges) inthe original resolution images are found progressively,starting from the low-density features in the lowestresolution level of the image pyramid. ATIN-form DSMis reconstructed from the matched features at eachpyramid level by using the constrained Delauneytriangulation method. This TIN is in turn used in thesubsequent pyramid level for derivation of approxima-tions and adaptive computation of selected matchingparameters. Finally and optionally, least-squares match-ing methods are used to achieve more precise results forall matched features and for the identification of somefalse matches.

Details of this matching approach will be provided inthe following paragraphs; further descriptions can befound in Zhang and Gruen (2004) and Zhang (2005).

2.1. Image preprocessing

In order to reduce the effects of the radiometricproblems, such as strong bright and dark regions, and tooptimize the images for subsequent feature extractionand image matching, a new preprocessing method wasdeveloped. This combines an adaptive smoothing filterand the Wallis filter, and it operates on both 8- and morethan 8-bit/pixel images. Firstly, an adaptive smoothingfilter proposed by Saint-Marc et al. (1991) is applied to

DSM generation approach.

197metry & Remote Sensing 60 (2006) 195211reduce the noise level and to sharpen edges and preservefine detail such as corners and line end-points. Next, theWallis filter is applied to enhance and sharpen thealready existing texture patterns. In Fig. 2, an exampleof applying the preprocessing method to an IKONOSimage is shown. Detailed image information is en-hanced and made more suitable for further processing.This is indicated by the road networks and small clustersof houses in the large shadow areas, which are caused bynon-optimal sensor and sun elevation angles (19 in thiscase).

After the image preprocessing, the image pyramid isgenerated starting from the original resolution images.Each pyramid level is generated by multiplying ageneration kernel, which reduces the resolution by afactor of three. The pyramid level number is a pre-defined value that is either input by the user or can bedetermined according to the height range of the imagingarea.

2.2. Multiple primitive multi-image matching

The multiple primitive multi-image (MPM) matchingprocedure is the core of our developed approach foraccurate and robust DSM generation. Results from thisapproach can be used as approximations for the refined

ter inf

grammatching procedure, which employs least-squaresmatching methods. In the MPM approach, the matchingis performed with the aid of multiple images (two ormore) incorporating multiple matching primitives,namely feature points, grid points and edges. Localand global image information is integrated and a coarse-to-fine hierarchical matching strategy is used. The MPMapproach consists mainly of three integrated subsys-tems: the point extraction and matching procedure, theedge extraction and matching procedure, and therelaxation-based relational matching procedure.

2.2.1. Matching through the image pyramidsThe MPM matching approach starts with an initial

matching at the lowest resolution pyramid level, wherethe influence of image noise is reduced and coarseapproximate values are sufficient to stay within the pull-

Fig. 2. IKONOS image pre-processing, road networks and housing clusimage (right).

198 L. Zhang, A. Gruen / ISPRS Journal of Photoin range of the matching procedure. At each pyramidlevel, an intermediate terrain surface is reconstructedfrom the mass points and edges. It is modeled by thetriangular irregular network (TIN) using a 2D con-strained Delauney triangulation method. The constraintsrefer to the appropriate use of edges in the Delauneytriangulation; that is, all edges are constrained to besides of triangles and mass points are introduced in thetriangulation so that the vertices of triangles will alwaysbe points. This TIN-form intermediate surface model isin turn used in the subsequent pyramid levels forproviding the approximations and for adaptivelycomputing the matching parameters. Thus, while thematching procedure is going through the imagepyramids, the surface model computed from the higherlevel of the image pyramid is successively refined at thelower level and, finally, the dense and accurate surfacemodel is reconstructed.An automatic blunder detection procedure isperformed at each pyramid level in order to deletesome mismatches. For blunder detection, we model thesurface locally by a polyhedron. For each point, a localneighborhood is defined, which is approximated by aplane. This neighborhood is not allowed to extendbeyond edges. Suppose p is the standard deviation ofthe plane estimation. Any point whose residualexceeds 2.5p is considered a mismatch. If surfacediscontinuities are not clearly defined as edges, therelated points may be interpreted as mismatches andremoved.

In our approach, the initial DSM for the highest levelis firstly extracted by the geometrically constrainedcross-correlation (GC3) algorithm based on a regiongrowing matching strategy. This method, which isdescribed in Section 2.2.2, uses the already measured

ormation in large shadow areas (left) are enhanced in the pre-processed

metry & Remote Sensing 60 (2006) 195211control points and tie points as seed points and matchesthe points under the assumption that points in a localneighborhood should have similar terrain height values(Otto and Chau, 1989). Then, the initial surface model isfiltered with the median filter in order to eliminate somemismatches. This method is justified because the terrainsurface can be treated as continuous and smooth in thelowest resolution image.

2.2.2. Feature point extraction and matchingThe Lue (1988) operator is used to extract well-

defined feature points. Feature points often correspondto points at locations of terrain height change so thatsome represent significant points for the DSMgeneration. Also, feature points are usually the pointsthat are most suitable for accurate and reliablematching. In our implementation, the chosen referenceimage is divided into small image patches. Only one

feature point will be extracted in each image patch.The density of the feature points can be controlled bythe size of the patches.

In order to determine the correspondences of thegiven points on search images, we have developed anew flexible and robust matching algorithm, thegeometrically constrained cross-correlation or GC3

method, in order to take advantage of the multipleimages. The algorithm is an extension of the standardcross-correlation technique and is based on theconcept of multi-image matching guided from objectspace. This allows reconstruction of 3D objects bymatching all available images simultaneously, withouthaving to match all individual stereo-pairs and mergethe results.

Consider an IKONOS image triplet, as shown inFig. 3. The middle image is chosen as the referenceimage and denoted as I0 and the other two imagesare search images denoted as Ii, i=1, 2. In this case,

values of Z0Z and Z0+Z, respectively, alongthe image ray Cp0. If we back-project the pointsbetween Pmin and Pmax onto the search images, thecorresponding segments of the quasi-epipolar linesfor the point p0 can be easily defined. The correctmatches pi, i=1, 2 in the search images Ii, i=1, 2must lie along the corresponding quasi-epipolar linesegments.

Let I0(p) and Ii(p) be the image intensity values ofthe reference and the ith search image, respectively. Inthe reference image, we define a correlation windowW around the point p0. We assume that an approxi-mate DSM is known either as a horizontal plane orfrom matching results at a higher level of the imagepyramid. If we project this window onto the ap-proximate DSM through the so-called mono-restitutionprocedure, we obtain a piece of surface patch in objectspace. Then, we back-project this surface patch ontothe search images, thus generating the corresponding

199L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 195211we have two stereo pairs: the pairs I0 I1 and I0 I2.For a given point p0 in the reference image, we canestablish the image ray Cp0 (here C denotes theperspective center related to point p0) on which thecorrespondence of p0 in object space should lie,according to the known image orientation para-meters. By intersecting the image ray Cp0 with ahorizontal plane defined by a given approximateheight Z0, we obtain P0 (X0, Y0, Z0) in object space.The approximate height Z0 may have an incrementZ, such that the correct position of P0 in objectspace should lie between Pmin and Pmax, with heightFig. 3. Multiple image matchingimage window in the search images. We have namedthis process the correlation window warping proce-dure. Through this reshaping procedure, a squarecorrelation window in the reference image can becorrelated with windows of different size, shape andorientation in the search images. Therefore, multipleimages with different image scale and orientation canbe matched in a straightforward manner, and distor-tions caused by terrain relief and imaging geometrycan be compensated.

The normalized correlation coefficient (NCC) valuebetween the corresponding correlation windows in thewith the GC3 algorithm.

gramreference image and the ith search image can now bedefined with respect to the height Z for p0 as:

NCCi p0; Z

XsaW

I0s I0 IisiZIiXsaW

I0sI02r X

saW

IisiZIi2r 1

where

I0 1m n

XsaW

I0 s ;

Ii 1m n

XsaW

Ii si Z i 1; 2:

Here, W and s denote the correlation window in thereference image and a pixel in this window, respective-ly;m and n are the dimensions of the correlation windowW and si(Z) is the corresponding point to s in the ithsearch image. As mentioned before, si(Z) can becomputed through the correlation window warpingprocedure. The intensity values for point si(Z) areinterpolated from the ith search image via the bilinearinterpolation method.

As can be seen from Eq. (1), the value of NCCi isdefined with respect to the height value Z, which couldbe any value between Z0Z and Z0+Z. Thus, givena point in the reference image, as well as itsapproximated height Z0 and an increment Z in objectspace, the NCC functions for all individual stereo pairsare defined within a unique framework. We then followthe procedure proposed by Okutomi and Kanade (1993),whereby instead of computing the correct match of pointp0 by evaluating the individual NCC functions betweenthe reference I0 and search image Ii, i=1, 2, we definethe sum of NCC (SNCC) for point p0 with respect to Z as

SNCC p0; Z 12X2i1

NCCi p0; Z : 2

Therefore, by finding the value Z: Z [Z0Z, Z0+Z], which maximizes the SNCC function, we canobtain the corresponding height value for point p0. Here,the height increment Z determines the search distancealong the corresponding quasi-epipolar lines. Throughthe definition of the SNCC function, which simplyaccumulates the NCC functions of cross-correlationfrom all the stereo pairs, the correct match or correctheight in object space for a given point in the referenceimage can be obtained. In general, the matching

200 L. Zhang, A. Gruen / ISPRS Journal of Photocandidates show maxima in the SNCC function andeach peak of the function SNCC corresponds to anobject point with a certain height value. These objectpoints are defined in the GC3 algorithm as the matchingcandidates for the given point. The method can be easilyextended to a more general case, which is suitable forn+1 (n1) images:

SNCC p0; Z 1nXni1

NCCi p0; Z : 3

An example of high-resolution airborne linear arrayimage strips, with 5-cm footprint, is shown in Fig. 4 inorder to highlight the ability of the GC3 algorithm tosolve the multiple solution problem. Fig. 4(b) shows thatit is very difficult to determine the correct match by justevaluating each individual NCC value. However, theSNCC value shows a sharp and clear maximum at thecorrect match, even within a large search distance.

It can be seen that, just by replacing the sensormodels, the GC3 algorithm can also be applied toimages from the current emerging frame-based digitalcamera systems such as the UltraCamD fromVexcel. These systems can provide highly redundantimage data sets (e.g. 80% forward overlap and 60%side overlap) and together with the high radiometricquality of the imagery will certainly provide anincreased amount of automation and thus enhance thequality of the photogrammetric outputs (Thurgood etal., 2004).

For each feature point on the reference image, theresult is that one or several matching candidates arecomputed. The correct match is determined by takingthe following quality measures: (a) The correct matchshould have a clear and sharp maximum SNCC value,so if there is one or more than one candidate and thevalue of the second SNCC peak is less than half or onethird that of the first SNCC peak, the peak that has thelargest SNCC value should represent the final correctmatch. (b) By using the same matching parameters, thepoint can be matched from the search images to thereference image and, if the differences between thisinverse matching and the direct matching is less than 1.5pixels in image space, the candidate is taken to be thecorrect match. Points satisfying these two measures willbe indicated as successful matches and they areconsidered to have only one unique matching candidate,while others are considered to have multiple candidatesand the final match will be determined by using arelational matching procedure, as described in Section2.2.5.

For each point on the reference image, a reliability

metry & Remote Sensing 60 (2006) 195211indicator is assigned according to the SNCC value and

the sharpness of the SNCC function for that matchingcandidate. The reliability indicator essentially servesthree main purposes. Internally, it functions as an initialprobability and is updated iteratively in the relationalmatching procedure. Externally, it gives certain reliabil-

ity measures for the matching results and can be furtheremployed for operator editing of the results. Finally, itserves as a weighting factor for other subsequentprocedures that utilize the matching results, for exampleinterpolating the DSMs.

201L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 1952113Fig. 4. GC matching with six high-resolution airborne linear array imageIndividual NCC functions and the SNCC function determined by a height ins of 5 cm footprint from two strips with changing flight directions.crement of 10.0 m are shown in (b).

gram2.2.3. Grid point extraction and matchingIn the case of image regions with poor texture or even

no texture information, feature points cannot be extractedand the matching with only feature points will conse-quently generate holes in these regions. Thus, in additionto feature points, we use also points, which are definedover an image space grid in the reference image.

Grid points are points determined at given positionsthat are generally uniformly distributed over the wholeimage. Compared to the feature points, the choice ofgrid points is blind and thus many grid points may lie inpoorly textured regions or occluded areas. The searchfor the match of a given grid point has a higherpossibility of yielding an ambiguous match and even nomatching candidate. In the proposed implementation,the matching candidates for the grid points are firstlycomputed by the GC3 algorithm and their final match isdetermined by imposing a local smoothness constraintthrough the relaxation-based relational matching proce-dure that is described in Section 2.2.5.

2.2.4. Edge extraction and matchingIn general,mass points are the predominant features for

the reconstructed DSM and they form the overall shape ofthe terrain surface. However, to reconstruct a DSM fromhigh-resolution images over steep mountainous or urbanareas the problems caused by surface discontinuities,occlusions and significant perspective projection distor-tion must be taken into account. The matching of edges isa possible solution for these problems since line featuresare important for capturing and modelling terrain featuressuch as ridgelines and breaklines. To design an effectiveand reliable edgematching algorithm,we have to considerthe following situations that appear frequently in high-resolution images:

(a) The corresponding edge in the search image maybreak into more than one segment due to imagenoise, occlusions, shadows and deficiencies in theadopted edge extraction algorithms. Some weakedges may appear only in the reference image andnot in the search images.

(b) The corresponding edge in different images mayhave a completely different shape due to theperspective projection, so that one should notassume that the two edges are almost exact copiesof each other.

(c) There may be many similar edges in the searchimages.

In our approach, the well-known Canny operator

202 L. Zhang, A. Gruen / ISPRS Journal of Photo(Canny, 1986) is used to locate the intensity disconti-nuities. Then, edges are linked into free-form edgesthrough a local processing that analyses the character-istics of these pixels in a small neighborhood. Thisapproach is carried out independently for the threeimages. Only edges above a minimum length (e.g. 15pixels) are considered for matching.

The edge matching procedure presented here is basedupon evaluation of the local geometric and photometricattributes of edges for the solution of disambiguities (fordetails, refer to Zhang, 2005). The procedure consistsmainly of the following three steps:

(1) For each edge in the reference image, a prelim-inary list of candidates is built up based uponsimilarity measures and constraints. The epipolargeometry constraint and the approximated DSMderived from the higher-level of the imagepyramid are used to restrict the number of possiblematches. After restricting the necessary searchspace, matching candidates for edge pixels areprovided by using the GC3 algorithm. Then, thepossible candidates are further checked based onsimilarity measures, which are computed from thegeometric and region attributes of the edges.Thus, the number of candidates can be furtherreduced by excluding those failing to meet certainthresholds for their similarity measures.

(2) For each given edge, the remaining candidates arechecked for consistency and compared to eachother, and finally the best pair is chosen. Theconsistency checking is based on the figuralcontinuity constraint and it is solved by aprobability relaxation method. In principle, thismethod imposes the figural continuity constraintand examines the candidates by computing howmuch support they receive from their localneighborhood along the edge. The candidate thatgains the highest support as the correct match foredge pixels is selected. For each edge, the edgepixels that have only one candidate to serve as ananchor in this relaxation method.

(3) The 3D edges are reconstructed in object spaceand expressed by a linear B-spline function.

The proposed edge matching algorithm has beenimplemented and many tests have been carried out withimages over different terrain and land cover types,including mountainous areas, rural and urban regions.The algorithm works quite successfully with differentimage datasets, with the success rate depending upon theimage resolution and terrain type. Usually, a high

metry & Remote Sensing 60 (2006) 195211success rate (more than 75%) can be achieved with high-

resolution satellite images such as the IKONOS andSPOT5 images. As can be seen in Fig. 5, even in steepmountainous areas there are many successfully matchedline features, which are necessary for the modelling ofsuch rough terrain.

2.2.5. Relational matching with probability relaxationtechnique

Although the GC3 algorithm with multiple imagesand self-tuning correlation parameters can reduce thepossibility of mismatches caused by image noise,repetitive texture structures and surface discontinuitiesto a certain extent, it relies on the similarity measure(SNCC) between image intensity patterns in the localneighborhood of a point to determine the matches.Therefore, it is in principle a local matching process.With the local matching process, a point in the referenceimage may match equally well with a number of pointsin the search images. In addition, some points (e.g. gridpoints) located in poorly textured areas have a higher

and base our method on the work of Christmas et al.(1995). The algorithm imposes a piecewise smoothnesssurface constraint, which is bounded by edges. Itexamines the matching candidates by computing howmuch support they receive from their local neighbor-hood and, finally, selects the candidate that gains thehighest support as the correct match. The importantcharacteristics of this method that distinguishes it fromthe local matching are its compatibility coefficientfunction and its smoothness constraint satisfactionscheme. With the smoothness constraint, homogeneousor poorly textured areas can be bridged over byassuming that the terrain surface varies smoothly overthe area. At the same time, the surface discontinuitiescan be preserved because the smoothness constraintscannot cross edges.

In briefly describing the basic algorithm, suppose aiis a point in the reference image and let j (j=1, ,mj)be its mj candidate matches. As a result from the GC

3

algorithm, each matching candidate j relates to an

203L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 195211possibility of yielding multiple candidate matches. Inorder to solve these problems, the final match should beachieved in two stages: a local and a global stage. Thelocal matching finds the possible matching candidates,while the global matching stage is responsible forimposing global consistency among the candidates inorder to disambiguate the multiple candidates and avoidmismatches.

We use a relaxation-based relational image matchingalgorithm to solve the global correspondence problemFig. 5. Edge matching with IKONOS images over rough and steep mountmodelling such terrain.object point with coordinates of (Xj, Yj, Zj). P(i, j) is theprobability of matching j to ai. Similarly, let ah be oneof the points within a local neighborhood of point aiwith k (k=1, ,mk) its corresponding candidatematches. P(h, k) denotes the probability of matchingk to ah. Given two neighboring points and theircorresponding matching candidates (ai, j) and (ah, k),simply denoted as (i, j) and (h, k), respectively, wedefine the following compatible coefficient function C(i,j; h, k) according to Prazdny's (1985) rules. Thisainous areas. The matched edges, shown in white, are necessary for

(c) According to the updating rule (Eq. (5)), the newmatching probability at the nth iteration iscomputed for each point and one of its candidates.

(d) Ideally, the updating process will terminate whenan unambiguous matching result is reached, that iswhen each point ai is matched with one candidatewith probability 1.0, the probabilities for all othercandidate matches for this point being zero. Inpractice, the process is terminated if any one of thefollowing two conditions holds: For each point ai,one of the match probabilities P(i, j) (j=1, ,m)exceeds 1, where 1 (for example, we setthe value of to 0.1), and the pre-defined numberof iterations has been reached.

(e) When the updating process is terminated, thematch which gains the highest probability P(i, j)(j=1,,m) is chosen as the final match for pointai.

2.3. Least-squares approach to refined matching

grammetry & Remote Sensing 60 (2006) 195211function quantifies the compatibility between the match(i, j) and a neighboring match (h, k):

C i; j; h; k Tbdih

exp DZ2

b2d2ih

!4

where

DZ ZkZj:In Eq. (4), Z expresses the difference of the terrain

heights between point ai and its neighboring point ah,and dih is the distance of two points ai and ah in imagespace. As can be seen from the equation, the bigger Zor the larger dih, the smaller the compatibility. Thiscorresponds to imposing a surface smoothness constraintamong the matching candidates. However, the smooth-ness constraint might be violated in areas that containsurface discontinuities. Therefore, we introduce aweighting factor T to control the continuity of the terrainsurface. In the case where the connecting line betweenpoints ai and ah crosses a successfully matched edge, Twill be set to a very small value; otherwise, T equals 1.0.The scaling factor is empirically set to a constant.

The compatible coefficient function C(i, j; h, k) playsan important role in the relaxation matching approach.The global consistency of matching can be achieved byan iterative scheme where the probabilities P(i, j) areupdated by the following rule:

Pn1 i; j Pni; jQni; jX

s1

mj

Pni; sQni; s5

where

Qni; j Y

IhaXai

Xk1

mkPnh; kCi; j; h; k

Here, (ai) is the neighborhood of point ai and n isthe iteration number. The quantity Q(n)(i, j) expressesthe support the match (i, j) receives at the nth iterationstep from the matches (h, k) in its neighborhood (ai).

After the definition of the compatible coefficientfunction C(i, j; h, k) and the updating rule, therelaxation-based relational matching procedure worksas follows:

(a) The iteration scheme can be initialized by assign-ing the corresponding reliability indicator such asthe SNCC values to P(0)(i, j).

(b) For each pair of points within a small localneighborhood and their corresponding matchingcandidates, the compatible coefficient function is

204 L. Zhang, A. Gruen / ISPRS Journal of Photocomputed using Eq. (4).Least-squares matching methods are used in ourapproach to achieve potentially sub-pixel accuracyresults for all matched features and for the identificationof some false matches. For this, a modified multiphotogeometrically constrained matching (MPGC) algorithmis used. The MPGC combines grey level matching withgeometrical constraints derived from multiple image rayintersection conditions and a priori knowledge about theimage orientation (Gruen, 1985; Baltsavias, 1991). ItFig. 6. Sub-image of the nadir-viewing image in dataset T_DEC_O,which shows snow, forest covered areas and long shadows.

(seen

205L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 195211permits a simultaneous determination of pixel andobject coordinates and allows for simultaneous match-ing with any number of images.

In order to process linear array images, the standardMPGC algorithm has been extended by integrating thegeometric constraints derived from linear array sensormodels. The DSM computed from the MPM approach

Fig. 7. Shaded terrain model (5 m grid spacing) of the whole study areaT_DEC_N.provides quite good approximations and certainlyincreases the convergence speed. Furthermore, theinitial values of the shaping parameters of the MPGCalgorithm can be predetermined by using the imagegeometry and the derived DSM. Further details areprovided in Zhang (2005).

In order to match the edges, a simplified version ofleast-squares B-spline snakes (LSB-snakes) representedby parametric linear B-spline functions in object space isimplemented, as proposed by Li (1997). LSB-snakes

Table 1Main characteristics of the three IKONOS image triplets acquired over the s

Dataset Image no. Acquisition date G

T_DEC_O 135251_000 2003-12-25 2135251_100 2003-12-25 2135254_000 2003-12-25 2

T_DEC_N 163003_000 2003-12-25 2163003_100 2003-12-25 2163004_000 2003-12-25 2

T_OCT 157928_000 2003-10-12 2157928_100 2003-10-12 2157928_200 2003-10-12 2can be seen as an extension of the standard MPGCalgorithm. With this method, the parameters of linear B-spline functions of the edges in object space are directlyestimated, together with the matching parameters in theimage space for the multiple images.

2.4. Summary of matching approach

from the north) generated by image matching and the IKONOS triplet-The proposed matching approach is characterized bythe following features:

(1) Multiple image matching: We have developed anew flexible and robust matching algorithm, theGC3 method, in order to take advantage of themultiple images. The algorithm is based on theconcept of multi-image matching guided fromobject space and allows reconstruction of 3D

tudy area

eneration date Sensor azimuth Sensor elevation

004-01-19 180.39 62.95004-01-19 72.21 82.15004-01-19 128.17 82.62005-03-02 180.39 62.95005-03-02 72.21 82.15005-03-02 128.17 82.62004-02-11 10.74 77.85004-02-11 4.69 85.26004-02-11 197.09 71.95

206 L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 195211

objects by matching all available images simulta-neously, without having to match individual

3. Performance evaluation

the m

207L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 195211stereo-pairs separately and then merge the results.(2) Matching with multiple primitives: We have

developed more robust hybrid image matchingalgorithms by taking advantage of both area-based matching and feature-based matchingtechniques and utilizing both local and globalimage information. In particular, an edge match-ing method has been combined with a grid pointmatching method through a probability relaxa-tion-based relational matching process. The use ofedges leads to the preservation of surfacediscontinuities, while grid points bridge areaswith little or no texture.

(3) Self-tuning matching parameters: The adaptivedetermination of the matching parameters resultsin a higher success rate and less mismatches.These parameters include the size of thecorrelation window, the search distance and thecorrelation threshold values. This is done byanalyzing the matching results at the previousimage pyramid level and using them at thecurrent level.

(4) High matching redundancy: With the proposedmatching approach, highly redundant matchingresults, including points and edges, can begenerated and such are suitable for representingvery steep and rough terrain because terrainmicrostructures and surface discontinuities canbe well preserved. Moreover, this high redundan-cy also allows for automatic blunder detection.

(5) Efficient surface modeling: The object surface ismodeled by a TIN generated by a constrainedDelauney triangulation of the matched points andedges. A TIN is suitable for surface modelingbecause it integrates all the original matchingresults, including points and line features, withoutany interpolation. It is adapted to describecomplex terrain types that contain many surfacemicrostructures and discontinuities.

(6) Coarse-to-fine hierarchical strategy: The algo-rithm works in a coarse-to-fine multi-resolutionimage pyramid structure and obtains intermediateDSMs at multiple resolutions. Matches on low-resolution images serve as approximations torestrict the search space and to adaptively com-pute the matching parameters.

Fig. 8. Shaded terrain models of three sub-areas: the lower one shows

one the area around the town of Thun. Left: laser DSM (2 m average point dgrid).The proposed approach for DSM generation has beenverified extensively with several high-resolution satel-lite imagery datasets, such as IKONOS and SPOT5images, over different terrain types, including ruggedmountainous areas, and rural and urban areas. In thefollowing, we will report in detail about the evaluationof DSM extraction from IKONOS images over atestfield in Thun, Switzerland. The test field featuresaccurate ground control points (GCPs), a 1600-m heightrange, variable land cover and sub-optimal imagingconditions (snow, long shadows). The accuracy tests arebased on the comparison between the reference datafrom an airborne laser scanner and the automaticallyextracted DSMs. Other processing and evaluationresults of IKONOS and SPOT5 HRS/HRG can befound in Zhang and Gruen (2004), Poli et al. (2004),Baltsavias et al. (2005) and Poon et al. (2005).

The testfield is an area around the town of Thun andconsists of a steep mountainous region in the south-western part and smooth hilly regions in the middle andnorthern parts. Thun township is located in the lowermiddle part of the study area. The whole area is about1120 km2 and 30% is covered by forests (Fig. 6). Thesite has an elevation range of more than 1600 m (seeshaded terrain model in Fig. 7) and the land cover ishighly variable.

Three IKONOS image triplets (each covering ca.1120 km2) were acquired of the test area, each tripletconsisting of a forward/backward stereo-pair and a nadirimage. The first two image triplets (T_DEC_O andT_DEC_N) of Table 1 are actually the same acquisi-tions, but were processed with different calibrationparameters. In dataset T_DEC_O (old) the three lineararray lines, which together, after optical butting,constitute the virtual sensor line in the focal plane,were not correctly aligned. In dataset T_DEC_N (new),this error (a possible shift, causing parallaxes in flightdirection) was corrected by Space Imaging. About 70%of the area was covered by snow in these two triplets,while the sun elevation was very low at about 19causing long shadows and low contrast. Another triplet(T_OCT) was acquired during autumn with no snow,less shadows and better illumination conditions (Table1). All IKONOS images were Geo, 11-bit, with DRA(Dynamic Range Adjustment) off, and 1-m panchro-matic (PAN) and 4-m multispectral (MS) channels (for

ountain area, the middle one trees and vegetated areas, and the upper

istance); right: results of matching from the T_OCT triplet (5 m point

DSM generation only the PAN images were used). Forall images, the RPCs were provided in the metadata files.

In order to quantitatively evaluate the accuracy of thegenerated DSM, a 2-m irregularly spaced laser DSM,with an accuracy of 0.5 m (1) for bare ground areas and1.5 m for vegetation areas, was used as reference data.The laser DSM, which only covers the southern part ofthe study area, was acquired in the year 2000 from theSwiss Federal Office of Topography, Bern (Swisstopo).

In order to precisely georeference the IKONOSimages, about 50 well-distributed GCPs were collectedwith differential GPS in March 2004. The measurementaccuracy was about 0.25 m. Only 39 of these could bemeasured precisely in the images. With the aid of theGCPs, all IKONOS triplets were orientated with RMSE

more than 70% of the points have differences of less

Table 3DSM accuracy evaluation results (in m) for the triplet T_DEC_N

Terraintype

No. of comparedpoints

RMSE Average RMSE(95)

Average(95)

B1 7,037,578 1.15 0.31 0.73 0.37B2 7,993,875 1.90 0.34 0.93 0.35B3 9,763,257 2.14 0.29 1.19 0.30C 2,794,389 3.38 0.55 2.41 0.55V 8,689,642 8.05 1.58 W1 28,689,642 4.90 0.50 4.23 0.50

Table 4DSM accuracy evaluation results (in m) for the triplet T_OCT

Terraintype

No. of comparedpoints

RMSE Average RMSE(95)

Average(95)

B1 7,037,578 1.41 0.22 0.95 0.21B2 7,993,875 1.77 0.29 1.09 0.29B3 9,763,257 1.75 0.29 1.07 0.29C 2,794,389 2.83 0.25 2.08 0.25V 8,689,642 6.61 1.97 W1 28,689,642 4.25 0.40 2.96 0.39

208 L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 195211values of about half a meter in planimetry and betterthan 1 m in height. For more detailed results oforientations with different GCP distributions, seeEisenbeiss et al. (2004) and Baltsavias et al. (2005).

After all IKONOS image triplets were preprocessedby a method proposed in Baltsavias et al. (2001) andPateraki and Baltsavias (2002), the proposed matchingapproach was applied. The three images of each tripletwere matched simultaneously in order to achieve morereliable results. Some areas such as lakes and rivers weremanually defined as dead areas via a user-friendlyinterface and were excluded from matching. Taking thetriplet T_DEC_N as an example, the matching approachresulted in about 11 million points and 800,000 edges,of which more than 80% were labeled as highly reliable(quality indicator N0.75).

Finally, three 5-m regular-grid DSMs were interpo-lated from the raw matching results. Fig. 7 shows theshaded terrain model for the whole study area and Fig.8 shows a comparison of the laser DSM and the results

Table 2DSM accuracy evaluation results (in m) for the triplet T_DEC_O,with B1bare ground, B2bare ground (including mountainousarea), B3bare ground (including mountainous and shadow areas),Ccity area only, Vtrees/bushes area only, W1whole area andW2whole area without trees/bushes areas

Terraintype

No. of comparedpoints

RMSE Average RMSE(95)

Average(95)

B1 7,037,578 1.27 0.82 0.93 0.89B2 7,993,875 1.84 0.92 1.04 0.92B3 9,763,257 2.11 0.80 1.20 0.80C 2,794,389 3.34 0.30 2.36 0.30V 8,689,642 8.16 1.68 W1 28,689,642 4.93 1.13 4.24 1.14W2 18,022,149 2.74 0.70 1.45 0.69RMSE (95) and average (95) are the RMSE and average (signed)height difference values after excluding the 5% largest differences.of matching for three sub-areas. The resulting DSMreproduced quite well not only the general features ofthe terrain relief, but also small geomorphological andother features visible in the IKONOS images. The DSMshows many topographic details and features like smallvalleys in the mountains, detailed patterns related tostreets and buildings in urban areas, linear featuresrelated to highways and main road networks, sparsetrees, small clusters of houses and forest areas.

A quantitative evaluation of the DSM was conductedby comparing it with the laser DSM using nearly 28million elevation points over the whole study area. Weshow here the results of the raw computations, withoutany a posteriori manual editing procedure applied.Tables 24 give the DSM accuracy evaluation results.We computed the differences as reference DSM minusthe interpolated heights from our generated DSM. Theaccuracy of the raw generated DSM, excluding forestand vegetated areas, is between 1.2 and 3.4 m dependingon the terrain relief and land cover. If 5% of the pointswith the largest errors are deleted, these values drop to0.7 and 2.4 m, respectively. The results can besummarized as follows:

A high accuracy of 1 m or even sub-meter level canbe achieved in bare ground areas. We could not selecttruly bare ground areas; instead, our areas stillcontain many sparse trees and small clusters ofhouses. The analysis shows that in bare ground areas

W2 18,022,149 2.54 0.35 1.41 0.34W2 18,022,149 2.05 0.16 1.32 0.16

than 1 m. Around the areas of sparse trees and smallhouses, the resulting DSM is lower than the laserDSM. This can be expected because usually thesesmall features were either smoothed out by ourmatching method or removed in the automatedblunder detection procedure.

A bias of about 0.31.7 m can be observed in theT_DEC_O results. This was caused mainly byinterior orientation errors, as described below (seealso Baltsavias et al., 2005). A part of the differencesis also due to the 3-year difference between acquisi-tion of the laser DSM and the IKONOS images, andregarding trees due to their different state (in Octobertrees are with leaves, in December without).

In urban and forest areas, the accuracy becomesworse, which is due to the fact that the reference lasermeasurements and the DSM determined in matchingmay refer to partly different objects. Usually, thegenerated DSM is higher than the laser DSM in forestareas (the laser can penetrate the tree canopy) and

narrow low-lying objects (like streets in very denseresidential areas). However, for large buildings, thetwo DSMs coincided quite well.

Other factors that influenced the matching were thelong and strong shadows and occlusions, especially inthe mountain areas, and very low textured snow areas.However, after our particular preprocessing, the longand strong shadows did not hinder DSM generation inbare ground areas because they seemed to generatemore image texture information with less noise.

In steep mountain areas (a slope of more than 70),there are also some blunders with more than 400-mheight difference. They are mainly caused byocclusions. In addition, the smoothness constraintssmoothed out some steep and small features of themountain areas (mainly in shadow areas) becausethere were not enough extracted and matched edges.

In addition, the difference values show some stripe-like patterns for the triplet T_DEC_O in Fig. 9 (left).In these stripes, the average height jump at profiles

EC_Oy). Th

209L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 195211Fig. 9. Accuracy analysis based on height differences for triplets T_Ddifferences (in green and red negative and positive values, respectivel

height differences were computed from the reference DSM minus the interpoalong four profiles (P1 to P4).(left) and T_DEC_N (right). Top: the 2D distribution of the heighte following bar shows the color coding of the difference values. The

lated heights from the IKONOS DSM. Bottom: the height differences

gramP3 and P4, which were in well-textured open and notrugged areas, was 1.5 m. At P1 and P2, thecorresponding value was 1.3 m, corresponding to aparallax error in flight direction of about 0.70.8pixels, while the parallax error in across-flightdirection was only 0.150.2 pixels. A first thoughtwas that these stripes could be due to laser scanningerrors (i.e. neighbouring laser scanner strips mayexhibit offsets and tilts if the strips are not tiedtogether appropriately in a common adjustment).However, the distance between the stripes observedin Fig. 9 (left) was too large compared to the laserscanner swath width. The reason for this parallaxerror lies in a shift between the three individual lineararray chips, which are optically butted in the focalplane to form the complete array. For further details,see Baltsavias et al. (2005). In the dataset T_DEC_N(see Fig. 9, right) this error was corrected by SpaceImaging such that the results improved to the levelanticipated. The influence of interior orientationerrors on the 3D positioning accuracy was alsoobserved in the Y-RMSE values of the orientationprocedure (see also Baltsavias et al., 2005).

The differences between the laser DSM and thematching results of all triplets showed also someother systematic artifacts, as indicated by the arrowsin the right part of Fig. 9. These show up in all threedatasets and can thus be attributed to errors in thelaser DSM. It is very interesting to see thatsophisticated matching methods and satellite imagesfrom 680 km flying height can reveal errors inairborne laser scanning, although this is certainly nota cost-effective method for quality control of DSMsgenerated by laser scanning.

Taking all the above factors into account, it becomesclear that IKONOS has a very high geometricaccuracy potential and with sophisticated matchingalgorithms a height accuracy (1) of around 1.0 mcan be achieved in bare ground areas.

4. Conclusions

In this paper, we have reported upon an advancedmatching approach for automatic DSM generation fromhigh-resolution satellite images. It can provide dense,precise and reliable results. The method uses a coarse-to-fine hierarchical solution with an effective combina-tion of several image matching algorithms and auto-matic quality control. We have developed a matchingstrategy combining local point matching of feature andgrid points, robust edge matching and a relaxation-based

210 L. Zhang, A. Gruen / ISPRS Journal of Photoglobal relational matching. The DSMs are generated bya combination of matching results of feature points, gridpoints and edges. The strategy allows us to bridge overareas with little or no texture and at the same timemaintain the important contribution of edges in objectand image space.

We have presented the results of the processing ofIKONOS triplet images over a testfield in Thun,Switzerland, which had accurate GCPs, 1600-m heightrange, variable land cover and sub-optimal imagingconditions. The results were compared to reference datafrom airborne laser scanning. As evidenced by the visualinspection of the results, our matching method canreproduce not only the general features of the terrainrelief, but also detailed features. The results were partlyinfluenced by the sub-optimal accuracy of the referenceDSM and its temporal difference from the acquisition ofthe IKONOS images. The largest errors usuallyoccurred in areas of shadow, steep mountains and treeand building covered areas, with the best results beingobtained in bare ground areas. The RMS errors for thewhole area, excluding areas of trees and vegetation,were 23 m. If the bias introduced by trees andbuildings is removed, we can expect a height accuracyof one pixel or even better from IKONOS imagery as abest case scenario. One problem of our automatic DSMgeneration approach is the automated detection of smallblunders, which still may exist in the results. Thisconstitutes a relevant topic for further research.

Acknowledgements

We acknowledge the support of both Space Imaging,USA, who made available all the IKONOS imagesreferred to in this article, and the Swiss Federal Office ofTopography, Bern, who provided the laser DSM. Wealso would like to acknowledge our colleagues whowere of great help in the data processing.

References

Baltsavias, E.P., 1991. Multiphoto geometrically constrained match-ing. PhD Dissertation, Report No. 49, Institute of Geodesy andPhotogrammetry, ETH Zurich, Switzerland.

Baltsavias, E.P., Pateraki, M., Zhang, L., 2001. Radiometric andgeometric evaluation of IKONOS geo images and their use for 3Dbuilding modeling. Joint ISPRS Workshop on High ResolutionMapping from Space 2001, Hannover, Germany, 1921 Septem-ber. On CD-ROM.

Baltsavias, E.P., O'Sullivan, L., Zhang, C., 2004. Automated RoadExtraction and Updating using the ATOMI System-PerformanceComparison between Aerial Film, ADS40, IKONOS and Quick-Bird Orthoimagery. International Archives of the Photogrammetry,

metry & Remote Sensing 60 (2006) 195211Remote Sensing and Spatial Information Sciences 35 (Part B4),10531058.

Baltsavias, E., Zhang, L., Eisenbeiss, H., 2005. DSM generation andinterior orientation determination of IKONOS images using atestfield in Switzerland. ISPRS Hannover Workshop 2005 onHigh-Resolution Earth Imaging for Geospatial Information,

Okutomi, M., Kanade, T., 1993. A multiple-baseline stereo. IEEETransactions on Pattern Analysis and Machine Intelligence 15 (4),353363.

Otto, G.P., Chau, T.K.W., 1989. A region-growing algorithm formatching of terrain images. Image and Vision Computing 7 (2),8394.

Pateraki, M., Baltsavias, E., 2002. Adaptive multi-image matchingalgorithm for the airborne digital sensor ADS40. Asian Conferenceon GIS, GPS, Aerial Photography and Remote Sensing Mapasia

211L. Zhang, A. Gruen / ISPRS Journal of Photogrammetry & Remote Sensing 60 (2006) 195211Canny, J.F., 1986. A computational approach to edge detection. IEEETransactions on Pattern Analysis and Machine Intelligence 8 (6),679698.

Christmas, W.J., Kittler, J., Petrou, M., 1995. Structural matching incomputer vision using probabilistic relaxation. IEEE Transactionson Pattern Analysis and Machine Intelligence 17 (8), 749764.

Di, K., Ma, R., Li, R., 2003. Automatic shoreline extraction from high-resolution IKONOS satellite imagery. Proceedings of ASPRS 2003Conference, Anchorage, Alaska, May 59. On CD-ROM.

Eisenbeiss, H., Baltsavias, E.P., Pateraki, M., Zhang, L., 2004.Potential of IKONOS and QUICKBIRD imagery for accurate 3D-point positioning, orthoimage and DSM generation. InternationalArchives of the Photogrammetry, Remote Sensing and SpatialInformation Sciences 35 (Part B3), 522528.

Fraser, C., Hanley, H.B., 2003. Bias compensation in rational functionsfor IKONOS satellite imagery. Photogrammetric Engineering andRemote Sensing 69 (1), 5357.

Fraser, C., Baltsavias, E.P., Gruen, A., 2002. Processing of IKONOSimagery for sub-meter 3D positioning and building extraction.ISPRS Journal of Photogrammetry and Remote Sensing 56 (3),177194.

Grodecki, J., Dial, G., 2003. Block adjustment of high-resolutionsatellite images described by rational polynomials. Photogram-metric Engineering and Remote Sensing 69 (1), 5968.

Gruen, A., 1985. Adaptive least squares correlation: a powerful imagematching technique. South African Journal of Photogrammetry,Remote Sensing and Cartography 14 (3), 175187.

Gruen, A., Zhang, L., 2003. Automatic DTM generation from TLSdata. In: Gruen, Kahman (Eds.), Optical 3-D MeasurementTechniques VI, vol. I. ISBN: 3-906467-43-0, pp. 93105.

Gruen, A., Br, S., Bhrer, Th., 2000. DTMs derived automaticallyfrom DIPSwhere do we stand? Geoinformatics 3 (5), 3639.

Hu, X., Tao, C.V., 2003. Automatic extraction of main-road centerlinesfrom IKONOS and quick-bird imagery using perceptual grouping.Proceedings ASPRS 2003 Conference, Anchorage, Alaska, May59. On CD-ROM.

Jacobsen, K., 2003. Geometric potential of IKONOS- and Quick-Bird-Images. In: Fritsch, D. (Ed.), Photogrammetric Weeks '03,pp. 101110.

Krau, T., Reinartz, P., Lehner, M., Schroeder, M., Stilla, U., 2005.DEM generation from very high resolution stereo satellite data inurban areas using dynamic programming. ISPRS HannoverWorkshop 2005 on High-Resolution Earth Imaging for Geospa-tial Information, Hannover, Germany, 1720 May. On CD-ROM.

Li, H.H., 1997. Semi-automatic road extraction from satellite andaerial images. PhD Dissertation, Report No. 61, Institute ofGeodesy and Photogrammetry, ETH Zurich, Switzerland.

Lue, Y., 1988. Interest Operator and Fast Implementation. Interna-tional Archives of Photogrammetry and Remote Sensing 27 (PartB3), 491500.2002, Bangkok, Thailand, 79 August. On CD-ROM.Poli, D., 2004. Orientation of satellite and airborne imagery from

multi-line pushbroom sensors with a rigorous sensor model.International Archives of the Photogrammetry, Remote Sensingand Spatial Information Sciences 35 (Part B1), 130135.

Poli, D., Zhang, L., Gruen, A., 2004. SPOT-5/HRS stereo imageorientation and automatic DSM generation. International Archivesof the Photogrammetry, Remote Sensing and Spatial InformationSciences 35 (Part B1), 421432.

Poon, J., Fraser, C., Zhang, C., Zhang, L., Gruen, A., 2005. Qualityassessment of digital surface models generated from IKONOSimagery. Photogrammetric Record 20 (110), 162171.

Prazdny, K., 1985. Detection of binocular disparities. BiologicalCybernetics 52 (2), 9399.

Saint-Marc, P., Chen, J.-S., Madioni, G., 1991. Adaptive smoothing: ageneral tool for early vision. IEEE Transactions on PatternAnalysis and Machine Intelligence 13 (6), 514529.

Shan, J., 2003. On the quality of automatic building extraction fromIKONOS imagery. Proc. ASPRS 2003 Conference, Anchorage,Alaska, May 59. On CD-ROM.

Stolle, F., Schultz, H., Woo, D.-M., 2005. High-resolution DEMgeneration using self-consistency. ISPRS Hannover Workshop2005 on High-Resolution Earth Imaging for Geospatial Informa-tion, Hannover, Germany, 1720 May. On CD-ROM.

Thurgood, J.D., Gruber, M., Karner, K., 2004. Multi-ray matching forautomated 3D object modeling. International Archives of thePhotogrammetry, Remote Sensing and Spatial InformationSciences 35 (Part B3), 140144.

Toutin, Th., 2004. Comparison of stereo-extracted DTM from differenthigh-resolution sensors: SPOT-5, EROS-A, IKONOS-II, andQuickBird. IEEE Transactions on Geoscience and Remote Sensing42 (10), 21212129.

Zhang, L., 2005. Automatic Digital surface model (DSM) gene-ration from linear array images. PhD Dissertation, Report No.88, Institute of Geodesy and Photogrammetry, ETH Zurich,Switzerland. URL:http://e-collection.ethbib.ethz.ch/ecol-pool/diss/fulltext/eth16078.pdf.

Zhang, L., Gruen, A., 2004. Automatic DSM generation from lineararray imagery data. International Archives of the Photogrammetry,Remote Sensing and Spatial Information Sciences 35 (Part B3),128133.Hannover, Germany, 1720 May. On CD-ROM.

Multi-image matching for DSM generation from IKONOS imageryIntroductionMatching approachImage preprocessingMultiple primitive multi-image matchingMatching through the image pyramidsFeature point extraction and matchingGrid point extraction and matchingEdge extraction and matchingRelational matching with probability relaxation technique

Least-squares approach to refined matchingSummary of matching approach

Performance evaluationConclusionsAcknowledgementsReferences