A two-dimensional production system for grouping Persistent Scatterers inurban High-Resolution SAR scenes

Lukas Schack, Alexander Schunert, Uwe SoergelInstitute of Photogrammetry and GeoInformation, Leibniz Universitat Hannover

schack |schunert |soergel@ipi.uni-hannover.de

Abstract

Modern spaceborne SAR sensors like TerraSAR-Xoffer ground resolutions of about one meter in rangeand azimuth direction which allows for the discrimina-tion between different facade elements. Those objectsoften feature a trihedral structure, which leads to astrong radar response due to triple-bounce reflection.The resulting radar signal is frequently observed to betemporally stable and usually shows a large amplitude(Persistent Scatterer, PS). Our aim is to aggregate singlePS to lattices using Gestalt theory. This is a very import-ant step in the process of unveiling the physical natureof PS since it simplifies the fusion with supplementa-ry data like oblique view aerial images. We use a twostage 2D production system which exploits the know-ledge abut mapping of 3D objects into the SAR imaginggeometry. In the first step, the production system groupsthose PS, which are vertically aligned in the real world,to rows. In a second step, those groups are merged toregular lattices with the second orientation correspon-ding to the horizontal alignment of facade elements. Theresults show the benefit of aggregating points to latticesby the possible distinction of facade orientations.

1. Introduction

Persistent Scatterer Interferometry (PSI) offers thepossibility to monitor surface deformation on a sparsegrid of temporally stable points, the so called PersistentScatterers (PS) [5]. Using spaceborne SAR data of thehighest resolution (about one meter on the ground), avery high density of such points can be observed. Parti-cularly facades accommodate a plethora of PS, which isdue to typical facade objects like windows or balconies.Those objects usually form rectangular structures oftenleading to a strong and stable reflection mechanism. Si-gnal reflection from trihedral structures of only 8 cmside length may already cause PS [2]. Although PSI is a

quite established technique, the question about the phy-sical nature of the PS (i.e. what is the actual structurecausing the PS) remains still unanswered in the majo-rity of cases. This is quite problematic since a localiza-tion of the estimated deformation is vital in monitoringtasks. In order to investigate this, a fusion of the PS setswith oblique view aerial images is useful. However, it isnot possible to align both datasets with an accuracy nee-ded to unambiguously determine the correspondence ofa single PS to an entity in the optical image (for instan-ce a point of interest). A solution to this problem maybe the assignment of PS groups to sets of entities in theoptical image. In this work a method to find regular pat-terns in PS sets is proposed. This constitutes one part ofthe fusion methodology described above.

Manmade objects often consist of regularly arrangedstructures. In this paper, we concentrate on the regulari-ty of windows on facades which we represent as latticeswhich two main directions correspond to the verticaland horizontal alignment of its windows. Our produc-tion system takes advantage of the oblique SAR geo-metry: All vertically aligned objects, e.g. windows ofconsecutive levels, are depicted into the same azimuthbut increasing range coordinate. This holds for all ob-jects with same X- and Y- but different Z-coordinate ina local world coordinate system regardless of the anglebetween the facade normal and the SAR sensor’s line ofsight. We use this geometric property as prior knowled-ge which is valid for all SAR imaging sensors.

Gestalt theory uses the principles of human percep-tion to describe regularities in data. Approaches of fin-ding manmade structures in PS sets using Gestalt theoryand a good introduction to this theory can be found in[7] and [10].

2. Physical nature of PS

There are many conceivable applications for latti-ce detection in PS sets. One is unveiling the physicalnature of PS which helps to understand and interpret

high-resolution SAR data. Another aim is improvingthe height estimation of PS. Both applications introduceprior knowledge gained by the production system intoexisting algorithms.

The physical nature of PS describes its counterpart inthe real world. Windowsills and walls are forming struc-tures that give rise to strong signal (e.g., corner reflec-tors) but it is still unclear what physical and especiallygeometrical properties those window structures have tofulfill. One approach to unveil these properties is to useray tracing methods [1]. In such simulation experimentsa detailed 3D model of a building is necessary.

Another approach is to fuse regular lattices found inPS point clouds to regular lattices identified in obliqueview aerial images. It is especially reasonable to fusethose datasets since both SAR images as well as obli-que optical imagery are acquired in a side looking geo-metry. Thus, facades are visible in both data sets. Here,the proposed production system can provide the latticespresent in the PS point sets. Extracting regular latticesin oblique photographs can be realized by finding de-formed lattices in a Markov Random Field framework[8]. Aligning lattices from both sources could unveil thephysical nature of PS by exploiting the high resolutionof aerial imagery which allows distinguishing objects incm range.

3. Point selection

The presented approach aims at grouping two-dimensional point sets. Given a stack of 20 TerraSAR-XHigh-Resolution Spotlight images we have to condensethe data to a pointwise representation of relevant struc-tures like facades. A SAR image is a two-dimensionalcylindrical mapping of the real world to the SAR imagecoordinate system defined by the slant-range and azi-muth coordinate. The latter is oriented parallel to the sa-tellites trajectory, whereas the former coincides with theline of sight of the SAR sensor. The measured complex-valued signal is the integral over the third coordinate,corresponding to the angle of the cylindrical mapping.Since the SAR imaging geometry is oblique, the signalof facade elements is visible in the images. A detailedintroduction into SAR geometry can be found in [4].As PS are temporally stable they show a highly cohe-rent phase after removing atmospheric effects as wellas high amplitude compared to clutter [5]. Therefore,we preselect PS by oversampling the mean amplitudeimage and finding its local maxima. Subsequently, weselect that subset which exhibit a temporal coherence[3] higher than 0.7 which yields a PS set of around4,000 PS for a scene of approximately 0.6 km2.

Figure 1. Scheme of aggregating points torows. For every point P the local neigh-borhood ΩP (here: ΩP ∈ N1 to N6) isused to project probable locations of PS.If a PS is found, it is added to the row. Theblue box shows a range-row because thisrow is oriented along the range coordina-te. Green circles mark points belonging toa row through P and one point of ΩP .

4. A 2D production system

The key idea of the 2D production system approachis based on Gestalt theory. This theory dates back tothe beginning of the last century [11] and was intro-duced into computer vision in the 80’s [6]. Instead ofusing radiometric properties of single pixels we try togroup pixels to rows and rows to lattices. This increaseof complexity is facilitated by the use of prior knowled-ge, namely the exploitation of the SAR imaging proper-ties as well as general assumptions about the occurrenceof windows in a facade.

4.1 Step 1: Aggregating points to rows

Given a point set distributed in the range-azimuthplane we first store it in a kd-tree structure allowing toefficiently access the local neighborhood of a specificpoint. The point set is resampled in azimuth directionto obtain an isometric ground resolution δg . The set ofN neighbors of a point P in the local neighborhoodΦ is denoted as ΩP . We choose Φ by considering thewindow distribution at usual facades. Assuming a mi-nimum vertical and horizontal window spacing of 2 mfor average multilevel buildings and given the isome-tric ground resolution δg in m

pixel of the SAR system wederive the lower bound Φmin = 2m

δg. To restrict compu-

Figure 2. Two examples of row redundan-cy. Upper example: The same points areincluded in the two oppositely orientatedrows. Lower example: The short row is asubset of the longer one.

tational load as well as to avoid finding regularities thatdo not correspond to actual facade elements, we confinethe upper bound to five times the lower one. Therefore,ΩP consists of all points that fulfill

Φmin ≤ Φ ≤ 5Φmin (1)

For every point in ΩP we conduct a search for rows inall possible directions. Figure 1 shows this procedureschematically. Starting from P the distance and direc-tion to every point in ΩP are used to predict probablepositions of points belonging to a row through P andΩp. The projected position is extended by a small tole-rance and the successor is added to the row if a pointcan be found here. This step is repeated until no morepoints can be added to the row. Taking the pair P −N5

of the shown example in figure 1, the row can be exten-ded once. Proceeding from the newly added point, nosuccessor can be found and the search ends.

To keep track of redundancies (i.e. two rows sharetwo or more points) in the aforementioned algorithm weuse an adjacency matrix. In this case just the longest rowis kept while the other is dropped. Figure 2 shows twoexamples of row redundancy. After step 1 all points aregrouped to rows or discarded if no valid row could becreated.

4.2 Step 2: Aggregating rows to lattices

Instead of aggregating rows to lattices of maximalsize we want to find those lattices whose two directionscorrespond to the vertical and horizontal alignment ofwindows at a facade. Due to the aforementioned pro-perty of vertically aligned objects in the SAR geometry,we start the lattice forming with rows oriented parallelto the range axis. These rows are called range-rows.

The appearance of the horizontal alignment of win-dows depends on the orientation of the facade to theSAR sensor. Due to the lack of prior knowledge aboutthis orientation we exploit the assumption that all co-lumns of windows in a facade consist of the same num-ber of elements. Therefore the horizontal orientation isthe one which occurs most often for every point in a

Figure 3. Scheme of the assumption thatall columns consist of the same num-ber of facade elements. Red orientation issupposed to correspond to horizontal ele-ment alignment because it occurs mostoften for all points in the range-row (bluebox).

range-row. This holds for all shear mappings of a re-gular grid. Figure 3 shows this approach schematically.For the range-row under investigation (blue box), rowswith the red orientation occur for all three points whilethe orange and green rows only occur twice and oncerespectively. This approach is quite robust to missingpoints as long as the prevalent direction is chosen forevery range-row.

The lattice is then formed by creating meshes out ofthe points belonging to the range-rows and the rows cor-responding to the horizontal alignments. Similar to theavoidance of redundancy mentioned in 4.1 we store in-formation about already treated rows in a graph struc-ture [9]. As a result we obtain lattices of PS in the SARimaging geometry.

5. Results

Different building complexes in the city center ofBerlin were examined for lattices at the facade. Figu-re 4 shows the mean amplitude image (in SAR geome-try) overlaid with the rows corresponding to the hori-zontal alignment of facade elements. The colors codethe orientation of the row. Most relevant facade structu-res are detected. It is striking that the horizontal windowrows build clusters of uniform orientation. Three prin-cipal orientations can be identified (light blue, red andyellow). Since the aggregation to lattices is not done yet,some incorrect rows disturb the result.

Aggregating lattices overcomes these disturbances.Figure 5 shows the same scene but with rows aggrega-ted to lattices. All incorrect rows are discarded due tomissing lattice affiliation. The resulting three groups of

Figure 4. Mean amplitude image overlaidwith PS (blue points) and rows correspon-ding to horizontal window alignments. Formost facades the correct orientation isfound.

facade orientations are in compliance with the expecta-tion about different facades in one area: Most of themare oriented parallel to streets and perpendicular to eachother.

6. Conclusion

We propose a two-dimensional production systemwhich groups single PS to regular lattices whose twodirections correspond to the vertical and horizontal ali-gnment of facade elements. Our approach exploits theoblique SAR imaging geometry as well as the assump-tion that the elements of interest in one facade are ali-gned in horizontal rows of same size. It uses Gestalttheory to aggregate points to rows and rows to latticesin two steps.

The approach contributes to the long-term goal of abetter understanding and interpretation of SAR acqui-sitions. In future work we will introduce the elevationof PS and extend the production system to group threedimensional point clouds.

Literatur

[1] S. Auer, S. Hinz, and R. Bamler. Ray-tracing simu-lation techniques for understanding high-resolution sarimages. IEEE Transactions on Geoscience and RemoteSensing, 48(3):1445–1456, mar. 2010.

[2] R. Bamler, M. Eineder, N. Adam, X. Zhu, and S. Gern-hardt. Interferometric potential of high resolutionspaceborne sar. Photogrammetrie - Fernerkundung -Geoinformation, 2009(5):407–419, nov. 2009.

[3] C. Colesanti, A. Ferretti, F. Novali, C. Prati, and F. Roc-ca. Sar monitoring of progressive and seasonal ground

Figure 5. Mean amplitude image overlaidwith found lattices. Color codes the ori-entation of horizontal window alignments.Three principal orientations can be detec-ted.

deformation using the permanent scatterers technique.IEEE Transactions on Geoscience and Remote Sensing,41(7):1685 – 1701, jul. 2003.

[4] I. Cumming and F. Wong. Digital processing of synthe-tic aperture radar data: algorithms and implementati-on. Number 1 in Artech House remote sensing library.Artech House, 2005.

[5] A. Ferretti, C. Prati, and F. Rocca. Permanent scatterersin sar interferometry. IEEE Transactions on Geoscienceand Remote Sensing, 39(1):8–20, jan. 2001.

[6] D. Marr. Vision: a computational investigation into thehuman representation and processing of visual informa-tion. W.H. Freeman, 1982.

[7] E. Michaelsen, U. Soergel, and U. Thoennessen. Per-ceptual grouping for automatic detection of man-madestructures in high-resolution sar data. Pattern Recogni-tion Letters, 27(4):218 – 225, mar. 2006.

[8] M. Park, K. Brocklehurst, R. Collins, and Y. Liu. Defor-med lattice detection in real-world images using mean-shift belief propagation. IEEE Transactions on PatternAnalysis and Machine Intelligence, 31(10):1804 –1816,oct. 2009.

[9] E. Scheinerman. Matgraph: a matlab tool-box for graph theory, 2008. Retrieved fromhttp://www.ams.jhu.edu/ ers/matgraph.

[10] U. Stilla, E. Michaelsen, U. Soergel, and K. Schulz. Per-ceptual grouping of regular structures for automatic de-tection of man-made objects. In Geoscience and RemoteSensing Symposium, 2003. IGARSS ’03, volume 6, pa-ges 3525 – 3527, jul. 2003.

[11] M. Wertheimer. Laws of organisation in perceptualforms. Psychologische Forschung, 4:301–350, 1923.First published as ’Untersuchungen zur Lehre von derGestalt II’, Translation published in W. Ellis (1938) Asource book of Gestalt psychology, pp. 71-88, London:Routledge & Kegan.