LAND-COVER CLASSIFICATION OF SAR IMAGES BY COMBINING LOW-LEVEL FEATURES AND CATEGORY CONTEXT

Yongke Ding, Lizhong Qiu, Qiuze Yu, Wenxian Yu, Xingzhao Liu

Department of Electronic Engineering, Shanghai Jiao Tong University

ABSTRACT A novel land-cover classification framework for HR SAR images which combines low-level features and category context is presented in this paper. We use patch-based fea-tures for low-level information extraction, including aver-age intensity, texture within a patch and the super texture we proposed to model the texture similarity of neighboring patches. To represent the local category context of SAR images, we propose the label layout filter. This work re-solves local ambiguities of low-level features from a cate-gory context perspective. The framework demonstrates good performance in both accuracy and visual appearance for HR SAR scene interpretation.

Index Terms— Land-cover classification, low-level feature, super texture, category context, label layout filter

1. INTRODUCTION Land use mapping using SAR data plays an important role in many and diverse SAR applications and the availability of high-resolution (HR) SAR images from TerraSAR-X, COSMO-SkyMed, Radarsat-2, etc., has opened new oppor-tunities in land-cover classification [1]. HR SAR data pro-vides more information on scene context and visual ap-pearance, which is pushing toward the emergence of novel methodologies for exploiting radiometric, spatial and cate-gory context features for the category extraction.

This paper presents a land-cover classification frame-work for SAR images based on the combination of low-level features and category context. Firstly we extract patch-based low-level features to classify the SAR images into four classes as in [2]: urban area (UA), woodland (WL), open area (OA) and water (W). Then we improve the classi-fication through appearance regularization based on label layout filter.

The main contributions of our work are (1) introduc-tion of a novel feature, super texture, to model the texture context between image patches and (2) proposal of the label layout filter model which provides a simple but efficient representation of images from the category context perspec-tive. Incorporation of the category context helps to resolve the ambiguities of pre-classification using low-level features.

Experimental results of full-scene TerraSAR-X images prove the effects and efficiency of our methods.

2. LOW-LEVEL FEATURE EXTRACTION We use patch-based features for efficient low-level infor-mation extraction, including average intensity, texture within a patch and super texture between patches. The av-erage intensity I represents the average intensity of all pix-els in a patch. The texture T of a patch represents the local image heterogeneity by means of the coefficient of variation, which is a conventional and straightforward method to de-scribe the local development of speckle in SAR data [2, 3]. The super texture Tsup models the texture context of neigh-boring patches by measuring the heterogeneous coefficient of texture values of the patches in the neighborhood.

Suppose μi gives the mean backscatter intensity value and σi represents the standard deviation of the backscatter intensity for all the pixels of patch i in the SAR image, then

iI �� and /i iT � �� for patch i. 2.1. Super texture We propose super texture, to model the pattern of texture fineness of different classes. In contrast to texture T defined on the pixel level, super texture measures the similarity of textures between neighboring patches, thus providing clues for texture context of neighboring patches. Super texture is defined as,

sup 2

( ) ( )

1 1( )Tj T j

j N i j N iT

T T Tn n

��

� � �

� � �� � (1)

where T� is the average of T and T� is the standard de-viation of T in the neighborhood N(i) of patch i, n is the number of patches in N(i). A 5×5 neighborhood is exploited in our work.

Theoretically, the super texture value will be low for object class with a finer texture and high for class with a coarser texture. We take woodland and open area separation to analyze the experimental characteristics of the proposed feature, as in Fig. 1. In the figure, the feature demonstrates good separating capacity of object classes with similar scat-tering intensity but different coarseness of textures, such as woodland and open area.

3489978-1-4673-1159-5/12/$31.00 ©2012 IEEE IGARSS 2012

(a)

WL

OA

(b) (c) Fig. 1. Characteristics of super texture for separating WL

and OA. (a) Histogram of super texture supT and (b) scat-ter plot of supT vs texture T, derived from image tiles solely covering WL and OA. (c) Slices of WL and OA.

3. LABEL LAYOUT FILTER: MODELING

CATEGORY CONTEXT AND SPATIAL LAYOUT This section represents the proposal of the label layout filter (LLF) and its applications to modeling various contexts to achieve appearance regularization of categories in SAR images. 3.1. Label layout filter The label layout filter (LLF) is proposed to model the class distribution behavior and visual context appearance of clas-ses for SAR images. The label layout filter is defined on a neighboring region Ri of pixel or patch i of the image. A rectangular region, also called a box, is investigated in this paper for efficiency. Assume l denotes the class label of one land-cover type and L is the label map from pre-classification using low-level features. We focus on four land-cover types: urban area, woodland, open area and wa-ter, and use UA, WL, OA and W for short. The response of label layout filter for pixel i is the proportion of pixels under the region Ri that have label l,

,1( ) [ ]

area( )ii

R l jj Ri

v i L lR �

� �� (2)

We can design a filter bank by combining the label layout filters operating on the bounding boxes of different directions with respect to pixel i. An example is the 8-direction label layout filter bank shown in Fig. 2(a). In the figure, d1 denotes distance between the boxes and position i, and d2 - d1 refers to the width of the boxes at radial direction

of position i. The filter response of label l in all the defined boxes Ri of a filter bank is,

,1( ) ( )

num( ) ii

l R lRi

v i v iR

� � (3)

where num(Ri) is the number of boxes defined around posi-tion i. For 8-direction filter bank, num(Ri) = 8.

The label layout filter provides context clues like: (1) which categories exist around position i; (2) dimensional orientation of a certain category; (3) combination of various categories in different regions or boxes. The proposed filter is of a similar form with the texture-layout filter in [4]. While the texture-layout filter is based on the texton map [5], the label layout filter proposed in this paper is based on the label image obtained from pre-classification of SAR images using low-level features.

The filter responses can be efficiently computed over a whole image with integral images [6]. Fig. 2(b) illustrates the process: the raw label map is separated into 4 channels (one for each class) and then, for each channel, a separate integral image is calculated. The integral images can be used to compute the LLF responses in constant time. If ( )ˆ lL is the integral image of L for label channel l, then the label feature response is computed as, ( ) ( ) ( ) ( )

[ , ]ˆ ˆ ˆ ˆ( ) ( ) ( ) ( ) ( )

i

l l l lR l br bl tr tlv i L p L p L p L p� � � � (4)

where pbr, pbl, ptr and ptl denote respectively the bottom right, bottom left, top right and top left corner points of rectangle R [4, 6]. 3.2. Appearance regularization using LLF The proposed LLF is a general model for representing spa-tial context and visual appearance of categories in the im-ages. We explore the ambiguity of pre-classification using low-level features and develop several LLF-based methods to model various contexts of land-cover classes, summa-rized in Table 1. We use 8-direction label layout filter bank, and vl(i) (l = UA, WL, OA, W) is as defined in Eq. (3).

4. LLF-BASED CLASSIFICATION SCHEME AND EXPERIMENTAL RESULTS

Based on the low-level features and the proposed context appearance regularization method discussed above, a classi-fication scheme combining low-level features and category context is proposed. Firstly we extract patch-based features including average intensity, texture within a patch and super texture between patches to pre-classify the SAR images into UA, WL, OA and W. And then we improve the pre-classification using Context_1 to Context_5, the LLF-based appearance regularization methods in Table 1. We estimate the classification thresholds through automati-cally analyzing the histograms of the employed features over the selected test regions, plus manual fine adjustment of parameters.

0 0.2 0.4 0.6 0.8 10

0.02

0.04

0.06

0.08

0.1

0.12

0.14

super-texture value

num

ber o

f pat

ches

woodlandopen area

0 0.2 0.4 0.6 0.80

0.5

1

1.5

super-texture value

text

ure

valu

e

woodlandopen area

3490

upper-right box

lower-left box

left-upperbox

upper-left box

lower-right box

right-lowerbox

right-upperbox

left-lowerbox

d1 d2

i

(1)iR (2)

iR

(3)iR

(4)iR

(5)iR(6)

iR

(7)iR

(8)iR

label map L = label 1 L(1)+label 2 L(2)+label 3 L(3)+label 4 L(4)

UA WL OA W

(a) (b)

Fig. 2. Label layout filter. (a) 8-direction filter bank model. (b) Separating the label map into multiple channels.

Table 1. Modeling various contexts using LLF

Name Ambiguity analysis after pre-classification LLF-based method Context_1 Small isolated areas: small areas of WL in OA, small

area of OA in WL and small area of W close to UA; caused by ambiguities of low-level features; should be classified as the surrounding land-cover type

For Li = WL, if WL ( ) 0v i � and OA 1( )v i (0 < ԑ1 < 1), then set Li = OA; and similar pro-cessing for OA and W

Context_2 Small urban area: urban at the border of woodland (or water) is caused by scattering variance between WL and OA (or W); should be classified as WL; along the water is more likely woods than human settlements [1]

For Li = UA and UA ( ) 0v i � , if W 1( )v i or

WL 2( )v i (0 < ԑ1, ԑ2 < 1), then set Li = WL

Context_3 Urban-surrounded area: mainly woodland; caused by shadows between buildings or homogeneous areas like parking lots; should be classified as UA

For a larger d1 (than in Context_1 and Con-text_2) and Li ≠ UA, if UA 1( )v i (typically 0.7), then set Li = UA

Context_4 Wood-surrounded area: mainly open area; caused by shadows between the height-varying woods or the weak-scattering woods; should be classified as WL

For a larger d1 and Li = OA, if WL 1( )v i (typ-ically 0.7), then set Li = WL

Context_5 Directive/line-like structure: mainly strips of woodland; caused by scattering variance between a ridge (weak scattering) and its neighboring fields (strong scattering relatively); should be classified as OA

For a larger d1 and Li = WL, if UA ( ) 0v i � ,

W ( ) 0v i � , ( ) 1,WL( )k

iRv i when k=1, 2, 5, 6

and ( ) ,WL( ) 0k

iRv i � when k=3, 4, 7, 8, then it

indicates a vertical WL strip across OA, so set Li = WL; and similar processing for WL strip in other directions

For experiments, we use a full-scene TerraSAR-X im-

age of Rosenheim, Germany to evaluate our method. The image is 9504×8330 with resolution of 1.5m. Accuracy be-fore and after appearance-based regularization using LLF is 76.17% and 80.03%, as shown in Table 2. Classification results are presented in Fig. 3. The introduction of label layout filter improves the accuracy of land-cover classifica-tion and enhances the visual appearance of land-cover re-gions.

Table 2. Classification accuracy

Accuracy (%) UA WL OA W Total Before LLF

regularization 74.30 68.90 81.07 77.80 76.17

After LLF regularization 78.26 77.50 82.39 77.84 80.03

5. CONCLUSION AND PERSPECTIVES

This paper has presented a land-cover classification frame-work for HR SAR images which combines low-level fea-tures and category context. We use patch-based features for low-level information extraction, including average intensi-ty, texture within a patch and the proposed super texture which models the texture similarity of neighboring patches. The label layout filter is proposed to represent local catego-ry context of SAR images. This paper resolves local ambi-guities of low-level features from a category layout per-spective. The framework demonstrates good performance in both accuracy and computation for HR SAR scene interpre-tation. For future work, we will introduce CRFs [7] to com-bine the low-level features and category context in one

3491

model and implement automatic learning and inference of the proposed scene interpretation method for SAR images.

(a) (b)

UA

WL

OA

W

(c) (d) Fig. 3. Classification results of a TerraSAR-X image. (a)

Original SAR image. (b) Ground truth. (c) Label map after pre-classification using low-level features. (d) Final result

after LLF-based appearance regularization.

6. ACKNOWLEDGMENT This work has been supported by the National Natural Sci-ence Foundation of China under Grant 61174196 and Na-tional Hi-Tech Research and Development Program of China (863 Program) under Grant 2007AA120206. We thank our colleagues of Geospatial Information Technology Research Center, Shanghai Jiao Tong University.

7. REFERENCES [1] F. Dell'Acqua, P. Gamba, and G. Lisini, “Rapid mapping of

high resolution SAR scenes,” ISPRS Journal of Photogrammetry and Remote Sensing, vol. 64, no. 5, pp. 482-489, Sep, 2009.

[2] T. Esch, A. Schenk, T. Ullmann et al., “Characterization of Land Cover Types in TerraSAR-X Images by Combined Analysis of Speckle Statistics and Intensity Information,” IEEE Transactions on Geoscience and Remote Sensing, vol. 49, no. 6, pp. 1911-1925, 2011.

[3] F. T. Ulaby, F. Kouyate, B. Brisco et al., “Textural Infornation in SAR Images,” Geoscience and Remote Sensing, IEEE Transactions on, vol. GE-24, no. 2, pp. 235-245, 1986.

[4] J. Shotton, J. Winn, C. Rother et al., “TextonBoost for Image Understanding: Multi-Class Object Recognition and Segmentation by Jointly Modeling Texture, Layout, and Context,” International Journal of Computer Vision, vol. 81, no. 1, pp. 2-23, Jan, 2009.

[5] J. Malik, S. Belongie, T. Leung et al., “Contour and texture

analysis for image segmentation,” International Journal of Computer Vision, vol. 43, no. 1, pp. 7-27, 2001.

[6] P. Viola, and M. Jones, “Rapid object detection using a boosted cascade of simple features,” Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol 1, pp. 511-518, 2001.

[7] X. M. He, R. S. Zemel, and M. A. Carreira-Perpinan, “Multiscale conditional random fields for image labeling,” Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Vol 2, pp. 695-702, 2004.

3492