application of level set methods in sar image ......3. segmentation of high-resolution sar images...

1

NSFC-RSE Workshop on Emerging Directions in Image Processing and Understanding

Application of Level Set Methods in SAR Image Segmentation

Zong-Jie Cao

School of Electronic Engineering University of Electronic Science and Technology of China

Email: [email protected]

October 20, 2011

2

Outline

1. Introduction of SAR Images and Level Set MethodsSAR: Principles, Systems and ApplicationsLevel set method: concepts and methods

2. An Unified Energy Functional for SAR Image SegmentationSAR image modelApplication for Target Extraction in SAR ImagesMultiphase SAR image partition

3. Segmentation of High-Resolution SAR ImagesProblem caused by high-resolutionSegmentation of HR SAR Images based on G0 Model

4. Segmentation of Polarimetric SAR ImagesPolSAR images and polarimetric featuresCombing of polarimetric features and a vector valued level set methods

5. Future Work

3

Outline


2. An Unified Energy functional for SAR Image SegmentationSAR image modelApplication for Target Extraction in SAR ImagesMultiphase SAR image partition



5. Future Work

4

Basic Principles of SAR

SAR: Synthetic Aperture Radar

Radar Frequency: P, L, C, X, Ku band

ACTIVE IMAGING SENSORSProvide the maps of remote targets on a terrain by transmitting microwave and detecting its reflected responses.Real Aperture RadarSynthetic Aperture Radar

PASSIVE IMAGING SENSORSProvide the maps by detecting the reflected or emitted electromagnetic radiation from natural sources.Camerasspectrometermicrowave radiometer

Signal Processing: Coherent ProcessingRange: pulse compressionAzimuth: synthetic aperture processing

5

Basic Principles of SAR

Different SAR imaging modes are implemented by adjusting the relative movement

6

SAR Systems

7

SAR Systems

Airborne SAR

Spaceborne SAR

LynxPAMIRMISARE-SAR

RADARSAT-IITerraSAR-XRADARSAT

ERS-1Lacrosse

199920032003199420052006200219911988

10cm×10cm10cm×10cm0.5m×0.5m1.5m×1.5m

3m×3m＜1m×1m25m×25m26m×28m1m×1m

KuX

KaP, L, C, X

CXCCX

Typical SAR Systems

8

Advantages compared to optical remote sensing

– day and night operation (independence of sun illumination)

– no effects of atmospheric constituents (multitemporal analysis)

– sensitivity to dielectric properties (water content , biomass, ice)

– sensitivity to surface roughness ( ocean wind speed)

– sensitivity to man made objects

– sensitivity to target structure (use of polarimetry)

– accurate measurements of distance (interferometry)

– subsurface penetration

SAR Systems

9

Crop classification by using a AIRSAR full polarization SAR data (D. H. Hoekman et.al., 2011).

Top left: L-band radar image(R-HH,G-HV,B-VV); Top right: Classification result; bottom: legend

Soil moisture retrieve by using a TerraSAR-X data (M. Kseneman et.al., 2011). Top: The SAR

image of a agricultural field with low vegetation cover

SAR is an efficient tool in remote sensing.

Applications of SAR

Agriculture Survey

10

Oil spill segmentation (F. Galland et al., 2004)Left: ASAR –Envisat image of the Prestige tanker oil

slick near Galicia;Right: Oil spill segmentation result

Applications of SAR

Environment Protection

Ice thickness estimation of Greenland(Paden et al., 2010)

Up: Flying pathRight: Estimation result

11

Road network extraction (K. Heman et.al., 2010)

Top: A TerraSAR-X image of the city Guan Xian, China

Bottom: Extracted roads (blue-main roads, red and green- side roads)

Building reconstruction(F. Galland et.al., 2004)

Up: SAR image of TerraSAR-X;Bottom: Results of 3D building reconstruction

Applications of SAR

Urban Construction

12

Flood monitoring of the Tagus river in Portugal by using Envisat-ASAR images,

(M. Silveira et.al., 2009)Top: Oct. 1997; Bottom: Nov.1997

Landslide area monitoring in Wenchuanafter the earthquake of May 12, 2008 by

using RADARSAT-II SAR images.

Applications of SARNatural Disaster Monitoring

13

Applications of SAR

Missile Launcher

Helicopter Identification

High resolution SAR images of MiniSARfrom SANDIA Lab

SAR GMTI results displayed on Google Earth (DLR, 2009)

Military Surveillance

14

Missions of SAR Image Processing

ClassificationSegmentation

Parameter InversionTarget Recognition

Speckle Reduction

Interferometry

15

visual information losing Speckle noise due to coherent imaging

Heterogeneous caused by strong reflectors

Geometric distortion due to distance sampling

Disadvantages of SAR Image Data

16

Segmentation of SAR Images

A key but difficult problem in SAR image understanding due to less information and severe speckle noise

Target Extraction Scene Partition

Various methods have been applied• Thresholding• Edge Detection• Region merging/growing

17

Segmentation of SAR Images

Purpose of the study

Solving the SAR image segmentation problems by using level set approaches

Problems that we are concerned about

– Single channel SAR image segmentation

– Multi-region partition

– High resolution SAR image segmentation

– Full polarimetric SAR image segmentation

Basic idea

How to efficiently use the SAR image information in the framework of level set method.

18

Level Set Methods for Image Segmentation

Basic IdeaImage contours is represented as the zero

level set of an function (level set function) defined in a higher dimension.

Image contours is implicitly evolved by evolving level set function.

General Algorithm Steps

0t

∂Φ=

∂

Lagrangian formulation

Eulerian formulation

ε∂∂Φ

ε∂∂Γ

( )ε Γ

Gradient flow

0ε∂=

∂Φ

Euler-Lagrange formulation

19

Advantages of Level Set methods

Naturally representation of regions and edges

– Smooth and closed boundary is good for shape analysis and recognition applications

– Easily handle topological changes

Various image information can be integrated into a unified framework

– Edge, intensity, color, statistical property, shape…

Solid theoretical foundation and feasible tools

– PDE, differential geometry, calculus of variation,

– Optimizing methods, numerical solution…

20

Variational Level Set methods

ProfitsEnergy functional is directly defined on level set functionPDE formulation is directly derived from energy functionalMore convenient and natural for incorporating region-based propertyinformation and boundary-based information in one energy formulation

0ε∂=

∂Φε∂

∂Φ( )ε Φ 0

t∂Φ

=∂

Algorithm Steps

21

Outline





5. Future Work

22

Two assumption is made:

1. The non-overlapping and non-vacant assumption

2. The i.i.d distributed assumption

Image Model for Segmentation

1Ω2Ω

3Ω

4Ω 5Ω

1= ,N

k i jk =Ω Ω Ω Ω = ∅U I

Let is the intensity SAR image, and is composed of N regions ,and each region corresponds to a boundary curve

:u RΩ → 1

Nk k =

Ω

Pr( ) ( ( ); )k k kp u∈Ω =x x θ

1Pr( | ) ( ( ); )

k

Nk kk

u p u= ∈Ω

Λ = ∏ ∏ xx θ

Given a partition of the image domain , the conditioned probability of the whole intensity image is:

1

Nk k =

Λ = Ω

, 1,2...,kC k N=

23

An Unified Energy Functional

• The regional maximum-likelihood based energy:

1

( , ) log Pr( | )

log ( ( ); )k

rN

k kk

E u

p u d= ∈Ω

Λ = − Λ

= ∑ ∫x

θ

x θ x

• The edge information based energy:

( )1( ) | |

k

Nb k C

E g u ds=

Λ = ∇∑ ∫

( , ) ( , ) ( )r bE E EΛ = Λ + Λθ θ

• The regional term is derived from the image model, it is robust to speckle noise since it takes the random property of pixel’s value• The edge-based term can be considered as a weighted length of the boundary curves. It has two effects: to ensure the smoothness of the boundary and to help locating the boundary more precisely

24

Application to Target Extraction

There are only two regions: target and backgroundLevel set representation of regions and boundary

The intensity in each region is assumed to follow the Gamma distribution

Energy functional based on variational level set method

| , ( ) 0 | , ( ) 0 | , ( ) 0

T

B

C = ∈ Ω Φ =⎧⎪Ω = ∈ Ω Φ >⎨⎪Ω = ∈ Ω Φ <⎩

x x xx x xx x x

TΩ BΩ

( )( , ) ( ) log ( ( ); ) 1 ( ) log ( ( ); )

(| |) | ( ) |

T T B BE H p u d H p u d

g u H d

µ µΩ Ω

Ω

Φ = − Φ − − Φ

∇ ∇ Φ

∫ ∫∫

µ x x x x

+ x

1( )( )( ( ); ) , ,

( )k

LL Lu

k kk k

L uP u k T BL e µµ

µ µ

−

−⎛ ⎞= ∈⎜ ⎟Γ ⎝ ⎠

xxx

| , ( ) 0 | , ( ) 1 | , ( ) 0

T

B

C HHH

= ∈ Ω ∇ Φ ≠⎧⎪Ω = ∈ Ω Φ =⎨⎪Ω = ∈ Ω Φ =⎩

x xx xx x

1, 0( ) 0, 0H⎧ Φ>⎨Φ = Φ<⎩

25

Curve Evolution by Energy Minimization

Minimizing E with respect to

Minimizing E with respect to

Φ

( ) ( )( ) log T

B T B

E u u gt

µδ κµ µ µ

⎛ ⎞∂Φ ∂= − = Φ + − +⎜ ⎟∂ ∂Φ ⎝ ⎠

x x

,T Bµ µ

( )( )

( ) ( ) 1 ( ) ( ),

( ) 1 ( )T B

H u d H u d

H d H dµ µΩ Ω

Ω Ω

Φ − Φ= =

Φ − Φ∫ ∫

∫ ∫x x x x

x x

Detailed Algorithm

1. Initialization of the level set function

2. Do until convergence

• Distribution parameter estimation

• Level set function evolving

3. Get the segmentation resultis the Dirac functionis the curvature of the curve

The parameter updating equations are obtained by maximum-likelihood estimation

( )δκ

26

Result

Initial curves

Extractionresults

Mean gray valuepresentation

80%15%

Geodesicactive

contourmethod

85%80%

Segmentationwithout

boundaryinformation

96%93%

Extractionmethod of the unified

energy

After Lee

filtering

Beforefiltering

Comparison results of three algorithms

Target Extraction with MSTAR data

27

Multiregion SAR Image Segmentation

The extension of two-region case to multiregion case is nontrivial

Based on the hierarchical model, the region indicator function can be defined as

(a) Chan-Vese MultiphaseModel

(b) HierarchicalModel

Two kinds of multiphase strategy

( )

( )

11 2 1

11 2 1

( , ,..., ) ( ) 1 ( ) , 1,2..., 1

( , ,..., ) 1 ( )

kk N k ll

NN N ll

M H H k N

M H

−

=

−

=

Φ Φ Φ = Φ − Φ = −

Φ Φ Φ = − Φ

∏∏

28

Energy Functional and Curve Evolution

Energy functional

Curve evolution equations

– Updating

– Updating distribution parameter

1

1 1( , ) log ( ( ); ) (| |) | ( ) |N N

k k k lk lE M p u d g u H dµ −

= =Ω Ω= + ∇ ∇ Φ∑ ∑∫ ∫Φ µ x x x

( )1

11( ) 1 ( )k N jk

k i k j kj kik

MH g

tδ ξ ξ κ−

= +=

∂⎡ ⎤∂Φ= Φ − Φ − +⎢ ⎥∂ ∂Φ⎣ ⎦

∑∏

log ( ) /k k kuξ µ µ= + x

( ) /k k kM u d M dµΩ Ω

= ∫ ∫x x x

, 1,2... 1k k NΦ = −

, 1,2...k k Nµ =

29

Comparison of the Chan-Vese model and the hierarchical model Segmentation of a real SAR image

Result

30

Outline





5. Future Work

31

High-resolution SAR Images

To improve resolution is a goal of SAR technologyHR sensors: TerraSAR-X, COSMO-SkyMed, RAMSES….Although HR images have advantages, they also brings challenges for interpreting:

– SNR becomes lower– Strong reflectors are present – The terrain fluctuation can’t be neglected

Visual Comparison of SAR Images of Different Resolutions

Very high-resolution(0.3 x 0.5 m)

High-resolution(3 x 3m)

Low-resolution(25x6 m)

32

The G0 Model

The G0 model for amplitude SAR images

The G0 model for intensity SAR images

Parameters: n-number of looks, -shape parameter, -scale parameter

Advantages:– Clear physical understanding of the model

– More suitable for describing high-resolution SAR images

– Very flexible

( )( ) ( )( )

02 1

2

2 ( , )( ( , ); , , ) , , , , ( , ) 0

( , )

n nG

A n

n n I x yP I x y n n I x y

n nI x yαα

αα γ α γ

γ α γ

−

−

Γ −= − >

Γ Γ − +

( )( ) ( )( )

01( , )

( ( , ); , , ) , , , , ( , ) 0 ( , )

n nG

I n

n n I x yP I x y n n I x y

n nI x y αα

αα γ α γ

γ α γ

−

−

Γ −= − >

Γ Γ − +

a γ

33

Modeling of High-resolution SAR Images

Types of eight regions

1. Thick woods

2. Target and grass

3. Sparse woods

4. Mixture of different vegetation types

5. Grass

6. Bare soil

7. Target and wood

8. Road

Four distributions are used for comparison:

1. Weibull 2. Rayleigh/Gamma

3. K distribution 4. G0 distribution

An ADTS SAR Image (0.3mx0.3 m)

34

Modeling Performance Comparison

The KL distance is used to evaluate the modeling performance for different distributions

The Gamma model performs worst when modeling heterogeneous areas

G0 model is flexible enough to model different terrain types

KL distance for different regions

( || ) ( || )KLD D q p D p q= +

2( )( || ) ( ) log ( )( )

q wD q p q w dwp w

= ∫

where

35

Using the G0 Model in Level Set Segmentation

(a) Initial curves (b) Result with G0 model

(c) Result with Gamma model

36

Outline





5. Future Work

37

Polarimetric SAR Data

Full polarimetric data can provide more information than singlepolarization image

Data form:

hh hv

vh vv

s ss s

⎛ ⎞= ⎜ ⎟

⎝ ⎠SScattering Matrix:

Target Vector: [ ]1 22

TP hh vv hh vv hvs s s s s= + −k

Vectorizationusing Pauli

basis

Coherency Matrix: 3 31

1 N l l Hp plN

×=

= ∈∑T k k £

Multilook using incoherent averaging

The level set segmentation method for single channel SAR images can not be directly used for dealing with PolSAR data.

38

Polarimetric Features

Multi-channel features– Span:

– Polarization ratio

– Polarization correlation coefficient

Statistical distribution feature– Complex Gaussian distribution for target vector

– Complex Wishart distribution for coherency matrix

Decomposition features– Coherent Decomposition: Pauli, Krogager, Cameron

– Non-coherent Decomposition: Cloude-Pottier, Freeman-Durden, Yamaguchi

2 2 2( ) | | | | 2 | |hh vv hvspan S s s s= + +

Decomposition features are more simple than original polarimetric data.

39

Cloude-Pottier Decomposition

The coherency matrix can be decomposed into three scattering mechanism by eigenvalues and eigenvectors:

Define , three parameters can be defined as:

3 3

3 3 31 1

[cos sin cos sin sin ]i i

H Hi i i i i

i i

j j Ti i i i i i

T u u T U U

u e eδ γ

λ λ

α α β α β= =

= = = Σ

=

∑ ∑

3

1/i i i

jp λ λ

=

= ∑

3

31

logi ii

H p p=

= −∑Entropy:

3

1i i

i

pα α=

= −∑Mean scattering angle:

2 3

2 3

p pAp p

−=

+Anisotropy:

40

Physical Understanding of the H-α-A Parameters

The entropy H is a indicator of the randomness of the global scattering mechanism.

Alpha is a indicator of the mean scattering mechanism.

A is a complementary parameter to H. It measures the relative importance of the second and the third scattering mechanism.

(a) Span (b) Entropy

(c) Alpha (d) Anisotropy

41

Combing with the Vectored Chan-Vese Model

The decomposition parameters are put into a vector

Then the PolSAR image is segmented by using a vector valued Chan-Vese Model

Advantages:– Combing the polarimetric features with the level set segmentation method. As the method is region based, the segmented results is robust to noise.

– The segmentation method of the proposed method can be related to scattering mechanism thanks to the using of decomposition parameters.

– The computation burden is reduced compared to the methods which deal with the original matrix data.

( )( )

( )( ) ( )( )( )2 21 2

E ,

1 1[ ( ) , ( ) 1 , ]

H x y dxdy

H x y dxdy H x y dxdyM M

µ φ

φ φ

Ω

Ω Ω

= ∇

+ − + − −

∫∫

∫∫ ∫∫v c v c

[ ]TH Aα=v

42

Results

(a) Initial curves (c) Results with Wishart

distribution

(b) Results with the feature vector

(a) Entropy (b) Alpha (c) Anisotropy (d) Result

43

Outline





5. Future work

44

Future Work

Issue One:advanced image

processing approaches and

theories

Level SetPCA, ICASVMActive LearningGraph Theory…

Issue Two:Efficient use of SAR

image data characteristics

IntensityProbability distributionTexturePhasePolarimetric property…

PASTDevelopment of SAR systems

PRESENT and FUTURESAR Image Processing,

Understanding and Their Applications

We hope for the broad discussion and cooperation with you

Issue Three:Advanced application

of SAR in different areas

AgricultureDisaster PreventionCity PlanningMilitary Defense…

+ +

45

THANKS!(Q & A)

application of level set methods in sar image ......3. segmentation of high-resolution sar images...

Documents