an efficient local chan–vese model for image segmentationhuanxu4/papers/wangpr10.pdflocal...

Pattern Recognition 43 (2010) 603 -- 618

Contents lists available at ScienceDirect

Pattern Recognition

journal homepage: www.e lsev ier .com/ locate /pr

An efficient local Chan–Vesemodel for image segmentation

Xiao-Feng Wanga,b,c, De-Shuang Huanga,∗, Huan Xua,b

aIntelligent Computing Lab, Hefei Institute of Intelligent Machines, Chinese Academy of Sciences, P.O. Box 1130, Hefei Anhui 230031, ChinabDepartment of Automation, University of Science and Technology of China, Hefei Anhui 230027, ChinacKey Lab of Network and Intelligent Information Processing, Department of Computer Science and Technology, Hefei University, Hefei Anhui 230022, China

A R T I C L E I N F O A B S T R A C T

Article history:Received 28 September 2008Received in revised form 14 May 2009Accepted 2 August 2009

Keywords:Extended structure tensorImage segmentationIntensity inhomogeneityLevel set methodLocal Chan–Vese model

In this paper, a new local Chan–Vese (LCV) model is proposed for image segmentation, which is builtbased on the techniques of curve evolution, local statistical function and level set method. The energyfunctional for the proposed model consists of three terms, i.e., global term, local term and regularizationterm. By incorporating the local image information into the proposed model, the images with intensityinhomogeneity can be efficiently segmented. In addition, the time-consuming re-initialization step widelyadopted in traditional level set methods can be avoided by introducing a new penalizing energy. To avoidthe long iteration process for level set evolution, an efficient termination criterion is presented whichis based on the length change of evolving curve. Particularly, we proposed constructing an extendedstructure tensor (EST) by adding the intensity information into the classical structure tensor for textureimage segmentation. It can be found that by combining the EST with our LCV model, the texture imagecan be efficiently segmented no matter whether it presents intensity inhomogeneity or not. Finally,experiments on some synthetic and real images have demonstrated the efficiency and robustness of ourmodel. Moreover, comparisons with the well-known Chan–Vese (CV) model and recent popular localbinary fitting (LBF) model also show that our LCV model can segment images with few iteration timesand be less sensitive to the location of initial contour and the selection of governing parameters.

© 2009 Elsevier Ltd. All rights reserved.

1. Introduction

Image segmentation has always been a fundamental problem andcomplex task in the field of image processing and computer vision.Its goal is to change the representation of an image into somethingthat is more meaningful and easier to analyze [1]. In other words,it is used to partition a given image into several parts in each ofwhich the intensity is homogeneous. Up to now, a wide variety ofalgorithms have been proposed to solve the image segmentationproblem. Researchers have also done great efforts to improve theperformance of the image segmentation algorithms.

Active contour model, or snakes, proposed by Kass et al. [2], hasbeen proved to be an efficient framework for image segmentation.The fundamental idea of active contour model is to start with acurve around the object to be detected, and the curve moves towardits interior normal and stops on the true boundary of the objectbased on an energy-minimizing model. The main drawbacks of this

∗ Corresponding author.E-mail addresses: [email protected], [email protected] (X.-F. Wang),

[email protected] (D.-S. Huang), [email protected] (H. Xu).

0031-3203/$ - see front matter © 2009 Elsevier Ltd. All rights reserved.doi:10.1016/j.patcog.2009.08.002

method are its sensitivity to initial conditions and the difficultiesassociated with topological changes like the merging and splittingof the evolving curve. Since the active contour model was proposed,many methods have been proposed to improve it, in which level setmethod proposed by Osher and Sethian [3] is the most importantand successful one.

Level set method is based on active contour model and particu-larly designed to handle the segmentation of deformable structures.Generally, the classical active contour model uses spline curves tomodel the boundary of an object. However, the level set method isto use a deformable curve front for approximating the boundary ofan object. In the level set framework, the curve is represented bythe zero level set of a smooth function, usually called the level setfunction. Moving the curves can be done by evolving the level setfunctions instead of directly moving the curves. Therefore, levelset methods exhibit interesting elastic behaviors and can efficientlyhandle the topological changes which is also a main advantage com-pared with classical active contour model. Formally, the evolution ofthe curve is driven by a time-dependent partial differential equation(PDE) where the so-called velocity term reflects the image featurescharacterizing the object to be segmented [4].

Generally, a classical level set framework consists of an implicitdata representation of a hypersurface, a set of partial differential

http://www.sciencedirect.com/science/journal/pr

http://www.elsevier.com/locate/pr

mailto:[email protected]




604 X.-F. Wang et al. / Pattern Recognition 43 (2010) 603 -- 618

equations (PDEs) that govern how the curve moves, and the cor-responding numerical solution for implementing this method oncomputers [5].

It should be mentioned that the early edge-based level set meth-ods [6–8] usually depend on the gradient of the given image for stop-ping the evolution of the curve. Therefore, these methods can onlydetect objects with edges defined by the gradient. However, the cor-responding discrete gradients are generally bounded and the energyfunctional will hardly approach zero on the boundaries in practice.So, the evolving curve may pass through the true boundaries, espe-cially for the models in [6–8].

Recently, region-based level set methods [10–13] have been pro-posed and applied to image segmentation filed by incorporatingregion-based information into the energy functional. Unlike edge-based level set methods using image gradient, region-based meth-ods usually utilize the global region information to stabilize theirresponses to local variations (such as weak boundaries and noise).Thus, they can obtain a better performance of segmentation thanedge-based level set methods, especially for images with weak objectboundaries and noise. Among the region-based methods, Chan–Vesemodel [10] is a representative and popular one.

Based on the Mumford–Shah functional [9] for segmentation,Chan and Vese [10] proposed an easily handled model, or calledChan–Vese (CV) model, to detect objects whose boundaries are notnecessarily detected by the gradient. Mumford–Shah model wasfirstly proposed as a general image segmentation model by Mum-ford and Shah in [9]. Using this model, the image is decomposedinto some regions. Inside each region, the original image is approxi-mated by a smooth function. The optimal partition of the image canbe found by minimizing the Mumford–Shah functional. Chan andVese successfully solved the minimization problem by using level setfunctions, which utilized the global image statistics inside and out-side the evolving curve rather than the gradients on the boundaries.

CV model has achieved good performance in image segmentationtask due to its ability of obtaining a larger convergence range andhandling topological changes naturally. However, it still has someintrinsic limitations. First, CV model generally works for images withintensity homogeneity since it assumes that the intensities in eachregion always maintain constant. Thus, it often leads to poor seg-mentation results for images with intensity inhomogeneity due towrong movement of evolving curves guided by global image infor-mation. Second, the segmentation of CV model is usually dependenton the placement of the initial contour, especially for the compli-cated images. Sometimes, the different results will be obtained onthe same image by using different initial contours. Thus, the place-ment of initial contour is still an important issue for CV modelto get successful segmentation in complicated images. Third, the CVmodel may become time-consuming if the periodical re-initializationstep is adopted, which is a technique for periodically re-initializingthe level set function to a signed distance function during the evo-lution. It has been regarded as a numerical remedy for maintainingstable curve evolution and ensuring precise results, which also leadsto time-consumption as the side effect.

To solve the limitations of CV model, many efficient implemen-tation schemes have been proposed [14–20]. For example, in [14],Vese and Chan extended their original model in [10] by using a mul-tiphase level set formulation. However, the involved computation inthis model is very expensive, which also limits its applications inpractice. In addition, to reduce the computational cost, this methodusually requires that the initial contour should be near to the objectboundaries. In [15], an initialization scheme for the CVmodel was in-troduced, in which the initial curve is found by searching among theexternals of the fidelity term in [10]. However, this one-dimensionalsearch method is also time-consuming and fails to work when thegray difference between object and background is small. Xia et al.

[16] proposed another initialization method which generates ini-tial closed curves by iteratively connecting edge points obtained bycanny detector and morphological filter. This method can efficientlywork for some simple images. To reduce the computational load ofcurve evolution for CV model, the implementation schemes with-out solving the PDEs were proposed [17,18]. However, they are stillsensitive to the selection of initial curves and sometime sensitiveto noise. Li et al. [19] proposed a so-called penalizing energy whichacts as a metric to characterize how close the level set function toa signed distance function. This metric can also be adopted by CVmodel to avoid the re-initialization step. In [20], the penalizing en-ergy proposed in [19] and a discrimination function based on colorinformation was combined into the CV model for segmenting thecolor images.

To efficiently perform the segmentation of images with intensityinhomogeneity, a new class of models has been proposed which notonly utilize region-based techniques but also incorporate the benefitsof local information. There have been several literatures [12,21–27]which are relevant to the existing works. Paragios et al. [12] pre-sented a method in which edge-based energies and region-basedenergies were explicitly summed to create a joint energy which wasthen minimized. In [21], Sum et al. took a similar approach and min-imized the sum of a global region-based energy and a local energybased on image contrast. Brox et al. [22] proposed the idea of in-corporating localized statistics into a variational framework whichshows that segmentation with local means is a first order approxi-mation of the piecewise smooth simplification of the Mumford–Shahfunctional. Piovano et al. [23] employed convolutions to quickly com-pute localized statistics and yielded results similar to piecewise-smooth segmentation in a much more efficient manner. The work ofAn et al. [24] also noted the efficiency of localized approaches ver-sus full piecewise smooth estimation and introduced a way in whichlocalizations at two different scales can be combined to allow sen-sitivity to both coarse and fine image features. In [25], the authorsproposed a similar flow based on computing geodesic curves in thespace of localized means rather than approximating a piecewise-smooth model. In [26], a novel localization framework was proposedwhich allows the region-based energy to be localized in a fully vari-ational way so that objects with heterogeneous statistics can be suc-cessfully segmented with the localized energies. Recently, Li et al.proposed an efficient region-based level set method by introducinga local binary fitting (LBF) energy with a kernel function [27]. TheLBF model enables the extraction of accurate local image and can beused to segment the images with intensity inhomogeneity. It has at-tracted extensive attentions for its good segmentation performancein limited iterations. However, the LBF model usually needs to per-form four convolution operations at each iteration, which greatlyincreases the computational complexity. In addition, it is also sen-sitive to the selection of governing parameters and the location ofinitial curve.

For CV model using the intensity average only, texture image seg-mentation is another difficult issue since intensity averages cannotrepresent the texture information inside and outside the target ob-jects. In many texture images, due to the difference of the intensityaverages of neighboring textures, the small textures in objects willbe segmented while the desirable whole objects will be not sepa-rated. Therefore, other information should be introduced for textureimage segmentation. Chan and Vese [10] suggested using texture in-formation or features extracted from the original image, such as thecurvature or the orientation of level sets, to overcome the difficulty.However, the proposed texture information in [10] cannot work wellin many complicated texture images due to their simple properties.Note that the texture image segmentation greatly relies on the ex-traction of suitable texture information from the image. Recently,Gabor filters have been efficiently incorporated into level set

X.-F. Wang et al. / Pattern Recognition 43 (2010) 603 -- 618 605

methods and CV model for the texture image segmentation[12,28,29]. Unfortunately, Gabor filters have the fatal drawback thatthey induce a lot of redundancies and thus lots of feature channels[30]. As another efficient texture representation, the structure ten-sor [31,32] is a kind of low dimensional feature computed from thespatial derivatives of the image. It is a common tool for local ori-entation estimation and image structure analysis which is formedas the outer product of the image gradient with itself. So far, thestructure tensor has been applied in many image segmentationalgorithms, most notably, in the early geodesic active contoursframework [33–35].

In this paper, we proposed a so-called local Chan–Vese (LCV)model which utilizes both global image information and local im-age information for image segmentation. The energy functional forthe proposed model consists of three parts: global term, local termand regularization term. By using the local image information, theimages with intensity inhomogeneity can be efficiently segmentedin limited iterations. Moreover, the time-consuming re-initializationstep widely adopted in traditional level set methods can be avoidedby introducing a new penalizing energy to the regularization term.As a result, the time-consumption is greatly decreased. Specially, theevolving curve in level set evolution process can automatically stopon true boundaries of objects according to a termination criterionwhich is based on the length change of evolving curve. Finally, weproposed constructing an extended structure tensor (EST) by addingthe intensity information into the classical structure tensor for tex-ture image segmentation. By incorporating the EST into the proposedLCV model, the texture image can be easily segmented no matterwhether it presents intensity inhomogeneity or not. Moreover, thecomparisons with the CV model and recent LBF model show thatour LCV model can segment ordinary/texture images with or with-out intensity inhomogeneity in fewer iterations. Particularly, it canbe found that our model is less sensitive to the location of initialcontour and the selection of governing parameters.

The rest of this paper is organized as follows: In Section 2,we briefly review the Mumford–Shah model and Chan–Vese (CV)model. Our local Chan–Vese (LCV) model is presented in Section 3. InSection 4, the proposed model is validated by some experimentson synthetic and real images. Finally, some conclusive remarks areincluded in Section 5.

2. Previous works

2.1. Mumford–Shah model

The Chan–Vese model is the curve evolution implementationof a piecewise-constant case of the Mumford–Shah model [9]. TheMumford–Shah model is an energy-based method introduced byMumford and Shah via an energy functional. The basic idea is to finda pair of (u,C) for a given image u0, where u is a nearly piecewisesmooth approximation of u0, and C denotes the smooth and closedsegmenting curve. The general form for the Mumford–Shah energyfunctional can be written as

EMS(u,C) =∫�

|u0(x, y) − u(x, y)|2 dxdy

+ �∫�\C

|∇u(x, y)|2 dxdy + � · Length(C), (1)

where � and � are positive constants, � denotes the image domain,the segmenting curve C ⊂ �. To solve the Mumford–Shah problem istominimize the energy functional over u and C. Note that the removalof any of the above three terms in (1) will result in trivial solutionsfor u and C [9]. However, with all three terms, it becomes a difficultproblem to solve since u is a function in the N-dimensional space

(N = 2 in 2D image segmentation), while C is an (N−1)-dimensionaldata set.

2.2. Chan–Vese model

The Chan–Vese (CV) model is an alternative solution to theMumford–Shah problem which solves the minimization of (1) byminimizing the following energy functional:

ECV (c1, c2,C) = � · Length(C) + �1 ·∫inside(C)

|u0(x, y) − c1|2 dxdy

+ �2 ·∫outside(C)

|u0(x, y) − c2|2 dxdy, (2)

where �, �1 and �2 are positive constants, usually fixing �1 = �2 = 1.c1 and c2 are the intensity averages of u0 inside C and outside C,respectively.

To solve this minimization problem, the level set method [3] isused which replaces the unknown curve C by the level-set func-tion �(x, y), considering that �(x, y)>0 if the point (x, y) is inside C,�(x, y)<0 if (x, y) is outside (x, y), and �(x, y)=0 if (x, y) is on C. Thus,the energy functional ECV (c1, c2,C) can be reformulated in terms ofthe level set function �(x, y) as follows:

ECV� (c1, c2,�) = � ·∫�

�(�(x, y))|∇�(x, y)|dxdy

+ �1 ·∫�

|u0(x, y) − c1|2H�(�(x, y))dxdy

+ �2 ·∫�

|u0(x, y) − c2|2(1 − H�(�(x, y)))dxdy, (3)

where H�(z) and �(z) are, respectively, the regularized approxima-tion of Heaviside function H(z)and Dirac delta function (z) as fol-lows:

H(z) ={1 if z�0,

0 if z<0,(z) = d

dzH(z). (4)

Thisminimization problem is solved by taking the Euler–Lagrangeequations and updating the level set function �(x, y) by the gradientdescent method:

��t

= �(�)[�div

( ∇�|∇�|

)− �1(u0 − c1)

2 + �2(u0 − c2)2], (5)

where c1 and c2 can be, respectively, updated at each iteration by

c1(�) =∫� u0(x, y)H�(�(x, y))dxdy∫

� H�(�(x, y))dxdy,

c2(�) =∫� u0(x, y)(1 − H�(�(x, y)))dx dy∫

�(1 − H�(�(x, y)))dxdy. (6)

The main advantages of this model are: First, it can deal with thedetection of objects whose boundaries are either smooth or not nec-essarily defined by gradient. In such cases, the edge-based level setmethods commonly fail and result in boundary leakage [36]. Second,it does not require image smoothing and thus can efficiently processthe images with noise. Therefore, the true boundaries are preservedand could be accurately detected. Third, it can automatically detectinterior contours with the choice of Dirac delta function (z) thathas non-compact support.

However, CV model also has some drawbacks which have beendescribed in Section 1, i.e., the unsuccessful segmentation of im-ages with intensity inhomogeneity (as shown in Fig. 3(c)), the sen-sitivity to the placement of initial contour and the extraordinarytime-consumption if re-initialization step is adopted for maintainingstable curve evolution and ensuring more precise results.


3. Local Chan–Vese model

In this section, we shall present and discuss the details ofour proposed local Chan–Vese (LCV) model and its numericalimplementation. What should be stressed is that our model is de-fined based on the techniques of curve evolution, local statisticalfunction and level set methods. It is well-known that some tradi-tional level set methods used either the image gradient [6–8] or theglobal information [10,11] to drive the evolving curve(s) towardsthe true boundaries. However, none of them can achieve success insegmenting images with intensity inhomogeneity. In the proposedmodel, we combined both global and local statistical information toovercome the inhomogeneous intensity distribution in some imagesand provided more satisfying segmentation result. Particularly, byincorporating a so-called extended structure tensor(EST) into theproposed LCV model, the texture image can also be segmented nomatter whether it presents intensity inhomogeneity or not. Theoverall energy functional in our proposed LCV model ELCV consistsof three parts: global term EG, local term EL and regularization termER. Thus the overall energy functional can be described as

ELCV = · EG + � · EL + ER. (7)

3.1. Global term

The global term EG is directly derived from (2) in the Chan–Vesemodel, in which it is also called the fitting term. It can be seen thatthe global term is defined based on the global properties, i.e., theaverages of u0 inside C and outside C, which is stated as follows:

EG(c1, c2,C) = F1(C) + F2(C) =∫inside(C)

|u0(x, y) − c1|2 dxdy

+∫outside(C)

|u0(x, y) − c2|2 dxdy, (8)

Using the level set formulation, the boundary C is represented bythe zero level set of a Lipschitz function � : � → R.

�(x, y)

⎧⎪⎨⎪⎩

> 0 if (x, y) is inside C,

= 0, (x, y) ∈ C,

< 0 if (x, y) is outside C.

(9)

Accordingly, the global term in (8) can be rewritten so as to eval-uate the level set function � on the domain �:

EG(c1, c2,�) =∫�

|u0(x, y) − c1|2H(�(x, y))dxdy

+∫�

|u0(x, y) − c2|2(1 − H(�(x, y)))dxdy, (10)

where H(z) is the Heaviside function described in (4).Usually, after (10) comes to a steady state, or approximately to

be zero, the evolving curve C (zero level set of �) will separate theobject from the background. However, for the images with inten-sity inhomogeneity, the final obtained curve C can hardly divide theimage into object region and background region even after a longiteration time. The reason is that the global term assumes that theimage intensity is piecewise constant like the CV model. Thus, theaverages c1 and c2 actually act as the global information and cannotrepresent the inhomogeneous intensities of object region and back-ground region in the images with intensity inhomogeneity. So, toachieve a good performance in segmenting the images with intensityinhomogeneity, the local image information needs to be included.

3.2. Local term

Before introducing the local term, we should firstly discuss theintensity inhomogeneity problem. Intensity inhomogeneity usuallyarises from the imperfect factors of acquisition process for ordinaryimages ormedical images, such as non-uniform daylight and artificialillumination, static field inhomogeneity, radio-frequency excitationfield non-uniformity and inhomogeneity of reception coil sensitivity,etc. These inhomogeneities are known to appear in images as sys-tematic changes in the local statistical characteristics of target object.Although the presence of intensity inhomogeneity is usually hardlynoticeable to a human observer, many image processing methods,including image segmentation methods, are highly sensitive to thespurious variations of image intensities since they are based on theassumptions that the intensities in each region are constant.

The generally accepted assumption on intensity inhomogeneityis that it manifests itself as a smooth spatial varying function overthe image [37]. The most common model in describing the acquiredimages X′ with intensity inhomogeneity effect is

X′ = BX + N, (11)

where X is the inhomogeneity-free image, B denotes the intensityinhomogeneity field and N is the noise. To simplify the computation,the noise is often ignored. Also, there have been theoretical mod-eling approaches to approximate the intensity inhomogeneity field.However, due to the complexity that causes the intensity inhomo-geneity, it is difficult for ones to model the intensity inhomogeneityunder a variety of image acquisition conditions [38].

Since the intensity inhomogeneity is slowly varying in the im-age domain, its spectrum in frequency domain will be concentratedin the low-frequency area. Thus, the intensity inhomogeneity effectmainly influences the non-contour pixels in the image, whereas forthe pixels belonging to contour, this influence is less. Motivated bythis observation, we proposed incorporating the local statistical in-formation into the level set method for segmenting the images withintensity inhomogeneity effect. It should be noted that we do not tryto eliminate the intensity inhomogeneity from the images which isstill not a completely solved problem [38].

Here, the local term is introduced in (12) which uses the localstatistical information as the key to improve the segmentation ca-pability of our model on the images with intensity inhomogeneity.

EL(d1,d2,C) =∫inside(C)

|gk ∗ u0(x, y) − u0(x, y) − d1|2 dxdy

+∫outside(C)

|gk ∗ u0(x, y) − u0(x, y) − d2|2 dxdy, (12)

where gk is a averaging convolution operator with k×k size window.d1 and d2 are the intensity averages of difference image (gk∗u0(x, y)−u0(x, y)) inside C and outside C, respectively.

The assumption behind the proposed local term is that smallerimage regions are more likely to have approximately homogeneousintensity and the intensity of the object is statistically different fromthe background. It is significative to statistically analyze each pixelwith respect to its local neighborhood. The most simple and faststatistical information function is the average of the local intensitydistribution, the rationale being that if the object pixels are brighterthan the background, they should also be brighter than the average.It should be noticed that the size of the neighborhood has to beproperly selected so as to cover sufficient object and backgroundpixels, which may make the local term less sensitive to the existenceof noise. However, at a larger neighborhood size, the local term willprobably lose some fine detail of images. So, it needs to be combinedwith the global term in the image segmentation process.

By subtracting the original image from the averaging convolutionimage, the contrast between foreground intensities and background


intensities can be significantly increased. Note that the differenceimage (gk ∗ u0(x, y) − u0(x, y)) with higher image contrast is still noteasily to be segmented due to the weak object boundaries and com-plicated topological structure. It needs under a level set evolution forobtaining better segmentation result. Thus, the structure of the fit-ting term in (8) is adopted, i.e., local fitting term, with the differenceimage being used instead of original image. It can be obviously seenthat the local fitting term keeps decreasing while the curve evolvestowards the true boundaries of objects in difference image, and thetrue boundary C∗ is the minimizer of the following local fitting term:

infC(FL1(C) + FL2(C)) ≈ 0 ≈ FL1(C

∗) + FL2(C∗), (13)

where (FL1(C) + FL2(C)) denotes the local fitting term.In the same manner as global term, the local term (12) can also

be reformulated in terms of the level set function �(x, y) as follows:

EL(d1, d2,�)

=∫�

|gk ∗ u0(x, y) − u0(x, y) − d1|2H(�(x, y))dxdy

+∫�

|gk ∗ u0(x, y) − u0(x, y) − d2|2(1 − H(�(x, y)))dx dy. (14)

3.3. Regularization term

In order to control the smoothness of the zero level set and fur-ther avoid the occurrence of small, isolated regions in the final seg-mentation, we add to the regularization term a length penalty termL(C) which is defined related to the length of the evolving curve C.Let C be a smooth closed planar curve C(p) : [0, 1] → � parameter-ized by parameter p ∈ [0, 1]. The length functional can be written as

L(C) =∮C

dp. (15)

Here, through replacing the curve C by the level set function�(x, y), L(C) can be reformulated as

L(� = 0) =∫�

|∇H(�(x, y))|dxdy =∫�

(�(x, y))|∇�(x, y)|dxdy, (16)

where H(z) is Heaviside function and (z) Dirac delta function, whichhave been described in (4).

The use of length penalty term implies that the evolving curveC which minimizes the overall energy functional should be as shortas possible. It imposes a penalty on the length of the curve thatseparates the two phases of image, i.e., foreground and background,on which the energy functional will make a transition from one ofits values, c1(d1), to the other, c2(d2).

In many situations, the level set function will develop shocks,very sharp and/or flat shape during the evolution, which in turnmakes further computation highly inaccurate in numerical approxi-mations. To avoid these problems, it is necessary to reshape the levelset function to a more useful form, while keeping the zero locationunchanged. A common numerical scheme is to initialize the function�(X, t = 0) as a signed distance function before the evolution, andthen re-initialize the function �(X, t) to be a signed distance functionperiodically during the evolution, which can be written as

�(X, t) =

⎧⎪⎨⎪⎩dist(X,Ct) if X is inside Ct ,

0, X ∈ Ct ,

−dist(X,Ct) if X is outside Ct ,

(17)

where dist(X,Ct) is the shortest Euclidean distance of X to the pointson the evolving curve Ct at time t.

It is crucial to keep the evolving level set function as an approx-imate signed distance function during the evolution, especially inthe neighborhood around the zero level set [5]. The most straight-forward way of implementing the re-initialization operation is toextract the zero level set and then explicitly compute the distancefunction from it. However, this method is generally time-consuming.To overcome this difficulty, a now widely accepted method has beenproposed [39] in order to re-initialize the level set function by solv-ing the following partial difference equation:

��t

= sign(�0)(1 − |∇�|), (18)

where �0 is the function to be re-initialized, and sign(�0) is the signfunction. When the steady state of Eq. (18) is reached, � will be adistance function with the same zero level set as �0 despite �0 is adistance function or not. This is commonly known as the standardre-initialization procedure.

Another equivalent approach is to solve the following eikonalequation:

|∇�| = 1, (19)

with the boundary condition � = 0 on {�0 = 0}.Re-initialization has been extensively used as a numerical rem-

edy for maintaining stable curve evolution and ensuring desirableresults in the level set methods. Unfortunately, it is obviously a dis-agreement between the theory of level set method and its imple-mentation, since it has an undesirable side effect of moving the zerolevel set away from its original location. Moreover, it is quite com-plicated and time-consuming, and when and how to apply it is stilla serious problem [40]. Accordingly, some fast techniques [41,42]were proposed for finding a solution to these problems. Among thesemethods, fast marching method [42] is a representative and popu-lar one. It is the optimal technique for solving the Eikonal equationF|∇�| =1, where F denotes the speed of interface. For more detailedtechnical description about fast marching method, readers can re-fer to literature [42]. Though fast marching method is more efficientthan traditional approach, the computational time is still large.

In this paper, we did not directly use the re-initialization step tokeep the level set function as a signed distance function but add tothe regularization term a penalty term as follows:

P(�) =∫�

12(|∇�(x, y)| − 1)2 dxdy (20)

which can force the level set function to be close to a signed distancefunction.

Actually, this penalty term is more like a metric which charac-terizes how close a function � is to a signed distance function. Themetric plays a key role in the elimination of re-initialization in ourmethod. To explain the effect of the penalty term P(�), we give itsgradient flow as follows:

∇2� − div( ∇�

|∇�|)

= div[(

1 − 1|∇�|

)∇�

]. (21)

Notice that the above gradient flow has the factor(1−(1/|∇�|) whichcan act as the diffusion rate. If |∇�|>1, the diffusion rate is positiveand the effect of the penalty term is the usual diffusion, i.e. making� more even and therefore reduce the gradient |∇�|. If |∇�|<1, thepenalty term has effect of reverse diffusion and therefore increasethe gradient. It can be seen that the penalty term continually adjuststhe deviation of level set function� from the signed distance functionin the evolution process. Therefore, it can naturally and automaticallyforce the level set function to be an approximate signed distancefunction during the evolution.


So, in the proposed model, the regularization term ER should becomposed of two terms:

ER(�) = � · L(� = 0) + P(�)

= � ·∫�

(�(x, y))|∇�(x, y)|dxdy +∫�

12(|∇�(x, y)| − 1)2 dxdy,

(22)

where � is the parameter which can control the penalization effectof length term: if � is small, then smaller objects will be detected;if � is larger, then larger objects are detected.

3.4. Level set formulation

In the level set formulation, the curve C is represented by thezero level set of a Lipschitz function �. The overall energy functionalin (7) can be further described as follows:

ELCV (c1, c2, d1, d2,�) = · EG(c1, c2,�) + � · EL(d1,d2,�)

+ ER(d1,d2,�), (23)

where and � are two positive parameters which govern the tradeoffbetween the global term and the local term. In fact, and � shouldbe set according to the intensity inhomogeneity presenting in theimages. For images without intensity inhomogeneity, the value of is suggested to be near or equal to that of �. If images present distinctintensity inhomogeneity, the value of should be selected less thanthat of � so as to restrict the intensity inhomogeneity. It shouldbe noticed that the case of = 0 may be acceptable in segmentingsome images. However, it is not suggested since the global termcan sometimes have a restriction effect on noise and maintain theboundary details. In our experiments, we usually fixed � = 1 andthen dynamically adjusted the value of according to the intensityproperty of images.

In general, the image segmentation process can be equivalentlytransformed into finding a solution that minimizes the ELCV by evolv-ing level set function �. The Heaviside function H(z) and the DiracDelta function (z) described in (4) are then applied to divide thelevel set function into three parts, i.e., the part inside C, the part out-side C and the part on C. For practical and feasible implementation,H�(z) is chosen as a non-compactly supported, smooth and strictlymonotone approximation of H(z), which can be written as

H�(z) = 12

∣∣∣∣1 + 2�arctan

∣∣∣∣ z�∣∣∣∣∣∣∣∣ , � → 0. (24)

The regularized approximation �(z) of Dirac delta function (z)is correspondingly computed by

�(z) = 1�

· ��2 + z2

. (25)

So, the overall energy functional can then be rewritten as

ELCV (c1, c2, d1, d2,�)

=∫�( · |u0(x, y) − c1|2 + � · |gk ∗ u0(x, y) − u0(x, y) − d1|2)

× H�(�(x, y))dxdy +∫�( · |u0(x, y) − c2|2

+ � · |gk ∗ u0(x, y) − u0(x, y) − d2|2)(1 − H�(�(x, y)))dxdy

+(� ·∫�

�(�(x, y))|∇�(x, y)|dxdy+∫�

12(|∇�(x, y)|−1)2 dxdy

),

(26)

where gk is the averaging convolution operator with k× k size win-dow for local information detection.

Here, the gradient descent method is used to compute the min-imizer of (26). For a fixed level set function �, we minimize the

energy functional in (26) with respect to two pairs of constants:c1 and c2, d1 and d2. By calculus of variations, it can be shown thatthe constant functions c1(�), c2(�), d1(�) and d2(�) that minimizeELCV (c1, c2, d1, d2,�) for a fixed function � are given by

c1(�) =∫� u0(x, y)H�(�(x, y))dxdy∫

� H�(�(x, y))dxdy, (27a)

c2(�) =∫� u0(x, y)(1 − H�(�(x, y)))dx dy∫

�(1 − H�(�(x, y)))dx dy, (27b)

d1(�) =∫�(gk ∗ u0(x, y) − u0(x, y))H�(�(x, y))dxdy∫

� H�(�(x, y))dx dy, (27c)

d2(�) =∫�(gk ∗ u0(x, y) − u0(x, y))(1 − H�(�(x, y)))dx dy∫

�(1 − H�(�(x, y)))dx dy. (27d)

Keeping c1, c2, d1 and d2 fixed, andminimizing the overall energyfunction ELCV in (26) with respect to �, we can deduce the associatedEuler–Lagrange equation for �. The minimization of (26) can be doneby introducing an artificial time variable t�0, and moving � in thesteepest descent direction to a steady state with the initial conditiondefined in (28b) and boundary condition defined in (28c):

��t

= �(�)[−((u0 − c1)2 + �(gk ∗ u0(x, y) − u0(x, y) − d1)

2)

+ ((u0 − c2)2 + �(gk ∗ u0(x, y) − u0(x, y) − d2)

2)]

+[��(�)div

(∇�∣∣∇�

∣∣)

+(

∇2� − div

(∇�∣∣∇�

∣∣))]

, (28a)

�(0, x, y) = �0(x, y) in �, (28b)

��−→n = 0 on ��, (28c)

where −→n denotes the exterior normal to the boundary ��.In the above partial differential equation, the Neumann bound-

ary condition in (28c) is chosen as the boundary condition. Usually,Neumann boundary condition has many advantages. First, it is easyto implement since there are no values to assign for � at the bound-ary. Second, it implies that the solution of (26) satisfies a maximumprinciple. Moreover, the gaps of the curve C may appear only whenadvancing the zero level set which would change its topology, andcannot come from outside of � because of the spurious values cre-ated by the Neumann boundary condition.

3.5. Termination criterion for curve evolution

Evolving curve C, or zero level set function, will gradually splitaccording to the topological structure of objects and evolves towardsthe true boundaries of objects in images. When evolving curves fi-nally arrive at the position of true boundary C∗, the curves shouldstop evolving. Now, a problem of when and how the curves auto-matically stop evolving or what termination criterion is for curvesevolution is arising.

It can be obviously seen that the global term and local term keepdecreasing while the curve evolves towards the true boundaries ofobjects and the true boundary C∗ is the minimizer of the global termand local term, which can be written as follows:

infC( · EG(C) + � · EL(C)) ≈ 0 ≈ · EG(C∗) + � · EL(C∗). (29)

Therefore, we can approximately obtain the energy minimizationvalue by

infc1,c2,d1,d2,C

(ELCV (c1, c2, d1, d2,C)) ≈ � · L(C∗). (30)


Actually, (30) can be the termination criterion for curves evolu-tion. But, it cannot be directly implemented in practice because ofits approximate equal sign. In our numerical solution, we shall useanother form of this termination criterion by determining the lengthof evolving curve L(C(t)) at each iteration. When curve reaches thetrue boundary, it is obvious that the variable L(C(t)) will remain al-most constant. Here, we shall give the termination criterion for thecurve evolution in an image as follows:

Termination criterion: If the absolute value of the change of thecurve length |L(C(t))−L(C(t−1))| remains smaller than a given thresh-old length over a fixed threshold of iterations Tit , the evolution ofcurves will be stopped.

The pseudocode of this termination criterion is listed as follows:

k = 1;. . . . . .if |Length(C(t)) − Length(C(t − 1))| � length then{ if k = Tit then stop evolution of curve;k = k+1;} else k = 1;end

If the condition |L(C(t)) − L(C(t − 1))| � length is firstly satisfiedat a later stage, it is unnecessary to stop the evolution of curveimmediately. Instead, we shall examine whether or not the conditionis maintained over a fixed threshold of iterations Tit before stoppingthe evolution of curve. This examination must be performed becausethe evolution process often slows down temporarily even before trueboundary is reached. In practice, the choice of the two thresholds isflexible. Generally, we always set Tit = 10 and length = 5.

Remark 1. Note that the energy functional (26) of proposed modelis non-convex and sometimes it may have multiple local minima,these being properties inherited from the Mumford–Shah model. Toimprove the ability of curve to approach the true boundary, the localinformation is incorporated into the energy functional. The globalterm is also employed to restrain the noise and maintain the bound-ary details. In practice, we do not guarantee that our model can al-ways converge to a global minimizer. Sometimes, a local minimizermay be obtained, but close to a global minimizer. In this case, theenergy functional will become stationary and the proposed termi-nation criterion can also be applied.

3.6. Numerical implementation of the model

The partial differential equation in the continuous domain de-fined in (28a) can be solved by a finite difference method in numer-ical scheme. All the spatial partial derivatives are approximated bythe central difference and the temporal partial derivatives are ap-proximated by the forward difference.

Then, (28a) can be discretized using the forward difference asfollows:

�n+1i,j − �n

i,j

�t= L(�n

i,j), (31)

where �t is the time-step and L(�ni,j) is the numerical approximation

of the right-hand side in (28a).The corresponding curvature � = div(∇�/|∇�|) in the L(�n

i,j) canbe discretized using a second-order central differencing scheme:

� = div( ∇�

|∇�|)

= �xx�2y − 2�xy�x�y + �yy�

2x

(�2x + �2

y)3/2

, (32)

where �x, �y, �xx, �yy and �xy are computed as follows:

�x = 12h

(�i+1,j − �i−1,j), �y = 12h

(�i,j+1 − �i,j−1),

�xx = 1h2

(�i+1,j + �i−1,j − 2�i,j), �yy = 1h2

(�i,j+1 + �i,j−1 − 2�i,j),

�xy = 1h2

(�i+1,j+1 − �i−1,j+1 − �i+1,j−1 + �i−1,j−1), (33)

where h is the grid spacing. Eq. (28a) is then implemented as follows:

�n+1i,j − �n

i,j

�t

= �(�ni,j){−((ui,j − c1(�

n))2 + �(gk ∗ ui,j − ui,j − d1(�n))2)

+ ((ui,j − c2(�n))2 + �(gk ∗ ui,j − ui,j − d2(�

n))2)}

+ [� · �(�ni,j)� + (�n

i+1,j + �ni−1,j + �n

i,j+1 + �ni,j−1 − 4�n

i,j − �)],(34)

where � and � are computed according to (25) and (32), respectively.

3.7. Texture image segmentation

Usually, the texture image segmentation algorithms include twosteps: First, a texture representation is selected and correspond-ing texture features are extracted from initial images. Second, someobjective function can be defined using the texture features, andthe segmentation is formulated as an optimization or minimizationproblem. In this paper, the extended version of classical structuretensor was proposed to act as the texture feature and further incor-porated into the LCV model for texture image segmentation.

For a scalar image I, the classical structure tensor J� is ob-tained by Gaussian smoothing of the tensor product of the imagegradient, i.e.:

J� = K� ∗ (∇I∇IT ) =(

K� ∗ I2x K� ∗ IxIy

K� ∗ IxIy K� ∗ I2y

), (35)

where K� is a Gaussian kernel with standard deviation �, and sub-scripts x and y denote the partial derivatives. For vector-valued im-ages, the following expression is employed:

J� = K� ∗⎛⎝ n∑

i=1

∇Ii∇ITi

⎞⎠ . (36)

It can be seen from (35) that the classical structure tensor yieldsthree feature channels for each scale. The matrix representation us-ing the image gradient allows the integration of information froma local neighborhood without considering cancellation effects. Sucheffects would appear if gradients with opposite orientation were in-tegrated directly [43]. Moreover, the usage of Gaussian smoothingwill cause the classical structure tensor robust to noise and make theorientation estimation at certain pixel to be performed. Comparingthe number of features obtained by the classical structure tensor tothat of Gabor filters reveals that the degree of freedom for the ori-entation known from Gabor filters is replaced by the smoothed ver-sions of the image derivatives [30]. Since the image derivatives holdthe whole orientation information, the components of the structuretensor are as powerful for the discrimination of textures as a wholeset of Gabor filters with a fixed scale. Due to the mixed matrix com-ponent, texture discrimination of classical structure tensor is alsofully rotation invariant.

However, the classical structure tensor has the disadvantage ofnot using any intensity information (or color information in the case


of vector-valued images) at all. When the intensity (color) of textureobject is different from that of background in a texture image, theintroduction of intensity information will be of benefit to the tex-ture object segmentation. Moreover, if texture images present thedistinct intensity inhomogeneity as we discussed in Section 3.2, thesegmentation using the classical structure tensor will also fail evenif the LCV model with local term is adopted. The reason is due tothe fact that the intensity information is missed of which the LCVmodel is strongly dependent. Therefore, we consider constructing anextended structure tensor (EST) by incorporating the intensity infor-mation into the classical structure tensor for texture image segmen-tation.

For a scalar image I, the extended structure tensor (EST) JE� canbe defined as follows:

JE� = K� ∗ (vvT ) =⎛⎝ K� ∗ I2x K� ∗ IxIy K� ∗ IxIK� ∗ IxIy K� ∗ I2y K� ∗ IyIK� ∗ IxI K� ∗ IyI K� ∗ I2

⎞⎠ , (37)

where v = [Ix Iy I]T .For vector-valued images, the following expression can be used:

JE� = K� ∗⎛⎝ N∑

i=1

vivTi

⎞⎠ , (38)

where vi = [Ii,x Ii,y Ii]T .

Compared with the classical structure tensor, the EST yields sixfeature channels for each scale, with three of which containing theintensity information. After the EST is computed, the LCV modelcan be adopted to address texture image segmentation by replacingthe original image u0 in (28) with the average of all the channelsJE�,i (i = 1, 2 . . . 9) belonging to the EST JE� in (37), i.e.:

uE� = 19

9∑i=1

JE�,i, (39)

where � is the standard deviation of Gaussian kernel.

3.8. Description of algorithm steps

Now, we can describe the steps of our LCV model as follows:Step 1: Input the original image u0. If the image is texture im-

age then compute the corresponding extended structure tensor JE�according to (37) and replace u0 with uE� in (39) (the average of ninechannels of the extended structure tensor JE�).

Step 2: Set the initial curve CI in u0. Set the value of time-step �t,the grid spacing h and � in (34). In practice, �t=0.1 and h=�=1. Setthe window size k of averaging convolution operator in local term.Set the values of the controlling parameter of global term, , the con-trolling parameter of local term, �, and length controlling parameter,�, in regularization term according to the following criterions:

(a) If images present the intensity homogeneity, then the value of is near or equal to that of �. In practice, = � = 1;

(b) If images present distinct intensity inhomogeneity, then thevalue of should be less than that of �. In practice, � = 1;

(c) If � is small, then smaller objects will be detected; if � is larger,then larger objects will be detected. Usually, � is formatted by� = o ∗ 2552, o ∈ [0, 1].

Step 3: Evolve level set function � according to (28) and itsnumerical solution scheme described in (34).

Step 4: Extract the evolving curve C from the zero level setfunction.

Step 5: Judge whether the termination criterion described inSection 3.5 is satisfied or not. If yes, the algorithm is stopped; oth-erwise, go to Step 3.

4. Experimental results

In this Section, we shall present the experimental results of ourlocal Chan–Vese (LCV) model on some synthetic and real images. Theproposed model was implemented by Matlab 7 on a computer withIntel Core 2 Duo 2.2GHz CPU, 2G RAM, and Windows XP operatingsystem. The processing time referred later in this section starts afterchoosing the initial contour. We used the same parameters of thetime-step �t = 0.1, the grid spacing h= 1, � = 1 (for H�(z) and �(z)),the window size of averaging convolution operator k = 15, Tit = 10and length = 5 (to be used in the proposed termination criterion),�=2 (the standard deviation of Gaussian kernel in extended structuretensor) for all the experiments in this section. It can be found thatwe were able to get good segmentation results on a wide range ofimages with these parameters.

The couple of controlling parameters of global term and localterm , � should be set according to the image intensity property asdescribed in Section 3.8. Generally, we fixed � = 1 and dynamicallyadjusted the value of . In our experiments, has two correspondingvalues: 0.1 and 1 for images with/without intensity inhomogeneity.The length controlling parameter � also has a scaling role like . Ifwe have to detect all or as many objects as possible and the objectsof any size, then � should be small. Otherwise, � should be larger. Inour experiments, two corresponding values of �, 0.01∗2552 and 0.1∗2552, are adopted. In conclusion, there are totally two parameterswhose values need to be dynamically adjusted in our experiments,i.e., and �.

We firstly considered the simplest case: segmentation of imageswith the intensity homogeneity. Fig. 1(b)–(d) show the segmentationprocess of a noisy synthetic image using the proposed LCV model.The robustness property of our model is due to the usage of globalterm and length penalty term. It can be seen from Fig. 1(a) thatthere are four separated shapes within the original image. To showthe high efficiency of our model, the initial contour was placed atthe bottom left corner unlike other methods in which the initialcontour was usually placed in the center of images or touches thetarget objects. Afterwards, the initial contour evolves and furthersplits into several parts according to the topological structure oftarget shapes, as shown in Fig. 1(c). Finally, the evolving curves stopon the true boundary of each shape after 40 iterations, as can be seenin Fig. 1(d). Since the parameter of evolution termination criterionTit = 10 is predefined before the evolution process, it is obvious thatthe evolving contour has already arrived at the true boundary at the30th iteration. It should be noticed that this conclusion is suitablefor all experiments using our model in this paper.

Fig. 2 shows the segmentation process of a real image of DNAchannel using the LCV model. The objective is to extract the nucleiwhich appear much brighter than the background in the DNA chan-nel, as shown in Fig. 2(a). Some nuclei are very close to each other,and traditional thresholding methods may fail to segment them. Theinitial contour was still placed at the bottom left corner, as shownin Fig. 2(b). After 4 iterations, the evolving curves successfully ap-proach and surround all the nuclei, as shown in Fig. 2(c). It can beseen from Fig. 2(d) that the majority of nuclei have been segmentedafter 10 iterations while two pairs of nuclei are still connected. Theevolving curves continue to split and all nuclei are successfully seg-mented at the 25th iteration (accomplished at the 15th iteration dueto Tit = 10, as shown in Fig. 2(e)). Here, � is set to 0.01× 2552 sincethe objective is to segment those small nuclei.

In the next experiments (Figs. 3–5), we shall illustrate the abilityof the proposed LCV model to segmenting images with the intensity


Original Image Initial Contour 20 iterations Final Contour, 40 iterations

Fig. 1. Noisy synthetic image segmentation using the proposed LCV model: (a) original noisy image; (b) initial contour; (c) intermediate segmentation result at the20th iteration; and (d) final segmentation result at the 40th iteration. Size = 114×112, = � = 1, � = 0.1 × 2552, and processing time = 3.1 s.

Original Image

Initial Contour 4 iterations

10 iterations 15 iterations Final Contour, 25 iterations

Fig. 2. Segmentation of a real image of DNA channel using the LCV model: (a) original image; (b) initial contour; (c) intermediate segmentation result at the 4th iteration;(d) intermediate segmentation result at the 10th iteration; (e) intermediate segmentation result at the 15th iteration; and (f) final segmentation result at the 25th iteration.Size = 217×160, = � = 1, � = 0.01 × 2552, and processing time = 5.2 s.

Original Image Initial ContourFinal Contour,100 iterations

Final Contour,19 iterations

Fig. 3. The comparisons of the CV model and the proposed LCV model on segmenting a synthetic image with the intensity inhomogeneity: (a) original image; (b) initialcontour; (c) final segmentation result using the CV model; and (d) final segmentation result using the proposed LCV model. Size = 88×85, = 0.1 (LCV), � = 1 (LCV),� = 0.01 × 2552(CV/LCV), and processing time = 5.3 s (CV), 1.1 s (LCV).


Original Image Initial Contour Final Contour, 50 iterations Final Contour, 40 iterations



Fig. 4. The comparisons of the CV model and the proposed LCV model on segmenting three real blood vessel images with the intensity inhomogeneity. The first column:original images. The second column: initial contours. The third column: final segmentation results using the CV model. The fourth column: final segmentation results usingthe proposed LCV model. Size = 150×147, 103×131, 110×110, = 0.1 (LCV), � = 1 (LCV), and � = 0.01 × 2552 (CV/LCV).

inhomogeneity. Fig. 3 shows the segmentation results for the well-known synthetic image using both the CV model and the proposedLCV model. It can be seen from Fig. 3(a) that the intensity decreasesgradually from the left to the right. The initial contour was placedat the intersectional region of high intensity area and low intensityarea, as shown in Fig. 3(b). The failing segmentation of this syntheticimage using the CV model is illustrated in Fig. 3(c), which showsthat the evolving curve of the CV model cannot pass through theintersectional region of high intensity area and low intensity areaeven after 100 iterations. Fig. 3(d) shows the segmentation resultof the LCV model where the evolving curve of the LCV model cansuccessfully stop on the boundary of object after 19 iterations. Here,the value of the controlling parameter =0.1 is less than that of �=1to maintain a restriction effect on the intensity inhomogeneity.

Fig. 4 shows the segmentation results for three real blood vesselimages with the intensity inhomogeneity using both the CV modeland the proposed LCVmodel. To achieve a fair comparison, we addedthe penalty term described in (20) into the CV model to avoid there-initialization step. In addition, the proposed termination criterionhas also been applied to the CV model. It can be seen from the thirdcolumn of Fig. 4 that the CV model failed to segment all three imageswith the intensity inhomogeneity as we anticipated. The reason isdue to the inherent disadvantage of not using the local information.The fourth column of Fig. 4 shows that the proposed LCV model cansuccessfully segment the images in the first column. It should benoticed that the iteration times will be efficiently decreased if the

initial contours are placed on certain part of the objects, as shownin Fig. 5.

Here, we also used the recent popular local binary fitting (LBF)model [27] to segment the images in Fig. 4. As we discussed inSection 1, the LBF model usually needs to perform four convolutionoperations at each iteration and is sensitive to the selection of gov-erning parameters and the location of initial contour. We tried manytimes and selected the best governing parameters � = 0.001 × 2552

(the length controlling parameter), sigma=5/5/3.5 (the standard de-viation of Gaussian kernel for three images) and the initial contourfor the LBF model. In contrast, the parameters of the LCV model arethe same as the ones in Fig. 4. Fig. 5 illustrates the segmentation re-sults of the LBF model and the proposed LCV model with the sameinitial contours. Both models have succeeded in the segmentationtask. Their iteration numbers and processing time for segmentingimages in Fig. 5 are presented in Table 1. It can be seen from Table 1that the iteration number and processing time for the LCV model areboth less than that of the LBF model for all three image segmenta-tion. Note that the LCVmodel only needs to perform one convolutionoperation before level set evolution. Considering that the parame-ters and initial contours of the LBF model are selected elaborately,so the LCV model is proved to be more efficient in segmenting theimages with the intensity inhomogeneity.

The next three experiments (Figs. 6–8) are focused on illustrat-ing the ability of segmenting texture images of the LCV model com-bined with the extended structure tensor (EST). Fig. 6 shows the


Initial Contour Final Contour, 28 iterations Final Contour, 27 iterations



Fig. 5. The comparisons of the LBF model and the proposed LCV model on segmenting the images in Fig. 4. The first column: initial contours. The second column: finalsegmentation results using the LBF model. The third column: final segmentation results using the LCV model.

Table 1Iteration number and processing time for the LBF model and proposed LCV model in segmenting the images in Fig. 5.

The image from the first row The image from the second row The image from the third row

Iteration number Processing time (s) Iteration number Processing time (s) Iteration number Processing number

LBF 28 4.9 50 6.8 59 7.6LCV 27 2.7 29 2.4 25 2.9

segmentation process of three kinds of synthetic texture images.The first kind of image contains one texture object (the first row), thesecond kind of image contains three objects with the same texture(the second row), and the third kind of image contains two objectswith different texture (the third row). According to the algorithmsteps described in Section 3.8, the average of nine channels of theextended structure tensor (EST) is utilized for level set evolution in-stead of original image. Since the texture objects are usually locatedin the center parts of original images, the center parts of the com-puted averages are always in high-frequency area. Correspondingly,the boundary parts of the computed averages are in low-frequencyarea. Thus, the boundary parts can be firstly captured by the ini-tial contours at the image corner. The second column shows the in-termediate segmentation results for three texture images after one

iteration. It can be seen that the small initial contours in the first col-umn have quickly expanded and involve the majority of the images.After 4 or 5 iterations, the evolving curves successfully approach thetrue boundaries of texture objects, as shown in the third column. Itcan be seen from the fourth column that all texture objects are suc-cessfully segmented. Fig. 7 shows the segmentation result for a realgiraffe image which contains nature texture produced from the up-per body of the giraffe. It can be seen from Fig. 7(c) that the desiredupper body of giraffe was successfully segmented.

Fig. 8 shows the segmentation results for a texture image withthe intensity inhomogeneity using the LCV model combined withthe classical structure tensor and the EST. First, the classical struc-ture tensor was incorporated into the LCV model to segment thistexture image. It can be seen from Fig. 8(c) that the LCV model


Initial Contour 1 iterations 4 iterations Final Contour, 15 iterations



Fig. 6. Segmentation of three synthetic texture images using the LCV model combined with the EST. The first column: initial contours. The second column: intermediatesegmentation result. The third column: intermediate segmentation result. The fourth column: final segmentation results. Size = 140×130, 128×128, 128×128, = � = 1, and� = 0.01 × 2552.

Original Image Initial Contour Final Contour, 29 iterations

Fig. 7. Segmentation of a real giraffe image using the LCV model combined with the EST: (a) original image; (b) initial contour; and (c) final segmentation result.Size = 279×205, = � = 1, and � = 0.01 × 2552.

combined with the classical structure tensor failed to segment theimage. The reason is due to the fact that the intensity inhomogene-ity effect can be regarded as a smooth spatial varying function overthe original image and the classical structure tensor does not containany original intensity information. Since the intensity information isincorporated into the EST, the segmentation process would not be

influenced by the intensity inhomogeneity any further. The segmen-tation result illustrated in Fig. 8(d) has also proved the efficiency ofthe EST.

As we discussed in Section 3.3, re-initialization can be regardedas a numerical remedy for maintaining stable curve evolution inthe level set methods. However, it is usually quite complicated and



Fig. 8. Segmentation of a synthetic texture image with the intensity inhomogeneity using the LCV model combined with the classical structure tensor and the EST:(a) original image; (b) initial contour; (c) final segmentation result using the LCV model combined with the classical structure tensor; and (d) final segmentation resultusing the LCV model combined with the EST. Size = 256×256, = 0.1, � = 1, and � = 0.01 × 2552.

Table 2The performance comparisons of proposed LCV model using the fast marching method and the penalty term in segmenting the images in Figs. 2 and 3.

Fig. 2(a), size = 217×160 Fig. 3(a), size = 88×85

Average computationaltime (s)

Iterationnumber

Total processingtime (s)

Average computationaltime (s)

Iterationnumber

Total processingtime (s)

Fast marching method 0.0938 80 23.1 0.0351 27 2.4Penalty term 0.0313 25 5.2 0.0156 19 1.1

time-consuming. Therefore, the fast marching method [42] was pro-posed for finding a solution to these problems. In this paper, weadded a penalty term into the regularization term to completelyavoid the re-initialization step. To further show the advantage ofthis penalty term, the fast marching method was also used and com-pared with our non-re-initialization solution scheme. Accordingly,the penalty term was removed from the energy functional and thefast marching method was adopted to re-initialize level set func-tion to a signed distance function at every iteration. Table 2 showsthe performance comparisons of proposed LCV model using the fastmarching method and the penalty term in segmenting the images inFigs. 2 and 3. Both images were successfully segmented by the LCVmodel using two solution schemes. It can be seen from Table 2 thatthe average computational time of fast marching method is greaterthan that of penalty term. In addition, the iteration number for theLCV model using fast marching method are greatly increased, espe-cially in segmenting the larger image in Fig. 2. As a result, the totalprocessing time for the LCV model using fast marching method isobviously greater than that for the LCV model using penalty term. Itcan be concluded that the penalty term is a good substitute for fastmarching method while solving the re-initialization problem in theproposed LCV model.

Here, we also provided more experiments on four different typesof real images to further demonstrate the performance of the pro-posed LCV model. It should be noted that all the initial contourswere placed in the background area, as shown in the second columnof Fig. 9. The first row of Fig. 9 shows the segmentation of a wristX-ray image with obvious intensity inhomogeneity. The true bound-ary of the wrist is covered by certain blurry light, which will resultin wrong segmentation result if only the global information is used.The successful segmentation result using the proposed LCV model isillustrated in the third image of the first row of Fig. 9. The secondrow shows the segmentation of a real starfish image with naturetexture. Thus, the LCV model combined with the EST was adoptedand the successful segmentation result is shown in the third imageof the second row of Fig. 9. The third row shows the segmentation

of a real garden image in which segmenting the fire hydrant is ourobjective. Obviously, the segmentation process will be influenced bythe existences of wall, gate and grass if the traditional solutions areused. Although this image does not contain obvious texture, the ESTcontaining intensity information can still be adopted for eliminatingthe influences of wall, gate and grass. The segmentation result in thethird image of the third row shows that the LCV model combinedwith the EST successfully segmented the fire hydrant while the wall,the gate and the grass were not included. As mentioned above, theLCV model combined with the EST can also segment the texture im-age with the intensity inhomogeneity, as can be seen from the syn-thetic texture image segmentation in Fig. 8. Next, we shall providea real case to further validate this conclusion. The fourth row showsthe segmentation of a real corpus callosum MR image with intensityinhomogeneity. The target is to segment the blurry corpus callosumregion which is surrounded by the dim brain tissues. To eliminate theinfluences caused by both surrounding brain tissue and the intensityinhomogeneity effect, the EST was incorporated into LCV model forsegmentation with the controlling parameters = 0.1 and � = 1. Itcan be seen from the third image of the fourth row that the blurrycorpus callosum region was successfully segmented.

Remark 2. In general, the Courant–Friedrichs–Levy (CFL) rule willrestrict the allowable step size of the partial differential equation(PDE) when the explicit numerical scheme in Section 3.6 is applied.That is, for any given grid spacing h and the velocity F, we requireF · �t�h, where �t is the time-step. Thus, the CFL condition placesan upper bound on the time-step and hence on the speed of curveevolution. In our experiments, we used the same evolving parame-ters of the time-step �t=0.1 and the grid spacing h=1. It seems thatthe proposed LCV model can sometimes segment the target objectsin very few iterations, especially in Figs. 2 and 6. As we discussedin Section 3.2, the utilization of local term can significantly increasethe contrast between foreground intensities and background in-tensities. Thus, the ability of evolving curve to approach the trueboundary has also been greatly improved while compared with






Fig. 9. Segmentation of four different types of real images using the LCV model. The first column: original images. The second column: initial contours. The third column:final segmentation results using the proposed LCV model with different schemes. Size = 164×323, 300×294, 255×230, 238×159, = 0.1, � = 1 (the first and the fourth row), = � = 1 (the second and the third row), and � = 0.01 × 2552.

traditional level set methods. Moreover, a larger window size of con-volution operator k= 15 was adopted which also implicitly updatesthe grid spacing in the numerical solutions. As a result, the smallinitial contour can quickly expand and approach the true boundaryeven starting from the image corner, as can be seen from Fig. 2(c).For texture image segmentation, the average of nine channels of theextended structure tensor (EST) is utilized for level set evolution in-stead of original image. Since most texture objects are located in thecenter part of the original image, the center part of the computed

average is in high-frequency area. Correspondingly, the boundarypart of the computed average is in low-frequency area which canbe firstly captured by the initial contour at the image corner. Hence,the evolving curves will expand to surround the center part of imageand quickly approach the true boundaries of texture objects underthe dual effect of the EST and the local term. The second to thefourth columns of Fig. 6 properly demonstrate this process. By incor-porating the local term and the EST, our solution scheme adds someflexibility to the CFL constraint, which makes the curve evolution


maintain stable and converge in fewer iterations while segmentingsome ordinary images and texture images. However, it should be em-phasized that the CFL rule still takes effect in our numerical scheme.It can be seen from the third and the fourth rows of Fig. 9 that theiteration times for two segmentations are relatively large due to thecomplicated background.

5. Conclusions and future works

In this paper, we propose a new local Chan–Vese (LCV) modelfor image segmentation, which is based on the techniques of curveevolution, local statistical function and level set theory. The energyfunctional for the proposed model consists of global term, local termand regularization term. By incorporating the local image informa-tion into our model, the images with intensity inhomogeneity can beefficiently segmented. To avoid the time-consuming re-initializationstep, a new penalizing energy is introduced into the regularizationterm. Moreover, a termination criterion based on the length of theevolving curve is proposed to ensure that the evolving curve canautomatically stop on the true boundaries of objects. Particularly, byadding the intensity information into the classical structure tensor,an extended structure tensor (EST) is constructed for texture imagesegmentation. Combining the EST with the proposed LCV model,the texture image can be efficiently segmented no matter whetherit presents intensity inhomogeneity or not. Finally, experimentson some synthetic and real images have demonstrated the desiredsegmentation performance of our proposed model for the (texture)images with or without intensity inhomogeneity. The comparisonswith the CV model and the LBF model also show that the proposedLCV model has a much faster convergence speed and less sensitivityto the location of initial contour and the selection of governingparameters.

It should be noted that the proposed LCVmodel is efficient for thetwo-modal (phase) images, which usually generate two segments,i.e., foreground and background. As a bimodal model, it cannot si-multaneously detect multiple objects in different intensities and thetriple junctions. In our future work, we will extend the current LCVmodel by using the multi-layer idea (transformation of single phaseto layer) or multiple level set functions. Further extension can bemade to obtain amultimodal LCVmodel, so that multi-modal (phase)images and regions with triple junctions can be segmented.

Acknowledgements

This work was supported by the grants of the National Sci-ence Foundation of China, Nos. 60873012, 60805021, 60705007& 30700161, the grant from the National Basic Research Programof China (973 Program), No.2007CB311002, the grants from theNational High Technology Research and Development Program ofChina (863 Program), Nos. 2007AA01Z167, the grant of the GuideProject of Innovative Base of Chinese Academy of Sciences (CAS),No.KSCX1-YW-R-30, the grant of the Graduate Students' ScientificInnovative Project Foundation of CAS (X.-F. Wang).

Supplementary material

Supplementary data associated with this article can be found inthe online version at doi:10.1016/j.patcog.2009.08.002.

References

[1] G.S. Linda, C.S. George, Computer Vision, Prentice-Hall, New Jersey, 2001.[2] M. Kass, A. Witkin, D. Terzopoulos, Snakes: active contour models, Int. J. Comput.

Vision 1 (4) (1987) 321–331.[3] S. Osher, J.A. Sethian, Fronts propagating with curvature-dependent speed:

algorithms based on Hamilton–Jacobi formulations, J. Comput. Phys. 79 (1)(1988) 12–49.

[4] A. Gelas, O. Bernard, D. Friboulet, R. Prost, Compactly supported radialbasis functions based collocation method for level-set evolution in imagesegmentation, IEEE Trans. Image Process. 16 (7) (2007) 1873–1887.

[5] Y.H. Tsai, S. Osher, Total variation and level set based methods in image science,Acta Numer. (2005) 1–61.

[6] V. Caselles, F. Catte, T. Coll, F. Dibos, A geometric model for active contours inimage processing, Numer. Math. 66 (1) (1993) 1–31.

[7] V. Caselles, R. Kimmel, G. Sapiro, Geodesic active contours, Int. J. Comput. Vision22 (1) (1997) 61–79.

[8] R. Malladi, J.A. Sethian, B.C. Vemuri, Shape modeling with front propagation:a level set approach, IEEE Trans. Pattern Anal. Mach. Intell. 17 (2) (1995)158–175.

[9] D. Mumford, J. Shah, Optimal approximation by piecewise smooth functionsand associated variational problems, Commun. Pure Appl. Math. 42 (1989)577–685.

[10] T.F. Chan, L.A. Vese, Active contours without edges, IEEE Trans. Image Process.10 (2) (2001) 266–277.

[11] A. Tsai, A. Yezzi, A.S. Willsky, Curve evolution implementation of theMumford–Shah functional for image segmentation, denoising, interpolation,and magnification, IEEE Trans. Image Process. 10 (8) (2001) 1169–1186.

[12] N. Paragios, R. Deriche, Geodesic active regions and level set methods forsupervised texture segmentation, Int. J. Comput. Vision 46 (3) (2002) 223–247.

[13] S. Gao, T.D. Bui, Image segmentation and selective smoothing by usingMumford–Shah model, IEEE Trans. Image Process. 14 (10) (2005) 1537–1549.

[14] L.A. Vese, T.F. Chan, A multiphase level set framework for image segmentationusing the Mumford and Shah model, Int. J. Comput. Vision 50 (3) (2002)271–293.

[15] J.E. Solem, N.C. Overgaard, A. Heyden, Initialization techniques for segmentationwith the Chan–Vese model, in: Proceedings of the 18th International Conferenceon Pattern Recognition (ICPR'06), vol. 2, 2006, pp. 171–174.

[16] R.B. Xia, W.J. Liu, J.B. Zhao, L. Li, An optimal initialization technique forimproving the segmentation performance of Chan–Vese model, in: Proceedingsof the IEEE International Conference on Automation and Logistics, 2007,pp. 411–415.

[17] Y. Shi, W. Karl, A fast level set method without solving PDES, in: Proceedings ofthe IEEE International Conference on Acoustics, Speech, and Signal Processing(ICASSP'05), vol. 2, 2005, pp. 97–100.

[18] Y. Pan, J.D. Birdwell, S.M. Djouadi, Efficient implementation of the Chan–Vesemodels without solving PDEs, in: Proceedings of the International WorkshopOn Multimedia Signal Processing, 2006, pp. 350–354.

[19] C.M. Li, C.Y. Xu, C.F. Gui, M.D. Fox, Level set formulation without Re-initialization: a new variational formulation, in: Proceedings of the IEEEInternational Conference on Computer Vision and Pattern Recognition (CVPR)2005, vol. 1, 2005, pp. 430–436.

[20] L. Pi, C.M. Shen, F. Li, J.S. Fan, A variational formulation for segmenting desiredobjects in color images, Image Vision Comput. 25 (9) (2007) 1414–1421.

[21] K. Sum, P. Cheung, Vessel extraction under non-uniform illumination: a levelset approach, IEEE Trans. Biomed. Eng. 55 (1) (2008) 358–360.

[22] T. Brox, D. Cremers, On the statistical interpretation of the piecewise smoothMumford–Shah functional, Proc. Scale Space Var. Met. Comput. Vision 4485(2007) 203–213.

[23] J. Piovano, M. Rousson, T. Papadopoulo, Efficient segmentation of piecewisesmooth images, Proc. Scale Space Var. Met. Comput. Vision 4485 (2007)709–720.

[24] J. An, M. Rousson, C. Xu, �-Convergence approximation to piecewise smoothmedical image segmentation, Proc. Med. Imag. Comput. Comp. Assist. Interven.4792 (2007) 495–502.

[25] S. Lankton, D. Nain, A. Yezzi, . Tannenbaum, Hybrid geodesic region-basedcurve evolutions for image segmentation, in: Proceedings of the SPIE: MedicalImagining, vol. 6510, 2007, pp. 65104U.

[26] S. Lankton, A. Tannenbaum, Localizing region-based active contours, IEEE Trans.Image Process. 17 (11) (2008) 2029–2039.

[27] C.M. Li, C.Y. Kao, J.C. Gore, Z.H. Ding, Implicit active contours driven by localbinary fitting energy, in: Proceedings of the CVPR'07, 2007, pp. 1–7.

[28] G. Aubert, M. Barlaud, O. Faugeras, S. Jehan-Besson, Image segmentation usingactive contours: calculus of variations of shape gradients, Research Report,INRIA, July 2002.

[29] B. Sandberg, T. Chan, L. Vese, A level-set and gabor-based active contouralgorithm for segmenting textured images, Technical Report 39, MathematicalDepartment, UCLA, Los Angeles, USA, July 2002.

[30] M. Rousson, T. Brox, R. Deriche, Active unsupervised texture segmentation ona diffusion based feature space, in: Proceedings of the 2003 IEEE ComputerSociety Conference on Computer Vision and Pattern Recognition (CVPR'03),vol. 2, 2003, pp. 699-7-4.

[31] J. Bigun, G.H. Grandlund, Optimal orientation detection of linear symmetry,in: Proceedings of the IEEE First International Conference on Computer Vision(ICCV), 1987, pp. 433–438.

[32] J. Bigun, G.H. Grandlund, J. Wiklund, Multidimensional orientation estimationwith applications to texture analysis and optical flow, IEEE Trans. Pattern Anal.Mach. Intell. 13 (8) (1991) 775–790.

[33] C. Feddern, J. Weickert, B. Burgeth, Level-set methods for tensor-valued images,in: Proceedings of the Second IEEE Workshop on Variational, Geometric andLevel Set Methods in Computer Vision, 2003, pp. 65–72.

[34] Z. Wang, B. Vemuri, Tensor field segmentation using region based activecontour model, in: Proceedings of the Eighth European Conference on ComputerVision (ECCV'04), Springer Lecture Notes in Computer Science, vol. 3024, 2004,pp. 304–315.

doi:10.1016/j.patcog.2009.08.002


[35] S.M. Lee, A.L. Abott, N.A. Clark, P.A. Araman, Active contours on statisticalmanifolds and texture segmentation, in: Proceedings of the IEEE InternationalConference on Image Processing (ICIP), vol. 3, 2005, pp. 828–831.

[36] D.E. Maroulis, M.A. Savelonas, D.K. Iakovidis, S.A. Karkanis, N. Dimitropoulos,Variable background active contour model for computer-aided delineation ofnodules in thyroid ultrasound images, IEEE Trans. Inf. Technol. Biomed. 11 (5)(2007) 537–543.

[37] U. Vovk, F. Pernuš, B. Likar, A review of methods for correction of intensityinhomogeneity in MRI, IEEE Trans. Med. Imaging 26 (3) (2007) 405–421.

[38] Z.J. Hou, A review on MR image intensity inhomogeneity correction, Int. J.Biomed. Imaging 2006 (2006) 1–11.

[39] M. Sussman, E. Fatemi, An efficient, interface preserving level set redistancingalgorithm and its application to interfacial incompressible fluid flow, SIAM J.Sci. Comput. 20 (1999) 1165–1191.

[40] J. Gomes, O. Faugeras, Reconciling distance functions and level sets, J. VisualCommun. Imaging Representation 1 (11) (2000) 209–222.

[41] D. Adalsteinsson, J.A. Sethian, A fast level set method for propagating interface,J. Comput. Phys. 118 (1995) 269–277.

[42] J.A. Sethian, Level Set Methods and Fast Marching Methods, second ed.,Cambridge University Press, Cambridge, 1999.

[43] T. Brox, J. Weickert, B. Burgeth, P. Mrázek, Nonlinear structure tensors, ImageVis. Comput. 24 (1) (2006) 41–55.

About the Author—XIAO-FENG WANG received the B.Sc. degree in Computer and Science Technology from Anhui University, Hefei, China, in 1999, the M.Sc. degree inPattern Recognition and Intelligent System from Institute of Intelligent Machines (IIM), Chinese Academy of Sciences, Hefei, China, in 2005. He is now in pursuit for Ph.D.degree in Pattern Recognition and Intelligent System in University of Science and Technology of China (USTC). His research interests include pattern recognition, imageprocessing and data mining.

About the Author—DE-SHUANG HUANG received the B.Sc., M.Sc. and Ph.D. degrees all in Electronic Engineering from Institute of Electronic Engineering, Hefei, China,National Defense University of Science and Technology, Changsha, China, and Xidian University, Xian, China, in 1986, 1989 and 1993, respectively. During the 1993–1997period he was a postdoctoral student, respectively, in Beijing Institute of Technology and in National Key Laboratory of Pattern Recognition, Chinese Academy of SciencesBeijing, China. In September, 2000, he joined the Institute of Intelligent Machines, Chinese Academy of Sciences as the Recipient of “Hundred Talents Program of CAS”. FromSeptember 2000 to March 2001, he worked as Research Associate in Hong Kong Polytechnic University. From April 2002 to June 2003, he worked as Research Fellow inCity University of Hong Kong. From August to September 2003, he visited the George Washington University as visiting professor, Washington DC, USA. From October toDecember 2003, he worked as Research Fellow in Hong Kong Polytechnic University. From July to December 2004, he worked as the University Fellow in Hong Kong BaptistUniversity. Dr. Huang is currently a senior member of the IEEE.

About the Author—HUAN XU received the B.Sc. degree in Central China Normal University, majored in Computer Software Science and minored in Mathematics, in 2006,the M.Sc. degree in Pattern Recognition and Intelligent System from University of Science and Technology of China (USTC), in 2009. She is now in pursuit for Ph.D. degreein the Electrical and Computer Engineering Department of Louisiana State University. Her research interests include image analysis, visual and geometric computing.

an efficient local chan–vese model for image segmentationhuanxu4/papers/wangpr10.pdflocal...

Documents