computer vision center (cvc) arxiv:1507.06504v4 [cs.cv] 22 ... · protein aggregates detected by...

July 2, 2018 Computer Methods in Biomechanics and Biomedical Engineering gCMBguide

To appear in Computer Methods in Biomechanics and Biomedical EngineeringVol. 00, No. 00, Month 20XX, 1–18

RESEARCH ARTICLE

Active skeleton for bacteria modeling

Jean-Pascal Jacoba and Mariella Dimiccolib∗ and Lionel Moisana

aUniversite Paris Descartes, MAP5 (CNRS UMR 8145), Paris, France; bComputer Vision Center (CVC)

and University of Barcelona (UB), Barcelona Perceptual Computing Lab (BCNPCL), Barcelona, Spain

(v4.0 released February 2014)

The investigation of spatio-temporal dynamics of bacterial cells and their molecular components requiresautomated image analysis tools to track cell shape properties and molecular component locations insidethe cells. In the study of bacteria aging, the molecular components of interest are protein aggregatesaccumulated near bacteria boundaries. This particular location makes very ambiguous the correspon-dence between aggregates and cells, since computing accurately bacteria boundaries in phase-contrasttime-lapse imaging is a challenging task. This paper proposes an active skeleton formulation for bac-teria modeling which provides several advantages: an easy computation of shape properties (perimeter,length, thickness, orientation), an improved boundary accuracy in noisy images, and a natural bacteria-centered coordinate system that permits the intrinsic location of molecular components inside the cell.Starting from an initial skeleton estimate, the medial axis of the bacterium is obtained by minimizingan energy function which incorporates bacteria shape constraints. Experimental results on biologicalimages and comparative evaluation of the performances validate the proposed approach for modelingcigar-shaped bacteria like Escherichia coli. The Image-J plugin of the proposed method can be foundonline at http://fluobactracker.inrialpes.fr.

Keywords: bacteria modeling; medial axis; active contours; active skeleton; shape contraints.

1. Introduction

One of the fundamental issues addressed by Computational Cell Biology is the characterization ofspatio-temporal dynamics of bacterial cells and their molecular components (Slepchenko, Schaff,et al. 2002). The rapid development of techniques for fluorescence imaging in recent years hasopened new research opportunities, that need to cope with the availability of automated imageanalysis tools to be fully exploited. In the study of E. Coli aging (Lindner, Madden, et al. 2008),the molecular components of interest are protein aggregates accumulated at the old pole region ofthe aging bacterium. These subcellular components, visualized by fluorescence imaging techniques,need to be reliably associated to their host cells, which are generally visualized with phase-contrastmicroscopy. Figure 1 shows a pair of simultaneous frames extracted from a time-lapse phase-contrastand fluorescence image sequence. The automatic association between aggregates and bacteria re-quires a very accurate estimation of bacteria boundaries in this case, because aggregates accumulatenear these boundaries (see Figure 2). In addition to boundary accuracy, a requirement for bacteriamodeling is the possibility to derive accurate cell measurements such as length, width, orientation,perimeter, etc., that are fundamental characteristics of bacteria (Osborn and Rothfield 2007). Forinstance, in bacteria like E. Coli, tracking population variability over time helps to understandthe combinations of effects of genetic and environmental factors on cell phenotype. Finally, havingthe possibility to describe the location of subcellular components in an intrinsic cell-centered co-

∗Corresponding author. Email: [email protected]

1

arX

iv:1

507.

0650

4v4

[cs

.CV

] 2

2 D

ec 2

016


ordinate system may be very useful to characterize intracellular dynamics (Coquel, Jacob, et al.2013).

Figure 1. E. coli cells visualized through phase-contrast microscopy (left), and their molecular components (in this caseproteine aggregates) visualized through fluorescence microscopy (right). The cell size is approximately between 3 and 5 µm.

Figure 2. E. coli cells visualized through phase-contrast microscopy (left) and protein aggregates visualized through a fluo-

rescence microscopy (right). For a better visualization, here it is shown a saturated version of the original fluorescence image.Protein aggregates detected by Dimiccoli, Jacob and Moisan (2015) on the original fluorescence image, are visualized as white

points on left image. As it can be seen, they are often localized near boundaries making ambiguous the assignment to the hostcell. The cell size is approximately between 3 and 5 µm.

This paper aims to provide an algorithm able to extract from an image a complete parametricdescription of cigar- or rod-shaped bacteria (e.g., E. coli) that it contains. The underlying math-ematical representation is a curvilinear skeleton (also called medial axis), which defines, after anappropriate (non-constant) dilation, the boundary of the bacterium. This model is built by evolv-ing a first skeleton estimate so as to minimize an energy that promotes a good image fitting whileenforcing the expected properties of the cell shape. In the following, we present the state of theart related to the use of skeletons for shape characterization and active contours, on which theproposed model relies.

1.1 Background

Segmentation, the task of partitioning an image into coherent regions, is ill-posed and simplecontinuity or homogeneity assumptions cannot cope with the large variability of object appearances.

2


In some specific applications for which one expects to see objects from a particular class, priorknowledge (e.g., clues on appearance or shape) may be exploited to reduce the space of variations.Such cues have traditionally been used by the Computer Vision community for object recognition,but their usefulness in segmentation has only more recently reached a common consensus (Leibe,Leonardis and Schiele 2006). Appearance-based methods rely on features points as Scale-InvariantFeature Transform (SIFT) (Lowe 2001) but their efficiency has proved to be limited since in general,object identity is more a function of form that a function of appearance (Siddiqi, Kimia, et al.2001; Siddiqi and Kimia 1996, 1995; Leordeanu, Hebert and Sukthankar 2007). Furthermore, inapplications such as living cell imaging, where the intensities are reduced to a minimum to avoidphotodamage and photobleaching, key features may not be sufficiently available.

In the use of shape as an alternative or to reinforce the use of appearance, the skeleton, definedby Blum (Blum 1973) as the set of medial loci of the maximal disks contained inside the object,has received much attention in the literature (Bai and Latecki 2008; Siddiqi, Shokoufandeh, et al.1998; Zhu and Yuille 1995). Compared to other shape representation such as edge fragments andshock graphs, the skeleton offers more flexibility in modeling spatial relationships between partsand has a good repeatability/distinctiveness trade-off. Although its usefulness for shape matchingand classification on silhouettes has been recognized since the nineties, it is only recently thatskeletons have been exploited for non-rigid object detection (Bai, Wang, et al. 2009; Trinh andKimia 2011). In all these methods, the object is first recognized using reference skeletons, and thenthe skeleton is used to optimize an initial segmentation.

The idea of optimizing an initial segmentation, under some constraints, is at the core of activecontour (also called snakes) methods, which specifically refine an initial approximate contour ac-cording to the image data. Early active contour models (Kass, Witkin and Terzopoulos 1988; Osherand Sethian 1988; Cohen 1992; Caselles, Catte, et al. 1993) act by minimizing an energy functionconsisting of a data term (including image information), and a regularization term which, in gen-eral, imposes smoothness constraints on the object shape. According to the nature of the imageinformation included in the data term, the existing active contour methods can be categorized intotwo types: edge-based models and region-based models. Edge-based models use local image infor-mation —typically, gradient information— (Kass, Witkin and Terzopoulos 1988; Staib and Duncan1992; Park, Schoepflin and Kim 2001) to stop the contour evolution on the object boundary, whileregion-based approaches (Cohen, Bardinet and Ayache 1992; Ivins and Porrill 1995; Zhu and Yuille1996; Chesnaud, Refregier and Boulet 1999; Chan and Vese 2001; Jehan-Besson, Gastaud, et al.2003; Zhang, Zhang, et al. 2010) use global image features inside and outside the contour to con-trol the evolution. Region-based methods relying on local information are able to segment imageswith non-homogeneous intensities (Li, Kao, et al. 2007, 2008). However, as detailed by Wang et al.(Wang, Li, et al. 2009), such localization property introduces many local minima of the nonconvexenergy functional. Consequently, the result is more dependent on the initialization of the contour.The initialization issue itself is since long time the object of investigations (Cohen 1991; Cohenand Cohen 1993; Kass, Witkin and Terzopoulos 1988; Bajcsy and Kovacic 1989; Jain, Zhong andDubuisson-Jolly 1998; Cohen and Kimmel 1997; Amini, Weymouth and Jain 1990; Geiger, Gupta,et al. 1995) and, recently, it has been partially addressed by the formulation of convex energyfunctionals (Mao, Liu and Shi 2010; Thieu, Luong, et al. 2011), for which a single global min-imum exists. According to the contour parametrization method used to model the smoothnessconstraint included in the regularization term, the existing active contour methods can be cate-gorized into three classes: level sets snakes (Osher and Sethian 1988; Caselles, Catte, et al. 1993;Malladi, Sethian and Vemuri 1995), point-based snakes (Kass, Witkin and Terzopoulos 1988; Xuand Prince 1998) and parametric snakes (Staib and Duncan 1992; Brigger, Hoeg and Unser 2000).Level sets define the 2D-curve implicitly from an evolving surface in a 3D space. They easily enabletopology changes but are computationally expensive. The smoothness constraints are also implicit,that is, they are defined on the surface and not on the curve. Point-based snakes correspond to noparametrization or equivalently to a polygonal line (B-splines of degree zero). Parametric snakes de-fine the curve between knot points (Brigger, Hoeg and Unser 2000; Jacob, Blu and Unser 2004) via

3


a basis decomposition such as B-splines (Marc, Menet and Medioni 1990; Brigger, Hoeg and Unser2000) or Fourier exponentials (Staib and Duncan 1992). The overall smoothness is ensured withcurvature and eventually length minimization. When a shape-prior is introduced into point-basedor parametric snakes, a deformable template model is obtained (Jain, Zhong and Dubuisson-Jolly1998). The prior shape model may be built ad hoc with analytical formulas (Widrow 1973; Yuille,Hallinan and Cohen 1992; Jolly, Lakshmanan and Jain 1996; Lakshmanan and Grimmer 1996) orderived from a training set (Staib and Duncan 1992; Cootes, Hill, et al. 1993; Cootes, Taylor, et al.1995; Cootes and Taylor 1995; Davatzikos, Tao and Shen 2003). For analytic models, the shapedeformations are given by the model parameters (e.g., slope for straight lines, radius for circles).Mostly no probability distribution is needed. Training methods, usually called active shape mod-els (Staib and Duncan 1992; Cootes and Taylor 1995; Davatzikos, Tao and Shen 2003), are moreflexible since they can be applied easily to any reproducible shape. However, they require to definean average shape and deformation models, which could bias the result if, for example, the trainingset is too small.

1.2 Active skeleton

Instead of matching a given bacteria template to a previously computed edge map as in classicalrecognition problems, or evolving an initial contour toward the object outline as in classical active-contour methods, we propose an active skeleton model which evolves initial skeletal polygonallines toward the true medial axes of the bacteria. The advantage of the proposed approach forsegmentation is that it allows to introduce strong shape constraints adapted to the bacteria class(here, cigar-shaped bacteria), which improves the accuracy of the boundary location even in verynoised images.

The paper is organized as follows. In Section 2, we describe the skeletal model we are consid-ering, while Section 2.4 details the active skeleton initiation and the evolution process based on avariational (energy minimization) formulation. Section 3 provides results and comparisons to othersegmentation methods.

2. Method

2.1 Skeleton model

We model a digital image acquired by a contrast-phase microscopy as a real valued discrete functionu : Ω ⊂ Z2 → R, where Ω = [0, N ]× [0,M ] is a rectangle.

Definition 1. A ball B(x, r) centered at x and having radius r is a maximal disk of a shape F ifB(x, r) ⊂ F and ∀B(x′, r′) ⊂ F , B(x′, r′) 6= B(x, r)⇒ B(x, r) 6⊂ B(x′, r′)

We denote the maximal disks of a shape by the symbol Bm(x, r).

Definition 2. The morphological skeleton S ∈ Ω of a bacterium B on the image u is the set ofmaximal disks centers of B : S = x|∃r > 0 : B(x, r) ≡ Bm(x, r).

We represent a skeleton S by an ordered set of n points with associated radius value: S =(~xi, ri)i∈1..n, where ~xi = (xi, yi) ∈ Ω represents the center of the maximal disk of radius ri.The skeleton is hence composed of n − 1 segments sj = [ ~xj , ~xj+1], with j ∈ [1, ..., n − 1]. Giventhe skeleton S, the bacterium is built up by dilating the centers of the maximal disks accordingto their associated radius, which is linearly interpolated inside each segment. Let sj = [ ~xj , ~xj+1] asegment of the skeleton, then for each point ~xk of sj , the corresponding radius is computed as alinear interpolation of rj and rj+1: rk(λ) = (1− λ)rj + λrj+1, with 0 ≤ λ ≤ 1.

Given an initial skeleton, the optimization of its location strongly depends on the underlyingsegment dilations and in turn to the definition of distance used for the dilation. In the following,

4


we define two distance models and, according to them, we derive the expression of the dilationmodels.

2.2 Distance models

In this section, we first define a simplified model of distance of a point to a segment, say de, andthen we define an accurate orientation based distance, say do, that takes the radii difference intoaccount.

Definition 3. Simplified distance of a point to a segment. Let ~y = (x, y) ∈ Ω be any pointof the image and si = [~xi, ~xi+1] ∈ S any segment of the skeleton. Denoting by ~p1 the projection of~y on the line through ~xi and ~xi+1, ~p1 can be written as ~p1 = (1−λ1)~xi +λ1~xi+1, with λ1 ∈ R. Thesimplified Euclidean-based distance of a point ~y to a segment si is defined as follows.

de(~y, si) =

||~y − ~xi|| λ1 < 0||~y − ~xi+1|| λ1 > 1||~y − ~p1|| λ1 ∈]0, 1[

(2.1)

where || · || is the Euclidean norm. In Fig.3 (up) it can be observed that the distance coincideswith the Euclidean distance between two points only if the orthogonal projection of ~y on the linethrough ~xi and ~xi+1 lies on the segment si.

The value of λ1 can be computed from the scalar product < ~y − ~xi, ~xi+1 − ~xi >: λ1 =(x−xi)∆xi+(y−yi)∆yi

L2i

, where ∆xi = xi+1 − xi, ∆yi = yi+1 − yi, and Li =√

∆xi2 + ∆yi

2 (the

length of segment si). If the orthogonal projection ~p1 of ~y on si lies on si, then the distanceof ~y to si coincides to the distance of ~y to ~p1. In this case, such a distance can be written asd(y, si) =

√(x− xi − λ1∆xi)2 + (y − yi − λ1∆yi)2. By introducing λe:

λe =

0 λ1 ≤ 01 λ1 ≥ 1

λ1 = (x−xi)∆xi+(y−yi)∆yiL2i

0 < λ1 < 1(2.2)

the simplified distance from any point to a segment can be written as: de(~x, si) =√(x− xi − λe∆xi)2 + (y − yi − λe∆yi)2.

Definition 4. Orientation-based distance of a point to a segment.Let ~y ∈ Ω be any point of the image and si = [~xi, ~xi+1] ∈ S any segment of the skeleton. Let

∆ri = |ri+1−ri| be the difference between the radii value associated to the extremities of the segmentsi and Li the segment length.

• Case 1: Li > ∆ri: let ti be the common tangent to the circles with centers (~xi, ~xi+1) andradii (ri, ri+1) respectively, lying on the same half-plane than ~y compared to ti. Let y be theorthogonal projection of ~y over ti, and ~p2 the intersection of the line through ~xi and ~xi+1

with the line through ~y and y. Writing ~p2 as ~p2 = (1 − λ2)~xi + λ2~xi+1, with λ2 ∈ R, theorientation-based distance do is defined as follows: (see figure 3 (down)):

do(~y, si) =

||~y − ~xi|| λ2 < 0||~y − ~xi+1|| λ2 > 1||~y − ~p2|| λ2 ∈]0, 1[

where || · || is the euclidean norm.

5


• Case 2: Li ≤ ∆ri:

do(~y, si) =

||~y − ~xi|| ri > ri+1

||~y − ~xi+1|| ri < ri+1

Under the assumption Li > ∆ri, ~p2 = (1 − λ2)~xi + λ2~xi+1 with λ2 ∈ R. Let λ such that~p2−~p1 = λ(~xi+1−~xi). Then one has simply to replace λ1 with λ2 = λ1 +λ in the equations derivedfor the simplified distance.

Figure 3. Euclidean distance de(y, si) (up) and orientation-based distance do(y, si) (down) of a point y to a skeletal segment

si.

To explicit λ, one can compute the angle θ between ti and si according to sin(θ) = ∆riLi

. It comesthat

λ =de

Li

√L2i −∆r2

i

∆ri. (2.3)

Note again that λ is an algebraic value. To deal with the case Li ≤ ∆ri one can extend the definitionof λ, for example in the following manner:

λ =

1− λ1 ∆ri ≥ Li,−λ1 −∆ri ≥ Li,λ = de

Li√L2i−∆r2i

∆ri otherwise.(2.4)

Now one can compute in any case λ2 = λ1 + λ. Because of the restrictions of λ2, the definition canbe extended as:

λo =

0 λ2 ≤ 0,1 λ2 ≥ 1,

λ2 = λ1 + λ λ2 ∈]0, 1[,(2.5)

6


Finally, the general expression of do is as follows:

do(~x, si) =√

(x− xi − λo∆xi)2 + (y − yi − λo∆yi)2. (2.6)

2.3 Dilation models

Before introducing the dilations models derived from the above definitions of distance, we definethe distance of a point to as skeleton.

Definition 5. Distance of a point to a skeleton. Let S = (~xi, ri)i=1,...,n ∈ Ω a skeleton onthe image u : Ω→ R and y a point of Ω.

d(~y,S) = mini=1,...,n−1

d(~y, si)

Definition 6. Dilation model derived from the simplified distance of a point to a seg-ment. Let S = (~xi, ri)i∈1..n a skeleton and sj = [~xj , ~xj+1] a segment of S. For any point ~y ∈ Ω,let si = argmin

j∈[1,...,n]de(~y, sj) the segment that has minimal simplified distance from ~y. The simplified

dilation of S is given by the set of points such that their simplified distance to si is inside themaximal disks with interpolated radius: D(S) = ~x|de(~x, si) ≤ (1 − λe)ri + λeri+1 where λe isdefined in equation 2.2

Definition 7. Dilation model derived from orientation-based distance of a point to asegment. Let S = (~xi, ri)i∈1..n a skeleton and sj = [~xj , ~xj+1] the j− th segment of S. For anypoint ~y ∈ Ω, let si = argmin

j∈[1,...,n]do(~y, sj). The orientation-based dilation of S is:

D(S) = ~y|do(~y, si) ≤ (1− λo)ri + λori+1 where λo is defined in equation 2.5.

Figure 4. The dilation Do obtained by using the orientation-based distance do is represented by a full red line. The dilation

De obtained by using the simplified distance de differs by Do only in the region delimited by the two dashed vertical lines,where it is represented by a blue line. However, the difference between the two dilations is very small also for large difference

radii.

Denoting by Do and De the dilations obtained by using the distances do and de respectively, as it

can be observed on the cartoon example shown in Fig. 4: do(~y,S) = de(~y,S)√

1− (∆riLi

)2 − ri(λe)in the region delimited by the vertical dashed green lines, and Do ≈ De elsewhere. As it canbe appreciated, the dilated models differ slightly even if the difference between the radii is large.Since bacteria width has low variations, radii difference values are generally small compared tothe segment length. Hence, the difference between the dilation models would be not significant.

7


Additionally, when using the simplified distance, the corresponding dilation contains less derivativeterms and discontinuities, and consequently the variational optimization based on it would becomputationally faster and more stable numerically. Hereafter, to simplify the notations, we willuse the symbol d to denote either de or do, and r to denote either ri(λe) or ri(λo) according to thedesired distance model.

2.3.0.1 General remarks. Although the distances de and do are continuous, both dilation outlinesare made up of arc circles and straight lines. Note that the scale parameter of the outline is containedin the radius values: big or small outlines may be described with the same skeleton. Furthermore,some measures of interest inherent to the bacteria class are immediate with this representation:orientation, thickness, perimeter and length.

2.4 Active skeletons

2.4.1 Skeleton initialization

The optimization process relies on the availability of an initial skeleton for each bacterium. Inthis work, we derived it from the closed contours obtained by applying the bacteria segmentationmethod proposed in Primet, Demarez, et al. (2008). After computing the morphological skeletonthrough the use of morphological operators, a linear vectorization is performed by using an iter-ative method inspired to Wall and Danielsson (1984); Potier and Vercken (1994) but based onangle variations. The vectorization starts from the farthest point from the gravity center of theskeleton. To take into account the presence of possible holes, a spline-interpolation is used to builda continuous skeletal line, which is finally sampled uniformly according to the wished accuracy ofthe skeleton.

2.4.2 Skeleton optimization

Given an image u and the initial skeleton approximations S0i , i = 1, ..., n, the aim of the active

skeleton model is to find the skeletons Si, i = 1, ..., n corresponding to the true medial axisof bacteria by evolving the initial skeletons. This is achieved by minimizing an energy functionmade up of two terms: a data-fidelity term and a regularity term that incorporates bacteria shapeconstraints. To handle the specific issue of several non-ovelapping objects, we propose an additionalrepulsion energy term.

Figure 5. Pixel selection function used to express the spatial condition. For pixels inside the closed contours the signeddifference r(x,S)− d(x,S) has an absolute value less than 1, more than 1 otherwise. Therefore, the function being symmetric,

we have that: fps(d− r) + fps(r − d) = 1.

8


2.4.2.1 Data term. Images captured by a phase-contrast microscopy are characterized by a dif-ference of contrast between the foreground specimen and the background. Therefore, the data termhas to be small when the difference of the contrast between foreground and background is large.Since time-lapse imaging objects contours may have large and slow variations (that is, strong low-frequency components), the use of a local neighborhood allows us to have a reasonable local modelof the background. Typically, the data term corresponds to the difference between the average in-tensity inside the closed contour, and the average intensity in a ring surrounding it that correspondsto a high contrast between a bacterium and its local background. Since the objective function tobe minimized has to be differentiable, the spatial condition is expressed through a smooth pixelselection function fps.

fps(t) =

1+sin(π

2t)

2 t ∈ [−1, 1]0 t < −11 t > 1

(2.7)

The data term Ed is expressed as the difference between the internal energy Ein and the externalenergy Eout, which are defined as follows

Ein =∑x∈I

fps[r(x,S)− d(x,S)]fc(x)

fps[r(x,S)− d(x,S)]

Eout =∑

x∈I|d(x,S)<r(x,S)+ρ

fps[d(x,S)− r(x,S)]fc(x)

fps[d(x,S)− r(x,S)]

(2.8)

where fc is an increasing smooth contrast function allowing to adapt the imaging contrast to thedata, ρ is the width of the ring surrounding the closed contours and, since x ∈ si ⊂ S can be writtenas x = (1 − λ)xi + λxi+1, r(x,S) = (1 − λ)ri + λri+1. It is easy to see that fps[r(x,S) − d(x,S)]selects pixels inside the closed contour, whereas fps[d(x,S)− r(x,S)] selects outside pixels.

2.4.2.2 Regularity terms. Two regularity assumptions are embedded into the energy function byimposing a smooth segment angle variation and radii homogeneity.

Most of the bacteria are weakly bent, therefore the skeleton curvature has to be as small as pos-sible. The discrete curvature at point xi is measured by the angle of the arc at xi, that corresponds

to the angle αi = ~xi − ~xi−1, ~xi+1 − ~xi. Numerically, we compute sin(αi) = det(~xi−~xi−1,~xi+1−~xi)‖~xi−~xi−1‖‖~xi+1−~xi‖ . The

corresponding curvature-related regularization term is:

Ec =i=n−1∑i=2

sin2(αi) (2.9)

Since the bacterium thickness is quite homogeneous, the difference between the radii of theskeleton points, ri, i = 1, ..., n, should be small. We consider the median radius value defined asrmed = median

i∈[1..n](ri). The corresponding energy term is

Eh =

i=n∑i=1

(ri − rmed)2 (2.10)

2.4.2.3 Repulsion term. An additional repulsion term may be added when dealing with bacteriacolonies. Denoting by d(xi,Sk) the distance between a point xi ∈ Sl and the skeleton Sk, thecorresponding distance between xi and the bacteria is t = d(xi,Sk) − ri − rki , where rki is the

9


interpolated radius from Sk when computing the distance d(xi,Sk). Since t < 0 has to be avoided,by defining a repulsion function frep(t) minimal when t > 0, the repulsion energy term Er can bewritten as:

Er =∑k

∑xi∈Sl 6=Sk

frep(d(xi, Sl)− ri − rki ) (2.11)

2.4.2.4 Overall energy. The global energy to minimize combines all above energy terms: E =aEd + bEc + cEh + dEr, where a, b, c, d are positive weights. The energy minimization requiresto compute derivatives according to points coordinates and radii. All derivatives are given inAppendix.

3. Results

The parameters defined in last section - a, b, c, d, ρ, the contrast function fc and the repulsionfuncion frep - were set once for all when correctly calibrated on our dataset. The experimentalvalues were: a = 10, b = 1 , c = 0.01, d = 0.1, thickness ρ = 2 pixels. The contrast function weused is fc(x) = sin0.8 I(x), which enhances the contrast between inside and outside mean values ofbacteria. We have chosen a quadratic repulsion function to penalize bigger overlaps:

frep(t) =

(t−∆)2 if t < ∆0 otherwise

(3.1)

with ∆ = 0.3 pixels. The algorithm was implemented in C with the MegaWave2 library 1.

3.1 Method validation

Figure 6. The value u(x) of the pixel x is computed as a function of the distance of the pixel x from the skeletal segment sj(dashed line).

In this section, we compare our active skeleton model to the active contour model by using theHaussdorff distance to a synthetically generated ground truth on a set of synthetically generatedimages. Since both methods require a rough initialization we drew it by hand. To generate theground truth, as well as the test data, we started by considering a bacterial colony phase-contrastimage. First, we computed the bacteria skeletons by applying the proposed method and then, byrelying on the skeletons obtained, we recovered a synthetic image by computing each pixel value

1The software in C will be made publicly available with the publication of the article

10


u(x) as

u(x) = f

(infj

d(x, sj)

rj(x)

)(3.2)

where d(x, sj) is the distance of the pixel x to the segment sj of a skeleton S having as extremitiesthe pixels xj and xj+1 with associated radii rj and rj+1 respectively (see Fig. 6). f is the Gausserror function and rj(x) is the interpolated radium computed on the projection of x on sj (x′ in

Fig. 6). The interpolated radium value is computed as: rj(x) = αrj+βrj+1

α+β with α, β > 0.

Figure 7. Synthetic image, whose noised versions constitute the test data used for validation.

As test data, we used noised versions of the so generated synthetic image (see Figure 7), obtainedby adding a Gaussian white noise with increasing standard deviations. As ground truth, we usedthe boundaries from which the synthetic image has been generated. This is justified by the factthat, on the unnoised synthetic image, the average Haussdorff distances to ground truth of bothmethods are almost the same and of less than 1/10 of pixel (see Fig. 8) for a standard deviationof noise equal to zero.

Fig. 8 shows the Haussdorff distance to the ground truth of the active skeleton and the activecontour methods. Since the active contour model assumes that the object contours are smooth, wepreviously smoothed the image by using a Gaussian kernel of standard deviation σk = 0.5 beforeapplying this method. As it is shown in Fig.9, for relatively small values of the standard deviationof the Gaussian smoothing kernel, the performances of the active contour method slightly improves,since the ”edgness” of curves is enhanced by smoothing. However, there is an upper bound to thelevel of smoothing that can be applied which is related to the scale of the image structure we arelooking for. The proposed active skeleton model allows to enforce smoothness without becomingexcessively sensitive to noise by integrating an a priori on the shape which is itself smooth (rod orcigar-shape). It is worth to remark that on real images, the image contrast across cells boundariesis not constant as it is for our data test images. This greatly deteriorates the performances of theactive contour method as it can be appreciated in Fig. 10. This Fig. shows the interest of theproposed method in determining the association cell-aggregate. As detailed in section 1, solvingthis association requires accurate boundary estimation since fluorescent molecular components areusually concentrated at the poles of bacteria.

11


Figure 8. Haussdorff distance of the active contour method (in blue) versus the Haussdorff distance of the active skeleton

method (in red) as a function of the standard deviation of the Gaussian noise. As it can be observed, the active contour method

is much more sensitive to the noise.

Figure 9. Haussdorff distance to the ground truth of the active contour method for increasing levels of smoothness.

3.2 Application to the study of Escherichia Coli aging

In Escherichia coli (E. coli) bacteria, aging-related protein aggregates accumulate at the old poleregion of the aging bacterium. Studying the dynamics of these molecular components inside the cellsrequires automatic tools for 1) detecting protein aggregates in fluorescence images, 2) computingbacteria boundaries in the corresponding contrast-phase image, 3) assigning each protein aggregateto one cell, 4) expressing the coordinates of the protein aggregates in the basis composed of the celllong (the median line) and short axes (along the skeleton width). We used the method in Dimiccoli,Jacob and Moisan (2015) to detect protein aggregates in fluorescence images, the method Primet,Demarez, et al. (2008) for segmenting the cells in contrast-phase images, and the active skeletonmethod to refine bacteria boundaries prior to the affiliation of protein aggregates to cells. Thenthe final skeletons were used in two different manners. Firstly, to localize the protein aggregatesin the skeletal short and long axes referential. Then, the simple shape of the skeleton was used toestimate the total cell width and length as that of the respective skeleton.

As a convention, we refer below to the aggregate coordinate along the long axis as the x-coordinate

12


Figure 10. (Left) Cells boundaries obtained by applying the snakes method. (Right) Cells boundaries obtained by applying

the active skeleton method. Both used the method Primet, Demarez, et al. (2008) as initialization. Detected protein aggregatesare visualized as white points. The accuracy of the active skeleton method allows a correct assignment of protein aggregates to

cells.

Figure 11. (a) The x− and y− coordinates of protein aggregate inside the cell correspond to the long and short axis respectively.(b) Histogram of the x−component of 1,644 images associated to initial trajectories. To take into account the high variability

of cell lengths, the x-component was rescaled by division by the cell half-length, so that the cell poles are located at locations

-1.0 and 1.0 respectively. Image adapted from the biological study Coquel, Jacob, et al. (2013)

and that along the short axis as the y-coordinate (see Fig. 11 (a)). In Fig. 11 (b) is shown thehistogram of the x−component computed over 1,644 images. It can be seen that most aggregatesaccumulate along the center, that undergoes division, and the poles of bacteria. Fig. 12 schematizesthe increase of the cell half-length during growth that dominates the movement along the x-axis,also showing the time evolution of the mean displacement.

4. Discussion

This study has covered a number of aspects of the problem of investigating the dynamics of bac-teria cells and their molecular components. Overall the work reported produced numerical valuesfor validation results and experimental results, and the discussion will be organised to highlightdifferent parts of the process and to analyse the computational complexity. We will end this sec-

13


Figure 12. Evolution mean displacement along the x− (red) and y− axis (black). The movement along the x-axis dominatedthe cell half-length growth. Image adapted from the biological study Coquel, Jacob, et al. (2013).

tion by discussing the range of applicability of the proposed method. An example of initializationis shown on figure 13 (a): we choose a default number of four points per bacteria (and more ifthe spline interpolation is too far from the skeleton). Even if the morphological skeleton providesradii values at each skeleton point, we used a constant value corresponding to the minimal contactradius instead (see blue outlines). In this way, the influence of the initial segmentation obtainedby applying the method proposed by Primet el al. Primet, Demarez, et al. (2008) is minimized.The results correspond to what is expected: the different final skeletons are inside the bacteriaand the implicit outlines demarcate dark areas from light outsides. Figures 13 (b) and 13 (c) showfinal states from initial positions given in figure 13 (a). They highlight the interest of the repulsionenergy Er. The second segmentation seems fully coherent with the initial image.

From a biological point of view the segmented bacteria are too thin since they should be incontact according to the experimental conditions. Of course this is not the case for the abovesegmentation (e.g on figure 13 (c)) because we consider that there are bright pixels outside: thebright pixels isolate the bacteria from each other. For this experimental reason, the bacteria have tobe enlarged in some way. We propose two strategies to fix this problem. The first solution consistsin adding a positive coefficient αin < 1 in the expression Ed = αinEin − Eout. This enables oneto enlarge the bacteria according to the luminosity transitions as shown on figure 13 (d). Thoughtheoretically this is a good solution, the result is that different bacteria are more ore less dilatedaccording to their outline intensity. Moreover, when taking low values of αin (very big bacteria)the active skeleton may leak in the background. Another repulsion term from the bottom couldbe introduced but it supposes to set an additional arbitrary parameter: the minimal distance ofbacteria to the bottom ∆2. Then the result is biased like on red dots in Fig. 13 (e). For thebiological study we prefer another solution. We consider that bacteria are bigger than they appearon the image. Then the optimization is done by considering bigger bacteria in the repulsion energyterm Er (interaction between real bacteria) while having a thin eroded representation of bacteriafor the data confidence energy Ed. In other words, we assume that an eroded representation of thebacteria may fit the image. Let us call h the radius difference between real and thin bacteria. Toadapt consequently the repulsion term Er, ∆ in equation (3.1) is replaced by ∆ + h to preventthe overlapping of the physical bacteria. The optimization process produces thin bacteria. Then toobtain the physical bacteria from their thin image version, a homogeneous dilation is performedafter the optimization. In practice, it simply consists in adding h to all ri values. Figure 13 (f)presents a h = 1.5 pixels thickness difference.

The computational complexity of the proposed method depends on three parameters: the num-ber of active skeletons, the number of skeletal points used to model each of them and the imageresolution. More precisely, for one step of the gradient descent and for a single skeleton, the com-plexity of deriving the energy term is O(MN), where N is the number of skeleton points and M

14


(a) (b)

(c) (d)

(e) (f)

Figure 13. (a) Initial state of active skeletons (green) made up of at least 4 points with the corresponding implicit outlines

(blue). (b) Final state of active skeletons (green) and corresponding implicit outlines (blue) without the repulsion energy Er. (c)

Final state of active skeletons (green) and corresponding implicit outlines (blue) with the repulsion energy Er. The red skeletonwas stopped by Er. (d) Final state of active skeletons (green) and corresponding implicit outlines (blue) with a thickness

parameter αin = 0.8, still with the repulsion energy Er. (e) Final state of active skeletons with a thickness parameter αin = 0.6

and an additional bottom repulsion set to ∆2 = 2 pixels (effective on the red skeleton dots). (f) Final state of active skeletonswith a repulsion of ∆ + h = 3.3 pixels and a uniform dilation of h = 1.5 pixels.

is the number of pixels within a box surrounding the skeleton dilation. For K skeletons withoutrepulsion energy, the complexity is O(KMN). When considering the repulsion energy, it becomesO(KMN) +O(K2N2), where the second term corresponds to inter-bacteria distances. Since accu-rate segmentation of bacteria can be achieved with a few skeletal points, the algorithm complexitywill mainly depend on the image resolution and on the number of bacteria in this particular ap-plication.

We conclude by remarking that although this work was tailored to cigar- or rod-shaped bacteria,it would work as well for any kind of curved ovoid shape. For non-ovoid shapes, it could be adaptedby changing the dilation model. For more complicated ramified shapes, the skeleton model itselfshould be adapted. The utility spectra of the proposed method is not restricted to bacteria, but toall cases in which a precise segmentation of cigar- or rod-shaped object is needed and the imagequality is poor. For instance it could be very useful for segmenting images of insects in the studyof insect populations, or for the segmentation and recognition of tumors and cigar- or rod-shaped

15


cells.

5. Conclusions

This paper has proposed an active skeleton approach for bacteria modeling, that presents sev-eral advantages for the study of bacteria shape properties and the dynamics of their molecularcomponents. Indeed, the long and short axis of the skeleton define a reference system centeredto the bacterium, which is crucial to study the dynamics of subcellular components inside thecell. In addition, the computation of shape properties such as local orientation, thickness, lengthand perimeter from the skeleton representation is straightforward. Finally, bacteria boundaries arecomputed accurately even in very noised images by introducing implicitly smooth shape priorsinto the segmentation process. The improved accuracy is important to reduce the ambiguity of theassociation molecule/cell in time-lapse fluorescence and phase-contrast imaging.

The proposed model has been successfully used to study the dynamic of protein aggregates inE. Coli Coquel, Jacob, et al. (2013). Localization of proteins to specific positions inside bacte-ria is crucial to several physiological processes, including chromosome organization, chemotaxisor cell division and aging. The Image-J plugin of the proposed method can be found online athttp://fluobactracker.inrialpes.fr.

Further improvements could be discussed and developed according to the kind of data and tothe imaging task to deal with. For example, the fidelity term of the energy functional could bechosen differently from the literature on data energy terms for snakes. Ad-hoc initializations couldbe derived from morphological gray-level skeletons and segment extraction. Finally, the skeletonmodel could handle more complex shapes, with more branches and angle constraints, and eventuallywith other distance definitions relaxing the circular extremity of each branch.

Acknowledgement

This work has been partially supported by the project PAGDEG: Causes and consequences ofprotein aggregation in cellular degeneration. Funding: French ANR. The authors would like tothank Anne-Sophie Coquel for sharing the biological data and contributing to validate the proposedmethod.

References

Amini A, Weymouth T, Jain R. 1990. Using dynamic programming for solving variational problems in vision.IEEE Transactions on pattern analysis and machine intelligence:855–867.

Bai X, Latecki LJ. 2008. Path Similarity Skeleton Graph Matching. Pattern Analysis and Machine Intelli-gence, IEEE Transactions on. 30(7):1282–1292.

Bai X, Wang X, Latecki LJ, Liu W, Tu Z. 2009. Active skeleton for non-rigid object detection. In: ComputerVision, 2009 IEEE 12th International Conference on; Sep.; Kyoto. IEEE; p. 575–582.

Bajcsy R, Kovacic S. 1989. Multiresolution elastic matching. Computer Vision, Graphics, and Image Pro-cessing. 46(1):1–2.

Blum H. 1973. Biological shape and visual science (part I). Journal of Theoretical Biology. 38(2):205–287.Brigger P, Hoeg J, Unser M. 2000. B-spline snakes: a flexible tool for parametric contour detection. IEEE

Transactions on Image Processing. 9(9):1484–1496.Caselles V, Catte F, Coll T, Dibos F. 1993. A geometric model for active contours in image processing.

Numerische Mathematik. 66(1):1–31.Chan T, Vese L. 2001. Active contours without edges. IEEE Transactions on image processing. 10(2):266–

277.Chesnaud C, Refregier P, Boulet V. 1999. Statistical region snake-based segmentation adapted to different-

16


physical noise models. IEEE Transactions on Pattern Analysis and Machine Intelligence. 21(11):1145–1157.

Cohen L, Bardinet E, Ayache N. 1992. Surface reconstruction using active contour models. Report INRIA.Cohen L, Cohen I. 1993. Finite-element methods for active contour models and balloons for 2-D and 3-D

images. IEEE Transactions on Pattern Analysis and Machine Intelligence. 15(11):1131–1147.Cohen L, Kimmel R. 1997. Global minimum for active contour models: A minimal path approach. Interna-

tional Journal of Computer Vision. 24(1):57–78.Cohen LD. 1991. On active contour models and balloons. CVGIP: Image Underst. 53(2):211–218.Cohen LD. 1992. On active contour models. Springer Verlag.Cootes T, Hill A, Taylor C, Haslam J. 1993. The use of active shape models for locating structures in medical

images. In: Information Processing in Medical Imaging. Springer; p. 33–47.Cootes T, Taylor C. 1995. Combining point distribution models with shape models based on finite element

analysis. Image and Vision Computing. 13(5):403–409.Cootes T, Taylor C, Cooper D, Graham J, et al. 1995. Active shape models-their training and application.

Computer vision and image understanding. 61(1):38–59.Coquel A, Jacob J, Primet M, Demarez A, Dimiccoli M, Julou T, Moisan L, AB L. 2013. Localization of

Protein Aggregation in Escherichia coli Is Governed by Diffusion and Nucleoid Macromolecular CrowdingEffect. PLoS Comput Biol. 9(4).

Davatzikos C, Tao X, Shen D. 2003. Hierarchical active shape models, using the wavelet transform. IEEETransactions on Medical Imaging. 22(3):414–423.

Dimiccoli M, Jacob JP, Moisan L. 2015. Particle detection and tracking by a-contrario approach: applicationto fluorescence time-lapse imaging. arXiv preprint arXiv:150706266.

Geiger D, Gupta A, Costa L, Vlontzos J. 1995. Dynamic programming for detecting, tracking, and match-ingdeformable contours. IEEE Transactions on Pattern Analysis and Machine Intelligence. 17(3):294–302.

Ivins J, Porrill J. 1995. Active region models for segmenting textures and colours. Image and Vision Com-puting. 13(5):431–438.

Jacob M, Blu T, Unser M. 2004. Efficient energies and algorithms for parametric snakes. IEEE Transactionson Image Processing. 13(9):1231–1244.

Jain A, Zhong Y, Dubuisson-Jolly M. 1998. Deformable template models: A review. Signal Processing.71(2):109–129.

Jehan-Besson S, Gastaud M, Barlaud M, Aubert G. 2003. Region-based active contours using geometricaland statistical features for image segmentation. In: IEEE International Conference in Image Processing;vol. 2. Citeseer; p. 643–646.

Jolly M, Lakshmanan S, Jain A. 1996. Vehicle segmentation and classification using deformable templates.IEEE Transactions on Pattern Analysis and Machine Intelligence. 18(3).

Kass M, Witkin A, Terzopoulos D. 1988. Snakes: Active contour models. International journal of computervision. 1(4):321–331.

Lakshmanan S, Grimmer D. 1996. A deformable template approach to detecting straight edges in radarimages. IEEE Transactions on Pattern Analysis and Machine Intelligence. 18(4):438–443.

Leibe B, Leonardis A, Schiele B. 2006. An implicit shape model for combined object categorization andsegmentation. Lecture notes in computer science; vol. 4170. Springer Berlin Heidelberg. p. 508–524.

Leordeanu M, Hebert M, Sukthankar R. 2007. Beyond local appearance: Category recognition from pairwiseinteractions of simple features. CVPR.

Li C, Kao CY, Gore J, Ding Z. 2008. Minimization of region-scalable fitting energy for image segmentation.Image Processing, IEEE Transactions on. 17(10):1940–1949.

Li C, Kao CY, Gore JC, Ding Z. 2007. Implicit Active Contours Driven by Local Binary Fitting Energy.In: Computer Vision and Pattern Recognition, 2007. CVPR '07. IEEE Conference on; Jun. IEEE;p. 1–7.

Lindner AB, Madden R, Demarez A, Stewart EJ, Taddei F. 2008. Asymmetric segregation of protein aggre-gates is associated with cellular aging and rejuvenation. Proceedings of the National Academy of Sciences.105(8):3076–3081.

Lowe DG. 2001. Local feature view clustering for 3D object recognition. In: Computer Vision and PatternRecognition, 2001. CVPR 2001. Proceedings of the 2001 IEEE Computer Society Conference on; Kauai,HI, USA; vol. 1. Los Alamitos, CA, USA: IEEE; p. I–682–I–688 vol.1.

Malladi R, Sethian J, Vemuri B. 1995. Shape modeling with front propagation: A level set approach. IEEETransactions on Pattern Analysis and Machine Intelligence. 17(2):158–175.

17


Mao H, Liu H, Shi P. 2010. A convex neighbor-constrained active contour model for image segmentation.In: Image Processing (ICIP), 2010 17th IEEE International Conference on. IEEE; p. 793–796.

Marc P, Menet S, Medioni G. 1990. B-snakes: Implementation and application to stereo. In: IU Workshop.p. 720–726.

Osborn MJ, Rothfield L. 2007. Cell shape determination in Escherichia coli. Current opinion in microbiology.10(6):606–610.

Osher S, Sethian J. 1988. Fronts propagating with curvature-dependent speed: algorithms based on Hamilton-Jacobi formulations. Journal of computational physics. 79(1):12–49.

Park H, Schoepflin T, Kim Y. 2001. Active contour model with gradient directional information: Directionalsnake. IEEE Transactions on Circuits and systems for video technology. 11(2):252–256.

Potier C, Vercken C. 1994. Modelling digitized graphics. ENST-94-D-012. 94-012.Primet M, Demarez A, Taddei F, Lindner AB, Moisan L. 2008. Tracking of cells in a sequence of images

using a low-dimension image representation. In: Biomedical Imaging: From Nano to Macro, 2008. ISBI2008. 5th IEEE International Symposium on; May. IEEE; p. 995–998.

Siddiqi K, Kimia BB. 1995. Parts of Visual Form: Computational Aspects. IEEE Trans Pattern Anal MachIntell. 17(3):239–251.

Siddiqi K, Kimia BB. 1996. A shock grammar for recognition. In: Computer Vision and Pattern Recognition,1996. Proceedings CVPR '96, 1996 IEEE Computer Society Conference on; Jun. IEEE; p. 507–513.

Siddiqi K, Kimia BB, Tannenbaum A, Zucker SW. 2001. On the psychophysics of the shape triangle. VisionResearch. 41(9):1153–1178.

Siddiqi K, Shokoufandeh A, Dickenson SJ, Zucker SW. 1998. Shock graphs and shape matching. In: Com-puter Vision, 1998. Sixth International Conference on; Jan. IEEE; p. 222–229.

Slepchenko BM, Schaff JC, Carson JH, Loew LM. 2002. Computational cell biology: spatiotemporal simu-lation of cellular events. Annual review of biophysics and biomolecular structure. 31(1):423–441.

Staib L, Duncan J. 1992. Boundary finding with parametrically deformable models. IEEE Transactions onPattern Analysis and Machine Intelligence. 14(11):1061–1075.

Thieu QT, Luong M, Rocchisani JM, Viennet E. 2011. A convex active contour region-based model forimage segmentation. In: Proceedings of the 14th international conference on Computer analysis of imagesand patterns - Volume Part I; Seville, Spain. Berlin, Heidelberg: Springer-Verlag; p. 135–143. CAIP’11.

Trinh NH, Kimia BB. 2011. Skeleton Search: Category-Specific Object Recognition and Segmentation Usinga Skeletal Shape Model. Int J Comput Vision. 94(2):215–240.

Wall K, Danielsson P. 1984. A fast sequential method for polygonal approximation of digitized curves.Computer Vision, Graphics, and Image Processing. 28(2):220–227.

Wang L, Li C, Sun Q, Xia D, Kao CY. 2009. Active contours driven by local and global intensity fittingenergy with application to brain MR image segmentation. Computerized Medical Imaging and Graphics.33(7):520–531.

Widrow B. 1973. The rubber-mask technique. i. pattern measurement and analysis. Pattern Recognition.5(3):175–97.

Xu C, Prince J. 1998. Snakes, shapes, and gradient vector flow. IEEE Transactions on image processing.7(3):359–369.

Yuille A, Hallinan P, Cohen D. 1992. Feature extraction from faces using deformable templates. Internationaljournal of computer vision. 8(2):99–111.

Zhang K, Zhang L, Song H, Zhou W. 2010. Active contours with selective local or global segmentation: Anew formulation and level set method. Image Vision Comput. 28(4):668–676.

Zhu S, Yuille A. 1996. Region competition. IEEE trans on pattern analysis and machine intelligence.18(9):884–900.

Zhu SC, Yuille AL. 1995. FORMS: a flexible object recognition and modelling system. In: Computer Vision,1995. Proceedings., Fifth International Conference on; Jun. IEEE; p. 465–472.

18

computer vision center (cvc) arxiv:1507.06504v4 [cs.cv] 22 ... · protein aggregates detected by...

Documents