a discrete chain graph model for 3d+t cell tracking with ... · a discrete chain graph model for...

A Discrete Chain Graph Model for3d+t Cell Tracking with High

Misdetection Robustness

Bernhard X. Kausler1, Martin Schiegg1, Bjoern Andres1,2,Martin Lindner1, Ullrich Koethe1, Heike Leitte1,

Jochen Wittbrodt3, Lars Hufnagel4, and Fred A. Hamprecht1

1HCI/IWR, Heidelberg University, Germany2SEAS, Harvard University, United States

3COS, Heidelberg University, Germany4European Molecular Biology Laboratory (EMBL), Heidelberg, Germany

Contact: [email protected]

Abstract. Tracking by assignment is well suited for tracking a varyingnumber of divisible cells, but suffers from false positive detections. Wereformulate tracking by assignment as a chain graph–a mixed directed-undirected probabilistic graphical model–and obtain a tracking simul-taneously over all time steps from the maximum a-posteriori configura-tion. The model is evaluated on two challenging four-dimensional datasets from developmental biology. Compared to previous work, we obtainimproved tracks due to an increased robustness against false positivedetections and the incorporation of temporal domain knowledge.

Keywords: chain graph, graphical model, cell tracking

1 Introduction

One grand challenge of developmental biology is to find the lineage tree of allcells in a growing organism, i.e. the complete ancestry of each cell [17]. The chal-lenges encountered include (i) simultaneously tracking an unknown and variablenumber of cells; (ii) allowing for cell division; and (iii) high accuracy, becauseeach tracking error affects a complete subtree of the lineage.

We propose a tracking by assignment solution contributing these novelties:

1. In recent years graphical models proved to be exceptionally successful incomputer vision [4]. We adopted the approach and present the first proba-bilistic graphical model for cell tracking which can track an unknown numberof divisible objects that may appear or disappear and which can cope withfalse detections.

2. The model achieves robustness against noise detections by solving the track-ing problem for all time steps at once taking the expected time betweendivisions into account. The most likely cell lineage is obtained as the modelconfiguration with maximum a-posteriori (MAP) probability.

2 A Discrete Chain Graph Model for 3d+t Cell Tracking

Fig. 1: Maximum intensity projection of aDrosophila embryo and tracking: commonancestry is indicated by common color.

Our model1 2 3 4 5 · · · 189

1 2 3 4 5 · · · 189 1 2 3 4 5 · · · 189

Manual tracking Bise et al. [3]

Fig. 2: Lineage trees forDrosophila: exemplary manuallytracked lineage trees are compared tothe results of Bise et al. [3] and ourmodel.

3. While all literature with similar data focuses on single organisms, we presentresults on zebrafish and–for the first time–on Drosophila together with goldstandard lineages and benchmarking tools (as a supplementary).

We obtain the exact MAP solution by integer linear programming (ILP), and–when comparing to the gold standard lineages–obtain results that are superiorto existing methods.

2 Related Work

Tracking in science comprises many different application domains and approaches[23]. Our work is concerned with the tracking of a varying number of divisible ob-jects with similar appearance in images with its main application in cell lineagereconstruction. It has to be distinguished from tracking many similar particlesthat are not dividing [21].

When the temporal resolution of the raw data is high compared to the rateof change, derivative-based methods such as optical flow [18] or level sets [20]can be used to simultaneously segment and track the evolution of one or moretargets in spacetime.

A lower temporal resolution–as it is the case in our application–makes theproblem harder and in response, most algorithms then separate the detection / seg-mentation from the tracking problem: detections are found in each time step andfed into a tracking routine. Two different kinds of models are typically used: statespace models and assignment models. The Kalman filter, a linear Gaussian state

A Discrete Chain Graph Model for 3d+t Cell Tracking 3

space model, is the archetype of the former class, which interprets detections ascaused by a hidden target. It has been generalized in a number of ways allowingfor nonlinear motion models, discrete state spaces, non-Gaussian distributions[2, 7], multiple target hypotheses, or even an unknown (but fixed) number oftargets [8]. While state space models easily accommodate target properties suchas velocity, size, and appearance and are robust against noisy detections by de-sign, they are unfortunately hard to coax into dealing with a variable number ofdividing objects.

Tracking by assignment, on the other hand, treats every detection as a poten-tial target itself. It easily accommodates multiple or dividing objects; however,this increased flexibility must be reined in by enforcing the consistency (see sec-tion 3) of a tracking. This difficulty has so far been addressed in three ways.Firstly, approximate methods can be applied that forego a consistency guaran-tee [10]. Secondly, tracks can be generated hierarchically from tracklets [3, 15, 6],akin to the use of superpixels in image processing. Thirdly, tracking is performedacross pairs of frames only [11, 12, 19, 16].

Approach [3] is most similar to ours since it is–to our best knowledge–theonly other global model of detections and assignments over many time steps.Therefore we chose this method to compare with.

3 Graphical Model for Global Tracking by Assignment

Both state space and assignment models treat detection (segmentation) andtracking as separate problems. We will focus on tracking in this article andassume given detections with a high misdetection rate (≈ 10%).

Specifically, we propose a probabilistic model for the tracking problem thatmeets the requirements set out in the introduction and can give limited feedbackto the detection step by incorporating the possibility of spurious detections, i.e.allows to switch off erroneous detection hypotheses to address the high misde-tection rate. The model takes the form of a chain graph [9]—a directed graphicalmodel (Bayesian network) of “supernodes”, each of which consists of a condi-tional random field over a set of “subnodes”. A chain graph model turns out tobe necessary because neither Bayesian networks nor Markov random fields aloneare compatible with the independence assumptions imposed by tracking.

Our model contains two types of random variables: detection variables and

assignment variables. A binary detection variable X(t)i is associated with the ith

detected object (cell candidate) at time step t, and X (t) denotes the set of alldetection variables at time t. Values of these variables determine if the corre-sponding detections are accepted as a true object (cell) (and hence incorporatedinto the tracking interpretation), or rejected as a false alarm.1 Binary assign-

ment variables Y(t)ij are associated with every pair of cell candidates i, j at times

t ∈ [1, . . . , T − 1], t + 1, and Y(t) is the set of all assignment variables between

1 We deliberately do not model missed detections (i.e. objects, that are not visible inthe data) since they are very rare in the 3d images of typical applications.


steps t and t+ 1. A value of 1 expresses the belief that cell candidate j at timestep t + 1 is identical with, or a child of, cell candidate i at time step t, and avalue of 0 says that i and j are unrelated.

In addition, a number of natural consistency constraints are imposed on thesevariables by our application in developmental biology: First, each cell must have

at most one predecessor, i.e.∑

i Y(t)ij ≤ 1. Second, each cell must have a unique

fate—if it does not disappear (by leaving the data frame or other reasons), itcan either move to (be assigned to) a unique cell candidate in the next timeframe, or it can divide and be associated with exactly two cell candidates inthe next time frame. These children, in turn, may not have any other nonzeroincoming assignment variables. Third, there is a biological upper bound on thedistance traveled between time frames, excluding all assignments that are too farapart. This leads to a significant reduction in the number of required assignmentvariables.

These consistency constraints define which configurations are impossible, i.e.have zero probability. In the next two subsections, we describe how the proba-bilities of feasible configurations are defined by connecting variables into a chaingraph whose factors encode our knowledge about the plausibility of differenttrackings as a function of data-dependent properties such as distance traveled,probability of being a misdetection, etc.

Conditional random field over assignment variables Let us pretend forthe moment that the optimal values of the detection variables are already known.Then the probability of a configuration Y(t) of assignment variables connectingframes t and t+1 given the detection variables X (t) and X (t+1) can be expressedwith an undirected graphical model, in particular a conditional random field(CRF):

CRF(t) : P (t)(Y(t) | X (t),X (t+1)) (1)

where

P (t)(Y(t)|X (t),X (t+1)) =1

Z(t)

∏X

(t)i ∈X (t)

φ(t)i→(X

(t)i ,Y(t)

i→)∏

X(t+1)j ∈X (t+1)

φ(t)→j(Y

(t)→j , X

(t+1)j ) , (2)

and φ(t)i→ and φ

(t)→j denote the factors for outgoing and incoming transitions re-

spectively. Note that the CRF partition function Z(t)(X (t),X (t+1)) is only calcu-lated over the assignment variables, since the detection variables are consideredas given. A factor graph [14] representation of eq. (2) is shown in Fig. 3a. Each ofthe maximal cliques of the underlying undirected graph corresponds to a factorin the CRF. These CRFs will form the supernodes of the chain graph, while theindividual assignment variables are the subnodes.

Analyzing the graph with d-separation the implicit independence assump-

tions of the CRF can be identified: an assignment Y(t)ij is conditionally indepen-

dent of all assignments Y(t)kl involving different detection variables k 6= i, l 6= j,


(a) Undirected part of the chaingraph. A single assignment CRF isshown as a factor graph [14]. The darknodes represent fixed detection vari-ables X (at t and t+1) and the smaller,light nodes assignment variables Y.

(b) Directed part of the chain graph.The boxes symbolize supernodes containingassignment CRFs according to Fig. 3a.

Fig. 3: Chain graphical model for global tracking by assignment.

provided that all incoming assignments Y(t)→j of detection j and all outgoing

assignments Y(t)i→ of detection i are given:

Y(t)ij ⊥⊥ Y

(t)kl | Y

(t)→j ,Y

(t)i→ l 6= j, i 6= k . (3)

In other words, the assignment decisions only influence a small neighborhood ofother assignment variables directly. By fixing enough variables the CRF can evendecouple in two or more completely independent tracking problems—promotingthe tractability of the chain graph model.

Combining assignment CRFs in a chain graph In order to get rid of therequirement that detection variables are already known, we connect all CRFsdefined above by means of directed edges from detection variables to factors, ina manner analogous to a Bayesian network. The joint probability over all timesteps can now be factorized as

P (X ,Y) =

T∏t=1

∏X

(t)i ∈X (t)

Pdet(X(t)i ) ·

T−1∏t=1

P (t)(Y(t)|X (t),X (t+1)) , (4)

where Pdet(X(t)i ) is the probability that a detection hypothesis should be ac-

cepted into the tracking. The corresponding chain graphical model is sketchedin Fig. 3b. By treating each CRF as a single random variable (supernode)with as many states as configurations of the assignment variables (subnodes),d-separation can be easily applied to the directed part of the model. The (unob-served) supernodes have only incoming edges and are blocking the path betweendetection variables, revealing the following conditional independence relations:


1. Detection variables are independent from each other, i.e.Xi ⊥⊥ (X\Xi) ,∀Xi.2. Consecutive assignment variables are conditionally independent given the

connecting detection variables, i.e. Y(t) ⊥⊥ Y(t+1) | X (t+1).

These are rather strong independence assumptions and–of course–only an ap-proximation to the real world. Later, we introduce an extension (eqs. (11), (10))that, on the one hand, improves the model accuracy but, on the other hand,weakens these relations and makes inference harder.

Inference by Integer Linear Programming Under the above model, thesingle most likely tracking is given by the maximum a-posteriori (MAP) config-uration of variables. Finding this configuration can equivalently be restated asan energy minimization problem, by expressing the probability in eq. (4) as aGibbs distribution P (X ,Y) = 1

Z e−E(X ,Y) such that

argmaxX ,Y

P (X ,Y) = argminX ,Y

E(X ,Y) . (5)

Rewriting the chain graph distribution in (4) as a sum of energies E allows us toapply integer linear programming (ILP) for minimization. This approach has twoadvantages. Firstly, it allows us to use state-of-the-art ILP solvers that are able todetermine globally optimal solutions of (5). Any remaining shortcomings of ourtracking results can thus be attributed to our model and are not a consequence ofapproximate optimization. Secondly, we can easily exclude biological impossibletrackings by incorporating appropriate constraints in the linear program.

Application: Cell tracking Applying (5) to the chain graph model equa-tions (2) and (4) gives the energy function

E(X ,Y) =

T∑t=1

∑X

(t)i ∈X (t)

Edet(X(t)i )+

T−1∑t=1

∑i

Eout(X(t)i ,Y(t)

i→) +∑j

Ein(Y(t)→j , X

(t+1)j )

(6)

where Edet, Eout, and Ein are the energies corresponding to the factors Pdet,

φ(t)i→, and φ

(t)→j respectively.

The first term allows to incorporate evidence from the raw data regardingthe presence or absence of a cell at a location where a potential target was de-tected. In our experiments, we use Random Forest [5] to estimate the probability

P̂det

(X

(t)i

)of a detection being truly a cell, based on features such as intensity

distribution and volume of each cell candidate found by the detection module.In keeping with the definition of the Gibbs distribution, we get

Edet

(X

(t)i

)=

− ln(P̂det

(X

(t)i

)), X

(t)i = 1

− ln(

1− P̂det

(X

(t)i

)), X

(t)i = 0

(7)


The assignment energies Ein and Eout need to embody two kinds of priorknowledge: firstly, which trackings are consistent; and secondly, amongst all con-sistent trackings, which ones are plausible. To enforce consistency zero probabil-ity / infinite energy is assigned to cells dividing into more than two descendants,cells arising from the merging of two or more cells from the previous frame, andcell candidates that are regarded as misdetections.

To penalize implausible trackings, we need to parameterize our model ap-propriately. The model equations expose exactly five free parameters (Cinit,Cterm, Copp, w, d̄). Cinit imposes a penalty for track initiation (i.e. when acell (re-)appears in the data frame) and Cterm for track termination (i.e. whena cell disappears from the data frame or is lost due to cell death). To discour-age trivial solutions where too many objects are explained as misdetections, weexact a opportunity cost of Copp for the lost opportunity to explain more of thedata. Furthermore, each move is associated with a cost dependent on the squaredlength of that move. For cell divisions, finally, it is known that the descendantstend to be localized at an average distance d̄ from the parent cell, and punishdeviations from that expected distance.

All of the above can be summarized in the following assignment energies:

Eout(X(t)i ,Y(t)

i→) =

∞ , X(t)i = 1 ∧

∑j Y

(t)ij > 2 > 2 children

w((d− d̄)2

+(d′ − d̄)2), X(t)

i = 1 ∧∑

j Y(t)ij = 2 division

wd2 , X(t)i = 1 ∧

∑j Y

(t)ij = 1 move

Cterm , X(t)i = 1 ∧

∑j Y

(t)ij = 0 disappearance

Copp , X(t)i = 0 ∧

∑j Y

(t)ij = 0 opportunity

∞ , X(t)i = 0 ∧

∑j Y

(t)ij > 0 tracked misdetection

(8)

Ein(Y(t)→j , X

(t+1)j ) =

∞ , X(t+1)j = 1 ∧

∑i Y

(t)ij > 1 > 1 parent

0 , X(t+1)j = 1 ∧

∑i Y

(t)ij = 1 move

Cinit, X(t+1)j = 1 ∧

∑i Y

(t)ij = 0 appearance

0 , X(t+1)j = 0 ∧

∑i Y

(t)ij = 0 misdetection, no parent

∞ , X(t+1)j = 0 ∧

∑i Y

(t)ij > 0 misdetection with parent

(9)

Note that more informative features could be extracted from the raw data: forinstance, besides the length of a move, one could consider the similarity of theassociated cell candidates to obtain an improved estimate of their compatibility.However, it is then no longer possible to select appropriate values for the pa-rameters with a simple grid. Instead, a proper parameter learning strategy willbe needed, which will be a subject of our future research.

Domain specific knowledge: minimal cell cycle length The model may beextended further by incorporating domain specific knowledge. For instance, cellsmust pass through specific cell cycle states (interphase, prophase, metaphase,anaphase, telophase) and thus, there is a biological lower bound for the dura-tion of a cell cycle. In other words, it is impossible to observe a cell dividingtwice within a number of subsequent frames as it is the case in Fig. 5a. This


biological constraint can be integrated in our cell tracking model by expanding

the detection variables X(t)i to obtain the number of time steps since the last

division of an individual cell, i.e. X(t)i ∈ {0, 1, ..., τ}, where X

(t)i = 0 still stands

for misdetection and τ is a parameter for the minimal duration between divisionevents of an individual cell. Hence, the energy functions given above change only

slightly in that X(t)i = 1 becomes X

(t)i ≥ 1 and X

(t)i = τ for the division energy.

Besides, another two factors need to be introduced to incorporate the countingbetween successive detections. The corresponding energy functions are given by

Ecnt→(X(t)i ,Y(t)

i→,X(t+1)i→ ) =

∞ , Y

(t)ij = 1 ∧

∑k Y

(t)ik = 2 ∧X(t+1)

j 6= 1

∞ , Y(t)ij = 1 ∧

∑k Y

(t)ik = 1 ∧X(t)

i < τ ∧X(t+1)j 6= X

(t)i + 1

∞ , Y(t)ij = 1 ∧

∑k Y

(t)ik = 1 ∧X(t)

i = τ ∧X(t+1)j 6= τ

0 , otherwise

,

(10)

Ecnt←(X(t+1)j ,Y(t)

→j) =

{∞ ,

∑k Y

(t)kj = 0 ∧X(t+1)

j 6= 0 ∧X(t+1)j 6= τ

0 , otherwise, (11)

where X (t+1)i→ are the detection variables connected to X

(t)i through Y(t)

i→. Tech-nically, these rules are implemented as hard constraints on indicator variableseach representing a possible state of the detection variables. It should be notedthat with this modification, the detection variables are no longer independent.For that reason and due to the increased number of variables, computation timeincreases by a factor of 10.

4 Experiments

We applied the chain graph model to track cells during early embryogenesis ofzebrafish (Danio Rerio) and fruit fly (Drosophila): the GFP-stained nuclei wereimaged with different light sheet fluorescence microscopes.

4.1 Zebrafish Dataset

Keller et al. [13] recorded time-lapse volumetric images of developing zebrafishembryos (3d+t). A first cell lineage reconstruction of the blastula period based onthat data was reported by [16]. Its segmentation-based cell detection method hashigh recall (false negative rate of 4±1% for cell detection) but suffers in precisiondue to segmentation errors and noise. Cell tracking was done by matching thetime slices with minimal cost by integer linear programming. The test datasetconsisted of 80 frames of the animal pole from 66 cells up to ca. 2400 cells. Itslow segmentation precision warrants the application of the chain graph, whichexplicitly models noisy detections.

We implemented the method of Lou et al. [16] and replicated its benchmarkfor tracking the cells in the zebrafish dataset. The benchmark measures per-formance in terms of precision, recall, and f-measure2 of the reconstructed cell

2 precision: true pos./(true pos. + false pos.); recall: true pos./(true pos. + false neg.);f-measure: harmonic mean of precision and recall


Fig. 4: Drosophila dataset: syn-opsis of raw data and correspond-ing segmentation; shown is a detailof slice 33 at time step 7.

Table 1: Tracking results on the Ze-brafish dataset: our chain graph modelshows improved performance in terms of f-measure compared to [16].

Lou et al.[16] Chain Graph

precision 0.807 0.897recall 0.861 0.850f-measure 0.833 0.873

movements and divisions between two time slices against a manually obtainedtracking on the first 25 time slices. We use the segmentation result of [16] for afair comparison of the two tracking methods.

4.2 Drosophila Dataset

The other dataset consists of 3d image stacks of a fruit fly syncytial blastoderm(cf. Fig. 1) over 40 time steps. We segmented the cells with the freely avail-able ILASTIK software [22]. Subsequently we obtained a manual tracking tobenchmark the automatic methods.

Compared to the zebrafish dataset the fruit fly dataset has lower contrastresulting in more ill-shaped cell segmentations and false positive detections thatare hard to distinguish from the actual cells (see Fig. 4). On the other hand,cells reside only near the surface of the embryo, lowering the chance of wrongassignments to neighboring cells compared to the zebrafish embryo.

We examine the difference of our holistic approach over all time steps com-pared to a time slice by time slice tracking on that dataset. For the latter, wetake the Random Forest predictions to filter out false alarms beforehand (0.5probability threshold), add detection variables for all remaining objects to thegraphical model and fix them as ’active’. Under these conditions the single assign-ment random fields are independent and the model reduces to a time step-wisetracking approach. Finally, we implemented the method of Bise et al. [3] andevaluated it with the same dataset and benchmark.

4.3 Implementation

We implemented the chain graph model in C++ using the OpenGM [1] libraryand wrote a MAP inference backend based on CPLEX.3

Parameter values are set according to an optimal f-measure of successfullyreconstructed move, division, appearance, and disappearance events relative toa manually obtained tracking. We initialize the parameters with a reasonablesetting and fine-tune them with a search on a grid in parameter space.

3 A commercial (integer) linear programming solver. Free for academic use.


Table 2: Tracking results on the Drosophila dataset: a timestep-wise trackingwith previously filtered misdetections (first col.) is inferior to our full chain graph modeloptimized over all time steps (second col.). The third column shows the results of theextended chain graph with the condition that division events of a cell must at leastbe 3 time steps apart from each other. Finally, the method of [3] (fourth col.) showsdecent recall but suffers in precision.

FixedDetections

UnconditionedChain Graph

Chain Graphwith τ = 3

Bise et al.[3]

precision 0.889 0.953 0.956 0.550recall 0.933 0.957 0.960 0.718f-measure 0.911 0.955 0.958 0.623

Runtimes on the presented datasets are of the order of minutes—obviatingthe need for approximate inference at the current problem size.

Finally, we implemented the methods by [16] and [3] in C++ and foundoptimal parameters via grid search (improving over the ad hoc parametrizationapproach of [3]). Source code and parameters are provided as a supplementary.

4.4 Results

We perform experiments on two datasets. First, we compare our tracker againsta previously published method [16] on the zebrafish dataset. The results aresummarized in table 1. Second, we investigate the performance of the model onthe Drosophila dataset, comparing a chain graph having fixed detection variableswith an unconditioned chain graph and a chain graph with 4-state detectionvariables satisfying a minimal duration of 3 time steps between division eventsof a particular track. Additionally we put the chain graph model in perspectiveby comparing with a state-of-the-art cell tracking method by Bise et al. [3]. Thetracking results are presented in table 2.

Finally, table 3 shows the Drosophila results in terms of correctly and incor-rectly identified cells for both variants of the chain graph in contrast to humanannotations and the Random Forest classifier alone (which was used to fix thevariables).

5 Discussion

The MAP solution of the chain graph is obtained from all time steps at onceconsidering properties of complete tracks. In particular a track needs a minimalnumber of active transitions to overcome the track initialization and termina-tion costs together with the cost for misdetections. Cells that are only shortlypresent–due to low signal to noise or due to entering/leaving the data frame–will consequently be labeled as misdetections. On the other hand, cells that werebadly segmented and were classified as misdetections by the Random Forest may


Table 3: Cell identification performance on Drosophila dataset: compared toan object classifier, the chain graph recovers less actual cells (true positives), but findssignificantly more misdetections (true negatives). The unconditioned chain graph andthe variant with a minimal cell cycle length don’t differ significantly in identificationperformance (but in tracking performance: see table 2).

ClassifierUnconditionedChain Graph

Chain Graphwith τ = 3

Human

true positive 12391 12330 12346 12493true negative 1284 1607 1598 1810

false negative 102 163 147 -false positive 526 203 212 -

be set active as long as they reasonably continue a track (see Fig. 6 for an il-lustration of this behavior). The chain graph model can therefore be seen as aregularizer on the pure classifier output that trades some true positives (decreas-ing recall) against a significantly reduced number of false positives (increasingprecision).

The above interpretation is supported by the results. Compared to the perfor-mance of method [16] on the zebrafish dataset, our method loses 1.1 percentagepoints (pp) in recall but gains 9pp in precision leading to an overall better perfor-mance. More insight into this precision-recall trade-off can be gained by lookingat the cell vs. misdetection identification performance. Compared to the classifieralone, the (unconditioned) chain graph retrieves 61 less cells (less true positives),but labels 323 more misdetections correctly (more true negatives). (The chaingraph controlling for minimal cell cycle length exposes the same behavior withnonsignificant differences in performance.)

Since correct cell identifications are a precondition for a successful recoveryof move, division, appearance, and disappearance events, the higher object iden-tification rate should directly transfer to the tracking performance. The resultson the zebrafish are supporting this assumption, but could also be caused by thefact that our method employs a Random Forest classifier whereas the method of[16] just treats every segmented object as a cell. To exclude the influence of therandom forest we examined another variant of the chain graph model, where wefix the detection variables as a preprocessing step using the very same RandomForest predictions. Compared to the chain graph with previously fixed detectionvariables, the full (unconditioned) model gains 2.4pp and 4pp in terms of recalland precision, respectively. This is clear evidence that the performance gain overthe previously published step-by-step method is not only caused by the RandomForest classifier, but also by the holistic chain graph model that is optimizedover many time steps at once.Further improvements can be obtained by requiring a minimal time between twodivisions. Table 2 shows an increase in f-measure by 0.3pp for a minimal tem-poral distance of three (τ = 3) compared to the unconditioned model. At a first


(a) Impossibledivision

1 2 3 4 50.945

0.950

0.955

0.960

perform

ance

precisionrecallf-measure

1 2 3 4 5minimal duration between divisions τ

0.84

0.86

0.88

0.90

0.92

0.94

precision divisionsrecall divisionsf-measure divisions

minimal duration between divisions τ

(b) Performance on Drosophila depending on τ

Fig. 5: Extending the cell tracking model by the requirement of a minimalduration between division events: (a) shows a biologically implausible tracking(left) and the result of our model when requiring a minimal duration of τ = 3 (right).Performance results on the Drosophila dataset with varying τ are depicted in (b):shown is the performance for all events (left panel) and for division events only (rightpanel).

glance the gain seems nonsignificant, but the f-measure puts the same impor-tance on every type of tracking event. Since the number of move events exceedsthe number of divisions by a factor of 20, it is dominated by the former. Yet, thef-measure for divisions (exclusively) improves from 0.883 (precision 0.941, recall0.832) to 0.917 (precision 0.923, recall 0.911). This is a significant gain and veryimportant for an accurate assessment of cell ancestry since a single mistrackeddivision can spoil a whole subsequent lineage. In practice, setting τ to a lowvalue will already give a boost in performance with an acceptable overhead. Theeffect on the overall and the division performance when varying parameter τ isdepicted in Fig. 5b. The method of Bise et al. [3] shows convincing results inregions with high data quality (cf. Fig. 2). However, its f-measure is 33pp worsecompared to the chain graph. This is most likely caused by the high misdetectionrate of 13% and can be seen from the low precision of 0.550 and Fig. 2.

Still the performance of our approach could be further improved. A manualinspection of incorrectly tracked divisions reveals that the tracker can be con-fused by divisions happening in close local proximity. A better model for divisionsthat also incorporates the change in shape and the geometric motion pattern—descendants move away from the ancestor in roughly opposite directions—couldprevent such mistakes. Future datasets may contain cells that move in a more co-ordinated fashion. The presented move energies are optimal in case of Brownianmotion, but will most likely underperform for directed motions. In that case themodel can be extended with discrete momentum variables that are associatedwith the detected objects–analogous to our extension that controls the minimalcell cycle length. Finally, the model can easily be extended to consider mergingobjects (due to undersegmentation or 2d applications) by allowing more thanone active incoming variable.

6 Conclusion

We presented a chain graph model for tracking by assignment over many timeslices simultaneously. This model is highly robust against misdetections and


Fig. 6: Sample tracking sequence obtained with the chain graph model.The eight consecutive time steps show a detail of the Drosophila dataset projected to2d. The colored spots are segmented objects, where objects with the same color areassigned to each other. Dark gray objects are misdetections as indicated by ’inactive’detection variables. Ill-shaped objects will be treated as active, if they are bridgingtwo tracks in a sensible manner. On the contrary, cell-shaped objects may be labeledinactive, if they do not constitute a track of a certain length.

can consider temporal domain knowledge. It has been applied in the contextof 3d+t cell tracking and achieved better performance than two recently pub-lished methods ([16], [3]). Furthermore, it showed that globally optimal trackingby assignment over many time slices is superior to the popular time step-wiseapproach. The performance could be further improved by introducing bettermotion models for cell movements and divisions.

Acknowledgments This work has been supported by the German ResearchFoundation (DFG) within the programme “Spatio-/Temporal Graphical Modelsand Applications in Image Analysis”, grant GRK 1653.

References

1. Andres, B., Kappes, J.H., Koethe, U., Schnoerr, C., Hamprecht, F.A.: An empiricalcomparison of inference algorithms for graphical models with higher order factorsusing opengm. In: Goesele, M., Roth, S., Kuijper, A., Schiele, B., Schindler, K.(eds.) Pattern Recognition, LNCS, vol. 6376, pp. 353–362. Springer (2010)

2. Arulampalam, M., Maskell, S., Gordon, N., Clapp, T.: A tutorial on particle fil-ters for online nonlinear/non-Gaussian Bayesian tracking. Signal Processing, IEEETransactions on 50(2), 174 –188 (2002)

3. Bise, R., Yin, Z., Kanade, T.: Reliable cell tracking by global data association. In:IEEE International Symposium on Biomedical Imaging (ISBI 2011) (2011)

4. Blake, A., Kohli, P., Rother, C. (eds.): Markov Random Fields for Vision andImage Processing. MIT Press (2011)


5. Breiman, L.: Random forests. Machine Learning 45, 5–32 (2001)6. Brendel, W., Amer, M., Todorovic, S.: Multiobject tracking as maximum weight in-

dependent set. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2011). pp. 1273 –1280 (2011)

7. Doucet, A., Johansen, A.: A tutorial on particle filtering and smoothing: Fifteenyears later. In: Handbook of Nonlinear Filtering. Oxford University Press (2011)

8. Fox, E., Choi, D., Willsky, A.: Nonparametric Bayesian methods for large scalemulti-target tracking. In: Signals, Systems and Computers, 2006. ACSSC’06. For-tieth Asilomar Conference on. pp. 2009–2013 (2006)

9. Frydenberg, M.: The chain graph Markov property. Scandinavian Journal of Statis-tics 17, 333–353 (1990)

10. Jiang, H., Fels, S., Little, J.J.: A linear programming approach for multiple ob-ject tracking. In: IEEE Conference on Computer Vision and Pattern Recognition(CVPR 2007) (2007)

11. Kachouie, N.N., Fieguth, P.W.: Extended-Hungarian-JPDA: exact single-framestem cell tracking. IEEE Transactions on Bio-Medical Engineering 54 (2007)

12. Kanade, T., Yin, Z., Bise, R., Huh, S., Eom, S., Sandbothe, M.F., Chen, M.:Cell image analysis: Algorithms, system and applications. In: IEEE Workshop onApplications of Computer Vision (WACV) (2011)

13. Keller, P.J., Schmidt, A.D., Wittbrodt, J., Stelzer, E.H.: Reconstruction of ze-brafish early embryonic development by scanned light sheet microscopy. Science322, 1065–1069 (2008)

14. Kschischang, F., Frey, B., Loeliger, H.A.: Factor graphs and the sum-product al-gorithm. Information Theory, IEEE Transactions on 47, 498 –519 (2001)

15. Li, Y., Huang, C., Nevatia, R.: Learning to associate: HybridBoosted multi-targettracker for crowded scene. In: IEEE Conference on Computer Vision and PatternRecognition (CVPR 2009) (2009)

16. Lou, X., Kaster, F.O., Lindner, M.S., Kausler, B.X., Koethe, U., Jaenicke, H.,Hoeckendorf, B., Wittbrodt, J., Hamprecht, F.A.: DELTR: Digital embryo lin-eage tree reconstructor. In: IEEE International Symposium on Biomedical Imaging(ISBI 2011) (2011)

17. Meijering, E., Dzyubachyk, O., Smal, I., van Cappellen, W.A.: Tracking in cell anddevelopmental biology. Seminars in Cell & Developmental Biology 20 (2009)

18. Melani, C., Peyrieras, N., Mikula, K., Zanella, C., Campana, M., Rizzi, B.,Veronesi, F., Sarti, A., Lombardot, B., Bourgine, P.: Cells tracking in a live ze-brafish embryo. In: IEEE Engineering in Medicine and Biology Society (EMBS2007 ) (2007)

19. Padfield, D., Rittscher, J., Roysam, B.: Coupled minimum-cost flow cell trackingfor high-throughput quantitative analysis. Medical Image Analysis 15(4) (2011)

20. Padfield, D., Rittscher, J., Thomas, N., Roysam, B.: Spatio-temporal cell cyclephase analysis using level sets and fast marching methods. Medical Image Analysis13(1), 143–155 (2009)

21. Smal, I., Meijering, E., Draegestein, K., Galjart, N., Grigoriev, I., Akhmanova, A.,Van Royen, M., Houtsmuller, A., Niessen, W.: Multiple object tracking in molec-ular bioimaging by Rao-Blackwellized marginal particle filtering. Medical ImageAnalysis 12(6), 764–777 (2008)

22. Sommer, C., Straehle, C., Koethe, U., Hamprecht, F.A.: ILASTIK: Interactivelearning and segmentation toolkit. In: IEEE International Symposium on Biomed-ical Imaging (ISBI 2011) (2011)

23. Yilmaz, A., Javed, O., Shah, M.: Object tracking: A survey. ACM ComputingSurveys (CSUR) 38 (2006)

a discrete chain graph model for 3d+t cell tracking with ... · a discrete chain graph model for...

Documents