segmentation of medical images under topology constraints florent ségonne computer science and...

Segmentation of Medical Images under Topology Constraints

Florent Ségonne

Computer Science and Artificial Intelligence Laboratory, MIT

Athinoula Martinos Center for Biomedical Imaging, MGH, Boston

Most macroscopic brain structures have the topology of a sphere.

Cortical surface C can be considered to have the topology of a sphere S

GOAL: Achieve Accurate and Topologically Correct Segmentations of Anatomical

Structures from Medical Images

Definition of Topological Defects

Background

TOPOLOGICAL DEFECT: deviation from spherical topology

• Handles or holes

• Cavities

• Disconnected components

Importance of Accurate, Topologically-Correct Segmentations

Local functional organization of the cortex is largely 2-dimensional

From (Sereno et al, 1995, Science)

Cortical parcellation

Analysis of functional activitySpherical AtlasVisualization

• Shape Analysis • Presurgical Planning• Statistical analysis of morphometric properties

Aging Neurodegenerative diseases

Longitudinal studies of structural changesHemispheric asymmetry

• Image artifacts

- Partial voluming a single voxel may contain more than one tissue type.

- Bias field effective flip angle or sensitivity of

receive coil may vary across space.

- Tissue inhomogeneities even within tissue type (e.g. cortical gray matter), intrinsic properties such as T1,

PD can vary (up to 20%).

• Topological notions are difficult to integrate into

a discrete numerical framework

Difficulties of Achieving

Accurate, Topologically Correct Segmentations

Assigning tissue classes to voxels can be difficult

Partial volume effect is often the cause for an incorrect topology

Previous Work

Essentially two types of approaches:

1) Segmentation under topological constraint

2) Retrospective topology correction of segmentations

Previous Work

Local decision to preserve topology might lead to large geometrical errors

1) Segmentation under Topological Constraints

• Active contours: Triangulations: self-intersection

Dale et al.-99Davatzikos-Bryan-96MacDonald-et-al.-00

Level sets and Topologically constrained level-sets Zeng-Staib-Schultz-Duncan-99Han-Xu-Prince-03

• Homotopic digital deformations Mangin-et-al.-95

Poupon-et-al.-98Bazin-Pham-05

• Segmentation by registration and vector fields Karacali-Davatzikos-04Christensen-et-al.-97

Final surfaceDeformationInitial condition

Correct segmentation

Incorrect segmentation

Sensitivity to the initialization

Incorrect segmentation

Strict topology preservation is

restrictive

Previous Work

1) Limitations of Segmentations under Topological Constraints

• Sensitivity to initialization

• Strict topology preservation is restrictive

cannot distinguish handles from disconnected components or cavities

Section I

Location of the topological defects is hard to controlCorrection of the defects might not be optimal

Previous Work

2) Retrospective Topology Correction of Segmentations

• Digital binary images Shattuck-Leahy-01Han-Xu-Braga-Neto-Prince-02Kriegeskorte-Goeble-01

• Triangulations Guskov-Wood-01

Fischl-Liu-Dale-01

Topological defect Inaccurate correction Accurate correction

• Necessity to integrate additional information

• Solution is not necessarily obvious

Previous Work

Difficulty of finding the correct solution!

Topological defect Sagital view Corrected defect Sagital view

1) Limitations of Retrospective Topology Correction Methods

• Do not use all the available information

corrections based on the size of the defect only

• Do not generate several potential solutions to find the best one

cutting a handle or filling the corresponding hole may not generate the valid correction

Section II

Motivation

1) Existing methods are limited

- segmentation under strict topology preservation is too restrictive.

- retrospective topology correction methods do not use of additional information and cannot guarantee optimal topological corrections.

2) Discrete topological tools are restricted

Introduction

Background

Section I - Novel Approaches in Digital Topology

Section III - Manifold Surgery

Road Map

Background

Topology in Discrete Imaging

• General Notions of Topology

• Topology in Discrete Imaging

Topology: study of shape properties preserved through deformations, twistings, and stretchings, but no tearings. [Massey 1967]

=

Background

General Notions of Topology

• Continuous theory

• Different levels of equivalence

Background

Difficult to adapt into a discrete framework

• Intrinsic Topology: - properties preserved by homeomorphisms

- ignore the embedding space

• Homotopy type: - continuous transformations in the embedding space

Euler-characteristic , genus g of a surface S with V: # vertices E: # edges F: # faces

in any polyhedral decomposition of S

Background

= V-E+F

Very useful to check the topology type of a triangulation

= 8-12+6=2 = 16-32+16=0

Euler-characteristic is a topological invariant

But no localization of the topological defectsRelated to the genus of a surface = 2-2g

Genus of a surface is equivalent to the number of handles

• Adapt concept of continuity to a discrete framework

• Surfaces and 3D images

- Tessellations

- 3D digital images

Background

Topology in Discrete Imaging

use the notion of connectivity instead

Euler-characteristic

Theory of Digital topology

Theory of Digital Topology

• Replace the notion of continuity by one of connectivity

Choice of connectivity n=8 or n=4

NEED TO WORK WITH A PAIR OF CONNECTIVITIES TO AVOID TOPOLOGICAL PARADOXES.Intersecting curves or not?

Background

One connectivity for the object X and one compatible connectivity for the inverse object X

Background

4 different pairs of compatible connectivities for the foreground F and Background B

F B

6 26

26 6

6+ 18

18 6+

n=6 n=18 n=26

( , )n n

3D digital images: 3 different types of connectivity

How to characterize the topology type of a given voxel ?

How to detect topology changes ?

• Characterize the topology type of a given voxel• Simple point: point that can be added or removed from a digital object without changing its topology. They are characterized by:

•Local computations using the 3D neighborhood only Fast

Topological numbers (see Bertrand, 1994, Pat. Rec. Let.)

Background

and n nT T

1),(),( XxTXxT nn

Homotopic deformation: addition or deletion of simple points.

Background

simple point in red – non simple point in yellow

Problem: concept of simple point cannot distinguish different topological changes

(formation of handles ≠ formation of cavities ≠ formation of disconnected components)

Homotopic deformations are often too restrictive

Section I

Novel Approaches in Digital Topology

• On the Characterization of Multisimple Points

• Genus Preserving Level Sets

Strict preservation of the topology is a strong constraint

• Notion of simple point is too restrictive

Cannot distinguish different topological changes.

• Difficult to use topology preserving methods

Sensitivity to initialization, noise, unexpected cavities, …

• Interested in handles

Handles are hard to detect and retrospectively correct.

Disconnected components, cavities are less problematic.

Requires an extension of the concept of simple point

On the Characterization of Multisimple Points

Goal: Develop digital techniques to prevent formation of handles but not the generation of cavities or of disconnected components.

Definition: a multisimple point is a point that can be added or removed from a digital object X without changing the number of holes in X or the complement .


X

Need to find a characterization of multisimple points.


Can we use the topological numbers to characterize different topological changes ?

Topology type0 Isolated point

0 Interior point

1 1 Border point ( = Simple Point )

2 1 Curve point

>2 1 Curves junction

1 2 Surface point

1 >2 Surfaces junction

>1 >1 Surface(s)-curve(s) junction

nTnT

Voxel topology type and topological numbers

)18,6(),( nn

1),(0),( XxTXxT nn

1),(2),( XxTXxT nn

• Topological numbers are not able to characterize multisimple points

• Topological numbers are computed locally: no information about the global connectivity of the digital object

extension of the concept of Topological Numbers

• We introduce the set of neighboring components of a point x:

• We define two Extended Topological Numbers:

),( XxCn

( , ) and ( , )n n n nT C x X T C x X


How to integrate information about the global connectivity of the digital object ?


A point is multisimple if and only if:

andn n n nT T T T

Necessary and Sufficient Condition for a Point to be Multisimple ?

• Simple characterization of multisimple points.

• ‘Low’ computational complexity.

• deformations that do not generate/suppress any handles in the digital objects


1 2

1 1

n n

n n

T T

T T

2 2

1 1

n n

n n

T T

T T

Example 1),(2),( XxTXxT nn

• Computational complexity - Implementation using a grid of labels

assign a label to each component

- local computations using the grid of labels

merging = assigning the same label

splitting = assigning different labels (using region growing algorithms)

• New sets of transformations under topology control - Preservation of the genus

Components can split/merge/appear/disappear without generating any handles


Topology constrained Topology controlled with one single initial component

Topology controlled with several initial components

simple point

non simple point

Multisimple point

1) Multisimple Points preserve the genus of a digital object

2) Level sets require an underlying digital image to be implemented

The negative/positive grid points of a level set define set of connected components of a digital object

Apply the concept of multisimple point to

level sets in order to design a

Genus Preserving Level Set Framework

Genus Preserving Level Sets


Level set evolution equation Level set evolution equation with outward normal speed with outward normal speed ββ::

What is the level set method ?

Representing a manifold implicitly as the zero level set of a Representing a manifold implicitly as the zero level set of a higher-dimensional functionhigher-dimensional function (Osher-Sethian, 1988) (Osher-Sethian, 1988)

+ No parameterization

+ Easy computation of intrinsic geometric properties (e.g. normal, curvature)

+ Mathematical proofs and numerical stability

+ Topology changes handled automatically

+ No self-intersection in the resulting mesh

- Computationally expensive

narrow band algorithm

- Need for an isocontour extraction step

marching cubes algorithm

Level sets vs. explicit representations


Why using the level set methods ?


How to preserve the topology of level sets ?

Topology Preserving Level Sets (Han, Xu, Prince, 2003)Algorithm: for each grid point at each time step,- Compute the new value of the level set- Sign is about to change?

No: accept the new valueYes: check if the point is simple

SIMPLE: accept the new valueCOMPLEX: set +/-ε as the new level set value

Initial conditionInitial condition Traditional level Traditional level setssets

Topological level setsTopological level sets

+ Local computations only FAST Strict Preservation of the topology RESTRICTIVE

Algorithm: for each grid point at each time step,- Compute the new value of the level set- Sign is about to change?

No: accept the new valueYes: check if the point is simple- SIMPLE: accept the new value- COMPLEX: set +/-ε as the new level set value



Strict Topology Preservation

Algorithm: for each grid point at each time step,- Compute the new value of the level set- Sign is about to change?

No: accept the new valueYes: check if the point is simple- SIMPLE: accept the new value- COMPLEX: set +/-ε as the new level set value

- Compute the set of neighboring connected components from the grid of labels L- Check if the point is multisimpleMULTISIMPLE: accept the new value

update the grid of labels LCOMPLEX: set +/-ε as the new level set value


velocity field ( , ) ( ( ) ) ( , ) ( , )

( , ) 1Euler-Lagrange equation ( ) div( )

1

thres

thres

t I I H t t

tI I

t n

v x x x N x

x

Experiments: Simple Segmentation Tasks

with I(x) : intensity in the image at location x

Ithres : global intensity threshold

H(x,t) : mean curvature of active contour at location x

N(x,t) : normal of active contour at location x

: mixing parameter

Data termSmoother surfaces


Traditional level sets Topology preserving level sets Genus preserving level sets

Segmentation of a synthetic ‘C’ shape

Trade-off between traditional level-sets and topology preserving

level-sets

Less sensitive to initial conditions


Segmentation of blood vessels from MRA

Segmentation of blood vessels from MRA under two different initial conditions

movie


Topology preserving level sets Genus preserving level sets

OTHER EXAMPLES: square with cavities cortical segmentation initialized with 100 seeds

Advantages of Genus Preserving Level Sets


• Preserve the number of handles in the volume

• Components can be created or destroyed, can merge or split

• Less sensitive to initial conditions

• Less sensitive to noise/unexpected structures in the image

• ‘Low’ computational complexity (except for splits)

Section I: Contributions

Novel Approaches in Digital Topology

• Concept of Multisimple Point

• New sets of digital transformations

• Genus Preserving Level Sets

Section II

Manifold Surgery

Topology Correction of Cortical Surfaces

Surfaces with the same Euler-characteristic are homeomorphic

Manifold Surgery : Position of the problem

Segmentation under topological constraints is a difficult problem: partial voluming effect, noise, bias field, image inhomogeneity, …

many topological defects (handles) in the resulting cortical surface

GOAL: given a cortical surface C, detect and optimally correct the topological defects D

Manifold Surgery

• Approach

• Location of topological defects

• Optimal topology correction

Shrink-Wrap Methods cannot reach deep folds

Quasi-Homeomorphic mapping to locate and

correct the defects

Manifold Surgery

Manifold Surgery : Approach

Homeomorphic mapping M:C S– Transformation that is continuous, one-to-one with continuous

inverse M-1

– Jacobian is positive definite: JM > 0

– Jacobian is related to the areal distortion

– For triangulations, the areal distortion is an approximation of the Jacobian

S M CdA J dA

SM

C

AJ

A

CA SA

Manifold Surgery

Definition: Homeomorphism

Manifold Surgery

Manifold Surgery

• Approach



• Cortical surface C with correct topology is homeomorphic to

the sphere S:

- continuous, one-to-one, continuous inverse

- strictly positive Jacobian

• In the presence of topological defects (i.e. handles), no such mapping exists.

Search for a mapping that minimizes the regions with negative Jacobian

Quasi-homeomorphic mapping = maximally homeomorphic

Manifold Surgery

• Try to find a mapping that is maximally homeomorphic

The areal distortion is an approximation of the Jacobian

Quasi-Homeomorphic mapping

SM

C

AJ

A

Generate a mapping that minimizes the

regions with negative area

th

th

area of i face in at

area of i face in

ti

ti

A S t

A C

1

log(1 )with

ikR tFi

M i i oi i

AeE R R

k A

1) Initialize the mapping by inflation and projection

2) Minimize EM by gradient descent

Manifold Surgery

Location of topological defects

different steps

- inflation

- projection

- minimization of

negative regions

- defect = set of

overlapping faces

- back-projection

Manifold Surgery

Location of topological defects

Cortical surface with incorrect topology

Topological defect

Spherical projection

Manifold Surgery

Manifold Surgery

• Approach



The retessellation problem

defect in original cortical surface

corresponding spherical projection of defect

quasi-homeomorphi

c mapping

1) Discard all faces and edges in each defect D

2) Find a valid retessellation D for each defect D

Manifold Surgery

Finding a valid retessellation D for each defect D

- Geometrically Accurate, i.e. smoothness, location, self-intersection

Use Spherical Representation S

Use Cortical Representation C

- Geometrically Accurate, i.e. smoothness, location, self-intersection

Manifold Surgery

The retessellation problem

Constraining the Topology on S

Discard edges and faces

Retessellate

Find a retessellation with no self-intersecting edges on the sphere

D = 1

Manifold Surgery

Many potential retessellations exist!

Sedges={ …} ={e0 ,e1 ,…,eN} set of all potential edges

where N=n(n-1)/2 and n is the number of vertices in the defect

One candidate retessellation corresponds

to an ordering OD of the set Sedges

Retessellating = iteratively adding edges


Manifold Surgery

Retessellating = Knitting on the sphere

Edge ordering OD ={ …}

Different edge orderings (permutations) will generate different configurations in the original

cortical space


Manifold Surgery

Measuring the accuracy of D on C

• No self-intersection

• Smoothness of D

• MRI intensity I profile inside C - and outside C +

( | , ) ( | , ) ( | )

Likelihood PriorD D D

pT C I p I C T pT C

Bayesian Framework

Manifold Surgery

Prior = smoothness

Likelihood = MRI intensity profile

#vertices

1 21

( | ) p( (v), (v)| )D

i

pT C C

#vertices

i i1

( | , ) ( ( ) | , )

( ( ) | , )

p(g(v),w (v)| , )

wD Dx C

g Dx C

Di

p I C T p I x C T

p I x C T

C T

1 2, principal curvatures

volume

surface

Measuring the accuracy of on C

Manifold Surgery

Huge space to be searched!Discrete space No gradient information

Search : Genetic Algorithm

defect with n vertices N=n(n-1)/2 potential edges

For a patch, v-e+f=1 # of used edges 3n

Size of Space of potential retessellations

2

3

2

n

n

Manifold Surgery

Use of a Genetic Algorithm to explore the space of potential retessellations OD

• introduced in the 60s by John Holland as a way to import the mechanism of natural adaptation into computer algorithm and numerical optimization

• candidate solution = chromosome

• Fitness function = measure the evolutionary viability of each chromosome

• Genetic operations = mutations, crossovers

representation = edge ordering OD

posterior probability

Search : Genetic Algorithm

Manifold Surgery

• Mutations : swap intersecting edges in OD with probability pmut

• Crossovers : randomly combine two orderings by iteratively adding edges from each of them

• Parameters :population = 20 chromosomes

elite population = 40% - mutation = 30% - crossovers = 30%

pmut= 0.1 – stop after 10 populations without any change

Genetic OperationsManifold Surgery

Results: real MRI datasets• applied to hundred of brains (part of FreeSurfer)

typical brain segmentation = 40 defects (~50 vertices)

average time = ~3 hours (~ 5 minutes / defect)

• validation on 35 brains (70 hemispheres)

comparison with expert Hausdorff distance < 0.2mm

Manifold Surgery

Topological defect

Corrected defect

Results: real MRI datasets

Manifold Surgery

Topological defect Corrected defectInitial cortical surface

Sagital view Coronal view


Manifold Surgery

Topological defect

Corrected defect


Manifold Surgery

Convergence• Genetic Search versus Random Search

- Boost up average fitness of population- Speed up convergence by a log factor of 2

- Converge in few iterations

Manifold Surgery

Section II: Contributions

Manifold Surgery

• Method for optimally correcting the topology of cortical surfaces

• Genetic algorithm for solving the retessellation problem

Manifold Surgery

CONCLUSION

TO BE DONE…

IT”S GREAT WORK !

Manifold Surgery

Approach: retrospective topology correction of the cortical surface C

orig MRI segmentation

tessellate

surface locate defects correct defects

skull strip

spherical mapping

Discarding vertices on S

Spherical mapping

C

S

All vertices are included in the retessellation jagged surfaces

Solution: use the iterative retessellation to eliminate late vertices

During the retessellation, vertices that are included inside temporary triangles are discarded from the

tessellation

The space of all potential retessellations depends on the spherical mapping

Limitations: the mapping problem!

Spherical mapping

C

S Discard faces and edges

impossible

Retessellate

The mapping problem!

Solution: Generate several mappings corresponding to different configuration

- identify defects with large handles (geodesic distances)

- cluster proximal vertices

- generate several mappings using spring term force


Example

Topological defect

Spherical representation

Sagital view of topological defect

The mapping problem

Example

Topological defect Corrected defect

Spherical representation


Solution: Generate several mappings corresponding to different configuration

- spherical location is not appropriate

circle in 2D


- cluster proximal vertices in original space

- generate different mappings using a spring term force

The mapping problem

Original defect Final configuration

Example

Results: synthetic data

segmentation of medical images under topology constraints florent ségonne computer science and...

Documents

topological notions

topological defect sagital

spherical topology handles

correct solution

tissue type

sereno et

segmentation of medical

tissue inhomogeneities