2. tumor growth calculations imagesegmentation&
TRANSCRIPT
ì Image Segmentation Introduc*on
Radiotherapy treatment planning based on a tumor growth model 5
2. Tumor growth calculations
The purpose of the tumor growth model amounts to predicting the density of tumor cells
in the regions of the brain that appear normal on MRI. It is assumed that the growth
of the tumor is described phenomenologically by local proliferation of tumor cells and
migration into neighboring tissue.
(a) (b)
Figure 1. (a) post-gadolinium T1 weighted image of a glioblastoma located in theleft parietal lobe. (b) FLAIR image of the same tumor, showing the surroundingperitumoral edema. (Note that the right side of the image corresponds to the left sideof the brain to follow brain imaging conventions.)
2.1. Patient specific input data
The tumor growth calculations are based on MR images routinely acquired in clinical
practice. The images incorporated into the simulation process are T1, T2, FLAIR, and
T1 post gadolinium. The T1 post gadolinium image shows the vascularized gross tumor
volume, and the FLAIR image shows the surrounding edematous region. For illustrative
purpose, these two images are shown in figure 1 for a case discussed in detail in this
paper.
2.2. Data processing
In the first data processing step, all MR sequences are registered to the image with
highest spatial resolution. This is done using affine registration with 12 degrees of
freedom, utilizing the function FLIRT [23] as part of the toolbox FSL [24, 25]. In
the second step, a segmentation of the brain was obtained using the multimodal brain
tumor segmentation algorithm published in [26]. The algorithm is an Expectation-
Maximization based segmentation method, which uses a probabilistic normal tissue
Radiotherapy treatment planning based on a tumor growth model 5
2. Tumor growth calculations
The purpose of the tumor growth model amounts to predicting the density of tumor cells
in the regions of the brain that appear normal on MRI. It is assumed that the growth
of the tumor is described phenomenologically by local proliferation of tumor cells and
migration into neighboring tissue.
(a) (b)
Figure 1. (a) post-gadolinium T1 weighted image of a glioblastoma located in theleft parietal lobe. (b) FLAIR image of the same tumor, showing the surroundingperitumoral edema. (Note that the right side of the image corresponds to the left sideof the brain to follow brain imaging conventions.)
2.1. Patient specific input data
The tumor growth calculations are based on MR images routinely acquired in clinical
practice. The images incorporated into the simulation process are T1, T2, FLAIR, and
T1 post gadolinium. The T1 post gadolinium image shows the vascularized gross tumor
volume, and the FLAIR image shows the surrounding edematous region. For illustrative
purpose, these two images are shown in figure 1 for a case discussed in detail in this
paper.
2.2. Data processing
In the first data processing step, all MR sequences are registered to the image with
highest spatial resolution. This is done using affine registration with 12 degrees of
freedom, utilizing the function FLIRT [23] as part of the toolbox FSL [24, 25]. In
the second step, a segmentation of the brain was obtained using the multimodal brain
tumor segmentation algorithm published in [26]. The algorithm is an Expectation-
Maximization based segmentation method, which uses a probabilistic normal tissue
Radiotherapy treatment planning based on a tumor growth model 6
atlas as spatial tissue prior. For every voxel, it estimates the posterior probability for
three normal tissue classes (white matter, gray matter, and CSF‡), as well as the lesionoutlines on T1 post gadolinium and FLAIR. The result of the automatic segmentation
is inspected visually and minor corrections were performed in a post-processing step if
necessary. The segmentation result for the patient in figure 1 is shown in figure 2. In the
last step, the reference MR image is registered to the radiotherapy planning CT using
affine registration. The transformation matrix is saved and later applied to register the
simulated tumor cell density to the planning CT.
Figure 2. Segmentation of the brain into contrast enhancing core (white), peritumoraledema, white matter, gray matter, and CSF (black).
2.3. Underlying tumor growth model
It is assumed that tumor growth is described by two processes: local proliferation
of tumor cells and migration of cells into neighboring brain tissue. Mathematically,
this is formalized via the Fisher-Kolmogorov equation, a partial differential equation of
reaction-diffusion type for the tumor cell density c(r, t) as a function of location r and
time t:∂
∂tc(r, t) = ∇ · (D(r)∇c(r, t)) + ρc(r, t)
�1− c(r, t)
cmax
�(1)
where ρ is the proliferation rate which is assumed to be spatially constant, and D(r)
is the 3 × 3 diffusion tensor which depends on location r. The first term on the right
hand side of equation (1) is the diffusion term that models tumor cell migration into
neighboring tissue. The second term is a logistic growth term that describes tumor cell
proliferation. The tumor cell density c(t, r) takes values between zero and the carrying
capacity cmax. In this paper, the diffusion tensor is constructed as
D(r) =
�Dw · I r ∈ white matter
Dg · I r ∈ gray matter(2)
‡ In this work, we refer to all brain tissue that is neither white matter, gray matter, nor tumor as CSF.
Introduction
• segmenta*on problem is reduced to a registra*on problem
1. Atlas based segmenta/on
3. Segmenta/on as a classifica/on problem • assign each voxel to a *ssue class • Example: EM segmenta*on
2. Boundary detec/on • ac*ve contour models • graph cut algorithms • region growing
Approaches to image segmenta/on … many
EM-‐segmentation
Overview: Problem formula/on:
• segmenta*on as classifica*on problem • Example: brain *ssue segmenta*on
Probabilis/c genera/ve model of the data: • Gaussian mixture model • Expecta*on-‐Maximiza*on training algorithm
Incorpora/ng spa/al context: • Markov Random Field regulariza*on
Incorpora/ng prior knowledge:
• probabilis*c atlas as *ssue prior
Problem formulation
Mo/va/on:
3 main *ssue classes in the brain
• white maCer • gray maCer • cerebrospinal fluid (CSF)
These *ssues have characteris*c intensity values on MR images
• The voxels belonging to the same *ssue class form clusters in intensity space
+ one background class
Clusters in intensity space
Consider T1 and T2 image: 2-‐dimensional intensity space
Clusters in intensity space
white maCer
gray maCer
background
CSF
Problem formulation
Every voxel is a data point in M-‐dimensional image intensity space:
xi !"M
(M = number of images, typically M = 1, …, 10)
Voxel i is data point
Assump*on:
• the gray values in each *ssue class are Gaussian distributed • A voxel i belongs to *ssue class k with probability πk
Assume we have K *ssue classes
• for now: every voxel independent
(typically K = 4)
Gaussian mixture model
Probabilis/c genera/ve model of the data:
P x |!( ) = ! kP x µk,!k( )k=1
K
"
= ! k1
2!( )D/2 det(!k )exp "
12(x "µk )
T !k"1(x "µk )
#
$%
&
'(
)
*++
,
-..k=1
K
/
Mul*variate Gaussian Model
mixing coefficients
! = " k,µk,!k{ }k=1K
Parameters:
cluster means
covariance matrices
Gaussian mixture model
We need to do two things:
1. Learning the model parameters, given the data (i.e. the set of images)
è Maximum Likelihood es*mate
2. Inference = assigning a voxel to a /ssue class (i.e. segmenta*on)
è Calculate posterior probability of the *ssue class given a gray value vector and the model
Maximum likelihood estimate
L = P {xi}i=1N | ! k,µk,!k{ }k=1
K( ) = ! kP xi µk,!k( )k=1
K
"#
$%
&
'(
i=1
N
)
logL = logP {xi}i=1N | ! k,µk,!k{ }k=1
K( ) = log ! kP xi µk,!k( )k=1
K
"#
$%
&
'(
i=1
N
"
Maximum Likelihood es/mate Maximize the probability that the model generates the observed data
Likelihood function
Log-Likelihood
Towards EM
! * = argmax!
log " kP xi µk,!k( )k=1
K
"#
$%
&
'(
i=1
N
"
Difficulty:
• logarithm of a sum cannot be simplified • no analy*c solu*on
Towards EM
! * = argmax!
log " kP xi µk,!k( )k=1
K
"#
$%
&
'(
i=1
N
"
Recall: things would be easy without the sum
= argmax!
log 12"( )D/2 det(!k )
exp "12(xi "µk )
T !k"1(xi "µk )
#
$%
&
'(
)
*++
,
-..i=1
N
/
Towards EM
! * = argmax!
log " kP xi µk,!k( )k=1
K
"#
$%
&
'(
i=1
N
"
Recall: things would be easy without the sum
= argmax!
log 12"( )D/2 det(!k )
exp "12(xi "µk )
T !k"1(xi "µk )
#
$%
&
'(
)
*++
,
-..i=1
N
/
becomes a quadra*c func*on of the model parameters
Towards EM
µ =1N
xii=1
N
!
! =1N
xi "µ( ) xi "µ( )Ti=1
N
#
Analy/c solu/on for mean and covariance of a Gaussian
Latent variables
Reformulate problem:
Introducing latent variables
zik =1 (xigenerated by component k)0 else
!"#
$#
l introduce binary assignment variables
l observed data X: the data points xi
l latent variables Z: zik
Complete date likelihood
P X,Z |!( ) = ! kN xi µk,!k( )"# $%k=1
K
&i=1
N
&zik
P xi | zi,!( )
Likelihood of the joint distribu/on
Log-‐Likelihood:
P zi =1( )
logP X,Z |!( ) = zik log! k + logN xi µk,!k( )"# $%k=1
K
&i=1
N
&
EM
Intui/on:
logP X,Z |!( ) = zik log! k + logN xi µk,!k( )"# $%k=1
K
&i=1
N
&
1. For any fixed Z, op*mizing the model parameters (π,μ,Σ) becomes easy
2. For any fixed (π,μ,Σ), we can calculate the expected value of Z
Idea: replace zik by E[zik |θ] and itera*vely es*mate E[zik |θ] and op*mize θ=(π,μ,Σ)
EM algorithm for GMM
EM algorithm
E-Step: Evaluate zik = probabilitiy that data point xi was generated by component k (for the current parameter estimates θold)
M-Step: obtain new parameter estimates θnew
E[zik |!old ]=
" koldN xi |µk
old,!kold( )
! joldN xi |µ j
old,! jold( )
j"
! new = argmax!
E[zik |!old ] log" k + logN xi µk,!k( )"# $%
k=1
K
&i=1
N
&"
#'
$
%(
(can be done analytically)
Maximization step
M-Step: obtain new parameter estimates
! new = argmax!
E[zik |!old ] log" k + logN xi µk,!k( )"# $%
k=1
K
&i=1
N
&"
#'
$
%(
µknew =
1Nk
E[zik |!old ]
i=1
N
! xi
!knew =
1Nk
E[zik |!old ]
i=1
N
" xi #µknew( ) xi #µk
new( )T
! knew =
Nk
N
Nk = E[zik |!old ]
i=1
N
!
Solution
where
The general EM algorithm
P X,Z |!( ) = ! kN xi µk,!k( )"# $%k=1
K
&i=1
N
&zik
This is an instan/a/on of an EM algorithm
Marginalize over unknown Z
P X |!( ) = P X,Z |!( )Z! = ! kN xi µk,"k( )#$ %&
k=1
K
'zi
!i=1
N
'zik
= ! kN xi µk,!k( )k=1
K
"#
$%
&
'(
i=1
N
)
Complete data likelihood:
• observed data X • latent variables Z
This is our original likelihood func*on!
The general EM algorithm
E[zik |!old ]=
" koldN xi |µk
old,!kold( )
! joldN xi |µ j
old,! jold( )
j"
Reinterpret expectation of zαi
= P zik =1| xi,!old( )
Posterior probability of zik=1 given the model and the data
The general EM algorithm
LOOP
STOP if convergence criterion met
INITIALIZE θold
1) E-Step Evaluate the posterior probability P(Z|X,θold) given the current parameters
( )⎥⎦
⎤⎢⎣
⎡= ∑
Z
oldnew ZXPXZP )|,(log),|(maxarg θθθθ
2) M-Step Evaluate θnew given by
3) set θold ← θnew
( )),()|,(,| old
oldold
XPZXPXZPθθ
θ =
The general EM algorithm
Convergence:
The algorithm is proven to converge to a local maximum of the likelihood func*on
l In this general form, the EM rather refers to a meta-algorithm or class of related algorithms (as opposed to a specialized algorithm to solve a specific problem)
l A “good” instantiation of an EM algorithm is one where both the M-step and the E-step are relatively simple (with a closed form solution in the ideal case)
)|,()|( θθ ∑=Z
ZXPXP
(Lit: Bishop, chapter 9.4)
Remarks:
Example Radiotherapy treatment planning based on a tumor growth model 5
2. Tumor growth calculations
The purpose of the tumor growth model amounts to predicting the density of tumor cells
in the regions of the brain that appear normal on MRI. It is assumed that the growth
of the tumor is described phenomenologically by local proliferation of tumor cells and
migration into neighboring tissue.
(a) (b)
Figure 1. (a) post-gadolinium T1 weighted image of a glioblastoma located in theleft parietal lobe. (b) FLAIR image of the same tumor, showing the surroundingperitumoral edema. (Note that the right side of the image corresponds to the left sideof the brain to follow brain imaging conventions.)
2.1. Patient specific input data
The tumor growth calculations are based on MR images routinely acquired in clinical
practice. The images incorporated into the simulation process are T1, T2, FLAIR, and
T1 post gadolinium. The T1 post gadolinium image shows the vascularized gross tumor
volume, and the FLAIR image shows the surrounding edematous region. For illustrative
purpose, these two images are shown in figure 1 for a case discussed in detail in this
paper.
2.2. Data processing
In the first data processing step, all MR sequences are registered to the image with
highest spatial resolution. This is done using affine registration with 12 degrees of
freedom, utilizing the function FLIRT [23] as part of the toolbox FSL [24, 25]. In
the second step, a segmentation of the brain was obtained using the multimodal brain
tumor segmentation algorithm published in [26]. The algorithm is an Expectation-
Maximization based segmentation method, which uses a probabilistic normal tissue
Radiotherapy treatment planning based on a tumor growth model 5
2. Tumor growth calculations
The purpose of the tumor growth model amounts to predicting the density of tumor cells
in the regions of the brain that appear normal on MRI. It is assumed that the growth
of the tumor is described phenomenologically by local proliferation of tumor cells and
migration into neighboring tissue.
(a) (b)
Figure 1. (a) post-gadolinium T1 weighted image of a glioblastoma located in theleft parietal lobe. (b) FLAIR image of the same tumor, showing the surroundingperitumoral edema. (Note that the right side of the image corresponds to the left sideof the brain to follow brain imaging conventions.)
2.1. Patient specific input data
The tumor growth calculations are based on MR images routinely acquired in clinical
practice. The images incorporated into the simulation process are T1, T2, FLAIR, and
T1 post gadolinium. The T1 post gadolinium image shows the vascularized gross tumor
volume, and the FLAIR image shows the surrounding edematous region. For illustrative
purpose, these two images are shown in figure 1 for a case discussed in detail in this
paper.
2.2. Data processing
In the first data processing step, all MR sequences are registered to the image with
highest spatial resolution. This is done using affine registration with 12 degrees of
freedom, utilizing the function FLIRT [23] as part of the toolbox FSL [24, 25]. In
the second step, a segmentation of the brain was obtained using the multimodal brain
tumor segmentation algorithm published in [26]. The algorithm is an Expectation-
Maximization based segmentation method, which uses a probabilistic normal tissue
Radiotherapy treatment planning based on a tumor growth model 6
atlas as spatial tissue prior. For every voxel, it estimates the posterior probability for
three normal tissue classes (white matter, gray matter, and CSF‡), as well as the lesionoutlines on T1 post gadolinium and FLAIR. The result of the automatic segmentation
is inspected visually and minor corrections were performed in a post-processing step if
necessary. The segmentation result for the patient in figure 1 is shown in figure 2. In the
last step, the reference MR image is registered to the radiotherapy planning CT using
affine registration. The transformation matrix is saved and later applied to register the
simulated tumor cell density to the planning CT.
Figure 2. Segmentation of the brain into contrast enhancing core (white), peritumoraledema, white matter, gray matter, and CSF (black).
2.3. Underlying tumor growth model
It is assumed that tumor growth is described by two processes: local proliferation
of tumor cells and migration of cells into neighboring brain tissue. Mathematically,
this is formalized via the Fisher-Kolmogorov equation, a partial differential equation of
reaction-diffusion type for the tumor cell density c(r, t) as a function of location r and
time t:∂
∂tc(r, t) = ∇ · (D(r)∇c(r, t)) + ρc(r, t)
�1− c(r, t)
cmax
�(1)
where ρ is the proliferation rate which is assumed to be spatially constant, and D(r)
is the 3 × 3 diffusion tensor which depends on location r. The first term on the right
hand side of equation (1) is the diffusion term that models tumor cell migration into
neighboring tissue. The second term is a logistic growth term that describes tumor cell
proliferation. The tumor cell density c(t, r) takes values between zero and the carrying
capacity cmax. In this paper, the diffusion tensor is constructed as
D(r) =
�Dw · I r ∈ white matter
Dg · I r ∈ gray matter(2)
‡ In this work, we refer to all brain tissue that is neither white matter, gray matter, nor tumor as CSF.
Summary
Consider segmenta/on as a classifica/on problem
• each voxel is assigned to one of K *ssue classes • given: gray value features in M images
Here: considered Gaussian Mixture model (GMM) as genera/ve model of the image histogram
Segmenta*on consists in simultaneously es*ma*ng
Next /me:
1. *ssue class probabili*es E[zik] 2. cluster means, variances, mixing coefficients
• spa*al *ssue priors through probabilis*c atlas • spa*al context through Markov Random Field