Fast Segmentation of Vessels in MR Liver Images using Patient Specific Models

by

Sameer Zaheer

A thesis submitted in conformity with the requirements for the degree of Master’s of Health Science

Institute of Biomaterials and Biomedical Engineering University of Toronto

© Copyright by Sameer Zaheer 2013

ii

Fast Segmentation of Vessels in MR Liver Images using Patient

Specific Models

Sameer Zaheer

Master’s of Health Science

Institute of Biomaterials and Biomedical Engineering

University of Toronto

2013

Abstract

Image-guided therapies have the potential to improve the accuracy of treating liver cancer. In

order to register intraoperative with preoperative liver images, joint segmentation and

registration methods require fast segmentation of matching vessel centerlines. The algorithm

presented in this thesis solves this problem by tracking the centerlines using ridge and cross-

section information, and uses knowledge of the patient’s vasculature in the preoperative image to

ensure correspondence. The algorithm was tested on three MR images of healthy volunteers and

one CT image of a patient with liver cancer. Results show that in the context of join

segmentation registration, if the registration error is less than 2.0mm, the average segmentation

error is 0.73-1.68mm, with 88-100% of the vessels having an error less than a voxel length. For

registration error less than 4.6mm, the average segmentation error is 1.17-2.11mm, with 79-98%

of the vessels having an error less than a voxel length.

iii

Acknowledgments

I would like to thank Dr. James Drake for his supervision and support. I would like to

thank Dr. (Edward) Xishi Huang for his supervision and patiently teaching me important

concepts in mathematics, imaging and programming. I would like to thank my committee

members Dr. Allan Jepson and Dr. Walid Farhat for their constructive criticism. I would like to

thank Dr. Brian Carillo for his help in writing. I would like to thank Anwar Bari for helping me

process data. I would like to thank CIGITI members for their support.

I am grateful to my parents for always pushing me. I am grateful to my sisters and my

friends for their encouragement. I am grateful to God Almighty for giving me strength.

iv

Table of Contents

Acknowledgments .......................................................................................................................... iii

Table of Contents ........................................................................................................................... iv

List of Tables ................................................................................................................................ vii

List of Figures .............................................................................................................................. viii

Chapter 1 ......................................................................................................................................... 1

1 Introduction ................................................................................................................................ 1

1.1 Liver anatomy ..................................................................................................................... 1

1.2 Liver carcinoma .................................................................................................................. 2

1.3 Imaging liver cancer ........................................................................................................... 3

1.3.1 Ultrasonography ...................................................................................................... 4

1.3.2 Computed Tomography .......................................................................................... 4

1.3.3 Magnetic Resonance Imaging ................................................................................. 5

1.3.4 Comparison of the imaging modalities ................................................................... 7

1.4 Role of imaging in interventions ......................................................................................... 8

1.5 Image registration ............................................................................................................... 9

1.6 Thesis outline .................................................................................................................... 12

Chapter 2 ....................................................................................................................................... 13

2 Previous Work and Research Objective ................................................................................... 13

2.1 Vessel Segmentation ......................................................................................................... 13

2.1.1 Active Contours .................................................................................................... 14

2.1.2 Region-Growing ................................................................................................... 14

2.1.3 Centerline Tracking .............................................................................................. 15

2.2 Vessel Matching ................................................................................................................ 16

2.3 Research Objective ........................................................................................................... 17

v

Chapter 3 ....................................................................................................................................... 19

3 Algorithm for vessel segmentation .......................................................................................... 19

3.1 Vessel modeling ................................................................................................................ 19

3.1.1 Cross-Section ........................................................................................................ 20

3.1.2 Direction ............................................................................................................... 21

3.2 Centerline tracking ............................................................................................................ 23

3.2.1 Mathematical Representation of the Image .......................................................... 23

3.2.2 Vessel Center Likelihood ...................................................................................... 24

3.2.3 Generating Seed Point ........................................................................................... 25

3.2.4 Computing the Next Point .................................................................................... 26

3.2.5 Termination Condition .......................................................................................... 27

3.2.6 Radius Computation .............................................................................................. 28

3.3 Orientation Correction ...................................................................................................... 28

3.4 Bifurcation Correction ...................................................................................................... 29

Chapter 4 ....................................................................................................................................... 33

4 Experiments, Results and Discussion ...................................................................................... 33

4.1 Experiments ...................................................................................................................... 33

4.1.1 Image Acquisition ................................................................................................. 33

4.1.2 Guidance Model .................................................................................................... 33

4.2 Results and Discussion ..................................................................................................... 34

4.2.1 Subject A ............................................................................................................... 36

4.2.2 Subject B ............................................................................................................... 42

4.2.3 Subject C ............................................................................................................... 46

4.2.4 Subject D ............................................................................................................... 50

4.3 Accuracy ........................................................................................................................... 55

4.4 Speed ................................................................................................................................. 57

vi

Chapter 5 ....................................................................................................................................... 58

5 Conclusion and Future Direction ............................................................................................. 58

5.1 Contribution of This Thesis .............................................................................................. 58

5.2 Other clinical applications ................................................................................................ 59

5.3 Future Work ...................................................................................................................... 60

References ..................................................................................................................................... 61

vii

List of Tables

Table 1.1 The different phases of acquiring contrast enhanced images of the liver ....................... 5

Table 3.1 The description of the shape being described for various combinations of eigenvalues.

....................................................................................................................................................... 22

Table 4.1 MR images acquired using the following sequences and resolutions. ......................... 33

Table 4.2 Summary of the number of vessels segmented in each data set, the number of different

guidance models used on each image, and the total number of individual vessel segmentations 34

Table 4.3 Summary of accuracy and speed results of the algorithm. ........................................... 35

viii

List of Figures

Figure 1.1 The vascular anatomy of the liver. ................................................................................ 2

Figure 1.2 Example of a workflow diagram of image-guided treatment administration. ............. 10

Figure 1.3 Huang et al.’s joint registration and segmentation method to iteratively obtain an

accurate registration. ..................................................................................................................... 11

Figure 3.1 Ray-casting to determine center of cross-section.. ...................................................... 24

Figure 3.2 Illustration of the centerline extraction process. .......................................................... 25

Figure 3.3 Tracking the next point in the algorithm. .................................................................... 27

Figure 3.4 Illustration of the centerline extraction process. .......................................................... 28

Figure 3.5 Points used to compute bifurcation point.. .................................................................. 30

Figure 3.6 Corrected centerlines (dark blue dots and lines) at a branching point, after the new

point of bifurcation (green square) has been computed. ............................................................... 32

Figure 4.1 An example of the vessel centerline segmentation results (green) compared with the

gold standard (red) for the image of subject A. ............................................................................ 36

Figure 4.2 The SE vs. ME for all vessel centerlines segmented for subject A (length) ............... 37

Figure 4.3 The SE vs. ME for all vessel centerlines segmented for subject A (radius)................ 38

Figure 4.4 Percentage of vessels with an average error in each category vs. maximum ME for

subject A. ...................................................................................................................................... 39

Figure 4.5 The SE vs. ME for all bifurcation points segmented for subject A.. ........................... 40

Figure 4.6 Percentage of bifurcation points with an average error in each category vs. maximum

ME for subject A. .......................................................................................................................... 41

ix

Figure 4.7 An example of the vessel centerline segmentation results (green) compared with the

gold standard (red) for the image of subject B. ............................................................................ 42

Figure 4.8 The SE vs. ME for all vessel centerlines for segmented for subject B (length) .......... 43

Figure 4.9 The SE vs. ME for all vessel centerlines for segmented for subject B (radius). ......... 44

Figure 4.10 Percentage of vessels with an average error in each category vs. maximum ME for

subject B. ....................................................................................................................................... 45

Figure 4.11 An example of the vessel centerline segmentation results (green) compared with the

gold standard (red) for the image of subject C. ............................................................................ 46

Figure 4.12 The SE vs. ME for all vessel centerlines for segmented for subject C (length). ....... 47

Figure 4.13 The SE vs. ME for all vessel centerlines for segmented for subject C (radius).. ...... 48

Figure 4.14 Percentage of vessels with an average error in each category vs. maximum ME for

subject C. ....................................................................................................................................... 49

Figure 4.15 Subject D, a pediatric patient with a lesion in the liver. ............................................ 50

Figure 4.16 An example of the vessel centerline segmentation results (green) compared with the

gold standard (red) for the image of subject D. ............................................................................ 51

Figure 4.17 A close-up of the vessel centerline segmentation results (green) compared with the

gold standard (red) for the image of subject D. ............................................................................ 52

Figure 4.18 The SE vs. ME for all vessel centerlines for segmented for subject D (length). ....... 53

Figure 4.19 The SE vs. ME for all vessel centerlines for segmented for subject D (radius).. ...... 54

Figure 4.20 Percentage of vessels with an average error in each category vs. maximum ME for

subject D. ...................................................................................................................................... 55

1

Chapter 1

1 Introduction

Liver cancer is one of the leading causes of cancer related deaths in the world. Medical

imaging plays not only a diagnostic role, but can also be used to more accurately guide

treatments. At heart of image-guided interventions is the registration of preoperative and

intraoperative images. Fast and accurate vessel segmentation is often a critical step in clinically

relevant registration methods. Many segmentation techniques have been previously explored, but

they have not been fast robust or automated. These problems could be overcome by making use

of a priori information about the vasculature. This thesis proposes a ridge-tracking algorithm that

makes use of a previously segmented image of the patient to segment all subsequent images.

The objective of the first chapter is to provide the necessary background on liver

anatomy, liver cancer and its diagnosis and treatment, and the importance of vessel segmentation

in guiding therapies to result in better patient outcomes.

1.1 Liver anatomy

The liver is the largest internal organ in the human body, weighing 1.2-1.5 kg [21]. A

vital organ necessary for survival, the liver plays a major role in metabolism, protein synthesis,

hormone production and detoxification. It is located inferior to the diaphragm. During normal

respiration the liver and other nearby organs such as the pancreas and kidneys undergo motion.

The liver chiefly undergoes motion in the craniocaudal direction [26]. As it is a soft tissue, it also

undergoes deformation, including compression, stretching and shear [27].

The liver is a vessel rich organ that normally receives 25% of cardiac output through a

dual blood supply. The hepatic artery supplies with oxygenated blood from the aorta, and about

25% of the total blood volume received by the liver. The other 75% of the blood volume is

venous blood from the pancreas, intestines, and spleen supplied via the portal vein. The blood

leaves the liver through the hepatic venous system, which is typically made of three major veins:

the right, the middle and the left [22].

2

Figure 1.1 The vascular anatomy of the liver. The inferior vena cava (IVC) gives rise to the

right (RHV), middle (MHV) and left (LHV) hepatic veins. The portal vein (PV) and the

hepatic artery (HA) enter the liver alongside the common bile duct (CBD). The PV

bifurcates into right (RPV) and left (LPV), as does the HA (RHA and LHA) [54].

1.2 Liver carcinoma

There are two types of liver cancer, primary and metastatic. Together, they are one of the

leading causes of cancer death worldwide, being the second most frequent cause of cancer death

in men, and the sixth most frequent in women. In 2008, 695,900 deaths were attributed to liver

cancer, while another 748,300 patients were diagnosed with it. Primary liver cancer represented

the majority of liver cancer cases worldwide [30]. However, in the United States and Europe,

malignancies in the liver are mostly due to metastatic deposits originating in other organs [29].

3

Primary liver malignancies include hepatocellular carcinoma (HCC), hepatoblastoma and

cholangiocarcinoma, although HCC constitutes for about 85% of all primary malignancies. HCC

most often occurs in patients with cirrhosis, Hepatitis B and C infections, and history of alcohol

abuse. Such patients are often regularly screened for HCC. When the diagnosis of the disease is

done as a result of screening, the prognosis is better than when it occurs only after a patient

presents the symptoms of the disease, such as jaundice and abdominal pain. Yuen et al. showed

that 26.8% of patients diagnosed as a result of regular screening were eligible for curative

resection as opposed to only 7.9% of those presenting symptomatic disease [30].

Besides tumors that originate in the liver, tumors can metastasize to the liver from other

organs. In fact, after the lymph nodes, the liver is the organ most frequently involved in organ

metastases from tumors in the breast, colon, lung, pancreas and stomach [28]. There are at least

four reasons for this. Firstly, the dual blood supply of the liver provides metastatic cells with two

ports of entry. Secondly, the long, slow and tortuous character of hepatic microcirculation

provides circulating cancer cells with access to all parts of the liver and may facilitate their

mechanical arrest there. Thirdly, the liver contains rich cell populations that normally serve to

restore hepatic tissue under various pathophysiological circumstances; these same cells can be

exploited for growth by cancer cells. Finally, the local production of anti-inflammatory and

immune suppressing factors, which normally prevent damage to liver parenchyma, can also

contribute to the tolerance of cancer metastasis [31]. As in primary liver cancer, resection of the

tumor(s) is widely considered to be the only potentially curative treatment, although less than

20% of patients are eligible for such treatment [40].

1.3 Imaging liver cancer

Various medical imaging modalities are used to visualize the structure, function and

pathology of the liver. Medical imaging is critical in diagnosing liver cancer, determining the

treatment options for the patient, planning the treatment, and, more recently, in guiding the

intervention [8]. It is also important as a follow-up procedure during chemotherapy [34].

Imaging of the liver presents certain challenges. First, the liver is usually moving due to

respiration. It can be temporarily immobilized via a breath-hold, and the imaging modality must

complete acquisition within that time. Second, both benign and malignant lesions are common in

4

the liver, and the imaging modality must provide enough information to distinguish between

them.

Common modalities to image the liver are ultrasound, computed tomography and

magnetic resonance imaging. In all three imaging modalities, a contrast agent is often

administered intravenously to improve image quality. The acquisition is timed with agent’s

arrival into the relevant vasculature [23]. In the context of the liver, the agent travels first to the

hepatic artery, then the hepatic and portal vein systems and then may travel into the interstitial

fluid depending on whether it can pass out of the vasculature.

1.3.1 Ultrasonography

Ultrasonography (US) is an imaging technique that transmits high frequency sound

waves through the body, and captures their echoes to create an image. The modality is

inexpensive, safe, widely used in radiology and employs relatively small and portable

machinery. Conventional, gray-scale US is considered a “non-specific technique” in the

diagnosis of focal liver lesions (FLL). While tissue harmonic imaging, which exploits the non-

linear propagation of a sound-wave, significantly improves the spatial and contrast resolution of

conventional US, the detection rate of FLLs is only slightly improved. The use of Doppler US,

which can provide a quantitative measure of blood flow, is also limited since metastases are

overall poorly vascularized; although Doppler US can detect vascular invasion [34].

Two methods can significantly extend the role of US in liver cancer diagnosis: contrast

agents and intraoperative imaging. US contrast agents are commonly microbubbles, consisting of

gas in a protein, polymer or lipid shell, and can increase the contrast of vasculature. When the

agent is predominantly in arteries, it can be useful in imaging metastases, which, while overall

poorly vascularized, receive almost all their blood from the hepatic artery. Intraoperative, or

invasive US, can also significantly improve liver cancer diagnosis as the ultrasound probe can be

brought to the surface of the liver. This is useful because it can overcome the limited depth of

penetration of ultrasound due to attenuation and artifacts caused by tissue covering the liver.

1.3.2 Computed Tomography

Computed Tomography (CT) provides significantly better image quality than US and is

one of the most widely used imaging techniques. CT delivers electromagnetic radiation in the

5

range of 100 eV and 100 keV (“x-rays”) to the body and measures its attenuation. The

attenuation depends on the composition and density of tissue it along its beam path. Since it is

impossible to determine the exact 3-D volume based simply on a single 2-D projection image,

CT takes multiple images from different angles of the same volume and then reconstructs the

volume based on algorithms. The radiation used by CT is ionizing, and therefore potentially

harmful, limiting CT’s usefulness as a modality, particularly in pediatric patients.

CT is currently the most common imaging modality for detecting and characterizing liver

lesions. The development of multi-detector CT has allowed for a high resolution whole liver scan

to be acquired in 5-10 seconds. CT scans can be taken with or without a contrast agent. Scans

without a contrast agent (also called “precontrast”) are useful in showing calcified and

hemorrhagic metastases [34]. Scans taken with an iodinated agent intravenously injected can

improve the contrast between healthy and diseased tissue. There are three phases of contrast-

enhanced liver imaging, depending on how long after the injection the liver is imaged: arterial-

phase, portal-venous phase, and the equilibrium phase. Table 1.1 summarizes the location of the

agent at the time of acquisition and the type of tissue enhanced in each phase [33].

Table 1.1 The different phases of acquiring contrast enhanced images of the liver

Phase Acquisition time

(after contrast

injection)

Location of

contrast agent

Tissues enhanced

Arterial 15-25 seconds Hepatic arteries Hypervascular

malignancies (lesions

predominantly supplied

by the hepatic artery)

Portal-venous 40-70 seconds Hepatic and portal

veins

Liver parenchyma,

excluding hypovascular

lesions

Equilibrium 80-120 seconds Interstitium Edematous tissues, e.g.

neoplasms and inflamed

areas

1.3.3 Magnetic Resonance Imaging

Magnetic Resonance Imaging (MRI) is an imaging technique that exploits the nuclear

magnetic resonance (NMR) of hydrogen atoms inside the body. NMR is the phenomenon where

nuclei of certain atoms can respond to magnetic fields and create magnetic fields of their own. In

this model, nuclei are like bar magnetics that are always spinning, and their axis of rotation

6

normally points in an arbitrary direction [35]. Hydrogen nuclei, which make up the majority of

atoms inside the body, are especially magnetically susceptible. In MRI, the axes of rotation of

hydrogen nuclei are first aligned in the same direction using a strong static magnetic field. A

radiofrequency (RF) magnetic wave then deflects the axes of rotation (or simply “axes”) of the

hydrogen nuclei, causing them to tip into the transverse plane. The frequency of the RF wave

must be the same as the hydrogen nuclei’s Larmor frequency, which is the rate at which they

spin. As the axes of the nuclei then begin to “relax”, or return themselves to the original

alignment with the strong magnetic field, their magnetic field is measured. The strength of the

field is dependent upon the amount of hydrogen nuclei (“proton density”), which varies for

different tissue types, and gives the “image intensity”.

The nuclei relax in two primary ways: T2 and T1. When the nuclei axes are deflected,

they tip, generating a magnetic field transverse to the static magnetic field. T2 relaxation is the

exponential decay of this signal, which occurs because the precession is slightly different for

different nuclei and becomes incoherent over time. T1 relaxation is the phenomenon whereby the

deflected nuclei axes re-align themselves with the strong magnetic field. T2 and T1 relaxation

times vary for different tissues and can give excellent soft tissue contrast.

An MR dataset is typically acquired one slice at a time, where a slice is the plane normal

to the anteroposterior axis (defined in the thesis as the “z-axis”). In order to “select” a slice in the

body, a 1-D linear magnetic field gradient is applied during the period that the RF pulse is

applied, causing the magnetic field to vary linearly with the position on the z-axis. Since the

Larmor frequency is proportional the magnetic field, and because nuclei will only deflect if their

Larmor frequency is the same as the RF wave frequency, the position on the z-axis can be

selected by choosing the frequency of the RF wave.

While the z-axis information is determined before deflecting the axes, the x- and y-axis

information is determined during relaxation. A 1-D linear magnetic gradient is applied along the

x direction, increasing the precession where the magnetic field is stronger. Over time this results

in the phase progressively shifting along the x direction, which implies a particular spatial

frequency (depending on the strength and duration of application of the gradient). Applying a

similar gradient in the y direction gives spatial frequencies in that direction; together these

frequencies form “k-space”. Taking the 2-D inverse Fourier transform of k-space then gives the

7

image slice. The maximum spatial frequency sampled in k-space corresponds with the x and y

resolutions (“in-slice resolution”), while the distance between slices selected corresponds to the z

resolution (“slice thickness”). The order in which the deflections and gradients are applied

(“pulse-sequence”) affects the duration of acquisition.

Recent advances in both hardware and software have allowed for fast pulse sequences

that are suitable for liver imaging, including fast dynamic, parallel, and echo-planar imaging.

More recently, parallel acquisition techniques have been introduced, which have the potential of

reducing the acquisition time 2-4 fold [34], at the expense of poorer signal to noise and

reconstruction artifacts.

There are three different types of MRI sequences of the liver that are commonly acquired:

T2-weighted, T1-wieghted, and contrast enhanced Gradient Echo (GRE). T2-wieghted sequences

predominantly provide information on the fluid content, fibrotic tissue and iron content of

tissues. T1-wieghted images are very useful in lesion characterization. Much like US and CT

images, MRI images can be contrast enhanced; gadolinium chelate is typically used as a contrast

agent [34]. As shown in Table 1, the time of acquisition determines the type of tissue that is

maximally contrasted.

1.3.4 Comparison of the imaging modalities

US, CT and MRI are all commonly used to image the liver with similar sensitivity in

detecting liver cancer [34]. US is inexpensive and can be rapidly acquired. CT is more expensive

than US, but less so than MRI, and can also be rapidly acquired. However, CT exposes the

patient to potentially harmful ionizing radiation. MRI is expensive with a relatively long

acquisition time, but has few known harmful effects. CT and MRI also both allow for multi-

plane reformatting and 3-D display; this allows surgeons to plan for surgical resections.

MRI offers better image quality than CT and both offer superior image quality than US.

US images suffer primarily from poor contrast and effective resolution. US also has a limited

depth of penetration, this is especially true if higher frequencies are used to achieve a higher

resolution. CT images, while better than US images, don’t give as good a liver parenchyma to

lesion contrast as MRI [34]. Also, while both CT and MRI rely on contrast agents for improved

8

image quality, MRI contrast agents like gadolinium are not as often contraindicated as the

iodinated contrast agents used for CT [33].

For the above reasons, MRI is probably one of the best imaging modalities for the liver

due to its safety and image quality, with great potential for the future.

1.4 Role of imaging in interventions

Surgical resection is often considered the “only curative”… Patients can be considered

ineligible for surgical resection either due to the extent of disease or poor medical fitness or both

[46]. For those who are ineligible for resection, systematic chemotherapy is currently the only

significant treatment. However, more local, but invasive, treatments have recently been

developed. In radiofrequency ablation (RFA), alternating electric current is delivered to the

tumor via a needle, heating it to 50-100°C, which causes coagulation and tumor necrosis [47].

Other invasive methods for ablation include laser-induced thermotherapy, where laser energy is

deliver via inserted optical fibers, and microwave ablation, where microwaves (a type of

electromagnetic radiation) are delivered via a microwave antenna inserted into the tumor [48].

An example of a local but non-invasive treatment is focused ultrasound surgery (FUS), where

tumor temperature is increased by delivering mechanical energy via high intensity ultrasonic

waves. Because it is a noninvasive form of treatment, the energy is localized using some sort of

intraoperative imaging modality (usually MRI). In spite of challenges, including the fact that the

liver is often behind ribs which attenuate ultrasound, initial results are promising [49]. A far

more common non-invasive local treatment is radiation therapy.

Intraoperative imaging (i.e. images acquired during intervention), is increasing used in

the treatment of liver cancer. It is an essential part of interventions such as radiation therapy [36],

radiofrequency ablation [38] and high frequency ultrasound (HIFU) ablation, where the

interventions are either non-invasive or minimally invasive, and intraoperative imaging is

necessary to direct treatment to diseased tissue while sparing healthy tissue. Intraoperative

imaging is also increasingly used in surgical resection of the liver tumors [37], where it can

increase the accuracy of the resection, leading to clear tumor margins, improved outcomes, and

an increased number of patients eligible for resection [38].

9

However, there are inherent limitations to intraoperative imaging. Due to time constraints

associated with a single breath-hold (reportedly 12-16 seconds [57]), and the duration of the

treatment, it is logistically difficult to complete a comprehensive liver scan intraoperatively.

Furthermore, intraoperative imaging is usually limited to non-contrast enhanced images, and

does not embed the clinician’s treatment plans. For this reason, using the information provided

by pre-operative images is necessary to guide the intervention.

In order to use the pre-operative images, they must be mathematically mapped onto the

intraoperative images in what is known as registration. From this mapping, a real-time update of

the location of intervention can be displayed in reference to the detailed pre-operative images

and plans [39]. This real-time information can be used to much better improve accuracy of the

treatment. Figure 1.1 shows a workflow diagram of how preoperative and intraoperative imaging

would be incorporated to deliver therapy. It could apply to, for example, radiation therapy or MR

guided ultrasound ablation.

1.5 Image registration

In the context of liver cancer treatment, the image registration must take into account not

just liver motion, but also liver tissue deformation, and be rapid enough to be implemented in the

workflow outlined above.

Most deformable image registration methods can be divided into two categories:

intensity-based and feature based. Intensity-based methods, including Fourier methods and

Mutual Information (MI) methods, correlate the voxel intensities in both images. Because these

methods take only voxel intensities into account, without any structural analysis, they are

susceptible to intensity changes caused by noise [11], and the presence of contrast in the

preoperative image. Since the method of acquisition of pre-operative and intra-operative images

is often different, these methods require the normalization of intensities and the resampling of

one image to the resolution of another [12].

10

Figure 1.2 Example of a workflow diagram of image-guided treatment administration.

Preoperative images are acquired and the treatment is planned before the intervention,

which is boxed in blue. After the intervention is prepared, the treatment is administered

using image guidance, boxed in pink, as many times as necessary until the required dose

has been administered. In between the administrations the patient is freely breathing.

Feature-based methods attempt to match the corresponding features of both images, and

are faster and more robust than most other methods [13,14]. These methods’ speed advantage

over intensity-based methods stems from the fact that features are more sparse than voxels [5]. In

order for these methods to be accurate, the precise segmentation and correspondence of the

features they rely upon is critical. In the context of the liver, vessels are ideal features because

they are well distributed throughout the liver and capture its non-rigid transformations [7]. While

microvasculature may not be detectable in medical images, the larger vessels are quite

prominent. The registration would then use these more visible vessels as features to generate a

transformation.

11

Recently, Huang et al. showed a joint registration and segmentation method where

vessels are iteratively registered and segmented, with each process exploiting the information

from the previous one [41] (see Figure 1.3). Osario et al. have presented a similar scheme [42].

These methods use corresponding vessel centerlines in both images, as opposed to segmented

vessel volumes, as features. This is because vessel centerlines, a 1-D curve in 3-D space, capture

the location of the vessel with far less information than vessel volumes, making registration

efficient. Because the registration aims to minimize the distance between features in both

images, correspondence between those features is necessary. This is not a trivial task considering

that vascular trees are dense and vessels look quite similar.

Figure 1.3 Huang et al.’s joint registration and segmentation method to iteratively obtain

an accurate registration. Vessel centerline extraction is a critical part of this method.

12

Thus the segmentation of vessel centerlines in the liver allows for fast and accurate image

registration, which is a critical component of image-guided treatments of liver cancer.

1.6 Thesis outline

The remainder of the work is organized as follows.

Chapter 2 discusses the previous work done in rapidly segmenting vessel centerlines in

an image and establishing correspondence between two sets of centerlines. It identifies the

absence of a fast method that has been used to do both. Noticing that few of the previous

methods have used a priori information about the liver vasculature, the research goal is defined

as developing an algorithm that accurately and rapidly segments liver vessel centerlines using

patient specific models.

Chapter 3 presents the proposed algorithm to meet the research goal. It describes the

vascular model and centerline tracking scheme used to segment vessel centerlines, and the

method to correct centerline orientation.

Chapter 4 discusses the results of using the algorithm to segment centerlines in sets of

MR and CT images of healthy and diseased individuals. The strengths and weakness of the

accuracy and speed of the algorithm are discussed.

Finally, Chapter 5 summarizes the contribution of this thesis, suggesting future direction

for the work and presenting ideas on integrating this algorithm into clinical procedures.

13

Chapter 2

2 Previous Work and Research Objective

As shown in the previous chapter, there is a demonstrated need for fast and accurate

segmentation of vessel centerlines in the liver. There also needs to be a correspondence between

two sets of segmented vessel centerlines so that the registration may determine a transform that

minimizes the distance between corresponding centerlines. In order to be clinically relevant the

speed of these methods is critical. Ultimately faster algorithms leave more time for registration,

image acquisition and treatment administration. This chapter looks at the previous work done in

the field before identifying the gap that this thesis aims to fill.

2.1 Vessel Segmentation

A preponderance work has been done in segmenting vessels in medical images. However,

much of the work is computationally intensive and/or requiring significant user interaction, and

therefore clinically irrelevant for rapid image guidance. A hallmark of most computationally

cheap segmentation techniques is that they explore the image only very sparsely. Another

important aspect of fast techniques is their parallelizability. This is because computationally

complex methods can sometimes be significantly speeded up using a GPU based

implementation. Erdt et al, for example, presented a vessel segmentation method where the

filtering step for a liver volume took 26-32s on a CPU, but only 1.6-2.0s with a GPU

implementation [45]. (They did not report the total times for the entire segmentation process).

Interestingly, Osario et al, the only other group to use segmentation in a joint

segmentation-registration framework, have reported a segmentation time of 3s per vessel, with

the total segmentation-registration time of the entire liver upwards of 22 minutes [42]. Their

segmentation involves first computing the likelihood of a vessel being present at each voxel, and

then skeletonizing likelihood images to generate centerlines, before implementing another

method to determine correspondence.

Most vessel segmentation can be categorized by their vessel extraction schemes as active

contours, region growing, or centerline-based methods [20].

14

2.1.1 Active Contours

Active contours methods delineate the outline of the vascular region by evolving an

interface (contour in two dimensions, surface in three dimensions) to minimize its energy.

External energy is derived from the image, while the internal energy of an interface is based on

its geometry and regularity.

The deformable interface can be represented explicitly in a parametric form, also called

“snakes”. Segmentation using parametric contours is robust to image noise and gaps because it

constrains the interface to be smooth and can be simple to implement and computationally

efficient. However, it is topologically inflexible to splitting or merging of parts. Such topological

challenges can be handled using “level set” methods, which represent contours implicitly [50].

However, these methods are computationally intensive [20], making them unsuitable for online

use.

2.1.2 Region-Growing

Region-growing methods begin with at least some voxels identified inside a vessel

(“seeds”), and classify neighboring voxels as vessel or non-vessel, incrementally, until a

termination condition is met. Classic region growing approaches were voxel-wise processes that

were prone to false positives (“leaking”) and false negatives (“holes”). This can be rectified with

growth limiting criteria, or using region growing to segment both vessel and non-vessel voxels

(“competitive region growing”). The region being grown can be made spatially coherent by

imposing conditions on the interface of the region (which is similar to the active contours

approach). One way of doing that is through ordered region growing (ORG) approach, where

regions for inclusion are ranked in accordance to their potential effect on the geometry of the

growing front. Fast Marching methods, where voxels are visited based on their geodesic

distance from seeds, have been used to segment vasculature [20]. All region growing methods

give a true segmentation of the image, i.e. they classify the voxels into vessel and non-vessel.

However, as mentioned in 2.4, efficient registration requires that centerlines be computed from

the segmentation. The resulting skeletonization problem is not trivial as it can result in false

branches.

Yim et al. used ORG to segment vessels, followed by skeletonization to extract the

centerline, and achieved this in 10-20 s. In order to avoid false branches, the method required the

15

user to input the maximum radius and pruned all branches smaller than that [51]. However, since

a liver data set would contain vessels that vary widely in radius, such an approach would require

significant user interaction increasing the actual time taken by the algorithm.

2.1.3 Centerline Tracking

A more straightforward approach to extracting vessel centerlines is by directly tracking

them, as in centerline-tracking approaches. These methods output focus on tracking the center

points of a vascular tree, not its entire volume. Just as region growing methods can be augmented

to later extract the centerline from vascular volumes, centerline tracking methods can be

followed by one of several methods to generate the vessel volume from its centerline: 2D cross-

section segmentation, 3D constrained active contours, or generalized cylinders [20].

Like region-growing, centerline-tracking requires initial seeds inside the vessel. Many

previous authors have relied on user specified seeds, making the speed of their methods

dependent upon user interaction. Some methods, not concerned with the processing time, use

seeds generated, more or less, arbitrarily [16]. Many of these seeds are later discarded if they are

determined to be false positives, or if they have been visited via tracking from a different seed

[52]. Another method is to select seeds only at branchpoints, which can be detected pre-

segmentation using a corner-detector algorithm, or during tracking as a result of cross-sectional

changes associated with passing a bifurcation [53].

Tracking methods rely on one or more models to determine direction and recenter each

tracked point in the vessel cross-section. There are three models that can be used: ridges, cross-

section, and tubular. Ridge-tracking assumes the center line of the vessel is a “ridge”. However,

at very low contrast, the ridge will be sensitive to noise. Cross-section tracking, which models

the vessel cross-section as an ellipse, is more suitable for such low contrast. But this method is

not suitable for small vessels (usually less than 3 voxels in diameter) [15]. By contrast, ridge

tracking is suitable for vessels as small as 1 mm in diameter [16]. Tubular models of blood

vessels have been developed to deal with both low contrast and small vessel diameter. These,

however, take several minutes to compute [15], and therefore unsuitable for applications that

require near real-time operation. Thus a combination of ridge and cross-section methods can

provide a robust and accurate model for the vessel, yet still be fast enough to be suitable for near

real-time applications.

16

Aylward, in particular amongst ridge-tracking algorithms has developed a fast and

accurate ridge-tracking algorithm and optimized it using dynamic parameters [16]. The

algorithm further had the advantage of modeling the vessel as curving centerline, with varying

radii. This reduces the amount of data required to represent a vessel, therefore making feature-

feature registration more computationally efficient. However, the authors did not propose an

efficient method for defining a search space. The search space could either be defined by a user,

or arbitrarily the entire image. The former case reduces automation, and the latter case increases

computational complexity. In the latter case, Aylward’s method would also have to be followed

by a method that established the correspondence between the vessels segmented in the

intraoperative and preoperative images.

2.2 Vessel Matching

As mentioned in section 2.5, correspondence between features is necessary to obtain a

transformation. The problem is similar to the “assignment problem” in computer science: given

two sets (of possibly unequal sizes), assign each member in the smaller set exclusively to one

member in the larger set. A brute force approach could lead a combinatorial explosion. One

efficient approach is to use the Hungarian algorithm, where the cost is the linear combination of

Mahalanobis distance and orientation similarity [55]. However, the accuracy of this approach

depends on the initial alignment of the two sets, with a strong likelihood of the existence of local

minima in the cost, especially in dense vasculature. A more accurate alignment is likely to give

better results.

Thus, in the context of registration, the most robust methods solve for both the

correspondence and transformation, iteratively. The simplest of these methods is the iterative

closest point (ICP) method, whose steps can be:

1. Assign elements in one set to the elements in the second set based on the nearest

neighbor criteria,

2. Estimate a rigid transformation that would minimize the distance between

corresponding elements,

3. Apply the transformation to the moving set of elements, and start again at step 1.

17

The above algorithm still has drawbacks: the transformation is not necessarily rigid and

ICP may assign two elements to one element. ICP also generates local minima [56]. More

advanced approaches use the Expectation Maximization (EM) algorithm, where E-step estimates

the correspondence and the M-step finds the transformation. Osario et al use the Thin-Plate

Spline Robust Point Matching (TPS-RPM) algorithm. The algorithm is similar to ICP above.

However, in step 1, each element in the smaller set is assigned in a “soft” or “fuzzy” manner to a

spread of elements in the larger set. The spread is determined by the “annealing temperature”,

which is reduced at each iteration. In step 2, a non-rigid TPS transformation is used. The original

authors of this method reported the worst case computational complexity at O(N3), where N is

the number of points in the sets. Osario et al reported the time taken by correspondence-

transformation iteration to be on the order of several minutes [42]. If vessel centerlines in a set of

two images are to be used as corresponding features, the correspondence problem must be

solvable in near real-time.

2.3 Research Objective

One common aspect of all the algorithms mentioned above is that they don’t take into

account that an image of the same patient may already have been segmented (either by the said

algorithm or another algorithm). Detailed information about the patient’s vasculature is readily

available, if it has been segmented in the preoperative image. Unlike the intraoperative image,

the time limits on segmenting the preoperative image are far more relaxed and the preoperative

image is usually studied carefully anyway by the radiologist. This thesis hypothesizes that this

information can be quite powerful, especially in solving the correspondence problem. Thus the

thesis proposes using a patient specific model to segment vessel centerlines in liver images.

The objective of the thesis is to develop an accurate, fast and automatic vessel

segmentation algorithm that exploits patient-specific knowledge of the vasculature. The

algorithm should segment the vessel centerlines with an error less than a voxel length. This is

because reducing the error beyond a voxel length does not necessarily have any clinical

advantages, as clinicians do not aim to resolve tumor boundaries or surgical instruments at the

subvoxel level. The percentage of vessels that need to be correctly segmented should be as high

as possible; however, since no previous work has used patient-specific models to segment vessel

centerlines rapidly, the number achieved by this thesis will set the benchmark for future work.

18

The algorithm should execute in less than 2s, the lower the better. The reason for such a short

window is because breath-holds are typically 12-16 seconds[57], during which the image must

be acquired and registered and the treatment administered. Finally, the algorithm should execute

automatically, since there would not be enough time for user interaction. The first image taken

after the patient is in the image scanner may require a rough initial alignment from the user,

which is often the case in clinical protocols.

19

Chapter 3

3 Algorithm for vessel segmentation

This thesis proposes an algorithm that exploits the vascular segmentation of one image to

segment the vessels in another image of the same patient. The proposed algorithm takes as input

the intraoperative image (called the “image” in this section) in which vessels are to be segmented

and the vessel centerlines segmented in the preoperative image (called the “guidance model” in

this section). The algorithm outputs the vessel centerlines in the intraoperative image. The

algorithm assumes that the guidance model has been registered onto the image with some degree

of accuracy, and thus both are in the same coordinate frame of reference. For each centerline in

the guidance model, the algorithm searches for the corresponding centerline in the current image.

The segmentation of the centerline is initialized by determining an appropriate seed point

using the guidance model. It is then tracked point by point, and the radius is calculated at each

point. The tracking terminates once the length of the current vessel matches that of the guidance

model. Finally, the algorithm checks the segmented centerline to ensure it matches the guidance

model. If not, the algorithm backtracks and re-segments. Upon segmentation, the algorithm

corrects the bifurcations of the segmented tree of vessel centerlines.

3.1 Vessel modeling

Selecting appropriate models for vessel cross-section and direction are important to

ensure accurate and robust tracking on MR images of the liver vasculature.

The following notations are used in this chapter, given a point P in a 3-D image:

is used to denote a point in 3-D image coordinates

is the intensity of the image at point

20

is the Hessian matrix at point and scale σ

are the eignevalues of H, where

are the respective eignevectors of , and

3.1.1 Cross-Section

The vessel cross-section is theoretically a sharp edged circle or ellipse (anatomically this

corresponds to the fluid being surrounded by an endothelium). However, the edge often appears

slightly blurred in MR images, due to three artifacts.

Firstly, due to the partial volume effect, if a voxel straddles the boundary of a vessel, part of

the voxel will generate a vessel signal, while the other part will generate a non-vessel signal.

These will often average out and give the impression of blurring at the edge. This would also

apply to CT images.

Secondly, because of slice selection, as discussed in section 1.3.3, only nuclei in a slice have

their axes deflected. Since there is time between this axis deflection and the measuring of the

signal, some of the nuclei are inevitably replaced by non-deflected nuclei via blood flow. Due to

the no-slip condition of fluid flow, where flow at the boundary is assumed to be 0, the blood flow

is slower at the boundary than at the center of the vessel, making this effect the most pronounced

in the center of the cross-section [58]. Note that this results in the vessel cross-section being a

region of low intensity, which would be then used in the current algorithm by inverting the

intensity values.

Thirdly, the blurring could be the result of windowing in the k-space, which, for practical

reasons, can’t be sampled infinitely. The 1-D vessel cross-section and it’s k-space representation

can be written as, respectively,

and

. . After

applying windowing, the k-space representation becomes

,

21

and the cross-section is imaged in 1-D as

, where W is the

width of the window.

Previous literature has sometimes modeled the cross-section as a 2-D Gaussian distribution.

This was not observed in the MR images this thesis examined, nor could it be explained in terms

of the artifacts above. However, two important features were observed on the cross-section:

1. The center portion of the vessel cross-section has a higher intensity than the peripheral

regions; the center portion could be a single voxel or several voxels wide, depending

on the size of the vessel cross-section.

2. While the intensity drops off from the center portion of the vessel cross-section to the

background, the decrease is the sharpest at the edge of the vessel cross-section.

These two features will be exploited later in the chapter to determine the center of the vessel

cross-section and the radius.

3.1.2 Direction

In order to model the vessel direction, first consider the neighborhood of an image point.

+

If the neighborhood is approximated by the Taylor series expansion up to the second derivative,

the following is obtained,

22

The second coefficient of the final term in the above equation is the Hessian matrix,

which takes into account the second order intensity variation [24]. In fact, if the eigen-

decomposition of the Hessian matrix gives three distinct eigenvalues ( , the

corresponding eigenvectors (

are orthogonal. Each eigenvalue represents the sign and

magnitude of the second derivative in the direction of the corresponding eigenvector. Also recall

that the second derivative of a graph is negatively correlated with its curvature (where concave

up is defined as positive). So, for example, the eigenvector associated with the largest eigenvalue

would represent the direction in which the curvature (where concave up is positive) of the

intensity is the highest. Similarly, different possibilities of describe various structures,

described in the table below. Eigenvalues much larger than zero are not considered as those

would give rise to concave up curvatures, which would result in low intensity structures in the

image, whereas vessels in MR images are of a high intensity.

Table 3.1 The description of the shape being described for various combinations of

eigenvalues.

Description

≈ 0 ≈ 0 ≈ 0 Noise or uniformly bright/dark

<< 0 ≈ 0 ≈ 0 A flat “sheet” or “plate”

<< 0 << 0 ≈ 0 A “tube” or vessel

<< 0 << 0 << 0 An ellipsoid or “blob”

23

From Table 3.1 it can be seen that the eigen-decomposition of the Hessian computed near

the center of a vessel would result in ≈ 0, << , and ≈ . The last condition assumes

that if the cross-section of the vessel is approximated as an ellipse, the length of the major axis

would be similar to the minor axis. Thus, and

are the vectors representing directions of

high magnitude of (negative) curvature, which corresponds to the cross-section, whereas the

direction of lowest magnitude of curvature would correspond to the vessel direction.

3.2 Centerline tracking

The centerline of the vessel is tracked as a three dimensional intensity “ridge”, adapting a

similar tracking method described by Aylward [16]. The starting point of the tracking, the seed,

is derived from the guidance model. Parameters of scale and step size are calculated dynamically

based on radius and curvature, respectively. Radius calculation is based on maximizing the sum

of gradients along the circumference of a circle [17].

The assumption that the centerline of the blood vessels is an intensity ridge is justified as

it has been shown in 3.1.1 that the center of the vessel cross-section is higher in intensity than the

edges.

3.2.1 Mathematical Representation of the Image

Mathematically the vessel centerlines constitute a three-dimensional ridge. When viewed

on a cross-section, or a 2-D plane orthogonal to the direction of the vessel, the ridge appears as a

local maximum. If a Hessian matrix is computed at a voxel on this ridge, two of its eigenvalues

will be negative, the eigenvector corresponding to the largest eigenvalue will be tangent to the

direction of the vessel, and the other two eigenvectors will form a plane orthogonal to the

direction of the vessel. The Hessian matrix can be calculated by convoluting the image with the

first and second derivatives of a Gaussian, which is mathematically equivalent to taking the

derivative of the image after convolving with a Gaussian. The elegance of using the derivatives

of the Gaussian to compute the Hessian is that convolving with a Gaussian will reduce noise and

generate maximum intensity at the center of the cross-section, given the correct scale.

24

3.2.2 Vessel Center Likelihood

In 4.1.1, it was shown that the vessel cross-section is blurred for several possible reasons.

When the vessel diameter is less than 10 voxels, it has been observed that the point of maximum

intensity is close to the center. Otherwise, however, the central portion of the cross-section is of

uniform intensity (meaning the 1-D cross-section profile of the vessel has a plateau rather than a

peak). Because of noise, the point of maximum intensity is often not close to the center. In that

case vessel center likelihood is determined using a method adapted from Wink et al [18].

The likelihood of a point being at the

center of the cross-section is computed by casting four pair of opposite rays from it at angles 0˚

and 180˚, 45˚ and 225˚, 90˚ and 270˚, and 135˚ and 315˚. The ray computes the 1-D intensity

gradient at each point; the point of minimum gradient is assumed to be the boundary of the

vessel cross-section. To avoid false positives due to noise, the minimum gradient must be lower

than -30. If the point is at the center of the vessel cross-section then its distance to the edge of the

vessel should be equal in opposite directions.

Formally, let Q be the point under consideration, and be the distance between Q and

the boundary of the vessel along the bth

ray of the ath

pair of rays (see Figure 3.1). The likelihood

that point Q is the center of the vessel is,

Figure 3.1 Rays are casted from the point

Q (yellow) in eight directions. Each ray

halts at the point the intensity gradient is

minimum, with the length of the ray

indicating the distance between Q and the

vessel edge. Opposite rays of similar

length indicate a high likelihood of Q

being the center of the cross-section. In

this image, Q is off-center, yielding some

asymmetric pairs of rays.

25

It can be seen that this approach is more computationally intensive than simply finding a

maximum, and thus is only used for vessels with diameter equal to or larger than 10 voxels,

where there is observed to be a difference between center point and point of maximum intensity

of the cross-section.

3.2.3 Generating Seed Point

The starting point of the current vessel is determined using the starting point of its

guidance vessel. As shown in Figure 3.2, the guidance model (red) is superimposed on the

current image, via coarse registration. The starting point of the guidance vessel (G1) is placed in

the current image, and a number of particles are uniformly generated, using a Mersenne-Twister

distribution, in a cylinder with a normal in the direction of the guidance vessel (blue crosses in

figure 3). Each particle, S, is assigned a score based on the following equation,

where G1 and G2 are points from the guidance vessel as shown in Figure 3.2. The numerator

ensures a vessel in the current image with orientation closest to the guidance vessel is selected,

while the denominator ensures that distant vessels in the current image oriented in the same

direction as the guidance vessel are not matched. The numerator is squared to give orientation

matching more weight than distance. The particle with the highest score is chosen as the seed.

Figure 3.2 Illustration of the centerline extraction

process. Points G1, G2 are on the guidance model

(red line) The blue ‘+’ points are the seed

candidates generated around G1, with direction

associated with each candidate indicated. The seed

candidate with the highest score, a function of the

direction it is associated with, is indicated in light

blue.

26

The seed is then refined by searching for a local maximum of intensity along a straight

line starting at the seed and point in the direction of the gradient at the seed. This refined seed is

the first centerline point and should approximately be located on a ridge. After this first point is

computed, each subsequent point of the centerline is found by ridge tracking.

3.2.4 Computing the Next Point

Using the eigenvectors of the Hessian computed at a given point, PG, the next point is

first estimated, PN1, and then refined iteratively as PN2 and PN3. The next point is estimated using

the following equation

where the magnitude of t is the desired “step-size” for vessel tracking. Through testing, 0.4 times

shortest voxel length was found to work well as the step-size. If vector is pointing in the

direction of the previous point (i.e. pointing “backwards”) the value of t is negative, otherwise t

is positive. To compute PN2, a plane is defined as the following set of points, where is the

search radius.

The refined point is taken to be the point in with the highest intensity,

To compute PN3, a plane is defined as the following set of points,

Once again the refined point is taken to be the point in with the highest vessel center

likelihood,

27

Essentially the algorithm is creating a plane of the vessel cross-section at a given point,

shifting the plane in the direction of its normal, and finding the point with the maximum intensity

at this shifted plane, as depicted in Figure 3.3. This is similar to the pseudo-arc length numerical

continuation method, where a solution curve is parameterized in terms of its arclength, the next

solution is predicted by its tangent at a known solution point, and the next solution is “corrected”

by solving the set of equations that define the curve (this is usually approximated using a variant

of Newton’s method) [43, 44].

3.2.5 Termination Condition

Just as the algorithm needs a place to start tracking, it needs to know when to stop.

Currently it is proposed that the vessel extraction be terminated when the length of the current

vessel reaches that of the guidance vessel. After extracting each centerline, the points are

averaged using an eleven point long Gaussian-weighted window:

The first and last points are anchored while this smoothing is applied to prevent vessel

centerline shrinkage. This had the effect of smoothing the points, eliminating curvatures in the

final vessel due to inaccuracies, while preserving genuine curves in the vessel structure.

Figure 3.3 The tracking algorithm first

computes point B by adding A and , which is

a scalar times the eigenvector corresponding to

the largest eigenvalue, where the Eigenmatrix is

computed at point A. C is taken as the point of

maximum vesselness in the plane normal to .

D is taken as the point of maximum vesselness

in the plane normal to .

28

3.2.6 Radius Computation

To model the vessel as a tubular structure, both information about its centerline and its

radius at each centerline point are needed. To compute the radius at each point, an estimate is

needed. For the seed point of the current vessel, the radius is estimated to be that at its guidance

vessel seed. For all subsequent points the estimate is the computed radius at the previously

tracked point.

The algorithm computes the optimal radius, , by maximizing a medialness function

over radii values close to the estimated radius, ,

For practicality, discrete values of r, spaced at 0.1 times the voxel size is considered. This

requires linearly interpolating the intensity data. A convenient medialness function, M, has been

formulated by Pock et al. [17] to optimize the radius, r, at the refined point, ,

3.3 Orientation Correction

Often vessel tracking encounters an unknown bifurcation and may track the “wrong” one

of the two possible vessels. The algorithm evaluates every tracked segment to check if it has

tracked a “wrong” segment. If so, it backtracks and restarts the tracking from seed generation.

While tracking the current vessel centerline, every 4 mm of the current vessel centerline

is designated a segment (e.g. in Figure 3.4, segment from T1 to T2, or from T2 to T3). The

Figure 3.4 Illustration of the centerline

extraction process. The red line is the

guidance model, and points G1 to G4

delineate segments. The teal square points

are those segmented by the algorithm, and

points T1 to T4 delineate the tracked

segments.

29

guidance vessel centerline is treated similarly (e.g. in Figure 3.4, the portion of the centerline

from G1 to G2 is a segment). Furthermore, correspondence is established between segments of

the current vessel and guidance vessel by equating the distance of both segments to the

respective starting point of the centerline.

A current vessel segment is classified as “correct” if it meets the following criterion,

where = 0.65, corresponding to a maximum deviation angle between the corresponding

segments of ~50˚. This value was found to give the best result on the data tested.

If the segment is not correct, then the algorithm will eliminate that segment and backtrack

to the last good segment. It will attempt to regenerate it by placing the seeds in the same fashion

as described in 5.2.2. The model vessel segment corresponding to the “wrong” vessel serves as

the starting model segment. However, if the newly tracked segment again doesn’t satisfy the

relationship above, the tracking of the vessel terminates. It is better to produce no information on

a vessel than the wrong information.

3.4 Bifurcation Correction

Vessel centerlines, upon segmentation, often do not give accurate bifurcations. As can be

seen Figure 3.5, the centerlines bisect earlier than one would expect the bifurcation of the

vasculature. While the definition of bifurcation can be subjective, it should correspond to the

geometry of the vasculature and be robust to small changes in the centerline segmentation. For

this reason, the definition proposed by Piccinelli et al. was used [19]. It is a weighted average of

four points on the centerlines, reducing the impact of a single point on the bifurcation location.

Because the points are on the centerline and because it also uses the radius computed at the

centerline, it is robust to small changes in one measure that are not reflected in the other

measure.

30

Figure 3.5 Centerlines A and B are shown with their respective “tubes” shown as the

shaded regions (pink and blue). Points and

(black) are on the edges of the red and

blue circles, respectively, while and

(dark blue and dark red) are at the centers of

the two respective circles.

Assuming the centerline of a vessel is given, and its radius is known at each centerline

point, a “tube” can be constructed as shown for two vessels in Figure 3.5. and radii of a vessel at

each point. Given centerlines A and B, the following four points are defined:

1. and

are the points on centerlines A and B, respectively, that intersect tubes B

and A, respectively.

2. and

are points on centerlines A and B, respectively, whose distance from the

edge of the tube is the same as the distance from points and

, respectively. In

31

other words, these points are the centers of maximally inscribed spheres within tubes

A and B, respectively, that touch points and

, respectively.

The bifurcation point is then the weighted average of the four points:

where is the radius associated with point j on centerline i.

After the correct bifurcation is computed, the centerlines are re-generated in the vicinity

of the bifurcation. Because the correct bifurcation point is always farther from the root than the

bifurcation of the centerlines, part of the parent centerline has to be reconstructed, while the

children centerlines need to be modified to be connected to the new branchpoint. An example of

the results is shown in Figure 3.6. The new lines are reconstructed by fitting a second order

polynomial 3-D function to the following points:

For reconstructing the parent, the points used are: the new branchpoint, the old

branchpoint, the last few points of the parent centerline

For reconstructing each of the children, the points used are: the new branchpoint,

the first few points of the child centerline

If enough points are not available for a second order fit, a first order fit is generated.

32

Figure 3.6 Corrected centerlines (dark blue dots and lines) at a branching point, after the

new point of bifurcation (green square) has been computed. The uncorrected centerlines

(cyan dots and lines) had an incorrect point of bifurcation (red square). The blue centerline

between the red and green squares is generated by fitting a second order polynomial to the

two squares, and points on the blue line preceding the red square. Similarly the lines

between the green square and orange circles are regenerated by fitting a second order

polynomial to the shapes and some points outside the region. In the rare case when enough

points were not available, a first order polynomial was fit.

33

Chapter 4

4 Experiments, Results and Discussion

The above described algorithm was tested on MR and CT images of the livers of several

humans, including a patient with liver cancer. The results are that the algorithm can accurately

segment vessel centerlines.

4.1 Experiments

4.1.1 Image Acquisition

Dynamic MR images of three human volunteers (subjects A, B and C) were acquired in

the axial plane using a 1.5T GE scanner (GE Medical Systems, Milwaukee, WI). All three

images were from black blood pulse sequences and the acquisition details are shown in table 4.1.

A set of two images was acquired for each subject with a breath-hold at exhalation in the same

imaging session. The variance in MR images was due to the variance in the duration the

volunteers held their breath; the longer the breath-hold the finer the resolution of the acquired

image.

In addition, one set of two CT images (resolution 0.355 x 0.355 x 2.5mm) of a pediatric

patient (subject D) with liver cancer were used to evaluate the performance of the algorithm on a

liver image with a lesion, especially in the vicinity of the lesion.

Table 4.1 Three MR images were acquired using the following sequences and resolutions.

Subject Pulse sequence TR TE Flip angle Slice thickness In-plane resolution

A LAVA gradient echo

3.79ms 1.72ms 12˚ 1.5 mm 1.3 mm x 1.3 mm

B DwiSE/SE 1.3s 34ms 90˚ 5.0 mm 1.95 mm x 1.95 mm

C DwiSE/SE 1.6s 34ms 90˚ 3.0 mm 1.95 mm x 1.95 mm

4.1.2 Guidance Model

To prepare the guidance model, the volume of the liver was manually segmented and

processed in Slicer 3.0. As a pre-processing step, the image was contrast-enhanced using the

34

Vascular Modeling Toolkit (VMTK) Vessel Enhancement tool (steps: 10, diameter: 1.0-2.5

physical units, plate and line-line structures 0.5, blob-like structures: 0.5, contrast: 60). The

image was segmented using Easy Level Set tool (inflation: 10, curvature: 70, attraction to ridges:

60, iterations: 10). The centerlines were extracted using the Centerline Extraction tool. Because

this tool generates overlapping centerlines, in-house software separated the centerlines to what

we call “segmented format” for the guidance model. In this format, a vessel is defined from one

bifurcation point up to the next bifurcation point. The model was stored as a .vtk file with the

centerline points with corresponding radii at each point. As VMTK produces incorrect

bifurcations, and because correct bifurcations are necessary for a good guidance model,

centerline bifurcations had to be corrected as described in 3.4.

For each set of images, one image was registered to the second image using different

types of registration with a different set of parameters each time. Each registration was applied to

the guidance model associated with the set of images to bring it into the coordinate system of the

image to be segmented. The objective was to produce a wide variety of guidance models with

which to test the algorithm.

Table 4.2 Summary of the number of vessels segmented in each data set, and the number of

different guidance models used on each image, and thus the total number of individual

vessel segmentations

Subject Vessels segmented in each

segmentation

Number of segmentations Total number of vessels

segmented

A 102 10 1020

B 15 3 45

C 12 5 60

D 35 5 175

The results of the centerline extraction method described above were evaluated against a

“gold standard”. The gold standard was constructed in the same way as the guidance model was.

An example of the results of the segmentation and the gold standard are shown in Figure 4.1.

4.2 Results and Discussion

To quantitatively evaluate the extraction accuracy, the segmentation error (SE) of each

vessel centerline was calculated between segmented vessels and the gold standard, by averaging

35

the shortest distance between the two centerlines for each point on a segmented centerline. The

model error (ME) was calculated similarly, finding the average distance between the gold

standard and the model centerlines from which the segmentation is computed. The performance

of the algorithm is heavily dependent on the model it is given: a vessel centerline generated from

a model closer to the gold standard is more accurate than one generated from a model centerline

much farther than the gold standard. Therefore SE was plotted against ME for each subject and

horizontal lines are drawn to indicate the voxel length (taken to be the largest dimension of the

voxel) for the set of images.

Every vessel segmentation is classified in terms of its average error compared to the

voxel length (e.g. 1 voxel < SE < 2 voxels). As ME increases, the proportion of vessels in each

category of error is expected change. The “maximum model error” refers to the set of all models

whose error is less than or equal to a particular model error. (For example, “90% of vessels had

error < 1 voxel at maximum model error of 2.5 mm” means that of the segmentations

corresponding to all models with an average error ≤ 2.5 mm, the SE is less than a voxel length

90% of the time). The measure of maximum model error is relevant because it imposes a

constraint on the accuracy of the registration portion of the joint segmentation-registration

method which is one of the main applications of this segmentation algorithm.

Times were measured while running on a computer running Microsoft Windows XP with

a 3.2-GHz i3 processor and 3.5 GB of memory, and included the entire time required for

segmenting all vessels in the image.

Table 4.3 Summary of accuracy and speed results of the algorithm. *Features refer to

vessels in all cases except the one where it refers to bifurcation points.

Subject Average error of all features*

when maximum model error is: Percentage of features* with average error < voxel length,

when maximum model error is:

Average time to run each

segmentation

2.0 mm 4.6 mm 2.0 mm 4.6 mm

A 0.97 mm 1.28 mm 88% 79% 35.2s A (for

bifurcation points)

0.81 mm 1.01 mm 95% 89%

B 1.67 mm 2.03 mm 100% 98% 9.8s

C 1.68 mm 2.11 mm 93% 87% 5.2s

D 0.73 mm 1.17 mm 96% 82% 44.7s

36

A summary of the accuracy and speed are given in Table 4.3. Maximum model errors of

2.0 mm and 4.6 mm were used as they were quoted by Osario et al[42] and Huang et al[41],

respectively, as the maximum error for their registration algorithms in their joint-segmentation

registration framework. For subject A, additional analysis was performed for measuring the

accuracy of bifurcation points instead of the entire centerline. This analysis was not performed

for other data sets, as the number of bifurcation points was very small.

4.2.1 Subject A

Figure 4.1 An example of the vessel centerline segmentation results (green) compared with

the gold standard (red) for the image of subject A.

Data from subject was comprehensive in terms of vascular coverage, with 102 vessels

from the hepatic and portal system. Figure 4.1 shows that all the vessels were successfully

segmented in most cases. Figure 4.2 shows that SE is less than a voxel length for low ME.

Visually, there is no clear pattern of vessel length on SE or ME, but it appears that vessels with a

larger radius may have both smaller ME and SE (Figure 4.3). Figure 4.4 confirms the effect of

37

increasing maximum ME on decreasing accuracy. While the only category relevant to the goal of

the thesis is SE < voxel length, the other categories are shown to illustrate the distribution of

error.

The figures mentioned above all show the average error of vessel centerlines. Figure 4.5

displays the error of the bifurcation points only. SE for bifurcation point was measured as the

distance between the segmented bifurcation point and the bifurcation point in the gold standard;

ME was the distance between the bifurcation point in the model and that in the gold standard.

The SE of the bifurcation is comparably less than that of vessel centerlines. This is likely

because the location of the bifurcation point is influenced most heavily by the parent centerline

(since the algorithm segments from the root in the direction of the leaves). Parent vessels will

Figure 4.2 The SE vs. ME for all vessel centerlines segmented for subject A. Each blue

point represents the average error of a vessel centerline with a particular length. The

total number of points on the graph is 1020. The horizontal lines are at intervals of 1.5

mm (the voxel length for this data set).

38

have a larger radius than the children vessels, and Figure 4.3 shows that vessel with larger radii

tend to have lower ME and SE. Figure 4.6 confirms that bifurcation SE is less than the vessel

centerline SE, showing a higher percentage of vessel centerline whose SE < voxel length.

Figure 4.3

Figure 4.3 The SE vs. ME for all vessel centerlines segmented for subject A.

Each blue point represents the average error of a vessel centerline with a

particular radius. The total number of points on the graph is 1020.The

horizontal lines are at intervals of 1.5 mm (the voxel length for this data set).

39

Figure 4.4 Percentage of vessels with an average error in each category vs. maximum ME

for subject A.

40

Figure 4.5 The SE vs. ME for all bifurcation points segmented for subject A. Each blue

point represents the error of a bifurcation point. The total number of points on the graph is

420. The horizontal lines are at intervals of 1.5 mm (the voxel length for this data set).

41

Figure 4.6 Percentage of bifurcation points with an average error in each category vs.

maximum ME for subject A.

42

4.2.2 Subject B

Figure 4.7 An example of the vessel centerline segmentation results (green) compared with

the gold standard (red) for the image of subject B.

In the dataset from subject B, 15 vessels were segmented, including those from the

hepatic and portal systems. Figure 4.7 shows that most vessels were accurately segmented. From

Figure 4.8 it can be seen that most segmented centerlines have error less than 5mm, the voxel

length. This is confirmed by the distribution in Figure 4.10. Figures 4.8 and 4.9 don’t show any

vessel length or radius related pattern.

43

Figure 4.8 The SE vs. ME for all vessel centerlines for segmented for subject

B. Each blue point represents the average error of a vessel centerline with a

particular length. The total number of points on the graph is 45. The

horizontal lines are at intervals of 5 mm (the voxel length for this data set).

44

Figure 4.9 The SE vs. ME for all vessel centerlines for segmented for subject

B. Each blue point represents the average error of a vessel centerline with a

particular radius. The total number of points on the graph is 45. The

horizontal lines are at intervals of 5 mm (the voxel length for this data set).

45

Figure 4.10 Percentage of vessels with an average error in each category vs. maximum ME

for subject B.

60

65

70

75

80

85

90

95

100

1.5 2.5 3.5 4.5 5.5 6.5 7.5 8.5 9.5 10.5 11.5

Perc

enta

ge o

f ve

ssel

s in

err

or

cate

gory

Maximum model error

Effect of maximum model error on segmentation error

>3 voxel lengths

2-3 voxel lengths

1-2 voxel lengths

0-1 voxel lengths

Category of error

46

4.2.3 Subject C

Figure 4.11 An example of the vessel centerline segmentation results (green) compared with

the gold standard (red) for the image of subject C.

12 vessel centerlines from the hepatic system were segmented in the image from subject

C. A large number of vessel centerlines had SE < voxel length, especially for ME < 5mm. This is

confirmed in figure 4.14. Figures 4.12 and 4.13 do not reveal any length or radius related

patterns.

47

Figure 4.12 The SE vs. ME for all vessel centerlines for segmented for subject C.

Each blue point represents the average error of a vessel centerline with a

particular length. The total number of points on the graph is 60. The horizontal

lines are at intervals of 3 mm (the voxel length for this data set).

48

Figure 4.13 The SE vs. ME for all vessel centerlines for segmented for subject

C. Each blue point represents the average error of a vessel centerline with a

particular radius. The total number of points on the graph is 60. The

horizontal lines are at intervals of 3 mm (the voxel length for this data set).

49

Figure 4.14 Percentage of vessels with an average error in each category vs. maximum ME

for subject C.

50

4.2.4 Subject D

Figure 4.15 Subject D was a pediatric patient with a lesion in the liver.

The objective of testing the CT image of subject D, a pediatric cancer patient, was to

demonstrate that the algorithm could segment vessel centerlines even in the presence of a lesion.

While the focus of the thesis was to segment MR images, it can also be applied to CT images.

Figure 4.15 shows the lesion in the axial view. Figure 4.16 shows that the algorithm was able to

accurately segment 35 vessels. Figure 4.16 shows a close-up of the segmentation, revealing that

segmentation is accurate even when the vessel is near or partially inside the lesion volume.

Figures 4.18 and 4.19 show that most vessel centerlines have an average SE <voxel length and

this is confirmed by Figure 4.20.

LiverLi

ne of

lowest

possibl

e

Differe

ntial

Error

LesionSe

gmentati

on Error

> Model

Error

51

Figure 4.16 An example of the vessel centerline segmentation results (green) compared with

the gold standard (red) for the image of subject D. The volume of lesion is shown in blue,

along with a slice of the image.

52

Figure 4.17 A close-up of the vessel centerline segmentation results

(green) compared with the gold standard (red) for the image of subject

D. The centerlines indicated by yellow arrows are either partially inside

the lesion volume or in close proximity to it. The volume of lesion is

shown in blue, along with a slice of the image.

53

Figure 4.18 The SE vs. ME for all vessel centerlines for segmented for

subject D. Each blue point represents the average error of a vessel

centerline with a particular length. The total number of points on the graph

is 175. The horizontal lines are at intervals of 2.5 mm (the voxel length for

this data set).

54

Figure 4.19 The SE vs. ME for all vessel centerlines for segmented for subject

D. Each blue point represents the average error of a vessel centerline with a

particular radius. The total number of points on the graph is 175. The

horizontal lines are at intervals of 2.5 mm (the voxel length for this data set).

55

Figure 4.20 Percentage of vessels with an average error in each category vs. maximum ME

for subject D.

4.3 Accuracy

In all of the segmentation results, the number of vessel centerlines with an average error

of less than a voxel length was singled out. This is because accuracies better than the image

resolution would be clinically irrelevant as it is difficult to resolve parameters such as tumor

margin, needle or surgical tool position at a subvoxel level. It is also challenging to segment

vessel centerlines at a subvoxel level without some modeling of vascular anatomy and MRI

physics that is beyond the scope of this thesis. All other errors represent room for improvement

in the accuracy of the algorithm.

There are several explanations for error greater than voxel length. The first is that in a

small number of cases the “gold standard” is actually inaccurate. Since the gold standard was

56

generated using 3D Slicer and in the absence of easy editing tools in the software, there was no

quick and easy way of fixing this.

The second was the problem of correspondence. There were several instances of the

algorithm tracking the “wrong vessel”, with the orientation correction aspect of the algorithm

failing. This happened if the guidance model was closer to the incorrect vessel than to the vessel

that was meant to be segmented. Sometimes, if there was no vessel within the model’s search

space, the algorithm would instead tracked noise or other nonvascular structures. Had the

algorithm been able to predict vascular density and model error, it would be better able to define

a search space and thus avoid tracking the “wrong vessel”.

The third source of error was due to not finding the center of the cross-section. Neither of

the models discussed in section 3.2.2 were robust enough to always ensure that the center was

found accurately. There are a lot of models in the literature, many computationally efficient,

which should be explored in the future for use in this algorithm.

The fourth problem was that orientation correction was intended to detect tracking the

“wrong vessel” and correct the problem close to where it started spatially. However, in practice

this often happened a few millimeters downstream of when the problem occurred leading to

portions of the centerline that were completely incorrect. This problem could potentially be

solved by detecting bifurcations using the shape of the cross-section. So once the “wrong vessel”

has been detected, the algorithm would backtrack all the way to the last bifurcation found, which

would better localize where the “wrong vessel” starts.

The accuracy was seen to remain consistent with vessel length. There was, however, a

slight trend noticed where vessels with a larger radius had lower ME and SE than those with

smaller radius. This can be explained in several ways. Since vessels with larger radius represent

those closer to the root, the vascular density in their vicinity is lower, reducing the chances of

tracking something else. That the ME is lower for these cases suggests that the registration may

be yielding closer models for thicker vessels than thinner ones. This could be because thicker

vessels are more prominent features that would influence registration more than thinner ones.

The accuracy that this thesis deals with is dependent upon the registration error, with

segmentation error worsening with worsening model error. The segmentation works relatively

57

well for registration errors less than 5mm. Thus, the segmentation errors were given in terms of a

registration error. This allows the thesis to evaluate the accuracy of the segmentation

independent of the registration. However, understandably clinicians would be interested in the

final accuracy of the entire joint segmentation registration framework. Unfortunately, since the

registration portion of the framework was not available, the final accuracy couldn’t be tested and

is left as future work.

4.4 Speed

The execution of the algorithm took 35 seconds on the most comprehensive MRI dataset.

The goal was for the algorithm to execute in less than 2 seconds. However, the algorithm does

have the potential to execute within that time frame for several reasons.

The first reason is that the algorithm developed is highly parallelizable – the extraction of

one vessel centerline is totally independent of all others and can thus be done in parallel.

Previous work has shown that algorithms can be speeded up to 15 times with a GPU

implementation [45]. Therefore it is quite possible that the entire segmentation can be completed

in 2 seconds.

Secondly, the implementation wasn’t optimized. 40% of the time was consumed by a

single inefficient class that computed the Hessian over the entire image. The area explored for

vessel segmentation, however, is a small fraction of the entire image. Therefore an

implementation that computes the Hessian only as necessary could reduce the computation

significantly.

Finally, implementing the algorithm on a more high performance computer would

increase the speed.

Ultimately, the goal of the algorithm is to be used in a segmentation-registration method

to find a transformation for the preoperative image into the intraoperative image space. Thus, the

accuracy and speed that matter the most need to be evaluated from the segmentation-registration

method as a whole.

58

Chapter 5

5 Conclusion and Future Direction

Image-guided procedures are a promising avenue to treat liver cancer as they can

improve accuracy of treatments, and therefore patient outcomes. In the context of the liver, a

rapid registration of the preoperative and intraoperative images is required as treatments are

often administered within a single breath-hold. Feature-based registration is robust and

computationally efficient and uses corresponding features in both images to find an accurate

registration. Vessels are a well-distributed feature in the liver. The problem to be solved is the

fast and accurate segmentation of vessel centerlines in an intraoperative liver images and

corresponding them with vessel centerlines in a preoperative liver image.

Previous work in vessel segmentation has often been intended for offline use. While there

are fast vessel segmentation algorithms, they do not necessarily establish correspondence.

Solving the correspondence problem after the segmentation is computationally expensive. This

thesis approaches the problem with the aim of exploiting the information in the preoperative

image to segment the intraoperative image.

5.1 Contribution of This Thesis

The algorithm developed by this thesis project uses a previous segmentation of the vessel

centerlines in another image and superimposes it upon the image to be segmented (usually after

some sort of registration). This guidance model is used to generate centerline seeds, which are

then tracked until termination to generate vessel centerlines. If the wrong centerline is tracked,

the guidance model can be used to detect that and correct the orientation in order to track the

correct centerline. This elegantly solves the correspondence problem at the time of segmentation,

whereas previous works have treated the correspondence problem as one to be solved post-

segmentation.

The algorithm successfully segmented centerlines accurately and rapidly on four sets of

images. If the registration was constrained to 2.0mm, the average segmentation error was 0.73-

1.68mm, with 88-100% of the vessels having an error less than a voxel length. If the registration

59

error was less than 4.6mm, the average segmentation error was 1.17-2.11mm, with 79-98% of

the vessels having an error less than a voxel length.

5.2 Other clinical applications

There are many clinical applications to the work done in this thesis, beyond those that

have been discussed. The first possible expansion of the application of the segmentation

algorithm is to other organs. While the aim of the thesis was to segment the vessels of the liver,

the proposed algorithm is not specific to liver vessels, and can, in fact, be applied to the

segmentation of vessels in other vasculature rich organs such as the brain, kidneys and lungs.

Secondly, the segmentation only clinical application of segmentation discussed above was

registration. While this is indeed an important application, there are two other applications of

vessel segmentation: computer aided diagnosis and treatment planning.

Computer Aided Diagnosis (CAD) systems aim to detect, classify and quantify certain

tissues (including lesions) in 3D medical images and thereby estimate the probability of a disease

in a given patient [59]. These systems can save the time and labor required to interpret medical

images and increase the accuracy of detecting abnormalities. The segmentation of vessels in the

medical images can be an important step in CAD [60]. For example, hepatic vessel segmentation

in CT images is an important part of the CAD of hepatocellular carcinoma [60]. Another

example is the CAD of pulmonary emboli in CT pulmonary angiography images, which depends

on the segmentation of pulmonary vessels to define its search space [61]. In breast tumors, the

delineation of vessels is an important step since vessels can display similar contrast

enhancements as the tumors [62]. Thus, the proposed vessel segmentation could potentially be

used to aid CAD algorithms.

Treatment planning is an important part of intervention, and helps ensure effectiveness

and minimizes the complications that may result; planning is necessary to consistently achieve

high quality results [63]. Vessel segmentation is often a key part of this planning because it

allows the clinician to visualize the vasculature and perform quantitative analysis. For example,

in liver surgery, vessel segmentation enables the surgeon to visualize the vasculature, which is an

important part of the preoperative planning [64]. Since vessel segmentation also quantifies the

location and size of vessels, it is used to characterize the importance of certain vessels in

supplying and draining tissue, which in turn is used to compute the volume of tissue remaining

60

post-resection [65]. The segmentation of vessels can also be used to visualize cerebral

vasculature and plan the path of tumor biopsies or the placement of electrodes deep in the brain

for epilepsy evaluation [66]. The visualization can also assist in selecting the best treatment (e.g.

surgery, radiosurgery, embolization, etc.), evaluating surgical risk and localizing lesions [67].

Thus, the segmentation of vessels has applications in treatment planning.

5.3 Future Work

The algorithm needs further improvement in accuracy and speed before it can be tested in

the clinical context. Changes to the algorithm to improve accuracy are proposed in section 4.3.

As discussed in section 4.4, the algorithm is highly parallelizable, and major improvements in

speed could be achieved by implementing it using parallel programming.

After improvements in speed and accuracy have been achieved the algorithm needs to be

tested within a suitable joint segmentation registration framework to determine the actual error in

registering the intraoperative image to the preoperative one. In the thesis, the error is presented

as a function of registration error; however, clinicians are more concerned with the final

registration error.

After the joint segmentation registration method has been tested, it could be tested as part

of clinical trials. These trials would validate the method, and also elicit useful feedback from

clinicians. This would improve the method so that ultimately it can improve treatment accuracy,

creating better outcomes for patients living with liver cancer.

61

References

[1] K. S. Lehmann et al. “Portal Vein Segmentation of a 3D-Planning System for Liver

Surgery—In vivo Evaluation in a Porcine Model,” Annals of Surg. Oncology, vol. 15, no. 7, pp.

1899-1907, 2008.

[2] M. Lin et al. “Algorithm-Based Method for Detection of Blood Vessels in Breast MRI for

Development of Computer-Aided Diagnosis,” Jour. Of Magn. Resonance Imaging, vol. 30, pp.

817-824, 2009.

[3] M. Descoteaux, D. L. Collins, K. Siddiqui. “A geometric flow for segmenting vasculature in

proton-density weighted MRI,” Medical Image Analysis, vol. 12, pp. 497-513, 2008.

[4] Q. Shang et. al. “Adaptive Directional Region Growing Segmentation of the Hepatic

Vasculature,” Proc. Of the Soc. Of Photo-Optical Instr. Eng., vol. 6914, pp. F9141, 2008.

[5] S. Kawajiri et al. “Automated segmentation of hepatic vessels in non-contrast X-ray CT

images,” Radio. Physics and Tech, vol. 1, no.2, pp. 214-222, 2008.

[6] C. Zhou et al. “Automatic multiscale enhancement and segmentation of pulmonary vessels in

CT pulmonary angiography images for CAD applications,” Med. Physics, vol. 34, no. 12, pp.

4567–4577, 2007.

[7] X. Zhu. “Voles: Vascularity-Oriented Level Set Algorithm For Pulmonary Vessel

Segmentation In Image Guided Intervention Therapy,” IEEE Inter. Symp. on Biomedical Imag. -

From Nano to Macro, pp. 1247-1250, 2009.

[8] I. Bankman, Handbook of Medical Imaging: Processing and Analysis Management, San

Diego, CA: Academic Press, 2000.

[9] B. J. Erickson and B. Bartholmai. “Computer-Aided Detection and Diagnosis at the Start of

the Third Millennium,” Journal of Digital Imag, vol. 15, no. 2, pp. 59-68, 2002.

62

[10] S. P. Edwards. “Computer-Assisted Craniomaxillofacial Surgery,” Oral and Maxillofacial

Surgery Clinics of North America, vol. 22, no. 1, pp.117-134, 2010.

[11] N. Shin et al. “Computer-assisted Surgical Planning for Cerebrovascular Neurosurgery,”

Neurosurgery, vol. 41, no. 2, pp. 403-410, 1997.

[12] P. Markelj et. al. “A review of 3D/2D registration methods for image-guided interventions,”

Medical Image Analysis, vol. 16, pp. 642-661, 2012.

[13] M. Groher et al, “Planning and Intraoperative Visualization of Liver Catherizations: New

CTA Protocol and 2D-3D Registration Method,” Academic Radiology, vol. 14, no. 11, pp. 1325-

1340, 2007.

[14] B. Zitova and J. Flusser, “Image registration methods: a survey,” Image Vis. Comput. Vol.

21, no. 11, pp. 977–1000, 2003.

[15] O. Friman, M. Hindennach, C. Kühne. “Multiple hypothesis template tracking of small 3D

vessel structures,” Medical Image Analysis, vol. 14, pp. 160-171, 2010.

[16] S. R. Aylward. “Initialization, Noise, Singularities, and Scale in Height Ridge traversal for

Tubular Object Centerline Extraction,” IEEE Transactions on Medical Imaging, vol. 21, no. 2,

2002.

[17] T. Pock, C. Janko, R. Beichel and H. Bischof. “Multiscale Medialness for Robust

Segmentation of 3D Tubular Structures,” Proceedings of the Computer Vision Winter

Workshop, 2005.

[18] O. Wink and W. J. Niessen. “Fast Delineation and Visualization of Vessels in 3-D

Angiographic Images,” IEEE Trans. On Medical Imag., vol. 19, no. 4, pp.337-346, 2000.

[19] M. Piccinelli et al. "A Framework for Geometric Analysis of Vascular Structures:

Application to Cerebral Aneurysms," IEEE trans. on medical imaging, vol. 28, no. 8, pp. 1141-

1155, Aug. 2009.

[20] D. Lesage et. al. “A review of 3D vessel lumen segmentation techniques: Models, features

and extraction schemes,” Medical Image Analysis, vol. 13, pp. 819-845, 2009.

63

[21] S. Sherlock and J. Dooley, “Anatomy and function,” in Diseases of the Liver and Biliary

System. John Wiley & Sons, 2008, pp.1-4.

[22] L. Schlossberg and G. Zuidema, “The Liver,” in The Johns Hopkins Atlas of Human

Functional Anatomy. JHU Press, 1997, pp. 137-141.

[23] L. DeLeve and G. Garcia-Tsao, Vascular Liver Disease: Mechanisms and Management,

[24] A. Frangi et al, “Multiscale Vessel Enhancement Filtering,” Lecture Notes in Comput. Sci.,

vol. 1496, pp. 130-137, 1998.

[25] Y. Sato et al, “Three-dimensional multi-scale line filter for segmentation and visualization

of curvilinear structures in medical images,” Medical Image Analysis, vol. 2, no. 2, pp. 143-168,

1998.

[26] L. Cervino and S. Jiang, “Intrafraction Variations and Management Technologies” in

Adaptive Radiation Therapy, USA: CRC Press, 2011, pp. 204.

[27] P. Villard et al, “A Method to Compute Respiration Parameters for Patient-based

Simulators,” Medicine Meets Virtual Reality, no. 173, pp. 529-33, 2012.

[28] A. Khan et al (2012). Liver Metastases Imaging [Online]. Available:

http://emedicine.medscape.com/article/369936-overview

[29] M. Oliva and S. Saini, “Liver cancer imaging: role of CT, MRI, US and PET,” Cancer

Imaging, vol. 4, Spec No A, pp. S42–S46.

[30] A. Jemal et al. “Global Cancer Statistics,” CA Cancer J Clin, vol. 61, no. 2, pp. 69-90,

March/April 2011.

[31] N. Reau and F. Poordad, Primary Liver Cancer: Surveillance, Diagnosis and Treatment,

New York: Springer, 2012, pp. 1-12.

[32] F. Vidal-Vanaclocha, “Architectural and Functional Aspects of the Liver with Implications

for Cancer Metastasis,” in Liver Metastasis: Biology and Clinical Management, Springer, 2011,

pp. 9-42.

64

[33] Advanced liver diagnosis: Computed Tomography (CT) [Online]. Available:

http://www.liver-imaging.com/scripts/pages/en/liver_imaging/computed_tomography_ct/

[34] M. El Khodary et al, “Imaging of Hepatic Metastases,” in Liver Metastasis: Biology and

Clinical Management, New York: Springer, 2011, pp. 307-352.

[35] A. Berger, “How does it work?: Magnetic Resonance Imaging,” BMJ, vol. 324, no. 7328,

pp. 35, 2002.

[36] K. Brock, “Imaging and Image-Guided Radiation Therapy in Liver Cancer,” Semin

Radiation Onc, vol. 21, no. 4, pp. 247-255, 2011.

[37] P. Dumpuri et al, “Model-updated image-guided liver surgery: Preliminary results using

surface characterization,” Prog Biophys Mol Biol., vol. 103, no. 2-3, pp. 197-207, 2010.

[38] H. Rempp et al, “Image-Based Monitoring of Magnetic Resonance-Guided Thermoablative

Therapies for Liver Tumors,” Cardiovasc Intervent Radiol, vol. 35, no. 6, pp. 1281-94, Dec.

2012.

[39] D. Cash, “Concepts and Preliminary Data Toward the Realization of Image-guided Liver

Surgery,” Journ Gastrointest Surg, vol. 11, no. 7, pp. 844-59, 2007.

[40] D. Ribero, “Epidemiology and Natural History,” in Surgical Treatment of Colorectal Liver

Metastases, Milan: Springer, 2011.

[41] Huang et al. “Extraction of liver vessel centerlines under guidance of patient-specific

models,” in Engineering in Medicine and Biology Society, San Diego, CA, 2012.

[42] Osario et al. “Accurate CT/MR vessel-guided nonrigid registration of largely deformed

livers,” Med Phys, vol. 39, no. 5, May 2012.

[43] H. Keller, Lectures on Numerical Methods in Bifurcation Theory, Bangalore: Tata Institute

of Fundamental Research, 1986, pp.66.

[44] C. Kelley (2005). Arclength Continuation and Bifurcation[Online]. Available:

http://www4.ncsu.edu/~ctk/MA784/continue.pdf

65

[45] Erdt et al. “Automatic Hepatic Vessel Segmentation Using Graphics Hardware,” Lec. Not.

Comp. Sci., vol. 5128, pp. 403-412, 2008.

[46] A. Muratore and L. Capussotti. “Evolution of Resectability Criteria,” in Surgical Treatment

of Colorectal Liver Metastases, Turin: Springer, 2011, pp. 28.

[47] R. Cirocchi et al. “Radiofrequency ablation in the treatment of liver metastases from

colorectal cancer,” Cochrane Database Syst. Rev., no.6, Jun 13, 2012.

[48] T. Vogl et al. “Thermal ablation therapies in patients with breast cancer liver metastases: A

review,” Eur. Radiol, vol. 23, no. 3, pp. 797-804, 2013.

[49] K. Hynynen. “MRIgHIFU: A Tool for Image-Guided Therapeutics,” Journ. Magn.

Resonan. Imag., vol. 34, pp. 482-493, 2011.

[50] S. Weeratunga and C Kamath. “An Investigation of Implicit Active Contours for Scientific

Image Segmentation,” Vis Comm Image Process Conf, San Jose, CA, 2004.

[51] P. Yim et al. “Gray-Scale Skeletonization of Small Vessels in Magnetic Resonance

Angiography,” IEEE Trans. Med. Imag., vol. 19, no. 6, 2000.

[52] J. Tyrrell et al. “Robust 3-D Modeling of Vasculature Imagery Using Superellipsoids,”

IEEE Trans. Med. Imag., vol. 26, no. 2, 2007.

[53] Y Fridman et al. “Segmenting 3D Branching Tubular Structures Using Cores,” MICCAI

2003, LNCS, vol 2879, pp. 570–577, 2003.

[54] O. Catalano et al. “Vascular and Biliary Variants in the Liver: Implications for Liver

Surgery,” Radiographics, vol. 28, no. 2, pp. 359-78, 2008.

[55] E. Serradell et al. “Simultaneous Correspondence and Non-Rigid 3D Reconstruction of the

Coronary Tree from Single X-ray Images,” Intern Conf on Comp Vision, 2011.

[56] H. Chui and A. Rangarajan. “A new point matching algorithm for non-rigid registration,”

Comp. Vis. Image Underst., vol. 89, pp. 114-141, 2003.

66

[57] P. Giraud and A. Houle. “Respiratory Gating for Radiotherapy: Main Technical Aspects and

Clinical Benefits,” ISRN Pulmonology, vol. 2013, id. 519602, 2013.

[58] C. Ethier and C. Simmons. “Hemodynamics,” in Introductory Biomechanics: From cells to

Organisms, Cambridge: Cambridge University Press, 2007, p.136.

[59] B. J. Erickson and B. Bartholmai. “Computer-Aided Detection and Diagnosis at the Start of

the Third Millennium,” Journal of Digital Imag, vol. 15, no. 2, pp. 59-68, 2002.

[60] S. Kawajiri et al. “Automated segmentation of hepatic vessels in non-contrast X-ray CT

images,” Radio. Physics and Tech, vol. 1, no.2, pp. 214-222, 2008.

[61] C. Zhou et al. “Automatic multiscale enhancement and segmentation of pulmonary vessels

in CT pulmonary angiography images for CAD applications,” Med. Physics, vol. 34, no. 12, pp.

4567–4577, 2007.

[62] M. Lin et al. “Algorithm-Based Method for Detection of Blood Vessels in Breast MRI for

Development of Computer-Aided Diagnosis,” Jour. Of Magn. Resonance Imaging, vol. 30, pp.

817-824, 2009.

[63] S. P. Edwards. “Computer-Assisted Craniomaxillofacial Surgery,” Oral and Maxillofacial

Surgery Clinics of North America, vol. 22, no. 1, pp.117-134, 2010.

[64] Q. Shang et. al. “Adaptive Directional Region Growing Segmentation of the Hepatic

Vasculature,” Proc. Of the Soc. Of Photo-Optical Instr. Eng., vol. 6914, pp. F9141, 2008.

[65] K. S. Lehmann et al. “Portal Vein Segmentation of a 3D-Planning System for Liver

Surgery—In vivo Evaluation in a Porcine Model,” Annals of Surg. Oncology, vol. 15, no. 7, pp.

1899-1907, 2008.

[66] M. Descoteaux, D. L. Collins, K. Siddiqui. “A geometric flow for segmenting vasculature in

proton-density weighted MRI,” Medical Image Analysis, vol. 12, pp. 497-513, 2008.

[67] N. Shin et al. “Computer-assisted Surgical Planning for Cerebrovascular Neurosurgery,”

Neurosrugery, vol. 41, no. 2, pp. 403-410, 1997.