Download - Head and Neck Lymph Node Region Delineation with Auto-segmentation and Image Registration
11
Head and Neck Lymph Head and Neck Lymph Node Region Delineation Node Region Delineation with Auto-segmentation with Auto-segmentation and Image Registrationand Image Registration
Chia-Chi TengChia-Chi Teng
Department of Electrical EngineeringDepartment of Electrical Engineering
University of WashingtonUniversity of Washington
22
OutlineOutline
IntroductionIntroduction Related WorkRelated Work Lymph Node Region Contouring with Lymph Node Region Contouring with
Image RegistrationImage Registration Automatic Segmentation of Landmark Automatic Segmentation of Landmark
StructuresStructures Geometrical Feature Based SimilarityGeometrical Feature Based Similarity ResultsResults ConclusionConclusion
33
ContextContext
3D Conformal Radiotherapy 3D Conformal Radiotherapy (beams (beams are shaped to match the tumor)are shaped to match the tumor)
Intensity Modulated Radiation Intensity Modulated Radiation Therapy Therapy (controls intensity in small volumes)(controls intensity in small volumes)
55
MotivationMotivation
Improve the process of target Improve the process of target volume delineation for volume delineation for radiation therapy planning.radiation therapy planning.
Objective:Objective:– Auto-contour lymph node Auto-contour lymph node
regions.regions.– Initial focus on head and neck.Initial focus on head and neck.
66
ProblemProblem
Where are the lymph nodes?Where are the lymph nodes? Where are the lymph node regions?Where are the lymph node regions?
none of thestructures arelymph nodes
77
SolutionSolution
Create reference (canonical) models.Create reference (canonical) models. Map reference nodal regions to patients.Map reference nodal regions to patients.
88
System OverviewSystem Overview
Segmentation of Landmark Structures
Target CT Images
Projected Lymph Node Regions
Retrieve Similar Reference Models
3D Volume and Mesh of Mandible, Hyoid, ...
Image Registration
Reference Models with Lymph Node Regions
99
Image RegistrationImage Registration
Align the transformed reference image Align the transformed reference image ffRR ° ° gg to the target image to the target image ffT T . .
Find the optimal set of transformation Find the optimal set of transformation parameters parameters that maximize an image that maximize an image similarity function similarity function SS::
optimaloptimal = argmax = argmax SS(())
1010
Mattes’ MethodMattes’ Method
Similarity FunctionSimilarity Function
SS(()) = mutual_information( ffRR ° ° gg , f, fT T ))
Transformation FuncitonTransformation Funciton
gg((xx||) = ) = RR((x x - - xxCC) – ) – TT((x x - - xxCC) + ) + DD((xx||))
x x = [= [x, y, zx, y, z]]T T in the reference image in the reference image coordinates.coordinates.
1111
DeformableDeformable TransformationTransformation
Control points Control points (15*15*11).(15*15*11).
Each control point is Each control point is associated with a 3-associated with a 3-element deformation element deformation vector vector , describing , describing x-, y-, z-components x-, y-, z-components of the deformation.of the deformation.
1212
Project Target Lymph Project Target Lymph RegionsRegions
Image registration aligns Image registration aligns reference and target CT sets.reference and target CT sets.
Apply result transformation Apply result transformation gg to to reference lymph node regions.reference lymph node regions.
Incorporate anatomical landmark Incorporate anatomical landmark correspondences.correspondences.
Use surface mesh of outer body Use surface mesh of outer body contour, mandible, hyoid …contour, mandible, hyoid …
1313
Surface WarpingSurface Warping
Shelton’s method used to find Shelton’s method used to find
correspondences between surfaces.correspondences between surfaces. Energy based surface mesh warping.Energy based surface mesh warping.
E(C) = Esim(C) + Estr(C) + Epri(C)
C is the function which maps points from reference surface SSRR to target surface SST T .
1414
Landmark Landmark CorrespondenceCorrespondence
The deformation The deformation at landmark points at landmark points
kk = = kk kk
kk : points from reference surface mesh : points from reference surface mesh SSRR..
kk : corresponding locations on : corresponding locations on transformed reference surface transformed reference surface SSRR ° ° CC matching the target surface mesh matching the target surface mesh SSTT..
1616
Using Landmark Using Landmark CorrespondenceCorrespondence Deformation vectorsDeformation vectors DD((jj) ) are are
modified according to landmark modified according to landmark correspondencescorrespondences kk in the in the proximity of the control pointsproximity of the control points jj..
Landmark structures align better.Landmark structures align better. Faster convergence.Faster convergence.
1717
Reference Mattes w/ Landmark Target
Compare Image Compare Image Registration ResultsRegistration Results
1919
Automatic Automatic Segmentation of Segmentation of Landmark StructuresLandmark Structures Given: Cancer radiation treatment
patient’s head and neck CT image. Find:
– Skull base & thoracic inlet.– Anatomical structures:
cervical spine (white) respiratory tract (dark green) mandible (turquoise) hyoid (yellow) thyroid cartilage internal jugular veins (pink) carotid arteries (dark yellow) sternocleidomastoid muscles (light green,
orange)
2020
MethodMethod
2D knowledge-based segmentation– Based on Kobashi’s work– Dynamic thresholding– Progressive landmarking
Combined with 3D active contouring– Do not require successful 2D Do not require successful 2D
segmentation on every axial slicesegmentation on every axial slice– Initialize with 2D segmentation result
2222
2D/3D Iteration2D/3D Iteration
1 3 5
2 4 6
Identify objects that are easy to find, use them to find harder ones.
2424
Geometrical Feature- Geometrical Feature- Based SimilarityBased Similarity Given: A stored database DB of CT
scans from prototypical reference head and neck cancer patients and a single query CT scan Q from a target patient.
Find: Similarity between Q and each database image d in DB in order to find the most similar database images {ds}.
2525
StructuresStructures
Outer body Outer body contourcontour
MandibleMandible HyoidHyoid Internal Internal
jugular jugular veinsveins
2626
Feature TypesFeature Types
Simple Simple numeric 3D regional numeric 3D regional properties: volume and extents. properties: volume and extents.
VectorVector properties: relative properties: relative location between structures.location between structures.
ShapeShape properties: surface meshes properties: surface meshes of structures.of structures.
2727
Features for Similarity Features for Similarity MeasureMeasure
Volume and extents Volume and extents of the overall of the overall regionregion
Normalized Normalized centroidcentroid of hyoid and of hyoid and mandiblemandible
3D 3D centroid difference vector centroid difference vector between between mandible and hyoidmandible and hyoid
2D 2D centroid difference vectorscentroid difference vectors between between hyoid and jugular veinshyoid and jugular veins
Surface meshes Surface meshes of mandible and outer of mandible and outer body contourbody contour
2828
Mesh Feature Distance Mesh Feature Distance
Register reference mesh Register reference mesh SR and and target mesh target mesh ST with Iterative with Iterative Closest Point (ICP), result Closest Point (ICP), result T..
Hausdorff distance Hausdorff distance between between two aligned surface meshes, TSR and ST
),(max),( TSp
TRh STpdSTSdR
The Hausdorff distance is the maximum distance from anypoint in the transformed reference image to the test image.
2929
Feature Vector Feature Vector DistanceDistance Given feature vectors Given feature vectors FFdd and and FFQQ
for model for model dd and query and query QQ in the in the feature vector space feature vector space RRNN..
2
1
2
1
),(),(
iQ
N
iidiiQdF FFdwFFD
3030
EvaluationEvaluation Surface mesh distance after full Surface mesh distance after full
image registration image registration DDHH – slow. – slow. Feature vector distance Feature vector distance DDFF – fast. – fast.
DH
DF
corr_coef(corr_coef(DDHH, , DDFF) )
= 0.72= 0.72
Images with smallfeature vector distanceshould produce the bestresults after registration.
3131
Experiment ResultsExperiment Results
50 head and neck patient CT sets.50 head and neck patient CT sets. 34 subjects are segmented.34 subjects are segmented. 20 subjects with lymph node 20 subjects with lymph node
regions drawn by experts.regions drawn by experts. Image registrationImage registration
20 * (20 – 1) = 380 total cases.20 * (20 – 1) = 380 total cases.
3333
Auto-segmentation Auto-segmentation cont.cont. Incorrect SegmentationsIncorrect Segmentations
Carotid artery misidentified as Hyoid partly missing due tojugular vein due to surgery. too low inter-slice resolution.
3434
Successs Failure Incorrect % of success
Cervical Spine 34 0 0 100.00%
Respiratory Tract 34 0 0 100.00%
Mandible 34 0 0 100.00%
Hyoid 34 0 0 100.00%
ThyroidCartilage 33 0 1 97.06%
Left Internal Jugular Vein 27 3 4 79.41%
Right Internal Jugular Vein 31 1 2 91.18%
Left Carotid Artery 25 9 0 73.53%
Right Carotid Artery 30 4 0 88.24%
Left SCM 24 10 0 70.59%
Right SCM 25 9 0 73.53%
Auto-segmentation Auto-segmentation cont.cont.
3535
Image Registration Image Registration ResultsResults
Total cases Successful Success rate (%)
Mattes method 380 367 96.57%
New method using landmark correspondence 380 380 100.00%
Average Standard deviation
Mattes method 32 minutes 6 minutes
New method using landmark correspondence 26 minutes 5 minutes
Time of Convergence
Success/Failure
3636
Quantitative Quantitative Evaluation -Evaluation - Surface Surface Mesh DistanceMesh Distance
Projected Region SR ° ° ggColor is Color is distance to distance to truth.truth.
Ground Truth: Expert Drawn Target Region ST
DH(SR ° ° gg, ST, n) : Hausdorff distancen : lymph node region
3737
Average Standard deviation
Mattes method 2.85 1.44
New method using landmark correspondence 2.12 0.64
DH(SR ° ° gg, ST, 1B) for all SR, ST.
Measurement in centimeter.
Mattes distance larger thanlandmark distance.
3838
Average Standard deviation
Mattes method 1.02 0.51
New method using landmark correspondence 0.59 0.21
Mean_distance(SR ° ° gg, ST, 1B) for all SR, ST.
Measurement in centimeter.
3939
Similarity EvaluationSimilarity Evaluation
RH(i, Q) : the ith ranked reference subject for target Q based on the image registration results, DH.
RF(i, Q) : the ith ranked reference subject based on geometrical features, DF.
P(P(RRFF(1(1, Q, Q)) ==RRHH((11, Q, Q)) = 80%)) = 80%
P(P(RRFF(1(1, Q, Q)) ==RRHH((22, Q, Q)) = 10%)) = 10%
P(P(RRFF(1(1, Q, Q)) ==RRHH((33, Q, Q)) = 4%)) = 4%
Q
di
HausdorffDistance
FeatureDistance
RH(1, Q)
RH(2, Q)
RF(1, Q)
RF(2, Q)
4040
Similarity Evaluation Similarity Evaluation ExamplesExamples
DH DH
DF DF
corr_coef(corr_coef(DDHH, , DDFF) )
= 0.74= 0.74
corr_coef(corr_coef(DDHH, , DDFF) )
= 0.68= 0.68
4141
Similarity Evaluation – Similarity Evaluation – Surface Mesh DistanceSurface Mesh Distance
AveragAveragee
Standard Standard deviatiodeviationn
DDH H for the for the closest closest
reference subject reference subject to each target to each target based on feature based on feature distancedistance 1.28 1.28 0.310.31
DDH H for all reference for all reference
and target subjectsand target subjects 2.592.59 0.900.90Measurement in centimeter. So its better to find the closest subject.
4242
Qualitative Evaluation – Qualitative Evaluation – 1.11.1
Mattes Expert w/ Landmark Drawn
Clinically acceptable target projection.Clinically acceptable target projection.
4343
Mattes Expert w/ Landmark Drawn
Qualitative Evaluation – Qualitative Evaluation – 1.21.2 Clinically acceptable target projection.Clinically acceptable target projection.
4444
Mattes Expert w/ Landmark Drawn
Qualitative Evaluation – Qualitative Evaluation – 1.31.3 Clinically acceptable target projection.Clinically acceptable target projection.
4545
Mattes Expert w/ Landmark Drawn
Qualitative Evaluation – Qualitative Evaluation – 22 Clinically Clinically unacceptableunacceptable target target
projection.projection.
4646
ConclusionConclusion
Inter-subject image registration Inter-subject image registration technique shows promise for lymph technique shows promise for lymph node region auto-contouring.node region auto-contouring.
Knowledge-based auto-segmentation Knowledge-based auto-segmentation is useful for head and neck CT.is useful for head and neck CT.
Fast similar subject search is Fast similar subject search is possible and critical as reference possible and critical as reference database grows.database grows.
4747
Future WorkFuture Work
Integrate and evaluated in a clinical Integrate and evaluated in a clinical environment.environment.
Generalize to other types of cancer.Generalize to other types of cancer. Regional lymphatic involvement prediction.Regional lymphatic involvement prediction. Improve image registration results.Improve image registration results. Improve auto-segmentation results.Improve auto-segmentation results.
– Validation logicValidation logic– Knowledge-based 3D active contour constraintsKnowledge-based 3D active contour constraints
4848
AcknowledgementAcknowledgement
Linda ShapiroLinda Shapiro Ira KaletIra Kalet Jim BrinkleyJim Brinkley David HaynorDavid Haynor David MattesDavid Mattes Mark WhippleMark Whipple Jerry BarkerJerry Barker Carolyn RutterCarolyn Rutter Rizwan Rizwan NuraniNurani
4949
ContributionsContributions
The first auto target contouring tool for The first auto target contouring tool for radiation therapy. radiation therapy. (AMIA 2002)(AMIA 2002)
An auto-segmentation method combining 2D An auto-segmentation method combining 2D dynamic thresholding and 3D active dynamic thresholding and 3D active contouring. contouring. (IEEE CBMS 2006)(IEEE CBMS 2006)
An image registration method using landmark An image registration method using landmark correspondences in conjunction with mutual correspondences in conjunction with mutual information optimization. information optimization. (IEEE ISBI 2006)(IEEE ISBI 2006)
A patient similarity measurement using 3D A patient similarity measurement using 3D geometrical features of anatomical structures. geometrical features of anatomical structures. (IEEE ISBI 2007)(IEEE ISBI 2007)