recent clinical advances and applications for medical...
TRANSCRIPT
copy L Joskowicz 2011
Recent clinical advances and applications
for medical image segmentation
Prof Leo Joskowicz
Lab website httpwwwcshujiacil~caslabsite
copy L Joskowicz 2011
Key trends in clinical radiology bull Filmlight table Digital imagesscreen -- early 80rsquo
bull 2D X-rays US 25D CT MRI -- mid 80rsquo ndash 90rsquo
bull 25D CT MRI 3D visualization -- mid 00lsquo
bull 3D visualization 3D anatomical modeling -- now
bull 3D modeling Advanced modeling ndash coming soon
copy L Joskowicz 2011
Centerline and lumen
Colon and intestine surface Abdominal CT scan
What is a patient-specific 3D model
Surface model
Polyp
copy L Joskowicz 2011
What is the difference between 3D
visualization and 3D modeling
Visualization Model rendering
You do the interpretation
You fillomit missing info
Computer interprets
Explicit delineation
copy L Joskowicz 2011
Why patient-specific modeling
bull 3D visualization is great -- but it has limitationshellip
minus no explicit delineation
minus no validation ndash what you see is what is there
minus limited measurements
minus limited structures discrimination
bull 3D models allow
minus spatial and volumetric measurements ndash with validation
minus advanced analysis
minus wide variety of uses in the treatment cycle
minus reduction of radiologist time easier learning curve
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Copyright L Joskowicz 2007
Segmentation in commercial
systems
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Key trends in clinical radiology bull Filmlight table Digital imagesscreen -- early 80rsquo
bull 2D X-rays US 25D CT MRI -- mid 80rsquo ndash 90rsquo
bull 25D CT MRI 3D visualization -- mid 00lsquo
bull 3D visualization 3D anatomical modeling -- now
bull 3D modeling Advanced modeling ndash coming soon
copy L Joskowicz 2011
Centerline and lumen
Colon and intestine surface Abdominal CT scan
What is a patient-specific 3D model
Surface model
Polyp
copy L Joskowicz 2011
What is the difference between 3D
visualization and 3D modeling
Visualization Model rendering
You do the interpretation
You fillomit missing info
Computer interprets
Explicit delineation
copy L Joskowicz 2011
Why patient-specific modeling
bull 3D visualization is great -- but it has limitationshellip
minus no explicit delineation
minus no validation ndash what you see is what is there
minus limited measurements
minus limited structures discrimination
bull 3D models allow
minus spatial and volumetric measurements ndash with validation
minus advanced analysis
minus wide variety of uses in the treatment cycle
minus reduction of radiologist time easier learning curve
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Copyright L Joskowicz 2007
Segmentation in commercial
systems
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Centerline and lumen
Colon and intestine surface Abdominal CT scan
What is a patient-specific 3D model
Surface model
Polyp
copy L Joskowicz 2011
What is the difference between 3D
visualization and 3D modeling
Visualization Model rendering
You do the interpretation
You fillomit missing info
Computer interprets
Explicit delineation
copy L Joskowicz 2011
Why patient-specific modeling
bull 3D visualization is great -- but it has limitationshellip
minus no explicit delineation
minus no validation ndash what you see is what is there
minus limited measurements
minus limited structures discrimination
bull 3D models allow
minus spatial and volumetric measurements ndash with validation
minus advanced analysis
minus wide variety of uses in the treatment cycle
minus reduction of radiologist time easier learning curve
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Copyright L Joskowicz 2007
Segmentation in commercial
systems
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
What is the difference between 3D
visualization and 3D modeling
Visualization Model rendering
You do the interpretation
You fillomit missing info
Computer interprets
Explicit delineation
copy L Joskowicz 2011
Why patient-specific modeling
bull 3D visualization is great -- but it has limitationshellip
minus no explicit delineation
minus no validation ndash what you see is what is there
minus limited measurements
minus limited structures discrimination
bull 3D models allow
minus spatial and volumetric measurements ndash with validation
minus advanced analysis
minus wide variety of uses in the treatment cycle
minus reduction of radiologist time easier learning curve
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Copyright L Joskowicz 2007
Segmentation in commercial
systems
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Why patient-specific modeling
bull 3D visualization is great -- but it has limitationshellip
minus no explicit delineation
minus no validation ndash what you see is what is there
minus limited measurements
minus limited structures discrimination
bull 3D models allow
minus spatial and volumetric measurements ndash with validation
minus advanced analysis
minus wide variety of uses in the treatment cycle
minus reduction of radiologist time easier learning curve
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Copyright L Joskowicz 2007
Segmentation in commercial
systems
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Copyright L Joskowicz 2007
Segmentation in commercial
systems
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Segmentation in commercial systems
Copyright L Joskowicz 2007
copy L Joskowicz 2011
Copyright L Joskowicz 2007
Segmentation in commercial
systems
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Copyright L Joskowicz 2007
Segmentation in commercial
systems
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
3D models in the patient treatment cycle
Diagnosis Planning
Delivery
CAD mammography
Virtual colonoscopy
Neurosurgery -- trajectory
Orthopaedics -- fixation
Tumor follow-up
Implant location
MODEL Interventional
Radiology
Navigation
Robotics Evaluation
Training
Learning
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Diagnosis computer-aided radiology
stenosis
thrombus aneurism
tumor
volume
with Prof J Sosna
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
liver contour
blood vessels
tumors
4-phase CT dataset
kidneycontour
blood vessels
urinary vessels with Dr Y Mintz with Prof J Sosna
Diagnosis computer-aided radiology
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Planning neurosurgery
entry point
with Dr Y Shoshan
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Conventional Our method
Planning neurosurgery
with Dr Y Shoshan
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Planning orthopaedics
internal fixation external fixation
Fracture
fixation
alternatives
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011 with Dr E Peleg and Profs M Liebergall R Mosheiff
Planning orthopaedics
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Planning orthopaedics
with Dr E Peleg and Profs M Liebergall R Mosheiff
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
bull EM real-time tracking
bull US and X-ray imaging
add patient-specific
models from CT
Delivery interventional radiology
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Delivery intraoperative image guidance
augmented continuous X-ray fluoroscopy
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Delivery intraoperative image guidance
with Simbionix and Prof J Sosna
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Evaluation brain tumor follow-up
Dec 15 2009 June 9 2010
Disease
progression T2-weighted T1-weighted
Tumor internal components
solid enhancing cyst
with TA Sourasky and Dana Hospital
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
copy L Joskowicz 2011
Key issue model creation
Currently
bull mostly manual delineation
bull slice by slice
Desired
bull automaticnearly automatic ndash a few
clicks by physician
bull no technician
bull accurate and reliable
HARD NO METHOD
SUITABLE FOR ALL STRUCTURES
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Modeling requires segmentation
Very difficult task
ndash organpathology specific
ndash imagescanning protocol
ndash anatomical variability
ndash intensity values overlap
ndash structures proximity
Identify and delineate anatomical structure contours
1 2
3 4
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Clinical dataset challenges
proximity calcifications stenosis
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Segmentation ndash state of the art
bull Hundreds anatomical segmentation methods for a
variety of structures and imaging modalities
bull Families of techniques thresholding region
growing level sets active contours and more
bull Very few if any in routine clinical use
ndash huge gap between prototype and clinical use
ndash time-consuming fragile limited in scope
ndash require technical knowledge
ndash most have limited validation
ndash clinical benefits unproven
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
bull Segmentation specificity is unavoidable ndash each
anatomical structure and pathology has unique
characteristics and nearby structures
bull Organ structure and pathology-specific
algorithms heart liver long bones spine
bull Very laborious top develop and validate a
segmentation algorithm for each ndash trial and error
process many man-months
No universal segmentation method
Observations (1)
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Observations (2) bull Applications have different requirements wrt
ndash Accuracy
ndash Robustness
ndash User interaction
ndash Quality
bull Consider the different requirements of
ndash 3D visualization
ndash Training simulation
ndash FEA simulation
Defining application
requirements ahead of time is
essential
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Goals Clinical
bull Automatic or nearly automatic lt 5mins user time
bull Can be used by clinician without a technician
bull Produces robust results
Technical
bull Requires significantly less time than developing from
scratch
bull Takes into account intensity and shape
bull Incorporates shape and intensity priors
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
MIS validation
bull Algorithms without validation are clinically
worthless
bull Validation is with respect to a clinical task
bull Validation requires a ground truth for comparison
bull Physical anatomical models andor phantoms are
typically not available (except sometimes for bones)
bull Ground truth is usually obtained by manual
identification andor segmentation by a user
bull Experts in most cases radiologists
bull Extrinsic comparison compare vs other methods
Copyright L Joskowicz 2010
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Copyright L Joskowicz 2010
MIS validation issues
bull Large inter- and intra- variability across experts and clinical sites
bull May not be representative of population variability
bull Main quantitative parameters
ndash validation set size
ndash number and type of observers
ndash intra and inter-observer manual segmentation variability
ndash surface-based and volume-based error measures
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
MIS validation anatomical variability
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
stenosis
looping
narrowing
MIS validation anatomical variability
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
MIS validation metrics
bull Surface-based error measurements
ndash Mean surface distance
ndash RMS surface distance
ndash Maximum surface distance
bull Volume-based error measurements
ndash Dice coefficient
ndash Volumetric overlap error
Copyright L Joskowicz 2010
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
MIS validation metrics
Surface-based error measurements
bull Mean surface distance
bull RMS surface distance
bull Maximum surface distance
Copyright L Joskowicz 2010
Reference Result
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
MIS validation metrics
Volume-based error measurements
bull Volumetric overlap error
bull Dice similarity
Copyright L Joskowicz 2010
Reference Result
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Aorta
Int + Ext Carotids
All arteries
Subclavian Arteries Bifurcations
16mm (std=08mm)
13mm (std=10mm)
15mm
std=13mm 17mm
std=09mm
07mm
std=07mm
Surface
RMS
Common
Carotid
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
MIS validation observers variability
bull Intra-observer variability determine how much
variation there is when a single observer
produced the ground-truth segmentation
repeat 5 times the segmentation
bull Inter-observer variability determine how much
variation there is between multiple observers that
produced the ground-truth segmentation
ask 3 radiologists to do the
segmentation
bull Observer expertise and frequency variability
Copyright L Joskowicz 2010
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Study [Weltens 2001]
bull Axial MRI slice
bull Nine independent
observers
bull Repeated delineations
bull Interintra observer
variability 30
Key issue fuzzy tumor boundaries
Inter observer variability
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Intra-observer variability
8
50
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Carotid Lumen Segmentation and Stenosis
Grading Challenge -- The MIDAS Journal
Copyright L Joskowicz 2010
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
MIS validation manual annotation
Copyright L Joskowicz 2010
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
MIS validation ground-truth generation
centerlines
(manual)
contours
(manual)
Partial Volume
Segmentation
Reference
standard
from 3PVS
Copyright L Joskowicz 2010
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Measurements characteristics
bull Ground-truth not know
bull Repeatability intra-observer variability
bull Reliability inter observer variability
bull Measure correlation between repeated
measurements
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
bull Intrinsic uncertainty about the tumor volume
bull Accuracy for clinical significance is unknown
Inaccuracy gt Uncertainty
bull Uncertainty may not be improved
bull Goal improve accuracy to obtain
Inaccuracy vs Uncertainty
Results can be
meaningless
Inaccuracy lt Uncertainty
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
In summaryhellip
bull Medical image segmentation is on the risehellip
bull Basis of many clinical applications
bull Validation is a MUST
bull Many methods and approaches ndash do not re-invent
the wheel
bull I expect patient-specific 3D models to be in the
clinical mainstream in 3-5 years
Copyright L Joskowicz 2011
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Segmentation challenges
bull Rigorous quantification and evaluation of
segmentation algorithms performance
bull Shape priors for pathologies
bull Incorporation of functional information from
diffusionperfusion MRI fMRI PET into
segmentation algorithms
Copyright L Joskowicz 2011
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Summary (1)
bull Patient-specific anatomy model creation is
currently a major bottleneck in many clinical
applications
bull Automatic anatomical segmentation is essential for
model creation
bull Current tools are limited in scope and coverage
bull Clinical use requires the elimination of the
technician ndash model generation by the physician
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms
Summary (2) bull Great opportunity for the development and
incorporation of anatomy modeling tools in
commercial platforms
bull Growing need and variety of users for anatomical
models
ndash Training simulators surgery rehersal
ndash Intraoperative guidance
ndash Computer Aided Radiology
bull Service providers -- shifting paradigms