Organ localisation in Fetal MRIPhD started in October 2011
Kevin Keraudren
Steiner Unit, Hammersmith Hospital
June 8th, 2012
Introduction What about fetuses? Own research Future plans
1 Introduction
2 What about fetuses?
3 Own research2D detector for fetal headsEye detection
4 Future plans
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 2 / 31
Introduction What about fetuses? Own research Future plans
1 Introduction
2 What about fetuses?
3 Own research
4 Future plans
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 3 / 31
Introduction What about fetuses? Own research Future plans
Organ localisation
What?Automatic detection of organs: center, bounding box or roughsegmentation.
Why?Segmentation, measurements, optimised or standardisedvisualisations, browsable body (keyword queries), tagging...
How?Machine Learning methods, trained on relevant Training Data.
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 4 / 31
Introduction What about fetuses? Own research Future plans
Adults?2 main groups on this subject:
Microsoft (A. Criminisi)
Siemens (D. Comaniciu)
Main focus on CT data from adults:
CT on adult MRI on fetusModality intensities directly related to
tissuesrelative values and field in-homogeneities
Subject lay still, can hold breath,aligned with the scanner
movement, random orien-tation and position of thelimbs, bending of the spine
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 5 / 31
Introduction What about fetuses? Own research Future plans
Siemens: “Hierarchical Parsing and Semantic Navigation of Full Body CTData”, Seifert et al. (2009)
Slice detection Landmark detection Object detection+ shape refinement
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 6 / 31
Introduction What about fetuses? Own research Future plans
Microsoft: “Regression Forests for Efficient Anatomy Detection andLocalization in CT Studies”, Criminisi et al. (2011)
Automatic organ detection, repositioning of the 3D cameraand selection of the best colour transfer function
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 7 / 31
Introduction What about fetuses? Own research Future plans
1 Introduction
2 What about fetuses?
3 Own research
4 Future plans
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 8 / 31
Introduction What about fetuses? Own research Future plans
“Automatic Segmentation of Head Structures on Fetal MRI”, Anquez etal. (2009) [FEMONUM]
24 MRI volumes,30-35 gestational weeks,
0.94×0.94×4 mm3
no motion
Modeling the eyes fortemplate matching:
real eye, model and template
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 9 / 31
Introduction What about fetuses? Own research Future plans
“Automatic Segmentation of Head Structures on Fetal MRI”, Anquez etal. (2009) [FEMONUM]
24 MRI volumes,30-35 gestational weeks,
0.94×0.94×4 mm3
no motion
Graph-cut segmentationof the brain, with rotated
template around eyescenter
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 10 / 31
Introduction What about fetuses? Own research Future plans
“Segmentation of the Fetal Envelope on Ante-Natal MRI”, Anquez et al.(2010)
Semi-automatedregistration of a
simplified skeleton
Mean shape, graphcut initialisation andresulting segmentation
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 11 / 31
Introduction What about fetuses? Own research Future plans
“Detection of Fetal Anatomies from Ultrasound Images using aConstrained Probabilistic Boosting Tree”, Carneiro et al. (2008)
From an annotated database,learn to detect organs
Head and whole body
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 12 / 31
Introduction What about fetuses? Own research Future plans
1 Introduction
2 What about fetuses?
3 Own research2D detector for fetal headsEye detection
4 Future plans
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 13 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
Sagittal head detectorTraining data (Thanks Vanessa!)
303 tightly cropped images
Average image
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 14 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
Sagittal head detector
Viola&Jones detector fromOpenCV
Local Binary Pattern (LBP)features
Exhaustive search byrotating/mirroring new image
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 15 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
LBP features
6 7 8
4 5
1 2 3
black if∑Neighbour < ∑Middle
else white
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 16 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
LBP features
First tests of the detection process
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 17 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
Detection result
5min for a 1◦ rotation step,less than 1min if 8◦ rotation step.
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 18 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
Coronal head detectorTraining data
666 tightly cropped images
Average image
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 19 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
Transverse head detectorTraining data
246 tightly cropped images + mirrored
Average image
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 20 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
Neck region darker than the brain
Influence of background
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 21 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
Detection result
In volumes, multiple detections increase confidence
Movie as example of true and false detection
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 22 / 31
Introduction What about fetuses? Own research Future plans
2D detector for fetal heads
Room for improvement
Specific task, specific training data: no universal detector
Need for more testing, better separation of training and testingdata
Confidence score, and combining 3 detectors into a classifier
Combine with other detectors (eyes?) to increase confidence andorientation
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 23 / 31
Introduction What about fetuses? Own research Future plans
Eye detection
Eye detection pipeline
1 Slice by slice:1 Canny edge detection + template matching2 Intensity filtering
2 For the whole volume:1 Grouping candidate circles2 Forming candidate pairs (25 mm ≤ d ≤ 39 mm)3 Chan&Vese Level Sets segmentation + shape
checking4 Grouping candidate pairs
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 24 / 31
Introduction What about fetuses? Own research Future plans
Eye detection
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 25 / 31
Introduction What about fetuses? Own research Future plans
Eye detection
Problem of motion (here 1.3mm spacing)
Several results returned, ranked by number of detection
Is it worth having a more complex pipeline than FEMONUM?
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 26 / 31
Introduction What about fetuses? Own research Future plans
1 Introduction
2 What about fetuses?
3 Own research
4 Future plans
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 27 / 31
Introduction What about fetuses? Own research Future plans
Future plans
Validate and integrate current detectors
Target other organs: dataset of ' 50 fetuses with lungs, heart,kidneys, liver and whole body (Damodaram et al., 2012)
Regression Forests, multi-organs detection
Long-term goals:
Improve visualisation of fetal MRI data
(Semi-)Automatic building of catalogues of organs
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 28 / 31
Introduction What about fetuses? Own research Future plans
Thanks!
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 29 / 31
Introduction What about fetuses? Own research Future plans
Questions?
K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 30 / 31