Organ localisation in Fetal MRIPhD started in October 2011

Kevin Keraudren

Steiner Unit, Hammersmith Hospital

June 8th, 2012

Introduction What about fetuses? Own research Future plans

1 Introduction

2 What about fetuses?

3 Own research2D detector for fetal headsEye detection

4 Future plans

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 2 / 31

Introduction What about fetuses? Own research Future plans

1 Introduction

2 What about fetuses?

3 Own research

4 Future plans

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 3 / 31

Introduction What about fetuses? Own research Future plans

Organ localisation

What?Automatic detection of organs: center, bounding box or roughsegmentation.

Why?Segmentation, measurements, optimised or standardisedvisualisations, browsable body (keyword queries), tagging...

How?Machine Learning methods, trained on relevant Training Data.

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 4 / 31

Introduction What about fetuses? Own research Future plans

Adults?2 main groups on this subject:

Microsoft (A. Criminisi)

Siemens (D. Comaniciu)

Main focus on CT data from adults:

CT on adult MRI on fetusModality intensities directly related to

tissuesrelative values and field in-homogeneities

Subject lay still, can hold breath,aligned with the scanner

movement, random orien-tation and position of thelimbs, bending of the spine

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 5 / 31

Introduction What about fetuses? Own research Future plans

Siemens: “Hierarchical Parsing and Semantic Navigation of Full Body CTData”, Seifert et al. (2009)

Slice detection Landmark detection Object detection+ shape refinement

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 6 / 31

Introduction What about fetuses? Own research Future plans

Microsoft: “Regression Forests for Efficient Anatomy Detection andLocalization in CT Studies”, Criminisi et al. (2011)

Automatic organ detection, repositioning of the 3D cameraand selection of the best colour transfer function

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 7 / 31

Introduction What about fetuses? Own research Future plans

1 Introduction

2 What about fetuses?

3 Own research

4 Future plans

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 8 / 31

Introduction What about fetuses? Own research Future plans

“Automatic Segmentation of Head Structures on Fetal MRI”, Anquez etal. (2009) [FEMONUM]

24 MRI volumes,30-35 gestational weeks,

0.94×0.94×4 mm3

no motion

Modeling the eyes fortemplate matching:

real eye, model and template

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 9 / 31

Introduction What about fetuses? Own research Future plans

“Automatic Segmentation of Head Structures on Fetal MRI”, Anquez etal. (2009) [FEMONUM]

24 MRI volumes,30-35 gestational weeks,

0.94×0.94×4 mm3

no motion

Graph-cut segmentationof the brain, with rotated

template around eyescenter

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 10 / 31

Introduction What about fetuses? Own research Future plans

“Segmentation of the Fetal Envelope on Ante-Natal MRI”, Anquez et al.(2010)

Semi-automatedregistration of a

simplified skeleton

Mean shape, graphcut initialisation andresulting segmentation

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 11 / 31

Introduction What about fetuses? Own research Future plans

“Detection of Fetal Anatomies from Ultrasound Images using aConstrained Probabilistic Boosting Tree”, Carneiro et al. (2008)

From an annotated database,learn to detect organs

Head and whole body

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 12 / 31

Introduction What about fetuses? Own research Future plans

1 Introduction

2 What about fetuses?

3 Own research2D detector for fetal headsEye detection

4 Future plans

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 13 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

Sagittal head detectorTraining data (Thanks Vanessa!)

303 tightly cropped images

Average image

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 14 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

Sagittal head detector

Viola&Jones detector fromOpenCV

Local Binary Pattern (LBP)features

Exhaustive search byrotating/mirroring new image

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 15 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

LBP features

6 7 8

4 5

1 2 3

black if∑Neighbour < ∑Middle

else white

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 16 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

LBP features

First tests of the detection process

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 17 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

Detection result

5min for a 1◦ rotation step,less than 1min if 8◦ rotation step.

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 18 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

Coronal head detectorTraining data

666 tightly cropped images

Average image

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 19 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

Transverse head detectorTraining data

246 tightly cropped images + mirrored

Average image

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 20 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

Neck region darker than the brain

Influence of background

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 21 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

Detection result

In volumes, multiple detections increase confidence

Movie as example of true and false detection

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 22 / 31

Introduction What about fetuses? Own research Future plans

2D detector for fetal heads

Room for improvement

Specific task, specific training data: no universal detector

Need for more testing, better separation of training and testingdata

Confidence score, and combining 3 detectors into a classifier

Combine with other detectors (eyes?) to increase confidence andorientation

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 23 / 31

Introduction What about fetuses? Own research Future plans

Eye detection

Eye detection pipeline

1 Slice by slice:1 Canny edge detection + template matching2 Intensity filtering

2 For the whole volume:1 Grouping candidate circles2 Forming candidate pairs (25 mm ≤ d ≤ 39 mm)3 Chan&Vese Level Sets segmentation + shape

checking4 Grouping candidate pairs

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 24 / 31

Introduction What about fetuses? Own research Future plans

Eye detection

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 25 / 31

Introduction What about fetuses? Own research Future plans

Eye detection

Problem of motion (here 1.3mm spacing)

Several results returned, ranked by number of detection

Is it worth having a more complex pipeline than FEMONUM?

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 26 / 31

Introduction What about fetuses? Own research Future plans

1 Introduction

2 What about fetuses?

3 Own research

4 Future plans

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 27 / 31

Introduction What about fetuses? Own research Future plans

Future plans

Validate and integrate current detectors

Target other organs: dataset of ' 50 fetuses with lungs, heart,kidneys, liver and whole body (Damodaram et al., 2012)

Regression Forests, multi-organs detection

Long-term goals:

Improve visualisation of fetal MRI data

(Semi-)Automatic building of catalogues of organs

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 28 / 31

Introduction What about fetuses? Own research Future plans

Thanks!

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 29 / 31

Introduction What about fetuses? Own research Future plans

Questions?

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 30 / 31

Introduction What about fetuses? Own research Future plans

Manual segmentation with OpenCV Grabcut

20min for 30 slices

K. Keraudren (ICL) Organ localisation in Fetal MRI June 8th , 2012 31 / 31