sketch-based image-independent editing of 3d tumor segmentations using variational interpolation

© Fraunhofer MEVIS

Frank Heckel1, Stefan Braunewell1, Grzegorz Soza2, Christian Tietjen2, Horst K. Hahn1

Sketch-based Image-independent Editing of 3D Tumor Segmentations using Variational Interpolation

1 Fraunhofer MEVIS, Germany, 2 Siemens AG, Healthcare Sector, Imaging & Therapy Division, Computed Tomography, Germany

© Fraunhofer MEVIS2 / 15Frank Heckel et al. Sketch-based Image-independent Editing of 3D Tumor

Segmentations28. Sept. 2012

Motivation

Segmentation is one of the essential tasks in medical image analysis Many sophisticated automatic segmentation algorithms exist … … which might fail in some cases (low contrast, noise, biological

variability)

What to do? Manual segmentation? Takes too long Different algorithm? Might fail as well Locally correct the error!

Why do we need efficient segmentation editing tools?Solution Results Outlook Conclusion



Motivation

Requirements: Intuitive interaction in 2D – estimate the user’s intention in 3D Local modifications Real-time feedback Provide a general tool (for different objects and modalities) Be independent of the preceding automatic algorithm

The user expects the tool to allow him or her to correct all errors With only a few steps!

The segmentation problems are typically hard (noise, low contrast, …) Do not use the image!

What makes segmentation editing a difficult problem?Solution Results Outlook Conclusion



Solution

Use methods known from object reconstruction Contour-based representation Can be treated as a point cloud Reconstruct a smooth surface using variational interpolation

Segmentation Formulated as an Object Reconstruction Problem

0)()()(1

k

jjj cxwxPxf

Results Outlook Conclusion Motivation



Solution

Hole-handling: Recursively check the level of embedding Holes have an odd level Invert the sign of the normals

Segmentation Formulated as an Object Reconstruction Problem

without hole-handling

with hole-handling




SolutionSketch-based Editing in 2D

User input Correction result Edited region

Part containing the center of

gravity




Solution

We have to deal with imperfection:

Sketch-based Editing in 2Dadd

remov

e

add +

remov

e

replc

ae




Solution

A correction might generate new “holes”: Remove all contours whose level of embedding has changed

Sketch-based Editing in 2DResults Outlook Conclusion Motivation



Solution

Compute a correction depth

Reconstruct the new surfacebetween start and end

3D Extrapolation using Variational Interpolation

𝑠start=max❑

(𝑠−𝑑 (𝐶𝑠𝑢 ) , 𝑠min )

𝑑 (𝐶 𝑠𝑢 )=⌈max

𝑖 {min𝑗 {|𝐶𝑠𝑒 [𝑖 ]−𝐶𝑠𝑢[ 𝑗]|}}𝑑𝑠

⌉

𝑪𝒔𝒖

𝑪𝒔𝒆

𝒔𝐬𝐭𝐚𝐫𝐭

𝒔𝒆𝒏𝒅𝒔


𝒔



Solution

Making the correction local: Dilate the edited region

Duplicate it to all slices of the reconstruction Use new segmentation in this region only

3D Extrapolation using Variational Interpolation

𝑘=2 ⌈ 143√ 3𝑛4𝜋 ⌉+1

Sphere volume:→

𝒌




SolutionManual Correction Workflow

Corrections can be performed in any view User can arbitrarily switch between views

Previously performed corrections should be part of the new surface Keep all user-inputs and use them for reconstruction


user-input(1st step, axial

view) user-input(2nd step, sagittal view)



SolutionResults Outlook Conclusion Motivation



Results

Data: 89 tumors in CT (lung nodules, liver metastases, lymph nodes) Participants: 2 technical experts with 6+ years experience in tumor

segmentation and assessment Qualitative rating of the correction tool

Median: 4 steps (53s), Avg. time per step: 0.4s1

Rating

Meaning # Cases

Percentage

++ Perfect 14 15.7%+ Good 43 47.3%0 Acceptable 25 28.1%- Bad 6 6.7%-- Unacceptable 1 1.1%

1 Intel Xeon X5550 (2.66GHz), 12GB RAM, Windows 7 64-Bit, 4 cores used

Outlook Conclusion Solution

92.1%



Outlook

Main problem currently: Contradictory user inputs

Reconstruction is currently too slow for large objects (like the liver) Comparison the other methods

Strongly depends on the specific segmentation task and the experience and requirements of the users

Proposal: Segmentation editing challenge

Conclusion Results



Conclusion

Segmentation editing is an indispensable step in the segmentation process

Efficient editing in 3D is challenging

Sketching provides an intuitive interface for segmentation editing in 2D

We have proposed a general, efficient editing tool 2D corrections are extrapolated to 3D using object reconstruction Can be used for any 3D modality and any compact object

Outlook



[email protected] you!

sketch-based image-independent editing of 3d tumor segmentations using variational interpolation

Documents

d tumor segmentations28

solutionsketchbased

frank heckel et

modalitiesbe independent

segmentation problems

manual segmentation

efficient segmentation

appropriate segmentation