interaction techniques for medical visualization (ii) bernhard preim1/71

Interaction Techniques for Medical Visualization (II)

Bernhard Preim 1/71

Interaction Tasks and Techniques

Interaction Tasks• Selection in Volume Data • Insertion of Cutting Planes• Deformation of Volume Models• Exploration of Volume Cuttings• Measurings• Virtual Resection• Path Planning for Minimally-Invasive Surgeries

Bernhard Preim 2/71

Interaction Tasks and Techniques: Selection in Volume Data

• Selection by indication of coordinates• Picking in Volume Visualization

Use of several (orthogonal) viewsRestriction to one intensity areaRestriction to segmented objectsRestriction to a volume of interest

Bernhard Preim 3/71

Object Selection (1)

Selection of objects in scenes consisting of many objectsAll objects opaque → trivial (first hit)

Vessel trees of the liver

Bernhard Preim

Object Selection (2)

Semi-Transparent Structures → Which structure in the “pickray” shall be selected?

Vessel trees plus liver parenchyma

Bernhard Preim

Semi-Transparent Objects

Weakening of the “pickray” according to

a) the transparency of the hit objectsb) size of the objects

Rather small, opaque structures are selected

Bernhard Preim

Algorithm

Source: Mühler et al., IEEE TVCG, 2010

Bernhard Preim

Examples

Opaque objects in the transparent liver can be

selected.

Semi-transparent objects in front of similarly large opaque objects still

cannot be selected.

Bernhard Preim

Interaction Solutions

List with all objects along the pickrays availableScrolling through the individual

objects with the mouse wheel until the chosen object is selected.

Bernhard Preim

Interaction Tasks and Techniques: Insertion of Cutting Planes

• Restriction of the volume

• Movement of a cutting plane through the data set supports a fast diagnosis in case of high-resolution data (e.g. thorax CT)

• Slab Rendering: Coupling of 2 cutting planes (back and front clipping), whereas the distance

remains the same.

Compromise between 2D slice illustration and 3D overview display

• Combination of several clipping planes is possible (hardware support for up to 6 clipping planes)

• Application: DVR and MIP visualizations, especially for vessel diagnostics

Bernhard Preim 10/71


Slab Rendering (© H. Shin, MH Hannover)



Thinslab-Maximum Intensity ProjectionClinically approved for round lesion diagnostics of the lung, Data: RWTH Aachen (Prof. Günther) Volker Dicken, MeVis



Slab volume rendering of approx. 10 cm slices of CT thorax data.


Data: RWTH Aachen (Prof. Günther) Volker Dicken, MeVis


•Selective Clipping: The clipping plane affects only certain objects.

•In medical visualization with unsegmented data, transfer functions and clipping planes are combined to display interesting structures.

15/71Bernhard Preim

Interaction Tasks and Techniques: Virtual resection

Interaction Task: Definition of an arbitrarily formed 3D area that illustrates a resection.

Applications: • Discussion of therapy decisions, • Volumetry of the planned resection, • "Correction" of a visualization, • Computer-assisted trainingAspects:• Specification of the virtual resection (3D interaction)• Modification of a defined virtual resection (3D interaction)• Efficient update of the visualization with high quality of the

illustration of cross sections



Specification of the virtual resection1. Extrusion, e.g. with a prism

• Not for general resections; commercially available

2. Movement of a tool deletes contacted areas (Eraser)3. Drawing in 2D slices4. Drawing of a resection on the organ surface



(2) Specification through deletion with an Eraser

Interaction tasks:Translation and scaling of the Eraser

Variants:• Use of 3D input and bimanual interaction (rotation of the

model and resection)• Postprocessing of the resection via morphological image

processing (erosion, dilatation, closing of holes)• Evaluation of the resection in 2D slice illustration




Virtual resection for surgery training.

• Use of surgical devices that simulate the cutting.

• High-quality illustration of the resection (subvoxel resolution)

(2) Specification through deletion with an Eraser



Virtual resection with virtual scalpel, IMDM, Uni Hamburg (Pflesser et al.)


(3) Specification through drawing in the slices. Interpolation between manually processed layers.

Option: Utilization of the segmentation information3D visualizationg: Evaluation of the resection




(3) Specification through mapping on the organ surface


Some details:

(1) Lines are represented as point set.

(2) Main axis analysis to determine plane and grid size. Eigenvectors that correspond to the two largest eigenvalues span the plane.

(3) Points on the grid are tangentially shifted (in direction to the eigenvector that corresponds to the smallest eigenvalue), such that they are best possibly adopted to the mapped lines.

(4) Points can be shifted by the user (the area of influence is adjustable).



(3) Specification through mapping on the organ surface




Source: PhD Thesis, Stefan Zachow, Zuse Institute Berlin, 2004


Comparison of the methods

(1) Resection with Erasers: hardly manageable, too imprecise.(2) Resection through drawing in slices: very precise, when

mapping takes places in many layers.(3) Resection through deformation of grids: More

challenging interaction than (2), but better manageability than (1). Targeted resection of 3D structures is possible. For experienced users faster than (2).


Interaction Tasks and Techniques: Exploration of volume sections


Local Volume Rendering• More details visible through reduced transparency

when restricted to small ROI• malignancy of tumors better assessable in 3D due to

form criteria• Segmentation can be performed in small volumes

Interaction tasks:Definition of the local volume and adjustment of the local transfer function


Box clipping to define a detailed view for the assessment of brain vessels, application of a local TF, © Peter Hastreiter, Uni Erlangen



Local volume rendering and iso-surface rendering of a bronchial carcinoma to assess the tumor vascularization


Source: Dicken et al., BVM 2003, Data: Prof. Günther (RWTH Aachen)

Interaction Tasks and Techniques: Distance, angle and volume measurements

Motivation:

• Supplementation of the visual assessment of medical data via quantitative values. “To quantify is to know”.

• Evaluation (e.g. in case of vascular constriction and dilatation)

• Quantitative values to assses the severity of a disease (tumor staging) and for follow-up (evaluation of therapy success)

• quality assurance

• decision support w.r.t. the applicability of therapies



Essential Measures:• Mean gray values in a region in CT data:

Measure for the severity of osteoporosis Measure for the severity of a lung function impairment

• Angle: Assessment of malpositions (orthopedics, mouth/face/jaw surgery), criterion for the necessity of a surgery

• Distances between tumors and organ edges: Criterion for the applicability of thermoablations

• Vessel diameter: Criterion, if a clamped blood vessel needs to be reconstructed or not

• Volumes of individual pathologies or the sum of the volumes of all pathologies: Criteria for Therapy Success



Typical: Measurement in axial or reformatted 2D slices. Screenshots: Philips EasyVision Workstation




Planning of live donor liver transplantsQuestions:1. Existence of an accessory hepatic vein2. diameter of the vein3. Distance between the accessory vein and the junction of the veins (if diameter > 5 mm)

Screenshot: Dr. Wald, Lahey Clinic, Boston


Measurements in 2D and 3D• Selection of measuring points is more precise in 2D (each voxel

can be selected)• Distances and angles between 3D objects can only be roughly

approximated via measurements in a 2D slice.

Automatic and interactive measurementsSegmentation results can be used for automatic measurement.Examples:

Minimal distances between objectsExpansion of objectsAngle between the longest main axes of two objects



Simple 3D measurement via surface illustration (Philips EasyVision)



Combination of an inventor manipulator with measures describing the expansion, © Peter Hastreiter, Uni Erlangen


Use of a tracked ruler in an AR surrounding to estimate sizes© B. Reitinger, Uni Graz



Use of a measuring cup in an AR surrounding to estimate volumes© B. Reitinger, Uni Graz


3D widgets for distance measurementImportant aspects:• 3D geometry, • perspective illustration, • shadow projection, • labeling of the "distance lines"



Design considerations for appearance and behavior of the distance line


Interaction Tasks and Techniques: Distance measurements

Further variants of distance lines. (b,c,d) are 2D variants. (b) is without adaptation to the viewing direction → inappropriate.

Bernhard Preim 45

Variants for positioning the measured values (source: Roessling, 2009)


Path measurement as special form of distance measurement.Examples: Length of a stent implant for blood vessels, catheter length


Path measurement (Screenshot: Philips EasyVision)


3D widgets for angle measurement

Illustration of angle and angle legTemporary fade-in of transparent surfaces




3D widgets for angle measurement. Visual design, interaction, feedback and positioning of the measured value are important.

© B. Reitinger, Uni Graz


Combination of 2D and 3D visualization for angle measurement. Left: MPR generated from the 3 points of the angle




Automated measurement:• Expansion of objects• Minimal distances between

objects

Interaction Tasks and Techniques: Measuring

Versions of the placing of results of the automatic measurement (Roessling, 2009)


Interaction Tasks and Techniques: Measuring

Interactive measuring in 2D. Exact automatic measuring in 3D→ the difference leads to a different tumor stage.Right: Shortest distances between tumor and vesselSource: Roessling, 2010



Automated measurement:• Angle bewteen the longest main axes of two objects


Interaction Tasks and Techniques: Path Planning for Minimally-Invasive Surgeries

Medical Background:• Treatment of non-operable tumor patients (e.g. in case of a

bad general condition, adjacency to large blood vessels, in case of liver tumors: in case of a distinct liver cirrhosis)

• Tumors < 5 cm • Few tumors/metastases (<=5)



Goal:Placing an applicator to destroy a tumor, provided that vital structures are preserved.

• Difficulty: Poor visual control• Selection of entry and target point

(puncture in the body and target point)• Rough planning via 3D view,

setting of details in 2D• User assistance: Analysis of the emerging path

e.g. histogram displayIllustration of hit objects



Applications:• Destruction of brain tumors and liver

metastases

Applicator models:• Radio frequency therapy• Laser-induced interstitial

thermotherapy



Difficult situation: Tumor centrally in a vessel tree



Required views: 2D, 3D view, 3D detail



Intelligent planning:

Suggestion for target point: Focus of a tumorSuggestion for the direction: longest main axis of a tumorSearch for paths and evaluation in a local surrounding, where appropriateSuggestions for the duration and performance of the application dependent on the tumor size and simulation, where appropriate



Insertion:Besides a suitable visualization, a simulation of physical effects is important. Goal: determine if and how the tumor may be maximally destroyed (damage volume).Parameters are:

Duration and performace of the application andtissue-specific parameters, especially the cooling

effect of vessels




Neglection of the cooling effect of the

vessels

Considering the vessels

Visualization of Registered Volume Data Sets

Why?Overlapping of data

of several modalitiesof several sequences of a modaliy (e.g. MR sequences)of different points in time (follow-up)Preoperative data and intraoperative video

How?Matching: Rigid and non-rigid transformations, use of statistic information, registration based on landmarks


Visualization of Registered Volume Data Sets

Fusion of a CT and an MRI data set (left), Fusion of an MR angiography for vessel illustrationwith MRI (right), © Peter Hastreiter, Uni Erlangen


Literature

O. Konrad-Verse, B. Preim, A. Littmann: Virtual Resection with a Deformable Cutting Plane, Proc. of Simulation und Visualisierung, pp. 203-214, 2004http://www.vismd.de/lib/exe/fetch.php?media=files:hci:konrad-verse_2004_simvis.pdf

K. Mühler, C. Tietjen, F. Ritter, and B. Preim. The Medical Exploration Toolkit: An Efficient Support for Visual Computing in Surgical Planning and Training. IEEE Transactions on Visualization and Computer Graphics, 16(1)(1):133–146, 2010

B. Preim, C. Tietjen, W. Spindler, and H.-O. Peitgen. Integration of Measurement Tools in Medical Visualizations, In Proc. of IEEE Visualization, pages 21–28, 2002

B. Reitinger, D. Schmalstieg, A. Bornik, and R. Beichel. Spatial Analysis Tools for Medical Virtual Reality. In Proc. of IEEE Symposium on 3D User Interface, 2006 (3DUI 2006).

I. Rössling, C. Cyrus, L. Dornheim, P. Hahn, B. Preim, and A. Boehm. Interaktive Visualisierung von Abständen und Ausdehnungen anatomischer Strukturen für die Interventionsplanung. In Proc. of Bildverarbeitung für die Medizin (BVM), pages 381–385, Springer, 2009.http://www.vismd.de/lib/exe/fetch.php?media=files:measurements:roessling_2009_bvm.pdf

I. Rössling, C. Cyrus, L. Dornheim, A. Boehm, and Bernhard Preim. Fast and flexible distance measures for treatment planning. International Journal of Computer Assisted Radiology and Surgery, pages 633–646, 2010.http://www.vismd.de/lib/exe/fetch.php?media=files:measurements:roessling_2010_jcars.pdf

Bernhard Preim 66

interaction techniques for medical visualization (ii) bernhard preim1/71

Documents

interaction techniques

selected bernhard preim

bernhard preim171 slide

bernhard preim371 slide

liver bernhard preim

selection of objects

mevis slide

mevis bernhard preim1271