medical visualization and clinical applications

NA-MICNational Alliance for Medical Image Computing http://na-mic.org

3D Slicer and zSpace: Medical Visualization and Clinical Applications

Steve Pieper, PhD3D Slicer Chief Architect

Isomics is a small technology development company working closely with academic and commercial institutions in medical imaging research and software development.

Jayender Jagadeesan, PhDHarvard Medical School

Monday, October 21, 13

© 2013, All Rights Reserved

Goal and Outline

• Introduction to Our Software and Research (Steve)

• Example Research in Image Guided Therapy (Jay)

• Integration of zSpace (Steve)

Images: A. Golby, F. Talos, P. Black



Domestic Network of Peers• Leadership:

– BWH: Ron Kikinis, (Overall PI)• Algorithms

– Utah: Ross Whitaker (Core 1 PI)– MIT: Eric Grimson– UNC: Guido Gerig– MGH: Bruce Fischl, Dave Kennedy– GaTech: Allen Tannenbaum

• Engineering– Kitware: Will Schroeder (Core 2 PI)– GE: Jim Miller– Isomics: Steve Pieper– UCSD: Mark Ellisman– UCLA: Art Toga

• DBP 2004– BWH: Martha Shenton– Dartmouth: Andy Saykin– UCI: Steve Potkin– UofT: Jim Kennedy

• DBP 2007– UNC: H. Cody– BWH: M. Kubicki– Mind Institute: J. Bockolt– Queens University: G. Fichtinger

• DBP 2011– MGH: G. Sharp– Iowa: H. Johnson– Utah: R. MacLeod– UCLA: J. Van Horn

• Service– Kitware: Stephen Aylward

• Training– BWH: Sonia Pujol

• Core 6 Dissemination– Isomics: Steve Pieper– BWH: Tina Kapur

• Management– BWH: Sanjay Manandhar

Provided by Pieper, Kikinis



International Network of Peers



What is 3D Slicer?

• A platform for image analysis and visualization

• Current Release 4.3• A freely-downloadable program

– Source code and executables available for Windows, Linux, and Mac OS X

– All Code Reusable• Slicer is a research platform:

– NOT an FDA approved medical device – NOT finished – some parts will work

better than others



What: FDA or Research

Slide by Andras Lasso



Multi-Modality Visualization

• Integrated Display of What is Known About the Subject– Anatomical Space as Common

Coordinate System– Segmented Anatomy and

Volume Rendering for Context– Statistics Volumes– Interactive Visualization (View,

Visibility, Cropping, Slicing… )– Therapy Plans, Measurements,

Annotations– Analysis Results– Linked Views

Plesniak, Aucoin et al - BWHJakab and Berenyi - University of Debrecen



Registration

• Intra-subject– Pre-Intra-Post Procedure– Longitudinal Tracking of Disease Progression

• Inter-subject– Support Group Comparison (fMRI)– Map Anatomical Atlas to Individual

• Degrees of Freedom (DOFs) – Rigid (Rotation + Translation) – Similarity (Rigid + Uniform Scale)– Affine (Rigid + Nonuniform Scale and Shear)– Polyaffine (Locally Affine Interpolation)– B-Spline (Cubic Displacement) – Vector Field

Sota - BWH

Preprocedure MR Intraprocedure CT



Navigation & Guidance

22

• OpenIGTLink Standard– Tracked Instruments– Real-Time Images– Scanner and Robot

Control• BWH AMIGO

– 3T MR + PET CT in OR



Alignment of all pre-operative datasets to the intra-operative images achieved during the neurosurgery.

Provided by Archip, Warfield

Intra-procedural Mapping

Archip N, Clatz O, Whalen S, Kacher D, Fedorov A, Kot A, Chrisochoides N, Jolesz F, Golby A, Black PM, Warfield SK. Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery. Neuroimage. 2007 Apr 1;35(2):609-24



Extension Catalog



3D Slicer / AMIGO Parathyroid

Adenoma SurgeryCue Jay...



3D Slicer and zSpace



AMIGO CaseHub Display/Control Options

• Use-Case Needs Assessment• Clinician Control of Imaging• Collaborative Data Management

Workstation for Technical LeadMouse/Keyboard/Monitor

Wall Mounted MonitorsDisplay Only

Point of Care InterfaceTouch/Touchless Interaction

CaseHub Server

Imaging Modalities

MR, PET/CT, Fluroro, US

PACS

SPL Compute Resources

Medical Records, Real-Time Monitors,

Video

Vendor Workstations: Navigation, Planning (Brainlab, Siemens)



AMIGO CaseHub Display/Control Options

• Use-Case Needs Assessment• Clinician Control of Imaging• Collaborative Data Management

Workstation for Technical LeadMouse/Keyboard/Monitor

Wall Mounted MonitorsDisplay Only

Point of Care InterfaceTouch/Touchless Interaction

CaseHub Server

Imaging Modalities

MR, PET/CT, Fluroro, US

PACS

SPL Compute Resources

Medical Records, Real-Time Monitors,

Video

Vendor Workstations: Navigation, Planning (Brainlab, Siemens)

zSpace?



zSpace Slicer Prototype Module

• Developed with Scott Dolim of zSpace– Uses zSpace Python API– Written as Slicer Scripted Module– Integrates with Qt Event Loop– Controls VTK Camera

• Not Yet Publicly Available– zSpace to Create Slicer Extension

(Planned)



zSpace - Slicer Status

• Already:– Any Slicer Data in Displayable– Easy to Use– Ready for TLX-Style Human

Factors Testing• Not Yet:

– No Stylus Support– No GPU Volume Rendering

(Quad-Buffer Issue)– Works with Python 2.6, not 2.7



zSpace in AMIGO Issues

• No Table Space– Would Need Boom Mounted Version

• Surgeon May Not Be Able To Move– May Minimize Value of Head Tracking

• Glasses May Not Be Compatible with Other Eyewear

• Polarized Glasses May Disrupt Surgeon’s Normal Vision

• Only One Person Sees Correct View– Rest of Surgical Team Sees Garbage



zSpace/Slicer Clinical Research• Utility Will Be Highly Procedure-

Dependent• Screen Possible Target Areas

– Interventional– Needle Placement– Robot Control– In-Scanner Procedures



Clinical Research Strategy

• Phase I– Bench Testing– Pre-Procedure Planning/Review

• Phase II– Place in AMIGO Control Room– Intra-Procedural Imaging Review

• Phase III– Navigation– Patient Trials



Summary

• 3D Slicer is Free and Open Research Platform for Medical Image Computing

• AMIGO Environment Ideal for Novel Hardware/Software/Clinical Projects

• zSpace Integration Well Underway and Promising


medical visualization and clinical applications

Technology

rights reservedmonday

steve pieper bwh

medical imaging research

cody bwh

bwh jakab

medical visualization

sota bwh

robot control bwh amigo