na-mic national alliance for medical image computing large-scale computing frameworks for...
TRANSCRIPT
NA-MICNational Alliance for Medical Image Computing http://na-mic.org
Large-Scale Computing Frameworks for Developing Image Analysis Tools
Steve Pieper, PhD
Isomics, Inc.
Founder and CEO of Isomics, Inc. a technology development company that works closely with multiple research institutions on projects including NIH grants and software development. Much of this work is done in collaboration with the Surgical Planning Laboratory at BWH.
National Alliance for Medical Image Computing http://na-mic.org
2
Acknowledgments• F. Jolesz, R. Kikinis, C. Tempany, P. Black, S. Wells, CF. Westin, M. Halle, N.
Hata, T. Kapur, A.Tannenbaum, M. Shenton, E. Grimson, P.Golland, W.Schroeder, J. Miller, N. Aucoin, K. Hayes, A. Yarmarkovich, C. Lisle, D. Marcus, J. Miller, R. Gollub, S. Pujol, S. Barre, W. Plesniak, B. Fischl, D. Greve and many more….
V E R I TAS
National Alliance for Medical Image Computing http://na-mic.org
3
Overview
• Software Infrastructure– NA-MIC Kit Components– 3D Slicer
• Modules and Extensibility• Population Analysis• Organization and Community
National Alliance for Medical Image Computing http://na-mic.org
Large Scale Computing
• Working at a High Level– Major Functional Components are Readily Available– Tested, Cross Platform, Documented…
• Suite of Tools and Systems for Organizing and Processing– Databases and Batch Computing
• Mechanisms to Translate Research to Clinical Users– Plug New Research into a Usable Platform
• A Talented and Productive Community– Multidisciplinary– National and International
4
National Alliance for Medical Image Computing http://na-mic.org
5
NA-MIC Kit
• The NA-MIC Kit is a Collection of Software and Methodologies for Medical Image Computing
• Most components of the NA-MIC kit predate NA-MIC (National Alliance for Medical Image Computing)
National Alliance for Medical Image Computing http://na-mic.org
6
NA-MIC Kit
• Packaging of:– Applications– Algorithms (toolkits)– Methodologies
National Alliance for Medical Image Computing http://na-mic.org
7
NA-MIC Kit Components
• End User Application– 3D Slicer
• Image Analysis, Visualization, and GUI libraries– ITK, VTK, KWWidgets, Teem
• Archive and Distributed Processing Tools – XNAT, Batchmake, BIRN GRID tools
• Software Engineering Tools– CMake, CDash, CTest, CPack
http://www.na-mic.org/Wiki/index.php/SoftwareInventory
Provided by Pieper, Kikinis
National Alliance for Medical Image Computing http://na-mic.org
8
Visualization Toolkit - vtk
Open source toolkit for scientific visualization, computer graphics, and image processing
National Alliance for Medical Image Computing http://na-mic.org
10
KWWidgets• Usage
– Cross-platform GUI with open source license– Object Oriented C++ API (very VTK-like)
• Active Development– New widgets– Work flow support– File/directory browser
• Future– Tracing (for testing)– Registration inspection widget– Camera control widget– Interface to ITK (spatial objects)– Cleanup
• kwwidgets.org– cvs, dashboard, bug tracker, wiki…– BSD license
National Alliance for Medical Image Computing http://na-mic.org
11
NA-MIC Software Process
Source
CDashTesting DashboardVersion Control
Svn, CVS with web access
Developers review results
Developers check-in code
CMake, CTest
Multiplatform Software Compilation, Testing
National Alliance for Medical Image Computing http://na-mic.org
12
Development Methodology
6 months
4 months
2 months
Release Patch Nightly Continuous
Release X.Y
Release X.Y.1
Release X.Y.2
Release X.Y.3
Extreme lifecycle PrivateSandbox
NA-MICSandbox
Slicer
ITK
Dashboard
CMakeCTestCPack
Testing
National Alliance for Medical Image Computing http://na-mic.org
13
NA-MIC Software Process
• Emphasizes Cross Platform Testing
• Provides informal– Requirements specification– Design documents
• Which are captured via collaboration tools– Tcon, email, wiki, software documentation
• Traceability– Implicit in logs and collaboration
National Alliance for Medical Image Computing http://na-mic.org
14
What is 3D Slicer?• A platform for image analysis and
visualization• Current Releases 2.7 and 3.2
– 2.x most features and documentation
– 3.x focus of current activity• A freely-downloadable program
– Source code and executables available for Windows, Linux, and Mac OS X
• Slicer is a research platform:– NOT an FDA approved medical
device – NOT finished – some parts will
work better than others
National Alliance for Medical Image Computing http://na-mic.org
Slicer 3.2 May 2008
Kitware, Inc. GE Research Isomics, Inc.BWHUCLAUCSDU IowaMITGaTechUNCU UtahMGHUCIHarvardWashU…
NACNCIGTNA-MICmBIRNfBIRN…
http://slicer.org
National Alliance for Medical Image Computing http://na-mic.org
16
3D Slicer Numbers• Numbers May 2007:
Subversion Commits: 3,407
Lines of Code*: 371,428
Bugs & Features:
154 Submitted
63 Closed
Active Developers with svn access†: 33
Numbers July 2008:
Subversion Commits: 7303
Lines of Code*: 448,175
Developers: 42
Image provided by A. Golby, F. Talos, P. Black
*: find . -iname \*.h -o -iname \*.cxx -o -iname \*.tcl -o -iname \*.java -o -name \*.py | grep -v svn | xargs wc†: svn log | grep "^r" | cut -d " " -f 3 | sort | uniq | wc
National Alliance for Medical Image Computing http://na-mic.org
17
Slicer3 “Observer MVC” Pattern
• MRML (Model)– For Scene Description and Application State– MRML Nodes are Persistent and Undoable– Scene and Nodes are Observable– XML Serialization, Undo/Redo
• Logic Encapsulate VTK and ITK Pipelines (Controller)– Observe MRML to Configure Pipelines– Help Create/Manage Nodes– No UI Components (no Widgets, Actors, Mappers,
Renderers or RenderWindows)• GUI (View)
– Observe and Edit MRML– Interact with User and Display Hardware
Logic
MRML Nodes
GUI
Widgets Renderers
EditObserve
Observe
Observe
Edit
Edit
“Observe” means generic event mechanisms are used to pass information.“Edit” means code can directly call methods.
Example: GUI can call methods in Logic classes,but Logic cannot call GUI methods.MRML cannot call Logic or GUI methods.
There can be many observers for any event.
National Alliance for Medical Image Computing http://na-mic.org
18
Scene Description
Provided by D. Gering
National Alliance for Medical Image Computing http://na-mic.org
19
Image/Scene Management
• XML-Based MRML File Stores Scene Description
– Volumes (Images, Label Maps)
– Models– Hierarchical Affine
Transforms– Scene Data (Cameras,
Colors, Fiducials, etc).• Manipulated in World
Coordinates based on Patient RAS
Provided by S. Pieper
National Alliance for Medical Image Computing http://na-mic.org
20
Alignment of all pre-operative datasets to the intra-operative images achieved during the neurosurgery.
Provided by Archip, Warfield
Neurosurgery Example
Archip N, Clatz O, Whalen S, Kacher D, Fedorov A, Kot A, Chrisochoides N, Jolesz F, Golby A, Black PM, Warfield SK. Non-rigid alignment of pre-operative MRI, fMRI, and DT-MRI with intra-operative MRI for enhanced visualization and navigation in image-guided neurosurgery. Neuroimage. 2007 Apr 1;35(2):609-24
National Alliance for Medical Image Computing http://na-mic.org
Slicer3 Command Line Modules
Common architecture for interactive and batch
processing
Slices Courtesy Jim Miller
National Alliance for Medical Image Computing http://na-mic.org
Each module has …… an entry in the module menu
… a panel of user interface controls
National Alliance for Medical Image Computing http://na-mic.org
Slicer Modules
• Interactive Modules– Interact Closely with GUI– Volume Rendering, Models, Editor…
• Command Line Modules– “Batch Mode” (Can be run from command line or script)– Registration, Filtering, DTI Tractography…
• Both Interactive and Command Line Modules are Dynamically Loaded– They can be Built and Distributed Independent of Slicer
• Today we will look in detail only at Command Line Modules
National Alliance for Medical Image Computing http://na-mic.org
25
EM Interactive Module
MRML Logic GUI
•Global Parameters
•Hierarchical parameters
•Image data
•Segmentation output
•Manage MRML nodes
•API for access to parameters
•Manage hierarchy
•Segmentation algorithm
•Window to parameter set
•Wizard
•Interaction with images
Provided by B. Davis
National Alliance for Medical Image Computing http://na-mic.org
26
Command Line Module
MRML Logic CLI
•Global Parameters
•Hierarchical parameters
•Image data
•Segmentation output
•Manage MRML nodes
•API for access to parameters
•Manage hierarchy
•Segmentation algorithm
•XML Parameter File
•File I/O
•Batchable
Provided by B. Davis
National Alliance for Medical Image Computing http://na-mic.org
Existing Command Line Modules• Demonstration
– Execution Model Tour– Your new hello world module
• Converters – Create a DICOM Series– Dicom DWI loader– Dicom to Nrrd– GE Dicom to NRRD Converter– Orient Images
• Filtering – Calculate Volume Statistics– CheckerBoard Filter– Extract Skeleton– Histogram Matching– Otsu Threshold– Resample Volume/ Resample Volume 2– Voting Binary Hole Filling– Zero Crossing Based Edge Detection Filter– Filtering.Arithmetic:
– Add Images– Subtract Images
National Alliance for Medical Image Computing http://na-mic.org
Existing Command Line Modules• Filtering.Denoising
– Curvature Anisotropic Diffusion– Gaussian Blur– Gradient Anisotropic Diffusion
• Segmentation – EMSegment Simple– FreesurferSurfaceSectionExtraction– Otsu Threshold Segmentation– Simple region growing
• Filtering.Morphology – Grayscale Fill Hole– Grayscale Grind Peak
• Model Generation – Grayscale Model Maker– Label Map Smoothing– Model Maker– Multiple models example– Probe Volume With Model (Paint)
• Meshing.VoxelMesh – Voxel Meshing
– Median Filter
National Alliance for Medical Image Computing http://na-mic.org
Registration Command Line Modules
• Registration – Affine registration
– Deformable BSpline registration
– (Utah) Deformable BSpline registration
– Linear registration
– RealignVolume
National Alliance for Medical Image Computing http://na-mic.org
Diffusion Command Line Modules
• Diffusion Tensor – Resample DTI Volume
• Diffusion Tensor Estimation• Diffusion Tensor Scalar Measurements• Simple DWI IO Test• Rician LMMSE Image Filter• Tractography.Editor: ROISelect• Tractography.Seeding• Stochastic Tractography
– Generate Connectivity Map
– Stochastic ROI Tract Filter
– Stochastic Stochastic Tractography Filter
National Alliance for Medical Image Computing http://na-mic.org
Example Moduleshttp://www.nitrc.org/projects/slicer3examples/
http://www.na-mic.org/Wiki/index.php/Slicer3.2:Training
National Alliance for Medical Image Computing http://na-mic.org
Parameters
<integer> | <float> | <double> | <boolean> | <string> | <integer-vector> | <float-vector> | <double-vector> | <string-vector> | <integer-enumeration> | <float-enumeration> | <double-enumeration> | <string-enumeration> | <file> | <directory> | <image>[type="scalar|label|tensor|diffusion-weighted|vector|model"] | <geometry> [type="fiberbundle|model"] | <point>[multiple="true|false"] [coordinateSystem="lps|ras|ijk"] | <region>[multiple="true|false"] [coordinateSystem="lps|ras|ijk"]
• GUI Automatically Generated from XML
• C++ Argument Parsing Code Automatically Generated from XML
National Alliance for Medical Image Computing http://na-mic.org
Parameter Example
33
<?xml version="1.0" encoding="utf-8"?><executable> <description> HelloWorld Example</description>…<parameters> <label>Input/Output</label> <description>Input/output parameters</description> <image> <name>helloSlicerInputVolume</name> <label>Input Volume</label> <channel>input</channel> <index>0</index> <default>None</default> <description>Input volume</description> </image>…</parameters><parameters>
<label>Discrete Gaussian Parameters</label> <description>Parameters of the Discrete Gaussian Filter </description><double> <name>variance</name> <longflag>--variance</longflag> <description>Variance ( width of the filter kernel) </description> <label>Variance</label> <default>0.5</default></double></parameters></executable>
National Alliance for Medical Image Computing http://na-mic.org
C++ Module
34
#include <iostream> #include "HelloSlicerCLP.h" #include "itkImage.h" #include "itkImageFileReader.h" #include "itkImageFileWriter.h" #include "itkDiscreteGaussianImageFilter.h"
int main(int argc, char * argv []) { PARSE_ARGS; typedef itk::Image< short, 3 > ImageType; typedef itk::ImageFileReader< ImageType > ReaderType; typedef itk::ImageFileWriter< ImageType > WriterType; ReaderType::Pointer reader = ReaderType::New(); WriterType::Pointer writer = WriterType::New(); reader->SetFileName( helloSlicerInputVolume.c_str() ); writer->SetFileName (helloSlicerOutputVolume.c_str()); typedef itk::DiscreteGaussianImageFilter <ImageType, ImageType> FilterType; FilterType::Pointer filter = FilterType::New(); try { filter->SetInput(reader->GetOutput()); filter->SetVariance(variance); writer->SetInput(filter->GetOutput()); writer->Update(); } catch (itk::ExceptionObject &excep) { std::cerr << argv[0] << ": exception caught !" << std::endl; return EXIT_FAILURE; } return EXIT_SUCCESS; }
National Alliance for Medical Image Computing http://na-mic.org
Python ModuleXML = """<?xml version="1.0" encoding="utf-8"?> <executable> <category>Filtering.Denoising</category> ...
def toXML(): return XML;
def Execute ( inputVolume, outputVolume, conductance=1.0, timeStep=0.0625, iterations=1 ): print "Executing Python Demo Application!" Slicer = __import__ ( "Slicer" ); slicer = Slicer.Slicer() in = slicer.MRMLScene.GetNodeByID ( inputVolume ); out = slicer.MRMLScene.GetNodeByID ( outputVolume );
filter = slicer.vtkITKGradientAnisotropicDiffusionImageFilter.New() filter.SetConductanceParameter ( conductance ) filter.SetTimeStep ( timeStep ) filter.SetNumberOfIterations ( iterations ) filter.SetInput ( in.GetImageData() ) filter.Update() out.SetAndObserveImageData(filter.GetOutput()) return
National Alliance for Medical Image Computing http://na-mic.org
Behind the scenes
• Tasks queued for processing thread• Three types of modules:
– executable,
– shared object, and
– python modules
• Scalar images sent via files for executables and memory* for shared objects and python
• Vector images, tensor images, geometry, tables, transforms sent via files
• Scalars, file names, directories, fiducials, regions sent via command line
* VTK-based modules using scalar images are only supported as executable (command line) modules.
National Alliance for Medical Image Computing http://na-mic.org
BIRN Integration
BIRNBIRN
acquisitionprotocols
distortioncorrection
tools
local databases
workflows
analysis tools
populationstatistics
.
.
.
.
.
.
PubMedPubMed
IBVDIBVD
BrainInfoBrainInfo
Visualization&
Interpretation
Visualization&
Interpretation
(.xcat)(.xcat)
(.xar)(.xar)
National Alliance for Medical Image Computing http://na-mic.org
mBIRN Informatics
• BIRN Data Repository (BDR)• eXtensible Neuroimaging
Archive Tool (XNAT) http://www.xnat.org
• XML-Based Clinical Experiment Data Exchange Schema (XCEDE) http://www.xcede.org
• Stores Images, Demographics, Clinical Data, Analysis Results
• 3D Slicer Interoperability (currently read-only, read/write prototype exists)
National Alliance for Medical Image Computing http://na-mic.org
Query Atlas Module
• BIRN Scientific Interpretation Tool
– Compatible with fMRI and Morphometry Datasets in BIRN-
Standard Formats
• Provides a Link Between Images and Text Databases
– Uses Anatomic Labeling + Ontologies
– Links to Definitions, Publications, Quantifications
– E.g. Wikipedia, PubMed, IBVD, BrainInfo
National Alliance for Medical Image Computing http://na-mic.org
Query Atlas Features
• Input:.xcat or .qdec from .xar
• Hardware Accelerated
Interactive 3D Annotation
• Ontology Engine
– FreeSurfer, UMLS, BIRNLex,
NeuroNames, IBVD
– Interactive Browser
• Direct Browser Launch to
Search Sites using Selected
Terms
National Alliance for Medical Image Computing http://na-mic.org
BIRN Population Statistics
• Query, Design, Estimate, Contrast of Population Statistics– XNAT to Select
Subjects– FreeSurfer QDEC
Runs on Server– .QDEC file in .XAR
Web Download– Slicer3 Interactive
Visualization – Integrated with Query
Atlas
National Alliance for Medical Image Computing http://na-mic.org
44
Overall Goals
• Why Medical Research?– To Help Patients!
National Alliance for Medical Image Computing http://na-mic.org
45
NA-MIC Roots: NIH
• Roadmap Initiative• http://grants1.nih.gov/grants/guide/rfa-files/RFA-
RM-04-003.html– …will create a networked national effort to
build the computational infrastructure for biomedical computing in the nation…
– The establishment of the NIH NCBC was called for in the Biomedical Information Science and Technology Initiative report in 1999
National Alliance for Medical Image Computing http://na-mic.org
46
NA-MIC Governance
Software Sharing:
http://grants1.nih.gov/grants/guide/rfa-files/RFA-RM-04-003.html • …software should be freely available … • …permit the commercialization of
enhanced or customized versions … • …include the ability of researchers outside
the center and its collaborating projects to modify the source code and to share modifications …
National Alliance for Medical Image Computing http://na-mic.org
47
NA-MIC Kit Governance
• Free, BSD style Open-Source
• NA-MIC Provides “Reference Standard Distribution”
– Maintain Official Releases
– Avoid Code Forking, Avoid Non-Free Code Creeping into Core
• No “Knowing” Addition of Patented Techniques into NA-MIC Kit
• Technology Base for Suited for both Research and Commercialization
– Okay to build GPL or Closed SW on top of NA-MIC Kit
– But no GPL or Closed SW in NA-MIC Kit
• Emerging community process for contributions http://insight-journal.org• Support Required for Contributions (no “shoot and forget”)
• Copyright to Substantive Works Remains with Creators
– Available under NA-MIC Approved Licenses for Distribution with Kit
National Alliance for Medical Image Computing http://na-mic.org
48
FOSS – A Public Highway…
• Open-source is like a Public Road System– Provides Infrastructure
for a Variety of Uses– Driveways can Lead to
Anything:• a Public Park• a Private Facility
FOSS= Free Open Source Software
Provided by Pieper, Kikinis
National Alliance for Medical Image Computing http://na-mic.org
49
NA-MIC Policy
Industry
GPL
NA-MIC Kit
ResearchOnly
National Alliance for Medical Image Computing http://na-mic.org
50
NA-MIC: A Network of Peers• Leadership:
– BWH: Ron Kikinis, (Overall PI)• Core 1 Algorithms
– Utah: Ross Whitaker (Core 1 PI)– MIT: Eric Grimson– UNC: Guido Gerig– MGH: Bruce Fischl, Dave Kennedy– GaTech: Allen Tannenbaum
• Core 2 Engineering– Kitware: Will Schroeder (Core 2 PI)– GE: Jim Miller– Isomics: Steve Pieper– UCSD: Mark Ellisman– UCLA: Art Toga – WashU: Dan Marcus
• Core 3 DBP 2004-2007– BWH: Martha Shenton– Dartmouth: Andy Saykin– UCI: Steve Potkin– UofT: Jim Kennedy
• DBP 2007– UNC: H. Cody– BWH: M. Kubicki– Mind Institute: J. Bockolt– Queens University: G. Fichtinger
• Core 4 Service– Kitware: Will Schroeder
• Core 5 Training– MGH: Randy Gollub
• Core 6 Dissemination– Isomics: Steve Pieper, Tina Kapur
• Core 7 Management– BWH: S. Manandhar
Provided by Pieper, Kikinis
National Alliance for Medical Image Computing http://na-mic.org
51
Communication
Daily – e-mail, dashboards, wikis Weekly – telephone conferences Periodic – architecture reviews, workshops Semi-annual– Programmer/Project week Yearly – All Hands Meeting
National Alliance for Medical Image Computing http://na-mic.org
NA-MIC Project Week
• June 23-27, 2008– 120 Participants– Dozens of Projects– Hands On Software
Development
National Alliance for Medical Image Computing http://na-mic.org
55
Dissemination and Training
• National and International Events– MIT, MGH, UNC,
EPFL, NIH, UNM, UCSD…
• All Materials on Wiki• Project Weeks
– Full Week Each Summer
– ½ Week at Winter AHM
• Workshops– MICCAI 2005, 2006,
2007, 2008..– OHBM, RSNA,
Munich, NCI…
National Alliance for Medical Image Computing http://na-mic.org
Training Courses
• June In Munich– 2 Packed Days, 40 Participants– Cover Spectrum of Use and
Development• Data Visualization• Registration• Tracked Surgical Instruments• Developing Custom Modules
• Upcoming– NIH/NCI, Robarts Inst., Stanford,
MICCAI, RSNA, WashU• > 1,000 Participants Since 2004
National Alliance for Medical Image Computing http://na-mic.org
NAMIC Training Portfolio http://www.na-mic.org/Wiki/index.php/Slicer:Workshops:User_Training_101
• Google for “Slicer 101”
• Most training material is for Slicer2.x with increasing amounts of Slicer3 content
National Alliance for Medical Image Computing http://na-mic.org
58
It’s all great, but…
• A few caveats to keep in mind
• It’s a big, distributed group– Good spin: “Wisdom of the Crowds”– Bad spin: “Designed by Committee”
• It’s all research– Constantly evolving– Never enough documentation– Plurality of approaches
• The net result is overwhelmingly positive.
National Alliance for Medical Image Computing http://na-mic.org
59
Getting Involved
• Winter Project Week 2009– January, Salt Lake City, UT
• Summer Project Week 2009– June at MIT Stata Center, Cambridge MA
• Collaborations PAR-05-063– Automated FE Mesh Development
N. Grosland PI, U Iowa + Isomics, Inc.– Measuring Alcohol and Stress Interaction with Structural and Perfusion MRI
• J. Daunais PI, Wake Forest University and Virginia Polytechnic Institute + K. Pohl BWH
– An Integrated System for Image-Guided Radiofrequency Ablation of Liver Tumors
• K. Cleary PI, Georgetown + N. Hata BWH– http://www.na-mic.org/Wiki/index.php/Collaborator:Resources
National Alliance for Medical Image Computing http://na-mic.org
Summary
• End-to-End Platform for Translational Image Research• Open Source and Extensible• Standard Methodologies Facilitate Sharing and Communication
60