na-mic national alliance for medical image computing slicer architecture steve pieper, phd
DESCRIPTION
National Alliance for Medical Image Computing Execution Environment Slicer runs inside a VTK Tcl/Tk shell –Uses the vtk executable that gets built when Tcl wrapping is enabled for the VTK build or the wish84.exe shell on Windows –vtkSlicerBase is a tcl package that gets pulled in at run time –Each Module is a tcl package –Note that vtkSlicerBase and the Modules can be wrapped for other languages if desired (python, java)TRANSCRIPT
NA-MICNational Alliance for Medical Image Computing http://na-mic.org
Slicer Architecture
Steve Pieper, PhD
National Alliance for Medical Image Computing http://na-mic.org
Overall Goals• Execution Environment
– Command Line Args– Flow of Control, Startup Sequence
• Data Model– Mrml Tree– Mrml Nodes– Mrml Data
• Module Architecture– Standard Developer Widgets– GUI Modules– Editor Modules– I/O Modules
National Alliance for Medical Image Computing http://na-mic.org
Execution Environment
• Slicer runs inside a VTK Tcl/Tk shell– Uses the vtk executable that gets built when
Tcl wrapping is enabled for the VTK build or the wish84.exe shell on Windows
– vtkSlicerBase is a tcl package that gets pulled in at run time
– Each Module is a tcl package– Note that vtkSlicerBase and the Modules can
be wrapped for other languages if desired (python, java)
National Alliance for Medical Image Computing http://na-mic.org
Tcl Packages
• Includes Tcl code to be interpreted• Includes compiled C++ code as a
shared library (optional) – Can reference and subclass C++
classes from other packages• Package path that can be used to
find module-specific logos, data, etc
National Alliance for Medical Image Computing http://na-mic.org
Command Line Arguments usage: slicer2-<arch> \[options\] \[MRML file name .xml | dir with MRML file\] <arch> is one of win32.exe, solaris-sparc, or linux-x86 \[options\] is one of the following: --help : prints this message and exits --verbose : turns on extra debugging output --no-threads : disables multi threading --no-tkcon : disables tk console --load-dicom <dir> : read dicom files from <dir> --load-analyze <file.hdr> : read analyze file from <file.hdr> --load-freesurfer-volume <COR-.info> : read freesurfer files --load-freesurfer-label-volume <COR-.info> : read freesurfer label files --load-freesurfer-model <file> : read freesurfer model file --load-bxh <file.bxh> : read bxh file from <file.bxh> --script <file.tcl> : script to execute after slicer loads --exec <tcl code> : some code to execute after slicer loads --all-info : print out all of the version info and continue --enable-stereo : set the flag to allow use of frame sequential stereo --old-voxel-shift : start slicer with voxel coords in corner not center of image pixel
National Alliance for Medical Image Computing http://na-mic.org
Command Line Examples• slicer2-win32.exe mydata.xml
– reads Mrml scene file after slicer starts• slicer2-win32.exe –no-threads
– good for debugging ThreadedExecute methods• slicer2-win32.exe –load-dicom <dir>
– loads all the dicom volumes from a study directory• slicer2-win32.exe –exec set a 100
– executes tcl code on command line after slicer loads• slicer2-win32.exe –script testscript.tcl
– runs the tests script after slicer loads• slicer2-win32.exe testscript.tcl
– runs the script instead of booting slicer GUI
National Alliance for Medical Image Computing http://na-mic.org
Bootstrapping
• slicer2-win32.exe– essentially the same as calling
• slicer2-win32.exe launch.tcl– The launch.tcl script creates a subshell
with correct environment variables– Then execs vtk or wish84.exe to start
main slicer process with Go.tcl as a startup script
National Alliance for Medical Image Computing http://na-mic.org
Flow of Control Outline
• Base/tcl/Go.tcl is main entry point– Parse command line args– Starts tkcon console– Load vtkSlicerBase and all module
packages• Uses tcl ‘package require’ command
– Run MainBoot
National Alliance for Medical Image Computing http://na-mic.org
MainBoot Startup Sequence• Init Slicer Base and Modules
– MainInit – sets up base framework, appearance, etc– Create Main Viewer Windows– Call All <Module>Init Procs – set callback procs and defaults
• Set up VTK– MainBuildVTK – makes the cube and letters– Call All Module procVTK – module-specific VTK instances
• Build GUI– Build Main GUI– Call All Module procGUI – each module is given it’s tab frame
• Read Options.xml – per-installation defaults• Read Command Line MRML scene if specified• MainSetup – set view parameter defaults, etc• Enter Main Tcl Event Event Loop
National Alliance for Medical Image Computing http://na-mic.org
Global Tcl Arrays• Module()
– Module(*) – state about modules– Module($m,*) – common state about module $m
• <ModuleName>() – an array with the same name as the Module– Internal state of the module, including the variables that are tied to the
GUI controls• Volume() and Model()
– Two important modules– Volume($id,*), Model($id,*)
• SLICER()– Command line argument values and version info
• Gui()– Window handles and appearance settings
• Anno()– Viewer annotations state and settings
National Alliance for Medical Image Computing http://na-mic.org
Key VTK Class Instances
• Slicer– Instance of vtkMrmlSlicer is “heart” of the
volume display process• Mrml(dataTree) – note, this is a class
instance of vtkMrmlTree, not an array reference– Hook to the scene data
• viewRen– The vtkOpenGLRenderer in the Viewer
National Alliance for Medical Image Computing http://na-mic.org
Module VTK Class Instance Naming Convention• Name of the Module with variable name in parentheses is used as
name of VTK class instance• For Example:
catch “SiemensMosaicReader(imageAppend) Delete”vtkImageAppend SiemensMosaicReader(imageAppend) SiemensMosaicReader(imageAppend) SetAppendAxis 2
• Note: delete class instances after use, but it is also good practice to do the catch “<inst> Delete” to make your module re-entrant
• Note: the is Not standard Tcl practice – but this is used to avoid namespace conflicts between modules
• This convention is widely used in the slicer Tcl code, but it can lead to confusion because the class instances look like Tcl array references.
• Be careful!
National Alliance for Medical Image Computing http://na-mic.org
Mrml Data Model
• Tree-structured scene description– Accessible as Mrml(dataTree)
• Serialized to XML file• Compiled to VTK class instances in
memory for display and manipulation
National Alliance for Medical Image Computing http://na-mic.org
Mrml http://www.slicer.org/mrml
National Alliance for Medical Image Computing http://na-mic.org
Mrml XML<?xml version="1.0" standalone='no'?><!DOCTYPE MRML SYSTEM "mrml20.dtd"><MRML><Transform> <Matrix name='manual' matrix='1 0 0 -3 0 1 0 -12 0 0 1 28 0 0 0 1'></Matrix> <Volume name='SPGR' filePattern='%s.%03d' filePrefix='spgr/I' rasToIjkMatrix='0 -1.06667 0 142.187
0 0 -1.06667 98.1333 0.666667 0 0 59.9667 0 0 0 1' rasToVtkMatrix='0 -1.06667 0 142.187 0 0 1.06667 157.867 0.666667 0 0 59.9667 0 0 0 1' positionMatrix='0 0 1 -89.95 -1 0 0 133.3 0 1 0 -148 0 0 0 1' description='LR' colorLUT='0' window='146' level='76' applyThreshold='yes' lowerThreshold='17' upperThreshold='355' imageRange='1 124'></Volume>
</Transform>
<Volume name='all' filePattern='%s.%03d' filePrefix='labels/all' rasToIjkMatrix='0 -1.06667 0 128 0 0 -1.06667 128 0.666667 0 0 62 0 0 0 1' rasToVtkMatrix='0 -1.06667 0 128 0 0 1.06667 128 0.666667 0 0 62 0 0 0 1' positionMatrix='0 0 1 -93 -1 0 0 120 0 1 0 -120 0 0 0 1' description='LR' colorLUT='-1' labelMap='yes' interpolate='no' window='4' level='1' lowerThreshold='0' upperThreshold='0' imageRange='1 124'></Volume>
<Model name='Skin' fileName='models/Skin.vtk' color='Skin' visibility='yes'></Model><Model name='Ventricles' fileName='models/Ventricles.vtk' color='Ventricles' visibility='yes'></Model><Model name='Vessels' fileName='models/Vessels.vtk' color='Vessels' visibility='yes'></Model></MRML>
National Alliance for Medical Image Computing http://na-mic.org
Mrml XML
• Describes the data as it exists on disk
• Composes the scene by positioning data elements
• Retains visualization and other state parameters
National Alliance for Medical Image Computing http://na-mic.org
Data Model – the Mrml Tree
• Mrml Tree is an in-memory data description corresponding to the mrml xml file– vtkMrmlNode subclasses contain the
metadata– Mapping of metadata to run-time state is
managed mainly in Tcl– vtkMrmlData subclasses are helpers for
classes that need C++ help to efficiently manage runtime state
National Alliance for Medical Image Computing http://na-mic.org
vtkMrmlNode
National Alliance for Medical Image Computing http://na-mic.org
Mrml Nodes• vtkMrmlNode abstract base class
– int ID;– char *Description;– char *Options;– char *Name;– char *Title;– virtual void Write(ofstream& of, int indent);
• IsA vtkObject• Can be subclassed in run time loadable
Modules
National Alliance for Medical Image Computing http://na-mic.org
vtkMrmlModelNode• vtkMrmlModelNode Example Instance Variables
// Stringschar *ModelID;char *FileName;char *FullFileName;char *Color;
// Numbersfloat Opacity;
// Booleansint Visibility;int Clipping;int BackfaceCulling;int ScalarVisibility;
National Alliance for Medical Image Computing http://na-mic.org
vtkMrmlModelNode - methodsvtkMrmlModelNode -- Example Methods SetModelID with 1 arg GetModelID SetFileName with 1 arg GetFileName SetFullFileName with 1 arg GetFullFileName SetColor with 1 arg GetColor SetOpacity with 1 arg GetOpacity VisibilityOn VisibilityOff GetVisibility SetVisibility with 1 arg ClippingOn ClippingOff GetClipping SetClipping with 1 arg BackfaceCullingOn BackfaceCullingOff GetBackfaceCulling SetBackfaceCulling with 1 arg ScalarVisibilityOn ScalarVisibilityOff GetScalarVisibility SetScalarVisibility with 1 arg
National Alliance for Medical Image Computing http://na-mic.org
vtkMrmlModelNode DTD• XML Representation Example:
<Model name='Skin' fileName='models/Skin.vtk' color='Skin' visibility='yes'></Model>
• DTD entry from Base/tcl/mrml20.dtd:<!ELEMENT Model (#PCDATA | Fiducials)><!ATTLIST Model name NMTOKEN "" fileName CDATA #REQUIRED color NMTOKEN "" description CDATA "" opacity NMTOKEN "1.0" visibility (yes | no) "yes" clipping (yes | no) "no" backfaceCulling (yes | no) "yes" scalarVisibility (yes | no) "no" ignore (yes | no) "no" scalarRange NMTOKENS "0 100">
National Alliance for Medical Image Computing http://na-mic.org
How Mrml Files are Read• MainMrmlReadVersion2.x
– In Base/tcl/tcl-main/Parse.tcl– Reads xml file and returns tcl list of nodes and
values• MainMrmlBuildTreesVersion2.0
– In Base/tcl/tcl-main/MainMrml.tcl– Converts list into vtkMrmlTree and
vtkMrmlNode instances– Modules can define
Module($m,procMRMLLoad) to extend Mrml
National Alliance for Medical Image Computing http://na-mic.org
Compiling the Mrml Tree• MainUpdateMRML
– In Base/tcl/tcl-main/Main.tcl– Is called after read or after changes to the
Mrml Tree– Invokes Main*UpdateMRML
• Models, Volumes, etc• Instances VTK classes, reads data, changes render
properties, etc to make scene match the MRML Tree
• Keeps Dev* GUI elements in sync with Tree and Selections
– Invokes Modules($m,procMRML) for each Module
National Alliance for Medical Image Computing http://na-mic.org
Rendering
• To make the newly updated data visible to the user:– Render3D
• Redraw just the 3D view
– RenderSlice <s>• Redraw slice view 0, 1, or 2
– RenderSlices• Redraw all slice views
– RenderAll• Redraw 3D and Slices
National Alliance for Medical Image Computing http://na-mic.org
Editing the MrmlTree Examples
• MainMrmlAddNode <nodeType>– Creates a new vtkMrmlNode instance and adds it to
Mrml(dataTree) and returns nodeId– Use vtkMrmlNode methods to set data
• Mrml(dataTree) RemoveItem $node– Remove a node
• Mrml(dataTree) InsertAfterItem $nodeBefore $node– Add your new node to specific spot in the hierarchy
• DevCreateNewCopiedVolume <OrigId> {Description ""} { VolName ""}– Create a copy node of existing volume with optional
new name and description
National Alliance for Medical Image Computing http://na-mic.org
Iterating Through the Mrml Tree
set nitems [Mrml(dataTree) GetNumberOfItems]for {set widx 0} {$widx < $nitems} {incr widx} { set item [Mrml(dataTree) GetNthItem $widx] puts “$item is a [$item GetClassName]”}
National Alliance for Medical Image Computing http://na-mic.org
Anatomy of A Module• See
Modules/vtkCustomModule/tcl/@[email protected] – For example, Custom
• CustomInit– Analogous to a Constructor
• CustomBuildGUI• CustomEnter / CustomExit
– When user module pane is visible/hidden• Custom*
– Any other “Methods” for the Module– These procs Edit the MrmlTree and call
MainUpdateMRML and Render
National Alliance for Medical Image Computing http://na-mic.org
The Shared Dev* Procs• Base/tcl/tcl-shared/Developer.tcl
– Interact with MrmlTree– Use standard look and feel conventions
• DevAddSelectButton– Automatically updating popup menu for Volumes, Models, etc – Adds entries when objects are created
• DevUpdateNodeSelectButton – Updates to currently selected
• DevAddButton• DevAddLabel• DevAddFileBrowse• DevInfoWindow, DevErrorWindow, DevWarningWindow…• Etc…
National Alliance for Medical Image Computing http://na-mic.org
Event Management
• The <Module>Enter and <Module>Exit procs can be used to set up custom mouse/keyboard bindings
• Base/tcl/tcl-shared/Events.tcl– pushEventManager / popEventManager – Arguments are list of:
• Widget, event, callback
National Alliance for Medical Image Computing http://na-mic.org
Event Example
In EMSegmentInit: foreach s $Slice(idList) { set widget $Gui(fSl${s}Win) append EMSegment(eventManager) " \ {$widget <Control-Button-1> {EMSegmentBindingCallback Sample %x %y}}” }
In EMSegmentEnter: pushEventManager $EMSegment(eventManager)
In EMSegmentExit: pushEventManager $EMSegment(eventManager)
National Alliance for Medical Image Computing http://na-mic.org
Event Example (continued)
EMSegmentBindingCallback calls EMSegmentReadGreyValue $x $y 1
In EMSegmentReadGreyValue: set s $Interactor(s) ;# get the slice window where user clicked scan [MainInteractorXY $s $x $y] "%d %d %d %d" xs ys x y ;# convert to pixels Slicer SetReformatPoint $s $x $y scan [Slicer GetIjkPoint] "%g %g %g" xIjk yIjk zIjk ;# convert to array coords set xIjk [expr int($xIjk)] set yIjk [expr int($yIjk)] set zIjk [expr int($zIjk)]
# get the pixel data from the volume set ImageData [Volume($v,vol) GetOutput] set pixel [$ImageData GetScalarComponentAsDouble $xIjk $yIjk $zIjk 0]
National Alliance for Medical Image Computing http://na-mic.org
Progress Bar
• Progress Bar on Lower part of slicer control window
• For a vtkProcessObject instance X x AddObserver StartEvent MainStartProgress x AddObserver ProgressEvent "MainShowProgress x" x AddObserver EndEvent MainEndProgress
National Alliance for Medical Image Computing http://na-mic.org
Fiducials Exampleproc intensities {vol} { # Note: does not account for any transforms applied to Volume catch "m Delete" vtkMatrix4x4 m eval m DeepCopy [Volume($vol,node) GetRasToVtkMatrix]
set ids [FiducialsGetPointIdListFromName default] foreach id $ids { set pt [eval m MultiplyPoint [FiducialsGetPointCoordinates $id] 1] set i [expr int([lindex $pt 0])] set j [expr int([lindex $pt 1])] set k [expr int([lindex $pt 2])] set imageData [Volume($vol,vol) GetOutput] set pixel [$imageData GetScalarComponentAsDouble $i $j $k 0] puts "fiducial $id value $pixel at $i $j $k" }}
National Alliance for Medical Image Computing http://na-mic.org
Reader Modules Motivation
• vtkMrmlVolumeNodes can be created by – User Interface– Reading in XML File– Programmatically
• Once the Node exists, the reader proc provides the vtkImageData
National Alliance for Medical Image Computing http://na-mic.org
Reader Modules• Managed by Base/tcl/tcl-modules/Volumes.tcl• ReaderModule needs an entry point proc named
Vol<type>Init– For example: VolGenericInit– This will be invoked by VolumesInit
• ReaderModule sets – Volume(readerModules,<type>,procGUI)
• to fill in type-specific controls as sub-pane of Volumes Modules
– Module(Volumes,readerProc,<type>)• so that MainUpdateMRML has a hook to call the reader
when <type> if found in the FileType attribute of a vtkMrmlVolumeNode
National Alliance for Medical Image Computing http://na-mic.org
Editor Modules Motivation
• The Editor manages manually guided segmentation of a – source volume into a working labelmap
• Editor Modules add to the palette of tools
• Ed(editor) is an instance of vtkImageEditorEffects– Allows you to apply a filtering pipeline
to modify the labelmap
National Alliance for Medical Image Computing http://na-mic.org
Editor Modules• Similar to Readers, these are managed by
Base/tcl/tcl-modules/Editor.tcl• Editor Module needs an entry point proc named Ed<type>Init
– For example: EdWatershedInit– This will be invoked by EditorInit
• EditorModule sets – Ed(<type>,procGUI)
• to fill in type-specific controls on the this editor type’s own subframe of the editor
• Segmentation is done In the UI control callbacks– Create a vtk (or vtkITK-based) processing pipeline of
vtkImageToImageFilter subclasses – Set parameters from UI elements or Slicer data such as Fiducial
locations– call Ed(editor) Apply <pipeline start> <pipeline end>– Ed(editor) manages progress bar, updating label map, undo (single)
National Alliance for Medical Image Computing http://na-mic.org
Conclusions
• Slicer provides a usable and extensible application framework for medical image computing research
• A significant base code provides a reference for many common coding tasks and user interface paradigms
National Alliance for Medical Image Computing http://na-mic.org
Resources
• www.slicer.org• www.na-mic.org/Wiki• www.na-mic.org/Bug• www.na-mic.org/Testing• [email protected]• [email protected]
NA-MICNational Alliance for Medical Image Computing http://na-mic.org
Slicer Architecture SupplementAdding A New View OptionNicole Aucoin
National Alliance for Medical Image Computing http://na-mic.org
Adding a new option example
To add a new View option, X, to the Scene Options node so can save and restore it via Mrml file:
• cxx/vtkMrmlSceneOptionsNode.h– add vtkGet/SetStringMacros(ViewX)– declare char *ViewX
• cxx/vtkMrmlSceneOptionsNode.cxx– Add ViewX in constructor, Write, Copy,
PrintSelf
National Alliance for Medical Image Computing http://na-mic.org
Adding a new option, con’t
• tcl/tcl-main/MainView.tcl– MainViewInit: add to Module(View,presets)
list, set View(X) to same default value. – Add a new proc MainViewSetX, with default
arg set to default value– MainViewStorePresets: add setting X's Preset
from current value– MainViewRecallPresets: add setting X from
preset value
National Alliance for Medical Image Computing http://na-mic.org
Adding a new option, con’t
• tcl/tcl-main/MainOptions.tcl– MainOptionsRetrievePresetValues: set
Preset(View,n,X) from the node– MainOptionsUnparsePresets: set the
node's X from the Preset
National Alliance for Medical Image Computing http://na-mic.org
Adding a new option, con’t
• tcl/tcl-main/MainMrml.tcl:– MainMrmlBuildTreesVersion2.0: add to
parsing of SceneOptions node, refer to vtkMrmlSceneOptionsNode C++ code