Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
The CISST Software Package
Peter Kazanzides
Johns Hopkins University
www.cisst.org
November 16, 2005
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Outline
• Background and Motivation
• Where are we going?
• CISST Libraries
– Foundation libraries
– Real time support
• Development process and tools
• Tour of web site
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Historical Background
• ERC-developed software was captured in:– CIS library
• Common tools, such as logging• Vectors, matrices, transformations• Interface to tracking systems• Numerical methods, registration, …
– MRC library• Common interface to different robots• Essentially a “wrapper” around API for hardware
that provides low-level control
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Robot Controller Architecture
Supervisory/Trajectory Control (~100 Hz)
Application(non-real-time)
Ha
rdw
are
Read Sensors
Compute Joint Goals
Compute Goal on Trajectory
Interpolate Setpoint
Compute Control
Read Sensors
Servo Control(~1000 Hz)
Application
API
MRCcisstMRC
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Motivation for CISST Package
• Improve process, design, testing, portability, maintainability for open source release and to facilitate clinical use:– Programming standards– Design reviews– Portable build tools– Automated testing– User documentation
• Enable the development of real-time software for physical devices such as robots and trackers
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Motivation for Real Time Support
• Motivated by transition from motion controller boards (with processor and vendor’s software) to I/O boards (no processor) and research software
Motion Controller Boards
(intelligent hardware)
I/O Boards (non-intelligent)
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Example: Teleoperation of Snake Robot
LoPoMoCo
Control PC (RTAI/Linux) I/O and Amps
I/O and Amps
LoPoMoCo
Read Sensors
Compute Joint Goals
Compute Goal on Trajectory
Interpolate Setpoint
Compute Control
Read SensorsApplication
API
Read Sensors
Compute Joint Goals
Compute Goal on Trajectory
Interpolate Setpoint
Compute Control
Read SensorsApplication
API
Control PC (RTAI/Linux)
Read Sensors
Compute Joint Goals
Compute Goal on Trajectory
Interpolate Setpoint
Compute Control
Read SensorsApplication
API
Read Sensors
Compute Joint Goals
Compute Goal on Trajectory
Interpolate Setpoint
Compute Control
Read SensorsApplication
API
Master Robots
Slave Robots
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Example: Image-Guided Robot for Rodent Research
PC (Windows) Servo Control and Amps (Galil)
DMC-2143 RobotApplication
API
3D Slicer
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Where are we going?
C++ Software Libraries• cisst libraries
• other libraries
Frameworks• Based on system complexity
• Component of larger system
Binary components• hardware interfaces
• research algorithms
stat
ic li
nkin
g
frozen spots
hot spots
Distributed Interfacedy
nam
ic
load
ing
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
CISST Libraries
• (Mostly) Stable– Open source* Sept 2005
• cisstInteractive ~Nov 2005
• Beta version– Open source ~Jan 2006
• Work in process– cisstTracker ~Jan 2006
– cisstMRC ~June 2006
Foundation libraries
cisstCommoncisstVectorcisstNumericalcisstInteractive
Real Time Support
cisstOSAbstraction
cisstDeviceInterfacecisstRealtime
Interventional Devices
cisstTrackercisstMRC…
*www.cisst.org, current license based on Slicer 2.x, goal is Slicer 3.0
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
cisstVector
• Vectors, Matrices and Transformations• Extensive use of C++ templates (metaprogramming)• Fixed size and dynamic
– Fixed size especially suited for real-time code• Operations on slices and sub-regions• References to vectors and matrices
– Improves interoperability with other libraries• Matrices in row-major or column-major format• C++ wrapping of NetLib (numerical methods,
including CLAPACK, MINPACK)
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Fixed Size Vectors and Matrices
• Templated by:– Element type (int, double, etc.)– Dimension (number of rows, columns, etc.)
• Efficiency considerations (for templated dimension):– Loop is easy, but not efficient for small vectors:
int Sum() { sum = 0 for (i=0; i < _size; i++) sum += data[i]; return sum;}
– Recursive function also not efficient:int RecursiveSum(int size){ return (size == 1) ? data[0] : RecursiveSum(size-1) + data[size-1];}int Sum() { return RecursiveSum(_size); }
– Recursive template (template metaprogramming) is efficient:• Compiler “unwraps” recursive template into straight-line code
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Fixed Size Vectors and Matrices
• Operations provided by “recursive engines”:– Classify operations by:
• Number of operands
• Type of operands
• Type of result
• Storage location for result
– Small number of engines handle all operations• Example: same engine for addition, subtraction, …
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
cisstNumerical
• C++ Interface to NetLib code (Fortran)• Versions for fixed size and dynamic operands• Can pre-allocate solution object
– More efficient if multiple calls madevctDynamicMatrix<double> A(rows,cols,VCT_COL_MAJOR);
nmrSVDSolutionDynamic solution;
solution.Allocate(A);
while (1) {
vctRandom(A,0.0,10.0);
nmrSVD(A, solution); // A = U*S*V’
cout << solution.GetU() << …
}
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
cisst vs. VNL
• cisstVector/cisstNumerical benefits:– More efficient fixed size vectors (few loops)– Stride allows operations on slices (e.g., can
access matrix columns as vectors)– Matrices can be in column-major format (more
efficient with NetLib)
• VNL benefits:– Mature package, larger development community– More features (polynomials, Matlab file I/O, …)– Support for more compilers
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Python Interface
• Automatic wrapping of C++ for Python (Swig)• Object registry to share objects between languages• Can load cisst libraries into Python shell• Can start Python shell from C++ program• GUI features provided by cisstInteractive (using
wxWidgets)
C++ Robot Control Library
C++ Numerical Library
C++ Vector/ Matrix Libraries
Application Software
(Complex executable with
GUI tools)
SWIG Class Wrapper
Python Workspace (IRE)
SWIG Class Wrapper
SWIG Class Wrapper
C++ Object
SWIG Wrapper
Registry(Shared Object)
WxWidgets
Simple dialogs, frames, etc.
WxWidgets
Simple dialogs, frames, etc.Custom Python Tools
Error Log ControllerConfig File EditorExperiment Scripts
Ire.lnk
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Device Interface
• Device Hierarchies:
Tracker Tool
GetPosition
Robot Tool
GetPosition
MoveToPosition
Should be able to use robot in place of a tracker
Can lead to complexity!
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Device Interface
• Our solution: query with string to obtain command object (Command Pattern)
Tracker Tool
Provides
Robot Tool
Provides
ddiDeviceInterface
Configure
Provides
GetMethodByName
…
Initialize (e.g., from XML)
Return list of implemented operations (strings)
Return command object for specified operation (string)
Abstract Base Class
{ “GetPosition” } { “GetPosition”, “MoveToPosition” }
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Device Interface
· · · dev = ptr to devicedevConfigure(…);cmd = devGetMethodByName(“GetPosition”);
· · ·
public:
void Startup();
void Run();
};
· · ·cmdExecute(data);
· · ·
class myTask : public rtsTask {
private: ddiDeviceInterface* dev; ddiCommand* cmd;
Non-real-time
Real-time
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Real Time Support
1. Devices and Tasks should be interchangeable:– Example: Servo control via an intelligent device or via a
software task– Solution: Task class derived from device class, but also
includes a device
ddiDeviceInterface
rtsTask
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Real Time Support
2. Maintain time history of important state data– State Data Table (SDT), indexed by time and data id
3. Task communication with Command Pattern:– Read from Task SDT or Device– Write to Task Mailbox or Device– Command object can handle remote communications
High-level task (low frequency)
Low-level task (high frequency)
Device Interfacemailbox
SDT
mailbox
SDTHardwareCommand
objects
Command objects
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
rtsTask
ddiDeviceInterface
osaThread
osaThreadBuddy
rtsStateDataTable
vector <osaMailBox >
map <string, ddiCommandBase *>
StateDataTableMailBoxes
DeviceThread
ThreadBuddy
vector < rtsStateDataArrayBase *>
vector < rtsTimeTicks >
rtsTimeIndex
vector <string>
Ticks
StateVector
StateVectorDataNames
rtsStateDataArrayBase
template<_type>
rtsStateDataArray vector <_type>Data
ddiCommandBase
template<_device>
ddiCommand
_device* Device
bool (_device::*)() Action
Operations
PositionOutput Voltage
Desired Position
[0] [1] [2]
IndexReaderIndexWriter
Legend:Member ofInherits fromInstance of (snapshot)
cisstRealTime
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Development Tools
CTest
Build Environment
LaTeX
Doxygen
SWIG
Compile
Compile
Compile
Library Binaries(static & dynamic, e.g.,
cisstVector, cisstCommon)
Test Programs
Applications
Optional Interpreter (IRE)
Wrapped Source
Formatted Documentation(e.g., pdf, html)
(e.g., VC++, gcc/make)
Test Results
Link
CVSTrac (bug/feature requests)
CMake
CVSRepository(source control)
CMake Build Instructions
Documentation
Libraries
Test App
Applications
Scripts
CppUnitDart2
(dashboard)PyUnit
Link
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Quick Tour of the Web Site
www.cisst.org/resources/software
Copyright © CISST ERC, 2005NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology
Development Summary
• Tools adequately manage implementation, (unit) testing and maintenance phases of development– Automated testing will be hard for physical devices
• Much documentation is manually created (not enforced by the process)– Requirements, risk analysis, (high-level) design, validation– User guide, tutorial, quickstart
• We needed Dart sooner than we expected!– gcc is getting pickier!– Windows static/shared libraries (dll export)