copyright © cisst erc, 2004 nsf engineering research center for computer integrated surgical...

Copyright © CISST ERC, 2004NSF Engineering Research Center for Computer Integrated Surgical Systems and Technology

Medical Robotics and Computer-Integrated Surgery

Russell H. TaylorDept. of Computer Science/Radiology/Mechanical EngineeringDirector, Center for Computer-Integrated Surgical Systems and TechnologyThe Johns Hopkins University3400 N. Charles Street; Baltimore, Md. [email protected]

NSF Engineering Research Center for Computer-Integrated Surgical Systems and Technology


Disclosures• Russell Taylor is a member of the Scientific

Advisory Board of Integrated Surgical Systems and of Image Guide, Inc., and he consequently has (an exceedingly small) financial interest in these companies, and he may join other Advisory Boards.

• Some of Dr. Taylor’s patents have been licensed by Image Guide, Incorporated, and he consequently has a small financial interest in IG,

• Other of Dr. Taylor’s patents have been licensed by Intuitive Surgical Systems.

• ISS, IG, GE, Intuitive Surgical, Siemens, Medtronic, Burdette Medical, and several other companies whose systems were discussed in this talk are affiliates of the NSF CISST ERC, of which Dr. Taylor is the Director.

• Dr. Taylor also enjoys fishing


Computer-assisted planning

Patient-specific Model

Update Model

Computer- Assisted

Execution

Update Plan

Computer- Assisted

Assessment

Preoperative Intraoperative

Atlas

Postoperative

Patient




Update Model

Computer- Assisted

Execution

Update Plan

Computer- Assisted

Assessment


Atlas

PostoperativePatient

Surgical “CAD”

Surgical “TQM”

Surgical “CAM”




Update Model

Computer- Assisted

Execution

Update Plan

Computer- Assisted

Assessment


Atlas

Postoperative

Patient

Surgical

Assistants


Computer-assisted

simulation & planning


Update Model

Simulate and replan in real

time

Computer- Assisted

Assessment


Atlas

Postoperative

Patient

The grand unified system*

* From Turf Valley Workshop (March 2003)


Drivers for adoption• Error-free surgery

– Designs & systems to make things safer– Integration of imaging, computer assistance, etc.

• Enablers for otherwise infeasible interventions– Superhuman capabilities (access, precision, …)– Delivery systems for novel image-guided therapies

• Enablers for research & development– Consistency, capability & data gathering

• Capture of information about cases• Use of robots as training tools

– Mentoring– Simulation integrated with robotics



Research and technology barriers

Modeling & analysis• Segmentation• Registration• Atlases• Optimization• Visualization• Task characterization• etc.

Interface Technology• Sensing• Robotics• Human-machine

interfacesSystems• Safety & verifiability• Usability & maintainability• Performance and validation


• Building anatomic models• Data fusion• Reasoning about uncertainty• Managing complexity• Computational algorithms• Limited data

Modeling & Analysis

Core Challenge: Develop computationally effective methods for patient-specific modeling and analysis

Before

After


Modeling and Analysis Highlights

• Explosion of 3D imaging modalities

– MRI, CT, Ultrasound, etc.• Patient-specific models

based on images• Statistical atlases and

deformable models are emerging as crucial enabling technologies within CIS

• Beginning to see application of real-time computer vision methods


Interface Technology

• Imaging and sensing• Robotics• Human-machine interfaces

Core Challenge: Fundamentally extend the sensory, motor, and human-adaptation abilities of computer-based systems in an unusually demanding and constrained environment


Interface Technology HighlightRobotic devices for surgery and image-guided interventions


Systems Science

• Modularity & Standards• Robustness & Safety• Customization• Validation • Information management

Core Challenge: Develop architectures, building blocks, and analysis techniques that facilitate rapid development and validation of versatile CIS systems & processes with predictable performance

Engineering Research Center for Computer Integrated Surgical Systems and Technology


Surgical CAD/CAM• Goals: to develop and

evaluate systems that will incorporate surgical plans into surgical execution using computers and robotics.

• Research areas:

– Percutaneous therapy systems (needle placement)

– Patient specific models (image segmentation and registration)

– Statistical atlases (prostate, femur, pelvis)

– Robotics (remote center of motion, needle drivers)


Image-Guided Needle-Based Interventions

Potentially significant impact on medical practice

• Minimally invasive (compared to open surgery)

• Faster recovery• Less morbidity• Fewer complications• Lower cost• Repeatable in many indications

• Sharply increasing number of procedures

Challenging but also doable• Constrained process – Constrained process –

formally describableformally describable• Major challenges (in addition

to open/lap surgery):• no visibility• no access• no room to maneuver• no room to recover


Current clinical scope

Spine/Bone70% of population affected in lifetime

400,000 metastatic

cancer /year

LiverMetastasis from colorectal cancer

130,000 new /year

60,000 death /year

Hepatitis worldwide

Prostate200,000 cancers/year

1M biopsies /year

10M BPH currently

25% of men affected in lifetime

Why these?• Significant health problems• Right mix of challenge and doability• Clinical buy-in• Experience of investigators• Funding opportunities

United States numbers

Eye~100k/y retinal

occlusions,

>100k/y age-related macular degeneration

(AMD)

EarHearing loss of 30-35% of 65-

75 yo

40-50% over 75 yo

MACRO SCALE MICRO SCALE


Why these organ systems?

Humanitarian reasons• Significant health problems in US

and worldwide

Opportunistic reasons• Right mix of challenge and doability• Clinical buy-in• Funding opportunities• Experience of investigators• Publishing opportunities


Engineering Goals

1. “One-Stop Shopping”•Plan, Do, and Verify – in one sitting

2. “Plug & Play Surgery”•Rapidly configure for target organ,

imaging modality, and therapy•Predictable performance

3. Critical Mass of Components


-0.3

-0.2

-0.1

0

0.1

0.2

0.3

0 2 4 6 8 10

Dis

tanc

e (m

m)

Surgical Assistants• Goals: to develop and evaluate

systems that augment the physical and sensory performance of a surgeon during surgery

• Research areas:

– Microsurgery (tremor reduction, manipulation on small scale)

– Video augmentation (tool tracking mosaicing)

– Surgical enhancement (tracking surgical actions, virtual fixtures)


Handle force

Kv

Steady Hand Guiding for Microsurgery

R. Taylor & R. KumarFree hand motion Steady hand motion


Stable platform for interventions

R. Kumar, 2001


Evolution to human-machine partnership

• Situation assessment• Task strategy & decisions• Sensory-motor coordination

Augmentation System

• Sensor processing• Model interpretation• Display

atlases

• Manipulation enhancement

• Online references & decision support

• Cooperative control and “macros”

atlases

libraries

R. Taylor


Computer-assisted

simulation & planning


Update Model

Simulate and replan in real

time

Computer- Assisted

Assessment


Atlas

Postoperative

Patient

The grand unified system*



System barriers• Safety & verifiability

• “The usual”• No fly zones• Information-based supports

• Usability & maintainability• “The usual”• Surgical performance

analysis• Recording information

• Performance and validation• “The usual”• Accuracy propogation• Analysis-based generation

of test plans • Standards


Key need: Standards

• Interoperability• System/OR architectures• Information Interchange• Control• Human interfaces

•Performance measurement• Accuracy• Task performance• Outcomes

•Use/Practices

• Information archiving


A few “grand challenge” areas

• Grand Unified System – Real-time patient-specific models– Surgical total information management– Active human-machine partnerships– Correlate performance to outcome

• “Fanstastic Voyage”*– Micro-invasive interventions on tiny structures

• Surgery on cells & microstructures


Surgical Total Information Management

• Real time sensing & modeling of surgical situation– Patient-specific models– Surgical tools & tool-tissue interations– Surgeon movements– Etc.

• Application drivers– Assistance (robots & information support)– Outcomes assessment

• Technology drivers– Imaging, computer vision, image analysis– Sensors, sensor networks– Biomechanics, statistical modeling– Human-machine interfaces, haptics– Performance/task modeling


How can we get there?

Strong and committed teams

– Surgeons– Engineers– Industry

Focus on systems that address important needs

Rapid iteration with measurable goals

Have fun!


Bio

Russell H. Taylor has a B.E.S. from Johns Hopkins, and a Ph.D. in Computer Science from Stanford. He joined IBM Research, where he developed the Agent Markup Language (AML) robot language. Following an assignment in Boca Raton, he managed robotics and automation technology research activities at IBM Research until returning to full-time technical work, when he became a manager of Computer Assisted Surgery. Taylor then moved to Johns Hopkins University as a Professor of Computer Science. He is currently a Professor of Computer Science, with joint appointments in Radiology and Mechanical Engineering, and is the Director of the NSF Engineering Research Center for Computer-Integrated Surgical Systems and Technology at Johns Hopkins. Research interests include robot systems, programming languages, model-based planning, and the use of imaging, model-based planning, and robotic systems to augment human performance in surgical procedures. He is Editor Emeritus of the IEEE Transactions on Robotics and Automation, Fellow of the IEEE, and a member of various honorary societies, panels, editorial boards, and program committees.

copyright © cisst erc, 2004 nsf engineering research center for computer integrated surgical...

Documents

intuitive surgical systems

impact of computer

surgical planning

technology slide

surgical practice slide

nsf cisst erc

copyright cisst erc

technology prediction