IMAGE GUIDED NAVIGATION SYSTEM FOR MINIMALLY INVASIVE SURGERYHua ZhongOctober 2007

Outline

Minimally Invasive Surgery Registration and Navigation System

Overview Ultrasound Virtual Touch 4D Registration Non-rigid Registration Visualization

What is minimally invasive surgeryBenefits & limitations

Minimally Invasive Surgery

Minimally Invasive Surgery

Catheter

Minimally Invasive Surgery

Minimally Invasive Surgery

What are currently available systemsHow do they workLimitations

Registration & Navigation Systems

Registration & Navigation Systems

CT image segmented 3D model reconstructed from CT

Registration & Navigation Systems

Magnetic Transmitter

Bed

Catheter

Catheter Tip

Example system: Biosense Carto Merge System

Registration & Navigation Systems Limitations 1: radiation Limitations 2: manually collect surface

points Limitations 3: low accuracy

Touch with ultrasoundUltrasound image plane thickness correctionRegistration with virtual touch points

Ultrasound Virtual Touch

Ultrasound Virtual Touch

Ultrasound Virtual Touch

Ultrasound Virtual Touch

Position Sensor Video

Ultrasound Virtual Touch

Virtual Touch Video

Ultrasound Virtual Touch

Virtual TouchVirtual Touch12000+ points ~3 minutes12000+ points ~3 minutes

Manual physical touchManual physical touch 88 points ~20 minutes88 points ~20 minutes

Ultrasound Virtual Touch

31 63 127 1278 127810

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Point Set Size

Re

gis

tra

tio

n S

uc

ce

ss

ful R

ate

Registration Successful Rate VS. Point Set Size

6%

20%

50%

96%99%

31 63 127 1278 127810

1

2

3

4

5

6

7

8

9

10

11

12

Point Set Size

Av

era

ge

Re

gis

tra

tio

n E

rro

r (m

m)

Average Registration Error VS. Point Set Size

11.58mm

8.78mm

4.91mm

0.95mm

0.09mm

Ultrasound Virtual Touch

Ultrasound Image Thickness

Ultrasound Image Thickness

Ultrasound Image Thickness

Thickness of ultrasound image plane, also called beam width: 5-8mm

Size of left atrium: 70-100mm First we need to measure the thickness

Measure Image ThicknessRichard, B.: Test object for measurement of section thickness at us. Radiology 211 (1999) 279-282Skolnick, M.: Estimation of ultrasound beam width in the elevation (section thick-ness) plane. Radiology 180 (1991) 286-288

45˚

Band Width

Transducer

Measure Image Thickness

Precise built of model Precise motion of the plane or multiple

planes Can we reduce all those restrictions?

Measure Image Thickness

Transducer

Measure Image Thickness

β

Measure Image Thickness

Measure β in ultrasound images

Measure Image Thickness

W is the width of the white band in ultrasound imageα is the angle of the sloped plane which can be measured

α= 45˚ , β= 0˚ : Thickness = w

Measure Image Thickness

Move the slope around Measure thickness at various depth Interpolate sample thickness to generate

a continuous function: Thickness = f(depth)

Measure Image Thickness

Correct Error from Image Thickness

1. Position and orientation of the ultrasound image plane.

2. Thickness of the image at point o.

3. Local normal of the object surface.

If we know these 3 things, we can correct the error!

Correct Error from Image Thickness

Correct Error from Image Thickness

Correct Error from Image Thickness

Correct Error from Image Thickness

Correct Error from Image Thickness1. Position and orientation of the

ultrasound image plane. 2. Thickness of the image at point o. 3. Local normal of the object surface.

YesYesYesYes

??

Correct Error from Image ThicknessEstimate local surface normalEstimate local surface normal By fitting local registration points

By pre-registration

Correct Error from Image ThicknessEstimate local surface normalEstimate local surface normal

Estimated surface normal will only be used to determine which of P1 or P2 is the real surface point.

It doesn’t need to be very accurate.

Correct Error from Image Thickness1. Position and orientation of the

ultrasound image plane. 2. Thickness of the image at point o. 3. Local normal of the object surface.

YesYesYesYes

YesYes

Correct Error from Image Thickness Experiment Design:1. Different group with various average

intersecting angles

Correct Error from Image Thickness The error is dependent on intersecting

angle: smaller the angle larger the error

Prager, R.W., Rohling, R.N., Gee, A.H., Berman, L.: Rapid calibration for 3-dfreehand ultrasound. In: Ultrasound in Medicine and Biology. (1998) 24(6):855-869

a

bo

o’ a

b

o

o’

Correct Error from Image Thickness Experiment Design:1. Different group with various average

intersecting angles2. Register to the surface model without

correction3. Register to the surface model with

correction4. Separate evaluation point set

Correct Error from Image Thickness Expected result

Intersecting angle

Regis

trati

on E

rror

Un-correct points

Corrected points

Correct Error from Image Thickness

Correct Error from Image Thickness Registration error around 1.8-1.9mm Average accuracy boost: 20.45% Consistent accuracy across the spectrum

Conclusion for “Virtual Touch” Faster More accurate More consistent

Time-space registration for heart model with full motion.

4D Registration

4D Registration

4D Heart Model Movie

4D Registration

Capture multiple 3D models during a cardiac cycle

4D Registration

Heart shape changes from cardiac cycle Heart shape also change from breath

cycle

Lung

Left atrium

4D Registration

Shape changes from breath

Location measured at heart wallShif

t d

uri

ng d

iffere

nt

bre

ath

phase

(m

m)

4D Registration

Two cycles are independent to each other Have to remove one cycle and

synchronize all shape change to the other cycle

Remove breath cycle with high-frequency ventilation machine: it pumps in and out a little air at a high frequency, and the pressure in lung can be seen as constant.

4D Registration

4D Registration

Given SpaceRegistration

SurfacePoints

HeartModel

Given Time RegistrationR,T

S

4D Registration

EM Hidden variable: time-alignment Iterate: 1. fix time alignment find space

registration2. fix space registration find time

alignmentvideo

Some small non-homogeneous local shape changes cannot be found by global rigid registration. We add a non-rigid local registration component.

Non-rigid Registration

Non-rigid Registration

Location measured at heart wallShif

t d

uri

ng d

iffere

nt

bre

ath

phase

(m

m)

Non-rigid Registration

Non-rigid Registration

For any surface points For any surface points ppii its nearest point on the model is its nearest point on the model is CCpipi

After registration, our system can visualize the heart model and catheters in various ways to make navigation easy and intuitive.

Visualization

Visualization

Visualization

Visualization

Visualization

video

Acknowledgement

Takeo Kanade, David Schwartzman, Yanxi Liu, George StettenBranislav Jaramaz, Jianbo Shi, Louise Ditmore, Sharon & Deborah

Questions

Thank you!