active tremor compensation in handheld … active tremor compensation in handheld instrument for...

86
1 Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological University Singapore [email protected]

Upload: nguyenduong

Post on 15-Mar-2018

224 views

Category:

Documents


3 download

TRANSCRIPT

Page 1: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

1

Active Tremor Compensation in Handheld Instrument for Microsurgery

Wei Tech AngSchool of Mechanical & Aerospace Engineering

Nanyang Technological UniversitySingapore

[email protected]

Page 2: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

2

Contributors

Cameron N. RiviereAssociate Research Professor

David Y. ChoiPh.D. Student

Si Yi KhooResearch Engineer

Medical Robotics Technology CenterThe Robotics InstituteCarnegie Mellon UniversityPittsburgh, PA, USA

Wei Tech AngAssistant Professor

Mounir KrichaneExchange Student (EPFL)

Robotics Research Centre &Sch. of Mechanical & Aerospace Eng.Nanyang Technological UniversitySingapore

Page 3: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

3

Microsurgery with Active Handheld Instrument

Visuomotor Control System

Noisy, Tremulous Motion

Motion Sensing

Visual Feedback

Micron

VitreoretinalMicrosurgery

Estimation of erroneous motion

Tip manipulation for active error compensation

Task Definition

Problem Analysis

Engineering Solutions

Technical Details

Page 4: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

4

Vitreoretinal Microsurgery

Removal of membranes ≤ 20 µm thick from front or back of retina

Page 5: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

5

Vitreoretinal Microsurgery

Injection of anticoagulant using intraocular cannulation to treat retinal vein (~∅100 µm) occlusion

Page 6: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

6

Vitreoretinal MicrosurgeryTremor: under microscope

Page 7: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

7

Involuntary Hand Movement and Microsurgery

Page 8: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

8

Involuntary Hand Movement and Microsurgery

Complicate microsurgical procedures and makes certain delicate interventions impossibleImpact on microsurgeons

2 of 10 surgeons become microsurgeonsFactors affecting tremor

Fatigue – strenuous exercise etc.Caffeine/alcohol consumptionLack of practice – long vacation etc.Age – experience vs hand stability

Microsurgeons’ consensus:10 µm positioning accuracy

Page 9: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

9

Involuntary Hand Movement of Healthy Human

Physiological TremorRoughly sinusoidal motion, 8-12 Hz ≤ 50 µm rms in each principal axis

Non-tremulous ErrorsMyoclonic jerk, drift etc.Aperiodic, may be in the same frequency band as voluntary motionLarger amplitude: > 100 µm

Page 10: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

10

Microsurgery with Active Handheld Instrument

Visuomotor Control System

Noisy, Tremulous Motion

Motion Sensing

Visual Feedback

Micron

VitreoretinalMicrosurgery

Estimation of erroneous motion

Manipulation of tip for active error compensation

Page 11: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

11

Robotic Error Compensation Approaches

Telerobotic systems: Zeus (Computer Motion) & Da Vinci (Intuitive Surgical)

Master-Slave manipulators

Erroneous motion filtered by motion scaling

Page 12: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

12

Robotic Error Compensation Approaches

‘Steady-hand’ robot:Russell Taylor et al., Johns Hopkins University

Surgeon and compliant robot hold tool simultaneouslyForce feedback‘Third hand’ operation

Erroneous motion damped by rigidity of robot

Page 13: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

13

Robotic Error Compensation Approaches

Active Handheld Instrument:Paolo Dario, Scuola SuperioreSant’Anna, Pisa, ItalySame concept

Page 14: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

14

Comparison of Robotic Solutions

Telerobotics‘Steady Hand’ robot Active Handheld Instrument

CheapUnobtrusiveSaferLimited workspaceNo motion scalingNo ‘third hand’

Obtrusive

Unobtrusive

< US$15K

> US$150K> US$1M

Page 15: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

15

Micron Current Prototype

Length: 180 mm long Diameter: Ø20(16) mmWeight <100 g9 DOF inertial and magnetic sensing system at the back end3 DOF piezoelectric driven parallel manipulator at front end with disposable surgical needle

180 mm (w/o needle)

Ø20 mm

Ø16 mm

Manipulator System

Sensing System

Disposable surgical needle

Page 16: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

16

System OverviewMotion of instrument

Magnetometer-aided all-

accelerometer IMU

ADCSensor fusion

Estimation of erroneous

motion

Erroneous tip displacement

WDtremor(3×1)

Inverse kinematics

Joint variables λ1, λ2, λ3

DAC

Power Amplifier

Piezoelectric-driven parallel

manipulator

V1, V 2, V 3

Motion of instrument tipWDvoluntary (3×1)

Host PC

Inverse feedforwardcontroller

BA(6×1), BM(3×1)

Forward kinematics

WDB(3×1), WΘB(3×1)

WDtip(3×1)

Sen

sing

Filtering

Man

ipulatio

n

Page 17: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

17

Microsurgery with Active Handheld Instrument

Visuomotor Control System

Noisy, Tremulous Motion

Motion Sensing

Visual Feedback

Micron

VitreoretinalMicrosurgery

Estimation of erroneous motion

Manipulation of tip for active error compensation

• Resolution

• Accuracy

Page 18: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

18

Sensing System Design

Magnetometer-aided all-accelerometer inertial measurement unit (IMU):

3 dual-axis miniature MEMS accelerometers Analog Devices ADXL-203: 5mm x 5mm x 2mm, < 1gThree-axis magnetometerHoneywell HMC-2003: 26mm x 19mm x 12mm, <10g

Housed in 2 locations

Back Sensor Suite

Sensing direction

Front Sensor Suite

ZB

XB

YB

Dual-axis accelerometer

Dual-axis accelerometers

Tri-axial Magnetometer

Manipulator System

Page 19: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

19

Sensing Modality

Internally referenced sensors because:Less obtrusiveExternally referenced sensors require a line of sightResolution:Inertial sensor < 1 µmExternally referenced (e.g. Optotrak): ~ 0.1 mm

All accelerometers because:Low cost, miniature gyros too noisy→ Poor sensing resolutionNavigation/tactical grade gyros - too expensive and bulky

Page 20: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

20

Differential Sensing Kinematics

Body acceleration sensed by accelerometer at location i:

Differential Sensing

onsAccelerati inducedRotation−

×Ω+×Ω×Ω++= BiBiCGi PPgAA

Xw

P23

P13

P12

P3

P2P1

Yw

Zw

W

B 2

3

1

Y2

Z2

( )

⎥⎥⎥

⎢⎢⎢

⎡••

=⎥⎥⎥

⎢⎢⎢

•=

⎥⎥⎥

⎢⎢⎢

••=

=×Ω+×Ω×Ω=−=

z

y

x

ijijij

aAaA

aA

jiPAAA

12

122323

13

13 ,,

3 2, 1,,,][]][[

Centripetal Acceleration

Tangential Acceleration

Measurement

Page 21: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

21

Differential Sensing Kinematics

3 unknowns:Ω = [ωx ωy ωz]T

3 differential acceleration measurements:AD = [a13x a23y a12z]T

Solve system of nonlinear equations by Gauss-Newton or Levenberg-Marquart methodNumerical instability

Assume Ω2 ≈ 0, solve for analytically

ΩXw

P23

P13

P12

P3

P2P1

Yw

Zw

W

B 2

3

1

Y2

Z2

Page 22: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

22

Sensing KinematicsUpdating quaternions:

Directional Cosines matrix

Gravity Removal: WAE = WCBBA – Wg

Tip Displacement:

⎥⎦

⎤⎢⎣

⎡Ω−

Ω×Ω=ΩΩ=

×

××

0][

21~ ),()(~)(

31

1333Ttqttq

⎥⎥⎥

⎢⎢⎢

+−−+−−−+−++−−−+

=23

22

21

2010322031

103223

22

21

203021

2031302123

22

21

20

)(2)(2)(2)(2)(2)(2

qqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqqq

CBW

tipBB

BW

t

TtE

Wtip

Wtip

W PtCddATtPtP ])[()()()( ×Ω++−= ∫ ∫−

τττ

Page 23: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

23

Sensing Resolution (Error Variance) Analysis

Sensing resolution dependent on sensor noise floorAngular Sensing

Sensing equation:Aij = f(Ω)= ([Ω×] [Ω×] + [ ×])Pij

Covariance:C(Aij) = C(Ω) Pij

Pij↑, C(Ω)↓

Pij

σAx2

σAy2

σAy2

σAx2

σωx2 orσωy

2

Back sensor suite

Front sensor suite

Ω

Page 24: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

24

Proposed All-Accelerometer vs Conventional Inertial Measurement Unit

All-accelerometer IMUMaximized Pij, with physical constraint of a slender handheld instrument

Conventional IMU (3A-3G)Tokin CG-L43D rate gyros x 3

96.9% / 32x4.42 × 10-21.41ωz

99.3% / 130x1.08 × 10-21.41ωx & ωy

Noise reduction / resolution

improvement

6A Error std. dev.

(deg/s)

3G-3AError std. dev.

(deg/s)

Page 25: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

25

Angular Sensing Resolution Comparison

Small angular velocity & sensor noise floor

0 50 100 150 200 250 300 350 400 450 500-2

0

2

4

6

8

0 50 100 150 200 250 300 350 400 450 500-2

0

2

4

6

8

0 50 100 150 200 250 300 350 400 450 500-2

0

2

4

6

8

ωz

(deg

/s)

ωx, ω

y(d

eg/s

)

ωG

(deg

/s)

Time (ms)Time (ms)

Tokin Gyroscope

All-accelerometer IMU

Page 26: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

26

Sensing Resolution (Error Variance) Analysis

Translational Sensing2 accelerometers in each sensing direction:

Sensing resolution improves by a factor of 2½

Better orientation estimation → more complete removal of gravity → better translation estimation

2111222

AiA

AjAiA

σ=σ→

σ+

σ=

σ

Page 27: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

27

Microsurgery with Active Handheld Instrument

Visuomotor Control System

Noisy, Tremulous Motion

Motion Sensing

Visual Feedback

Micron

VitreoretinalMicrosurgery

Estimation of erroneous motion

Manipulation of tip for active error compensation

• Resolution

• Accuracy

Page 28: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

28

Integration Drift of Inertial Sensors

Integration driftErroneous DC Offset

Ramp QuadraticError accumulates and grows unbounded over time

Poor sensing accuracy

∫∫0 0.2 0.4 0.6 0.8 1

-1

-0.5

0

0.5

1Acc

eler

atio

n (m

m/s

2 )

0 0.2 0.4 0.6 0.8 10

0.02

0.04

0.06

Vel

ocity

(mm

/s)

0 0.2 0.4 0.6 0.8 1

0

0.01

0.02

0.03

Time (s)D

ispl

acem

ent (

mm

)

Mean = 0.05 mm/s2

Page 29: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

29

Measurement Model

Measurement model allows error analysis and compensationMeasurement Model = Physical (Deterministic) Model + Stochastic Model

Measurement Error

Measurement Model• Physical

• Stochastic

Compensation

Page 30: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

30

Accelerometer Model

CalibrationRecord accelerometer outputs at orientation inline (V+g) and opposite (V-g) to gravity

Linear modelAcceleration, A = (Vo–B)/SF gScale factor, SF = ½(V+g−V-g) V/gBiasB = ½ (V+g+V-g) V

β = 0°

+90°

180°

90°

α = 180°

- 90°- 90°

β

α

y

x

g

Page 31: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

31

Experimental Observations

-1 -0.5 0 0.5 11.5

2

2.5

3

3.5

Acceleration (g)

Acc

eler

omet

er O

utpu

t (V

)

A

B+

+

CW: A+ → B-

CCW: B+ → A-

g

-1 -0.5 0 0.5 11.5

2

2.5

3

3.5

Acceleration (g)

Acc

eler

omet

er O

utpu

t (V

)

α±150

α±90

g

α = 90°, β = -180° to 180°

α±30

α = 30° & 150°, β = -180° to 180°

Page 32: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

32

Phenomenological ModelingBias, Bx(Vy, Vz) = Bx+gx(Vy)+hx(Vz)Scale Factor,SFx(Vz) = rx2Vz

2 + rx1Vz + rx0

Model

Ax = (Vx− Bx(Vy, Vz)) / SFx(Vz)

-1 -0.5 0 0.5 11.06

1.07

1.08

1.09

1.1

1.11

z-Acceleration (g)

Sca

le F

acto

r (V

/g)

1.5 2 2.5 3 3.5 4-0.03

-0.02

-0.01

0

0.01

0.02

0.03

0.04

y-Accelerometer Output, Vy (V)

Res

idue

Rxy (V

)-1 -0.5 0 0.5 1

-5

0

5

10

15x 10-3

z-Acceleration (g)R

esid

ue R

xz (V)

gx(Vy) = px2Vy2 + px1Vy + px0

hx(Vz) = qx2Vz2 + qx1Vz + qx0SFx(Vz) = rx2Vz

2 + rx1Vz + rx0

In plane cross- axis effect

Out of plane cross- axis effectScale Factor

Page 33: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

33

Sensing Experiment - Translation

Motion generator10 Hz, 50 µm p-p sinusoid

Displacement SensorInfrared interferometer (Philtec, Inc., Model D63)Sub-micrometer accuracy

Rotary Stages Accelerometer

Displacement Sensor

Motion Generator

Motion

Page 34: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

340 0.2 0.4 0.6 0.8 1-500

-400

-300

-200

-100

0

100

200

Time (s)

Acc

eler

atio

n (m

m/s

2 )Sensing Results - Translation

0 0.2 0.4 0.6 0.8 1-150

-100

-50

0

50

100

150

200

Time (s)

Acc

eler

atio

n (m

m/s

2 )

--89.7Error Reduction (%)

<1<531*Proposed Physical Model

6272300Linear Model

Scale Factor (mm/s2)Bias (mm/s2)Rmse (mm/s2)

Linear Model Proposed Physical Model

* ADXL-203 rated rms noise = 22.1 mm/s2

Interferometer

Accelerometer

Page 35: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

35

Stochastic Model

Random noise analysis by Allan Variance method Dominant accelerometer noise types:

Velocity random walkWhite noise in acceleration

Acceleration random walkWhite noise in jerk

Trend / Bias InstabilityTemperature drift

10-2 100 102 104100

101

102

Velocity random walk

Acceleration random walk

τ (s)

σ(τ)

(m/s

/s)

Trend / Bias Instability

Page 36: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

36

Temperature Drift

Time varying zero biasHeating up of internal circuitrySteady state: 2-12 hours

Solutions:ModelingWait for steady stateOvenization

Heat up sensor using power resistorChanges sensor behavior

0 2 4 6 8 10 122.64

2.645

2.65

Bia

s (V

)0 2 4 6 8 10 12

1.075

1.08

1.085

1.09

1.095

Time (hr)S

cale

Fac

tor (

V/g

)

Steady State

Page 37: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

37

Sensor Fusion via Cascaded Two-Stage Kalman Filtering

Motion

Bias

+

_Scale Factor

Local Gravity

Misalign-ment

Correction

Location & Orientation

Error

Magnetic North

Augmented State

Kalman Filtering

Acc. Trend

Acceleration Random

Walk1/s

Velocity Random Walk

+

+

Acc.

Mag.

White Noise 1/s

White Noise

+ +

Physical Model

+Norma-lization

Misalign-ment

Correction

Bias Orientation Error

Q-based Kalman Filtering

Kinematics

Tool tip displacement

_Numerical

Transform-ation Error

Page 38: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

380 2 4 6 8 10

-50

0

50

100

Augmented State Kalman Filtering

Time domain sensor fusionAugmented state dynamic equation:

[ ][ ][ ][ ][ ]

[ ][ ][ ][ ][ ] ( )

[ ][ ]

[ ]][

0

0

][

20000

][10000

1000001000010

00211

]1[

11111

221

2

kw

k

kk

kc

TkTcTc

kx

kbkbkakaku

A

T

T

TT

kx

kbkbkakaku

arw

vrw

a

u

a

u⎥⎥⎥⎥⎥⎥

⎢⎢⎢⎢⎢⎢

+

⎥⎥⎥⎥⎥⎥

⎢⎢⎢⎢⎢⎢

++

+

⎥⎥⎥⎥⎥⎥

⎢⎢⎢⎢⎢⎢

⎥⎥⎥⎥⎥⎥⎥

⎢⎢⎢⎢⎢⎢⎢

=

+

⎥⎥⎥⎥⎥⎥

⎢⎢⎢⎢⎢⎢

+++++

β

ξξ Velocity Random

WalkAcceleration Random Walk

Quadratic Model of Bias Drift

⎪⎩

⎪⎨

⎧x

⎩⎨⎧

b

State Transition Matrix

Vel

ocity

(mm

/s2 )

Time (s)

KF

ASKF

Page 39: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

39

Orientation Fusion

Source 1: Differential sensing kinematicsΩA[k] from differential accelerationState transition matrix:

Dynamic state equation: qA[k + 1] = F[k]qA[k] + γ[k]Orientation defined by quaternion

+ : high resolution− : drift

)44()44( ][~2

][sin

][1

2][

cos][ ×× += kΘk

kΩI

kkF

θθ

TkTkk TA

AAA ⎥

⎤⎢⎣

⎡Ω−

Ω×Ω=ΘΩ=θ

×

××

0][

21][~ ;][][

31

1333

Page 40: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

40

Orientation Fusion

Source 2: TRIADGravity vector & Magnetic North vector are non-collinearTRIAD algorithm:

zB = –gB/||g||; yB = zB×NB/|| zB×NB ||; xB = yB×zB

WCB = [xB yB zB]

+ : non-drifting– : poor resolution

zw

yw

xw

Nw

gw

Page 41: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

41

Quaternion-based Kalman Filtering - Gravity Tracking

Source 1: ΩAHigh resolution, drifting

Source 2: NB + gBNoisy, non-drifting

Quaternion-based KF: Q-KFReduced noise, non-drifting

0 200 400 600 800 1000 12000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

0 200 400 600 800 1000 12000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

ΩA

0 200 400 600 800 1000 12000

0.05

0.1

0.15

0.2

0.25

0.3

0.35

0.4

Time(ms) Time(ms)

Nor

mal

ized

Gra

vity

Nor

mal

ized

Gra

vity

Nor

mal

ized

Gra

vity

NB + gB

Q-KF

Page 42: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

42

Sensing Experiment - Orientation

Page 43: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

43

Translational Sensing Accuracy

No non-drifting referencePoor translational sensing accuracy

Not important if tremor is separable from drift and intended motion

0 500 1000 1500 2000-20

-10

0

10

20

0 500 1000 1500 2000-10

0

10

20

30

40

0 500 1000 1500 2000-10

0

10

20

30

40

Tremor Filtering

-

+

Sensed motion

Intended+Drift

Tremor

Page 44: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

44

Microsurgery with Active Handheld Instrument

Visuomotor Control System

Noisy, Tremulous Motion

Motion Sensing

Visual Feedback

MICRON

VitreoretinalMicrosurgery

Estimation of erroneous motion

Manipulation of tip for active error compensation

• Real-time

• Phase lag

Page 45: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

45

Frequency Selective Filters Phase Characteristics

Physiological tremor has a distinct frequency bands:8 – 12 Hz

Voluntary motion: ≤ 1 Hz; Electrical noise: >> 12 HzClassical frequency selective bandpass / band-stop (notch) filters

Phase lag ≡ Group (time) delayFiltered signal is a time delayed version of the actual sensed motionUnacceptable condition for real-time error canceling application: compensating action might worsen the error

Page 46: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

46

Zero Phase Filtering

Separation of tremor from the intended motion without introducing phase lag

Prediction/projection capabilityAdaptive

Non-linear phase response of IIR filter, i.e. phase characteristic changes with frequency

Two proposed algorithmsWeighted-frequency Fourier Linear Combiner (WFLC)Adaptive Phase Compensating Band-pass Filter

Page 47: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

47

Fourier Linear Combiner (FLC)

Truncated Fourier series to adaptively estimate amplitude and phase of periodic signal with known frequency (ω0)

Page 48: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

48

Weighted-frequency Fourier Linear Combiner (WFLC)

Extends FLC to also adaptively estimate the frequency using another LMS algorithmBand-pass filter to select the band of interest

Assumption: rate of change of the dominant input signal frequency is slow

Zero-phase notch (band-stop) filter effect

FLC

WFLC

Signal input

ω0

Tremor

BandpassFilter

Page 49: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

49

Weighted-frequency Fourier Linear Combiner (WFLC)

Page 50: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

50

Weighted-frequency Fourier Linear Combiner (WFLC) Experiment

1 DOF motion canceling experimentAve. rms tremor amplitude reduced 69%Stability problem

Double adaptive algorithm

Page 51: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

51

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1

-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Filter Phase Characteristics

Low-pass filters create phase lagHigh-pass filters create phase lead

0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1-1

-0.8

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

TimeInput Output

Low-pass Filter High-pass Filter

Time

Page 52: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

52

Adaptive Phase Compensating Band-pass Filter

The idea:To design a cascaded low-pass and high-pass filters such that phase lag of low-pass is compensated by phase lead of high-pass for a certain known input frequencyWFLC to estimate the instantaneous frequency

MotivationMost sensors come with built-in low pass filters

Page 53: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

53

Implementation

Design LP-HP filter pairs with equal and opposite phase characteristicsFilter design frequency band: 3 – 7 HzFilter type: Elliptical 2nd

order Roots of transfer function can be modeled by a linear function

LP HP

Roots of Filter Transfer Functions

× - Poles ο - Zeros

7 Hz

7 Hz

3 Hz3 Hz

5.67 Hz

5.67 Hz

Page 54: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

54

Adaptive Phase Compensating Band-pass Filter

Page 55: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

55

Motion Canceling Experiment

Motion generator oscillating at 2.0 HzCamcorder recording a rectangular target

25 frames per secondImage post processing to simulate real-time compensation

Motion generator

Page 56: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

56

Adaptive Phase Compensating Band-pass Filter

Rmse:Raw footage = 2.61 pixelsCompensated = 0.66 pixelsError reduction: 75%

Original Compensated

Page 57: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

57

Microsurgery with Active Handheld Instrument

Visuomotor Control System

Noisy, Tremulous Motion

Motion Sensing

Visual Feedback

MICRON

VitreoretinalMicrosurgery

Estimation of erroneous motion

Manipulation of tip for active error compensation

•High precision tracking control

• Mechanism design

Page 58: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

58

Piezoelectric Actuator Hysteresis

+ : High bandwidthFast responseHigh output force

– : Hysteresis

~15% of max. displacement

0 20 40 60 80 1000

2

4

6

8

10

12

14

Voltage (V)

Dis

plac

emen

t (µm

)

Page 59: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

59

0 20 40 60 80 10010

20

30

40

50

60

70

80

90

0 0.2 0.4 0.6 0.8 1-20

0

20

40

60

80

100

Commercial Piezo-System with Feedback Controller

Piezo-driven 3 axis micro-positionerPolytec-PI, Germany, NanoCubeTM P-611>$ 10,000Feedback sensors: strain gagesTracking a 10 Hz, 100 µm p-p sinusiod

Hysteresis still presentLow-pass filtered behavior

Time (s)

Dis

plac

emen

t (µm

)

Measured Desired

Dis

plac

emen

t (µm

)

Voltage (V)

Page 60: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

60

Open-loop Feedforward Controller with Inverse Hysteresis Model

Develop a mathematical model that closely describes the hysteretic behavior of a piezoelectric actuatorExistence of an inverse hysteresis model

0 2 4 6 8 10 12 140

20

40

60

80

100

0 20 40 60 80 1000

2

4

6

8

10

12

14

0 2 4 6 8 10 12 140

2

4

6

8

10

12

14

Feedforward controller Piezoelectric

Desired Displacement (µm)

Actuator Response (µm)

Voltage(V)

Displacement (µm)

Dis

plac

emen

t (µm

)

Voltage (V)

Vol

tage

(V)

Actuator Response (µm)

Displacement (µm)

Dis

plac

emen

t (µm

)

Desired Displacement

(µm)

Page 61: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

61

Prandtl-Ishlinskii (PI) Operator

Rate independent backlash operator:Hr = maxx(t) – r, minx(t) + r, y0Linearly weighted superposition of backlash operators:

r1

-r1

y

x

wh1

ri

y

x

Whi = Σwhi

[ ] )(,)( 0 tyxHwty rTh=

Page 62: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

62

Modified PI Operator

Backlash operators are symmetric but real hysteresis is not

Modeling saturation by linearly weighted superposition of dead-zone operators:

)]([)(

0 ),(0 ,0,)(max

)]([

tySwtz

dtyddty

tyS

dTs

d

⋅=

⎩⎨⎧

=>−

=

PI modelReal

Hysteresis

di

z

yd0 d1

∑=

=i

jsjsi wW

0

z

yd >0

ws

d =0

Page 63: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

63

Modified PI Operator

( ) )]](,[[][)( 0 tyxHwSwtxtz rThd

Ts ⋅⋅=Γ=

x(t) y(t) z(t)

x(t) z(t)

time time

Backlash operators Saturation operators

Γ

Page 64: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

64

Inverse Modified PI Hysteresis Model

Hysteresis Model

Inverse Model

Page 65: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

65

Inverse Modified PI Operator

( ) )](],ˆ[[]ˆ[ ,,,10,, tyzSwHwtz

dT

srT

h ⋅⋅=Γ−

)(ˆ tz )(tx

1−Γtime time

)(ˆ tz )(tx

Inverse saturation operators

Inverse Backlash operators

Page 66: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

66

Piezoelectric Hysteresis is Rate Dependent

Basic assumption of Prandtl-Ishlinskii operator:

Hysteresis is rate independent

Our observation:Hysteresis is ratedependent

Tremor frequency is time varying and person specific

8-12 Hz

25 Hz

15 Hz

5 Hz

Page 67: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

67

Rate-dependent PI Operator

Assume saturation is rate independentSum of the backlash weights up to ri (slope of the hysteresis curve at interval i) is linearly dependent on actuation rate Whi = Σwhi; 0 200 400 600 800 1000 1200

-0.6

-0.4

-0.2

0

0.2

0.4

0.6

0.8

1

Whi

Actuation rate (µm/s)

( ) ( ) nitxcxWtxW ihihi ...0 ),()( 0 =+=

Page 68: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

68

Rate Dependent PI Hysteresis Model

Rate dependent model:

Also a PI type → Inverse existsRate dependent inverse model

)]](,[))(([)](,[)( 0 tyxHtxwSwtxxtz rT

hdT

s ⋅⋅=Γ=

( ) )](],ˆ[[))((]ˆ[ ,,))((

,10, tyzSwHtxwtz d

Tstxr

Th ′

− ⋅⋅=Γ

Page 69: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

69

Open-Loop Feedforward Controller with Inverse Rate-Dependent PI Hysteresis Model

Measured Desired

0 0.05 0.1 0.15 0.2-2

0

2

4

6

8

10

12

14

Dis

plac

emen

t (µm

)

Error

Rate-dependent

Time (s)

0 0.05 0.1 0.15 0.2-2

0

2

4

6

8

10

12

14

Error

Rate-independent

Time (s)

Dis

plac

emen

t (µm

)0 0.05 0.1 0.15 0.2

-2

0

2

4

6

8

10

12

14

Time (s)

Dis

plac

emen

t (µm

)

WithoutModel

Tracking single frequency stationary sinusoids

Page 70: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

70

0 0.05 0.1 0.15 0.2-2

0

2

4

6

8

10

12

14

Open-Loop Feedforward Controller with Inverse Rate-Dependent PI Hysteresis Model

Tracking multiple frequency non-stationary sinusoids

0 0.05 0.1 0.15 0.2-2

0

2

4

6

8

10

12

14

Time (s)D

ispl

acem

ent (µm

)

Error

Error

Measured Desired

Without model.

Rate- independent.

0 0.05 0.1 0.15 0.2-2

0

2

4

6

8

10

12

14

Dis

plac

emen

t (µm

)

Time (s)

Error

Dis

plac

emen

t (µm

)

Rate- dependent.

Page 71: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

71

Dynamic Motion Tracking Results

5.38.017.3

0.59 ± 0.060.89 ± 0.041.91 ± 0.08max error ± σ(µm)

1.42.89.2

0.15 ± 0.0030.31 ± 0.031.02 ± 0.07rmse ± σ (µm)

Rate-dependent

Rate-independent

Without model

(%)amplitude p-prmse

(%)amplitude p-p

errormax

rms noise of interferometer = 0.01 µm

Page 72: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

72

Tremor Tracking Results

Tracking recordings of real tremor using 1 piezoelectric stackRmse = 0.64% of max ampl.; Max error = 2.4% of max ampl.

0 0.2 0.4 0.6 0.8 2-2

0

2

4

6

8

10

12

14DesiredObtainedError

0 0.2 0.4 0.6 0.8 1-2

0

2

4

6

8

10

12

14DesiredObtainedError

Without Model Rate-Dependent Controller

Time (s) Time (s)

Dis

plac

emen

t (µm

)

Dis

plac

emen

t (µm

)

Page 73: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

73

Microsurgery with Active Handheld Instrument

Visuomotor Control System

Noisy, Tremulous Motion

Motion Sensing

Visual Feedback

MICRON

VitreoretinalMicrosurgery

Estimation of erroneous motion

Manipulation of tip for active error compensation

•High precision tracking control

• Mechanism design

Page 74: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

74

Design of Parallel Mechanism

Page 75: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

75

Manipulator Design

3 DOF piezoelectric-driven parallel manipulator1 actuator per axis, max effective stroke = 12.5 µmMotion amplification = 9.4x, total stroke > 100 µm

Tool tip approximated as a point, hence only 3 DOF manipulationParallel manipulator design because

Rigidity, compactness, and design simplicity

Located at the front end to balance the weight of the instrument

Page 76: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

76

Design of Parallel Mechanism - New

IEEE EMBS 2005 (Shanghai) Best Student Design Competition winner

David Choi et al.Monolith design using Stereolithography∅22 x 58 mm

Page 77: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

77

Manipulator Kinematics

Inverse kinematicsModeled as Lee & Shah RS-typeClosed-form solution

No internal singularity

Page 78: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

78

Workspace Analysis

Xmax, Ymax = 650 µmZmax = 100 µmTremor Space = ∅ 50 µm

Tremor SpaceManipulator Workspace

Page 79: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

79

Manipulator 3D Tracking Result

Tracking planar circle of ∅200 µmMean tracking rmse ~ 12.1 µm

Page 80: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

80

Microsurgery with Active Handheld Instrument

Visuomotor Control System

Noisy, Tremulous Motion

Motion Sensing

Visual Feedback

Micron

VitreoretinalMicrosurgery

Estimation of erroneous motion

Manipulation of tip for active error compensation

Page 81: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

81

Motion Canceling Experiment

Translation onlyGenerated motion: 9 Hz, 91 µm p-p

Ave compensated tip motion over 10 runs = 60 µm p-p34.3% reduction3D optical sensing system

< 1.0 µm rms accuracy

8.8 9 9.2 9.4 9.6 9.8 10

-40

-30

-20

-10

0

10

20

30

40

Tip Displacement in Z

time (seconds)

Dis

plac

emen

t (m

icro

ns)

0 5 10 15 20 250

5000

10000

15000

Frequency Response without Compensation

Mag

nitu

de

0 5 10 15 20 250

5000

10000

15000

Frequency Response with Compensation

Frequency (Hz)

Mag

nitu

de

Page 82: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

82

Motion Canceling Experiment

Page 83: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

83

Micron Canceling Hand Tremor

Total range: 52% reduction

RMS amplitude: 47% reduction

0 1 2 3

-60

-30

0

30

60

time (s)

disp

lace

men

t (µm

) compensated

uncompensated

Page 84: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

84

Other Applications

Surgery and DiagnosticsActive compensation of periodic human physiological and pathological motion

Beating heart, breathing, etc.Pathological tremor due to Parkinson’s diseases, multiple sclerosis, etc.

Military optical tracking devicesHandheld, vehicle & ship mounted

Consumer camera/camcorder

Page 85: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

85

Other Applications: Cell Manipulation / Dissection

Page 86: Active Tremor Compensation in Handheld … Active Tremor Compensation in Handheld Instrument for Microsurgery Wei Tech Ang School of Mechanical & Aerospace Engineering Nanyang Technological

86

Questions & Comments

Wei Tech AngSchool of Mechanical & Aerospace Engineering

Nanyang Technological UniversitySingapore

[email protected]