Thursday,December 20,2018

End-point stiffness estimation ofthehumanarm inHuman-RobotInteraction

Supervisor:Prof.ssaElenaDeMomi

LuisaCucugliato872310luisa.cucugliato@mail.polimi.itCo-supervisor:JacopoBuzzi

Context

Human-RobotInteraction (HRI)

“When humans, robots andtheenvironment cometocontact witheach other andform atighty coupled dynamical system toaccomplish atask”

[A.Ajoudani, A.Zanchettin.S.Ivaldi.A.Albu-Shaffer etal.,“Progressandprospects oftheHuman-Robot Collaboration”, HAL,2016]

Luisa Cucugliato 2

Human-RobotInteraction

RESEARCHCHALLENGE: Enable natural andintuitive human-robot communication

Safety

Stability

Trasparency

3Luisa Cucugliato

Humandynamics control

CentralNervousSystem

Arm jointconfiguration

Muscleco-contraction

Humandynamicproprieties

Humans perform awidevariety ofmovements byadjusting themechanical impedanceparameters oftheir musculoskeletal system inmotion

DAMPING,INERTIA,STIFFNESS

4Luisa Cucugliato

Humandynamics control

CentralNervousSystem

Arm jointconfiguration

Muscleco-contraction

Humandynamicproprieties

Controller

Robotdynamicproprieties

Humans perform awidevariety ofmovements byadjusting themechanical impedanceparameters oftheir musculoskeletal system inmotion

STIFFNESS

DAMPING,INERTIA

4Luisa Cucugliato

Humanarm stiffness

• It is themain responsible ofmuscular-skeletal stability• It is thefirstelement modulated inneural-motor controlstrategies

Thearm stiffness is themuscles response toexternal perturbations fromequilibrium point

STIFFNESSMODULATION

Kinematic:armconfiguration variations

Dynamic:muscles co-contraction level variations

+

Humanarm second-order linearmodel:

k

b

mF

5Luisa Cucugliato

Humanarm stiffness

Geometrically described as anellipsoid:

• Orientation• Eccentricity• Size

Parameterschange wrt thestiffness modulation strategy

[N.Hogan,“Adaptive control ofmechanical impedance bycoactivation ofan- tagonist muscles,”IEEETRANSACTIONSONAUTOMATICCONTROL,1984]

Majorprincipal axis

Minorprincipal axis

x

y

SVD

Principal axes Axes direction

6Luisa Cucugliato

Thesis goal

Toestimate thehumanarm end-point stiffness asdescriptor ofhumanintentionduringphysicalHRI

Creation ofastiffnessacquisition set-up

Implentation ofastiffness

estimation method

Stiffness analysis formultiplepositionsandmuscle co-contraction

levels

1 2 3

7Luisa Cucugliato

Stiffness Investigation:measurment

End-point stiffness is acquired measuring thehumanarm restoring force as response torobotend-effector displacementduring aphysical HRI

8Luisa Cucugliato

Stiffness Investigation:methods

METHODSKnown displacement Frequency perturbation

• Known displacements onplane• Linearrelationbetween forceand

displacement

• Donot explain thewhole harmdynamics

[Mussa-Ivaldi andE.Bizzi,“Neural, mechanical, andgeometric factors subserving arm postureinhumans,”TheJournalofNeuroscience Vol.5,No.10,1985]

Luisa Cucugliato 9

Stiffness Investigation:methods

METHODSKnown displacement Frequency perturbation

• Known displacements onplane• Linearrelationbetween forceand

displacement

• Donot explain thewhole harmdynamics

• Stochastic displacement inspace• Frequency description ofhuman

arm characteristics

• Provide inertia,damping andstiffness

[E.Perreault, R.Kirsch, A.M.Acosta ,“Multiple-input, multiple-output system identification forcharacterization oflimb stiffness dynamics”,Biological Cybernetics, 1999]

Luisa Cucugliato 9

Stiffness Investigation:methods

METHODSKnown displacement Frequency perturbation

• Known displacements onplane• Linearrelationbetween forceand

displacement

• Donot explain thewhole harmdynamics

• Stochastic displacement inspace• Frequency description ofhuman

arm characteristics

• Provide inertia,damping andstiffness

Luisa Cucugliato 9

Workflow

End-point perturbation(Stiffness acquisition)

User FORCE

DISPLACEMENT

Post-processing

SVD

Arm end-pointstiffness estimation Geometrical

representation

10Luisa Cucugliato

Workflow

End-point perturbation(Stiffness acquisition)

User FORCE

DISPLACEMENT

Post-processing

SVD

Arm end-pointstiffness estimation Geometrical

representation

Multiplerobotpositions

RobotController

3

5cm

5cm

Robothandle

Grid ofequidistant points

1 2 4

5 6 7 8

10Luisa Cucugliato

Workflow

End-point perturbation(Stiffness acquisition)

0%

25%

50%

75%

MVC

User FORCE

DISPLACEMENT

Post-processing

SVD

RawEMG

EMGacquisition

Arm end-pointstiffness estimation Geometrical

representation

EMGprocessing Co-contraction indexcomputation

COC

Graphical interface

COCvisual feedback

Multiplerobotpositions

RobotController

3

5cm

5cm

Robothandle

Grid ofequidistant points

1 2 4

5 6 7 8

Luisa Cucugliato 10

Experimental set-up

Robotic arm Forcesensor Inertial-EMGsensors

Collaborativerobot

7-axis(7DoFs)

Low weight (16Kg)

Highperformanceandsafety

KUKALWR4+

MATERIALS

Wearable controldevice

IMUs (9-axis)

8sEMG sensors

M3815C MYOArmband

Load cell

FullWeastonebridgeconfiguration

Luisa Cucugliato 11

Experimental set-up

Robotic arm Forcesensor Inertial-EMGsensors

Collaborativerobot

7-axis(7DoFs)

Low weight (16Kg)

Highperformanceandsafety

KUKALWR4+

Wearable controldevice

IMUs (9-axis)

8sEMG sensors

M3815C MYOArmband

Load cell

FullWeastonebridgeconfiguration

( $

MultipleRobotpositions

MODALITIES

3

5cm

5cm

Robothandle

Grid ofequidistant points

1 2 4

5 6 7 8

0%

25%

50%

75%

MVC

8EEpositionsUser

4COC-Indexlevels

MultipleCo-contracion levels

x

y

Luisa Cucugliato 11

Acquisition protocol

Robotic arm

1

23

4

5

7

86

Visualinterface

Grid ofrobotpositions Rigid Armband

RobotbaseRF

Forcesensor

ShoulderRF

Inertial-EMGsensors

Luisa Cucugliato 12

Implementation: ROSframework

Robotic Arm

Joints torque

Joints angles

250Hz

ForceSensor Inertial-EMGsensor

Kinematiccomputation

50Hz

200Hz

Arm orientation

Muscles EMG

Sub-sampling

2KHz250Hz

250Hz250Hz COC-Index

computation

Forcex3

Torquex3

UserDataset

Luisa Cucugliato 13

COC-Indexcomputation

PROCESS

v Signal normalization

v Acquisition agonist-antagonist EMG

v Pre-processing:

v COC-Indexcomputation

• Offsetremoval• Rectification

• Smoothing (Moving Average)

Luisa Cucugliato 14

COC-Indexcomputation

v Signal normalization

v Acquisition agonist-antagonist EMG

v Pre-processing:

v COC-Indexcomputation

• Offsetremoval• Rectification

• Smoothing (Moving Average)

PROCESS

EMGmax: BicepsEMGmin:Triceps

Luisa Cucugliato 14

COC-Indexcomputation

PROCESS

EMGmax: TricepsEMGmin:Biceps

v Signal normalization

v Acquisition agonist-antagonist EMG

v Pre-processing:

v COC-Indexcomputation

• Offsetremoval• Rectification

• Smoothing (Moving Average)

Luisa Cucugliato 14

FrequencyMethod1/2

Humanhand sphericalpathway

1.Robothandle displacement:stochastic perturbation inspace (20mminx,y,z)

Luisa Cucugliato 15

FrequencyMethod1/2

2.Acquisition oftheforceand displacement at therobotEnd-Effector (250Hz)

ForcesensorM3815C

Inversekinematic(KUKAjoints angles)

1.Robothandle displacement:stochastic perturbation inspace (20mminx,y,z)

LPF:15Hz

LPF:15Hz

Luisa Cucugliato 15

FrequencyMethod1/2

3.F/dMIMOsystem NON-parametric identification frequency response (0-10Hz)

zh(t)

Fx(t)[N]

Fy(t)[N]

Fz(t)[N]

x(t) [m]

y(t) [m]

z(t) [m]

1.Robothandle displacement:stochastic perturbation inspace (20mminx,y,z)

CdF(s)=cross-spectra input-output,Sd(s) =auto-spectra input-inputH(s)=transferfunction output/input

2.Acquisition oftheforceand displacement at therobotEnd-Effector (250Hz)

[3x1] [3x3] [1x3]

i,j=1,2,3

Luisa Cucugliato 15

FrequencyMethod2/2

4.Secondorder fitting ofthenonparametric transferfunction (0-10Hz)

fitting =system zeros

=system pole

[A.Ajoudani andA.Bicchi, “Tele-impedance:Teleoperation withimpedance regulation using abody-machineinterface,” International Journal ofRobotic Research, 2012]

Luisa Cucugliato 16

FrequencyMethod2/2

4.Secondorder fitting ofthenonparametric transferfunction (0-10Hz)

fitting

5.Parametric identification ofthehumanhand dynamic proprieties

Inertia matrix

Damping matrix

Stiffness matrix

Luisa Cucugliato 16

=system zeros

=system pole

Trials

• 2right-handed users

• 4trialsforeachposition(varing theCOC-level)

• 8trialsforeachposition

Stiffness geometricalrepresentation

Results analysis

2x4x8total trials xy

z

End-point stiffness ellipsoid

Luisa Cucugliato 17

Result:Stiffness Ellipses

• Male• 24years old• 185cm

User1: User2: • Female• 23years old• 160cm

User’s shoulderUser’s shoulder

Luisa Cucugliato 18

Result:Stiffness Orientation

𝜙 = angle the major axis formswrt the line connectingshoulder and wrist

x

y𝜙 = 2.8°±1.1° 𝜙 = 2.6°±1.4°

User1 User2

Stiffness ellipses orientation aproximatelly alligned withtheshoulder-wrist axis (directionality)

Shoulder-wrist line Shoulder-wrist line

Luisa Cucugliato 19

Result:Stiffness Orientation

𝜗 =anglethemajoraxisformswrt y-axis onplane

x

y𝜗 = 54.4° − 9.3°

x

y𝜗 = 65.2° − 12.6°

User1 User2

Stiffness ellipses orientation changes according toeachhumanarm position onplane

MajoraxisMajoraxis

Luisa Cucugliato 20

Result:Stiffness Eccentricity

x

y

x

y

User1 User2

Stiffness ellipses eccentricity (Ecc) showsaanisotropic behavior

• Highter eccentricity fordistal positions (Kinematic dependency)• Highter eccentricity forhighco-contraction level (Dynamic depencency)

𝐸𝑐𝑐 = 0.43 ±0.29𝐸𝑐𝑐 = 0.46 ±0.35

Luisa Cucugliato 21

Result:Stiffness Size

25% 50% 75%COC Index:

25% 50% 75%COC Index:

y[m

]

User1

User2

Stiffness ellipses size (ellipse area)is influenced bythemucles co-contraction levels

0.50-0.5-1-1.5

0.50-0.5-1-1.5x [m]

Coc: 25%Coc: 50%Coc: 75%

Coc: 25%Coc: 50%Coc: 75%

1.2

1

0.8

0.6

0.4

0.2

0

-0-2

-0-4

1.2

1

0.8

0.6

0.4

0.2

0

-0-2

-0-4

x [m]

Position4

y[m

]y

[m]

Ellipse size increases withco-contraction index

Luisa Cucugliato 22

Result:Stiffness Size

25% 50% 75%COC Index:

25% 50% 75%COC Index:

y[m

]

User1

User2

Stiffness ellipses size (ellipse area)is influenced bythemucles co-contraction levels

0.50-0.5-1-1.5

0.50-0.5-1-1.5x [m]

Coc: 25%Coc: 50%Coc: 75%

Coc: 25%Coc: 50%Coc: 75%

1.2

1

0.8

0.6

0.4

0.2

0

-0-2

-0-4

1.2

1

0.8

0.6

0.4

0.2

0

-0-2

-0-4

x [m]

Position4

y[m

]y

[m]

Ellipse size increases withco-contraction index

∆𝑠𝑖𝑧𝑒 = 50.61%

Luisa Cucugliato 22

Result:Stiffness Ellipsoidz

[N/m

]

y [N/m] y [N/m]x [N/m]

z[N

/m]

x [N/m]

Position4

Luisa Cucugliato 23

Conclusions andFuturework

Lademomi nonvuolechedicacheilmetodoèoffline

• Onlineestimation ofthehumanarm stiffness bymeans ofthefrequency-basedmethod mplementation

• Evaluationofkinematic anddynamic influences onthestiffness estimated bytheproposed method

Luisa Cucugliato 24

Conclusions andFuturework

Lademomi nonvuolechedicacheilmetodoèoffline

Increasing number oftrials invastigating theinter-subject variability

Implementing the robot controlusing the estimated stiffness ascommand reference

Adopting EMG sensorsmore accurate todistinguish the musclescross-talking

• Onlineestimation ofthehumanarm stiffness bymeans ofthefrequency-basedmethod mplementation

• Evaluationofkinematic anddynamic influences onthestiffness estimated bytheproposed method

Luisa Cucugliato 24

Thursday,December 20,2018

End-point stiffness estimation ofthehumanarm inHuman-RobotInteraction

Supervisor:Prof.ssaElenaDeMomi

LuisaCucugliato872310luisa.cucugliato@mail.polimi.itCo-supervisor:JacopoBuzzi

Thank you!