collision safety for physical human-robot collaboration

Jae-Bok Song

School of Mechanical EngineeringKorea University

Seoul, Korea

IROS 2015 Workshop “Physical Human-Robot Collaboration”

Collision Safety for

Physical Human-Robot Collaboration

OutlineOutline

Human-Robot Interaction

3-Step Safety Strategy

• Collision Prediction & Avoidance

• Collision Detection & Reaction : Active Safety

• Collision Absorption : Passive Safety

Advanced Collision Detection

• Sensorless Collision Detection

• Collision Detection Index (CDI): Frequency-based

• Collision Detection Index (CDI): Projection-based

Collision Analysis & Simulation

2

Summary

Physical Human-Robot Interaction • Frequent contacts between humans and robots

• Sharing the same workspace Collaborative robots

Human-robot collisionHuman-robot collision Need for collision safetyNeed for collision safety

Safety Strategies

3

Before collision VisionAvoidance

Collision Torque sensingDetection

After collision SpringAbsorption

Collision Safety

3-Step Safety Strategy

4Safe Physical Human-Robot Interaction (pHRI)

Step 1Prediction

Step 2 Active Safety

Step 3 Passive Safety

Collision Safety !!

Collision PredictionNon-contact sensors

Path RegenerationPath planning

Safe MotionCollision avoidance

Collision DetectionSensor: JTS, skin

Collision Reaction

Sensorless: current monitoring

Collision AbsorptionSafe Joint Mechanism

Limit switch

Collision ReactionEmergence stopReflex motion

Approachto Human

Fail

Success Success Success

Fail

Collision Prediction

• Based on noncontact sensors

- Vision sensors, Kinect sensors

- Ultrasonic sensors

Step 1: Collision Prediction and Avoidance 5

<Collision avoidance using Kinect>

Kinect

Capacitive sensor

Camera

Ultrasonic sensor

Collision prediction and avoidance

No

Yes

Trajectorygeneration

Original path

Danger?Human

approach

New path

6Step 1: Collision Prediction and Avoidance

Use of Ultrasonic Sensors

• Multiple sensors needed

• d < 0.3m → Warning

• d < 0.1m → Emergence Stop

<Collision avoidance using ultrasonic sensors>

• Occlusion Multiple cameras

• Sensitive to lighting conditions

Problems with Vision System

Collision Detection & Reaction @ KU

• Detection: Disturbance observer + JTS(Joint Torque Sensor)

• Reaction: Different reaction modes

Step 2: Active Safety 7

<Collision detection with styrofoam> <Collision detection & reaction with chest>

Principle of collision detection

• Human-robot collision

External force applied to a robot

External torque generated at each joint

8

Normal operation

Collision Detection using Disturbance Observer (DOB)

Human-robot collision

Collision can be detected by monitoring external torque.

)(),()( qgqqqCqqMj ++= τ )(),()( qgqqqCqqMextj ++=− ττ

* τj : joint torque, τext : external torque

External torque estimation

• External torque:

9Collision Detection using DOB

External torque estimation

for collision detection

)}(),()({ qgqqqCqqMjext ++−= ττ

<Joint torque sensor (JTS)> <Joint module>

<Motor current & Friction model>

- Measurement of acceleration

Use of additional sensors

Impractical solution

- Computation of acceleration Numerical differentiation of

encoder signal

Noise due to differentiation

- Sensor based solution

- Sensorless solution

Disturbance observer (DOB)• Basic disturbance observer

Collision Detection using DOB 10

)(ˆ sD

++

−= )(

)()()(

)(1)(1

)()()()(ˆ sD

sGsGsN

sGsU

sGsGsQsD

nnn

)()()(ˆ sDsQsD ≈

Human-Robot Collision Detection

Robust control

Fault Detection & Isolation

Adaptive control

Applications

(if G(s)/Gn(s) ≈ 1, N(s) ≈ 0 )

• Disturbance observer

External Torque Estimator

• External torque estimator based on disturbance observer

11

)(ˆ sextτ

)(sq

)()()(ˆ ssQs extext ττ = )()(ˆ sKs

Ks extext ττ+

=

Collision Detection using DOB

Input Joint torque

• External torque estimate:

System (robot arm joint)

Output Joint velocity

Disturbance External torque

External torque estimator

(Q(s): Low pass filter))()()(ˆ sDsQsD =

Collision detection based on external torque• External torque estimate in time domain

• Generalized momentum: (De Luca, 2003)

12Collision Detection using DOB

<Example of typical case><Collision detection algorithm>

)()(ˆ sKs

Ks extext ττ+

= −++−= dldlqgqqqCqqMK extjext )](ˆ)}(),()({[ˆ τττ

])ˆ)(),(([ˆ pdtqgqqqCK eT

jext −−−+= τττ

External torque estimation without the acceleration information

qqMp )(=

extτ̂?|ˆ| thext ττ ≥

ext

τ̂

13

Demonstrations

Collision DetectionCollision Detection

7 DOF manipulator

Specifications

Weight 15 kg TCP speed 1 m/s

Payload 7 kg Acc. 5 m/s2

Reach 780 mm DOFs 7

RTOS TwinCAT Control period 1 ms

Safe Joint Mechanism (SJM)• Passive joint mechanism consisting of

springs and cam-cam follower mechanisms

• Nonlinear spring system

• High stiffness for positioning accuracy

• Low stiffness for collision safety

• Small & Lightweight

• Automatic return to home position

Operation of SJM• Normal operation

stiff arm accurate positioning

• Emergency (large impact)

soft arm shock absorption

Step 3: Passive SafetyStep 3: Passive Safety

0

10

20

30

40

50

0 10 20 30Displacement (mm)

40

Working region

Safe region

Unsafe region60

High stiffness spring

Inaccurate positioning

Dangerous

Low stiffness spring

Certain collision force

14

Static collision

Passive Safety: Demo 15

<Industrial robot with SJMs>

<Balloon & can> <Shoulder collision>

Advanced Collision Detection

16

1. Sensorless Collision Detection

2. Collision Dection Index (CDI)

• Frequency-based CDI

• Projection-based CDI

Drawbacks of Sensor-based Collision Detection

• Costly solution due to the use of sensors

• Not applicable to industrial manipulators

Need for collision detection without the use of extra sensors

Sensorless Collision Detection

• Estimation of joint torques using the motor current and friction model

17Sensorless Collision DetectionSensorless Collision Detection

Estimation of joint torques without sensors

SensorlessSensorless

<Joint torque sensor> <Motor current> <Friction model>

Estimation of joint torque

18

Power transmissionPower transmission

Friction torque

,im ατ = fmj n τττ −=

τm : Motor torque

α : Torque constant

i : Motor input current

n : Speed reduction ratio

( ))(),()( qgqqqCqqMinfext ++−=+ αττ

Sensorless Collision DetectionSensorless Collision Detection

Friction torque model

≥+≠<=<

=εττεττεττ

τ||if),()sgn(

0and||if),sgn(0and||if),sgn(c

qqqqqqq

vc

dhs

dh

f

Identification of

unknown parameters

IROS 2015, S.D. Lee, M.C. Kim, J.B. Song “Sensorless Collision Detection for Safe Human-Robot Collaboration”

Friction torque identification using least-squares technique

Friction torque observerFriction torque observer

Friction modelFriction model Identification Identification == ̅ ′ =

Data setData setRegressor

LS technique

Data

acquisition

Analysis on friction torqueAnalysis on friction torque

)(ˆ sr

)(sq

Estimation of joint torque

Sensorless Collision DetectionSensorless Collision Detection 19

≥+≠<=<

=εττεττεττ

τ||if),()sgn(

0and||if),sgn(0and||if),sgn(

qqqqqqq

vc

dhs

dhc

f

20

Demonstrations

7 DOF robot arm

Specifications



Reach 780 mm DOFs 7

RTOS TwinCAT Control period 1 ms

Collision detection without the use

of any extra sensors

Human-robot collisionHuman-robot collision

Sensorless Collision DetectionSensorless Collision Detection

Demonstrations

21Sensorless Collision DetectionSensorless Collision Detection

6 DOF industrial manipulator

Specifications



Reach 1044 mm DOFs 6

5 DOF collaborative robot arm

Specifications

Weight 125kg TCP speed 1.15 m/s


Reach 2105 mm DOFs 5

22

Human-robot cooperation

Collision Detection for Human-Robot CollaborationCollision Detection for Human-Robot Collaboration

Contact task

Unexpected collision

SAFE

DAN

GER

τext

τext

Need for New Collision Detection algorithm

Motivation

Handling of payload Physical interaction

Various tasks of collaborative robots

Generation of external torque collision ?

23

Torq

ue (N

m)

Frequency-based Approach

• Rate of change of external force: Frequency-based Collision Detection Index

- Safe Intended Contact : Relatively slow rate of change

- Dangerous Unexpected Collision : relatively fast rate of change

Need for an observer that detects only the fast-changing external torque

Add a high-pass filter to the conventional collision detector

Frequency-based Collision Detection IndexFrequency-based Collision Detection Index

Collision detection of unexpected collision

• Threshold: ±0.5 Nm

• Intended contact

- Maximum Residual: 0.2 Nm < threshold

• Unexpected collision

- Maximum Residual: 2.2 Nm > threshold

24

Intended contact Collision

Frequency-based Collision Detection IndexFrequency-based Collision Detection Index

25Frequency-based Collision Detection IndexFrequency-based Collision Detection Index

Limitations of Frequency-based Approach

• No guarantee that intended contact force is always low frequency

• No guarantee that unexpected collision force is always high frequency

Examples: Collisions in low velocity, clamping

• No clear frequency threshold to distinguish collision from external torque

Box assemblyBox assembly Measured contact forceMeasured contact force

FxFyFz

Need for more accurate but practical solution

26

Cases Source of τext Collision

(EE or Body)Applications

w/o collision w/ collision

Fce

Fcb1.

none

case 1-1

Position control (painting, welding)

τce case 1-2

τcb

Fi

Fp Fg

PayloadFce

Fcb2.

τp

case 2-1

Position control with payload

(pick-and-place, material handling)

τp + τce case 2-2

τp + τcb

Fe

Fce

Fcb3.

τe

case 3-1

Force control (grinding, hand

guiding),

τe +τce case 3-2

τe +τcb

Projection-based Collision Detection Index Projection-based Collision Detection Index

Subspace Projection based Approach

• Types of tasks for human-robot collaboration

27

Cases Collision detection index Detectable collision

Available arms

Fce

Fcb1.

τext Any robot arms

Fi

Fp Fg

PayloadFce

Fcb2.

extpp JJI τ)( +−

6~7 DOF robot arms

Fe

Fce

Fcb3.

extTT JJI τ))(( +−

7 DOF robot arms


Subspace Projection based approach

• Collision detection strategy for human-robot collaboration

28

extFextF• If Fext = Fp

CDI : zero vector

- Fp = (0, 1, 1) in the yz plane ( only payload)

- Fext = (1, 1, 1) in the xyz space ( col. Included)

- Projection of Fext into the x axis (orthogonal to

the yz plane)

- Collision force Fc = (1, 0, 0)

pcext FFF +=

• If Fext ≠ Fp

CDI : not zero vector


Projection based Approach

• Main idea of proposed collision detection method

Example of subspace projection (in Cartesian space)Example of subspace projection (in Cartesian space)

29

CDI : decoupled with τp & sensitive to τc

mnSp −=⊥ )dim(

cpp

pppcpp

extpp

JJI

JJIJJI

JJICDI

τ

ττ

τ

)(

)()(

)(

+

++

+

−≈

−+−≈

−=


Collision Detection for Handling a Payload (Case 2)

• Available for 6 – 7 DOF robot arms pcext τττ +=

30Projection-based Collision Detection Index Projection-based Collision Detection Index

Experimental results

• Collision detection for various payloads (w/o payload 1kg 2kg)

CDI

(Nm

)

-20-10

01020

0 2 4 6 8 10 121kg 2kg

The developed CDI can detect a collision for unknown payloads.

31

CDI : decoupled with τe & sensitive to τc

cTT

eTT

cTT

extTT

JJI

JJIJJI

JJICDI

τ

ττ

τ

))((

))(())((

))((

+

++

+

−≈

−+−≈

−=

Collision Detection for Human-Robot CollaborationCollision Detection for Human-Robot Collaboration

Collision detection for Contact Task (Case 3)

• Physical interaction based on force applied to its end-effector

• External force on the end-effector intended interaction force

• External force on the body unexpected collision force

ecext τττ +=

Experimental results

• Collision detection during hybrid force/position control

•) Hybrid force/position control

32Collision Detection for Human-Robot CollaborationCollision Detection for Human-Robot Collaboration

- Intended interaction force for

impedance control in the x direction

- Collision between human and

manipulator< Written letters: IRL >

Collision detection

Human-robot collaboration in car assembly line

Scenario for human-robot collaboration

Case 1: Approaching Case 3: Physical interaction Case 2: Handling of payload

33Projection-based Collision Detection Index Projection-based Collision Detection Index

Human-robot collaboration

(Pick and place) (Hand guiding)(Position control)

34Collision Detection for Human-Robot CollaborationCollision Detection for Human-Robot Collaboration

Collision detection strategy

Case 1: Approaching Case 3: Physical interaction Case 2: Handling of payload

Normal operation:

Collision:

0=extτ

cext ττ =

Normal operation:

Collision:

pcext τττ +=

Normal operation:

Collision:

ecext τττ +=

pext ττ = eext ττ =

CDI CDI CDIextτ extpp JJI τ)( +− ext

TT JJI τ))(( +−

Detectable collision Detectable collision Detectable collision

Collision Analysis & Simulation

35

Safety criteria for safety evaluation

Various Safety CriteriaVarious Safety Criteria 36

• ISO 10218-1

- Collaborative operation with humans

- vTCP<0.25m/s, FTCP<150N, Pmax<80W

• Human pain tolerance [Yamada, 1996]

- Static collision (v<0.6m/s)

- F<50N

• Head Injury Criterion (HIC)

- Automobile crash test

- HIC<650 prob(AIS≥3)<0.05

- Used to be the most popular index

• Too restrictive criteria Limitation of performance

• Too generous for a robot arm- Low collision speed- HIC saturation with increasing mass No robots become dangerous at 2m/s. [Haddadin, 2008]

Safety evaluation of human-robot collision

Real impact test

Simulation S/W

Collision analysis

• Real impact test & evaluation• Using a crash-test dummy

• Features

+ Most realistic data available

- Considerable cost and time for tests

- Need to construct a robot

Safety EvaluationSafety Evaluation 37

DLR – Haddadin


Real impact test

Simulation S/W

Collision analysis

• Collision simulation• Using simulation S/W

• Features

+ Relatively reliable results

+ No need to construct a robot

- Expensive S/W

38Safety EvaluationSafety Evaluation

MADYMO S/W


Real impact test

Simulation S/W

Collision analysis

39Safety EvaluationSafety Evaluation

Bicchi ‘04

Morita ‘00

• Collision analysis and evaluation• Analytic method

• Features

+ No need to construct a robot

+ Low cost and easy application

- Less reliable data

Injury tolerance of body parts

40Various Safety CriteriaVarious Safety Criteria

Cranial bone [SAEJ885, 1980] Fracture tolerance

Frontal 4.0 kNTemporal 3.12 kNOccipital 6.41 kN

Facial bone[Nahum, 1972 & 1976] Fracture tolerance

Mandible (C) 1.89 kNMandible (L) 0.82 kNZygomatic 0.85 kkN

Maxilla 0.62 kNNasal 0.342 kN

Chest Injury toleranceCompression criterion

[Lau, 1983] 22mm

Viscous criterion[Lau, 1986] 0.5m/s

Abdominal[Miller, 1989] Injury tolerance

Liver 310kPaLower abdomen 3.76kN

Neck (indirect impact) Injury tolerance

Shear[Mertz, 1993]

3.1kN @ 0msec1.5kN @ 25-35msec1.1kN @ 45msec

Tension[Mertz, 1993]

3.3kN @ 0msec2.9kN @ 35msec1.1kN @ 60msec

Compression[Mertz, 1993]

4kN @ 0msec1.1kN @ 30msec

Extension[Mertz, 1967] 57Nm

Flexion[Mertz, 1967] 87.8Nm

Bending angle[Gadd, 1971]

Extension: 80°Lateral: 60°

Neck (direct impact) Injury toleranceThyroid and cricoid

[Melvin, 1973] 0.337 kN

Lower extremities[Devore, 1999] Injury tolerance

Femur 3.8kNTibia 5.4kN

Upper extremities[Begeman, 1999] Injury tolerance

Humerus 1.96kNElbow 1.75kN

Forearm 1.37kN- KR6@2m/s No injury[Haddadin, ‘09]

• Neck injury

• Thyroid and cricoid cartilages

- Upper end of airway passage

- Fracture force : 337 N

Obstruction of airflow

• Head injury

• Nasal bone

- Protrusion of head

- Weakest bone of head

- Fracture force : 342 N

Comminuted fracture

Safety criteria (Collision force)

Safety criterion for service robots (blunt impact)

Safety CriteriaSafety Criteria 41

Hybrid III

HuRoCol (Human-Robot Collision Analysis)

Parameters of collision model

• Human (Hybrid III 50th percentile male)

• Weight: 4.5kg(head), 1.5kg(neck), 71kg(body)

• Neck stiffness: 0.44Nm/deg

• Robot arm

42HuRoCol: Model ParametersHuRoCol: Model Parameters

Robot arm model

Human model

Head-Neck Model (3 DOF)

• Head: Revolute joint (OC), Neck stiffness

• Neck: Revolute joint (C7), Neck stiffness

• Body: Prismatic joint

43HuRoCol: Collision modelHuRoCol: Collision model

Collision model

Chest Model• Lobdell [17]): 2 DOF

• Lumped-mass model of anteroposterior thoracic impact

• To obtain uncoupled inertia matrix

Dummy mass is added between kve and cve

- kr : rib cage and directly coupled viscera

- cb : air in lungs and blood in the vessels

- kve and cve : viscoelastic tissue such as thoracic muscle tissue

44HuRoCol: Collision modelHuRoCol: Collision model

x5

xy

x6

kr

cb

x7

kve cve

Various collision cases

45HuRoCol : Collision modelHuRoCol : Collision model

Unconstrained human

Partially constrained human

Impact to head Impact to neck Collision model

xz

Wall

xz

Wall

Impact to head Impact to neck

Constrained human

Impact to head Impact to neck

Solution:

• Matlab/Simulink

- 4th and 5th-order Runge-Kutta method

46HuRoCo : Solution MethodHuRoCo : Solution Method

( ))()()(),()( 1 qDqGqKqqCFqMq −−−−= −

Robotica 2015, J.J. Park, J.B. Song, S. Haddadin, “Collision analysis and safety evaluation using a collision model for a frontal robot-human impact”

Collision with unconstrained human- Impact to the neck is more dangerous than impact to the head.

( airway obstruction)

47

HuRoCol : Analysis ResultsHuRoCol : Analysis Results

Impact to head

Impact to neck

xz

Robot link

O.C.

C7

Body 0

100

200

300

400

500

0.4 0.6 0.8 1.0 1.2Time (s)

Col

lisio

n fo

rce

(N) 407 N

342 N

Nasal bone fracture

Ang

le (d

eg)

Disp

lace

men

t (cm

)

Time (s)

Time (s)

O.C.+C7

Collision

C7O.C.

020406080

0 0.5 1.0 1.5 2.0

00.20.40.6

0 0.5 1.0 1.5 2.0

CollisionBody

Collision with partially constrained human- Impact to the neck is more dangerous than impact to the head.

48


Impact to head

Impact to neck

xz

Wall

xz

Wall

Col

lisio

n fo

rce

(N)

Ang

le (d

eg)

Col

lisio

n fo

rce

(N)

Ang

le (d

eg)

49


Collision with constrained human- Impact to the neck is more dangerous than impact to the head.

Impact to head

Impact to neck

xz

Wall

0.4 0.6 0.8 1.0 1.2Time (s)

Nasal bone fracture

342 N

425 N

0

100

200

300

400

500

Design of safe robot arm- Design of the robot arm can be modified according to analysis results.- Mass (inertia), length, velocity…

HuRoCol : Design of Safe Robot ArmHuRoCol : Design of Safe Robot Arm

Impact to neck

Inertia of robot link (1.5m/s)

Velocity of robot(2.5kg)

0.4 0.6 0.8 1.0 1.2Time (s)

344 N

423 N

0

100

200

300

400

500Thyroid & cricoid fracture (337N)

1.5 m/s1.2 m/s

1.0 m/s

289 N

0.4 0.6 0.8 1.0 1.2Time (s)

337 N

423 N

0

100

200

300

400

500

2.5 kg

2.3 kg

2.1 kg210 N

Thyroid & cricoid fracture (337N)

• The robot arm with SJM can provide much higher safety. • Design of the robot arm can be modified according to analysis results.

HuRoCol : Design of Safe Robot ArmHuRoCol : Design of Safe Robot Arm

Impact to head

Impact to neck

xz

Robot link

O.C.

C7

Body

Analysis versus Dummy crash-test• KUKA KR6 (inertia: 67 kg)• Unconstrained human Close agreement with dummy crash-test data

Impact to head

HuRoCol : Verification 1HuRoCol : Verification 1

xz

Robot link

O.C.

C7

Body

Haddadin, ICRA ‘09

Col

lisio

n fo

rce

(N)

Impact to head

HuRoCol : Verification 2HuRoCol : Verification 2

xz

Robot link

O.C.

C7

Body

Haddadin, ICRA ‘09

Analysis versus Dummy crash-test• KUKA KR500 (Refl. inertia: 1870 kg)• Unconstrained human Close agreement with dummy crash-test data

• Safe Joint Mechanism: passive approach, infinite bandwidth

• Frequency-based Collision Detection

Intended contact: low frequency & Collision: high frequency

• Projection-based Collision Detection Index

Any collision regardless of frequency and magnitude of collision

• Safety Criterion: fracture force of thyroid & cricoid cartilages for neck injury

- The most appropriate safety indicator for a service robot

• Proposed collision model and analysis

• Accurate model

- More reliable analysis results for human-robot collisions

• Evaluation in the robot design phase

- Can save time and cost associated with collision tests

54Summary

Thank you!

Q & A

collision safety for physical human-robot collaboration

Documents