learning and control with movement primitives in multiple ...€¦ · coordinate system 2: this is...

Sylvain Calinon Robot Learning & Interaction Group Idiap Research Institute Martigny, Switzerland

Learning and control with movement primitives in multiple coordinate systems

RIBA healthcare robot, RIKEN i-limb prosthetics, TouchBionics

Humanoid diver, Stanford Robonaut, NASA

Learning from demonstration as an intuitive interface to transfer skills to robots

Statistical learning dynamical systems +

We look for a modular representation of movements and skills that can learn from wide-ranging data, that can adapt to new situations in a fast manner, that can exploit the robot embodiment, and that is robust to perturbations.

System plant

Model predictive control (MPC) Do it smoothly! Track path!

How to solve this cost function? • Pontryagin’s maximum principle Riccati equation

• Dynamic programming • Linear algebra with stacked

vectors [Tanwani and Calinon, IEEE RA-L 1(1), 2016]

state variable (position+velocity) control command (acceleration) tracking weight matrix control weight matrix

System plant

Model predictive control (MPC) Do it smoothly! Track path!

[Tanwani and Calinon, IEEE RA-L 1(1), 2016]

Model predictive control (MPC)

[Tanwani and Calinon, IEEE RA-L 1(1), 2016]

Sharing of synergies with:

Model predictive control (MPC)

[Tanwani and Calinon, IEEE RA-L 1(1), 2016]

Minimal intervention Safe/compliant robot controller

Transition and state duration

Stepwise sequence with:

Transfer of controllers from demonstration

Demonstration Reproduction

[Calinon, Intelligent Service Robotics 9(1), 2016]

Hold

ing

a cu

p ho

rizon

tally

Bi

man

ual

coor

dina

tion

in a new situation…

Extension to multiple coordinate systems

Coordinate system 1: This is where I expect

data to be located!

Coordinate system 2: This is where I expect

data to be located!

[Calinon, Bruno and Caldwell, ICRA’2014][Calinon, HFR’2016]

Product of linearly transformed Gaussians

Track path in coordinate system j

MPC considering multiple coordinate systems

Do it smoothly!

1

2

2 2

2

1

2

Set of demonstrations Reproduction in new situation

New position and orientation of coordinate

systems 1 and 2

Two candidate coordinate systems (P=2)

[Calinon, Bruno and Caldwell, ICRA’2014][Calinon, HFR’2016]

MPC considering multiple coordinate systems

1

2

In many robotics problems, the parameters describing the task or situation can be recast as some form of coordinate systems or locally linear transformations

[Calinon, Bruno and Caldwell, ICRA’2014][Calinon, HFR’2016]

Track path in coordinate system j Do it smoothly!

MPC considering multiple coordinate systems

Learning of a controller that adapts to new situations while regulating its gains according to the precision and coordination required by the task

In many robotics problems, the parameters describing the task or situation can be recast as some form of coordinate systems or locally linear transformations

Track path in coordinate system j Do it smoothly!

[Calinon, Bruno and Caldwell, ICRA’2014][Calinon, HFR’2016]

Adaptation of movements to different shapes

[Calinon, Alizadeh and Caldwell, IROS’2013]

Candidate coordinate

system

[Silverio et al., IROS’2015]

[Rozo et al., IROS’2015]

[Rozo et al., IEEE T-RO 32(3), 2016]

Bimanual coordination and co-manipulation

Dr Leonel Rozo João Silvério

Joint space constraints & tasks prioritization

Task space (operational space)

Joint space (configuration space)

Task parameters as Jacobian operators

Task parameters as null space projection operators

[Silverio, Calinon, Rozo and Caldwell (submitted)] [Calinon, ISRR’2015]

Demonstration on COMAN

Left

han

d

prio

rity

Righ

t han

d

prio

rity

Reproduction on WALKMAN

Learning from demonstration can be applied to various forms of robots and applications

(2015-2018)

(2015-2018)

(2012-2015)

Stiff Compliant

[Bruno, Calinon, Malekzadeh and Caldwell, ICIRA, LNCS 9246, 2015]

s=0 s=1

Continuous arm index s MPC for continuum robots

Flexible robots in minimal invasive surgery

Collaboration letting the surgeon control the tip, while the robot exploits the remaining degrees of freedom that do not interfere with the control of the tip

Insertion Retraction

[Bruno, Calinon and Caldwell, Autonomous Robots, 2016]

Robotic dressing assistance

Dressing assistance for: - Putting on a coat - Putting on shoes Requires to extend movement primitives to reaction, force and impedance primitives

Telemanipulation with underwater robot

Onshore Offshore

Same model (TP-HSMM) used for classification on the one side, and synthesis on the other side, with local adaptation to the task parameters

Being skillful =

exploiting variability and

correlation

Needs correction

Does not need correction

Recognition & synthesis of motion primitives

Semi-autonomous teleoperation as

a form of human-robot collaboration

[Tanwani and Calinon, IEEE RA-L 1(1), 2016]

Telemanipulation with underwater robot

• Model predictive control (MPC) can be smoothly combined with learning from demonstration (LfD) for both planning and control problems

• The parameters of the cost function in MPC can be learned from demonstration, with weights as full precision matrices, instead of predefining those manually as diagonal matrices

• LfD can be extended to the consideration of multiple coordinate systems and to the learning of controllers Minimal intervention strategy that is safer for the users

• Extending LfD to collaborative skills and teleoperation provides new perspectives in shared control by exploiting the learned task variations and task synergies to cope with perturbations

Conclusion

Contact: sylvain.calinon@idiap.ch http://calinon.ch

Source codes: http://www.idiap.ch/software/pbdlib/

Ajay Tanwani

Emmanuel Pignat Noémie Jaquier

Dr Ioannis Havoutis

Photo: Basilio Noris

Collaborators at Idiap: