questions for a cooperative robot: what, who, where, when ...contributions / bricks for the...
TRANSCRIPT
![Page 2: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/2.jpg)
What is sure
The autonomous robots, when they will be really used, will very seldom be alone
![Page 3: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/3.jpg)
A prediction: “20xy – in a mall”
A team of cleaning robots
Several semi-autonomous
wheelchairs
Several assistive or companion robots
A group of service robots
![Page 4: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/4.jpg)
Two types of interactions
domain and task dependent interaction that will be necessary within a team of robots e.g. perform cooperatively a cleaning task, or a surveillance mission
and also
generic interaction abilities, when a robot “encounters” a robot that is dedicated to another activity e.g. efficient avoidance and space sharing, between a cleaning robot and a surveillance robot
![Page 5: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/5.jpg)
5
A shift in the perspective: From a single robot to human-robot teams
![Page 6: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/6.jpg)
a dream and a challenge
The robot was alone
Autonomous Robotics
a hero
Very capable
Very intelligent
You give it a goal/task
It does the job
… end it is ready for a new adventure
![Page 7: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/7.jpg)
7
Human Robotic teamwork
![Page 8: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/8.jpg)
Participating labs:
LAAS, ToulouseEPFL, LausanneIPA, StuttgartKTH, StockholmU. KarlsruheU. BielefeldU. HertfordshireU. AmsterdamVU. BrusselsGPS, Stuttgart
Funded by the
EU FP6 -IST- FET
“Beyond Robotics” Program
The robot companion / assistant
![Page 9: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/9.jpg)
9
Human-Robot Exploration
Interactive
Dialogue intensive
Robots need to keep up with humans
EVA Assistant Tasks
Robots carrying equipment (mules)
Robots in advance of humans (scouts)
Robots maintaining services (data connectivity)
Robonaut: cooperative deployment of equipment
![Page 10: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/10.jpg)
10
Toward the collaborative robot
Task oriented Interaction
Task involves another partner
Questions of task decomposition, task allocation, task achievement
Reporting and acting on a task while it is executed
Reasoning abilities and models of the robot of the others
Collaborative schemes, Communication, Social behaviour, space and activity sharing
Questions for a cooperative robot: what, who, where, when, how?
![Page 11: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/11.jpg)
Contributions / Bricks for the collaborative abilities of a robot
Plan-merging protocol (when and how)
Task allocation protocol (who)
Opportunistic behaviour enhancement (who, what and how)
An architecture for multi-robot cooperation (all..)
Cooperation through assistance (who and how)
Multi-robot manipulation: the IKEA problem (how and where)
Adaptive Formation of UAVs in a hostile environment (who and how)
SHARY: collaborative human-robot task achievement (what, how and when.)
Proactive and safe motion in a dynamic environment (how)
Navigation in presence of humans ... user studies (how)
Guiding a person (how and when)
Manipulation in presence/interaction with humans ..(where and how)
Perspective taking (where)
HATP: a Human Aware Task Planner (what, who and how)
![Page 12: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/12.jpg)
List of co-authors
Rachid Alami, Samir Alili, Luis Aguilar, Gérard Bailly, Silvia Botelho, Ludovic Brethes, H. Bullata, Stéphane Cambon, Hung Cao, Maxime Cottret, Raja Chatila, Aurélie Clodic, Patrick Danes, Kerstin Dautenhan, Xavier Dollat, Frédéric Elisei, Isabelle Ferrane, Sarah Fleury, Paul Gaborit, Jérémie Gancet, Malik Ghallab, Fabien Gravot, Martin Haegele, Gautier Hattenberger, Matthieu Herrb, Guillaume Infantes, Felix Ingrand, Sylvain Joyeux, Maher Khatib, Ken Koay, Madhava Krishna, Jens Kubacki, Simon Lacroix, Jean-Paul Laumond, Christian Lemaire, Thomas Lemaire, Frédéric Lerasle, Efrain Lopez Damian, Jerome Manhes, Philippe Marcoul, Luis Marin, Paulo Menezes, Vincent Montreuil, Christopher Nehaniv, Christopher Parlitz, Samir Qutub, Frédéric Robert, Daniel Sidobre, Thierry Siméon, Akin Sisbot, Thierry Vidal, Mick Walters, Britta Wrede, Sara Woods.
![Page 13: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/13.jpg)
1 - Plan-merging protocol (when and how)
In multi-robot context
How to deal with resource conflicts in a distributed way in a dynamic environment ?
![Page 14: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/14.jpg)
task task task
The central station allocates (incrementally) individual tasks to robots
![Page 15: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/15.jpg)
Task
Multi-Robot
Execution Context
New Coordinated plan
Plan-Merging: Each robot adapts incrementally its plan to comply with the
multi-robot context
![Page 16: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/16.jpg)
New
coordination
plan needed
Plan Merging Protocol
![Page 17: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/17.jpg)
New
coordination
plan needed
Critical
section
obtained
Plan Merging Protocol
![Page 18: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/18.jpg)
Critical
section
obtained
Collect plans
Construct
global plan
Insert plan
New
coordination
plan needed
Plan Merging Protocol
![Page 19: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/19.jpg)
Collect plans
New
coordination
plan needed
Critical
section
obtained
Construct
global plan
Insert plan
Plan Merging Protocol
![Page 20: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/20.jpg)
Collect plans
Failure Success
New
coordination
plan
New
coordination
plan needed
Critical
section
obtained
Construct
global plan
Insert plan
Plan Merging Protocol
![Page 21: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/21.jpg)
Treillis
No Yes
PMO
deferred
New
coordination
plan needed
Critical
section
obtained
New
coordination
plan
PMO
failure
Deadlock
treatement
Wait PMO
events
Plan Merging Protocol
![Page 22: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/22.jpg)
Four trajectories in Space-Time
Plan-merging at a crossing
![Page 23: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/23.jpg)
Intricate motion coordination
Thesis of F. Robert and Samer Qutub (1996)
![Page 24: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/24.jpg)
Multi-robot hétérogènes
Paradigme
d’Insertion
de Plan Etendu
3D – Robots hétérogènes
Nombreux ddl - Priorités
(F. Gravot)
![Page 25: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/25.jpg)
Variants
Insert
Insert_Modify
Insert_Replan
Priorities
Discrete (Crossing paths)
Continuous (Convoy mode)
![Page 26: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/26.jpg)
2 - Task allocation protocol (who)
How to distribute a set of partially ordered task to a set of autonomous robots ?
![Page 27: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/27.jpg)
Task allocation
A mission is a set of partially ordered tasks, defined as set of goals to be achieved.
Tasks should/are allocated to the robots based on their capabilities and on their execution context.
Not necessarily distributed. However, task allocation is essentially based on proper or local information.
Negotiation process
![Page 28: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/28.jpg)
Idle
Best
candidate
Candidate
Eval
Cost
Planning
Make an offer
for a task
allocation
Loses the role
of best candidate
Task Allocation process
Mission
Tasks ready
for planning
Execution
event
Bothelo 1999
![Page 29: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/29.jpg)
Task Planning based Task allocation
Tasks may be allocated (and re-allocated when necessary) incrementally through a negotiation process between robot candidates.
This negotiation is combined with a task planning and cost estimation activity which allows each robot to decide its future actions taking into account:
its current context and task,
its own capacities
as well as the capacities of the other robots.
![Page 30: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/30.jpg)
Distributed Incremental Task Allocation
[Lemaire-2004]
Distributed task allocation based on
auction
Set of places to visit with temporal and
communication constraints
Incremental re-evaluation based on the
robots « load »
![Page 31: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/31.jpg)
3 - Opportunistic behaviour enhancement (who, what and how)
In a multi-robot context
Observing a set of incrementally refined plans and detecting potential sources of inefficiency
![Page 32: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/32.jpg)
Task achievement in a multi-robot context
The tasks cannot be directly « executed » but require further refinement
Each robot synthesizes its own detailed plan.
We identify two classes of problems related to the distributed nature of the system:
1. coordination to avoid and/or solve conflicts and
2. cooperation to enhance the efficiency of the system.
![Page 33: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/33.jpg)
Dili
Aller
à P1
Ouvrir
P1
Fermer
P1
Aller
à S2 Aller
à zone-libre
H2Bis
Aller
à P1
Ouvrir
P1
Fermer
P1
Aller
à S2 Aller
à zone-libre
Début
Fin
Ouvrir P1
Fermer P1
Dili
Début
Fin
Ouvrir P1
Fermer P1
H2Bis
Début
Fin
Ouvrir P1
Fermer P1
Emploi fusionné
Utilisation Coopérative d ’une étape
Botelho, DARS 2000
![Page 34: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/34.jpg)
4 - An architecture for multi-robot cooperation
![Page 35: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/35.jpg)
Multi-robot control cooperation
What decisional levels ?
What models, protocols and algorithms at each level ?
What consequences on the robots and on the tasks to perform ?
![Page 36: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/36.jpg)
An architecture for multi-robot cooperation
![Page 37: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/37.jpg)
Evolution of a robot plan
Coordinated plan (partially freezed)
Actions not yet validated in the multi-robot context
Signals to other robots
Signals from other robots
![Page 38: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/38.jpg)
Operations on Plans
Updating the plan when an event occurs: start actions, signal events to other robots:
Incremental planning for Robot i:
Collecting Coordinated Plans
Plan Merging Operation
Uij
k
j
k
iCPGP =
CPik+1
= UPDATE(e , CPik )
),PLAN(11 ++
=k
i
k
i
k
iGCPIP
),PMO( 11 ++=
k
i
k
i
k
iIPGPCP
![Page 39: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/39.jpg)
Adaptative move from completely distributed to centralized decision
![Page 40: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/40.jpg)
An example
Mission Decomposition
Task Allocation
Tasks
Task
Task Allocation
task
Task Allocation
Task achievement
Task
Mission
T1
T2 T3
T4 T5
Task achievement Task achievement
![Page 41: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/41.jpg)
Distributed Decisional activity display
future past present
Redundant
actions wait
opportunism
![Page 42: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/42.jpg)
42
La simulation
![Page 43: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/43.jpg)
Other examples R0 and R2 are slower. They
decide not to help R1 to
clean ROOM1
The cooperative
mechanisms are inhibited.
The robots only
COORDINATE.
![Page 44: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/44.jpg)
Workload for each robot
![Page 45: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/45.jpg)
The COMETS fleet
Marvin (TUB) Karma (LAAS) Heliv (Helivision)
COMETS: Real time coordination and control of multiple
heterogeneous Unmanned Aerial Vehicles (UAV)
- Forest fire alarm detection
- Alarm confirmation / fire monitoring
- Mapping
Environment surveillance:
(Work of Gancet, Lacroix)
![Page 46: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/46.jpg)
5 - Cooperation through assistance
![Page 47: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/47.jpg)
A plan management component
a plan model
generic enough to represent planner results
expressive enough for intricate execution & incremental plan refinement
a plan management
component
execution and adaptation of plans
Plan
generation
Decision
control
Plan manager
Executive
Robot
Functional layer
Thesis of Sylvain Joyeux (2006)
![Page 48: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/48.jpg)
A UAV for traversability mapping of a terrestrial robot
Proposes the
plan change
RemoteMapping
Bmap::TravMapping
RegionsOfInterest
Accepts and modifies the
transaction
Bmap::TravMappingPOM::Localization
planned_by
realized_by
P3d::TrackPath
Nav::Path
Nav::PathPlanning
Nav::MoveTo(to: C)
RemoteMapping
Bmap::TravMapping
The UAV can
start planning
RemoteMapping
RemoteMapping
Bmap::TravMapping
UAV Rover
![Page 49: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/49.jpg)
Collaborative task achievement involving close interaction
![Page 50: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/50.jpg)
6 - Adaptive Formation of UAVs in a hostile environment (who and how)
Task “internal” to the team:
- Choose an adequate configuration (position and role) based on the current context (threats, path) and to the UAVs abilities
- Plan and coordinate spatial reconfiguration of the formation
Thesis of Gautier Hattenberger 2007
![Page 51: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/51.jpg)
![Page 52: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/52.jpg)
7 - Asymov: A planner for Cooperative assembly
The IKEA problem
Plan:
- “transit” and “transfer” motions
- Individual (synchronized) or cooperative motion
- Choose graps and placements (how and where)
- Based on a general formulation of the manipulation
- Formal link between geometric and symbolic reasoning
(Thesis of Gravot and Cambon – based on work of Simeon, Laumond, Cortes )
![Page 53: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/53.jpg)
8 - A task-oriented architecture for an interactive robot
the IAA (InterAction Agent) represents the human
state, abilities and preferences.
Task-Oriented
How to perform a task, in presence or in interaction with humans, in the best possible way
Efficiency, Safety, Acceptability, Intentionality
Planning and On-Line Deliberation
Anticipation, Reasoning
![Page 54: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/54.jpg)
HRI Robot Decisional Abilities
Planners and Interaction Schemes that will allow the robot:
to elaborate plans
and to perform its tasks
While taking into account explicitly the constraints imposed by:
the presence of humans,
their needs and preferences.
![Page 55: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/55.jpg)
Supervision of H/R task achievement
Robot Searches for interaction when left alone
Establishes a common task
Programming a H/R task involving several perception and interaction modalities
Abandons mission if guided person stops following
Rackham at « Cité de l’Espace »:
![Page 56: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/56.jpg)
Integrative approach for a robot that acts in interaction with humans
Work on Collaborative / Interactive task achievement
based on a study of human-robot interaction
inspired from Joint activity / Teamwork
concretized as a set of robot decisional abilities
is progressively producing a coherent basis for Joint Human-Robot Activity
![Page 57: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/57.jpg)
SHARY Supervisor for Human Aware Robot « Ynteraction »
Continuous planning: Context dependent task refinement
Joint tasks (joint activity / teamwork)
Joint Goal
Task refinement: plans / recipes
Maintains common ground through a set of communication acts that support the interactive task achievement:
deciding who speaks when
establishing facts that must be agreed upon ..
Monitors human performance and commitment
Thesis of Aurelie Clodic (2007)
![Page 58: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/58.jpg)
Handing an object to a person
tvb Where is Thierry ?
Thierry does not take the bottle « Disturbed » attention
![Page 59: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/59.jpg)
Handing a bottle to a person Predictability, Common Ground, Responsiveness
![Page 60: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/60.jpg)
Serve a drink High-level internal structure
60
![Page 61: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/61.jpg)
Collaborative task achievement
61
![Page 62: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/62.jpg)
When to release the object ?
![Page 63: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/63.jpg)
Building a « good » plan
Managing Joint task achievement
Legibility of robot actions and intentions (intentionality)
Acceptability of robot actions
Compliance with “conventions”
Coherent attitudes and behaviours
Constraints on robot plans
![Page 64: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/64.jpg)
Consequences
On robot goals management
On planning:
high-level (symbolic)
and “geometric” level
On task achievement
monitoring and adapting to human commitment
On interplay between Dialog and Decision
![Page 65: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/65.jpg)
9 - A safe (but efficient) path taking into account sensor scope and robot and
(unknown) mobile obstacles (humans) dynamics
A typical path computed by a motion planner
Path as obtained after minimization of trajectory time
Work of M. Krishna
![Page 66: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/66.jpg)
10 - Navigation in presence of humans
Classical Motion Planning methods do not take into account specifically the presence of humans: obstacle free paths, coordination for non-collision or dead-lock avoidance
Need to generate robot motion that is acceptable, legible and compliant with social rules
![Page 67: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/67.jpg)
10 - Navigation in presence of humans
User trials performed at Univesity of Hertfordshire
![Page 68: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/68.jpg)
Real-time cost evaluation:
distance, posture, visibility
Human-friendly navigation
Incremental path adaptation
![Page 69: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/69.jpg)
Serving Dinner
Approaching too much Better behavior
![Page 70: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/70.jpg)
Crossing
Avoiding to loom too close
![Page 71: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/71.jpg)
Teamwork-based guiding behaviour
Human follows “freely” .. robot complies
Human “breaks” the “task” from time to time
(Work of Amit Pandey)
![Page 72: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/72.jpg)
Undesirable Placements /Motions
11 - How to hand an object to a person?
“acceptable” placements
![Page 73: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/73.jpg)
« Double-Grasp » for handing objects
Thesis of Efrain Lopez Damian
![Page 74: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/74.jpg)
Kinematic reachability
Field of sight
Trajectory and Motion dynamics
How to hand an object to a person?
![Page 75: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/75.jpg)
Handing an object to person
The object should be placed in a safe and comfortable position.
3 different HRI properties are defined and represented as 3D cost grids around the human
Reflects the effort to see
a point
Visibility Arm Comfort(right/left)
Proportional to the
distance to human Combination of d.o.f difference
and potential energy
Safety
![Page 76: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/76.jpg)
Handing an object to person 1- Calculating object position
3 grids are combined to form a final grid that merges all these properties.
The cell with minimum cost is chosen to be the place where robot will place the object.
Dist > Vis >AC Vis > Dis > AC AC > Vis > Dis
![Page 77: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/77.jpg)
Human Aware Manipulation Planner (HAMP) Calculating robot path
No human aware motion Human aware motion with 2
motion tasks:
• Follow the object path
• Look to the object
![Page 78: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/78.jpg)
Pick and Give
![Page 79: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/79.jpg)
79
Situation where the human does not give a
« hint » (does not choose where the task has
to be performed) .. The hands are even hidden
Situation where the human chooses where
the task should be performed
Handing an object to person (with perception)
![Page 80: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/80.jpg)
12 - One key robot capability: reasoning about placements and perspectives
Relative Placement and Motion with respect to humans and objects in an environment
Reasoning on the human (and the robot) perception and manipulation abilities
In order to answer a number of questions such as: Can the human see that object ? Can the human see the a given part of the robot ? (perspective)
Can human reach an object (grasp)
Where to place the robot in order to be able to see simultaneously an object, the hand and the face of a human partner (home tour, object handing)
![Page 81: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/81.jpg)
13/10/08
Perspective Placement
Robot (sensor) placement that satisfies: task feasibility, sensor placement for task monitoring (servoing), visibility by the person.
Pointed object not visible from the current Robot configuation
Robot moves to see the pointed object
![Page 82: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/82.jpg)
Choosing an adequate position, posture and sensor placement
(Thesis of Luis Felipe Marin)
Position to speak / to hand over an object
(human sitting, table is obstacle for
placement not for vision or speech
Position to to hand over an object to
a( person standing … avoid to « bother »
the other person.
Validity, Compliance with user preferences, Reachability,
![Page 83: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/83.jpg)
Go-to-look : Different contexts: obstacles
![Page 84: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/84.jpg)
13 - Human Aware Task Planning (HATP)
Designed to produce plans “socially acceptable” for an assistive robot.
Link with geometric issues: perspective taking
providing context for socially acceptable motion
![Page 85: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/85.jpg)
HATP key features
the use of a temporal planning framework with the explicit management of two time-lines (for the human and the robot),
a hierarchical task structure allowing incremental context-based refinement and fully compatible with the BDI approach adopted for the robot supervisor
a plan elaboration and selection algorithm that searches for plans with minimum cost and that satisfy a set of so-called “social rules''.
Thesis of V. Montreuil
![Page 86: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/86.jpg)
HATP Planning process
Social rules allow a specification of social conventions:
Undesired states
Undesired sequences
Avoiding plans intricacy
Minimising human effort
Practical planning approach in order to be able to build a planner that can effectively be used on-line (HTN planning)
![Page 87: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/87.jpg)
HATP example (1) promoting plans with less intricacies
Work of S. Alili
![Page 88: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/88.jpg)
HATP example2: various reactions to contingencie
![Page 89: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/89.jpg)
Conclusion
Collaborative skills involve a number of issues which were not discussed
Sensory-motor abilities
Dialog
Learning
Architectural issues
![Page 90: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/90.jpg)
Conclusion
Toward teams of robots and humans / ambient intelligent systems
Far more elaborate sensory-motor skills
Finer models integrating uncertainty
More flexible architecture
Devising models and algorithms with larger applicability context
Learning how to interact, how persons interact
![Page 91: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/91.jpg)
ADREAM initiative
Ubiquitous Robotics, Ambiant Intelligence
Devices and (micro)systems Micro-systems
Energy
Communication
Development and deployment technologies Embedded systems
Network and protocols
Resilience and Privacy issues
Robotics and decisional systems
![Page 92: Questions for a cooperative robot: what, who, where, when ...Contributions / Bricks for the collaborative abilities of a robot Plan-merging protocol (when and how) Task allocation](https://reader031.vdocuments.us/reader031/viewer/2022013019/5e4132394d038e03072a05a6/html5/thumbnails/92.jpg)
92
Thank you …