how a cooperative behavior can emerge from a robot team antonio d'angelo univ. of udine...

How a Cooperative Behavior can emerge from a Robot Team

Antonio D'Angelo Univ. of Udine

Emanuele Menegatti Univ. of Padua

Enrico Pagello Univ. of Padua

How a Cooperative Behavior can emerge from a Robot Team A. D'Angelo et. al.

Background and Motivations

behavior-based robotics

group of interacting autonomous robots

biologically motivated

using stigmergy

exploiting both implicit and explicit communication

deliberative versus reactive

using dynamical role assignment


Preceding Works

Reactve Architecture for RoboCup Competition [IJCAI-97] : implicit coordination was first devised

Implicit Coordination in Multi-Agent Systems [DARS-98] : the idea of stigmergy was introduced

Emergent Cooperative Behaviour [IAS-98] : macroparameters and voidance

Cooperative behaviors in multi-robot systems [RAS-29(1)] : some properties of voidance

Emergent behaviors of a robot team performing cooperative tasks [AR-17(1)] : quality functions and roles


Why Multi-Robot Systems (MRS) have been so successful ?

In challenging application domains, MRS can often deal with tasks that are difficult, if not impossible, to be accomplished by an individual robot.

A team of robots may provide redundancy and contribute cooperatively to solve the assigned task, or they may perform the assigned task in a more reliable, faster, or cheaper way beyond what is possible with single robots.


What a Cooperative Multi-Robot System is ?

• While Dudek [Autonomous Robots 1996] and Cao [Autonomous Robots 1997] gave a taxonomy, we prefer to identify several primary research areas

• Early research goes to• Cellular Robotics by Fukuda, IECON 1987] and Cyclic Swarm

by [Beni, Intelligent Control 1988]• Multi-Robot Motion Planning by [Arai, IROS 1989]• ACTRESS Architecture by [Asama, IROS 1989]

• Cooperative Robotics research field is so new that no topic can be considered mature


Research roots for Cooperative Multi-Robot Systems

Cooperative mobile robotics research began after the new behavior-based control paradigm

Brooks 1986, Arkin 1990 Since behavior-based paradigm is rooted in biological inspirations,

many researchers found it instructive to examine the social characteristics of insects and animals

The most common application is using simple local control rules of various biological societies, like ants, bees, and birds, for similar behaviors in MRS

MRS can flock, disperse, aggregate, forage, and follow trails, etc.


New and interesting research issues

The dynamics of ecosystems has been applied to MRS to demonstrate Emergent Cooperation

Cooperation in higher animals, such as wolf packs, has generated significant study in Predator-Prey Systems

Pursuit policies relay expected capture times to the speed and intelligence of the evaders and the sensing capabilties of the pursuers

Competition in MRS, such as in higher animals including humans, is being studied in domains such as multi-robot soccer.


Inherently cooperative tasks [1]

A particular challenging domain for MRS is the one whose tasks are inherently cooperative, that is, tasks in which the utility of the

action of one robot depends upon teammates’ current actions Inherently cooperative tasks cannot decomposed into

independent sub-tasks to be solved by a DARS Team success throughout task execution is measured by the

combined actions of the robot team, rather than by individual actions


Inherently cooperative tasks [2]

More recently identified biological topics of relevance are: Imitation in higher animals to learn new behaviors Physical Interconnectivity by insects such as ants, to

enable collective navigation over challenging terrains How to maintain Communication in a distributed animal

society


Communication versus Cooperation [1]

Communication issue in MRS started since the inception of Distributed Autonomous Robots Systems (DARS) research.

Distinctions between Implicit and Explicit Communication are usually made:

Implicit communication occurs as a side-effect of other actions, or “through the world”

Explicit communication is a specific act designed solely to convey information to other robots on the team.


Communication versus Cooperation [2]

Communication affects the performance of MRS in a variety of tasks

even a small amount of information can lead to great benefit

The challenge is to maintain a reliable communication even when connections between robots may change dynamically and unexpectedly

setting up and maintaining distributed network


Architecture and Task Planning [1]

Research in DARS has focused on the development of architectures, task planning capabilities, and control addressing the issues of:

action selection communication structure heterogeneity versus homogeneity of robots achieving coherence among team actions resolving conflicts, etc.


Architecture and Task Planning [2]

Each architecture focuses on providing a specific type of DARS capability:

fault tolerance swarm control human design of mission plans role assignment, etc.


Motion Coordination The Theoretical Background

A popular topic in MRS is the Motion Coordination. It includes: multi-robot path planning formation generation and formation keeping, traffic control, etc.

These issues are now well understood, although demonstration of these techniques in physical MRS (rather than in simulation) has been limited.

Computational Complexity strongly limit applicability to dynamic real-time robot teams: the problem of moving multiple objects in a shared environment has been showed to be PSPACE-hard by Hopcroft, Schwartz & Sharir, 1984


General research issues introduced into RoboCup since the beginning

• RoboCup is organized in Leagues, each one having its own architectural constraints.

• Research issues are slightly different one to another. For simulated leagues:

• Teamwork among agents

• Agent modelling, from primitive skills to complex ones

• Multi-agent learning, for on-line and off-line learning of

simple skills, as well as more complex strategy, etc.


Issues in Real RoboCup Robots

For real leagues:

• Real-time global or distributed perception from different

sensing sources

• Individual mechanical skills of the physical robots

• Strategic navigation and actions, etc.


A Hybrid Architecture for MRS

We suggest to use an Hybrid architecture where the Deliberative part and the Reactive part can take mutual advantages.

We introduced Robot Schemas at the low level, as building blocks to grow-up complex behaviors from simple ones, according to Arbib and Arkin :

Behaviors are chunks of basic knowledge of how to act and perceive.

Each behavior is implemented with a schema composed by a motor schema, representing the physical activities a perceptual schema which includes the sensing


The Perceptual/Motor Schema

At each level, the primitive control component is a behavior built by perceptual and motor schemas only.The lower reactive level uses only information coming from sensors, and feeds the motors with appropriate commands.

It can elaborate on some perceptual patterns generated by other individual robots, both opponents and temmates.


The Layered Levels of Control

By releasing a behavior, we fire an activation-inhibition mechanism, built on some given evaluation condition rule, at some level of abstraction.

Simple Behaviors like defendArea, or carryBall, are implemented as motor schemas accessing directly the robot effectors.

Basic Behaviors, like playDefensive, and chaseBall, are obtained by simply appending two perceptual schemas seeBall and haveBall.

playDefensive : seeBall --> defendArea chaseBall : haveBall --> carryBall

Since a primitive behavior results in appending just one perceptual schema to one motor schema, at the reactive level we obtain sensori-motor coordinations


An Abstract Architecture

Compound behaviors appear only at higher level, when they may receive more structured information about the environment.

Only the higher deliberative levels refer to cooperative capabilities that any robot could exhibit as a teammate, while a cooperative behavior is going to emerge.


Software architecture: the ArtiFact implementation

• We designed a new hybrid deliberative reactive architecture.

• The classic deliberative paradigm (Sense-Reason-Act) has been evolved reinforcing reactive behaviors.

• A direct link between sense and act has been introduced to speed-up the reactive response of the robot

• Thus, deliberative conditions can be bypassed for certain inputs which need more reactive behaviors


A Functional Architecture [1]


A Functional Architecture [2]

• The architecture of each single robot shows • An inner loop, for close feedback, • An outer looop, for high level reasoning.

• To allow cooperation with teammates, two sensorial sources can input asynchronously both• Environment constraints (the “Ruler”) • Information about teammates (the “Teamplay”)


ADE Software Environment

• All software runs within the frame of ADE , a multi-thread distributed real-time environment working under Linux-OS

• ADE allows to create a set of processes structured as threads.

• Each thread can communicate, through message passing, with other threads of same process, and also with other processes running on other processors

• If a segmentation fault happens, it is possible to kill the thread that caused the error


On Role Allocation in RoboCup [1]

Inspired by Stone and Veloso’s pioneering work, many teams employ role-based coordination, in which robots can take on different static roles within the team

Although it would be possible to statically assign roles once forever, most teams switched to dynamic role allocation, by solving an iterated assignement problem, where the current allocation is re-evaluated periodically 10 times for each millisecond


On Role Allocation in RoboCup [2]

Given N robots, N prioritized (weighted) single-robot roles, and some estimates of how well each robot can be expected to play each role, assign robots to roles so as to maximize the overall expected performance

Gerkey and Mataric [Springer Book on RoboCup2004] showed that this technique is an instance of the canonical Greedy algorithm for Optimization theory


Adopted solutions in RoboCup Teams

RoboCup role allocation problem is similar to task allocation problem for MRS in order to cooperatively achieve the goal, where a time-extended role concept replace that of a transient task

CS Friburg Team used a distributed role allocation mechanism in which two robots may exchange roles only if both want to do it, both moving to a higher-utility role for themselves.

ART Team, as well as early, Artisti Veneti Team, ordered the roles in a descending priority, and then assigned each to the available robot with the highest utility.


Utility Functions

Multi-robot role allocation is a dynamic decision problem, that varies in time, according to the environmental changes,

Utility concept rely on the fact that each individual robot can somehow internally estimate the value (i.e. the cost) of executing an action

In RoboCup it is common to compute utility as the weighted sum of factors like distance to target, distance to ball, defence-offense coonfigurations, etc.

The computation is affected by sensor noise, general uncertainties, and environmental changes

Given the utility value Uij of each robot i for each role j, find the highest utility Uij, assign robot i to role j, and iterate


Master/Supporter Roles Coordination

• Consider the coordination between two robots carrying the ball towards the opponent’s goal:

• We may indentify a Master Role and a Supporter Role• Roles can be played at different responsibility levels:

• Can be >>> Assume >>> Acquire >>> Advocate• Ball assignments depend on Ball Possesses

• HaveBall condition allows to discriminate which robot is really carrying the ball

• It is an Environment constraint acting as a kind of Macroparameter, evaluated by different teammates

• It allows to synchronize the activation of a new cooperation pattern


Notifying Roles

• Roles can be switched provided a notification is exchanged between teammates

• A notification implies a communication between teammates on

first-notified/first-advocated basis• A notify(Role) rule is:

Supporter (mate) -->> reply (role, mate) Master (mate) --> request (role, mate)

• Environment Rules require that a Master role must be advocated, whereas a Supporter role should be acquired.

• haveBall and notify(Role) are the two allowed asynchronous communication from outside for a single robot


Implementing Clamping Behaviors [1]

A role is switched from acquire to advocate, or from assume to acquire, provided a notification is made to its teammate

Two complex Clamping Behaviors for Master and Supporter can be constructed from notify (x) and haveBall (z)

The Master robot shows a chase_ball behavior haveBall (me) & not haveBall (mate) -->> acquire (Master) Acquire (Master) & Notify (Master) -->> advocate (Master)


Implementing Clamping Behaviors [2]

The Supporter robot shows an approach_ball behavior Not acquire (Master) & canBe (Supporter) -->> assume

(Supporter) Assume (Supporter) & Notify (Supporter) -->> acquire

(Supporter) The robot chasing the ball suggests a teammate to become supporter

by advocating a master role, and forcing the other robot to acquire a supporter role by approaching the ball.


Conclusions

How much deliberative process should be endowed in the distributed control of a robot team to play a soccer game cooperatively?

To force coordination when the dynamics of the game is not quite fast, we have tried to drive a cooperative task by a dynamical role assignment, switching from the implicit team assessment, given by the evaluation of quality functions, to the explicit negotiation on the first notified/first advocated basis

how a cooperative behavior can emerge from a robot team antonio d'angelo univ. of udine...

Documents

robot systems mrs

robot team antonio dangelo

individual robot

team of robots

robot systems ras

dangelo et

voidance cooperative

mrs mrs