Multi-robot planning and execution for surveillance missions

Patrick Béchon, Guillaume Casanova, Charles Lesire,Magali Barbier, Guillaume Infantes, Cédric Pralet, Christophe Grand

ONERA – The French Aerospace Lab, Toulouse, Franceprenom.nom@onera.fr

Abstract

This paper presents an architecture for planningand executing a multi-robot mission in presenceof disturbances, including intermittent communi-cation. The mission we consider is a surveillancemission in which the team of robots must ex-plore an area and observe a set of zones of in-terest. The planning algorithm we have devel-oped, HiPOP, takes into account the mission con-straints, including task durations, communicationconstraints, visibility of zones of interest, capabili-ties of robots. . . From the mission description andadditional user knowledge represented by abstractmethods, it builds a temporal plan using partial-order planning techniques. The execution of thistemporal plan is then made on a distributed way:each robot is responsible of executing its own tasks.If some delays occur, the robots will try to main-tain the feasibility of the plan by exchanging macro-scopic temporal information between their commu-nication constraints. When the plan is not feasi-ble anymore, or if a higher level disturbance occurs(like a robot being out of order), HiPOP is asked torepair the plan in order to reallocate the remainingobservation tasks. This architecture has been de-ployed on a team of heterogeneous robots, and anapplication to a surveillance mission by two aerialvehicles and two ground vehicles is shown at theend of the paper.

1 Introduction

Multi-robot missions address surveillance of criti-cal areas, information retrieval after disasters, ex-ploration of unknown areas, search and rescue, etc.In order to deploy a team of autonomous robots,

we must be able to provide a multi-robot plan, i.e.an allocation and scheduling of the tasks the robotsmust fulfill. Such a plan must be synthesized whiletrying to first optimize some criteria (e.g., missionduration, zone coverage, expectation to find an ob-ject of interest, etc.) and second fulfill some con-straints. These constraints may be physical con-straints (e.g., speed of robots, separation distancebetween aerial robots), or mission constraints thatmay come from the mission operator specifying themission.

In the surveillance mission we consider here, theoperator may require regular status of execution,hence requiring to plan regular rendezvous whenrobots will be in communication with the missioncontrol center to report on the progression of theirmission. Moreover, the operator may require somespecific behavior, like having a robot inspect a spe-cific zone, or having a robot perform some tasks ina specific order.

To deploy a robust team of robots in such sce-narios, we have developed a multi-agent planningand execution architecture based on:

• a hierarchical partial-order planner (HiPOP[1]) that plans high-level actions for the teamof robots from the mission description and con-straints;

• a distributed execution algorithm (DIP-M [2]that manages temporal constraints betweenagents’ tasks and that propagates these con-straints when a disturbance occurs (a delay inthe fulfillment of a task or a new temporal con-straint given by the operator);

• a distributed replanning framework that usesthe HiPOP algorithm on specific problems (de-

1

pending on the actual situation) to repair thecurrent plan for a group of robots.

This architecture has been designed while havingin mind that communication may be intermittentduring the mission execution. The distributed algo-rithms then take into account that either there maybe some delays in communications, or that only asubgroup of the involved robots can communicateat a given time.

The paper is organized as follows. The next sec-tion describes the related works regarding multi-robot planning and execution architectures. Sec-tion 3 describes how the mission is modeled. Theplanning algorithm, HiPOP, and the way it is usedfor planning the surveillance mission are describedin section 4. Section 5 presents the execution ofthe mission, including the temporal propagationand the distributed replanning framework. Finally,section 6 presents some experimental scenarios andsome results on the field.

2 Related worksSome works focus on allocating exploration tasksto several robots, but either do not consider com-munication constraints (using frontier-like explo-ration [3] or a segmentation of the environment [4]),or try to maintain communication capabilities atany time by deploying a network infrastructure [5,6]. Some approaches use opportunistic communi-cations to optimize the plan [7], but do not enforcethem. These approaches do not consider time con-straints between tasks, and when synchronizationis explicitly modeled, it is focused on spatial syn-chronization [8]. Other approaches propose mech-anisms to maintain the plan consistency at execu-tion. In [9], robots regularly communicate to up-date temporal constraints between their tasks inorder to maintain the global plan consistency. Inthe exploration mission considered in [10], offlinetask scheduling and online plan flexibility are com-bined, and robots adapt their plans by exchang-ing messages for satisfying task constraints again.However, associated communication tasks are notexplicitly included within plans. Moreover, thesetasks are considered as not subject to failures.

In another direction, multi-robot task schedul-ing deals with time constraints such as task prece-dence or synchronization [11]. In [12, 13], tasks

are scheduled using an auction algorithm to mini-mize their delays. Mixed Integer Linear Program-ming (MILP) is used in [14] to solve task allocationproblems with constraints modeled using Allen’salgebra, and in [15] to find trajectories of robotsthat must meet the already planned trajectories ofrobots to be recharged. In these works, once tasksare scheduled, no communication occurs to shareinformation and maintain plan consistency.

In probabilistic domains, [16] proposes to broad-cast history of actions and observations when aninconsistency is detected between the current ob-servation and the belief state of a local POMDP.[17] uses a DecMDP model with communicationuncertainty, while [18] defines local interactionswithin a so-called IDMG model. While these ap-proaches generate policies which include communi-cation tasks, they do not integrate temporal con-straints between tasks.

3 Mission modeling

3.1 Mission description

We consider the problem of exploring an area us-ing heterogeneous autonomous robots, subject tothe supervision of human operators . When defin-ing the mission, human operators first define somezones to be observed by the robots. These zonesare considered as observation tasks that will be allo-cated to the robots depending on their capabilities(some robots cannot see under trees, or cannot en-ter buildings). In this work, robots are assumed tobe individually able to localize themselves, to plantrajectories, and to perform navigation tasks in theenvironment, the latter being possibly mapped on-line using robot sensors.

In the mission considered, human operators needto regularly monitor mission execution. Due to in-termittent or unreliable communications betweenrobots, and between robots and the operators, thisonline monitoring is defined as a time rate at whicheach robot has to report an execution status forits plan. This execution status may for instancecontain the current robot position and the list ofzones it has observed. Due to communication andmotion constraints, aerial robots, which can movefaster, are used to collect other robots data and tocommunicate these data to the operators. Report-

2

ing tasks correspond to temporal and spatial ren-dezvous, during which robots share information.

3.2 Mission data

The mission consists in observing a set of nz zonesusing nr robots.

• The team of robots is R = {ri}1≤i≤nr.

• The set of zones is Z = {zi}1≤i≤nz .

• Each robot ri can reach a set of navigationpoints P (ri).

• For each robot ri, function Oi gives the navi-gation points from which a zone zj can be ob-served by ri. We have Oi(zj) ⊆ P (ri).

• For communications, we consider a function Cthat gives, for each couple of robots (ri, rj) theset of points from which robots ri and rj cancommunicate. We have: C(ri, rj) ⊆ P (ri) ×P (rj).

We have developed a simple interface to help theoperator preparing the mission (Fig. 1). It allowsto define, on a geo-referenced map of the exploredarea, the zones zi that must be observed, as wellas the set of navigation points P (rj) for each robotor each group of robots. In the mission presentedin the figure, all the ground robots share the sameset of navigation points, whereas navigation pointsfor the aerial vehicles are disjoint to enforce somesafety constraints.

4 Mission planning

To execute a mission, an initial plan is computed of-fline, taking into account all the robots and manda-tory rendezvous to synchronize them. Planningsuch a mission requires temporal reasoning aboutthe actions durations and the time constraints be-tween agents and the environment. For theseproblems, Partial-Order Planners (POP) are wellsuited. Moreover, to improve the efficiency andscalability of the planning process, we developeda planning algorithm, HiPOP [1], mixing partial-order planning and hierarchical planning that helpsrepresenting some expert knowledge about the mis-sion.

Figure 1: Operator interface for mission prepara-tion: zones are represented as red ‘+’, navigationpoints for ground vehicles are blue ‘x’.

4.1 HiPOP

The state of the world is described as a set of posi-tive literals. A problem is defined by a set of avail-able actions, a set of literals representing the initialstate, and a set of literals representing the goal.

An action is defined by a set of preconditions(identifying the states in which the action is appli-cable), a set of effects (the changes involved by theaction), a duration, a possible set of partial plans,called methods, that are used to instantiate a stepof this action and a set of conflicts used by HiPOPto compute the threats caused by abstract steps.

Actions without methods and without conflictscan be executed and are called elementary actions.The other actions are called abstract and cannotbe executed. They must be "instantiated" (ie. re-placed by one of their methods) for the plan to bevalid.

The aim of the planning algorithm is to find asequence of actions that reaches the goal from theinitial state. The algorithm explores the space ofpartial plans to find a plan achieving every literal ofthe goal. Since an action can be scheduled multipletimes, it is represented as a step in a plan, i.e. aninstance of an action with starting and finishingtimepoints.

A partial plan is a set of steps with constraintsbetween either timepoints (then called temporalconstraints) or between actions preconditions andeffects (then called causal links).

The HiPOP algorithm then reasons on the flaws

3

of partial plans. Those flaws, each of them pre-venting the plan to be an executable solution tothe problem, can be of three types:

• Open link: a precondition of a step is not guar-anteed;

• Threat: a step could delete a literal while acausal link is active;

• Abstract flaw: a step of an abstract action isnot yet instantiated.

The planning algorithm is an adaptation of aPartial Order Planning, shown on Algorithm 1.The PopBestPlan and PopBestFlaw procedures arebased on heuristics described in [1]. The Resolversprocedure computes all partial plans that solve aparticular flaw, also taken from HiPOP. The Ini-tialPartialPlan procedure returns a minimum plan:no steps, no causal links or temporal links and onlyopen links corresponding to the goal as flaws.

Algorithm 1: Basic POP algorithm1 Π = {InitialPartialPlan(I,G)} ;2 while Π 6= ∅ do3 P = PopBestPlan(Π) ;4 if F(P ) = ∅ then5 return P ;6 f = PopBestFlaw(F(P )) ;7 Π = Π ∪ Resolvers(A, P, f) ;8 return ∅

Deadlines We also allow the definition of dead-lines as goals (literals) that must be achieved at aset time, before the end of the plan. We model itas a step inserted into the initial plan.

Temporal flexibility A Simple Temporal Net-work [19] over the timepoints of the steps can beassociated with each partial plan. The presenceof the STN allows the final plan to be temporallyflexible: each step is not associated with an ab-solute time but with some temporal constraints.It is possible to compute absolute bounds (earliestand latest dates) and to update those bounds whennew information is acquired (for instance when an-other point is executed). The plan computed bythe planning algorithm is exported with all those

constraints to allow the execution to use this STN.

Allowed list Planning with abstract and elemen-tary actions together allow the planner to return aplan that does not conform to the user intend bydisregarding abstract actions and by planning onlywith elementary actions. To avoid this, the user-defined knowledge also explicitly contains the list ofactions that can be inserted to solve an open link.This is the list of actions that can be used as rootactions: steps using this action can be inserted inthe plan without being the child of another step.

Low priority predicate For the planner to useseveral layers of abstraction together, it has to fo-cus on some part of the problem first. For instance,HiPOP may have to reason about a team of robotshauling a shipment split in several packages. Insuch a case, HiPOP should reason about the posi-tion of the team and the shipment, not about theposition of individual robots or individual packages.HiPOP usually sorts its flaws by type: it will solveall open links before solving abstract flaws. So ifan open link to an individual position is created, itwill be solved alongside open link to team position,increasing the time needed to solve the problem.

The solution used is for the user to specify alist of low-priority predicates. Open links to thosepredicates will be solved after all the abstract links.This allows the planner to instantiate all the stepsbefore taking care of adding some steps to do thetransition between the abstract steps.

4.2 Modeling the mission for HiPOP

The mission is modelled by several elementary ac-tions :

• move(ri, from, to) with ri ∈ R, from, to ∈P (ri)

2: move the robot ri between from fromto to. A robotic library (Gladys [20]) is usedfor each robot and each couple of point to com-pute if the move if possible and its durationgiven the a-priori 3D model of the environ-ment.

• obs(ri, p, z) with ri ∈ R, z ∈ Z, p ∈ Oi(z). Itsduration is set to a fixed value and it createsthe effect explored(z).

4

• com(ri, rj , pi, pj) with ri, rj ∈ R2, (pi, pj) ∈C(ri, rj).

The goal of the mission is to acheive explored(z)for all z ∈ Z.

In addition to the elementary actions, some hi-erarchical actions are introduced, representing pa-trols. A patrol for a robot ri is defined as a set ofwaypoints p0, ..., pn where each pj ∈ P (ri). Eachpatrol is encoded as an abstract action with onlyone method. Its effects are to explore all the zonesvisible from at least one point pj . The patrols allowthe user to specify the kind of trajectory he wantsthe robots to follow and to reduce the overall seachtime.

In addition, the user can specifcy communicationpoints. A point is defined by a pair of robots riand rj , a date and an optionnal couple of points(pi, pj) ∈ C(ri, rj). It mandates the plan to inserta communication between ri and rj at the givendate. If no points are given, it is the reponsabilityof the planner to choose them among C(ri, rj).

5 Mission execution and adap-tation

5.1 Plan representation and execu-tion

Given a plan, the execution algorithm keeps theSTN associated with the plan. At any point, ifa controllable timepoint has all its preceding time-points executed, its execution is launched. Control-lable timepoints are the start points of elementarysteps. When the algorithm is notified at the end ofa step, it updates the STN with the execution ofthe associated uncontrollable timepoint.

Action execution is forwarded to the control ar-chitecture of each robot: we do not manage, e.g.trajectories of the robots to join observation points,but only ask the control architectures of robots toreach these points.

Some disturbances may then occur at execution.These disturbances may concern: (1) lateness inaction execution, when an action successfully endsbut later (or earlier) than expected; (2) failures ofexecution, when an actions fails (e.g., a waypointis actually unreachable). While the latter requiresa replanning of the mission, the former could be

dealt with at the execution level, by trying to mod-ify STN constraints without modifying the actionsthemselves.

5.2 Distributed execution to facetiming disturbances

In order to manage timing disturbances at execu-tion in a consistent way, we have to distribute theexecution process. To do so, we represent the planas a Multi-agent STN (MaSTN [21]). Formally, anMaSTN is defined by a set of N local STNs, oneper agent and a set edges which connect the localSTNs, and that represent external constraints byconnecting local vertices of different agents.

In [2] we have proposed several distributed al-gorithms to propagate constraints between agentsin order to find a new consistent MaSTN aftera timing disturbance. We have decided to inte-grate the DIP-M (Distributed Incremental Propa-gation with Macro-information sharing) algorithminto our multi-robot planning and execution archi-tecture. In DIP-M, each agent reasons about twokinds of constraints: (1) its own local temporal con-straints, and (2) a macroscopic view of the temporalconstraints of other agents; this macroscopic viewprovides distance constraints between external ver-tices; each agent is responsible for sending updateson its own macroscopic view, and hence never re-veals its internal edges.

During execution, each robot maintains his ownMaSTN taking into account the execution time ofactions and disturbances. The DIP-M algorithmsignals the planner when the MaSTN is found in-consistent.

5.3 Plan repairPlan repair is needed when the plan cannot beachieved anymore, either due to action failures, ordue to timing delays that cannot be managed atexecution (the MaSTN becoming inconsistent).

The plan repair process (Fig. 2) uses HiPOP asa standalone planner. The general principle is togive to HiPOP an initial partial plan from whichthe search will start, instead of letting HiPOP startfrom an empty plan. To ensure completeness, weiteratively compute a starting partial plan by re-moving steps in the failed plan until a solution isfound or everything is removed.

5

Figure 2: Repair process

The repair process has as main steps:

• Remove obsolete steps (2): remove all the stepsthat are in the failed plan but whose actionactions are not doable anymore. This also re-moves useless steps, i.e. steps that do not havea chain of causal links to one of the goals.

• Violated deadlines (3-4): A deadline d is vi-olated if the STN associated with the plan istemporally consistent without d but inconsis-tent with. In this case, all steps between thecurrent time and the deadline times are re-moved.

• Solving (5-6): the current plan is used as astarting plan Π for the planning algorithm (Al-gorithm 1).

• Remove more steps (7-9): at each new call itremoves all steps that were previously causallylinked to an already removed step. The stepsthat have already been executed cannot be re-moved, nor can be the steps introduced to rep-resent a deadline.

When repairing we assume that we are repairingthe global plan. This means that when a reparationis needed, one agent (called repairer) is tasked torepair the plan of every agent it can communicatewith (called reachable agents). Usually the repaireris the agent that detected the flaw in the currentplan). It will first querry the other robots for their

plan and merge them in a global plan. All the ac-tions of the agents that could not respond (calledunreachable agents) are considered locked, meaningthat they will not be modified by the repair pro-cess. This global plan is then repaired and the newplan is dispatched to all the reachable robots. Thisway the global plan can be change with no impacton the plan of unreachable robots.

If no solution can be found, it is possible to mod-ify the plan of unreachable robots. The modifica-tion can only happen after a scheduled communica-tion with a reachable robot. This means that dur-ing this communication, the reachable robot willsend the new plan to the unreachable robot. If theunereachable robot already repaired its own plan,a new repair is triggered.

6 ExperimentsThis multirobot planning and execution architec-ture is applied in the context of the Action project1.The objectives of the ongoing experiments are todeploy a team of heterogeneous robots to observe apartially known area in order to detect some possi-ble intruders.

The mission we are preparing for now involvestwo aerial robots and two ground robots. The areato observe is about 200m width by 200m height,and is represented on Fig. 1. It contains 17 zonesand 2 mandatory communication points between aground robot and an aerial robot. The initial planis found in less than 1 second while exploring 85plans. It contains 67 actions for a plan of about 9minutes.

At execution, we have to face several distur-bances. First, action may be delayed, which is ifpossible managed through distributed propagationof the MaSTN. Second, we may loose a robot, e.g.if the localization process embedded in the robotfails and the robot cannot navigate anymore. Insuch a case, we replan the mission for the remain-ing robots, considering the observations that havealready been done by live robots, and the actionsof the lost robot that must be replanned.

The mission can be simulated with the MORSEsimulator [22]. A screenshot is presented on Fig. 3

A sample execution is shown on Fig. 4. Duringthe first 300 seconds, no unexepected events ap-

1http://action.onera.fr

6

Figure 3: Screenshot of the two ground robots inthe MORSE simulator

pear. Communication are mandated at t = 280s.So each robot waits to communicate and then car-ries on with its plan (Fig. 4a). This wait is im-portant if the previous actions are delayed. Att = 300s, the green robot is killed. Figure 4b showsthe plan of each agent: the green robot stops andevery other robot is unaffected. After that, the bluerobot is notified of the loss of the green robot (rep-resenting communication from the green robot or adetection from the blue robot). It replans for ev-eryone, thus pausing its own execution. The otherrobots do not pause the execution of their currentaction but would not start any action until the newplan is available. Since only the other ground robot(top red) could replace the green robot, it got newactions to perform (Fig. 4c).

7 Conclusion and future works

In this paper, we presented an architecture for plan-ning and executing a multi-robot mission. Thisarchitecture uses a hybrid planner that mixespartial-order planning and hierarchical informa-tion, HiPOP. From the plan resulting from HiPOP,we have explained how we can monitor the execu-tion in a distributed way, resulting in a mechanism,DIP-M, that adapts the temporal plan to timingdisturbances at execution, while possible. Other-wise, or when a major disturbance occurs (like loos-ing a robot), we use HiPOP again to repair themission, while taking into account the set of robotswith which the repairing robot can communicate ata given time.

This architecture has been applied to a surveil-lance mission by a team of four robots (two aerial

and two ground robots) on the field. The finaldemonstration will take place in mid-June. By mid-October, we will apply the same architecture on asurveillance mission involving 2 aerial robots and6 ground robots on the field, and extra simulatedrobots to have a team of 12 autonomous robots.

References[1] P. Bechon, M. Barbier, G. Infantes, C. Lesire,

and V. Vidal. HiPOP: Hierarchical Partial-Order Planning. In STAIRS, Prague, CzechRepublic, 2014.

[2] G. Casanova, C. Lesire, and C. Pralet. Man-aging Dynamic Multi-Agent Simple TemporalNetwork. In AAMAS, Istanbul, Turkey, 2015.

[3] H. Hourani, E. Hauck, and S. Jeschke.Serendipity Rendezvous as a Mitigation ofExploration’s Interruptibility for a Team ofRobots. In ICRA, Karlsruhe, Germany, 2013.

[4] K. Wurm, C. Stachniss, and W. Burgard. Co-ordinated multi-robot exploration using a seg-mentation of the environment. In IROS, Nice,France, 2008.

[5] Y. Pei and M. Mutka. Steiner traveler: Re-lay deployment for remote sensing in hetero-geneous multi-robot exploration. In ICRA,Saint-Paul, MN, USA, 2012.

[6] P. Abichandani, H. Benson, and M. Kam. Ro-bust Communication Connectivity for Multi-Robot Path Coordination using Mixed Inte-ger Nonlinear Programming: Formulation andFeasibility Analysis. In ICRA, Karlsruhe, Ger-many, 2013.

[7] C. Sung, N. Ayanian, and D. Rus. Improv-ing the Performance of Multi-Robot Systemsby Task Switching. In ICRA, Karlsruhe, Ger-many, 2013.

[8] B. Coltin and M. Veloso. Optimizing forTransfers in a Multi-Vehicle Collection andDelivery Problem. In DARS, Baltimore, MD,USA, 2012.

[9] G. Kaminka, A. Yakir, D. Erusalimchik, andN. Cohen-Nov. Towards collaborative task

7

(a) Before the loss of a robot every-thing is nominal

(b) After the loss of a the greenrobot : the other robots are unawareof the loss of a robot and continue

(c) When the blue robot is notifiedof the loss of the green robot, it re-plans and add action to the otherground robot (top red robot)

Figure 4: Some timelines of execution

and team maintenance. In AAMAS, Honolulu,Hawaii, 2007.

[10] A. Korsah, B. Kannan, B. Browning,A. Stentz, and B. Dias. xBots: An approach togenerating and executing optimal multi-robotplans with cross-schedule dependencies. InICRA, Saint-Paul, MN, USA, 2012.

[11] Y. Zhang and L. Parker. Multi-Robot TaskScheduling. In ICRA, Karlsruhe, Germany,2013.

[12] S. Ponda, J. Redding, H. Choi, J. How,M. Vavrina, and J. Vian. Decentralized Plan-ning for Complex Missions with DynamicCommunication Constraints. In ACC, 2010.

[13] L. Luo, N. Chakraborty, and K. Sycara. Dis-tributed Algorithm Design for Multi-RobotTask Assignment with Deadlines for Tasks. InICRA, Karlsruhe, Germany, 2013.

[14] M. Koes, I. Nourbakhsh, and K. Sycara. Con-straint optimization coordination architecturefor search and rescue robotics. In ICRA, Or-lando, FL, USA, 2006.

[15] N. Mathew, S. Smith, and S. Waslander. AGraph-Based Approach to Multi-Robot Ren-dezvous for Recharging in Persistent Tasks. InICRA, Karlsruhe, Germany, 2013.

[16] F. Wu, S. Zilberstein, and X. Chen. On-line planning for multi-agent systems withbounded communication. Artificial Intelli-gence, 175:487–511, 2011.

[17] L. Matignon, L. Jeanpierre, and A. Mouaddib.Coordinated Multi-Robot Exploration UnderCommunication Constraints Using Decentral-ized Markov Decision Processes. In AAAI,2012.

[18] M. Spaan and F. Melo. Interaction-drivenMarkov games for decentralized multiagentplanning under uncertainty. In AAMAS, 2008.

[19] R. Dechter, I. Meiri, and J. Pearl. Tempo-ral constraint networks. Artificial intelligence,49(1):61–95, 1991.

[20] Pierrick Koch, Cyril Robin, and Arnaud De-groote. Gladys: 0.2.5, 2015. URL: http://dx.doi.org/10.5281/zenodo.14085, doi:10.5281/zenodo.14085.

[21] J. Boerkoel, L. Planken, R. Wilcox, andJ. Shah. Distributed Algorithms for Incremen-tally Maintaining Multiagent Simple TemporalNetworks. In ICAPS, Rome, Italy, 2013.

[22] Gilberto Echeverria, Séverin Lemaignan, Ar-naud Degroote, Simon Lacroix, Michael Karg,Pierrick Koch, Charles Lesire, and SergeStinckwich. Simulating complex robotic sce-narios with morse. In SIMPAR, pages 197–208, 2012.

8