real-time motion planning of multiple agents and...

REAL-TIME MOTION PLANNING OF MULTIPLE AGENTS

AND FORMATIONS IN VIRTUAL ENVIRONMENTS

by

Yi Li

M.Sc., The Royal Institute of Technology (KTH), 2001

A THESIS SUBMITTED IN PARTIAL FULFILLMENT

OF THE REQUIREMENTS FOR THE DEGREE OF

DOCTOR OFPHILOSOPHY

in the School

of

Engineering Science

c© Yi Li 2008

SIMON FRASER UNIVERSITY

Fall 2008

All rights reserved. This work may not be

reproduced in whole or in part, by photocopy

or other means, without the permission of the author.

APPROVAL

Name: Yi Li

Degree: Doctor of Philosophy

Title of thesis: Real-Time Motion Planning of Multiple Agents and Formations in

Virtual Environments

Examining Committee: Dr. Daniel C. Lee

Chair

Dr. Kamal Gupta, Senior Supervisor

Professor, School of Engineering Science

Dr. Shahram Payandeh, Supervisor

Professor, School of Engineering Science

Dr. Binay Bhattacharya, Supervisor

Professor, School of Computing Science

Dr. Richard T. Vaughan, Internal Examiner

Assistant Professor, School of Computing Science

Dr. Dinesh Manocha, External Examiner

Professor, The University of North Carolina at Chapel Hill

Date Approved:

ii

Abstract

In this thesis, we studied the problem of real-time motion planning of multiple agents and multiple

formations in virtual environments and games. In many such applications, agents move around

and their motions must be planned. We are especially interested in motion planning in Real-time

Tactical (RTT) games because they offer a challenging problem setting due to the following aspects:

multiple agents, real-time, dynamic obstacles, complex environments, coherence of the agents (e.g.,

formations), and inexpensive pre-processing.

We use the (basic) continuum model — a real-time crowd simulation framework based on the

Fast Marching Method (FMM) — extensively in this thesis, because it unifies global planning and

local planning (e.g., collision avoidance). Since the basic model may fail to generate collision-

free paths in certain constrained situations (e.g., when agents pass through narrow passages) due

to deadlocks amongst the agents, we propose to use a principled and efficient AI technique for

decision making and planning (i.e., Coordination Graph (CG)) to avoid such deadlocks in the narrow

passages.

Next, we present the adaptive multi-resolution continuum model — an extension to the basic

continuum model — to plan motions of multiple agents in virtual environments. It allows each

agent to have its own goal compared to the basic model, where agents have to be grouped into a few

groups, while retaining the advantages of the basic model such as unified global planning and local

planning.

Finally, we present a flexible virtual structure approach to the multi-agent coordination problem

in virtual environments. The approach conceives of agents in a formation as if they lie on an elastic

shape, which is modeled using the Boundary Element Method (BEM). Due to the BEM’s boundary-

only nature, even a formation with a large number of agents can be deformed in real-time. This

flexible virtual structure approach for formation control is then combined with the continuum model

to plan motions of multiple formations in virtual environments. To the best of our knowledge, this

iii

motion planning algorithm for multiple formations is the first one that does not use ad-hoc and local

approaches and hence agents in a formation do not split easily from the formation.

We believe that these three algorithms can be used as basic motion planning toolkits toward

enhancing the capabilities of RTT games.

iv

To my parents.

v

Who drives the horses of the sun

Shall lord it but a day;

Better the lowly deed were done,

And kept the humble way.

The rust will find the sword of fame,

The dust will hide the crown;

Ay, none shall nail so high his name

Time will not tear it down.

The happiest heart that ever beat

Was in some quiet breast

That found the common daylight sweet,

And left to Heaven the rest.

— The Happiest Heart,JOHN VANCE CHENEY, 1913

vi

Acknowledgments

This thesis dissertation marks the end of a long and eventful journey for me. I would like to gratefully

acknowledge all the people who supported me along the way. Above all I would like to acknowledge

the tremendous sacrifices that my motherYun Maand my father Dr.Ming Li made to ensure that I

had an excellent education. For this and much more, I am forever in their debt. It is to them that I

dedicate this dissertation.

I am grateful to my Ph.D. supervisorKamal Gupta, whose thoughtful guidance and continuing

support have lead me to grow immensely over the last five years. During these years, I was fortunate

to have some very nice lab-mates:Yifeng Huang, Moslem Kazemi, Lila Torabi, Pengpeng Wang, and

Zhenwang Yao. It will be boring without you. I also like to thank prof.Shahram Payandehfor his

valuable contributions to this thesis and all other committee members for accepting the invitation to

be part of my committee.

When I was an exchange student in Houston, Texas, USA during the academic year 1999-2000,

I was fortunate to take a couple of robotics courses from prof.Guanrong (Ron) Chenat University of

Houston and prof.Lydia Kavrakiat Rice University. I must have enjoyed those classes immensely,

because I made eventually the decision to become a Ph.D. student in the field of robotics.

I visited the Visual Agent Laboratory (VAL) of the Toyohashi University of Technology (TUT)

in Toyohashi, Japan from October 15, 2006 to January 15, 2007. I would like to thank prof.Shigeru

Kuriyama for his kindness. The three months I spent at TUT has definitely broadened my view

and changed the course of my research. I am very grateful to the Royal Swedish Academy of

Engineering Sciences (IVA) and the Hans Werthen foundation who made the trip possible. Finally,

I would like to thank Dr.Kevin T. Chuat Princeton University for providing me with a customized

version of their Level Set Method Library (LSMLIB) and Mr.Brian Corrie at the IRMACS Centre

at Simon Fraser University for his help in parallel implementation of coordination graphs.

This research has been enabled by the use of WestGrid computing resources, which are funded

vii

in part by the Canada Foundation for Innovation, Alberta Innovation and Science, BC Advanced

Education, and the participating research institutions. Work on this thesis was supported in part, by

a grant from Natural Sciences and Engineering Research Council (NSERC) of Canada.

viii

Curriculum Vitae

Yi Li studied electrical engineering at the Royal Institute of Technology (KTH), Stockholm, Sweden

between 1996 and 2001. In the academic year 1999/2000, he was an exchange student at University

of Houston and Rice University in Houston, TX, USA. After receiving the M.Sc. degree from KTH

in 2001, he worked at Ericsson AB in Kista, Sweden as a software engineer and developed software

for Ericsson’s WCDMA Radio Base Stations. Between 2003 and 2008, he studied as a Ph.D. student

under the supervision of Prof. Kamal Gupta at Simon Fraser University, Burnaby, Canada.

ix

Publications

Conference Papers

1. Yi Li and Kamal Gupta. Real-time motion planning of multiple formations in virtual en-

vironments: flexible virtual structures and continuum model. In Proceedings of IEEE/RSJ

International Conference on Intelligent Robots and Systems, 2008.

2. Yi Li and Kamal Gupta. Motion planning of multiple agents in virtual environments on paral-

lel architectures. In Proceedings of IEEE International Conference on Robotics and Automa-

tion, pages 1009 - 1014, 2007.

3. Yi Li and Kamal Gupta. A hybrid two-layered approach to real-time motion planning of mul-

tiple agents in virtual environments. In Proceedings of IEEE/RSJ International Conference on

Intelligent Robots and Systems, pages 4362 - 4367, 2006.

4. Yi Li and Kamal Gupta. Large-scale agent formations in virtual environments using linear

elastic shapes. In Proceedings of International Conference on Computer Animation and Social

Agents, pages 91 - 96, 2005.

5. Yi Li, Kamal Gupta, and Shahram Payandeh. Motion planning of multiple agents in virtual

environments using coordination graphs. In Proceedings of IEEE International Conference on

Robotics and Automation, pages 380 - 385, 2005.

Videos

1. Yi Li and Kamal Gupta, Motion Planning of Multiple Agents in Virtual Environments (Video).

AAAI-07 (the Twenty-Second Conference on Artificial Intelligence) AI Video Competition,

Vancouver, BC, Canada, 2007.

x

Contents

Approval ii

Abstract iii

Dedication v

Quotation vi

Acknowledgments vii

Curriculum Vitae ix

Publications x

Contents xi

List of Tables xiv

List of Figures xv

1 Introduction 1

1.1 Motion Planning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.1.1 Configuration Space . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

1.1.2 Three General Path Planning Approaches . . . . . . . . . . . . . . . . . . 5

1.1.3 Random Sampling Based Approaches . . . . . . . . . . . . . . . . . . . . 6

1.1.4 Motion Planning of Multiple Robots . . . . . . . . . . . . . . . . . . . . . 7

1.1.5 Motion Planning of Multiple Robots in Dynamic Environments . . . . . . 8

1.1.6 Motion Planning of Multiple Robots as a Group/Formation . . . . . . . . . 10

xi

1.2 Thesis Problems and Overall Solutions . . . . . . . . . . . . . . . .. . . . . . . . 13

1.2.1 Motion Planning in Static Environments with Narrow Passages . . . . . . . 13

1.2.2 The Adaptive Multi-resolution Continuum Model . . . . . . . . . . . . . . 15

1.2.3 Flexible Virtual Structures and Motion Planning of Multiple Formations . . 17

1.2.4 Path Quality . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 18

1.3 Thesis Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

1.4 Thesis Outline . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

2 Preliminaries 22

2.1 The Fast Marching Method . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

2.2 The Continuum Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26

3 Motion Planning in Environments with Narrow Passages 28

3.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 29

3.2 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.2.1 Coordination Graphs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

3.3 Overall Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

3.4 Information Extraction from Binary Occupancy Grids . . . . . . . . . . . . . . . . 32

3.5 Motion Coordination with Coordination Graphs . . . . . . . . . . . . . . . . . . . 34

3.5.1 Construction of Coordination Graphs . . . . . . . . . . . . . . . . . . . . 34

3.5.2 Computation of Optimal Joint Actions . . . . . . . . . . . . . . . . . . . . 37

3.5.3 Deadlock Free Optimal Joint Actions . . . . . . . . . . . . . . . . . . . . 38

3.6 Parallel Processing . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.6.1 Supervisor-Worker Paradigm . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.6.2 Job Scheduling . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.6.3 Asynchronous Message Delivery . . . . . . . . . . . . . . . . . . . . . . . 40

3.7 Experiments and Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.7.1 Hardware and Software Setup . . . . . . . . . . . . . . . . . . . . . . . . 41

3.7.2 Parallel Performance Analysis . . . . . . . . . . . . . . . . . . . . . . . . 42

3.7.3 Usability in Real-Time Applications . . . . . . . . . . . . . . . . . . . . . 44

3.8 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45

4 Motion Planning of Multiple Agents 51

4.1 Problem Definition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

xii

4.2 The Adaptive Continuum Model . . . . . . . . . . . . . . . . . . . . . . . .. . . 52

4.3 Extension: Multi-Resolution Channel Construction . . . . . . . . . . . . . . . . . 54

4.4 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56

4.5 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5 Motion Planning of Multiple Formations 64

5.1 Problem Description . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

5.2 Overall Solution . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 66

5.3 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

5.3.1 The Boundary Element Method . . . . . . . . . . . . . . . . . . . . . . . 67

5.4 A Flexible Virtual Structure Approach For Formation Control . . . . . . . . . . . . 73

5.4.1 Real-Time Formation Deformation . . . . . . . . . . . . . . . . . . . . . 73

5.4.2 Fast Collision Detection For Self-Collision Free Deformation . . . . . . . 74

5.4.3 Generation Of Deformations . . . . . . . . . . . . . . . . . . . . . . . . . 75

5.5 Motion Planning Of Formations . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

5.5.1 Construction Of Unit Cost Field . . . . . . . . . . . . . . . . . . . . . . . 76

5.5.2 Moving Formation Along Gradient: Construction Of Waypoints . . . . . . 77

5.5.3 Social Potential Fields . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78

5.5.4 Formation Deformation . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

5.6 Experiments And Results . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

5.6.1 The Flexible Virtual Structure Experiments . . . . . . . . . . . . . . . . . 80

5.6.2 The Motion Planning Experiments . . . . . . . . . . . . . . . . . . . . . . 82

5.7 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

6 Conclusion and Future Work 99

6.1 Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

6.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 101

A DVD Movies 103

Bibliography 104

xiii

List of Tables

3.1 Runtime of one CG task. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.2 Runtimes of 40 equal-size CG tasks. . . . . . . . . . . . . . . . . . . . . . . . . . 43

3.3 Runtimes of unequal-size CG tasks (with and without scheduling). . . . . . . . . . 43

4.1 Speed comparison between the basic continuum model and our adaptive approach. 60

5.1 Computation time of the BEM matrices in millisecond and average computation

time (over 20 different deformations) for one deformation in millisecond for the Vee

formation in Fig. 5.2. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

5.2 Average computation time for one deformation in millisecond for the infantry for-

mations. K is the number of the control nodes,E is the number of control nodes,

andN is the number of agents. The number of deformations was 20 in each case. . 81

5.3 Running Time (millisecond/cycle). . . . . . . . . . . . . . . . . . . . . . . . . . . 84

xiv

List of Figures

1.1 The continuum model can compute collision-free paths for multiple agents in real-

time when the agents are moving in an environment without any narrow passages

(Fig. 1.1a), but it may fail if the environment contains narrow passages (Fig. 1.1b).

For example, as shown in Fig. 1.1b, agents #2 and #5 are in deadlock because agent

#2 is moving right and trying to exit the narrow passage while agent #5 just entered

the passage and is moving left. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

1.2 When a formation in [70] encounters an obstacle that’s within its front projection,

the formation is split into two formations. As soon as these formations have passed

the obstacle, they merge back into one formation. . . . . . . . . . . . . . . . . . . 11

1.3 Two groups of agents are moving in an environment with one roundabout and four

entrances. Agents must move counterclockwise inside the roundabout. . . . . . . . 15

1.4 Paths for 20 agents (on a 64×64 grid) constructed by our adaptive multi-resolution

continuum model. The simulation cycle ran at 40 hertz (i.e., five times faster than

the basic model). . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

1.5 Paths for four formations in a virtual environment (64×64 grid) without static ob-

stacles. Each formation’s initial configuration and final configuration are shown as

a number of black circles and black discs (which represent the agents), respectively. 19

2.1 Discretized grid structure [91], where the scalar fieldsg andφ are stored inside each

grid cell, whereas the anisotropic fields such asf , C, and∇φ are stored at each face

between two neighboring grid cells. The fieldsf , g, andC are set in Section 2.2,

whereasφ is the solution to (2.1) computed using the FMM. . . . . . . . . . . . . 24

xv

3.1 Narrow passages (from left to right): one lane passage (e.g., one lane bridge), 3-way

intersection (e.g., T intersection, Y intersection), 4-way intersection (i.e., crossroad),

and circular intersection (i.e., roundabout). . . . . . . . . . . . . . . . . . . . . . . 29

3.2 Coordination graph for a coordination problem with 4 agents. . . . . . . . . . . . . 31

3.3 An environment with two narrow passages. . . . . . . . . . . . . . . . . . . . . . 46

3.4 An environment with one roundabout. . . . . . . . . . . . . . . . . . . . . . . . . 47

3.5 The rendering is decoupled from the supervisor and its workers. The processes

communicate with each other through System V message queue and shared memory. 47

3.6 Speedups for 40 equal-size CG tasks. . . . . . . . . . . . . . . . . . . . . . . . . . 48

3.7 Speedups for unequal-size CG tasks (with and without scheduling). . . . . . . . . 48

3.8 Two groups of agents are moving in an environment with one narrow passage and

three entrances. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.9 Two groups of agents are moving in an environment with one roundabout and four

entrances. Agents must move counterclockwise inside the roundabout. . . . . . . . 50

4.1 Reducing the domain for potential constructions from all grid cells (left) in the vir-

tual environment to a channel (right). . . . . . . . . . . . . . . . . . . . . . . . . . 53

4.2 0→ 0 paths for 20 agents. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

4.3 2→ 0 paths for 12 agents in an environment with narrow passages. . . . . . . . . . 62

4.4 0→ 0 paths for 5 agents in an environment with dynamic obstacles. . . . . . . . . 63

5.1 A formationR i is mapped from frameFfi , where it is defined, to the environment’s

frame of referenceFw. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 85

5.2 A Vee formation located on an imaginary circular elastic shape can be deformed in

real-time using BEM by displacing the (red in the color version) box control nodes

located on the elastic shape. The displacements of the control nodes are represented

by arrows. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

5.3 Path for a rank formation in a virtual environment (64×64 grid). The formation’s

initial configuration and final configuration are shown as a number of black circles

and black discs (which represent the agents), respectively. . . . . . . . . . . . . . . 87

5.4 Paths for three Vee formation in a virtual environment (64×64 grid). Each forma-

tion’s initial configuration and final configuration are shown as a number of black

circles and black discs (which represent the agents), respectively. . . . . . . . . . . 88

xvi

5.5 Paths for four formations in a virtual environment (64×64 grid) without static ob-

stacles. Each formation’s initial configuration and final configuration are shown as

a number of black circles and black discs (which represent the agents), respectively. 89

5.6 Paths for four formations in a virtual environment (64×64 grid) with static obsta-

cles. Each formation’s initial configuration and final configuration are shown as a

number of black circles and black discs (which represent the agents), respectively. . 90

5.7 The rank formation in [70] is made smaller by moving its agents closer to each other

so that the smaller formation can fit through the small gap between the obstacles. . 91

5.8 Boundary Element Method. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 92

5.9 Construction of waypoints for formationR i at timet + ∆t. . . . . . . . . . . . . . . 93

5.10 Formations and their deformations. . . . . . . . . . . . . . . . . . . . . . . . . . . 94

5.11 An infantry of soldiers move around in a virtual environment (64× 64 grid) with

three static obstacles. Figure No. 1 out a series of 4. . . . . . . . . . . . . . . . . . 95







xvii

Chapter 1

Introduction

Motion planning has been traditionally studied in the area of robotics. In recent years the techniques

are increasingly used in virtual environments and games. In many such applications, agents (called

entities in [40]) move around and their motions must be planned. There are four different types

of motions in virtual environments and games:navigation, animation, manipulation, andcamera

motion. In this thesis, we focus on the navigation part (i.e., how each agent finds a route to a

particular goal while avoiding collisions with obstacles and other agents), because this artificial

intelligence (AI) problem is probably the most common AI problem in the game industry [68].

The objects of the navigation in games is to take agents to their goals while avoiding both static

obstacles (e.g., buildings, water, fences) and dynamic/movable obstacles (e.g., vehicles, floating

bridges). The game design may also require that agents avoid areas with hazardous conditions (e.g.,

agents may prefer open spaces over narrow passages because the narrow passages are easy for the

enemies to defend). These factors add to the realism of the game and should be taken into account

in the game design [68].

We are especially interested in motion planning in Real-time Tactical (RTT) games. RTT is

a genre of computer games that could be characterized as war-games in real-time and the players

in RTT games do not take “turns” like conventional strategy video or board games. We focus on

RTT games in this thesis because they offer a challenging problem setting owing to the following

aspects [65, 66]:

1. Multiple agents: There are large numbers of agents moving around in the virtual environ-

ments and those agents must avoid each other. Each agent typically has 2 or 3 degrees of

freedom (DOFs).

1

CHAPTER 1. INTRODUCTION 2

2. Real-Time: The agents’ motions must be computed in real-time. Thus, the planner must be

able to perform a path query in real-time while the game is being played.

3. Dynamic: Virtual environments contain not only static obstacles, but also dynamic obstacles.

4. Complexity: Virtual environments can be complex.

5. Coherence: A collection of agents in a virtual environment may share a common goal. The

resulting motions of the agents in the same group planned by a planner must be perceived by

humans as being coherent (i.e., they must stay together and follow the same route to the goal).

6. Inexpensive pre-processing: A planner may not spend more than a few seconds in the pre-

processing phase because (1) a gamer may only tolerate a few seconds of delay when start-

ing/loading the game and (2) virtual environments are often generated by computers when the

game is being started/loaded.

The key underlying problem is toplan motions of multiple agents in real-time and with a short

pre-processing phase. In this thesis, we assume that there is no uncertainty in the agents’ motions

and the virtual environments are known. We study not only motion planning of multiple individual

agents, but also motion planning of multiple formations of agents, because agents are often grouped

together into formations in RTT games. Aformation is a coherent group of agents establishing and

maintaining some predetermined geometrical shape by controlling the positions of orientations of

each individual agents relative to the group. For example, military ground forces (e.g., infantry,

cavalry), military aircraft, naval vessels are often deployed in tactical formations (e.g., column, line,

square, wedge/v formations).

The agent-based approaches such asflocking[73], steering behaviors[74], and an algorithm [95]

based onReciprocal Velocity Obstacles[94], where motion for each agent is computed separately,

are widely used in the current generation of RTT games for simulation of multiple agents’ motions.

Because agent-based approaches combine the local collision avoidance with global path planning

(e.g., the multi-agent navigation algorithm in [95] uses a pre-computed roadmap for global path

planning andReciprocal Velocity Obstaclesfor local navigation and collision), conflicts arise in-

evitably between the local goals and the global goals (e.g., in area of high congestion). A real-time

crowd simulation framework called theContinuum Model1 [91] addresses this problem by unify-

ing global planning and local planning and thus agents modeled by the continuum model do not

1Thecontinuum modeladapts a continuum perceptive in contrast to the agent-based approaches. It computes a set ofpotential fields over the domain that guide all agents’ motions simultaneously.


face conflicting requirements between global planning and local planning. However, the continuum

model has to group the agents into a few groups (at most 4 in [91]) with all agents in a group sharing

the same goal in order to maintain real-time performance. In addition, the continuum model (and the

agent-based approaches) may fail to generate collision-free paths due to deadlocks in certain con-

strained situations (e.g., in narrow passages). Finally, the continuum model has only been applied to

motion planning of multiple agents, not groups/formations of agents. The quality of group behavior

in the current generation of RTT games is in general not good because they use ad-hoc and local

approaches and hence agents in a group split easily from the group and take different paths to the

goal (of the group) [40].

In this thesis, we build upon the continuum model and present three algorithms as basic motion

planning toolkits that can be used toward enhancing the capabilities of RTT games. In the first part of

the thesis, we use coordination graphs, a principled AI approach, in conjunction with the continuum

model, that allows us to plan motions of multiple agents in real-time while avoiding deadlock in

two dimensional static virtual environments with narrow passages. In the second part, we present an

adaptive extension to the continuum model to the problem of real-time motion planning of multiple

(up to 40 in the experiments on a dual-core PC) agents with independent goals in two dimensional

dynamic virtual environments. The third and final part deals with realistic modeling of formations

and motion planning of multiple formations in two dimensional virtual environments with dynamic

obstacles. These three algorithms can be used as basic motion planning toolkits toward enhancing

the capabilities of RTT games.

Note that both the agent-based approaches and the continuum model run a continuoussimulation

cycleof planning(e.g., computing potential functions) andexecution/acting(e.g., moving opposite

the gradient of a potential function computed in the planning step) because no agent in RTT games

has prior knowledge of the future movements of other agents or dynamic obstacles. In order for an

agent to react timely on changes, the cycle must be run at a high frequency. Consequently, we plan

motions of multiple agents/formationsonline in real-time in this thesis (as opposed to theoffline

approach widely used in the robotics community).

In the remainder of this chapter, we elaborate on the motion planning problem in Section 1.1.

We formally define the problems to be studied in this thesis and present the overall solutions in

Section 1.2. We detail the contributions and the outline of the thesis in Sections 1.3 and 1.4, respec-

tively.


1.1 Motion Planning

In robotics, the objective of motion planning is to compute a sequence of admissible motions for one

robot in order to change the state of the world [54]. The simplest type of motion planing problem

is the path planning problem (i.e., computing a path for a robot that takes the robot from its initial

position to its goal position without colliding with the given static obstacles).

Most path planning algorithms transform the path planning problem of a robot in the workspace

into a simple problem of a moving point by mapping the robot to a point in the configuration

space [58]. The continuous configuration space is then sampled (for high dimensional C-spaces) [42,

43, 55, 59] or can even be discretized (for low-dimensional C-spaces) and a graph is constructed to

represent the connectivity of the continuous configuration space. Finally, the graph is searched for a

path that takes the robot to its goal without collisions using standard graph search techniques, such

as the Dijkstra’s algorithm or the A* algorithm. Note that most practical path planning algorithms

(for high dimensional configuration spaces) are not complete (i.e., they may fail to find a path even if

one exists and notify when no path exists), because achievingcompletenessis often computationally

intractable, so they trade-off completeness for computational efficiency.

1.1.1 Configuration Space

A robot moves in workspaceW (eitherR2 or R3). If the position of every point in the robot can

be uniquely determined byd parameters, then the configuration of the robot is ad-dimensional

parameter vector and it is a point in thed-dimensional configuration spaceC . For example, all

agents in this thesis move in aR2 workspaceW and the configuration of each agent is its position

(x,y) and orientationθ , whereθ ∈ [−π,π). When a robot at a configurationq is collision-free (with

obstacles and itself), the configurationq is called free. The free configuration spaceC f ree is then

defined as the subset of all free configurations. The configuration space obstacleCobs is simply the

complement of all free configurations.

With the notion of the configuration spaceC , the path planning problem for a robot becomes that

of finding a path for a point (that represents the robot) in the free configuration spaceC f ree. More

formally:

Definition 1.1.1 (Path Planning Problem). Given a robot’s initial configurationqinit and a desired

goal configurationqgoal, find a path betweenqinit andqgoal that lies in the free configuration space

C f ree of the robot .


1.1.2 Three General Path Planning Approaches

A path planning algorithm usually represents the connectivity of the free configuration spaceC f ree

with a graph and then searches for a path betweenqinit andqgoal in C f ree for the robot. There are three

general approaches for construction and representation of the connectivity graphs:roadmap, cell

decomposition, andpotential field. For simple path planning problems in two or three dimensional

configuration spaces, these approaches are fast, but in their original forms, they do not scale up

for high dimensional path planning problems (four or more). In the following, we present some

examples of those three approaches for path planning problems in two dimensional configuration

spaces.

A roadmapapproach represents the connectivity of the free configuration spaceC f ree with a

graphG of one dimensional curves (i.e., the roadmap). Once the graphG is constructed, it is

searched for a collision-free path (i.e., the motion of the robot is restricted to the curves inG. In two

dimensional polygonal configuration spaces, the visibility graph [61] and the Voronoi diagram [63]

are two complete roadmap-based approaches: there is a collision-free path in the configuration

spaceC betweenqinit andqgoal if and only if there is such a path in the corresponding graphs. The

visibility graph approach captures the connectivity ofC in a visibility graphGvis: the nodes ofGvis

represent the vertices of the polygonal obstacles plusqinit andqgoal, and the edges ofGvis represent

the straight-line path between two nodes that does not intersect the obstacles. The Voronoi diagram

approach captures the connectivity ofC in a Voronoi diagram. Paths produced by the visibility

graph approach graze the obstacles, while paths produced by the Voronoi diagram approach have

the maximum clearance. Note that Voronoi diagram is related to medial axis in the sense that the

Voronoi diagram of the edges of a simple polygon is also known as the medial axis (or skeleton) [19].

A cell decompositionapproach first divides the free configuration spaceC f ree into simple, canon-

ical regions called cells and then represents the connectivity ofC f ree with a graphGcell, where each

node ofGcell represents a cell and two nodes that represent two adjacent cells are connected with an

edge. A path betweenqinit andqgoal is simply a channel of adjacent cells inC f ree. Compared to the

visibility graph approach and the Voronoi diagram approach, the cell decomposition approach is not

a complete approach, but aresolution-completeapproach (i.e., if a path exists, it can find it only if

the grid resolution is fine enough).

A potential fieldbased approach [45, 4] constructs an artificial potential functionU(q) over

the free configuration spaceC f ree to guide the robot fromqinit to qgoal, whereU(q) consists of

two components: an attractive componentUa(q) and and repulsive componentUr(q) (i.e.,U(q) =


Ua(q)+Ur(q)). Ua(q) pulls the robot towardsqgoal whileUr(q) steers the robot away from obstacles.

The main advantage of the potential field approach is thatU(q) specifies the motion of the robot

given anyq∈ C . However, sometimes the robot may fail to reachqgoal whenU(q) has local minima

and one of them traps the robot. If the robot is trapped, it can make some random moves to help itself

to escape from the local minimum [54]. Note that when the potential functionU(q) is constructed

over a grid,U(q) is essentially a heuristic function for graph search on the grid.

A recent potential field type approach (i.e., the continuum model [91]) defines potentials as an

implicit Eikonal equation and then solves it using an efficient method called theFast Marching

Method (FMM) [92, 80]. For a grid withN cells, the computational efficiency of the FMM is

O(N logN); it takesN steps to touch each grid cell, where each step takesO(logN). The FMM is

based on a node update formula which is consistent with the underlying continuous state space [1]

and consequently the solutions obtained by the FMM are better approximations of the continuous

and optimal paths [60] as compared to A* and Dijkstra’s algorithm. Thus, the FMM is preferable

over the latter algorithms. Because each potential field generated by the FMM has only one global

minimum at goal and henceno local minima, the continuum model can simply compute the optimal

path of an agent using gradient descent. Another benefit of the FMM is that its efficiency is not

affected by the complexity of (i.e. number and shapes of obstacles) of the virtual environment [69].

Finally, the underlying FMM [92] in the continuum model allows the continuum model to handle

circuitous routes (e.g., roundabouts) and hence enables the robot to follow the direction of the traffic

flow inside the roundabouts.

1.1.3 Random Sampling Based Approaches

During the past decade and a half, random sampling based approaches have become widely used be-

cause (1) they can handle high-dimensional configuration spaces and (2) they are easy to implement

due to the availability of collision checking libraries (e.g., RAPID – Robust and Accurate Polygon

Interference Detection [29], PQP – A Proximity Query Package [29, 53], PIVOT – Proximity In-

formation from Voronoi Techniques [35, 36]). For example, a probabilistic roadmap (PRM) [43]

planner SBL [76] (for Single-query, Bi-directional, Lazy in checking collision) spends an average

of 443 seconds on a PC with 1 GHz Pentium III processor and 1 GB RAM running Linux to solve

motion planning problems with 6 robot arms, where each arm has 6 DOFs (i.e., 36 DOFs in total).

Random sampling based approaches can be divided into two classes:multi-query planningand

single-query planning. A multi-query planner proceed in two phases: a construction phase and a


query phase. During the construction phase, the planner tries to capture the connectivity of the free

configuration spaceC f ree by samplingC at random and retaining the free configurations (i.e., the

milestones) as nodes in roadmap (graph)G. Each edge of the roadmapG represents a collision-

free straight-line path between the two configurations in the edge’s two nodes. Onceqinit to qgoal

are connected to the graph, the graph is searched using the Dijkstra’s shortest path algorithm in

the query phase to find a path betweenqinit and qgoal. The multi-query planning is suitable for

static environments, because once the roadmap is constructed, multiple queries can be performed

in the same environment quickly. However, if environments are dynamic, single-query planning

should be used instead. A single-query planner (e.g., Ariadne’s clew algorithm (ACA) [59] and

Rapidly-exploring Random Trees (RRT) [55, 51]) does not perform pre-computation to capture the

connectivity of the free configuration spaceC f ree. Instead it processes a single query as fast as

possible by building a small roadmap on the fly, where the roadmap is typically constructed by

expanding two trees (one rooted atqinit and the other rooted atqgoal) until one milestone in one tree

is connected with one milestone in the other tree.

In addition tocompletenessand resolution completeness, there is a third form of the notion

of completeness calledprobabilistic completeness: random sampling based approaches may not

terminate when there is no solution, but if a solution exists, the probability that the random sampling

based approaches find it converges to one as the time goes to infinity. In fact, both the multi-query

planners and the single-query planners converge exponentially [38, 52, 88] under some geometric

assumptions on the configuration space. A complete motion planner that combines approximate cell

decomposition with probabilistic roadmaps was presented in [98]. However, it has only been applied

to rigid robots with low DOFs (3-DOFs and 4-DOFs).

1.1.4 Motion Planning of Multiple Robots

The standard approaches for motion planning of multiple robots can be divided into two major

categories:centralized planninganddecoupled planning[54].

The centralized methods consider all agents as one robotic system with many degrees of free-

dom, and its time complexity is exponential in the dimension of the composite configuration space.

For example, the average runtime of the centralized planner SBL [76] for the simplest problem

in [76] increases from 0.26 sec to 28.91 sec when the number of robot arms increases from 2 to 6.

Thus, centralized planning is not suitable for motion planning of multiple robots with real-time con-

straints (e.g., motion planning of multiple agents in virtual environments such as computer games).


Decoupled planning is less expensive than centralized planning because the decoupled methods

break a problem into simpler sub-problems and hence lower dimensional spaces are searched, but it

is not complete and may fail to find solutions even if they exist [44] and the resulting paths can be

far from optimal. There are two main decoupled planning techniques:prioritized planning[21] and

path coordination[54]. In the former, one plans path for one robot at a time. Each robot treats the

robots for which the motions have been planned as moving obstacles. This requires a core planner

with dynamic obstacle avoidance [41, 37]. Runtime of a recent prioritized approach [93] grows

quadratically with the number of robots: if the number of robots isn, then a trajectory is planned

n times to avoidO(n) other robots. In the implementation of [93], the planner takes 2 seconds to

coordinate motions of 20 robots (roadmap construction time not included); beyond that number, it

slows down substantially due to quadratic dependence on the number of robots. A reactive style

prioritized method based on potential fields [45] was presented in [97]. Although it can operate

in real-time, it cannot handle problems involving highly concave obstacles. The path coordination

approach [62] first plans a path for each robot separately. The robots have to move along these fixed

independent paths, but their motions are coordinated to avoid mutual collisions using coordination

diagram. The path coordination approach proposed in [62] can only handle two robots. A method

based on a bounding box representation of the obstacles in the coordination diagram was proposed

in [83]. Since this method does not compute the exact shape of the obstacles, it is able to coordinate

motions of more than 100 robots. However, it cannot operate in real-time (for example, it needs 39.7

sec to coordinate motions of 32 robots).

1.1.5 Motion Planning of Multiple Robots in Dynamic Environments

There are two broad types of the planning problem in dynamic environments: the motions of the

obstacles are either known (i.e., given beforehand) or unknown (i.e., no prior information about the

movements of the obstacles).

When the dynamic obstacles’ motions are known beforehand, the concept of theconfiguration-

time spaceCT [54] can be used to solve the planning problem, whereCT is defined by incorpo-

rating timeT with the configuration spaceC as an additional dimension. For example, when the

dynamic obstacles are polygons and move piecewise-linearly in two-dimensional workspace, the

configuration-time space is three-dimensional and can be constructed explicitly. When the robot’s

velocity is unbounded, the planning problem can be solved in polynomial time using a vertical trape-

zoidal decomposition [77]. However, note that the motions of the dynamic obstacles in computer


games are normally not given beforehand.

When the motions of the dynamic obstacles are unknown beforehand, the motion planning prob-

lem of multiple robots becomes more complex. Algorithms for motion planning of multiple robots

in dynamic environments are divided into two classes:path modificationandreplanning.

Path modification methods such aselastic bands[71] andelastic strips[11] ensure a collision

free path by moving or deforming the path based on the movements of the dynamic obstacles. How-

ever, both elastic bands and elastic strips methods cannot handle changes in free-space connectivity.

A recent path modification method was presented in [23]. This physically-based, adaptive

roadmap based algorithm was implemented using a mass-spring simulation framework on top of

a roadmap-based motion planner to represent deformable links between nodes in a roadmap to cap-

ture the connectivity of the free space. Like other roadmap-based approaches, the algorithm in [23]

generates nodes randomly and its efficiency is affected by the complexity of (i.e. number and shapes

of obstacles) of the virtual environment [69]. Consequently, there is no guarantee on the optimality

of the resulting paths.

Instead of modifying the path, replanning methods replan at each step. Because the high cost of

performing the updates, the D* deterministic planning algorithms in [48, 84] make use of solutions

from previous steps instead of starting from scratch at each step.

A recent replanning method was presented in [87]. It constructsmulti-agent navigation graph

(MaNG) dynamically using discrete Voronoi diagrams computed on graphics hardware and the re-

sulting paths had maximal clearance. However, the maximal clearance is not always desirable be-

cause an agent may want to stay close to obstacles to avoid being detected by its enemies. Even

though graphics hardware can compute discrete Voronoi diagrams extremely fast, the performance

of [87] is limited by the fact that there exists overheads in transferring data to and from graphics

hardware.

As mentioned, the agent-based approaches combine the local collision avoidance with global

path planning and hence conflicts arise inevitably between the local goals and the global goals

(e.g., in the area of high congestion). The continuum model [91] (a replanning method) solves this

problem by unifying global path planning and local collision avoidance, because it is based on a

continuum perspective rather than per-agent dynamics as the name implies. The continuum model

can simulate motions of thousands of agents by grouping the agents together into a few (at most four

in [91]) groups, where all agents in each group share the same goal, and computing a single dynamic

potentialφ using the FMM for each group in each update cycle. From a planning perspective, use of

the continuum model for planning multiple agents is essentially a decoupled prioritized approach,


1

2

3

4

56

7

8

Sample No. 2100

(a) An environment without any narrow passages.

1234 5

6

Sample No. 1500

(b) An environment with one narrow passage.

Figure 1.1: The continuum model can compute collision-free paths for multiple agents in real-timewhen the agents are moving in an environment without any narrow passages (Fig. 1.1a), but it mayfail if the environment contains narrow passages (Fig. 1.1b). For example, as shown in Fig. 1.1b,agents #2 and #5 are in deadlock because agent #2 is moving right and trying to exit the narrowpassage while agent #5 just entered the passage and is moving left.

where all other agents are considered as obstacles for a given agent, albeit run at real-time rate. Even

though the continuum model could yield a solution in a large majority of cases, while retaining the

critical real-time character, it is not complete. Particularly, in constrained regions such as narrow

passages, it may not find a solution even if one exists. In our experiments (see Fig. 1.1), we found

that agents get stuck rather easily in narrow passages due to deadlocks.

1.1.6 Motion Planning of Multiple Robots as a Group/Formation

Multiple robots move sometimes as a group or even as a formation. For example, groups of agents

in RTT games move often in formations (i.e., they retain the “overall” geometric shape of the group,

as it moves around). In computer games, flocking [73] is widely used for simulating group move-

ment. To achieve flock-like motion, boids2 in the same flock try to stick together while avoiding

collision with each other and obstacles in their environment in order. Several steering behaviors

were introduced in [74] so that boids can achieve higher level goals. Global information in the

form of a roadmap of the environment was added in [7] to enable even more sophisticated flocking

2Generic simulated flocking creatures are calledboidsin [73].


behaviors (e.g., homing, exploring, and shepherding). Nevertheless, boids can easily get stuck in

cluttered environment because all boids act based on local information only and the group can be

broken up while moving towards its goal. When a formation in today’s RTT games needs to pass

through obstacles, the formation often breaks at one or multiple split points as shown in Fig. 1.2.

This dilutes the visual impact of seeing a formation move efficiently around the virtual environment.

split point

Figure 1.2: When a formation in [70] encounters an obstacle that’s within its front projection, theformation is split into two formations. As soon as these formations have passed the obstacle, theymerge back into one formation.

Two methods that guarantee coherence of the group were presented in [40, 39]. In [40], a group

of agents is enclosed by a deformable rectangle (i.e., the group shape) and PRM is used to plan

the global motions of the group shape. Local motions of the agents inside a deformable rectangle

are determined by group potential fields. The agents’ total motions are given by combining the

global motions of the groups and the local motions of the agents. Because this method is based on

PRM, it is not suitable for real-time applications such as computer games. The real-time method

presented in [39] finds first a path (called the backbone path) for a single agent. The backbone path

is then extended to a corridor using the clearance along the path. The agents can move freely inside

this corridor. A group region is a part of the corridor in which all agents must remain. Increasing

the maximum area of the group region allows the group to spread more along the backbone path.


Increasing the maximum corridor width makes the group wider, but decreases the longitudinal dis-

persion if the group region’s area is kept constant. An approach based on social potential force fields

is used in [39] to generate the paths inside the corridor. First, a repulsive force between the agents are

required to avoid collision between the agents. Second, a repulsive force from the boundary of the

corridor inward onto the agents forces the agents to stay inside the corridor. Third, a driving force

is applied to move the agents forward. Even though these two methods can guarantee coherence of

the group, they cannot move the agents in the group in formation.

Local motions of agents in a formation can be generated using one of three common formation

control approaches:the leader-follower approach[17], the behavior based approach[3], and the

virtual structure approach[56]. In the leader-follower approach, the leader moves along a path,

while the followers maintain desired distance and orientation to the leader. The leader-follower

approach is simple and has good scalability, but the approach cannot maintain the formation if a

follower is perturbed. The basic idea behind the behavior based approach is to prescribe several

desired behaviors (e.g., collision avoidance, formation keeping, target seeking) to each agent, and

the control action of each agent is then determined by the weighted average of the control for each

behavior. The main disadvantage of this approach is that it is difficult to explicitly define the group

behavior; hence, the behavior based approach is inadequate when the formation shape needs to be

changed (e.g., a formation may need to be deformed in order to pass through narrow passages). The

virtual structure approach to formation control [56] forces all robots in a formation behave as if they

are embedded in a single rigid structure and consequently the formation control is straightforward.

However, no automatic reconfiguring strategy was proposed in [56] to deform formations.

There has been some mention of a couple of ideas for motion planning of multiple groups of

agents (not in formations though) in [89], where each group of agents is modeled by the approach

presented in [39]. The first idea is based on prioritized planning for individual characters/robots [93].

Because all agents in a group have the same goal, the group can be represented by a single entity –

the group shape – that surrounds the agents in the group. The prioritized planner presented in [93]

is then used to plan the paths for multiple groups (i.e., the group shapes). In contrast to the first

idea, the second idea does not coordinate motions of different groups and paths for the groups are

planned independently. The motion for each group is planned using social force fields that move all

agents in the group along a backbone path with sufficient clearance, as in [39]. To plan motions of

multiple groups, an extra repulsive force between agents from different groups is added for collision

avoidance between those agents. To the best of our knowledge, these two ideas have not been

implemented.


1.2 Thesis Problems and Overall Solutions

As mentioned, the (basic) continuum model [91] is a real-time crowd simulation framework based

on the Fast Marching Method. The main advantage of the basic model is that it unifies global

planning and local planning (e.g., collision avoidance) and thus agents modeled by the basic model

do not face conflicting requirements between global planning and local planning. In addition, the

continuum model solves many challenges offered by RTT games:multiple agents (although the

continuum model can only plan motions of a small number of agents with independent goals in

real-time),real-time, dynamic, complexity, andinexpensive pre-processing. Therefore, the basic

continuum model is used extensively in this thesis.

However, the basic model may fail in certain constrained situations (e.g., when agents pass

through narrow passages as shown in Fig. 1.1b) to generate collision-free paths due to to deadlocks

amongst the agents. Another major drawback of the basic model is that it can only deal with a small

number of agents (or groups of agents) with independent goals. This is because it builds a potential

field (over the entire grid) for each separate goal, and the processing time for underlying computation

is such that only a few such fields can be built while maintaining real-time performance. Therefore

the basic model groups multiple agents together and and let them share the same potential field and

the same goal in order to maintain the real-time performance. Finally, the basic model is designed

for motion planning of multiple agents, and it does not address a key aspect of RTT games (i.e.,

coherence) and hence it does not plan motions of multiple formations, which are widely used in

RTT games.

We present three algorithms in this thesis to address these shortcomings of the basic continuum

model and we believe that they can be used as basic motion planning toolkits toward enhancing the

capabilities of RTT games. We assume in this thesis that all virtual environments are given as binary

occupancy grids.

1.2.1 Motion Planning in Static Environments with Narrow Passages

First, we propose to use a principled and efficient AI technique for decision making and planning

(i.e., Coordination Graph (CG) [34, 50]) to avoid such deadlocks in the narrow passages. One could,

in effect, think of placing an imaginary red-green traffic light at each entrance of a narrow passage.

If the light is red, all agents in the entrance have to wait until the light turns green, and only one of

them is allowed to enter the narrow passage at a time. Since only traffic lights that belong to the

same narrow passage and agents that are located in the same entrance need to coordinate, it is clear


that each traffic light/agent only has to coordinate with a small subset of other traffic lights/agents.

Such dependencies are exploited by coordination graphs. Furthermore, all traffic lights/agents in

a coordination graph share the same utility (i.e., the total utility or the value function [33]). This

is a so called coordination game (i.e., fully cooperative strategic game) [64], and the total utility is

assumed to be a linear combination of local value functions [33], where each local value function

involves only a few traffic lights/agents. The optimal joint action that maximizes the total utility is

then computed using the variable elimination algorithm [32]. The combination of the basic contin-

uum model and coordination graphs allows us to plan agents’ motions in a variety of complex and

constrained situations, such as narrow passages, intersections, roundabouts (i.e., follow the direction

of the traffic flow).

Overall, the algorithm runs as follows. In the pre-processing phase, we identify narrow passages,

entrances to the narrow passages, and open spaces using two elementary operations in mathematical

morphology,dilation and erosion, and theregion growing[28] procedure. At runtime, we use

the continuum model to plan motions of the agents by computing a single dynamic potentialφusing the FMM for each group in each update cycle and move each agent of the group opposite the

gradient of the potential function until all agents have reached their goals. Furthermore, we construct

coordination graphs and then compute optimal joint actions of the agents in the neighborhood of the

narrow passages to avoid deadlocks.

Because we have to process multiple coordination graphs in each update cycle, we speedup the

computation by processing them in parallel in a supervisor-worker paradigm on Symmetric Mul-

tiprocessing (SMP) systems. An SMP system has two or more identical processors connected to

a single shared main memory. We consider SMP systems because, first of all, the single shared

main memory allows the processors exchanging/sharing data efficiently. Moreover, our work is mo-

tivated by computer game applications. Although multi-core x86 processors are either dual-core or

quad-core for now, processor manufacturers are adding more cores to their multi-core processors

(e.g., Sun’s UltraSPARC T2 delivers eight cores). We show, via a parallel implementation on a SGI

UltimateVision with 24 processors, that our algorithm scales well with number of processors. Un-

fortunately, we could not render efficiently in 3D on the SGI UltimateVision because images/frames

must be sent through a rather slow network to our dual-core PC and displayed remotely. Conse-

quently, no rendering was done on the SGI UltimateVision. Hence we tested the entire algorithm

(i.e., including rendering) on the dual-core PC. Our current implementation can coordinate motions

of about ten agents in real-time (on the dual-core PC). In Fig. 1.3, our algorithm coordinated actions

of 8 agents (divided into 2 groups) in real-time for deadlock avoidance in the roundabout and its


entrances.

12

34

56

7

8

Sample No. 2000

Figure 1.3: Two groups of agents are moving in an environment with one roundabout and fourentrances. Agents must move counterclockwise inside the roundabout.

1.2.2 The Adaptive Multi-resolution Continuum Model

Next, we present an adaptive extension to the basic continuum model [91] to the problem of real-time

motion planning of multiple agents in two dimensional dynamic virtual environments. In this adap-

tive approach, the potential field computation for each agent is restricted to a channel (i.e., a subset

of the environment), which is constructed/updated at a very low frequency, or whenever it is blocked

by some dynamic obstacles (e.g., doors). This affords us a substantial speedup in the algorithm, and

maintains the advantage of the continuum model such as unified global planning and collision avoid-

ance, and its ability to handle dynamic obstacles. We further present a multi-resolution extension

of our adaptive approach. Simulations show that that our adaptive multi-resolution approach, while

maintaining a real-time performance, (i) handles a significantly larger number of individual agents

with independent goals, (ii) handles bigger grids than the continuum model, and (iii) provides a

natural way to steer agents from narrow passages to open spaces.

In our adaptive extension to the basic continuum model, the potential field computation (and the


resulting path) for each agent is restricted to achannel[67], a subset of the grid cells of the environ-

ment. This reduces the domain over which the potential field is constructed, thereby significantly

speeding up the computation. The intuition behind our adaptive approach is that once an agent’s

global path is set, there is no need to update it as frequently as the agent’s local path. The local

path has to be updated in each simulation cycle for obstacle avoidance, but the global path only

changes gradually (unless it is suddenly blocked by dynamic obstacles). The construction/update of

the channel itself requires (i) computing the potential field over the entire domain, (ii) computing

a backbone path derived from the potential field by following the negative of its gradient, and (iii)

growing the backbone path by a given margin. However, it is done at a very low frequency (less

than once per second) or whenever the channel becomes invalid (e.g., when it is blocked by some

dynamic obstacles or a change in the environment, such as a door closing). Note that the notion of

channels is very similar to the notion of corridors in [13, 24, 39], where each corridor is also con-

structed by growing a free region on either side of a backbone path. In [67], a channel was used to

control the amount of data to be searched, while a corridor in [13, 24, 39] was used to allow an agent

deviate locally from the global path (e.g., to avoid collisions with dynamic obstacles). Channels in

this thesis serve both purposes.

The adaptive continuum model is subsequently extended by constructing channels around back-

bone paths found at coarser resolution grids, and then planning motions of the agents at theoriginal

higher resolution grid. Although our original motivation behind the multi-resolution extension was

further speedup in computation as in [67], the extension has a nice intrinsic advantage in that it

prefers open areas because narrows passages may appear as blocked at the coarse levels. This is a

desirable trait since otherwise deadlocks may occur in narrow passages. Note that the coarse level

grids are only used for agents that need to avoid narrow passages. If a path is not found at a coarse

level, we move on to the next higher resolution level until a path is found (if it exists). Therefore, our

method will not fail even if the paths that pass through narrow passages are the only options. We use

wavelets [18] for representing multi-resolution grids due to its flexibility. Because we assume that

virtual environments are given as occupancy grids, we use the simplest form of wavelets, the un-

normalizedHaar basis. However, if the environments are natural, rough terrains (e.g., mountains),

we can simply switch to more advanced wavelets with better approximation properties [67].

We have implemented this adaptive multi-resolution continuum model and our simulations show

that that our approach can plan motions of a larger number of individual agents on larger grids

compared to the basic continuum model without any pre-processing at all. Via simulations, we

show that our current implementation can plan motions of 40 agents on 256×256 grids in real-time


on a standard dual-core PC. For example, 20 agents’ paths constructed by our approach are shown

in Fig. 1.4, where the agents’ start positions and goal positions are marked as grey circles (green

on color prints) and black discs, respectively, along with their IDs and static obstacles are drawn in

dark grey (red on color prints).

1

2

3

4

56

7

8

9

10

11

12

13

14

15

16

17

18

19

20

1 2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Sample No. 2000

Figure 1.4: Paths for 20 agents (on a 64× 64 grid) constructed by our adaptive multi-resolutioncontinuum model. The simulation cycle ran at 40 hertz (i.e., five times faster than the basic model).

1.2.3 Flexible Virtual Structures and Motion Planning of Multiple Formations

Finally, we propose a new approach to the problem of coordinating multiple agents to move in a

tightly controlled formation in a two-dimensional workspace, where each formation is modeled by

our flexible virtual structure approach for formation control. All agents in the formation are con-

ceived as being located on the elastic shape that can be deformed in a controlled/ordered manner

and in real-time by simply displacing multiple control points/nodes placed on the elastic shape.

The deformation of the elastic shape induces a natural and intuitive looking deforming of the for-

mation, which is a key strength of our approach. While there are many methods for modeling of

deformable objects (see the background section for a brief recapitulation of these methods), we

chose the Boundary Element Method (BEM) [9], a well-known physically based algorithm, primar-

ily because it enables us to deform the formation in a controlled and ordered manner and in real-time


by displacing a few control nodes.

Thisflexible virtual structure approach for formation control is then integrated with the contin-

uum model [91] to plan the motion of a formation in real-time in virtual environments with static and

dynamic obstacles. To plan motion of a formation, we reduce the formation to a point by growing

the obstacles (a standard technique in path planning literature [15]) and then apply the continuum

model to compute a potential field. From the gradient of the potential, we can generate a sequence

of waypoints for each agent of the formation and then usesocial potential fields[3] to move the

agent between its waypoints.

Please note that even though the continuum model can plan motions of thousands of agents (as

long as they are divided into a few groups, as mentioned above), the agents in each group do not

behave as if they are in a formation, since there is no such constraint. In fact, agents in the same

group can be far away from each other and the continuum model requires only that they share the

same goal position. By combining the continuum model and our flexible virtual structure approach

for formation control, we can plan motions of multiple formations in real-time, while each formation

deforming smoothly and in a controlled manner, as needed to navigate through narrow passages. If

the planner is not able to plan the motions of the formations, they are allowed to pass through each

other (e.g., when they have to pass through the same narrow passage), and our approach usessocial

potential fields[3] to guarantee that agents in those formations do not collide with each other. In the

experiments, we plan motions of multiple (at most four) formations, where many formations have

around 20 agents each. We chose this size because it is consistent with RTT games where platoons

of soldiers move in formations (see, for example game “Empire Earth II”). Although our approach

is easily applicable to much larger formations. For example, paths for 4 different formations are

shown in Fig. 1.5. Note that the formations changed their orientations gradually and two of them

deformed during the simulation.

1.2.4 Path Quality

In motion planning literature, a short path is often treated as a good path because it takes longer to

execute a longer path [25]. For example, sampling-based planners create low-quality paths contain-

ing unnecessary and jerky motions due to their probabilistic nature, and many techniques have been

proposed to reduce length of path generated by these planners [25]. However, a short path is not

always a good path (e.g., in the thesis), because the shortest path for an agent may take it through

areas with hazardous conditions (e.g., areas such as narrow passages controlled by the enemies).


1

2

3

4

1

2

3

4

Sample No. 4041

Figure 1.5: Paths for four formations in a virtual environment (64×64 grid) without static obstacles.Each formation’s initial configuration and final configuration are shown as a number of black circlesand black discs (which represent the agents), respectively.

Instead, the optimal path of an agent in this thesis is the one that minimizes a linear combination of

1) path length, 2) time to the goal, and 3) discomfort per time unit along the path (see Chapter 2).

One could define and then optimize a quality measure3 of the composite path. Our focus, from

a gamer’s point of view, has been on real-time behavior (e.g., the motion planner must react imme-

diately when dynamic obstacles appear / disappear). In addition, in the motion planning literature,

often the effort is devoted to finding a collision-free path (a hard enough problem), and not the qual-

ity of path per se, which, it is assumed, a subsequent optimization step would yield. Consequently,

we have not attempted to define and optimize any quality measure of the composite path in this

thesis.

3In many cases, a single scalar-valued function (e.g., the average arrival time of the agents or the arrival time of thelatest agent) is optimized [96].


1.3 Thesis Contributions

1. Motion Planning in Static Environments with Narrow Passages

We integrate coordination graphs with the continuum model and thus address a major draw-

back of the continuum model [91]: it may fail to generate collision-free paths due to deadlocks

amongst the agents in certain constrained situations such as when agents pass through narrow

passages. Our current implementation can coordinate motions of about ten agents in real-time

on a dual-core PC.

2. The Adaptive Multi-resolution Continuum Model

We propose an adaptive multi-resolution extension to the continuum model to plan motions

of multiple agents with independent goals (as compared to the basic continuum model). This

speeds up potential field computation for each agent by restricting the computation to a chan-

nel (i.e., a subset of the environment). As a result, the adaptive multi-resolution continuum

model can plan motions of a larger number of agents with independent goals as compared

to the basic continuum model. Our current implementation can handle up to 40 agents with

independent goals in real-time, whereas the basic model can handle only about four [91]. Fur-

thermore, the multi-resolution part of our continuum model can steer the agents away from

narrow passages to open spaces.

3. Flexible Virtual Structures and Motion Planning of Multiple Formations

We propose a new approach to the problem of coordinating multiple agents to move in a tightly

controlled formation in a two-dimensional workspace. Agents in a formation are conceived

as being part of a two-dimensional elastic shape modeled using the boundary element method

(BEM), and the formation can be deformed in a natural and controlled manner and in real-time

by displacing a few control nodes on the elastic shape. Next, we present a novel approach

for real-time motion planning of multiple formations in virtual environments with dynamic

obstacles by combining the continuum model and our virtual structure approach for formation

control in virtual environments. To the best of our knowledge, this motion planning algorithm

for multiple formations is the first one that does not use ad-hoc and local approaches and

hence agents in a formation do not split easily from the formation.


1.4 Thesis Outline

We introduce the Fast Marching Method (FMM) [92, 79] and the (basic) continuum model [91]

we will use throughout this thesis in Chapter 2. In Chapter 3, we use Coordination Graph (CG) in

conjunction with continuum model to avoid deadlocks in the narrow passages. In Chapter 4, we

build upon the basic continuum model and present an adaptive multi-resolution continuum model

to the problem of motion planning of multiple agents in virtual environments, where each agent

has its own goal. In Chapter 5, we present a virtual structure approach for formation control in

virtual environments. Next, we present an approach for real-time motion planning of multiple for-

mations in virtual environments with dynamic obstacles by combining the basic continuum model

and our virtual structure approach. We close the thesis with concluding remarks and future work in

Chapter 6.

Chapter 2

Preliminaries

We explain theFast Marching Method(FMM) [92, 79] and theContinuum Model[91] in this chap-

ter. The FMM is a consistent and accurate numerical algorithm based on entropy-satisfying up-

wind scheme and fast sorting technique for solving the nonlinear Eikonal equation. The FMM

is also highly efficient and it can solve the Eikonal equation on a rectangular orthogonal mesh in

O(N logN), whereN is the total number of grid cells. The continuum model is a real-time crowd

simulation framework based on the Fast Marching Method. The main advantage of the continuum

model is that it unifies global planning and local planning (e.g., collision avoidance) and thus agents

modeled by the continuum model do not face conflicting requirements between global planning and

local planning.

2.1 The Fast Marching Method

Given a two-dimensional virtual environment and an agent’s initial positionxa and (global) goalgb1,

the optimal pathp is the one that takes the agent fromxa to gb while minimizing∫

pCds, where the

unit cost fieldC is an anisotropic field (i.e.,C depends on both position and direction) [91]. Suppose

that there is a potential functionφ(x), wherex represents position. The gradient descent contours of

φ(x) that satisfies the Eikonal equation

‖∇φ(x)‖ = C, C > 0, φ(gb) = 0 (2.1)

1The Fast Marching Method does not restrict the goal to a single position. For example, the goal in Section 2.2 isgiven as a collection of grid cells.

22

CHAPTER 2. PRELIMINARIES 23

are the optimal paths [47], whereφ is a scalar field and it only depends on position (i.e.,x). The

right-hand side of (2.1),C > 0, and the boundary condition thatφ equal 0 at the agent’s (global)

goalgb) are the inputs to the Eikonal equation and are assumed to be known.

Consider the case of

‖∇φ(x)‖ = 1, φ(gb) = 0 (2.2)

If the agent’s goalgb is a single point, the solution to this Eikonal equation withC = 1 on the right-

hand side is just the distance togb and it can be easily constructed by expanding a boundary curve

from gb (i.e., initially, the boundary curve is a circular curve located atgb) in a normal direction

with unit speed becauseC controls the expanding speed of the boundary curve and it is equal to

1 everywhere in this case. However, whenC is not equal to 1 everywhere, this boundary curve

may eventually cross itself and we get a multi-valued “swallowtail” solution [80]. The solution we

are looking for is the one that satisfies the entropy condition (i.e., no multi-valued “swallowtails”).

This entropy-satisfying solution can be constructed by imagining the boundary curve as a source

for a propagating flame and requiring that a point in the virtual environment stays burnt once the

expanding front passes it and ignites it.

Another way to obtain this entropy-satisfying solution comes the mathematical theory ofvis-

cosity solutionsor (weak solutions). Viscosity is a measure of the resistance of a fluid to being

deformed and it is commonly perceived as “thickness” or resistance to flow. Here, the idea of vis-

cosity is used to smooth out the corner in the propagating boundary curve by adding a smoothing

term to the Eikonal equation (2.1). Consider the associated “viscous” partial differential equation

given by

‖∇φ(x)‖ = C+ ε∇2φ(x) (2.3)

It is shown in [81] that the viscous termε∇2φ(x) smooths out sharp corners in the solution and

eliminates multi-valued “swallowtails” asε → 0.

The Fast Marching Method is a numerical method that automatically extracts this viscous limit.

In fact, there are two different Fast Marching Methods: Tsitsiklis’ algorithm [92] and Sethian’s

algorithm [79]. We use Tsitsiklis’ algorithm in this thesis because it is efficient when agents must


follow circuitous rounds such as roundabouts [91].

Before we can use the Fast Marching Method to compute an approximation of the solution

to the nonlinear Eikonal equation (2.1), the two-dimensional virtual environment is meshed into a

rectangular orthogonal mesh withN grid cells. Note that the scalar fieldφ is stored inside each grid

cell, whereas the anisotropic fieldC is stored at each face between two neighboring grid cells. When

computing a anisotropic field (e.g.,C), we have to iterate over each grid cell and then over each of

the four faces of the grid cell. For example, when computingC for grid cellM in Fig. 2.1, we iterate

over the grid cell’s four faces to computeCM→i , wherei ∈ {N,S,W,E}. Note that other scalar fields

and anisotropic fields in the figure are used by the continuum model.

g,φ

fM→E,CM→E, fE→M,CE→M,∇φ

N

W M

S

E

Figure 2.1: Discretized grid structure [91], where the scalar fields g andφ are stored inside eachgrid cell, whereas the anisotropic fields such asf , C, and∇φ are stored at each face between twoneighboring grid cells. The fieldsf , g, andC are set in Section 2.2, whereasφ is the solution to (2.1)computed using the FMM.

Suppose that we want to solve the Eikonal equation (2.1) for cellM in Fig. 2.1, we computeφM

using the finite difference approximation to (2.1) (see [91]):

(

φM−φmx

CM→mx

)2

+

(φM−φmy

CM→my

)2

= 1 (2.4)

wheremx andmy represent the grid cells in theupwinddirection (i.e., they are the less costly adjacent

grid cells (of cellM) along thex- andy-axes, respectively).

mx = argmini∈{W,E}

{φi +CM→i}, my = argmini∈{N,S}

{φi +CM→i} (2.5)


Thex-dimension (ory-dimension) is dropped from (2.4), ifmx’s both neighbors (ormy’s both neigh-

bors) have infinite cost and consequently not defined.

Since the finite difference approximation in (2.4) is an upwind approximation, the solution at cell

M (i.e.,φM) depends only on the smaller adjacent values. This causality relationship is the key to the

efficiency of the Fast Marching Method because this relationship allows us to build the solution in

the order of increasing values ofφ . OnceφM is set, the gradient∇φ is simply the difference between

φM and the potentials at the neighboring grid cells in the upwind direction.

The Fast Marching Method is described in Algorithm 1. An efficient mini-heap structure is used

by the FMM to find theCANDIDATEcell with the lowest value ofφ . Given a heap withN elements,

the mini-heap structure can change any element and reorder the heap inO(logN) steps. Since it

takesN steps to touch allN grid cells, where each step takesO(logN), the computation efficiency

of the Fast Marching Method isO(N logN).

Algorithm 1 : The Fast Marching Method

Assignφ the value of 0 inside the agent’s goalgb and tag the grid cells in the goal as1.1

KNOWN;Assign the value of∞ to all other grid cells and tag those cells asUNKNOWN;1.2

Tag theUNKNOWNcells adjacent to allKNOWNcells asCANDIDATEcells;1.3

Computeφ of those newly taggedCANDIDATEcells by solving a finite difference1.4

approximation to the Eikonal equation (2.1);repeat1.5

Tag theCANDIDATEcell with the lowest value ofφ asTRIAL ;1.6

Add all UNKNOWNcells adjacent to theTRIAL cell to theCANDIDATEset, then1.7

remove them from theUNKNOWNset;Add theTRIAL cell to theKNOWNset, then remove it from theCANDIDATEset;1.8

Recompute the values ofφ at allCANDIDATEcells adjacent to theTRIAL cell;1.9

until all grid cells are KNOWN;1.10

The Fast Marching Method is very similar to Dijkstra’s algorithm [20], which is a graph search

algorithm for computing shortest paths on a discrete graph. Both the FMM and Dijkstra’s algorithm

propagate the values in the same order: Dijkstra’s algorithm also keeps track of the “current smallest

cost” for reaching a graph vertex and uses it to update the adjacent graph vertices that are directly

connected to the graph vertex with the smallest cost by graph edges. However, the FMM exploits

this idea in the context of an approximation to the underlying partial differential equation and uses

an update formula that is consistent with the underlying continuous state space. For example, paths

generated by Dijkstra’s algorithm are restricted to graph edges, whereas paths generated by the FMM


can pass on mesh edges. Thus, the FMM tends to produce shorter paths than Dijkstra’s algorithm

(e.g., the FMM is more accurate for the Euclidean norm) [1].

2.2 The Continuum Model

Given the unit cost fieldC, the Fast Marching Method computes the potentialφ in O(N logN), where

N is the number of grid cells. An agent can then move from anywhere (not just its initial position

xa) in the virtual environment in the direction opposite the gradient∇φ until it reaches the goalgb

(given as a collection of grid cells in [91]), because potentials generated by the FMM are free of

local minima analytically. Thus, a group of agents can share the same potential field as long as they

all have the same goal. This is utilized by the continuum model [91] to plan motions of a crowd of

agents.

In the continuum model, the unit cost fieldC in [91] is chosen in such a way that the optimal

path of the agent minimizes a linear combination of 1) path length, 2) time to the goalgb, and 3)

discomfort per time unit along the path, that is,

C = α + β1f+ γ

gf

(2.6)

whereα , β , andγ are weights,f is thespeed field, andg is thediscomfort field. Both fields are

user assigned as follows. The discomfort fieldg is a scalar field. Given two positionsx1 andx2

and the discomfortsg(x1) andg(x2), whereg(x1) < g(x2), an agent prefers the position with the

lower discomfort (i.e.x1), all other things being equal. For example, a discomfort is added to the

front of each agent so that two agents anticipate one other when they cross. The speed fieldf is an

anisotropic field and its value depends not only on the terrain, but also on the density of the agents

in the neighborhood. At low density,f depends only on the terrain and is equal to thetopographical

speed fT . When the density of the agents in the neighborhood increases,f becomes more and more

dominated by the movements of the neighboring agents.

To simulate motions of a crowd of agents, the continuum model first divides the agents into a

few groups before it constructs a unit cost fieldC and then computes a single dynamic potential

φ for each group in each simulation cycle by solving the Eikonal equation (2.1) using the FMM.

An optimal path for each agent can then be determined by backtracking from any position (e.g., its

initial position xa) to its goalgb by moving opposite the gradient of the potential function∇φ(x)

scaled by the speed at that point:


x =− f (x,θ)∇φ(x)

‖∇φ(x)‖(2.7)

where x and f (x,θ) denote the velocity and the speed evaluated in the direction of motionθ ∈{N,S,W,E}, respectively. Because speedf and discomfortg of one group affect all other groups,

motions of agents in different groups are coupled.

The continuum model for crowd simulation is outlined in Algorithm 2. Note that the last step

(i.e., minimum distance enforcement) is necessary because the continuum model without the mini-

mum distance enforcement step can only ensure that no two agents intersect up to the resolution of

the grid, and therefore two agents may collide if they are located in the same grid cell. To enforce a

pair-wise minimum distance between the agents, the continuum model simply iterates over all pairs

within a threshold distance and pushes them apart.

Algorithm 2 : The Continuum Model for Crowd Simulation

foreach simulation cycledo2.1

Construct the density field;2.2

foreach groupdo2.3

Construct the unit cost fieldC;2.4

Construct the potentialφ and its gradient∇φ ;2.5

Update agents’ locations;2.6

end2.7

Enforce the minimum distance between the agents;2.8

end2.9

Chapter 3

Motion Planning in Environments with

Narrow Passages

The continuum model [91] can plan motions of multiple agents at interactive rates, but it may fail

to generate collision-free paths due to deadlocks in certain constrained situations (e.g., in narrow

passages) as shown in Fig. 1.1b where two agents are in deadlock in the narrow passage. In this

chapter, we combine the continuum model with coordination graphs (CG) [34, 50] by designing

and incorporating rules into the coordination graphs to coordinate motions of agents at runtime

and locally to avoid deadlocks in constrained regions, such as narrow passages. These rules are

easily interpretable and very flexible (i.e., existing rules can be easily removed and new rules can

be easily added). Because we have to process multiple coordination graphs in real-time in order

to obtain optimal joint actions for deadlock avoidance, we speedup the computation in this chapter

by processing the graphs in parallel on Symmetric Multiprocessing (SMP) systems. Our current

implementation can coordinate motions of about ten agents in real-time on the PC.

This chapter is organized as follows. We formally define the problem to be solved in Section 3.1.

We explain the concept the continuum model in Section 3.2. The overall solution is presented

in Section 3.3. We explain in Section 3.4 how to extract information (i.e., open spaces, narrow

passages, and entrances) from a given environment. Coordination graphs are used in Section 3.5 to

coordination motions of the agents for deadlock avoidance. A parallel procedure for construction

of multiple coordination graphs and computation of multiple optimal joint actions is presented in

Section 3.6. We explain also how to decouple the rendering from the planning in that section. In

Section 3.7, we describe the experiments, and the results obtained. We conclude in Section 3.8.

28

CHAPTER 3. MOTION PLANNING IN ENVIRONMENTS WITH NARROW PASSAGES 29

3.1 Problem Definition

Let k groupsG1, . . . ,Gk of robot agents be in a static two-dimensional environment (given as a

binary occupancy grid), where each agent has three degrees of freedom (two for translation and

one for rotation) and the radius of the bounding circle for the agent isr. The agents in each group

have different initial positions, but share the same goal (given as a collection of grid cells).We

assume that the environment/workspace consists of open spaces and narrow passages that connect

the open spaces to each otherand hence we do not consider workspaces that consist solely of

narrow passages (e.g., mazes). We define a narrow passage as region where width is such that two

agents cannot pass side by side. In our implementation, we assume that the width of a grid cell is

less than 4r. Hence, a passage is considerednarrow if it is one grid cell wide (this can be easily

adapted for more general cases though). As shown in Fig. 3.1, we consider four different types

of narrow passages in this thesis:one lane passage, 3-way intersection, 4-way intersection, and

circular intersection/roundabout. The task is to plan for each agent, in real-time and with a very

short (a couple of seconds) preprocessing phase, a path that takes the agent from its initial position

to its goal without colliding with other agents and static obstacles. Note that a short preprocessing

phase is essential for motion planning in computer games, because virtual environments in games are

often generated automatically during the preprocessing phase and game players may only tolerate a

few seconds of delay.

Figure 3.1: Narrow passages (from left to right): one lane passage (e.g., one lane bridge), 3-wayintersection (e.g., T intersection, Y intersection), 4-way intersection (i.e., crossroad), and circularintersection (i.e., roundabout).


3.2 Background

3.2.1 Coordination Graphs

Suppose we have a collection of agentsA = {A1, . . . ,Am}. All agents are acting in a space described

by a set of discrete state variables,X = {X1, . . . ,Xn}. A statex = {x1, . . . ,xn} is an assignment

to each state variableXi. Each agentA j chooses an actiona j from a finite set of possible actions

Dom(A j). We want to select an optimal joint actiona= {a1, . . . ,am} that maximizes the total utility

(or value function)Q = ∑ j Q j(x,a), whereQ j(x,a) is agentA j ’s local value function [33]. The

local value functionQ j(x,a) consists of a set of value rules. A value rule describes a context — an

assignment to state variables and actions — and a value increment (positive or negative reward). If

a context applies, the corresponding value increment is added to the total utility. In this chapter, all

action and state variables are binary variables. Negates ofa andx are denoted as ¯a (or¬a) andx (or

¬x), respectively.

Each node of a coordination graph represents one agent. There is a local value function for each

agent. Local value functionQ j is influenced by action of agentA j and possibly other agents. If

agentAi influences agentA j ’s local value functionQ j , then nodej has an incoming edge from node

i. Agent A j is the child agent and agentAi is the parent agent. This means that agentA j has to

coordinate its action with agentAi. A coordination graph captures local coordination requirements

between agents, because two agents need to coordinate their actions if and only if the corresponding

nodes are interconnected.

A sample coordination graph for a coordination problem with 4 agents is shown in Fig. 3.2.

AgentA1 has to coordinate with agentsA2, A3, andA4. Since node 2 and node 3 are not connected

by an edge, agentsA2 and A3 do not need to coordinate with each other. AgentsA3 and A4 do

not need to coordinate with each other either. The coordination dependencies are represented by

directed edges. For example, node 2 is the child and node 1 is the parent, implying that decisions

made by agentA1 affect agentA2.

For the example in Fig. 3.2, agentA1’s local value functionQ1 is

〈a1∧a3∧x : −100〉

〈a1∧a4∧ x : 100〉

wherea1, a3, anda4 are actions of agentsA1, A3, respectiveA4 andx is a state. Actions of agents

A3 andA4 affect agentA1, therefore there are two incoming edges to node 1, one from node 3, and


1 2

3 4

Figure 3.2: Coordination graph for a coordination problem with 4 agents.

the other from node 4. If agentA1 observes that the current state isx, the second value rule does not

apply anymore and can be removed. This leaves us with only the first value rule and it states that the

total utility will be reduced by 100 ifa1 = true anda3 = true.

To find an optimal joint action, we delete all value rules which do not apply after the current

states have been observed. An efficient variable elimination algorithm is then used to find an optimal

joint action of the agents in combination with a message passing scheme [34]. Nodes are eliminated

one by one from the graph. Each node collects all rules that involve it from its neighbors and

computes its optimal conditional strategy. Then the optimal conditional strategy is distributed to the

node’s parents. The elimination order does not affect the result (i.e., optimal joint action). When the

last node is removed, a second pass is performed in the reversed order where each node distributes

its decision to its neighbors and this enables the neighbors to fix their final strategies. During the

process, some new rules are introduced. The cost of the whole algorithm is polynomial in the

number of new rules generated [33]. Note that the variable elimination algorithm is an exact method

that always reports the optimal joint action that maximizes the total utility, even if the coordination

graph has cycles [49]. However, for connected graphs with cycles, the worst-case complexity of

the variable elimination algorithm is exponential in the number of nodes [49]. We refer the reader

to [34, 49] for details on computing optimal joint actions using the variable elimination algorithm.

3.3 Overall Solution

The overall motion planning algorithm that combines the continuum model with coordination graphs

is given in Algorithm 3. Recall that we have to enforce the minimum distance between two agents

because the continuum model only guarantees that they do not intersect if they are located in two

different grid cells.


Algorithm 3 : Motion Planning of Multiple Agents in Static Environments with NarrowPassages

/ * Preprocessing * /Identify all open spaces, narrow passages, and entrances;3.1

/ * Runtime * /foreach update cycledo3.2

forall groupsdo3.3

Construct the unit cost fieldC;3.4

Construct the potentialφ and its gradient∇φ ;3.5

end3.6

forall agentsdo3.7

Observe the agent’s state;3.8

end3.9

Construct coordination graphs and compute optimal joint actions;3.10

Update the agents’ locations;3.11

Enforce the minimum distance between agents;3.12

Render in 3D;3.13

end3.14

3.4 Information Extraction from Binary Occupancy Grids

We model the environment as a binary occupancy grid (i.e., the grid cells have values in{0,1}). The

grid cells that have value 0 represent free areas, whereas the grid cells that have value 1 represent

obstacles. We use a slightly modified version of the topology-based maps extraction technique —

a combination of morphological operations and morphological watersheds [28] — proposed in [22]

to extract information from a grid (i.e., identify open spaces, narrow passages, and entrances to the

narrow passages). It essentially works as follows. First, we apply a morphological closing operation

to the binary occupancy grid that represents the environment. Denote the grid asI , then the closing

of I by astructural element Kis the erosion of the dilation ofI ,

I •K = (I ⊕K)⊖K (3.1)

where⊕ and⊖ denote the dilation and erosion, respectively. This results in closing all narrow

passages and separating the free space in the environment into distinct connected components of

open spaces, which are identified by using theregion growing[28] procedure. Narrow passages are

then determined by appropriate difference operations over these computed maps.


We choose a 2×2 structural element Kfor both dilation and erosion operations:

K =

1 1

1 1

(3.2)

The environment in Fig. 3.3a has three open spaces connected by two narrow passages. The nar-

row passage on the left hand side has three entrances, and the one on the right hand side has two

entrances. We first compute the dilationD of I by K (i.e., D = I ⊕K) which closes all narrow

passages and creates three separate open spaces (Fig. 3.3b). Note that some cells in open spaces

are also marked as obstacles by the dilation step. Next, we compute the erosionE of D by K (i.e.,

E = D⊖K) to remark those cells as free (Fig. 3.3c). Observe that the narrow passages are still

closed in Fig. 3.3c. Both dilation and erosion takeO(N) time, whereN is the number of grid cells.

In the erosionE of D, only grid cells belonging to open spaces are marked as free. To distinguish

the three open spaces from each other, we use the region growing procedure by starting with a set

of “seed” grid cells chosen randomly from the black cells in Fig. 3.3c and from them grow three

regions by appending to the seeds those neighboring grid cells that have the same gray level (i.e.,

black) as the seeds. The free grid cells in Fig. 3.3a, that do not belong to the open spaces, belong of

course to one of the two narrow passages (Fig. 3.3e), and they can be easily identified by performing

difference operation on mapsI (Fig. 3.3a) andE (Fig. 3.3c).

To identify all entrances to a narrow passage, we consider the narrow passage’s neighboring

open spaces in turn. For a pair of narrow passage and open space, we first find the grid cellps in

narrow passage (note that we consider a passage narrow if it is one grid cell wide) that shares a

common side with one of the grid cells in the open space. The entrance to the narrow passage is

the set of the open space grid cells inside a disk with centerps and radiusε > 0 (we setε to 5 grid

cells). We allow entrances to intersect, because each entrance is uniquely defined by a pair of narrow

passage and open space. For example, among the five entrances shown in Fig. 3.3f, two entrances

intersect each other and share some common (marked with white) grid cells, but those two entrances

belong to two different narrow passages.

An environment with a roundabout is shown in Fig. 3.4a. We can easily identify the four open

spaces, the four entrances, and the single narrow passage using the mathematical morphology and

the region growing procedure as outlined above. The computed single narrow passage is shown in

Fig. 3.4b and it consists of a square ring and two additional grid cells at each side of the square ring

that connect the square ring to one of the four open spaces. To check whether the narrow passage


is a roundabout, we find an axis-aligned bounding box (AABB) that bounds the narrow passage.

Using the same approach we used to identify the open spaces, we segment the non-narrow passage

grid cells inside the AABB into (five) connected components of grid cells. For each connected

component, we check whether a grid cell in that connected component shares a common side with a

grid cell in the AABB. If none of the grid cells in the connected component is in the neighborhood

of the AABB, the narrow passage is a roundabout. For example, the narrow passage in Fig. 3.4b is

identified as a roundabout because the grid cells inside the square ring in Fig. 3.4b are surrounded

by the narrow passage and they do not share a common side with any grid cells in the AABB.

We briefly mention how we set the topographical speedfT here. Since we assume that each grid

cell represents either an obstacle or a flat free space,fT in this chapter is eitherfree speed ff ree or

0, where f f ree > 0. Recall thatfT is an anisotropic field and it is stored at each face between two

neighboring grid cells. When settingfT , we iterate over each grid cell and then over each of the

four faces of the grid cell.fT for one face is set tof f ree if and only if both the grid cell and the

neighboring grid cell represent free space, otherwise it is set to 0. In addition,fT is used to force

agents to follow the traffic flow (clockwise or counterclockwise) inside a roundabout by settingfT

for the face of each roundabout grid cell that faces the opposite direction of the flow to 0.

3.5 Motion Coordination with Coordination Graphs

As mentioned in the introduction, deadlocks are avoided via “traffic control” using coordination

graphs at the entrance to narrow passages via “imaginary traffic lights”. The status of lights at all

the entrances to the same narrow passage is coordinated so that at one time, only one of these lights

is green. A robot agent at an entrance is only allowed to enter the narrow passage if the traffic light

at that entrance is green. If there are multiple agents at the entrance, the closest one to the entrance

is allowed to enter the narrow passage. The rest of the agents at the entrance must wait (i.e., we stop

these agents by setting their velocities to zero). This is implemented via the value rules associated

with each coordination graph as explained below.

3.5.1 Construction of Coordination Graphs

For each narrow passage, we create one coordination graph for controlling the “traffic lights” and

one coordination graph per entrance for controlling motions of agents. So, for a narrow passage

with Ne entrances, we will haveNe+1 coordination graphs in total.


Coordination Graph for Traffic Lights

Given a narrow passage, the imaginary traffic light placed atith entrance of a narrow passage is

denoted byAi, and is controlled as follows.

It has one boolean actionai :

• turn green(i): it changes the color of the traffic light to green.¬turn green(i) indicates that

the color is switched to red.

There are two boolean state variables associated with it:

• is green(i), indicates that the color of the traffic light at entrancei is green. ¬is green(i)

indicates that the traffic light is red.

• entranceis empty(i), indicates the entrancei is empty. Observe that only agents that are

heading toward the narrow passage (easily determined by following opposite the gradient of

the potential field generated by the FMM) connected to the entrancei are considered. If

all agents in the entrancei are moving toward the neighboring open space, the entrance is

considered to be empty.

In addition, there is one boolean state variable associated with the narrow passage connected to

all its entrances, denoted by narrowpassageis empty, which indicates that the narrow passage is

empty.

These action and state variables are then used to defineAi ’s local value functionQi as follows:

Qi(x,a) =

〈p1 ; ai = ¬turn green(i) : 10〉

〈p2 ; ai = turn green(i) ∧ is green(i) : 100〉

〈p3 ; ai = turn green(i)∧

a j = ¬turn green( j)∧

¬entranceis empty(i)∧

narrow passageis empty :

1000Ni−1e 〉∀ j 6= i

(3.3)

Qi consists of three value rules (i.e.,p1, p2, andp3) which specify the contribution to the total utility

given a specific context. The first rule indicates¬turn green (i.e., turn red) is the default action, and


hence low value increment. The second rule indicates that the traffic light at entrancei remains green

if it is already green. The third rule turns the traffic light at entrancei to green and the traffic lights

at all other entrances to the same narrow passage (i.e.,j, ∀ j 6= i) to red when the narrow passage is

empty and entrancei is not empty. Note that the value increment in rulep3 is chosen so that when

the narrow passage is empty and more than one entrance are occupied, only the traffic light at the

entrance with the highest ID number1 is set to green, and all others are set to red.

Coordination Graph for Controlling Robot Agents in an Entrance

We control motions of only those agents who are moving toward the narrow passage (easily deter-

mined by following opposite the gradient of the potential field given by the FMM from each agent’s

current position until we reach the narrow passage or an open space). Consequently, agents who are

moving away from the narrow passage will not be controlled by the coordination graph.

For each robot agentAk that is moving toward the narrow passage in entrancei, there is one

boolean action:

• enter(k): Ak is allowed to move toward the narrow passage.¬enter(k) indicates thatAk is not

allowed to move forward and must wait.

A boolean state variable is also defined:

• closestto narrow passage(k), indicates thatAk is the closest one to the narrow passage.

ForAk, its local value functionQk is defined as

Qk(x,a) =

〈p1 ; ak = ¬enter(k) : 100〉

〈p2 ; ak = enter(k)∧

closestto narrow passage(k)∧

is green(i) : 200〉

(3.4)

The first rule indicates thatAk is not allowed to move toward the narrow passage (i.e., it must wait)

by default. However, ifAk is the closest one to the narrow passage and the traffic light at the entrance

is green, the second rule overrides the first on and allowsAk to move toward the narrow passage.

1Other criterions could be used instead of the ID number.


When the narrow passage is a roundabout, there are no traffic lights placed at the entrances

to the roundabout. Instead, a robot agent has to yield for the traffic inside the roundabout. To

handle this situation, we replace the state variable isgreen in rulep2 with no traffic on left (or

no traffic on right, depending on the direction of the traffic flow inside theroundabout).

3.5.2 Computation of Optimal Joint Actions

Once all local value functions have been generated (i.e.,Ne+1 coordination graphs in total for each

narrow passage: first one coordination graph for controlling all the traffic lights and then one co-

ordination graph per entrance for controlling motions of robot agents), each agent determines its

current state and instantiates the current state on its local value function by discarding all rules that

are not consistent with the current state. The optimal joint action for the traffic lights is determined

first (by running the variable elimination algorithm on the coordination graph for the traffic lights),

followed by determining the optimal joint action for the robot agents (by running the variable elim-

ination algorithm on theNe coordination graphs for the robot agents, one at a time, the order being

immaterial).

Suppose that a narrow passage has two entrances and consequently two traffic lightsA1 and

A2. Assume that the narrow passage is empty (i.e., narrowpassageis empty), neither entrances is

empty (i.e.,¬entranceis empty(1) and ¬entranceis empty(2)), and only traffic lightA1 is green

(i.e., is green(1) and¬is green(2)), then the traffic lights’ local value functionsQ1 andQ2 become:

Q1(x,a) =

〈p1 ; a1 = ¬turn green(1) : 10〉

〈p2 ; a1 = turn green(1) : 100〉

〈p3 ; a1 = turn green(1)∧

a2 = ¬turn green(2) : 1000〉

(3.5)

Q2(x,a) =

〈p1 ; a2 = ¬turn green(2) : 10〉

〈p3 ; a2 = turn green(2)∧

a1 = ¬turn green(1) : 2000〉

(3.6)

Now an optimal joint action for the traffic lights can be computed, as mentioned earlier, using

a variable elimination algorithm in combination with a message passing scheme. The optimal joint

action for the example above is{a1,a2} (i.e., only traffic lightA2 is turned to green). Optimal joint

actions for robot agents in the entrances of the narrow passage are computed in the same way.


3.5.3 Deadlock Free Optimal Joint Actions

From local value functions (3.3) for traffic lights and local value functions (3.4) for robot agents, we

can easily prove that optimal joint actions of the traffic lights and the robot agents guarantee that the

robot agents do not get stuck in a narrow passage due to deadlocks.

Theorem 3.5.1.Given a narrow passage, optimal joint actions of its traffic lights (with local value

function Qi (3.3)) and robot agents (with local value function Qk (3.4)) avoid deadlocks in the narrow

passage.

Proof. Assume that the narrow passage is empty and at least one of theNe entrances are not empty,

then the third rule in (3.3) applies (for traffic lights at these non-empty entrances) and the corre-

sponding value increments are 1000Nne−1e : ne∈ [1,Ne], wherene denotes the IDs of the non-empty

entrances. Leth = max(nE). If h = 1 (i.e., there is only one non-empty entrance and its ID is equal

to 1), the entranceh = 1’s value increment is equal to 1000 and it clearly dominates the value incre-

ments from the first two rules. Forh > 1 (i.e., there is at least one non-empty entrance and its ID

is greater than 1), the entranceh’s value increment is also greater than the sum ofall other possibly

non-emptyentrances’ value increments, becauseNh−1e > ∑h−2

κ=0Nκe (this can be easily derived from

Geometric series[72]). Consequently, traffic light at entranceh is set to green while traffic lights at

all otherentrances are set to red (i.e., one and only one traffic light is switched to green) to maximize

the total utility (i.e., this action is optimal). Since the default action for a traffic light is turning to

red (the first rule in (3.3)) and it remains green if it is already green (the second rule in (3.3)), the

three rules in (3.3) guarantee that at most one traffic light at a certain time is green and it can only

be switched to green from red if the narrow passage is empty. Furthermore, when traffic light at an

entrance is green, multiple robot agents can enter from this entrance one by one (moving in the same

direction since they are all entering from the same entrance), because the two rules in (3.4)) ensure

that one and only one robot agent at the entrance is allowed to enter at one time. Consequently,

optimal joint actions of the traffic lights and the robot agents ensure that the robot agents do not get

stuck in the narrow passage due to deadlocks.

3.6 Parallel Processing

The most expensive step in Algorithm 3 is step 3.10, because we may have to process multiple

CG tasks, where eachCG taskconstitutes constructing a coordination graph and then performing

the variable elimination algorithm to compute an optimal joint action in each update cycle. For


processing more agents in real-time, we process multiple CG tasks in parallel on a Symmetric Mul-

tiprocessing (SMP) system in this section.

3.6.1 Supervisor-Worker Paradigm

We implemented the task parallelism in a supervisor-worker paradigm: a single supervisor, called

master, asks multiple workers, called slaves, to perform the CG tasks.

The supervisor-worker paradigm can be implemented with either threads or processes. Threads

should be used when the same complex data structures must be processed concurrently, and pro-

cesses should be used instead for less tightly coupled applications [75]. Therefore, we chose pro-

cesses for implementation of the supervisor-worker paradigm, because all workers perform their

tasks independently of one other and no data is passed between workers. Because creating pro-

cesses costs more than creating threads, we create and destroy processes only when the supervisor

and the workers initialize and terminate, respectively; hence no processes are created or destroyed

during runtime. Another advantage of processes over threads is that developing and testing pro-

cesses separately is easier. However, since each process has its own set of memory pages, we have

to use Interprocess Communication to let processes communicate with each other. We use System

V IPC for interprocess communication, in particularmessage queuesandshared memory segments.

As shown in Fig. 3.5, the processes are grouped into three groups: the rendering process, the

supervisor processes, and the worker processes. We decouple also the rendering (step 3.13 in Al-

gorithm 3) from the other steps in Algorithm 3 to ensure interactive frame rates. The supervisor

processes consist of a planner process and a message receiving process. All worker processes are

identical and they communicate with the supervisor processes through two message queues: the

supervisor sends messages to the workers through message queueA, and the workers send back the

results through message queueB.

In each update cycle, the planner process observes all agent’s states (step 3.7 in Algorithm 3)

as soon as it has computed all group’s potentials and gradients using the FMM (step 3.3 in Algo-

rithm 3). The planner process stores all agents’ states inside multiple messages before the messages

are appended to the message queueA. Once all messages are sent to the worker processes, the plan-

ner process moves on to the next step. As soon as a worker process is free, it pops the first message

in the message queueA. Using the data stored inside the message, the worker process performs

one CG task from step 3.10, and sends back the corresponding optimal joint action to the message


receiving process of the supervisor through the message queueB. When the message receiving pro-

cess pops a message in the message queueB, it writes the data (i.e., an optimal joint action) in the

message into the same memory it shares with the planner process. The shared data are only writ-

ten by one process (i.e., the message receiving process); hence, there is no need to use semaphores

for restricting access to the data. Similarly, the planner process passes the agents’ locations to the

rendering process (for rendering using OGRE [90]) via the memory shared by the two processes.

3.6.2 Job Scheduling

n CG tasks are performed in each update cycle. A CG task can only be processed by one processor,

and a CG task cannot be interrupted. All processors available for processing are identical. We want

to minimize the length of time denoted byM required to complete all CG tasks. This is a classical

scheduling problem of parallel identical processors and independent jobs(tasks) [2]. Since this

problem isNP-hard [10], we use a fast and effective heuristic procedure —Longest Processing

Time(LPT) [2, 10] — instead. The time complexity of the LPT isO(nlogn) [10]. Theabsolute

performance ratio RLPT for the approximation algorithm LPT is

RLPT =43−

13m

(3.7)

wherem is the number of processors (i.e., an LPT schedule can be up to 33% longer than an optimal

schedule in the worst case) [10].

In general, processing time for a coordination graph grows with the number of nodes in it.

Therefore, an LPT ordering of the CG tasks can be obtained by sorting the coordination graphs

according to the number of nodes.

3.6.3 Asynchronous Message Delivery

With multiple processes running in parallel, once all messages are sent by the planner process (part

of the supervisor) to the worker processes, the planner process does not wait for the results from

the worker processes via the message receiving process. Instead, it moves on to the next step (i.e.,

step 3.11, where agents’ locations are updated, in Algorithm 3) from step 3.10, while the worker

processes compute optimal joint actions. This is calledasynchronous message delivery. With it,

we can achieve higher parallel efficiency. However, on average, the worker processes still need to


compute optimal joint actions in less than about one-tenth of a second in order to avoid deadlocks

in the narrow passages.

During the simulation, the planner process maintains a table of boolean variables, with each

variable corresponding to a single agent. The size of the table is therefore equal to the number of

agents. The initial values of all table entries aretrue. Whenever an optimal joint action is received,

the planner process updates the entries of the corresponding agents with the values in the optimal

joint action. To prevent deadlocks, an agent’s velocity computed from the potential and the gradient

is reset to0 (i.e., we change the velocity instantaneously and this amounts to an infinite acceleration)

if the agent’s entry contains the valuefalse; hence the agent remains at its current location until the

value of the entry changes totrue.

3.7 Experiments and Results

3.7.1 Hardware and Software Setup

The experiments were performed on two different systems: an SGI UltimateVision and a Dell Opti-

Plex GX620. The SGI UltimateVision has 24 MIPS R16000 processors and runs IRIX 6.5.28. Each

MIPS processor runs at 700 megahertz, and has 4 megabyte L2 cache. The system has 14 gigabytes

of global shared memory, and it is visible to all 24 processors. The Dell OptiPlex GX620 has one

Intel Pentium D Processor 820 with two execution cores running at 2.8GHz and each core having

1MB level 2 cache (2MB in total), one 256MB ATI Radeon X600 PCIe video card, 4 gigabytes

of RAM, and runs Red Hat Enterprise Linux (RHEL) 4. The code was written in ANSI C/C++

and compiled using GNU GCC 3.4.3 on the Dell OptiPlex GX620 and GNU GCC 3.3 on the SGI

UltimateVision.

We chose the SGI UltimateVision to measure the parallel efficiency of our approach (specifically

constructing and processing coordination graphs in parallel using the supervisor-worker paradigm),

because the SGI UltimateVision has 24 processors. However, we could not render efficiently in 3D

on the SGI UltimateVision because images/frames must be sent through a rather slow network to

the OptiPlex and displayed remotely. Consequently, no rendering was done when we performed

experiments on the SGI UltimateVision. Instead, the OptiPlex was used to test the usability of our

approach in real-time application (i.e., with rendering) even though it has only two processing cores.


Table 3.1: Runtime of one CG task.CG Task Size (No. of CG nodes) 1 2 3 4 5 6Time in msec 2 10 20 64 107 160

3.7.2 Parallel Performance Analysis

In this subsection, we investigate whether we can obtained significant, scalable (i.e., we want to be

able to process more coordination graphs in real-time by adding processors) speedups by construct-

ing and processing coordination graphs in parallel using the supervisor-worker paradigm on the SGI

UltimateVision. An important parallel performance metric isspeedup S, which is defined as

S(Np,Pp) =Tseq(N)

T(Np,Pp), (3.8)

whereNp is the size of the problem,Pp is the number of processors,Tseq is the runtime of the

sequential program, andT(P) is the runtime of the parallel program.

To measure the parallel efficiency, we disabled the rendering process in Fig. 3.5 (due to the slow

connection between the OptiPlex and the SGI UltimateVision) and replaced the planner process

with a test driver process that enables us to perform some experiments in a controlled environment

using the SGI UltimateVision. The test driver process generates jobs and send them to the worker

processes (i.e., it performs only step 3.10 in Algorithm 3), where a job consists of one or multiple

CG tasks. The size of the job varies not only with the number of the CG tasks, but also with the size

of each CG task, which is equal to the number of nodes of the coordination graph inside the CG task,

and it ranges from 1 to 6 in our experiments. The test driver generates a job with a certain number

of CG tasks, where the sizes of the CG tasks may be equal to one another or different, because we

wanted to measure how the runtime of one CG task varies with its size and investigate how its size

affects the speedup. To measure runtime, a timer is started before the test driver sends messages to

the workers through message queueA; it is stopped once the test driver receives all optimal joint

actions from the message receiving process.

First, we measured how the runtime of one CG task varies with the size of the CG task. For each

job, the test driver sent 40 CG tasks of the same size to the workers for processing. The average

runtime for 1 CG task is shown in Table 3.1.

Next, we investigated how the size of a CG task (i.e., the number of nodes in a coordination

graph) affects the speedup. Three different numbers of nodes were tested: 2, 4, and 6. For each test,


the test driver sent 40 CG tasks of equal size to the workers. Runtime for 1, 2, 5, 10, and 20 workers

is shown in Table 3.2 and Fig. 3.6. The speedups for coordination graphs with all three different

numbers (i.e., 2, 4, and 6) of nodes are sub-linear due to thecommunication overhead. The effect of

the communication overhead is especially evident when the jobs are small and distributed to many

processors (e.g., when 40 coordination graphs with 2 nodes are processed by 20 workers).

Table 3.2: Runtimes of 40 equal-size CG tasks.No. of Processors(P) 1 2 5 10 20Time in msec (2 nodes) 412 210 86 51 33Time in msec (4 nodes) 2568 1279 521 268 149Time in msec (6 nodes) 6326 3148 1272 647 338

Another factor affecting speedups is the load balance between different processors. Ideally each

processor should perform the same amount of work. Given multiple CG tasks with mixed sizes, we

used the LPT scheduler to distribute CG tasks evenly among the available processors for a good load

balance. We tested the effect of the LPT scheduler with 20, 30, and 40 CG tasks. The size of each

CG task was determined randomly by the test driver, and it ranged from 1 to 6. Each CG task was

performed first without the LPT scheduler, and then with the scheduler. Runtime (averaged over 10

runs) is shown in Table 3.3. Speedups are shown in Fig. 3.7.

Table 3.3: Runtimes of unequal-size CG tasks (with and without scheduling).No. of Processors(P) 1 2 5 10 20

without schedulingTime in msec (20 CG tasks) 1081 569 270 171 157Time in msec (30 CG tasks) 1625 842 378 231 164Time in msec (40 CG tasks) 2139 1098 482 290 194

with schedulingTime in msec (20 CG tasks) 1079 525 223 155 150Time in msec (30 CG tasks) 1620 812 332 184 153Time in msec (40 CG tasks) 2139 1071 434 231 176

The data in Table 3.3 and Fig. 3.7 indicates that the parallel performance improves significantly

when the LPT scheduler is used. Observe that the speedups are limited by the processing time of the

biggest CG task (i.e., a CG task with size 6). Because it takes abut 160 msec (Table 3.1) to process

a CG task with size 6, multiple CG tasks can not be processed in less time than 160 msec regardless

of the number of processors used to process these CG tasks.


3.7.3 Usability in Real-Time Applications

We constructed a demo application that contains all components in Fig. 3.5 (i.e., rendering in 3D

using OGRE [90], the FMM based the continuum model, and deadlock avoidance using coordination

graphs) to test the usability of our approach in real-time application on the OptiPlex. The application

takes a static environment given as a grid andk groupsG1, . . . ,Gk of agents as the input, where all

agents in each group share the same goal, though their initial positions are different.

In addition to the rendering process and the two supervisor processes (i.e., the planner process

and the message receiving process), we created two worker processes. Because the OptiPlex’s

single Pentium D processor has two cores, the five processes we created are simply multiplexed

over the two cores by the system. However, running with fewer processors than processes can

hinder performance [14]. We gave the planner process higher priority than the other processes by

setting the planner process’ and other processes’nice valuesto 16 and 20, respectively. A process

with higher nice value runs at a lower priority [75].

We used the environments shown in Figs. 3.8 and 3.9 to test the runtime performance. Two

groups of agents were placed in the environments, where each group had four agents. Recall

that all agents in the same group share the same goal, but different initial positions. All figures

in Figs. 3.8 and 3.9 were drawn in 2D for clarity using the data collected during the 3D sim-

ulations. The environment in Fig. 3.8 contained a single narrow passage with three entrances;

hence four coordination graphs were constructed and processed in each update cycle during the

simulation. The single narrow passage in the environment in Fig. 3.9 is a roundabout; hence

the same number (four) of coordination graphs (one for each entrance) were constructed and pro-

cessed in each update cycle. By combining the continuum model and coordination graphs, we were

able to avoid deadlocks inside the narrow passages in Figs. 3.8 and 3.9. For example, Figs. 3.8b

and 3.8c show that agents #5 and #6 waited until agents #1 and #2 had exited from the nar-

row passage before they entered the narrow passaged (agent #6 first, followed by agent #5). In

Fig. 3.9, agents inside the roundabout have priority over the entering agents. For example, Figs. 3.9c

and 3.9d show that agent #5 waited outside the roundabout until agent #1 was no longer on its

left hand side (the traffic flowed counterclockwise inside the roundabout) before it entered the

roundabout. To enjoy the full visual impact, see thevideo accompanying this chapter (http:

//ramp.ensc.sfu.ca/people/liyi/LI_GUPTA_COORDINATION_VIDEO.mp4 ).


3.8 Conclusion

In this chapter, we showed how to combine the fast marching based continuum method with a

symbolic AI technique (i.e., coordination graphs) to plan in real-time motions of multiple agents

in static environments with narrow passages. The rules we designed for coordination graphs are

simple and intuitive, yet we were able to avoid deadlocks in narrow passages and achieve complex

behaviors such as agents’ behaviors inside roundabouts (i.e., follow the direction of the traffic flow).

In order to process multiple CG tasks in real-time, we proposed to process those tasks in parallel on

a SMP system. The task parallelism was implemented in a supervisor-worker paradigm with Unix

processes. The supervisor not only updates each agent’s position and orientation in each update

cycle, but also distributes jobs to the workers and receives results (i.e. optimal joint actions) from

the workers. A worker takes order from the supervisor, then constructs a coordination graph and

computes an optimal joint action.

We obtained significant, scalable speedups by constructing and processing coordination graphs

in parallel on the SGI UltimateVision with 24 processors. Unfortunately, we could not render effi-

ciently in 3D on the SGI UltimateVision because images/frames must be sent through a rather slow

network to our dual-core PC and displayed remotely. Consequently, no rendering was done on the

SGI UltimateVision. However, we tested the usability of our approach in real-time application (i.e.,

with rendering) on the PC. By decoupling the rendering and the planning, we were able to render

at a higher frame rate than 24 fps on the PC. Our current implementation can coordinate motions of

about ten agents in real-time on the PC.


5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(a) The environment (i.e.,I ).

5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(b) After dilation (i.e.,D = I ⊕K).

5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(c) After erosion (i.e.,E = D⊖K).

5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(d) The open spaces.

5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(e) The narrow passages.

5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(f) The entrances.

Figure 3.3: An environment with two narrow passages.


5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(a) The environment

5 10 15 20 25 30 35 40

5

10

15

20

25

30

35

40

(b) The roundabout.

Figure 3.4: An environment with one roundabout.

Data Sent

Shared Memory

Data Sent

Message Queue A

Message Queue B

Shared Memory

Worker Processes (CGs)

Supervisor

OSRendering Process

Message Receiving Process

(OGRE3D)

OS

(FMM)

Planner Process

Process No. 1

Figure 3.5: The rendering is decoupled from the supervisor and its workers. The processes commu-nicate with each other through System V message queue and shared memory.


1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200123456789

1011121314151617181920

No. of Processors

Spe

edup

s

6 nodes4 nodes2 nodes

Figure 3.6: Speedups for 40 equal-size CG tasks.

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 200

1

2

3

4

5

6

7

8

9

10

11

12

13

14

No. of Processors

Spe

edup

s

20 tasks without scheduling30 tasks without scheduling40 tasks without scheduling20 tasks with scheduling30 tasks with scheduling40 tasks with scheduling

Figure 3.7: Speedups for unequal-size CG tasks (with and without scheduling).


1

2

3

4 5 6

Sample No. 600

(a)

1

2

3

4 5

6

Sample No. 1000

(b)

1

2

3

4 5

6

Sample No. 1500

(c)

1

2

3

4

5

6

Sample No. 1900

(d)

12

3

4

5

6

Sample No. 2400

(e)

12

3

4

5 6Sample No. 2900

(f)

Figure 3.8: Two groups of agents are moving in an environment with one narrow passage and threeentrances.


1

2

3

4

56

7

8

Sample No. 500

(a)

1

2

3

4

56 7

8

Sample No. 1000

(b)

1

2

3

4

5

6 7 8

Sample No. 1300

(c)

1

23

4

5

6 7 8

Sample No. 1500

(d)

12

34

56

7

8

Sample No. 2000

(e)

1

2

3

4

5

6

7

8

Sample No. 2500

(f)

Figure 3.9: Two groups of agents are moving in an environment with one roundabout and fourentrances. Agents must move counterclockwise inside the roundabout.

Chapter 4

Motion Planning of Multiple Agents

In this chapter, we propose an adaptive continuum model forreal-timemotion planing of multiple

agents withindependent goals(i.e., each agent has its own goal) in a two dimensional dynamic vir-

tual environment given as a occupancy grid. Our objective is to retain the advantages of the basic

continuum model, while allowing each agent to have its ownindependentgoal (instead of sharing

the same goal with other agents). In our adaptive extension to the basic model, the potential field

computation (and the resulting path) for each agent is restricted to achannel[67], a subset of the

grid cells of the environment. This reduces the domain over which the potential field is constructed,

thereby significantly speeding up the computation. Next, we extend the adaptive continuum model

by constructing channels around backbone paths found at coarser resolution grids, and then plan-

ning motions of the agents at theoriginal higher resolution grid. It intrinsically prefers open areas

because narrows passages may appear as blocked at the coarse levels. This is a desirable trait since

otherwise deadlocks may occur in narrow passages. Simulations show that that our adaptive multi-

resolution approach, while maintaining a real-time performance, (i) handles a significantly larger

number of individual agents with independent goals, (ii) handles bigger grids than the basic contin-

uum model, and (iii) provides a natural way to steer agents from narrow passages to open spaces.

This chapter is organized as follows. The problem to be solved is formally defined in Section 4.1.

In Section 4.2, we present our adaptive continuum model for motion planning of multiple agents in

real-time, where each agent has its own goal. We extend the adaptive approach in Section 4.3 to

steer agents towards open spaces and away from narrow passages. We perform experiments with

multiple virtual environments of different sizes in Section 4.4 and conclude in Section 4.5.

51

CHAPTER 4. MOTION PLANNING OF MULTIPLE AGENTS 52

4.1 Problem Definition

Let agentsa1, . . . ,ak be in a two dimensional dynamic virtual environment, where each agent is

modeled by a disc of radiusr. The virtual environment is given as a binary occupancy grid, and

the motions of the dynamic obstacles are not known beforehand. In our application, an agent is

generally a few times smaller than a grid cell. The task is to plan for each agent, in real-time, a path

that takes the agent from its start position to its goal position without colliding with other agents and

obstacles.

4.2 The Adaptive Continuum Model

For each agentai , where 1≤ i ≤ k, we want to compute its optimal pathp∗i that takesai from its

start position to its goal position while minimizing∫

p∗iCids, where unit cost fieldCi depends on the

speed fieldfi and the discomfort fieldgi as shown in (2.6). Because we assume that a grid cell

represents either an obstacle or a flat free space, the topographical speedfT in this chapter is either

free speedf f ree or 0. When computing the speed fieldf , we iterate over each grid cell and then

over each of the four faces of the grid cell. The topographical speedfT for one face is set tof f ree

if and only if both the current grid cell and the neighboring grid cell represent free space, otherwise

it is set to 0. In addition, we increase discomforts at grid cells in the neighborhood of obstacles

to increase clearance. We refer the reader to the basic continuum model paper [91] for details on

constructions of the speed fields and the discomfort fields. Once the unit cost fieldCi is set, we

compute the potential functionφPi by solving the Eikonal equation (2.1) using the FMM, which is

the most expensive step in the basic continuum model [91] as mentioned in the introduction. In order

to achieve substantial speedups, we compute potentialφPi and gradient∇φPi over a channel (i.e., a a

subset of the grid cells) instead of the entire grid (see Fig. 4.1). Agentai then moves in the direction

opposite the gradient∇φPi scaled by the speed at the agent’s position. In each simulation cycle

(step 4.1 in Algorithm 4), all agents’ positions are updated. This is repeated until all agents reach

their goals. The algorithm is outlined in Algorithm 4. Note that Algorithm 4 iterates through all

agents instead of all groups as in [91] and the minimum distance between agents has to be enforced

because two agents in the same grid cell may collide [91].

To construct a channel for agentai , we apply the continuum model over the entire grid and use

the unit cost fieldCi to compute a backbone pathpi by moving in the direction opposite the gradient

until we reach the goal of agentai . Next, we growpi by a given marginm to yield a channelPi as


10 20 30 40 50 60

10

20

30

40

50

60

0

100

200

300

400

500

600

Potentialφ

10 20 30 40 50 60

10

20

30

40

50

60

50

100

150

200

250

300

350

400

Potentialφ

Figure 4.1: Reducing the domain for potential constructions from all grid cells (left) in the virtualenvironment to a channel (right).

Algorithm 4 : The Adaptive Continuum Model

foreach simulation cycledo4.1

foreach agent ai do4.2

Construct unit cost fieldCi;4.3

Construct channelPi (Algorithm 5);4.4

Construct potentialφPi and its gradient∇φPi over channelPi using the FMM;4.5

end4.6

Update the agents’ positions;4.7

Enforce the minimum distance between agents;4.8

end4.9

follows. For a cellui on the pathpi , a cellvi is added to the channelPi if the Manhattan distance

betweenui andvi is less thanm. Because the domain for potential constructions varies lineally with

m, the FMM computation can be speeded up by reducingm. Howeverm should not be too small,

for in that case, agentai may not be able to avoid collisions with other agents and obstacles in the

neighborhood of the backbone pathpi. We set the value ofm empirically. The computation of

channelPi is outlined in Algorithm 5.

Because channelPi (i.e., the domain for potential constructions for agentai) changes gradually

(e.g., it becomes shorter as the agent moves towards its goal), we updatePi (steps 5.7, 5.8, and 5.9 in

Algorithm 5) at a low frequency (i.e., 1/(k∆T), wherek is the number of agents and∆T is the chan-

nel update interval) instead of once in each simulation cycle. The potentialφPi and its gradient∇φPi


Algorithm 5 : Channel Construction for Agentai

input : unit cost fieldCi, potential over channelPi φPi

output: channelPi

∆T ← channel update interval (e.g., 0.5 sec);5.1

m← channel margin;5.2

ti ← time when the latest update (of channelPi) event happened;5.3

tc← current time;5.4

xi ← current position of agentai ;5.5

if (tc− ti) > k∆T or φPi(xi) is very highor φPi is not setthen5.6

Construct potentialφi and its gradient∇φi over the entire grid using the FMM;5.7

Find backbone pathpi by moving opposite the gradient∇φi ;5.8

Grow channelPi alongpi with marginm;5.9

end5.10

for agentai are updated in each simulation cycle (Algorithm 4). Note that a channel update cycle

for thek agents (i.e., time needed to update allk channels because we need to construct a channel

for each agent) lastsk∆T sec. In order to achieve a smooth simulation, although not shown in the

pseudo code here, we distribute the updates of allk channels evenly over the channel update cycle.

However,Pi has to be updated immediately if agentai is blocked by dynamic obstacle(s). When the

potentialφPi (xi), wherexi is the position of agentai , becomes suddenly very high (potentialφPi is

zero at the goal position of agentai ), it indicates that agentai has been blocked by dynamic obsta-

cle(s) (e.g., closed doors) andPi becomes invalid. In that case, we update channelPi for agentai

immediately so that the agent bypasses the dynamic obstacle(s). Overall, the impact of Algorithm 5

on the average running time of Algorithm 4 is negligible because we run steps 5.7, 5.8, and 5.9 in

Algorithm 5 at a low frequency.

4.3 Extension: Multi-Resolution Channel Construction

In this section, we construct multi-resolution unit cost fields using the discrete wavelet transform to

steer agents into open spaces and away from narrow passages.

A one-dimensional data sequenceSj for a given level j can be decomposed into a coarse res-

olution versionSj+1, with detailW j+1 using one pair of matricesA j andB j , known as aanalysis

filters [85, 86].


Sj+1 = A jSj (4.1)

W j+1 = B jSj (4.2)

This procedure can be applied recursively to decompose the original dataSj into a hierarchy of

coarser resolution versionsSj+1,Sj+2, . . . and detailsW j+1,W j+2, . . .. Although we are only inter-

ested in the coarser resolution versions. In the unnormalized Haar basis, matricesA j ,A j+1, . . . and

B j ,B j+1, . . . represent the averaging and differencing operations. In addition, the lengths of coarser

resolution version sequenceSj+1 and detail sequenceW j+1 is halved compared to the data sequence

Sj for particular levelj. To compute the wavelet transform of a two dimensional data set, we gener-

alize the Haar wavelet to two dimensions by applying the one-dimensional Haar wavelet transform

to each row of the original data set first, and then each column of the transformed rows [85, 86]. The

complexity of the wavelet transform isO(M), whereM is the size of the original data set.

Now we describe how to compute a multi-resolution unit cost field using the 2D wavelet trans-

form. We denote the unit cost field at resolution levell asCl , wherel ≥ 0, and higher value of

l represents coarser resolution levels, withl = 0 being the finest resolution. First we construct a

unit cost field over the virtual environment (given as a binary occupancy grid) as described in the

previous section and denote it asC0. To computeCl , wherel > 0, we divideC0 into four equal sized

setsC0N, C0

S, C0W, andC0

E according to direction, because unit cost fields are anisotropic fields. For

each setC0i , we apply the 2D wavelet transform on it to compute its coarser resolution versionsCl

i ,

wherei ∈ {N,S,W,E}, l = 1. . .Li, andLi is a suitable coarse level. Therefore, the unit cost field at

resolution levell (i.e.,Cl ) consists ofClN, Cl

S, ClW, andCl

E.

Next, we incorporate the multi-resolution unit cost field into Algorithm 5 and the result is out-

lined in Algorithm 6. Algorithm 4 is not changed, except that it calls Algorithm 6 instead of Algo-

rithm 5 now. Note that unit cost fieldCi in Algorithm 4 and unit cost fieldC0i in Algorithm 6 are the

same. In Algorithm 6, we need to choose a suitable resolution levelLi. With increasingLi, “wider”

narrow passages will be blocked. If we fail to find path at levelLi in Algorithm 6, we move down

to levelLi−1 which has higher resolution thanLi and try again. This step is repeated until we find

a path, denoted bypLii (assuming that one exists at level 0), and a channelPi is constructed along

it. Therefore, we will not miss the paths that pass through narrow passages if these are the only

choices. Note that when the environment is particularly cluttered, the benefit of the multi-resolution

extension (i.e., steering agents away from narrow passages) is not always attainable. Observe that


notationsPi, φPi , and∇φPi in Algorithm 6 do not have any superscripts, because the continuum model

without the minimum distance enforcement step can only ensure that no two agents intersect up to

the resolution of the grid and hence we always construct channelPi at level 0 even if the backbone

path it is constructed around was found at a coarser level.

Algorithm 6 : Multi-Resolution Channel Construction for Agentai

input : unit cost fieldC0i , potential over channelPi φPi

output: channelPi

∆T ← channel update interval (e.g., 0.5 sec);6.1

m← channel margin;6.2

ti ← time when the latest update (of channelPi) event happened;6.3

tc← current time;6.4

xi ← current position of agenti;6.5

if (tc− ti) > k∆T or φPi(xi) is very highor φPi is not setthen6.6

Li ← asuitable resolution level;6.7

for l = 1 to Li do6.8

ConstructCli using wavelets;6.9

end6.10

repeat6.11

Construct the potentialφLii and its gradient∇φLi

i over the entire levelLi grid;6.12

Find backbone pathpLii by moving opposite the gradient∇φLi

i , Li← Li−1 if6.13

failed;until path pLi

i is found;6.14

Grow channelPi alongpLii with marginm;6.15

end6.16

4.4 Experiments

We have performed a number of experiments to show the effectiveness of our adaptive multi-

resolution continuum model on a Dell OptiPlex GX620. The OptiPlex has one Intel Pentium D

Processor 820 with two execution cores running at 2.8GHz, one 256MB ATI Radeon X600 video

card, 4 gigabytes of RAM, and runs Red Hat Enterprise Linux (RHEL) 4. The code was written

in ANSI C/C++ and compiled using GNU GCC 3.4.3. All virtual environments and agents in the

experiments were rendered in 3D using OGRE [90] and in a separate process than the planning pro-

cess [57]. Because the screenshots do not reproduce well on paper, all figures in this section were

redrawn in postscript using the data collected during the experiments.


First, we simulated motions of 10, 20, and 40 agents on 64×64, 128×128, and 256×256 grids

to demonstrate the efficiency of our approach (we used∆T = 0.5 sec andm= 4 cells for all agents in

all experiments) compared to the basic continuum model, where we ran each simulation for 60 sec

and the number of agents in each group modeled by the basic continuum model was restricted to one

(i.e., each agent has its own goal). In this thesis, all grid cells are squares. For 64×64 and 128×128

grids, each grid cell is 500 units wide. For 256×256 grids, each grid cell is 250 units wide. The

radius of all agents is 50 units. The update frequencies of the main loops in the basic continuum

model and our approach (i.e., step 4.1 in Algorithm 4) for these cases are shown in Table 4.1, where

we denote a path computed at level 0 using a channel constructed around a backbone path found

at level Li as a typeLi 7→ 0 path. For example, 07→ 0 paths in Table 4.1 were computed using

channels constructed around backbone paths found at level 0 using Algorithm 5, whereas 17→ 0 and

2 7→ 0 paths were computed using channels constructed around backbone paths found at levels 1 and

2, respectively, using Algorithm 6. The data in Table 4.1 show that we have achieved substantial

speedups. For example, our approach is faster by a factor of 4 (i.e., it can perform 21 updates per sec

compared to less than 5 updates per sec using the basic continuum model) when planning motions

of 40 agents on a 64× 64 grid. On 256× 256 grids, the basic continuum model performed fewer

than one update per sec and resulted in sluggish simulations, whereas our approach speeded up

the computations by more than 8 times. Therefore, our approach can plan motions of more agents

with independent goals on bigger grids compared to the basic continuum model. Three virtual

environments (64× 64 grids) and the paths computed by our approach are shown in Fig. 4.2, 4.3,

and 4.4, where the agents’ start positions and goal positions are marked as grey circles (green on

color prints) and black discs, respectively, along with their IDs and static obstacles are drawn in dark

grey (red on color prints).

The adaptive aspect of our approach is the main contributor to the speedups. Particularly, at

lower resolution grids, the contribution comes entirely from it. For example, for 64× 64 and

128× 128 grids, the speedups for 0→ 0, 1→ 0, and 2→ 0 cases are essentially the same. The

multi-resolution aspect of our approach makes only slight contribution to the speedups for higher

resolution grids (e.g., 256× 256 grids). In these experiments, the multi-resolution aspect has less

impact on the speedups than the adaptive component because backbone paths are constructed at a

very low frequency.

The multi-resolution aspect of our approach has another benefit though; it steers agents away

from narrow passages to open spaces. This is desirable because otherwise deadlocks may occur

between agents moving through the same narrow passage. For example in Fig. 4.3, 2→ 0 path


for agent 1 does not pass through the narrow passages marked by grey rectangles (yellow on color

prints). Instead, it takes a much longer route because those narrow passages appear to be blocked at

level 2. When no path is found at a coarse level (e.g., level 2), a backbone bath is searched for at a

higher-resolution level (i.e., level 0 or 1). For example, path for agent 12 passes through one of the

narrow passages because there is no 2→ 0 path for agent 12. Therefore, we will not miss the paths

that pass through narrow passages if these are the only choices.

Our adaptive multi-resolution continuum model handles not only static obstacles, but also dy-

namic obstacles and changes in environments such as door closing. For each agent, all other agents

are treated as dynamic obstacles. As shown in Fig. 4.4, agent 4 changed its course and took a detour

shortly after it left its start position marked by a grey circles (green on color prints), because its

channel became suddenly blocked by a dynamic obstacle marked by grey rectangles (cyan on color

prints) and a new channel was constructed for the agent immediately around a backbone path that

does not intersect the dynamic obstacle.

To enjoy the full visual impact, see thevideo accompanying this chapter (http://ramp.

ensc.sfu.ca/people/liyi/LI_GUPTA_ADAPTIVE_VIDEO.mp4 ).

4.5 Conclusion

We have presented an adaptive multi-resolution approach to the problem of motion planning of

multiple agents in two dimensional dynamic virtual environments. The main contribution of it is

that our approach can handle significantly larger number of agents with independent goals, while

retaining the advantages of the basic continuum model such as unified global planning and collision

avoidance. The key to the efficiency of our approach is that we restrict the computation of each

agent’s potential field to a channel, where the channel is constructed/updated a very low frequency

or whenever it is blocked by dynamic obstacle(s). As an intrinsic byproduct, our approach can steer

the agents away from narrow passages to open spaces (if desired), but it will not miss paths that pass

through narrow passages if they are the only options. The experiments show that our adaptive multi-

resolution continuum model can plan motions of a larger number of individual agents on larger grids

compared to the basic continuum model.

Our adaptive multi-resolution approach has a couple of limitations. The adaptive extension to

the basic model restricts the potential field computation for each agent to a channel, therefore the

resulting path may differ from the optimal one constructed by the basic model. Furthermore, the

benefit of the multi-resolution extension (i.e., steering agents away from narrow passages) may not


always attainable when the environment is heavily cluttered with static/dynamic obstacles and even

agents. If all passages are narrow, then they will appear blocked at coarser resolutions and hence

one will have to go the the finest resolution and we loose the the benefit of the multi-resolution

extension.


Tabl

e4.

1:S

peed

com

paris

onbe

twee

nth

eba

sic

cont

inuu

mm

odel

and

our

adap

tive

appr

oach

.B

asic

Con

tinuu

mO

urA

dapt

ive

mul

ti-re

solu

tion

App

roac

hG

ridS

ize

No.

ofA

gent

sM

odel

0→

0P

aths

Spe

edup

1→

0P

aths

Spe

edup

2→

0P

aths

Spe

edup

(upd

ates

/sec

)(u

pdat

es/s

ec)

(rou

nded

)(u

pdat

es/s

ec)

(rou

nded

)(u

pdat

es/s

ec)

(rou

nded

)

64×

6410

17.8

994

.34

594

.34

594

.34

520

8.99

42.0

25

42.5

55

44.4

45

404.

5221

.37

521

.65

522

.08

5

128×

128

104.

3030

.96

731

.35

731

.15

720

2.16

15.0

27

15.1

77

15.0

47

401.

087.

627

7.79

77.

747

256×

256

100.

998.

068

8.40

88.

478

200.

494.

249

4.44

94.

469

400.

252.

088

2.20

92.

239


1

2

3

4

56

7

8

9

10

11

12

13

14

15

16

17

18

19

20

1 2

3

4

5

6

7

8

9

10

11

12

13

14

15

16

17

18

19

20

Sample No. 2000

Figure 4.2: 0→ 0 paths for 20 agents.


1

2

3

4

5

6

7

8

9

10

11

12

12

34

5

6

7

8

9

10

11

12

Sample No. 3500

Figure 4.3: 2→ 0 paths for 12 agents in an environment with narrow passages.


1

2

3

4

5

1

2

3

4

5

Sample No. 5750

Figure 4.4: 0→ 0 paths for 5 agents in an environment with dynamic obstacles.

Chapter 5

Motion Planning of Multiple Formations

In this chapter, we introduce a novel approach to the problem of coordinating multiple agents to

move in a tightly controlled formation in two dimensional virtual environments. Agents in a for-

mation are conceived as being part of a two dimensional elastic shape modeled using the Boundary

Element Method (BEM) [9], and the formation can be deformed in a natural and controlled manner

and in real-time by displacing a few control nodes on the elastic shape. This flexible virtual structure

approach for formation control is then integrated with the continuum model [91] to plan in real-time

motions of multiple formations in virtual environments with dynamic obstacles. Simulations cre-

ated with our algorithm run at interactive rates in quite complex environments. At runtime, different

formations normally avoid each other, but in case this is not feasible (i.e., planner is not able to

find collision-free motions for formations), we do allow them to go through each other (e.g., in nar-

row passages) without any collision between agents. In addition, each formation can be deformed

in real-time and the deformation is triggered either automatically (e.g., when the formation’s path

is blocked by dynamic obstacles) or manually. Via simulations, we show that our current imple-

mentation can plan at least four formations, each with about 20 agents, in real-time on a standard

PC.

This chapter is organized as follows. The problem to be solved is formally defined in Section 5.1.

In Section 5.2, we present the overall solution to the problem of coordinating multiple agents to move

in a tightly controlled formation in two dimensional virtual environments. A brief overview of the

BEM and some other methods for modeling deformable objects is then given in Section 5.3, before

we present the details of our flexible virtual structure approach to the formation control problem in

Section 5.4. In Section 5.5, we adapt the continuum model to plan motions of multiple formations in

real-time, where each formation is modeled by our flexible virtual structure approach for formation

64

CHAPTER 5. MOTION PLANNING OF MULTIPLE FORMATIONS 65

control. We perform various experiments in Section 5.6 and conclude in Section 5.7.

5.1 Problem Description

As mentioned in the introduction, our first problem is to plan motion of a single formation in static

and dynamic environments; we then extend it to planning the motions of multiple formations. How-

ever, we formulate the overall problem of planning for multiple formulations directly with the single

formation version being a special case of the general problem. It simply is more natural this way.

Givenk tightly controlled formationsR1, . . . ,Rk, we define each formationR i, where 1≤ i ≤ k,

in frameFfi . As shown in Fig. 5.1a, we designate one agent inR i as itsleaderand specify itsheading

with a free vector (i.e.,fi vi) associated with the center of the leader. We represent the coordinates of

l th agent in formationR i with respect to frameFfi by fi xli , where 0≤ l < Ni, whereNi is the number

of agents inR i. We usel = 0 to represent the leader ofR i . The configuration of a formationR i is

represented by the position of its leaderwx0i in the environment’s frame of referenceFw and angle

wθi the heading vector makes w.r.t. the horizontal axis ofFw, (i.e., the tuple (wx0i ,

wθi) represents a

configuration of the formationR i, as shown in Fig. 5.1b). Given a pointx and a heading directionv,

formationR i is mapped from frameFfi to Fw so that the leader ofR i is positioned atx (i.e.,wxi = x)

and the heading ofR i is aligned withv ( wvi = v) as shown in Fig. 5.1b. For simplicity, we model

agents in all formations as discs with the same radiusr and the same maximum speedfmax.

The problem then is to plan in real-time the motions of thek formations, from their respective

initial configurations to their respective goal positions while maintaining the formations. The for-

mations move in a two dimensional virtual environment (with dynamic obstacles), given as a binary

occupancy grid, and the motions of the dynamic obstacles are not known beforehand. We empha-

size collision avoidance between the formations (over finding shortest paths to goals) by placing a

relatively high value on one of the three weights (i.e.,γ) in the unit cost field (2.6). In the event the

formations are not able to avoid each other, they are allowed to pass through each other as long as

agents in those formations do not collide with each other. As explained later, we usesocial potential

fields[3] to control the agent’s local motions (including collision avoidance). Furthermore, we only

require that the leader of each formation reaches its goal position (i.e., we do not require that the

formation has to reach a certain goal orientation).


5.2 Overall Solution

First we discuss how a formationR is deformed. To deformR , we place theN disc agents inR on a

two dimensional elastic object along withK control nodes. We denote the centers of the disc agents

as internal nodes. The formationR is deformed by displacing theK control nodes. These control

nodes are user specified. Because the BEM is a physically based approach, there is an intuitive

connection between the displacements of the control nodes and the resulting deformations ofR .

For example, a Vee formation of 11 agents is shown in Fig. 5.2a and four user specified control

nodes are indicated by boxes (red in the color version). By displacing the four control nodes, the

Vee formation can be closed (Fig. 5.2c) and opened (Fig. 5.2e), in a very natural manner. The set of

allowable displacements is user-defined.

Given the displacements of theK control nodes, we want to compute the displacements of theN

internal nodes. We model the two dimensional elastic object with the BEM. As a first step in the pro-

cess, the boundary of the elastic object is meshed withquadratic elements(details later), and certain

matrices are computed offline. These matrices allow us to efficiently compute the displacements of

N internal nodes subsequently. Second, from the displacements of the control nodes, we compute

the boundary traction1 by solving a multi-dimensional minimization problem. Third, the boundary

displacements are obtained from the boundary traction. This is referred as theNeumann Problem

(i.e., compute the displacements of the boundary given the traction of the boundary). Once both

the boundary traction and the boundary displacements are known, the displacements of the internal

nodes (i.e., the agents) are easily obtained using the matrices computed earlier in the first step.

This flexible virtual structure approach to formation control is then integrated with the contin-

uum model to plan in real-time motions of multiple formations in frameFw. First, we reduce each

formation to a point by growing the obstacles and then use the continuum model to compute a po-

tential field for the formation. Next, from the gradient of the potential, we generate a sequence of

waypoints for each agent of the formation and then use social potential fields to move the agent

between its waypoints while avoiding collisions with other agents and obstacles. Finally, as shown

in Fig. 5.3, whenever a formation is blocked by obstacles, various “deformations” obtained by mov-

ing the control nodes are tried out and the one that leads to a collision-free motion and avoids the

obstacles is used.

1Traction is related to stress and is defined in the background section.


5.3 Background

In this section, we briefly recapitulate various methods for modeling deformable objects, and explain

the rationale for our choice of the Boundary Element Method. This is followed by a brief overview

of the BEM. Our treatment is mainly to give a basic overview for the reader, further details can be

obtained from the main references cited in each subsection.

5.3.1 The Boundary Element Method

An excellent survey of the work done in modeling deformable objects within the computer graphics

research community was presented in [27] and it focuses on physically based approaches such as

mass-spring models and finite element models. In mass-spring systems, an object is modeled as

a collection of point masses connected by springs in a lattice structure. Mass-spring systems are

widely used because they are easy to construct and enable real-time simulations of deformable ob-

jects. The main drawback of mass-spring systems is the difficulty of tuning spring constants, because

the constants’ proper values are not easy to derive from measured material properties. The Finite El-

ement Method (FEM) can simulate object deformations more physically realistic than mass-spring

systems, but mass-spring systems are still much more popular in computer graphics, especially in

real-time systems, than FEM due to FEM’s high computation requirements. A non-physical method

called Free-form Deformation (FFD) was presented in [78]. FFD models deformation by deforming

the space in which an object lies. It is extremely fast, but even an experienced user may need to

tweak many control nodes patiently to obtain desired deformations [8]. By combining FFD with

ChainMail [26], where ChainMail provides physical properties and FFD deforms the deformable

object, a single deformable object can be deformed in real-time with “reasonable” physical proper-

ties [8]. First, the deformable object is placed inside a bounding box. Because ChainMail performs

simple deformation calculations for each element in the bounding box, the bounding box can be

deformed fast towards a collision-free state if the box is colliding with obstacles. Once the bound-

ing box is collision-free, the deformable object inside the box is deformed by FFD according to the

shape of the box. However, given a formation of agents, the method proposed in [8] may produce

severe undesired local deformations of the formation. For example, as opposed to retaining the rank

formation but making it smaller by moving its agents closer to each other as shown in Fig. 5.7 so

that the formation can fit through the small gap between the obstacles, an agent in the rank formation

would only be displaced if the agent is subject to potential collision. As a result, the formation is no

longer rank formation. Instead, we choose the boundary element method because it enables us to


deform a formation in a controlled and ordered manner and in real-time.

We model formations using isotropic, homogeneous, and linear elastic materials [9]. Materials

are isotropic when their properties are direction independent. Homogeneous materials have the same

properties everywhere. Materials are linear if an unique (linear) relationship can be established

betweenstressσ andstrain ε , where stress is force per unit area inside the material and strain is

defined through a vector of displacementsu. Traction t is related to stress. Given a cutting plane,

traction is defined as the distributed force in any direction per unit area acting on the plane. Stress

and traction are related through a linear equation system [9]. The Boundary Element Method is then

used to solve this elasticity problem of isotropic, homogeneous, and linear elastic materials in two

dimensions.

Boundary Integral Equations And Fundamental Solutions

Boundary Integral Equations (5.1) represents a system of two integral equations which directly relate

tractiont and displacementsu on the boundary of an objectS.

u(P) =∫

StT(Q)U(P,Q)dS(Q)−

∫

SuT(Q)T(P,Q)dS(Q) (5.1)

where

u(Q) =

ux

uy

U(P,Q) =

Uxx Uyx

Uxy Uyy

(5.2)

t(Q) =

tx

ty

T(P,Q) =

Txx Tyx

Txy Tyy

(5.3)

As shown in Fig. 5.8a and Fig. 5.8b,P is the source point (located on the boundaryS) where unit

sources/loads are applied andQ is the field point (located on the boundaryS). U is the fundamental

solution (also called Green’s function or kernel) for the displacements andT is the fundamental

solution for the traction. The first subscript ofU (respectively,T) refers to the direction of a unit

load, and the second subscript relates to the direction of the displacement (respectively, traction) due

to the unit load.

The fundamental solutions for the displacement in thex andy directions due to a unit load in the

x-direction can be written as (Fig. 5.8a)


Uxx(P,Q) = C

C1ln

(

1r

)

+

(

rxr

)2

Uxy(P,Q) = C

(

rxr

)(

ry

r

)

(5.4)

whereC = 1/(8πG(1−ν)), andC1 = 3−4ν . G is the shear modulus andν is the Poisson’ ratio.

For a unit load in they-direction (Fig. 5.8b)

Uyy(P,Q) = C

C1ln

(

1r

)

+

(

ry

r

)2

Uyx(P,Q) = Uxy(P,Q)

(5.5)

The fundamental solutions for the traction due to a unit load atP in thex-direction are

Txx(P,Q) =−C2r

C3 +2

(

rxr

)2

cosθ

Txy(P,Q) =−C2r

2

(

rxr

)(

ry

r

)

cosθ −C3

ny

(

rxr

)

−nx

(

ry

r

)

(5.6)

whereC2 = 1/(4π(1− ν)), C3 = 1− 2ν , cosθ =(

rxr

)

nx +(

ry

r

)

ny. θ and the outward normal

directionn are defined in Fig. 5.8a and 5.8b.

For a unit load in they-direction we have

Tyy(P,Q) =−C2r

C3 +2

(

ry

r

)2

cosθ

Tyx(P,Q) =−C2r

2

(

rxr

)(

ry

r

)

cosθ −C3

nx

(

ry

r

)

−ny

(

rxr

)

(5.7)

The fundamental solutions listed here are for plane strain problems. For our plane stress prob-

lem, we simply substitute an effective Poisson’s ratio ofν = ν/(1+ ν) [9].

The fundamental solutions are often problematic to integrate when the source pointP coincides

with the field pointQ (as will occur, whenP = Q in Boundary Integral Equations (5.1)). The


fundamental solutionU has a lnr singularity and is known asweakly singular. The weak singularity

poses no great problems. The fundamental solutionT, on the other hand, has a 1/r singularity and is

known asstrongly singular, and it does pose serious problem for us, because the integral only exists

in the sense of aCauchy principal value(i.e., the integral can be evaluated on a contour around point

P only). However, it turns out that explicit calculation of these integrals can be avoided. We refer

the readers to [9, 30] for a complete treatment of the integrations of the fundamental solutions.

Numerical Implementation

For our two dimensional elasticity problem, one-dimensional elements are used to describe the

boundary. We choose quadratic elements over linear elements, because more complicated shapes

can be accurately described by a smaller number of elements with three nodes and three quadratic

shape functions. The quadratic shape functions are defined as

N1 = 12(ξ −1)ξ

N2 = 1−ξ 2

N3 = 12(ξ +1)ξ

(5.8)

whereξ is the intrinsic coordinate and−1≤ ξ ≤ 1. The three nodes are placed atξ = −1, ξ = 0,

respectiveξ = 1. The Cartesian coordinates of a point on elementeare

x =3

∑n=1

Nn(ξ )xen (5.9)

whereNn are element shape functions,x is a vector containing coordinates of a point on elemente

andxen is a vector of coordinates of thenth node of elemente. The boundary in Fig. 5.8c is described

by 10 quadratic elements. Each element shares its first and third nodes with its two neighboring

elements. This means that if a boundary consists ofE elements, there are 2E nodes on the boundary.

Now we can write the integral equation (5.1) in discretized form as

cu(Pi)+E

∑e=1

3

∑n=1

△Teniu

en =

E

∑e=1

3

∑n=1

△Uenit

en (5.10)

where


△Ueni =

∫

SeNn(ξ )U(Pi,ξ )dS(ξ )

△Teni =

∫

SeNn(ξ )T(Pi,ξ )dS(ξ )

(5.11)

The kernel shape functions products in (5.11) are integrated over elements, andQ in (5.1) has

been replaced by the local coordinateξ . In (5.10),E is the total number of elements, the number

of nodes per element is 3, 2× 1 vectorsuen and te

n are nodal values, andPi (known as collocation

point in a numerical method called the collocation method) is placed at one of the boundary mesh

nodes [9]. The variables△Ueni,△Te

ni, andc are 2×2 matrices. We refer to [30] for details of how

to evaluatec (often referred to as “free term”) implicitly.

Eq. (5.10) is then evaluated withPi corresponding to each node of the 2E nodes in the boundary

element mesh, and we get 2E vector equations (4E scalar equations) that can be used to solve the

elasticity problem. These equations can be expressed in matrix form as follows [30]

F′u = Gt (5.12)

where 4E×1 vectoru contains all nodal displacements, 6E×1 vectort contains all nodal traction,

F′ is a 4E×4E matrix, andG is a 4E×6E matrix. The size oft is 6E instead of 4E, because the

traction is not constrained to be continuous at the boundary of two neighboring elements, whereas

the displacements are constrained to be continuous [30].

A solution to theNeumann problem(i.e., computeu givent) exists if and only if the equilibrium

condition specified in (5.13) is satisfied.

∫

St(Q)dS(Q) = 0 (5.13)

However, there is not a unique solutionu to (5.12) and all solutions are related to each other

through a rigid-body transformation [30]. Since there are three degrees of freedom in 2D, the rank

of the matrixF′ is 4E−3; thereforeF′ is not invertible. We rewriteF′ using singular value decom-

position (SVD) of the form

F′ = UWVT (5.14)

whereW is a diagonal matrix. A unique solution to the Neumann problem is given by


u = At (5.15)

where 4E×6E matrix A = V[diag(1/w j )]UTG andw j are the singular values and the value 1/w j

is set to zero for the three singular values near zero [30]. This method finds the solution with the

smallest magnitude‖u‖2. A depends only on the object’s geometry so it can be computed offline in

O(E2) time.

Computing Displacements Of The Internal Nodes

Once both the boundary traction and the boundary displacements are known, we can easily obtain

N vector equations that can be used to compute the displacements of theN internal nodes. The

displacement at an internal nodePa inside the domain can be computed by simply rewriting (5.1)

u(Pa) =

∫

StT(Q)U(Pa,Q)dS(Q)−

∫

SuT(Q)T(Pa,Q)dS(Q) (5.16)

The discretized form of (5.16) is written as

u(Pa) =E

∑e=1

3

∑n=1

△Uente

n−E

∑e=1

3

∑n=1

△Tenue

n (5.17)

where

△Uen =

∫

SeU(Pa,Q)NndS(Q)

△Ten =

∫

SeT(Pa,Q)NndS(Q)

(5.18)

Apply (5.17) to allN internal nodes and obtainN vector equations (2N scalar equations) that

can be used to compute the displacements of the internal nodes. These equations can be expressed

in matrix form as follows [30]

uint = Gint t−Fintu (5.19)

where 2N× 1 vectoruint contains displacements ofN internal nodes, the dimensions of matrices

Gint andFint are 2N×6E and 2N×4E, respectively. These two matrices can be computed offline

in O(NE) time.


5.4 A Flexible Virtual Structure Approach For Formation Control

We now present the details of our flexible virtual structure approach to the formation control problem

in virtual environments.

5.4.1 Real-Time Formation Deformation

In order to deform a formationR by displacingK user-defined control nodes on the elastic object

(i.e., inside the boundary), we have to compute the traction of boundary nodest. In this section, we

drop the subscripti from R i and other notations for brevity. In Fig. 5.2a, we have a Vee formation

of 11 agents bounded by a circular boundary, and the four control nodes are designated by (red in

the color version) boxes. Given the desired displacements of the control nodesuctrld , we want to

calculatet that will bring about specified displacements of the control nodes. Because the control

nodes are located inside the boundary, the relationship between the computed displacements of the

control nodesuctrlc andt can be derived from (5.15) and (5.19)

uctrlc = (Gctrl −FctrlA)t (5.20)

where 2K× 1 vectoructrlc contains displacements ofK control nodes, the dimensions of matrices

Gctrl andFctrl are 2K×6E and 2K×4E, respectively. These two matrices can be computed offline

in O(KE) time.

Next, we define an error functionE(t) as theL2-norm of the difference between the desired

displacements of the control nodesuctrld and the computed displacementsuctrlc (i.e., (5.20))

E(t) = ‖uctrld −uctrlc‖ (5.21)

The nodal tractiont that minimizes the error functionE(t) can be computed by solving a multi-

dimensional minimization problem using anunconstrainedmulti-variable minimization technique

called theBroyden-Fletcher-Goldfarb-Shanno(BFGS) method [30]. However, if the BFGS method

is used directly, the resulting boundary tractiont produced by the BFGS method does not satisfy the

equilibrium condition (5.13) and may create numerical instabilities in the solution of the Neumann

problem.


To solve the constrained minimization problem, we recast the constrained problem into an un-

constrained form of reduced dimensionality [31]. We write the error function (5.21) in the form

E(Zy) = ‖uctrld −uctrlc‖ (5.22)

wherey ∈ ℜ6E−2 is an unconstrained vector,Z is a 6E× (6E−2) matrix, andt = Zy. Instead of

computingt ∈ℜ6E directly, we computey∈ℜ6E−2 using the BFGS method by minimizing the error

function (5.22).y is then mapped to a boundary tractiont which satisfies the equilibrium condition

(5.13) byZ (i.e., t = Zy). The matrixZ is derived from the equilibrium condition (5.13). Express

first the equilibrium condition (5.13) in discrete form

BT t =

0

0

(5.23)

whereB is a 6E×2 matrix. The QR decomposition forB can be written as

B = Q

R

0

=[

Q1 Q2

]

R

0

= Q1R (5.24)

whereQ is a 6E×6E orthogonal matrix,R is a 2×2 upper triangular matrix,Q1 is a 6E×2 matrix,

andQ2 is a 6E× (6E−2) matrix. The matrixZ is set toQ2.

Once the nodal tractiont has been determined, the boundary nodal displacementsu are obtained

using (5.15). The displacements of the internal nodes (agents)uint are obtained using (5.19).

Using this method, two different deformations of the Vee formation (Fig. 5.2a) are shown in

Fig. 5.2c and Fig. 5.2e, respectively, corresponding to the displacements of the control nodes.

5.4.2 Fast Collision Detection For Self-Collision Free Deformation

Since our agents are disc shaped, not point-agents, we need to rule out deformations that result

in agents colliding with each others (i.e., we retain only self-collision free deformations). This is

achieved by checking for collisions among the agents after each displacement of the control nodes.

By comparing each agent with every other one in formationR , the collision checking can be done

in O(N2) time, whereN is the number of agents inR . However, if R has large numbers (i.e.,

N > 50 [12]) of agents,spatial partitioning [12] is much faster than the naiveO(N2) collision


checking. First, we divide the two dimensional space whereR is located into a number of cells,

where each cell contains a list of all agents inside the cell. Next, we keep track of all agents while

R deforms. When an agent moves into a new cell, the agent is removed from its old cell’s list and

added to the new cell’s list. With such partitioning, the collision checking takes, on average,O(N)

time, because we need only test collision between an agent and the agents contained in the cells lie

within the agent’s neighborhood. The deformed formation is rejected if any two agents collide after

the deformation. Note that if the virtual environments are not given as occupancy grids, kinetic data

structures (KDS) [6, 5] could potentially be used instead of spatial partitioning.

5.4.3 Generation Of Deformations

We assume thatUctrld = (u1ctrld

, . . . ,umctrld

) – a user-defined list ofm displacements of theK control

nodes – is given and the output is a linear listU f ree of tuples ofuctrld (i.e., the desired displacements

of the control nodes) anduint (i.e., the displacements of the internal nodes (agents) due touctrld).

We can now generate deformations of formationR using the BEM as shown in Algorithm 7.

Algorithm 7 : Generating Deformations using the BEMinput : formationR with N agents, boundary of the elastic shape withE quadratic

elements,K control nodes, and a list of their displacementsUctrld .output: U f ree – a list of tuples ofuctrld anduint

offline : Compute the matricesFctrl , Gctrl , Gint , Fint , A, andZ.repeat7.1

uctrld ← pop(Uctrld);7.2

Compute the unconstrained vectory using the BFGS method;7.3

t = Zy , where the boundary tractiont brings about the desired displacements of the7.4

control nodesuctrld ;Compute the boundary nodal displacementsu usingu = At (i.e., (5.15));7.5

Compute the displacements of the internal nodes (agents)uint (i.e., the formation after7.6

the deformation) usinguint = Gint t−Fintu (i.e., (5.19));Check for collisions between agents after the deformation. If any two agents collide,7.7

the deformed formation is rejected. Otherwise,U f ree = U f ree∪ (uctrld,uint);until U ctrld is empty;7.8

The BEM matricesFctrl , Gctrl , Gint , Fint , A, andZ are pre-computed and depending on the

number of available processors, the loop in Algorithm 7 (i.e., Step 7.1) could be performed either

offline or online.

In the next section, we integrate this flexible virtual structure approach for formation control with


the continuum model to plan in real-time motions of multiple formations in two dimensional virtual

environments, where each formation may be deformed (triggered either automatically or manually)

at runtime.

5.5 Motion Planning Of Formations

We again adapt the continuum model to plan motions of multiple formations in real-time, where

each formation is modeled by our flexible virtual structure approach for formation control.

5.5.1 Construction Of Unit Cost Field

We first construct speed fieldfi and discomfort fieldgi for each formationR i in frameFw where the

formation’s motion is planned. Oncefi andgi are set, the formation’s unit cost fieldCi is given by

(2.6).

Speed

Since our goal is to plan motions of multiple formations and they should normally avoid each other,

we make the assumption in this chapter that the speed fieldfi does not depend on the density of the

agents. In addition, we assume also that a grid cell represents either an obstacle or a flat free space.

Consequently, thetopographical speed fT in this chapter is eitherfree speed ff ree or 0. Recall that

the speed field is an anisotropic fields and it is stored at each face between two neighboring grid

cells. When computing the speed fieldfi , we iterate over each grid cell and then over each of the

four faces of the grid cell. Speed for one face is set tof f ree if and only if both the grid cell and the

neighboring grid cell represent free space, otherwise it is set to 0. In addition, the anisotropic speed

field fi can be used to force formationR i to follow a flow (e.g., a water current). For example, we

can force formationR i to follow a circuitous route such as a roundabout, if speed for each face of

the roundabout grid cells is set tof f ree with the additional requirement that the face is facing the

allowable direction of the roundabout.

Discomfort

When planning path for formationR i, all other formations in the virtual environment are considered

to be obstacles forR i. Because we replan all formations’ paths in each cycle, every formation is

considered to be translating for such incremental motion. Consequently, a formation is simply a


vector of the placements of all agents in the formation. LetR j , where j = 1. . .k and j 6= i, represent

the formations other thanR i . FormationR i can then be reduced to a point by computingC -obstacle

(i.e., the configuration-space obstacles) ofR j due toR i,

CR j := {x0i : R i(x0

i )∩R j 6= /0} (5.25)

whereR i(x0i ) represents placements of all agents inR i with its leader atx0

i , CR j denotes theC -

obstacle and it can be computed usingMinkowski sum[19].

For both static and dynamic obstacles (other than formations), we could use the same Minkowski

sum based computation. However, Minkowski sum computation is expensive, especially when we

want to set as many unit cost fields as possible per second. Consequently, we only use Minkowski

sum for formations because they can come close to and even passing through each other. Instead of

Minkowski sum, we chose a more computationally efficient but conservative method for obstacles

(i.e., grow each occupied grid cell by the radiusrR i of R i(x0i )’s minimum bounding circleCR i ).

By adding discomfort to the resulting occupied/grown grid cells, formationR i will try to avoid

the other formations and obstacles because it prefers grid cells with lower discomfort. Last but not

least, we advance eachCR j by the velocity ofR j for several cycles and add to the discomfort field

gi along the way as in [91] so that two formations (i.e.,R j andR i) anticipate one another when they

cross perpendicularly.

Once unit cost fieldCi for formationR i is set, we construct potentialφi and its gradient∇φi using

the FMM so thatR i (specifically its leader) can move opposite the gradient∇φi until it reaches its

goal position. We now detail how this motion is carried out forR i.

5.5.2 Moving Formation Along Gradient: Construction Of Waypoints

Since formationR i is defined in frameFfi and its motion is planned in frameFw, we must mapR i

from frameFfi to frameFw. Furthermore, in order to move the formation, each of the agents must

be moved. The agents’ positions inFw (after the mapping) are treated aswaypointsand arenot used

for rendering directly. If we were to do the latter, formationR i would essentially behave like a rigid

body and its motion will not look natural at all. Instead social potential fields are used to move each

agent ofR i to its next waypoint, determined as below.

Let t denote the current time, and letR i ’s current configuration w.r.t.Fw be(wx0i (t),

w θi(t)), i.e.,

the leader ofR i is positioned atwx0i (t) and the heading ofR i is aligned with a unit vectorwvi(t)


as shown in Fig. 5.9, wherewvi(t) is determined bywθi(t). The next waypoint for the leader (i.e.,

waypoint at timet + ∆t) is given by:

wx0i (t + ∆t) =w x0

i (t)+w x0i (t)∆t (5.26)

wherewx0i (t) is given by (2.7) and it moves the leader opposite the gradient of the potential function

∇φ(wx0i (t)) scaled by the speed at that point. Note that the translational speed is limited tof f ree.

The waypoints for all other agents are determined as follows. Letwvi(t +∆t) =−∇wφi(wx0

i (t)), the

homogeneous transformationwT fi (t + ∆t) is then constructed to map formationR i from Ffi to pointwx0

i (t +∆t) and align its heading withwvi(t +∆t). (wx0i (t +∆t),w θi(t +∆t)) uniquely determinesR i ’s

configuration w.r.t.Fw at timet +∆t, wherewθi(t +∆t) is determined bywvi(t +∆t). Hence, we can

now determine the waypoints for agents other than the leader via a simple coordinate transformation.

wxli (t + ∆t) =w T fi (t + ∆t) fi xl

i (t + ∆t), where 0< l < Ni (5.27)

In this configuration, an agent in formationR i may have to exceed its maximum speedfmax to reach

its waypoint at timet + ∆t from its current position, because the rotation from−∇wφi(wx0

i (t−∆t))

to−∇wφi(wx0

i (t)) can be substantial.

To limit the rotation from−∇wφi(wx0

i (t−∆t)) to−∇wφi(wx0

i (t)), we can smooth unit cost field

Ci to increase the lower bound of the curvature radius of the optimal path [16]. However, the ef-

ficiency of the smoothing is limited for us because we have to represent virtual environments with

grids with coarse resolution (64×64) due to the real-time constraint. Instead, we setwvi(t +∆t) to a

vector lying in the middle ofwvi(t +∆t) andwvi(t) if any agent inR i needs to exceedfmax to reach its

waypoint at timet + ∆t. We repeat this process until no agent needs to exceedfmax. Consequently,

formationR i changes its orientation gradually.

5.5.3 Social Potential Fields

Once we have constructed a set of waypoints for formation inR i, we usesocial potential fields[3] to

controlR i ’s agents’ local motions. For each agent inR i, we draw the agent from its current position

to its next waypoint with an attractive force. We add also repulsive forces to each agent for collision

avoidance with other agents and obstacles in the neighborhood. The repulsive forces are necessary

because formationR i may pass through other formations and come close to obstacles when there


is no path that pass through areas with low discomfort (e.g., when it has to pass through narrow

passages).

Since we use social potential fields to move agents between their waypoints and avoid collisions,

the agents may run into local minima even though potentials generated by the FMM are free of local

minima analytically. In fact, we did encounter local minima during the experiments (e.g., when

agents go around corners of obstacles).

5.5.4 Formation Deformation

Because formationR i is modeled using our flexible virtual structure approach for formation control,

we can deformR i (by either displacing its control nodes and then computing the deformation in

real-time or fetching the pre-computed deformation) and this can be triggered either automatically

or manually. Our algorithm deforms formationR i automatically wheneverR i cannot reach its goal

position (i.e., wheneverφi(wx0

i (t)) – potential felt byR i at wx0i (t) – is very high because a high

potential means thatR i will collide with obstacles) and tries to find a collision-free path for the

deformed formationR i. The algorithm tries each deformation – determined byuint stored in each

tuple of the listU f ree computed in Algorithm 7 – and recomputesφi(wx0

i (t)). The process is repeated

until φi(wx0

i (t)) becomes low (i.e., there is a collision-free path for the deformed formationR i)

or all tuples inU f ree have been processed (i.e., formationR i cannot reach its goal even after the

deformation). In the latter case, the planner is deemed to have failed.

The pseudo-code for real-time motion planning of multiple formations is summarized in Algo-

rithm 8. It reports failure of finding a collision-free path for formationR i if φi(wx0

i (t)) remains very

high at Step 8.7 for all deformations ofR i (i.e., all uint in mathb fUf ree). In order to get smooth

simulation and rendering, we run Step 8.6 in a separate process than other steps in Loop 8.2 so that

we can update agents’ positions (using social potential fields) more frequently than the computation

of potential fields and waypoints.

5.6 Experiments And Results

We have performed a number of experiments and made a video to show the effectiveness of our

approach on a Dell OptiPlex GX620. The OptiPlex has one Intel Pentium D Processor 820 with two

execution cores running at 2.8GHz, one 256MB ATI Radeon X600 video card, 4 gigabytes of RAM,

and runs Red Hat Enterprise Linux (RHEL) 4. The code was written in ANSI C/C++ and compiled


Algorithm 8 : The Continuum Model for Formations

foreach cycledo8.1

foreach formationR i do8.2

Construct speed fieldfi , discomfort fieldgi , and unit cost fieldCi;8.3

Construct potentialφi and its gradient∇φi using the FMM;8.4

Construct the waypoints such that no agent inR i exceeds its maximum speedfmax;8.5

Update positions ofR i ’s agents using social potential fields;8.6

while ( φi(wx0

i (t)) is very highor a command is given by the user) do8.7

(uctrld ,uint)←GetOneTuple(U f ree);8.8

DeformR i by displacing its agents byuint ;8.9

end8.10

end8.11

end8.12

using GNU GCC 3.4.3. The video can be found athttp://ramp.ensc.sfu.ca/people/

liyi/LI_GUPTA_FORMATIONS_VIDEO.mp4 .

5.6.1 The Flexible Virtual Structure Experiments

First, we present experiments that illustrate effect ofE (number of quadratic elements),K (number

of control nodes), andN (number of agents), respectively on the computation times of the BEM

matrices (i.e.,A, Fctrl , Gctrl , Fint , andGint ) and the deformation time which is dominated by the

time it takes to solve the minimization problem using the BFGS method. We also present many

different formation deformations produced by the BEM to demonstrate its capability.

We use the Vee formation in Fig. 5.2 to illustrate effect of the number of quadratic elementsE on

the computation times of the BEM matrices and the deformation time. We bound the formation with

a circular boundary and mesh the boundary into 12 quadratic elements (i.e.,E = 12) (Fig. 5.2a), then

deform the formation, using four control nodes (i.e.,K = 4). Two different deformations are shown

in Fig. 5.2c and Fig. 5.2e, respectively. The formation with 24 quadratic elements (i.e.,E = 24) and

its deformations are also shown in Fig. 5.2 (i.e., Fig. 5.2b, Fig. 5.2d, and Fig. 5.2f, respectively). The

computation time for the BEM matrices and average computation time (averaged over 20 different

deformations) for one deformation are listed in Table 5.1. When we setE to 12 instead of 24, the

formation is deformed four times faster and hence we can conclude empirically that the deformation

time increases quadratically with the number of quadratic elementsE. At the same time, Fig. 5.2

clearly indicates that 12 quadratic elements can represent the elastic shape properly in numerical


modeling. Consequently, we have to choose anE small enough to achieve real-time performance

when deforming the formation and big enough to represent the elastic shape properly in numerical

modeling.

E = 12 E = 24

Fctrl 20 ms 20 msGctrl 10 ms 20 msFint 30 ms 60 msGint 20 ms 50 msA 140 ms 530 msTime per Deformation 152 ms 603 ms

Table 5.1: Computation time of the BEM matrices in millisecond and average computation time(over 20 different deformations) for one deformation in millisecond for the Vee formation in Fig. 5.2.

The effect of the number of agents in a formationN and the number of control nodesK on the

deformation computation time is checked using six infantry formations of various sizes and various

number of control nodes. A infantry formation with 21 agents (i.e.,N = 21) and two control nodes

(i.e., K = 2) is shown in Fig. 5.10a. Average computation time for one deformation are listed in

Table 5.2. The number of agents in a formationN does not significantly affect the deformation time,

because it takes only three matrix multiplications and one matrix subtraction (see (5.15) and (5.19))

to find the displacements of all agents once the minimization problem (independent ofN) is solved

andt is known. However, the data in Table 5.2 show empirically that the deformation time increases

linearly with the number of control nodesK.

K E N Time per Deformation

2 12 133 85 ms2 12 57 82 ms2 12 21 83 ms

4 12 99 151 ms4 12 55 149 ms4 12 21 150 ms

Table 5.2: Average computation time for one deformation in millisecond for the infantry formations.K is the number of the control nodes,E is the number of control nodes, andN is the number ofagents. The number of deformations was 20 in each case.

The infantry formation in Fig. 5.10a is bounded by an elliptic boundary, while the rest of forma-

tions in Fig. 5.10 are bounded by circular boundaries. The BEM even allows shapes with sharp edges


(e.g., rectangles) as boundaries; however, we find that deformations produced by smooth boundaries

look more natural. As shown in Fig. 5.10b, the infantry formation can by bent by displacing its 2

control nodes. The rank formation in Fig. 5.10c is controlled by 4 control nodes. By displacing the

control nodes, the agents’ positions can be scaled (Fig. 5.10d) to make the entire formation smaller

so that it can pass through a narrow passage. Our flexible virtual structure can achieve more com-

plicated deformations than bending and scaling. For example, we can change the infantry square

formation in Fig. 5.10e to the infantry “circle” formation in Fig. 5.10f. The infantry square forma-

tion in Fig. 5.10g is expanded by displacing 8 control nodes (i.e.,K = 8), and the result is shown in

Fig. 5.10h.

5.6.2 The Motion Planning Experiments

Next, we demonstrate the capability of Algorithm 8 through five experiments shown in Fig. 5.3,

5.4, 5.5, 5.6, 5.11, 5.12, 5.13, and 5.14. All virtual environments and agents were rendered in

3D using OGRE [90] and in a separate process than the planning process [57] that computed the

potential fields for allk formations. In addition, we pre-computed all formations’ deformations of-

fline using the BEM, because the OptiPlex has only two execution cores and not enough to compute

deformations in real-time.

Four screenshots in Fig. 5.11, 5.12, 5.13, and 5.14 show an infantry of 63 agents “bend” while

moving towards its goal. This deformation is useful in the military to encircle the enemy. Compared

to the formation in Fig. 1.2, the infantry in Fig. 5.11, 5.12, 5.13, and 5.14 kept the formation all

the way to the goal and did not split up to avoid the three static obstacles. Note that the screen was

updated at around 65 fps.

Fig. 5.3, 5.4, 5.5, and 5.6 were drawn in postscript using the data collected during the exper-

iments to show the formations’ initial configurations, final configurations, and the paths between

the configurations. In Fig. 5.3, 5.4, 5.5, and 5.6, static obstacles are drawn in dark grey (red in

color prints), each formation’s initial configuration and final configuration are shown as a number of

black circles and black discs, respectively. Note that the black circles/discs represent the agents that

belong to the formation.

• Dynamic Obstacles: The rank formation in Fig. 5.3 demonstrates the ability of our algorithm

to handle dynamic obstacles. When the two dynamic obstacles marked by light grey rectangles

(cyan on color prints) suddenly appeared, the rank formation deformed automatically to make

itself it smaller so that it can fit through the small gap between the two obstacles.


• Circuitous Routes: Our algorithm allows optimal paths that follow circuitous routes, because

it is based on the FMM presented in [92]. For example, in Fig. 5.4, our algorithm moved three

Vee formations counter-clockwise around the obstacle in the middle.

• Deformable Formations: Our algorithm can handle many different formations. Four very

different formations (No. 1: Vee formation; No. 2: rank formation; No. 3: file formation;

No. 4: infantry square formation) are shown in Fig. 5.5 and two of them (i.e., No. 1 Vee

formation and No. 4 infantry square formation) deformed during the simulation. Note that

the deformations were triggered manually.

• Complex Environments: Fig. 5.6 demonstrates that our algorithm can handle complex envi-

ronments with non-convex obstacles. The FMM can overcome local minima induced by e.g.

concavities because goal position (of each formation) is the only global minimum.

In addition, as shown in Fig. 5.11, 5.12, 5.13, 5.14, 5.3, 5.4, 5.5, and 5.6, all formations

changed their orientations gradually because our algorithm prevents agents from exceeding their

maximum speeds.

In each simulation cycle, we have to construct a potential fieldφ over the entire virtual envi-

ronment using the FMM for each formation. Therefore, givenn formations, then potential fields

(one for each formation) are computed inO(n) time in each cycle. This is verified empirically with

experiments on 64×64 grids, where 1, 2, 4, and 6 formations were selected, respectively, and each

experiment was run for 30 seconds. The average FMM running time of one cycle (i.e.,n times the

average running time of Step 8.4 in Algorithm 8) in Table 5.3 increases linearly withn. Because each

formation takes all other formations into account (through the Minkowski sum computations) when

it plans its next move and hence the unit cost fields are constructed inO(n2) time in each simulation

cycle. Therefore, each simulation cycle takesO(n2) time in our current implementation and this is

verified by the average total running time of one cycle (i.e., the average running time of Step 8.2

in Algorithm 8) in Table 5.3. This limits the number of formations our current implementation can

handle in real-time. However, if we utilizespatial partitioning[12] again and let a formation take

another formation into account if and only they are close to each other, then each simulation cycle

can be done, on average,O(n) time instead ofO(n2) time and hence more formations can be handled

in real-time.


Table 5.3: Running Time (millisecond/cycle).No. of Formations FMM Running Time Total Running Time

1 5 ms 12 ms2 10 ms 48 ms4 20 ms 184 ms6 30 ms 364 ms

5.7 Conclusion

In this chapter, we have introduced a real-time multi-formation navigation algorithm that combines

the continuum model for crowd simulation and our flexible virtual structure approach for formation

control in virtual environments. To the best of our knowledge, this motion planning algorithm for

multiple formations is the first one that does not use ad-hoc and local approaches and hence agents

in a formation does not split easily from the formation. We have shown that our method can handle

dynamic obstacles, circuitous routes, deformable formations, and complex environments in real-

time. At runtime, different formations normally avoid each other, but in case this is not feasible (i.e.,

planner is not able to find collision-free motions for formations), we do allow them to go through

each other (e.g., in narrow passages) without any collision between agents.


formation

leader

heading

fi x0i

R i

fi viFfi

(a) A Vee formationR i in frameFfi .

formation

wvi

R i

wx0i

wθi

Fw

(b) The Vee FormationR i in frameFw.

Figure 5.1: A formationR i is mapped from frameFfi , where it is defined, to the environment’s frameof referenceFw.


(a) A Vee formation and the meshed boundarywith 12 quadratic elements.

(b) A Vee formation and the meshed boundarywith 24 quadratic elements.

(c) Closing the Vee formation in Fig. 5.2a. (d) Closing the Vee formation in Fig. 5.2b.

(e) Opening the Vee formation in Fig. 5.2a. (f) Opening the Vee formation in Fig. 5.2b.

Figure 5.2: A Vee formation located on an imaginary circular elastic shape can be deformed in real-time using BEM by displacing the (red in the color version) box control nodes located on the elasticshape. The displacements of the control nodes are represented by arrows.


1

1

Sample No. 4756

Figure 5.3: Path for a rank formation in a virtual environment(64×64 grid). The formation’s initialconfiguration and final configuration are shown as a number of black circles and black discs (whichrepresent the agents), respectively.


1

2

3

1

2

3

Sample No. 1756

Figure 5.4: Paths for three Vee formation in a virtual environment (64×64 grid). Each formation’sinitial configuration and final configuration are shown as a number of black circles and black discs(which represent the agents), respectively.


1

2

3

4

1

2

3

4

Sample No. 4041

Figure 5.5: Paths for four formations in a virtual environment (64×64 grid) without static obstacles.Each formation’s initial configuration and final configuration are shown as a number of black circlesand black discs (which represent the agents), respectively.


1

2

3

4

1

2

3

4

Sample No. 6298

Figure 5.6: Paths for four formations in a virtual environment (64×64 grid) with static obstacles.Each formation’s initial configuration and final configuration are shown as a number of black circlesand black discs (which represent the agents), respectively.


Figure 5.7: The rank formation in [70] is made smaller by moving its agents closer to each other sothat the smaller formation can fit through the small gap between the obstacles.


(a) Notation for 2D fundamental solution (unit load inx-direction).

(b) Notation for 2D fundamental solution (unit load iny-direction).

(c) Boundary Element Mesh.

Figure 5.8: Boundary Element Method.


waypoints at

waypoints at

path segments

t +∆t

−∇wφi(wx0

i (t−∆t))

wvi(t +∆t)

−∇wφi(wx0

i (t))

t

wx0i (t) wvi(t)

wx0i (t +∆t)

wθi(t +∆t)wθi(t)

Fw

Figure 5.9: Construction of waypoints for formationR i at time t + ∆t.


(a) Infantry formation. (b) “Bent” infantry formation.

(c) Rank formation. (d) Compressed rank formation.

(e) Infantry square formation. (f) Infantry “circle” formation.

(g) Infantry square formation. (h) Expanded infantry square formation.

Figure 5.10: Formations and their deformations.


Figure 5.11: An infantry of soldiers move around in a virtual environment (64×64 grid) with threestatic obstacles. Figure No. 1 out a series of 4.

Chapter 6

Conclusion and Future Work

6.1 Conclusion

In this thesis, we studied the problem of real-time motion planning of multiple agents and multi-

ple formations in two dimensional virtual environments (e.g., environments in Real-time Tactical

games) and presented three algorithms to tackle the following challenges offered by RTT games:

1. Multiple agents: There are large numbers of agents moving around in the virtual environ-

ments and those agents must avoid each other.

2. Real-Time: The agents’ motions must be computed in real-time. Thus, the planner must be

able to perform path query in real-time while the game is being played.

3. Dynamic: Virtual environments contain not only static obstacles, but also dynamic obstacles.

4. Complexity: Virtual environments can be complex.

5. Coherence: The resulting motions of the agents in the same formation planned by a planner

must be perceived by humans as being coherent.

6. Inexpensive pre-processing: A planner may not spend more than a few seconds in the pre-

processing phase.

We have used the basic continuum model extensively in this thesis, because it unifies global

planning and collision avoidance, and the efficiency of the FMM is not affected by the complexity

of (i.e. number and shapes of obstacles) of the virtual environment [69]. However, the basic model

99

CHAPTER 6. CONCLUSION AND FUTURE WORK 100

may fail to plan deadlock free paths for multiple agents in narrow passages. We showed how to

combine the basic method with a symbolic AI technique (i.e., coordination graph) to plan motions

of multiple agents in real-time. The rules we designed for coordination graphs are simple and intu-

itive, yet we were able to achieve complex behaviors such as agents’ behaviors inside roundabouts

(i.e., follow the direction of the traffic flow). In order to process multiple CG tasks in real-time, we

proposed to process those tasks in parallel on a SMP system. The task parallelism was implemented

in a supervisor-worker paradigm with Unix processes. The supervisor not only updates each agent’s

position and orientation at each timestep, but also distributes jobs to the workers and receives re-

sults (i.e. optimal joint actions) from the workers. A worker takes order from the supervisor, then

constructs a coordination graph and computes an optimal joint action. We obtained significant, scal-

able (i.e., we can process more coordination graphs in real-time by adding processors) speedups by

constructing and processing coordination graphs in parallel. By decoupling the rendering and the

planning, we were able to render at a higher frame rate than 24 fps.

Next, we proposed an adaptive multi-resolution continuum model to plan motions of agents

with independent goals (i.e., increase the number of agents with independent goals compared to

the basic model, where agents have to be grouped into a few groups with all agents in a group

sharing the same goal in order to main real-time performance), while retaining the advantages of the

basic model. The key to the efficiency of our approach is that we restrict the computation of each

agent’s potential field to a channel, where the channel is constructed/updated a very low frequency

or whenever it is blocked by dynamic obstacle(s). As an intrinsic byproduct, our approach can steer

the agents away from narrow passages to open spaces (if desired), but it will not miss paths that pass

through narrow passages if they are the only options. The experiments show that our adaptive multi-

resolution continuum model can plan motions of a larger number of individual agents on larger grids

compared to the basic model.

Finally, we considered the problem of real-time motion planning of multiple tightly controlled

formations in two dimensional virtual environments, because agents in RTT games often move in

formations. We proposed a novel flexible virtual structure approach to the formation control prob-

lem. The approach conceives of all the agents in a formation as parts of an elastic shape, which is

modeled using the boundary element method. The flexible virtual structure approach is then inte-

grated with the basic continuum model to plan motions of multiple formations in real-time. To the

best of our knowledge, this multi-formation navigation algorithm is the first one that does not use

ad-hoc and local approaches and hence agents in a formation does not split easily from the forma-

tion. We have shown that our method can handle dynamic obstacles, circuitous routes, deformable


formations, and complex environments in real-time.

We believe that these three algorithms can be used as basic motion planning toolkits toward

enhancing the capabilities of RTT games.

6.2 Future Work

In this thesis, we assumed that agents know the current properties of all dynamic obstacles (i.e., we

do not take into account agents’ “line of sight” / visual occlusions). However, it is easy to incorpo-

rate partial visibility — visibility evaluations can be done using the fast marching method [81] —

into the adaptive multi-resolution continuum model where each agent has its own dedicated poten-

tial field. Although we prevented the agents in formations from exceeding their maximum speeds

so that the formations changed their orientations gradually, there is no explicit consideration for

dynamic constraints such as inertia of a moving agent. Such considerations will enhance the visual

impact, however, the challenge would be to meet the real-time requirement while respecting such

constraints [91].

When we coordinated motions of multiple agents using coordination graphs for deadlock avoid-

ance in narrow passages, we stopped these agents whose optimal actions are¬enter by setting their

velocities to zero. However, when there are many agents waiting at an entrance, they may block

agents who want to exit the narrow passage connected to the entrance. Instead, the waiting agents

should be allowed to move around in the entrance (e.g., step to the left/right).

We did not use coordination graphs to coordinate motions of multiple formations in narrow

passages, because we assumed that two formations are allowed to pass through each other in such

situations. Formations may be forbidden to pass through each other in some situations. It that case,

our coordination graphs based scheme can be to extended to coordinate multiple formations. By

adding more rules to the existing sets of rules, the scheme can also be extended to handle dynamic

obstacles. Furthermore, the coordination graphs based deadlock avoidance scheme can also be

incorporated into the adaptive multi-resolution continuum model. Even though the multi-resolution

extension can steer agents away from narrow passages to open spaces, the agents may have to pass

through narrow passages when paths solely in open space do not exist.

The running time of each simulation cycle in the motion planning algorithm we presented in this

thesis for multiple tightly controlled formations grows quadratically with the number of formations

because Minkowski sum computations between the formations is done naively (i.e., a formation,

when planning its next move, takes all other formations into account). More efficient data structures


should be investigated here to make the algorithm scale better with the number of formations. Fur-

thermore, the agents may run into local minima even though potentials generated by the FMM are

free of local minima analytically, because social potential fields are used to move agents between

their waypoints and avoid collisions.

In this thesis, we assumed that the virtual environments are given as binary occupancy grids.

This assumption in combination with the fact that only grids with relatively coarse resolution can be

used due to the real-time constraint means that polygonal obstacles can not be represented accurately

(by binary occupancy grids). We could partition the environments into unstructured meshes (instead

of grids). Note that this will also result in faster computing of potential fields [69] because far

fewer faces (than grid cells) will be needed. The fast marching method has been extended to handle

unstructured meshes [46, 82] and we believe that the algorithms presented in this thesis can also be

extended to handle virtual environments given as unstructured meshes. Another way to speed up

computation of potential fields is to perform the fast marching on multiple processes mapped onto

multiple processors. Potentials in this thesis were computed sequentially on a single process.

The three algorithms we presented in this thesis are primarily designed for virtual environments

with a centralized implementation. Although it is not our concern in this thesis, one could consider

extending it to multi-robot systems. Any such extension will need to consider several non-trivial

issues, such as distributed implementation, communication constraints, sensing, etc.

Appendix A

DVD Movies

The DVD attached forms a part of this work. The DVD may be played in any DVD player.

Movies:

• LI GUPTA COORDINATION VIDEO (1:46 minutes)

• LI GUPTA ADAPTIVE VIDEO (2:26 minutes)

• LI GUPTA FORMATIONS VIDEO (4:52 minutes)

103

Bibliography

[1] Ken Alton and Ian M. Mitchell. Optimal path planning under different norms in continuousstate spaces. InProceedings of the IEEE International Conference on Robotics and Automa-tion, pages 866–872, 2006.

[2] Kenneth R. Baker. Introduction to Sequencing and Scheduling. John Wiley and Sons Inc,1974.

[3] T. Balch and M. Hybinette. Social potentials for scalable multi-robot formations. InProceed-ings of IEEE International Conference on Robotics and Automation, pages 73–80, 2000.

[4] J. Barraquand and J. C. Latombe. Robot motion planning: A distributed representation ap-proach.The International Journal of Robotics Research, 10(6):628–649, 1991.

[5] J. Basch, J. Erickson, L. J. Guibas, J. Hershberger, and Li Zhang. Kinetic collision detectionbetween two simple polygons.ELSEVIER: Computational Geometry: Theory and Applica-tions, 27(3):211–235, 2004.

[6] J. Basch, L. J. Guibas, and J. Hershberger. Data structures for mobile data. InProceedings ofthe Eighth Annual ACM-SIAM Symposium on Discrete Algorithms, pages 747–756, 1997.

[7] O. Burchan Bayazit, Jyh-Ming Lien, and Nancy M. Amato. Better group behaviors in complexenvironments using global roadmaps. InProceedings of International conference on Artificiallife, pages 362–370, 2003.

[8] Osman Burchan Bayazit, Lien Jyh-Ming, and Nancy M. Amato. Probabilistic roadmap motionplanning for deformable objects. InProceedings of IEEE International Conference on Roboticsand Automation, volume 2, pages 2126–2133, 2002.

[9] Gernot Beer.Programming the boundary element method: an introduction for engineers. JohnWiley and Sons, 2001.

[10] Jacek Blazewicz, Klaus H. Ecker, Erwin Pesch, Gunter Schmidt, and Jan Weglarz.Schedulingcomputer and manufacturing processes. Springer, 2001.

[11] O. Brock and O. Khatib. Elastic strips: A framework for motion generation in human environ-ments.International Journal of Robotics Research, 18(6):1031–1052, 2002.

104

BIBLIOGRAPHY 105

[12] Mat Buckland.Programming game AI by example. Wordware Publishing, Inc., Plano, Texas,2005.

[13] Z. Butler. Corridor planning for natural agents. InProceedings of IEEE International Confer-ence on Robotics and Automation, pages 499–504, 2006.

[14] Rohit Chandra, Leonardo Dagnum, Dave Kohr, Dror Maydan, Jeff McDonald, and RameshMenon. Parallel programming in OpenMP. Morgan Kaufmann Publishers, San Francisco,CA, 2001.

[15] Howie Choset, Kevin M. Lynch, Seth Hutchinson, George Kantor, Wolfram Burgard, Lydia E.Kavraki, and Sebastian Thrun.Principles of Robot Motion: Theory, Algorithms, and Imple-mentations. The MIT Press, 2005.

[16] Laurent D. Cohen and Ron Kimmel. Global minimum for active contour models: A minimalpath approach.International Journal of Computer Vision, 24(1):57–78, 1997.

[17] A. K. Das, R. Fierro, V. Kumar, J. P. Ostrowski, J. Spletzer, and C. J. Taylor. A vision-basedformation control framework.IEEE Transactions on Robotics and Automation, 18(5):813–825, 2002.

[18] Ingrid Daubechies. Ten Lectures on Wavelets. SIAM: Society for Industrial and AppliedMathematics, 1992.

[19] Mark de Berg, Marc van Kreveld, Mark H. Overmars, and Otfried Schwarzkopf.Computa-tional Geometry: Algorithms and Applications. Springer-Verlag, 2000.

[20] E. W. Dijkstra. A note on two problems in connection with graphs.Numerische matematik 1,pages 269–271, 1959.

[21] M. Erdmann and T. Lozano-Perez. On multiple moving objects. InProceedings of IEEEInternational Conference on Robotics and Automation, volume 3, pages 1419–1424, 1986.

[22] E. Fabrizi and A. Saffiotti. Extracting topology-based maps from gridmaps. InProceedingsof IEEE International Conference on Robotics and Automation, volume 3, pages 2972–2978,2000.

[23] Russell Gayle, Avneesh Sud, Ming C. Lin, and Dinesh Manocha. Reactive deformingroadmaps: Motion planning of multiple robots in dynamic environments. InProceedings ofIEEE/RSJ International Conference on Intelligent Robots and Systems, 2007.

[24] Roland Geraerts and M. H. Overmars. The corridor map method: A general framework forreal-time high-quality path planning. InProceedings of IEEE International Conference onRobotics and Automation, 2007.

[25] Roland Jan Geraerts.Sampling-based Motion Planning: Analysis and Path Quality. PhDthesis, Universiteit Utrecht, Utrecht, the Netherlands, 2006.http://igitur-archive.

BIBLIOGRAPHY 106

library.uu.nl/dissertations/2006-0509-200010/index.htm , last ac-cessed: August 20, 2008.

[26] Sarah F. F. Gibson. 3d chainmail: a fast algorithm for deforming volumetric objects. InProceedings of Symposium on Interactive 3D Graphics, pages 149–154, 1997.

[27] Sarah F. F. Gibson and Brian Mirtich. A survey of deformable modeling in computer graphics.Technical Report TR-97-19, Mitsubishi Electric Research Lab., 1997.

[28] Rafael C. Gonzalez and Richard E. Woods.Digital Image Processing. Prentice Hall, UpperSaddle River, New Jersey, 2002.

[29] S. Gottschalk, Ming C. Lin, and Dinesh Manocha. OBBTree: A hierarchical structure for rapidinterference detection.Computer Graphics, 30:171–180, 1996.

[30] M. A. Greminger and B. J. Nelson. Deformable object tracking using the boundary elementmethod. InProceedings of IEEE Computer Society Conference on Computer Vision and Pat-tern Recognition, volume 1, pages 289–294, 2003.

[31] M. A. Greminger and B. J. Nelson. Boundary element deformable object tracking with equi-librium constraints. InProceedings of IEEE International Conference on Robotics and Au-tomation, pages 3896–3901, 2004.

[32] Carlos E. Guestrin, Daphne Koller, and Ronald Parr. Multiagent planning with factored MDPs.In Proceedings of Advances in Neural Information Processing Systems, pages 1523–1530,2001.

[33] Carlos E. Guestrin, Shobha Venkataraman, and Daphne Koller. Context-specific multiagentcoordination and planning with factored MDPs. InProceedings of National Conference onArtificial Intelligence, pages 253–259, 2002.

[34] Carlos Ernesto Guestrin.Planning under uncertainty in complex structured environments.PhD thesis, Stanford University, Stanford, CA, USA, 2003.http://ai.stanford.edu/

˜ guestrin/Publications/Thesis/thesis.pdf , last accessed: August 20, 2008.

[35] Kenneth E. Hoff, Andrew Zaferakis, Ming C. Lin, and Dinesh Manocha. Fast and simple 2dgeometric proximity queries using graphics hardware. InProceedings of ACM Symposium onInteractive 3D Graphics, pages 145–148, 2001.

[36] Kenneth E. Hoff, Andrew Zaferakis, Ming C. Lin, and Dinesh Manocha. Fast 3d geometricproximity queries between rigid and deformable models using graphics hardware acceleration.Technical Report TR02-004, Department of Computer Science, University ofNorth Carolina,2002.

[37] D. Hsu, R. Kindel, J. C. Latombe, and S. Rock. Randomized kinodynamic motion planningwith moving obstacles.International Journal of Robotics Research, 21(3):233–255, 2002.

BIBLIOGRAPHY 107

[38] D. Hsu, J. C. Latombe, and R. Motwani. Path planning in expansive configuration spaces.International Journal of Computational Geometry and Applications, 9(4/5):495–512, 1999.

[39] A. Kamphuis and M. H. Overmars. Finding paths for coherent groups using clearance. InProceedings of ACM SIGGRAPH/Eurographics symposium on Computer animation, pages19–28, 2004.

[40] A. Kamphuis and M. H. Overmars. Motion planning for coherent groups of entities. InPro-ceedings of IEEE International Conference on Robotics and Automation, volume 4, pages3815–3822, 2004.

[41] K. Kant and S. W. Zucker. Toward effcient trajectory planning: The path-velocity decomposi-tion. International Journal of Robotics, 5(3):72–89, 1986.

[42] L. E. Kavraki and J. C. Latombe. Randomized preprocessing of configuration for fast pathplanning. InProceedings of IEEE International Conference on Robotics and Automation,volume 3, pages 2138–2145, 1994.

[43] L. E. Kavraki, P. Svestka, J. C. Latombe, and M. H. Overmars. Probabilistic roadmaps forpath planning in high-dimensional configuration spaces.IEEE Transactions on Robotics andAutomation, 12(4):566–580, 1996.

[44] Lydia Kavraki and Steve LaValle.Motion Planning, chapter 5, pages 109–131. SpringerHandbook of Robotics. Springer, 2008.

[45] O. Khatib. Real-time obstacle avoidance for manipulators and mobile robots.InternationalJournal of Robotics Research, 5(1):90–98, 1986.

[46] R. Kimmel and J. A. Sethian. Computing geodesic paths on manifolds. InProceedings of theNational Academy of Sciences of the United States of America, volume 95, pages 8431–8435,1998.

[47] R. Kimmel and J. A. Sethian. Optimal algorithm for shape from shading and path planning.Journal of Mathematical Imaging and Vision, 14(3):237–244, 2001.

[48] S. Koenig and M. Likhachev. Improved fast replanning for robot navigation in unknown ter-rain. InProceedings of IEEE International Conference on Robotics and Automation, volume 1,pages 968–975, 2002.

[49] Jelle R. Kok. Coordination and Learning in Cooperative Multiagent Systems. PhD thesis,University of Amsterdam, Amsterdam, the Netherlands, 2006.http://staff.science.uva.nl/ ˜ jellekok/publications/2006/Kok06thesis.pdf , last accessed:August 20, 2008.

[50] Jelle R. Kok, Matthijs T. J. Spaan, and Nikos Vlassis. Multi-robot decision making usingcoordination graphs. InProceedings of International Conference on Advanced Robotics, pages1124–1129, 2003.

BIBLIOGRAPHY 108

[51] J. J. Kuffner and S. M. LaValle. RRT-connect: An efficient approach to single-query pathplanning. InProceedings of IEEE International Conference on Robotics and Automation,volume 2, pages 995–1001, 2000.

[52] A. M. Ladd and L. E. Kavraki. Theoretic analysis of probabilistic path planning.IEEE Trans-actions on Robotics and Automation, 20(2):229–242, 2004.

[53] Eric Larsen, Stefan Gottschalk, Ming C. Lin, and Dinesh Manocha. Fast proximity querieswith swept sphere volumes. InProceedings of IEEE International Conference on Roboticsand Automation, pages 3719–3726, 2000.

[54] J. C. Latombe.Robot Motion Planning. Kluwer Academic Publishers, Norwell, MA, USA,1991.

[55] S. M. LaValle and J. J. Kuffner. Randomized kinodynamic planning. InProceedings of IEEEInternational Conference on Robotics and Automation, volume 1, pages 473–479, 1999.

[56] M. A. Lewis and Kar-Han Tan. High precision formation control of mobile robots using virtualstructures.Autonomous Robots, 4(4):387–403, 1997.

[57] Yi Li and Kamal Gupta. Motion planning of multiple agents in virtual environments on parallelarchitectures. InProceedings of IEEE International Conference on Robotics and Automation,pages 1009–1014, 2007.

[58] T. Lozano-Perez and M. A. Wesley. An algorithm for planning collision-free paths amongpolyhedral obstacles.Communications of the ACM, 22(10):560–570, 1997.

[59] E. Mazer, J. M. Ahuactzin, and P. Bessiere. The ariadne’s clew algorithm.Journal of ArtificialIntelligence Research, 9:295–316, 1998.

[60] P. Melchior, B. Orsoni, O. Lavialle, A. Poty, and A. Oustaloup. Consideration of obstacledanger level in path planning using A* and fast-marching optimisation: comparative study.Signal Processing, 83(11):2387–2396, 2003.

[61] N. J. Nilsson. A mobile automation: An application of intelligence techniques. InProceedingsof International Conference on Artificial Intelligence, pages 509–520, 1969.

[62] P. O’Donnell and T. Lozano-Periz. Deadlock-free and collision-free coordination of two robotmanipulators. InProceedings of IEEE International Conference on Robotics and Automation,volume 1, pages 484–489, 1989.

[63] C. O’Dunlaing and C. K. Yap. A retraction method for planning the motion of a disc.Journalof Algorithms, 6:104–111, 1985.

[64] M. J. Osborne and A. Rubinstein.A Course in Game Theory. MIT Press, 1994.

[65] M. H. Overmars. Algorithms for motion and navigation in virtual environment and games. InProceedings of International Workshop on Algorithmic Foundations of Robotics, 2002.

BIBLIOGRAPHY 109

[66] Mark H. Overmars. Motion planning in virtual environments and games, 2004.http://www.give.nl/movie/publications/ERCIM.php , last accessed: August 20, 2008.

[67] D. K. Pai and L. M. Reissell. Multiresolution rough terrain motion planning.IEEE Transac-tions on Robotics and Automation, 14(1):19–33, 1998.

[68] Roger E. Pedersen.Game Design Foundations. Wordware Publishing, 2003.

[69] Clement Petres, Yan Pailhas, Pedro Patron, Yvan Petillot, Jonathan Evans, and David Lane.Path planning for autonomous underwater vehicles.IEEE Transactions on Robotics, 23(2),2007.

[70] Dave Pottinger. Implementing coordinated movement.Game Developer, February:48–58,1999.

[71] S. Quinlan and O. Khatib. Elastic bands: connecting path planning and control. InProceedingsof IEEE International Conference on Robotics and Automation, volume 2, pages 802–807,1993.

[72] Lennart Rade and Bertil Westergren.Beta: Mathematics Handbook: Concepts, Theorems,Methods, Algorithms, Formulas, Graphs, Tables. CRC Press, 1993.

[73] C. W. Reynolds. Flocks, herds and schools: a distributed behavioural model. InProceedingsof the 14th annual conference on Computer graphics and interactive techniques (SIGGRAPH’87), pages 25–34, 1987.

[74] C. W. Reynolds. Steering behaviors for autonomous characters. InProceedings of GameDevelopers Conference, pages 763–782, 1999.

[75] Marc J. Rochkind.Advanced UNIX Programming. Addison-Wesley Professional, 2004.

[76] G. Sanchez and J. C. Latombe. Using a PRM planner to compare centralized and decou-pled planning for multi-robot systems. InProceedings of IEEE International Conference onRobotics and Automation, volume 2, pages 2112–2119, 2002.

[77] J. Schwartz and M. Sharir. On the piano movers’ problem: Ii. general techniques for computingtopological properties of real algebraic manifolds.Advances in Applied Mathematics, 4:298–351, 1983.

[78] T. W. Sederberg and S. R. Parry. Free-form deformation of solid geometric models.ComputerGraphics, 20(4):151–160, 1986.

[79] J. A. Sethian. A fast marching level set method for monotonically advancing fronts. InPro-ceedings of the National Academy of Sciences of the United States of America, volume 93,pages 1591–1595, 1995.

[80] J. A. Sethian. Fast marching methods.SIAM Review, 41(2):199–235, 1999.

BIBLIOGRAPHY 110

[81] J. A. Sethian.Level Set Methods and Fast Marching Methods: Evolving Interfaces in Com-putational Geometry, Fluid Mechanics, Computer Vision, and Materials Science. CambridgeUniversity Press, Cambridge, UK, 1999.

[82] J. A. Sethian and A. Vladimirsky. Fast methods for the eikonal and related Hamilton-Jacobi equations on unstructured meshes.Proceedings of the National Academy of Sciences,97(11):5699–5703, 2000.

[83] T. Simeon, S. Leroy, and J. P. Lauumond. Path coordination for multiple mobile robots: aresolution-complete algorithm.IEEE Transactions on Robotics and Automation, 18(1):42–49,2002.

[84] A. Stentz. The focussed D* algorithm for real-time replanning. InProceedings of the Four-teenth International Joint Conference on Artificial Intelligence, volume 2, pages 1652–1659,1995.

[85] E. J. Stollnitz, A. D. DeRose, and D. H. Salesin. Wavelets for computer graphics: a primer 1.IEEE Computer Graphics and Applications, 15(3):76–84, 1995.

[86] E. J. Stollnitz, A. D. DeRose, and D. H. Salesin. Wavelets for computer graphics: a primer 2.IEEE Computer Graphics and Applications, 15(4):75–85, 1995.

[87] Avneesh Sud, Erik Andersen, Sean Curtis, Ming Lin, and Dinesh Manocha. Real-time pathplanning for virtual agents in dynamic environments. InProceedings of IEEE Virtual Reality,2007.

[88] P. Svestka and M. H. Overmars. Motion planning for car-like robots, a probabilistic learningapproach.International Journal of Robotics Research, 16:119–143, 1997.

[89] The MOVIE Team at Utrecht University. Coordinating motion for different groups. Tech-nical Report Deliverable 4.2, Utrecht University, 2006.http://www.give.nl/movie/viewPublicDeliverables.php , last accessed: August 20, 2008.

[90] The OGRE Team. OGRE 3D: Open source graphics engine.http://ogre3d.org/ , lastaccessed: August 20, 2008.

[91] Adrien Treuille, Seth Cooper, and Zoran Popovic. Continuum crowds. InProceedings of the33rd annual conference on Computer graphics and interactive techniques (SIGGRAPH ’06),pages 1160–1168, 2006.

[92] J. N. Tsitsiklis. Efficient algorithms for globally optimal trajectories.IEEE Transactions onAutomatic Control, 40(9):1528–1538, 1999.

[93] J. P. van den Berg and M. H. Overmars. Prioritized motion planning for multiple robots. InProceedings of IEEE/RSJ International Conference on Intelligent Robots and Systems, pages2217–2222, 2005.

BIBLIOGRAPHY 111

[94] Jur van den Berg, Ming C. Lin, and Dinesh Manocha. Reciprocal velocity obstacles for real-time multi-agent navigation. InProceedings of IEEE International Conference on Roboticsand Automation, 2008.

[95] Jur van den Berg, Sachin Patil, Jason Sewall, Dinesh Manocha, and Ming C. Lin. Interactivenavigation of individual agents in crowded environments. InProceedings of Symposium onInteractive 3D Graphics and Games (I3D), 2008.

[96] Jur Pieter van den Berg.Path Planning in Dynamic Environments. PhD thesis, Univer-siteit Utrecht, Utrecht, the Netherlands, 2007.http://igitur-archive.library.uu.nl/dissertations/2007-0404-200447/index.htm , last accessed: August20, 2008.

[97] C. W. Warren. Multiple robot path coordination using artificial potential fields. InProceedingsof IEEE International Conference on Robotics and Automation, pages 500–505, 1990.

[98] Liangjun Zhang, Young J. Kim, and Dinesh Manocha. A hybrid approach for complete motionplanning. InProceedings of IEEE/RSJ International Conference On Intelligent Robots andSystems, pages 7–14, 2007.

real-time motion planning of multiple agents and...

Documents