Heterogeneous Unmanned Networked Teams

George J. Pappas

School of Engineering and Applied Sciences

University of Pennsylvania

UXV Proliferation

UXV Proliferation

Heterogeneous Unmanned Networked Teams

Heterogeneous teams of UXVs must monitor and protect large and complex areas continuously

Search for threats, identify them, track them, neutralize them

Allocate and re-allocate tasks to different agents depending on sensor modalities, physical capabilities.

Persist in the presence of communication, sensor, mission constraints

Must dynamically collaborate with humans

The HUNT Mission

HUNT will push the state-of-the-art in complex, time-critical mission planning and execution for large numbers of heterogeneous vehicles collaborating with humans

Prior DoD efforts – ONR IA

Heterogeneous coverage with spatio-temporal constraints

Prior DoD efforts – ONR IA

Heterogeneous coverage with spatio-temporal constraints

Prior DoD efforts – DARPA HURT

Heterogeneous coverage with spatio-temporal specification

Prior DoD efforts – DARPA HURT

Heterogeneous coverage with spatio-temporal specification

Fundamental Challenges

Problems become quickly intractable - approaches are not inherently scalable to more than 4-5 agents

Coordination approaches do not explicitly incorporate differentiated roles based on individual UXV characteristics

Performance guarantees of safety, convergence, (sub)optimality, are very hard to establish

The HUNT Philosophy

HUNT will address these fundamental challenges by taking abroader interdisciplinary perspective to solve them:

1. (HUNTmates) Interdisciplinary team of researchers covering artificial intelligence, UXV control and robotics

2. (BioThinkTank) Biology, political science, cognitive psychology may offer solution templates in similar hard problems found in nature

HUNTmates

HUNTmates

Interdisciplinary team of pioneering researchers

Represent expertise in various forms of UXVs

Historical commitment to collaborative research

Historical commitment to bio-inspired approaches

Strong emphasis on formal approaches with guarantees

Impressive record of DoD transition activities

BioThinkTank

Daron Acemoglu, M.I.T.

Political economy

Coalition formation

Harvey Rubin, Penn

Genetic & metabolic networks,

persistence, tuberculosis.

David White, Penn

Animal behavior & communication

Evolutionary psychology.

John Vucetich, Michigan Tech

Population biology,

Wolf predatory behavior.

Simon Levin, Princeton

Mathematical biology

Evolutionary biology

David Skelly, Yale

Animal patterns in

Amphibious animals

Eric Horvitz, Microsoft

Cognitive computing

Human-automation

Julia Parrish, Washington

Cooperation in

Marine animals

BioThinkTank

Act as consultants for HUNTmates

Have expertise in traditionally separated domains

Are funded to visit all institutions and project meetings

As project evolves, consultants may be added or subtracted

Are NOT responsible for any project deliverables

HUNT Technical Approach

Task 1: Cataloging, modeling, and analysis of biological behaviors

Task 2: Biologically-inspired heterogeneous cooperation

Task 3: Cooperative behaviors in communication-degraded environments

Task 4: Distributed versus centralized optimization for networked control

Task 5: Embedded humans for mixed-initiative control

Task 6: Experimentation and validation

Task 1: Cataloging, modeling, and analysis of biological behaviors

Key Challenges:

Cataloging group behaviors in biology involving cross-species cooperation

Cataloging task allocation and role assignment in groups of intelligent animals

Develop mathematical models of heterogeneous and cross-species cooperation

Develop algorithms for automatic extraction of spatio-temporal behaviors

Personnel: Stephen Pratt (Lead) BioThinkTank , Vijay Kumar, Tucker Balch

Task 2: Biologically-inspired heterogeneous vehicle cooperation

Key Challenges:

Coalition formation and decision making for surveillance and coverage

Task and role assignments in heterogeneous teams

Biologically-inspired pursuit-evasion games for vehicle teams

Biologically-inspired formations of heterogeneous teams

Personnel: Ron Arkin (Lead) BioThinkTank, George Pappas, Vijay Kumar, Shankar

Sastry, Magnus Egerstedt, Dan Koditschek

Task 3: Cooperation in communication degraded environments

Key Challenges:

Cooperation over communication degraded environments

Distributed connectivity and topology control

Communication-aware motion planning and control

Personnel: Vijay Kumar (Lead) BioThinkTank, Ali Jadbabaie, Karl Hedrick, George

Pappas.

Task 4: Distributed versus centralized optimization for networked control

Key Challenges:

Optimization-based control for spatio-temporal specifications

Optimization based control for heterogeneous UXVs

Dual decomposition techniques for distributing optimization problems

Personnel: Ali Jadbabaie (Lead) BioThinkTank, Claire Tomlin, Vijay Kumar, Shankar Sastry,

George Pappas.

Task 5: Embedded humans for mixed-initiative systems

Key Challenges:

Stochastic hybrid systems control mixed-initiative systems

Compositionality of behaviors for responsiveness and robustness

Natural languages for mission specification

Personnel: Claire Tomlin (Lead) BioThinkTank, Vijay Kumar, Ron Arkin, George

Pappas.

Task 6: Experimentation and Validation

Key Challenges:

HUNT simulation for heterogeneous platform integration

Development of HUNT experimental testbeds

Individual and integrated experimentation

Personnel: Karl Hedrick (Lead) BioThinkTank, Vijay Kumar, Ron Arkin, Claire

Tomlin, George Pappas.

Project Schedule and Deliverables

Year 1 deliverables are all working papers

Year 2 deliverables also include algorithms and individual simulation experiments

Year 3 deliverables also include individual Tasks 5 and Task 6 experiments

Year 4 deliverables include an integrated experiment

Year 5 deliverables include an integrated demonstration

Institute Leads

Task Leads

TASK 1 LEAD TASK 2 LEAD TASK 3 LEAD TASK 4 LEAD TASK 5 LEAD TASK 6 LEAD

OVERALL LEAD

Synergetic ActivitiesLong record of interdisciplinary workshop organization, collaborative

research, integrated experiments, joint workshop organization, and joint publications.

Head start: ICRA 2008 Workshop on Cooperative Control of Multiple Heterogeneous UAVs for Coverage and Surveillance, Pasadena, CA.

Special Issue in IEEE Robotics and Automation Magazine (Deadline: October 15, 2008)

We will organize two high profile, community building workshops (one in base period, one in option period) with edited volumes as deliverables

Visiting students across institutions and cross-institute Ph.D. Committee supervision

DoD Transition

Impressive team record of industrial and DoD transition Berkeley UUVs transitioning to USNA and NUWC ONR AINS formation flying to ACR ONR STTR Penn/Lockheed contributes to ONR IA project

HUNT briefing to CNO Strategic Studies Group (Nov. 17)

HUNT may impact DARPA DSO FunBio program

HUNT research may impact ARL MAST Autonomy Project

Project Website

http://www.seas.upenn.edu/hunt/

Some solutions

Heterogeneous networks in complex environments

Mission specification languages

Heterogeneous Networks: MotivationConnected coverage requirement in mobile robotics applications (surveillance, coverage)

Problem: Deploy a network of robots so that communication is established between different locations

in complex environments.

Heterogeneous Networks: MotivationConnected coverage requirement in mobile robotics applications (surveillance, coverage)

Problem: Deploy a network of robots so that communication is established between different locations

in complex environments.

Heterogeneous Networks

Problem: Deploy a network of robots so that communication is established between different locations

in complex environments.

Heterogeneous Networks

Problem: Deploy a network of robots so that communication is established between different locations

in complex environments.

ConnectivityControl

LocationAssignment

Heterogeneous Networks

Problem: Deploy a network of robots so that communication is established between different locations

in complex environments.

ConnectivityControl

LocationAssignment

Leaders Relays

Heterogeneous Network

Challenges

Distributed Connectivity Control - Local estimates of the network topology

- Auction-based link deletion

Role Assignment in a Heterogeneous Team - Auction-based leader election / reelection (for leader failures)

- Leaders are assigned locations of interest

- Relays assist leaders in completing their tasks

Complex Environments - Environment interference on signal strength

- Geodesic paths

GRASP Lab floor

Target Assignment: Problem Definition

Problem: Given a group of robots and no a prioriassignment information, design distributed control lawsso that distinct agents are assigned to distinct targets.

Target Assignment: Problem Definition

Problem: Given a group of robots and no a prioriassignment information, design distributed control lawsso that distinct agents are assigned to distinct targets.

Our Approach

Desired Properties

Dynamically determinean assignment during navigationDistributed (local information)

Scalable (polynomial complexity)

From single-destination navigation functions…

(E. Rimon & D. Koditschek, 1992)

to multi-destination potential fields…

Discrete coordinationprotocols to ensureliveness & safety

Multi-Destination Potential Fields

Theorem: is free of local minima, other than the destinations.

Proof sketch: is harmonic

Coordinates of agent i and destination k, respectively.

Multi-Destination Potential:

Estimate of the available destinations:

Discrete Coordination ProtocolsAssumptions

Every robot knows the position of all destinations.Limited sensing/communication range R

Distributed CoordinationStep 1: “Select” an available target from a set .Step 2: Visit target and if it is free establish an assignment.Step 3: Update an estimate of taken and available targets (index sets) and

Discrete Coordination ProtocolsAssumptions

Every robot knows the position of all destinations.Limited sensing/communication range R

Distributed CoordinationStep 1: “Select” an available target from a set .Step 2: Visit target and if it is free establish an assignment.Step 3: Update an estimate of taken and available targets (index sets) and

Coordination via Market-Based Control

Market-Based CoordinationStep 1: Update an estimate of taken and available targets (index sets):

and

Step 2: Select an available target and an associated bid.Step 3: Among all neighbors bidding for the same target, the highest bid wins.

ChallengeNegotiate destinations before physically exploring them.

No specific destination to negotiate

Select a destination in a (eg. the closest one)

Subject to market-based negotiation

The Hybrid Agent

Assignment(Market-Based Coordination)

* Estimate available targets* Bid for an available target

* Among all neighbors bidding for the same target, the highest bid wins

Navigation

if target k is updated

positions, targets and bids from all neighbors

updated target and bid

current target k

Distributed: Only nearest neighbor information used

Scalable: At most O(n2) assignments explored before convergence

Provably Correct: Convergence to an assignment is guaranteed

Theorems & Results

Theorems & Side Results

Can handle communication limitations & time delays

Extendible to handle objectives such as collision and obstacle avoidance

Further Characteristics

M. M. Zavlanos, G. J. Pappas, IEEE T-RO, 2008.

Experimentation*

Platform: differential-drive robots (stepper motors)Tracking System: vision tracking system & robot odometry fused via Extended Kalman FilterImplementation: C++ using the open-source robotics software Player (TCP communications), part of the Player/Stage/Gazebo projectResults: Verify integrity and correctness of the asynchronous and parallel computation as well as message passing with time delays.

Courtesy of N. Michael and V. Kumar

*N. Michael, M. M. Zavlanos, V. Kumar and G. J. Pappas, ICRA 2008

Scalability & Potential Applications

Formation Stabilization

Self Assembly (Termite mounds)

Modular Robotics

M. M. Zavlanos, G. J. Pappas, IEEE CDC, 2007.

Scalability & Potential Applications

Formation Stabilization

Self Assembly (Termite mounds)

Modular Robotics

M. M. Zavlanos, G. J. Pappas, IEEE CDC, 2007.

1

2

3

4 5

6

Connectivity & Network Topology

Graph:

Vertex Set:

Edge Set:

Algebraic Representation

Adjacency Matrix Laplacian Matrix

Lemma: with eigenvector 1. Also, if , then is connected.

Potential Fields for Connectivity

State-Dependent Network:

Time Varying Edge Set:

Potential Field:

with a projection matrix to

Control Law:

M. M. Zavlanos, G. J. Pappas, IEEE T-RO, August 2007.

Connectivity modeled as an obstacle

Local ComputationGlobal Information

A Fully Distributed Approach

Desired Properties

Addition and deletion of links in Mobile Robotic NetworksAny network structure, from very sparse to very dense

Distributed (use only nearest neighbor information)Scalable (polynomial memory and computational complexity)

Local Potential Fields

Discrete CoordinationProtocols

Maintain LinksM. Ji and M. Egerstedt (2006)

Challenge

Control Link Deletions so that network connectivity is always guaranteed.

Discrete Coordination Protocols

Link Additions do not endanger connectivity

Challenge

Control Link Deletions so that network connectivity is always guaranteed.

Discrete Coordination Protocols

Link Additions do not endanger connectivity

Control Challenges

* At most one link deletion every time instant.* Agreement on the link to be deleted.

Connectivity Violation !

Market-Based

Coordination

If is connected,then is connected.

No Violation !

Connectivity Violation !

Network Estimate:

If then the is connected.

Agreement via Market-Based

Step 2: Select a link and a bid

Step 3: Initialize a set of max-bids Initialize a vector of tokens

Step 4: While do - Collect tokens from neighbors - Keep the highest bid

Step 5: If and select control

else if select control and return to Step 1.

All bids havebeen collected

Tie in the Bids

Step 1: Compute safe set of links to be deleted, i.e., links (i,j) such that

Integrating with Mobility

ObjectiveMobility does not destroy thenetwork structure controlled

by discrete coordination

Candidate agent to add a link (i,j) with

Candidate neighbor to delete a link (i,j) with

Non-neighbor agent

Definition of Links

Maintaining Links

ensured by

The Hybrid Agent

Navigation in the Config.-Spaceif neighbors are updated

positions, network estimates and bids from all neighbors

updated network estimate and bid

updated set of neighbors

Discrete Coordination ProtocolsMarket-Based Coordination for Link DeletionsLink Additions

New Links

Existing Links provided by Neighbors

Drift (negative gradient of a potential)

Maintain existing links with neighbors & collision avoidance

Distributed: Only nearest neighbor information used

Scalable: Memory cost is O(n2) due to adjacency matrixComputational cost is O(n3) due to eigenvalue computation

Provably Correct: Connectivity always guaranteed

Theorems & Results

Theorems & Side Results

Can handle communication limitations & time delays

Can handle robust notions of connectivity & collision avoidance

Achieving secondary objectives is not guaranteed

Further Characteristics

M. M. Zavlanos, G. J. Pappas, IEEE T-RO. (accepted)

Applications: FlockingAlignment:Steer towards the average heading of flockmates

Separation:Steer to avoid crowding of flockmates

Cohesion: Steer towards the average position of flockmates

Reynolds (1987)

Theorem: If the network remains connected, then all agent headingsconverge to a common value and the distances between them are stabilized

to a configuration where the group potential energy attains a minimum.

Tanner, Jadbabaie, Pappas (2007)Separation & CohesionVelocity alignment

Applications: FlockingAlignment:Steer towards the average heading of flockmates

Separation:Steer to avoid crowding of flockmates

Cohesion: Steer towards the average position of flockmates

Reynolds (1987)

Theorem: If the network remains connected, then all agent headingsconverge to a common value and the distances between them are stabilized

to a configuration where the group potential energy attains a minimum.

Tanner, Jadbabaie, Pappas (2007)Separation & CohesionVelocity alignment

Connectivity Preserving Flocking

Theorem:If the initial network is connected,

flocking is always guaranteed.

Maintain neighbor links & SeparationVelocity alignment

Distributed Connectivity Control

M. M. Zavlanos, H. G. Tanner, A. Jadbabaie, G. J. Pappas, IEEE TAC. (submitted)

Connectivity Preserving Flocking

Theorem:If the initial network is connected,

flocking is always guaranteed.

Maintain neighbor links & SeparationVelocity alignment

Distributed Connectivity Control

M. M. Zavlanos, H. G. Tanner, A. Jadbabaie, G. J. Pappas, IEEE TAC. (submitted)

Experimentation with Scarabs

N. Michael, M. M. Zavlanos, V. Kumar and G. J. Pappas, ISER 2008.

Heterogeneous Networks in Complex Environments

Distributed Connectivity Control - Local estimates of the network topology

- Auction-based link deletion

Role Assignment in a Heterogeneous Team - Auction-based leader election / reelection (for leader failures)

- Leaders are assigned locations of interest

- Relays assist leaders in completing their tasks

Complex Environments - Environment interference on signal strength

- Geodesic paths

GRASP Lab floor

Complex environments

1

2

3

4

Future Challenges

Heterogeneous Teams of Robots Integration of multiple-modality agents and possibly humans Robust execution in dynamic environments, abstract task specification Biologically inspired cooperation principles

Control of Robotic Networks Robotic networks in indoor environments or environments with obstacles Bid selection to maximize lifetime of the robotic network Stochasticity, i.e., failing communication links/robots Fundamental limits of distributed algorithms, i.e., amount of information required

Interdisciplinary insight from Political economics for auction designs Role allocation in ants

Some solutions

Heterogeneous networks in complex environments

Mission specification languages

Linguistic Mission Specification

Interface will depend on

1. Robot domain (mobile robots)2. Tasks (search missions)3. Environments

Interface should be formal, robot-independent, robust,

Challenge:expressivity

Challenge:executability

LANGU

AGE

ROBO

T

INTERFACE

Linguistic Mission Specification

Interface will depend on

1. Robot domain (mobile robots)2. Tasks (search missions)3. Environments

Interface should be formal, robot-independent, robust,

Challenge:expressivity

Challenge:executability

LANGU

AGE

ROBO

T

INTERFACE

Our approach uses Linear Temporal Logic

(LTL)

The setting

Kress-Gazit, Fainekos, Pappas, Sensor-based temporal logic motion planning, ICRA 2007

The setting

Kress-Gazit, Fainekos, Pappas, Sensor-based temporal logic motion planning, ICRA 2007

The setting

Kress-Gazit, Fainekos, Pappas, Sensor-based temporal logic motion planning, ICRA 2007

The setting

Kress-Gazit, Fainekos, Pappas, Sensor-based temporal logic motion planning, ICRA 2007

Mission specification“Waldo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms.

If at some point you see him, stop”

Constructing φ

We consider LTL formulas of the form:

Assumptions about

environment(another robot

or human)

Desired robot behavior

*Note that only if the assumptions are met ( is true),

the desired behavior is guaranteed ( must be true).

Example

Task: “Waldo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop”

Sensor (Input) propositions: X = {sWaldo}Robot (Output) propositions: Y = {r1, r2,…, r12}

Environment Assumptions Desired behavior

Initial Conditions

Transitions

Goals

Example

Task: “Waldo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop”

Sensor (Input) propositions: X = {sWaldo}Robot (Output) propositions: Y = {r1, r2,…, r12}

Environment Assumptions Desired behavior

Planning using SynthesisTask: “Waldo may be sitting in one of rooms 1, 3, 5 and 8. Starting in corridor 12, look for him in these rooms. If at some point you see him, stop”

r12r9r8

r5

r8

r9r10r5

r10

r11

r3r3 r11

r12

r9

r1r1sWaldo sWaldo

sWaldosWaldo

sWaldo

sWaldos W

aldo

sWaldo

3 4

8 7

6

5

12 109

11

1

2

Multi-robot specifications

• Naturally captured in a decentralized way

• The environment of each robot contains all other robots

Multi-robot scenario*

*Kress-Gazit et al, Valet parking without a valet, IROS 2007

Multi-robot scenario*

*Kress-Gazit et al, Valet parking without a valet, IROS 2007

DARPA’s Urban Challenge - NQE

H. Kress-Gazit and G. J. Pappas. Automatically Synthesizing a Planner and Controller for the DARPA Urban Challenge. IEEE CASE 2008

Robot moving

Robot stopping

Other vehicles

Obstacles

DARPA’s Urban Challenge - NQE

H. Kress-Gazit and G. J. Pappas. Automatically Synthesizing a Planner and Controller for the DARPA Urban Challenge. IEEE CASE 2008

Robot moving

Robot stopping

Other vehicles

Obstacles

Toolbox

Toolbox

Future directions

Multi-robot specification languages Coordination hidden or exposed to the interface?

Heterogeneous specifications Model heterogeneous constraints across sensor-robot predicates

More expressive logics/task description languages Spatial logics? “Go around the car in front of the red building”

Growing interaction with cognitive linguists and psychologists