Decentralised Coordination of Mobile Sensors using the Max-Sum Algorithm 

School of Electronics and Computer ScienceUniversity of Southampton{rs06r2, af2, acr, nrj}@ecs.soton.ac.uk

Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick Jennings

2

This presentation focuses on the use of Max-Sum to coordinate mobile sensors

Sensor Architecture

Decentralised Control using Max-Sum

Model

Value

Coordinate

Problem Formulation

The key challenge is to monitor a spatial phenomenon with a team of autonomous sensors

Sensors

The key challenge is to monitor a spatial phenomenon with a team of autonomous sensors

LimitedCommunication

The key challenge is to monitor a spatial phenomenon with a team of autonomous sensors

No centralised control

Spatial phenomena are modelled as a spatial field over two spatial and one temporal dimensions

The aim of the sensors is to collectively minimise predictive uncertainty of the spatial phenomenon

PredictiveUncertaintyContours

The main challenge is to coordinate the sensors in order to the state of these spatial phenomena

How to move to minimiseuncertainty?

To solve this coordination problem, we had to address three challenges

1. How to model the phenomena?2. How to value potential samples?3. How to coordinate to gather

samples of highest value?

The three central challenges are clearly reflected in the architecture of our sensing agents

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Model

Value

Coordinate

These three challenges are clearly reflected in the architecture of our sensing agents

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Model

The sensors model the spatial phenomenon using the Gaussian Process

Weak Strong

Spatial Correlations

The sensors model the spatial phenomenon using the Gaussian Process

Weak Strong

Temporal Correlations

The value of a sample is determined how much it reduces uncertainty

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Value

The value of a sample is based on how much it reduces uncertainty

But how to determine uncertainty reduction before collecting a sample?

The value of a sample is based on how much it reduces uncertainty

But how to determine uncertainty reduction before collecting a sample?

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

Gaussian Process not only gives predictions, but also confidence intervals

The value of a sample is based on how much it reduces uncertainty

But how to determine uncertainty reduction before collecting a sample?

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

Gaussian Process not only gives predictions, but also confidence intervals

Potential Sample Location

The value of a sample is based on how much it reduces uncertainty

But how to determine uncertainty reduction before collecting a sample?

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

Gaussian Process not only gives predictions, but also confidence intervals

Measure of uncertainty

The value of a sample is based on how much it reduces uncertainty

0 1 2 3 4 5 6 7 8 9 100

1

2

3

4

5

6

7

8

PredictionConfidence IntervalCollected Sample

Specifically, we use Entropy , as information metric

)(XH

The sensor agents coordinate using the Max-Sum algorithm

Samples sent toneighbouring agents

Samples received fromneighbouring agents

Information processing

Model of Environment

Outgoing negotiation messages

Incomingnegotiation messages

Value of potential samples Action

Selection

Move

Samples from own sensor

SensingAgent

Rawsamples

Coordinate

Using the Entropy criterion, the sum of the conditional values equals the team utility

)( 1XH )|( 12 XXH ),|( 321 XXXH

1U 2U 3U

),|()|()(),,( 213121321 XXXHXXHXHXXXH

iU

The key problem is to maximise the social welfare of the team of sensors in a decentralised way

M

iiiU

1

)(maxarg xx

Social welfare:

Mobile Sensors

1x

2x

3x

4x

5x

6x

7x8x

Variables Encode Movement

The key problem is to maximise the social welfare of the team of sensors in a decentralised way

1U

2U

3U

4U

5U

6U

7U8U

Utility Functions

The key problem is to maximise the social welfare of the team of sensors in a decentralised way

(These encode information value)

)( 33 xU

Localised Interaction

},,,{ 54313 xxxxx

1x

3x

4x

5x

The key problem is to maximise the social welfare of the team of sensors in a decentralised way

26

We can now use Max-Sum to solve the social welfare maximisation problem

Complete Algorithms

DPOPOptAPOADOPT

Communication Cost

Iterative AlgorithmsBest Response (BR)

Distributed Stochastic Algorithm (DSA)

Fictitious Play (FP)

Max-SumAlgorithm

Optimality

The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph

Variable nodes Function nodes

1x

2x

3x

1U

2U

3U

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by local computation and message passing

1x

2x

3x

1U

2U

3U

Variable nodes Function nodes

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by local computation and message passing

jiadjk

iikiji xrxq\)(

)()(

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

From variable i to function j

From function j to variable i

To use Max-Sum, we encode the mobile sensor coordination problem as a factor graph

1x

2x

3x

1U

2U

3U

Sensor 1Sensor 2

Sensor 3

Sensor 1

Sensor 2

Sensor 3

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Unfortunately, the straightforward application of Max-Sum is too computationally expensive

jiadjk

iikiji xrxq\)(

)()(From variable i to function j

From function j to variable i

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Unfortunately, the straightforward application of Max-Sum is too computationally expensive

jiadjk

iikiji xrxq\)(

)()(From variable i to function j

From function j to variable i

Bottleneck!

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Therefore, we developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Therefore, we developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

1. Try to prune the action spaces of individual sensors

2. Try to prune joint actions

ix

ij \x

The first pruning technique prunes individual actions by identifying dominated actions

The first pruning technique prunes individual actions by identifying dominated actions

1. Neighbours send bounds

↑ [2, 2]↓ [1, 1]

↑ [5, 6]↓ [0, 1]

↑ [1, 2]↓ [3, 4]

The first pruning technique prunes individual actions by identifying dominated actions

↑ [2, 2]↓ [1, 1]

↑ [5, 6]↓ [0, 1]

↑ [1, 2]↓ [3, 4]

2. Bounds are summed

↑ [8, 10]↓ [4, 7]

The first pruning technique prunes individual actions by identifying dominated actions

2. Bounds are summed

↑ [8, 10]↓ [4, 7]

↓ [4, 7]↑ [8, 10]

The first pruning technique prunes individual actions by identifying dominated actions

3. Dominated actions are pruned

[8, 10][4, 7]

X

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

We developed two general pruning techniques that speed up Max-Sum

Goal: Make as small as possible

1. Try to prune the action spaces of individual sensors

2. Try to prune joint actions

ix

ij \x✔

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

Sensor 1 Sensor 2 Sensor 3

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

Sensor 1 Sensor 2 Sensor 3

ijadjk

kjkjjiiij xqUxrj \)(\

)()(max)( xx

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

132 \)(},{

11 )()(max)(xjadjk

kjkjjxx

j xqUxr x

132 \)(},{

11 )()(max)(xjadjk

kjkjjxx

j xqUxr x

Sensor 1 Sensor 2 Sensor 3

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

),,(max)( 32},{

132

xxUr jxx

j

The second pruning technique reduces the joint action space because exhaustive enumeration is too costly

),,(max)( 32},{

132

xxUr jxx

j

),,,(),,,(max jj UU

),,,(),,,( jj UU...),,,(),,,( jj UU

The second pruning technique prunes the joint action space using Branch and Bound

Sensor 1

Sensor 2

Sensor 3

[7, 13][0, 4] [2, 6]

Sensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using Branch and Bound

[7, 13][0, 4] [2, 6]XXSensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using Branch and Bound

The second pruning technique prunes the joint action space using Branch and Bound

9 10 7 8

[7, 13][0, 4] [2, 6]XXSensor 1

Sensor 2

Sensor 3

The second pruning technique prunes the joint action space using Branch and Bound

9 10 7 8

[7, 13][0, 4] [2, 6]XX

X X XO

Sensor 1

Sensor 2

Sensor 3

This demonstration shows four sensors monitoring a spatial phenomenon

This demonstration shows four sensors monitoring a spatial phenomenon

Sensors

This demonstration shows four sensors monitoring a spatial phenomenon

UncertaintyContours

This demonstration shows four sensors monitoring a spatial phenomenon

The two pruning techniques combined prune 95% of the action space with 6 neighbouring sensors

2 2.5 3 3.5 4 4.5 5 5.5 60

25

50

75

100

Number of neighbouring sensors

% o

f joi

nt a

ction

s pru

ned

Avg.

Roo

t Mea

n Sq

uare

d Er

ror

Our Algorithm reduces Root Mean Squared Error of predictions up to 50% compared to Greedy

Our Al-gorithm

Greedy Random Fixed0.0

0.2

0.4

0.6

0.8

1.0

In conclusion, the use Max-Sum leads to an effective coordination algorithm for mobile sensors

1. Decentralised

In conclusion, the use Max-Sum leads to an effective coordination algorithm for mobile sensors

1. Decentralised

2. Fast

% P

rune

d

In conclusion, the use Max-Sum leads to an effective coordination algorithm for mobile sensors

1. Decentralised

2. Fast

3. Accurate predictions

% P

rune

d

Pred

ictio

n Er

ror

Our

Greedy

Random

Fixed

For future work, we wish to extend the algorithm to do non-myopic planning

In conclusion, the use Max-Sum leads to an effective coordination algorithm for mobile sensors

1. Decentralised

2. Fast

3. Accurate predictions

% P

rune

d

Pred

ictio

n Er

ror

Our

Greedy

Random

Fixed

Questions?