decentralised coordination of continuously valued control parameters using the max-sum algorithm...

Decentralised Coordination of Continuously Valued Control Parameters using the

Max-Sum Algorithm

Ruben Stranders, Alessandro Farinelli, Alex Rogers, Nick JenningsSchool of Electronics and Computer Science

University of Southampton{rs06r, af2, acr, nrj}@ecs.soton.ac.uk

2

This presentation focuses on the use of Max-Sum in coordination problems with continuous parameters

From Discrete to Continuous

Max-Sum for Decentralised Coordination

Empirical Evaluation

3

Max-Sum is a powerful algorithm for solving DCOPs

Complete Algorithms

DPOPOptAPOADOPT

Communication Cost

Iterative Algorithms

Best Response (BR)Distributed Stochastic

Algorithm (DSA) Fictitious Play (FP)

Max-SumAlgorithm

Optimality

Max-Sum solves the social welfare maximisation problem in a decentralised way

Agents


1x

2x

3x

4x

5x

6x

7x8x

Control Parameters


1U

2U

3U

4U

5U

6U

7U8U

Utility Functions


)( 33 xU

Localised Interaction

},,,{ 54313 xxxxx


Agents

M

iiiU

1

)(maxarg xx

Social welfare:

The input for the Max-Sum algorithm is a graphical representation of the problem: a Factor Graph

Variable nodes Function nodes

1x

2x

3x

1U

2U

3U

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by message passing

1x

2x

3x

1U

2U

3U

Variable nodes Function nodes

Agent 1Agent 2

Agent 3

Max-Sum solves the social welfare maximisation problem by message passing

jiadjk

iikiji xrxq\)(

)()(

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

From variable i to function j

From function j to variable i

Until now, Max-Sum was only defined for discretely valued variables

Graph Colouring)( iji xq )( iij xr

However, many problems are inherently continuous. Heading

andVelocity

Unattended Ground Sensor

ActivationTime

Autonomous Ground Robot

Thermostat

Preferred RoomTemperature

So, we extended the Max-Sum algorithm to operate in continuous action spaces

Discrete Continuous

We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)

We focussed on utility functions that are Continuous Piecewise Linear Functions (CPLFs)

)( iji xq )( iij xr

“Continuous” Graph Colouring

A CPLF is defined by a domain partitioning followed by value assignment

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

To make Max-Sum work on CPLFs, we need to define key two operations on them

jiadjk

iikiji xrxq\)(

)()(



ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx


jiadjk

iikiji xrxq\)(

)()(


From function j to variable i1. Addition

of two CPLFs

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx


jiadjk

iikiji xrxq\)(

)()(



2. Marginal Maximisation to a single variable

Addition of two CPLFs involves merging their domains, and then summing their values

1x

2x

1x

2x

1f 2f


1x

2x

1x

2x

1x

2x

1. Merge domains

1f 2f

21 ff


2. Sum Values

Marginal maximisation is the operation of finding the maximum value of a function, if we fix all but one variable

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

From function j to variable i:

Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

)(max)(\

jji

i fxgj

xx

Marginal maximisation involves finding the maximum value of a function, if we fix all but one variable

ijadjk

kjkjji

iij xqUxrj \)(\

)()(max)( xx

)(max)(\

ji

i fxgj

xx

),(max)( 2112

xxfxgx

Example: bivariate function:

Marginal maximisation involves the projection of a CLPF on a 2-D plane, and upper envelope extraction

),( 1 fxProject onto

axis


Project onto

axis),( 1 fx


Project onto

axis),( 1 fx

Result of projection


Extract UpperEnvelope


Extract UpperEnvelope

),(max)( 2112

xxfxgx

)( 1xg

We empirically evaluated this algorithm in a wide-area surveillance scenario

Dense deployment of sensors to detect activity within an urban environment.

Unattended Ground

Sensor

Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors

time

duty cycleDiscrete

time

duty cycle

time

duty cycle

Discretised time

Sensors adapt their duty cycles to maximise event detection by coordinating with overlapping sensors

time

duty cycleDiscrete Continuous

time

duty cycle

time

duty cycle

time

duty cycle

time

duty cycle

time

duty cycle

Continuous Max-Sum outperforms Discrete Max-Sum by up to 10%

0 5 10 15 20 25 300.75

0.80

0.85

0.90

0.95

1.00

ContinuousDiscrete

Discretisation

Solu

tion

Qua

lity

(as

frac

tion

of o

ptim

al)

Average Solution Quality over 25 Iterations

Tota

l Mes

sage

Siz

eContinuous Max-Sum leads to more effective use of communication resources than Discrete Max-Sum

0 5 10 15 20 25 300

5000

10000

15000

20000

ContinuousDiscrete

Discretisation

Total number of values exchanged between agents

In conclusion, we have shown that Continuous Max-Sum is more effective than Discrete Max-Sum

1. No artificial discretisation

time

time


1. No artificial discretisation Continuous

Discrete

2. Better solutions

time

timeSo

lutio

n Q

ualit

y


time

time1. No artificial discretisation Continuous

Discrete

2. Better solutions ContinuousDiscrete

3. Effective communication

Solu

tion

Qua

lity

Mes

sage

Siz

e

For future work, we wish to extend the algorithm to arbitrary continuous functions

For example, using Gaussian Processes


time

time1. No artificial discretisation Continuous

Discrete

2. Better solutions ContinuousDiscrete

3. Effective communication

Solu

tion

Qua

lity

Mes

sage

Siz

eQuestions?

decentralised coordination of continuously valued control parameters using the max-sum algorithm...

Documents