distributed cooperation and coordination using the max-sum algorithm

Distributed cooperation and coordination using the Max-Sum

algorithm

Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices Using the Max-Sum Algorithm. In: Seventh International Conference on Autonomous Agents and Multi-Agent

Systems (AAMAS-08), 12-16 May 2008, Estoril, Portugal.

Waldock, A., Nicholson, D. and Rogers, A. (2008) Cooperative Control using the Max-Sum Algorithm. In: Second International Workshop on Agent

Technology for Sensor Networks, Estoril, Portugal.

What is Max-Sum?

• Max-sum is a derivative of the sum-product algorithm, formed by transforming the max-product algorithm into log-space.

What is Sum-Product?

• General form of the forward-backward algorithm

• Iterative algorithm to compute the marginals of functions

• a.k.a Belief Propagation (when applied to probabilistic graphical models)– Information Theory– Machine Learning– Vision

An abstract problem

Consider a function, F, that is dependent on N variables, x = {x1 . . . xN}, and is defined as a product of M factors such that:

where each of the factors fm(xm) is a function of a subset xm of the variables that make up x.

An abstract problem

Find the marginal function zn(xn) that describes the dependency of F(x) on variable xn:

The marginal zn(a) is defined to be the sum of F(x) over all configurations of variables where

xn = a.

Factor graph representation

• Bipartite graph consisting of two types of nodes– Variable nodes– Function nodes

Factor graph representation

• Example: F = f1(x1,x2)f2(x2,x3)

What is Sum-Product?

• Given a factor graph

• Apply a local message passing procedure

• Compute the marginals of functions

Two local message types

• From variable to function:

• From function to variable:

Canonical update procedure

• Graph is cycle-free (a tree)

• Leaf nodes send identity messages to parents

• When a node receives a message on some edge, it computes and transmits the outgoing message for all other edges

• After variable node has received a message from every neighbor, it can compute its exact marginal value:

Message Passing

Message Passing

Message Passing

“Loopy” Belief Propagation

• Sum-product algorithm is only guaranteed to converge on graphs containing up to one loop

• Could oscillate, or fail to converge on cyclic graphs

• Empirically, it often converges pretty well on many types of graphs

“Loopy” Belief Propagation

• All variable messages initialized to 1

• All messages updated at every iteration

• Normalize variable to function message:

Choose αnm such that

What is Max-Product?

• Instead of computing the marginal functions, we can calculate arg maxx F(x)

• Replace function-to-variable message:

What is Max-Product?

• Now, when we take the product of the messages:

• For arg maxx F(x), each component is given by:

• A.k.a: Viterbi algorithm

What is Max-Sum?

• Suppose we have an F(x) that is the sum of function nodes, e.g.

• Use max-product in logarithm space

Two new local message types

• From variable to function:

• From function to variable:

What is Max-Sum?

• Now, when we take the sum of the messages:

Advantages

• Existing theoretical results• Simple, scalable local algorithm

– Exponential only in # of neighbors in graph

• Already decentralized and asynchronous– Partition nodes as desired– Only exchange local messages

• Can estimate variables before convergence

Caveats

• No strong theoretical results for loopy BP– Convergence not guaranteed

• Must exchange local messages between related factor and variable nodes

• Not necessarily easy to evaluate functions and compute messages

How to use Max-Sum to solve your distributed problem:

Construct utility function

How to use Max-Sum to solve your distributed problem:

Construct utility function

Turn it into factor

graph

Application #1:Decentralized Coordination

Farinelli, A., Rogers, A., Petcu, A. and Jennings, N. R. (2008) Decentralised Coordination of Low-Power Embedded Devices Using the Max-Sum

Algorithm. In: Seventh International Conference on Autonomous Agents and Multi-Agent Systems (AAMAS-08), 12-16 May 2008,

Estoril, Portugal.

Problem

• Solve graph coloring on sensor network

• Utility can be formulated for each agent:

Penalty for same color:

A priori color preference:

Factor-graph Formulation

• Utility function is only dependent on neighbors• Let each sensor maintain its state and utility

nodes:

• Exchange local messages as per Max-Sum

Comparison Algorithms• BR - Best Response

– Each agent chooses the “best” state for its variable (i.e. the one that minimizes the conflicts) according to the current states of its neighbors

– When a state change occurs, the agent sends the updated state to all of its neighbors

– Represents a lower bound on the performance of any algorithm since it uses the minimum computation and communication possible

• DSA - Distributed Stochastic Algorithm– Same as best response, except that an agent only actually performs the state

change according to a predefined probability (called the activation probability)– Whenever an agent’s preferred state actually changes, it sends a message to its

neighbors informing them of this fact– Set the activation probability to 0.6 (additional experiments indicate little

sensitivity to the exact value of this parameter in our setting)• DPOP - Dynamic Programming Optimization Protocol

– A complete algorithm that maintains optimality by preprocessing the constraint graph to produce a pseudo-tree, and then performs local message passing on this tree

Test Conditions

• Known colorable– 3-color random graphs – 10, 20, 30, 40, 50 nodes– Link density of 3– 50 instances

• ADOPT graph repository– Random graphs – 10, 14, 18, 25 nodes– Link density of 3– Not generally colorable with 3-colors

• Lattice graphs– 4-color colorable– 36, 49, 64, 81, 100 nodes– Nodes connected to eight neighbors

Conflicts over time

Conflicts over time

Conflicts over time

What happened here?

• Combination of small and large irregular loops

• Cycling causes poor convergence

• Modify utility function:

Summary of conflict results

Message Size

Message Loss

Hardware Implementation

• http://www.youtube.com/v/T6H1AwQ2gXw

• http://www.youtube.com/v/D6vWvs3Lsj0

Application #2:Cooperative Control

Waldock, A., Nicholson, D. and Rogers, A. (2008) Cooperative Control using the Max-Sum Algorithm. In: Second International Workshop on

Agent Technology for Sensor Networks, Estoril, Portugal.

Problem

• Sensor network doing distributed tracking• Sensors share information using

decentralized data fusion (DDF)

Problem

• Select control parameters for each sensor– Maximize sum of global info for all targets

Problem

• Define U as a summation of target factors Uj over only sensors that can observe target, e.g.

where Ub is defined as:

Problem

• Unfortunately, we can’t break into smaller factors:

≠

Factor-graph Formulation

• Each sensor maintains variable node for its own control parameter

• Define a function node for each target– Maintained by the sensor that first saw target

• Use DDF to determine edges of graph– Add/remove edges as target moves in and out of

sensor range

• Exchange local messages as per Max-Sum

Factor-graph Formulation

Sensor 3

θ2

Sensor 4

θ4

Sensor 2

θ2 UB

Sensor 1

θ1 UA

Sensor 5

θ5 UC

Test conditions

• Sensors initialized with weak priors and target responsibilities

• Performance evaluated using global information from one sensor

Global Information

Adjusting negotiation time

How to use Max-Sum to solve your distributed problem:

Construct utility function

Turn into factor graph