order optimal information spreading using algebraic gossipdhay/ind2011/16-18/avin.pdforder optimal...

Order Optimal Information SpreadingUsing Algebraic Gossip

Chen Avin, Michael Borokhovich, Keren Censor-Hillel, Zvi Lotker

Department of Communication Systems Engineering, Ben-Gurion University of the Negev, Israel

Computer Science and Artificial Intelligence Laboratory, Massachusetts Institute of Technology, USA

Israeli Networking Day 2011

(To be presented in PODC11)

Introduction Algebraic Gossip Research Goal Related Work Our Results Summary

Motivation

Wireless (sensor) networks and peer-to-peer networksneed efficient algorithms for information dissemination.In such networks, there is no central management entity,thus local, distributed algorithms are needed.Network Coding with gossip algorithms (a.k.a. AlgebraicGossip) will help us to achieve faster informationdissemination.We look at: k nodes want to disseminate their value to allother nodes in the network.

Information Spreading - The k -Dissemination Problem

x1 = 5 x2 = 15

x3 = 10 x4 = 12

A network represented by a graph G(V ,E). V = v1, v2, . . . , vn

k ≤ n values x1, x2, . . . , xk need to be distributed to all nodes

A node knows only its neighbors

Limited messages size

Gossip Algorithm – The Way Information is Spread

Gossip Algorithm:

1 Determines a communication partner(randomly) among neighbors.

Uniform gossip.Non uniform gossip.

2 Determines how the message is sent.

PUSH – a message is sent to the partner.PULL – a message is sent from the partner.EXCHANGE - PUSH and PULL.

In each round every node takes a gossip action

v8v1 v2

Gossip Algorithm:

2 Determines how the message is sent.

PUSH – a message is sent to the partner.PULL – a message is sent from the partner.EXCHANGE - PUSH and PULL.

Gossip Algorithm:

2 Determines how the message is sent.PUSH – a message is sent to the partner.PULL – a message is sent from the partner.EXCHANGE - PUSH and PULL.

v8v1 v2

Algebraic Gossip is Based on Random Linear Network Coding

Nodes (routers) can manipulate packets.All operations are in a field F so messages size is (almost)the same in both cases.

instead of sending randomly chosen values...

every message – a single value: msg = xi

nodes send random linear combinations

every message – linear equation: msg =∑

5x5+ 2x3

3x4 + 4x1

6x3+ x4

every message – linear equation

5x5+ 2x3

3x4 + 4x1

6x3+ x4

linear equations are stored in a matrix form:

4 3 7 62 0 0 71 1 0 00 2 1 5

x1x2x3x4

22457830

once a node has rank k – it finishes

only helpful messages are stored

messages that increase the rank of the matrix

5x5+ 2x3

3x4 + 4x1

6x3+ x4

5x5+ 2x3

3x4 + 4x1

6x3+ x4

4 3 7 62 0 0 71 1 0 00 2 1 5

x1x2x3x4

22457830

5x5+ 2x3

3x4 + 4x1

6x3+ x4

5x5+ 2x3

3x4 + 4x1

6x3+ x4

4 3 7 62 0 0 71 1 0 00 2 1 5

x1x2x3x4

22457830

5x5+ 2x3

3x4 + 4x1

6x3+ x4

5x5+ 2x3

3x4 + 4x1

6x3+ x4

4 3 7 62 0 0 71 1 0 00 2 1 5

x1x2x3x4

22457830

5x5+ 2x3

3x4 + 4x1

6x3+ x4

5x5+ 2x3

3x4 + 4x1

6x3+ x4

4 3 7 62 0 0 71 1 0 00 2 1 5

x1x2x3x4

22457830

So, Why Algebraic Gossip is Faster?

A B[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

Without Algebraic Gossip (Random Message Selection)

A has all values B is missing one value

A is sending a randomly chosen value

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

rank k

rank k − 1

With Algebraic Gossip (Random Linear Equations)

A has k independent equations B has k − 1 independent equations

A is sending a random linear equation

msg =[∑

], ai ∈ Fq

Pr (helpful) = 1− 1q

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

Pr (helpful) = qk−qk−1

qk = 1− 1q

A B[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

qk = 1− 1q

A B[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

qk = 1− 1q

A B[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

qk = 1− 1q

A B[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

Pr (helpful) = 1k

[x1, x2, x3, . . . xk ] [x1, x2=?, x3, . . . xk ]

msg = [xi ]

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

rank k

rank k − 1

msg =[∑

], ai ∈ Fq

qk = 1− 1q

Research Goal – Optimal Protocol for k -Dissemination Problem

1. Analyze uniform algebraic gossip for k -disseminationIs it optimal?For which graphs?

2. Study non-uniform gossip to achieve optimalk -dissemination

x1 = 5 x2 = 15

x3 = 10 x4 = 12

Related Work on Algebraic Gossip

Trivial lower bound – Ω(k), kn messages needed to be deliveredso k rounds.

[Deb et al., 2006] – (almost) Tight bound for the complete graph.Θ(k), for k ln3 n. (push/pull)

[Mosk-Aoyama and Shah, 2006] – n-dissemination. Upperbound for arbitrary graphs, based on conductance measure.The bound is not tight. For complete graph: O(n log n), for Ring:O(n2).

[BAL. ISIT10] – n-dissemination. Upper bound for any graph:O(∆n). Tight bound of Θ(n) for constant degree graphs. Worstcase graph for algebraic gossip (barbell): Ω(n2).

Open question: What graph property capture the stopping time?

v uv uw

[Haeupler. STOC11] – k -dissemination. Conductance andexpansion based arguments. Two parameters: γ and λ.

Tight bound for the case k = Ω(n): Θ(n/γ)

For k < o(n): O(k/γ + log2 n/λ). The bound is not tight fore.g., line: O(k + n log2 n), grid: O(k +

√n log2 n), binary

tree: O(k + n log2 n)Gave also results for dynamic networks

1st Result: k -Dissemination With Uniform Algebraic Gossip

Theorem 1For any graph with n nodes, diameter D, and maximumdegree ∆, stopping time of uniform algebraic gossip isO(∆(k + log n + D)) with high probability.

Corollary 1For any graph with n nodes and with constant maximumdegree, stopping time of uniform algebraic gossip is Θ(k + D)in the synchronous time model.

Tight for e.g., Line, Cycle, Grids, Binary Trees, etc.

The result holds for any gossip variation: Push, Pull, Exchange.

When does the uniform algebraic gossip perform bad? e.g.,Ω(kn) for a barbell graph.

2nd Result: k -Dissemination With TAG

Construct a spanning tree of thegraph using some gossip spanningtree protocol S.

The stopping time of S is t(S),and the diameter of the spanningtree is d(S).Spanning tree can be constructed e.g., by a randomizebroadcast protocol, or more sophisticated method.

Once the tree is constructed, every node knows its parent.

Perform algebraic gossip, where every node uses a singlecommunication partner – its parent.Notice, we have here non-uniform algebraic gossip.

The stopping time of S is t(S),and the diameter of the spanningtree is d(S).

Spanning tree can be constructed e.g., by a randomizebroadcast protocol, or more sophisticated method.

Theorem 2For any graph with n nodes,stopping time of TAG protocol is O(k + log n + d(S) + t(S))with high probability, where:t(S) – stopping time of the gossip spanning tree protocol S.d(S) – diameter of the spanning tree created by S.

Corollary 2

For any graph with n nodes, and for k = Ω(k),TAG protocol is order optimal for k -dissemination task, i.e.,the stopping time is Θ(n) with high probability.

Theorem 2For any graph with n nodes,stopping time of TAG protocol is O(k + log n + d(S) + t(S))with high probability, where:t(S) – stopping time of the gossip spanning tree protocol S.d(S) – diameter of the spanning tree created by S.

Corollary 2

For any graph with n nodes, and for k = Ω(k),TAG protocol is order optimal for k -dissemination task, i.e.,the stopping time is Θ(n) with high probability.

Proof Overview

k -dissemination with uniform algebraic gossip

Proof Overview – Converting a Graph to a System of Queues

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

initial graph with an arbitrary node v4

when v4 will finish the protocol?

BFS spanning tree rooted at v4

we ignore the messages coming from other edges

customers are helpful messages

initially, some nodes have helpful messages

customer arriving at some node, increases its rank by 1

once v4 receives k helpful messages it finishes

in a given round, v5 wakes up exactly once

v5 chooses v6 as a partner w.p. ≥ 1∆

the message will be helpful w.p. ≥ (1− 1q )

service time is geometrically distributed with p

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

p ≥?

p ≥ 1∆

p ≥ 1∆ (1− 1

Proof Overview – Exponential Servers Instead of Geometric

µ = p

If X ∼ Geom(p), and Y ∼ Exp(p), then: Pr(Y > t) ≥ Pr(X > t)

so, exponential server is slower than geometric

we replace servers, thus increasing the stopping time

µ = p

Line is Slower Than Tree

µ = p

v4µ µ µ

v4µ µ µλ

When does the

last customer

leave the system?

Reduce tree to line

Take all customers

out and use

Jackson theorem

µ = p

v4µ µ µ

v4µ µ µλ

When does the

last customer

leave the system?

Reduce tree to line

Take all customers

out and use

Jackson theorem

µ = p

v4µ µ µ

v4µ µ µλ

When does the

last customer

leave the system?

Reduce tree to line

Take all customers

out and use

Jackson theorem

Line of D Queues

Jackson’s Theorem: If utilization at every queue is less than 1, the equilibrium state distribution of numberof customers in each queue exists and it is given by:

For state (k1, k2, . . . , kn), and utilization ρi =λiµi

: π(k1, k2, . . . , kn) =n∏

ρkii (1− ρi ).

By setting λ = µ/2, we obtain: ρ < 1.

We add dummy customers according to stationary distribution. So, the real customers see stationarydistribution.

Time by which all the customers enter the system is O((k + log n)/µ), since λ = µ/2, and log n is neededfor high probability.

Time to cross one MM1 queue in the stationary state is exponentially distributed with µ− λ = µ/2.

So, the time needed to cross D MM1 queues is O((d + log n)/µ), where log n is needed for high probability.

v4µ µ µλ = µ

Last customer leaves after: O((k + log n + D)/µ) = O(∆(k + log n + D)) rounds

Line of D Queues

: π(k1, k2, . . . , kn) =n∏

ρkii (1− ρi ).

v4µ µ µλ = µ

Line of D Queues

: π(k1, k2, . . . , kn) =n∏

ρkii (1− ρi ).

v4µ µ µλ = µ

Line of D Queues

: π(k1, k2, . . . , kn) =n∏

ρkii (1− ρi ).

v4µ µ µλ = µ

Line of D Queues

: π(k1, k2, . . . , kn) =n∏

ρkii (1− ρi ).

v4µ µ µλ = µ

Line of D Queues

: π(k1, k2, . . . , kn) =n∏

ρkii (1− ρi ).

v4µ µ µλ = µ

Proof Overview

k -dissemination with TAGTree-Based Algebraic Gossip

TAG – Tree Based Algebraic Gossip Protocol

The proof is also based on analyzing a tree network of queues

The uniform gossip bound is

O(∆(k + log n + D))

The TAG based bound is:

O(t(S) + k + log n + d(S))

In the synchronous time model t(B) ≥ d(B). (B is a broadcastthat builds a tree)

For Round Robin Broadcast, t(BRR) = O(n) so for k = Ω(n) thestopping time of TAG is Θ(n)

Summary

When does uniform algebraic gossip is order optimal?

In graphs with constant maximum degreeBut not only, e.g., complete graph. so when exactly?

When does TAG is order optimal?

When k = Ω(n)In graphs with large weak conductance (see paper)When else? What spanning algorithm to use?

Thank you!

Summary

Thank you!

Summary

Thank you!

Deb, S., Médard, M., and Choute, C. (2006).Algebraic gossip: a network coding approach to optimalmultiple rumor mongering.IEEE Transactions on Information Theory,52(6):2486–2507.

Mosk-Aoyama, D. and Shah, D. (2006).Information dissemination via network coding.In ISIT, pages 1748–1752.

order optimal information spreading using algebraic gossipdhay/ind2011/16-18/avin.pdforder optimal...

Documents