Model Checking Nash Equilibria in MAD Distributed Systems

FMCAD 2008Formal Methods in Computer Aided Design

Portland, OR, USANovember 17 – 20, 2008

Federico Mari, Igor Melatti, Ivano Salvo, Enrico Tronci

Dep. of Computer Science,

University of Roma “La Sapienza”

Roma, Italy

Lorenzo Alvisi, Allen Clement, Harry Li

Dep. of Computer Science,

University of Texas at Austin

Austin, Texas, USA

2

SAD Distributed Systems

In a Single Administrative Domain (SAD) Distributed System all nodes belong to the same administrative domain.

… and its Administrator

An Administrative Domain

3

MAD Distributed SystemsIn a Multiple Administrative Domain (MAD) Distributed System each node owns its resources.

Administrative Domains …

… and their Administrators

4

Examples of MAD Systems

Cooperative Backup (e.g., each PC is an administrative domain)

Internet Routing (e.g., each router is an administrative domain)

Wireless Mesh Routing (e.g., each node is an administrative domain)

File Distribution (e.g., each PC is an administrative domain)

Archival Storage (e.g., each PC is an administrative domain (e.g. http://www.oracorp.com/Research/p2pStorage.html)

5

Node Behaviours in SAD Systems

Altruistic (or correct or obedient) nodes, that is nodes faithfully following the proposed protocol

Byzantine nodes, that is nodes that may arbitrarily deviate from the proposed protocol, for example, because of:

hardware failure software failures or malicious attacks.

6

SAD Correctness

A protocol P for a SAD Distributed system S is expected to tolerate up to f byzantine nodes. Thus typically correctness for SAD systems is a statement of the form:

Protocol P for system S satisfies property φ as long as there are no more than f byzantine nodes in S.

7

Node Behaviours in MAD SystemsAltruistic nodes (as in SAD).

Byzantine nodes (as in SAD).

Rational (or selfish) nodes.

That is, nodes whose administrators are selfishly intent on maximizing their own benefits from participating in the system.

Rational nodes may change arbitrarily the protocol if that is at their advantage. In particular rational nodes may change their hardware or software if that is at there advantage.

8

MAD Correctness (1)Problem

In a MAD system any node may behave selfishly. This rules out the classical approach of showing that a given property holds when there are no more than f byzantine nodes.

Solution

Show BAR (Byzantine, Altruistic, Rational) tolerance. Namely, a protocol is BAR tolerant if it guarantees the desired property despite the presence of Byzantine and rational players.

9

MAD Correctness (2)

Sufficient to show the following:

1. Show correctness when there are only Byzantine and Altruistic players

2. Show that no rational node has an interest in deviating from the proposed protocol.

Note that:

Point 1 above is SAD correctness (and can be done using well know model checking techniques)

Point 2 above amounts to show that the proposed protocol is a Nash equilibrium ... in a suitable sense. This is our focus here.

10

Outline

• Formal definition of Proposed Protocol and Mechanism.

• Formal definition of Nash equilibrium for mechanisms.

• Symbolic algorithm verifying that a given proposed protocol is a Nash equilibrium for a given mechanism.

• Experimental results showing feasibility of proposed approach.

11

MechanismAn n player mechanism M is a tuple <S, I, A, T, B, h, > such that• States: S = <S1, ... Sn>• Initial states: I = <I1, ... In>• Actions: A = <A1, ... An>• Underlying (Byzantine) behavior:

B = <B1, ... Bn>, Bi : S × Ai × Si Boole s.t.

No deadlock: s ai si’ s.t. Bi (s, ai, si’)

Deterministic: Bi (s, ai, si’) Bi (s, ai, si’’) (si’ = si’’)• Proposed Protocol:

T = <T1, ... Tn>, Ti : S × Ai Boole s.t.:

Realizability: Ti (s, ai) s’ Bi (s, ai, si’)

Nonblocking: s ai Ti (s, ai)

• Reward: h = <h1, ... hn> with hi : S × A • Discount: = < 1, ... n> with i (0, 1)

12

Mechanism Transition Relation

ByzantineAgent

AltruisticAgent

BTi (Z, s, ai, si’) =

Bi (s, ai, si’) if i Z

Bi (s, ai, si’) Ti(s, ai) if i Z

Let Z {1, ... n } (Byzantine agents)

Agents move synchronously:

BT(Z, s, a, s’) = BT1 (Z, s, a1, s1’) … BTn (Z, s, an, sn’)

13

Example of Mechanism

2 0 1

sleep

reset

gain

work

Bi :Ti : *

hi(s, <a-i, work>) = -1

hi(s, <a-i, sleep>) = hi(s, <a-i, reset>) = 0

hi(s, <a-i, work>) =

4 if s = <1, 1, … 1>

0 otherwise

… Working has a cost

… No cost for sleeping or resetting

… Reward only if everyone worked

i = 0.5

14

Paths in Mechanisms

A path in (M, Z) is a finite or infinite sequence = s(0) a(0) s(1) a(1) … s(t) a(t) s(t+1) …

s.t. BT(Z, s(t),a(t), s(t+1)) holds.

Let n = 2 and Z = {1}. An example of (M, Z) path is: = <0, 0> <sleep, work> <2,1> <reset, gain> <0, 0> <work, work> <1, 1> <gain, gain> <0, 0>…

2 0 1sleep/0

reset/0

gain/4- 0

work/-1Bi :Ti : *

M

Value of path for agent i: vi() = t=0… i t hi(s(t),a(t))

Value of path for agent 1: v1() = t=0…3 1

t h1(s(t),a(t)) = 1*0 + 0.5*0 - 0.25*1+ 0.125*4 = 0.25

Value of path for agent 2: v2() = t=0…3 2

t h2(s(t),a(t)) = -1*1 + 0.5*0 - 0.25*1+ 0.125*4 = -0.75

15

Strategies

A strategy is a finite or infinite sequence of local actions for a given player. For example, = <sleep, reset, work, gain> is a strategy for player 1.

where n = 2 and Z = {1}.

2 0 1sleep/0

reset/0

gain/4- 0

work/-1Bi :Ti : *

M

= < sleep, reset, work >

= <0, 0> <sleep, work> <2,1> <reset, gain> <0, 0> <work, work>

A strategy for player i agrees with (is associated to) path iff

16

Value of a Strategy

The value of strategy in state s for player i, vi(Z, s, ), is the minimum value of paths (with the same lenght of ) that agree with . That is:

vi(Z, s, ) = min {vi() | is an (M, Z) path that agrees with strategy of agent i}

2 0 1sleep/0

reset/0

gain/{4, 0}

work/-1Bi :Ti : *

M

For example, let = <work, gain>, then:

v1(, <0, 0>, ) = v1({1}, <0, 0>, ) = -1*1 + 0.5*4 = 1

v1({2}, <0, 0>, ) = v1({1, 2}, <0, 0>, ) = -1*1 + 0.5*0 = -1

In other words, we are assuming that all other players will play against i (pessimistic view). Namely, they will try to minimize i gain.

17

Value of a State

2 0 1sleep/0

reset/0

gain/{4, 0}

work/-1Bi :Ti : *

M

The value of state s at horizon k for player i, vik(Z, s), is the value of the best

strategy of length k for i starting at s. That is:

vik(Z, s), = max{vi(Z, s, ) | is a strategy of length k for agent i}

For example:

v12(, <0, 0>) = v1

2 ({1}, <0, 0>) = -1*1 + 0.5*4 = 1 (witness: <work, gain>)

v12 ({2}, <0, 0>) = -1*1 + 0.5*0 = -1 (witness: <work, gain>)

v12 ({1, 2}, <0, 0>) = 1*0 + 0.5*0 = 0 (witness: <sleep, reset>)

18

Worst Case Value of a State

2 0 1sleep/0

reset/0

gain/{4, 0}

work/-1Bi :Ti : *

M

The worst case value of state s at horizon k for player i, uik(Z, s), is the value

of the worst strategy of length k for i starting at s. That is:

uik(Z, s), = min{vi(Z, s, ) | is a strategy of length k for agent i}

We omit suprscript k when k = .

For example:

u12(, <0, 0>) = -1*1 + 0.5*4 = 1 (witness: <work, gain>)

u12 ({1}, <0, 0>) = 1*0 + 0.5*0 = 0 (witness: <sleep, reset>)

u12 ({2}, <0, 0>) = u1

2({1, 2}, <0, 0>) = -1*1 + 0.5*0 = -1 (witness: <work, gain>)

19

Computing Values of States

Proposition

The value of state s at horizon k for player i, vik(Z, s) can be computed using a

dynamic programming appraoch.

The worst case value of state s at horizon k for player i, uik(Z, s) can be

computed using a dynamic programming appraoch.

2 0 1sleep/0

reset/0

gain/{4, 0}

work/-1Bi :Ti : *

M

s

s1

s2

Sketch a

b

vik (Z,s) =

max{vi(Z, s, ) | is a strategy of length k for agent i} = max {h(s, a) + V(Z, s1), h(s, b) + V(Z, s2)}

20

Nash

Definition. Let M be an n player mechanism, f {0, 1, ... n} and > 0.

• M is -f-Nash for player i if

Z Pf([n] - {i}) s I, ui(Z, s) + vi(Z {i}, s)

• M is -f-Nash if it is -f-Nash for each player i [n].

Intuitively, a mechanism M is -f-Nash if, as long as the number of Byzantine agents is no more than f, no rational agent has an interest greater than in deviating from the proposed protocol.

Pf(Q) = subsets of Q of size at most f.

21

Finite and Infinite Paths

2 1 0 4 5 3d/3

e/-3 a/-1 c/1

b/0h/3

g/-3f/-1

i = 0.5

Bi :

Ti :

u1k(, 0) = -1 + (3/2) – (3/4) + (3/8) – … = -1 + (3/2) t=0, .. k-2(-1/2)t

= (-1)k/(1/2k-1) Strategy: a (d e)

v1k({1}, 0) = 1/(1/2k-1) Strategy: a (d e) when k is even

c (g h) when k is oddThus: if k is odd then u1

k(, 0) < v1k({1}, 0)

if k is even then u1k(, 0) = v1

k({1}, 0)Thus there is no k > 0 s.t. for all k > k, u1

k(, 0) v1k({1}, 0)

In other words, although the above mechanism is -0-Nash, there is no k > 0 s.t. the -0-Nash property can be proved by only looking at a finite prefix of length at most k.

Strategy: a d e d ……

22

Main TheoremLet M be an n player mechanism, f {0, 1, ... n}, > 0 and > 0. Furthermore, for each agent i let:• Mi = max{ |hi(s, a)| | s S and a A}• Ei(k) = 5 i

k Mi/(1 - i)• i(k) = max{ vi

k(Z {i}, s) – uik(Z, s) | s I and Z Pf([n] – {i}) }

• 1(i, k) = i(k) - 2Ei(k) • 2(i, k) = i(k) + 2Ei(k)

For each agent i let ki be s.t. 4 Ei(ki) < . Then:

• If i [ 2(i, ki) > 0] then M is -f-Nash.• If i [0 < 1(i, ki) ] then M is not -f-Nash.• Otherwise M is ( + )-f-Nash.

Proof idea: By computing an upper bound to the error we make by only considering paths of length up to k.

Not -f-Nash1(i, k) 2(i, k)

-f-Nash.( + )-f-Nash

23

Symbolic Algorithmfor i = 1, ... n do

Let k s.t. 4 Ei(k) < ;

Let b := <b1, ... bn> and Ci(b) := [j=1, ...n, ij bj f]

vi0(b, s) := 0, ui

0(b, s) := 0;for t = 1, ... k do

vit(b, s) := max{min {hi(s, <ai, a-i>) + i vi

t-1(s’, b) |

BT(b[bi := 1]), s, <ai, a-i>, s’) Ci(b) a-i A-i} | ai Ai};

uit(b, s) := min{min {hi(s, <ai, a-i>) + i ui

t-1(s’, b) |

BT(b[bi := 0]), s, <ai, a-i>, s’) Ci(b) a-i A-i} | ai Ai};

i := max{vik(s, b) - ui

k(s, b) | Init(s) Ci(b)}

1(i) := i – 2Ei(k); 2(i) := i + 2Ei(k);

if ( < 1(i)) return (FAIL)

if ( i (2(i) < )) return (PASS with )else return (PASS with ( + ))

24

Experimental Results (1)J-0 J-1 . . . . . J-(q–1)Jobs

T-0 T-1 . . . . . T-(q–1)Tasks T-2

Agent 1 task sequence: 0, 1, ... q-1Agent 2 task sequence: 1, 2, 3 ... q-1, 0Agent 3 task sequence: 2, 3, 4... q-1, 0, 1

An agent incurs a cost by working towards the completion of its currently assigned task. Once an agent has completed a task it waits for its reward (if any) before it considers working to the next task in its sequence. As soon as an agent receives its reward it considers working to the next task in its list.

A job is completed if for each task it needs there exists at least one agent that has completed that task. In such a case each of such agents receive a reward. Note that even if two or more agents have completed the same task all of them get a reward.

25

Experimental Results on a 64-bit Dual Quad Core 3GHz Intel Xeon Linux PC with 8GB of RAM

Agents Jobs

=

Tasks

Byz Nash CPU (Sec)

Mem (MB)

MaxBDD Present states Bits

Action bits

5 2 1 Pass 14.9 79.9 5.88+05 15 5

5 2 2 Fail 23.9 74.1 5.42+05 15 5

6 2 2 Pass 209 81.8 6.15+05 18 6

6 2 3 Fail 299 86 4.33+05 18 6

6 3 3 Pass 1680 277 5.62+06 24 6

6 3 4 Fail 1140 217 5.85+05 24 6

7 3 3 Pass 19,100 1850 2.29+07 28 7

7 3 4 Fail 22,200 2280 5.64+07 28 7

8 3 2 Pass 80,300 4660 5.53+07 32 8

8 3 3 N/A >127,000 >8000 >3.45+07 32 8

26

Conclusions

We presented:

• A Formal definition of Proposed Protocol and Mechanism.

• A Formal definition of Nash equilibrium for mechanisms.

• A symbolic algorithm verifying that a given proposed protocol is a Nash equilibrium for a given mechanism.

• Experimental results showing feasibility of our approach.

27

Thanks