polymatrix games: algorithms and applications · rahul savani department of computer science...

Polymatrix Games:Algorithms and Applications

Rahul Savani

Department of Computer ScienceUniversity of Liverpool

Tutorial at theConference on Web and Internet Economics

WINE 2015

Some of talk relates to joint work with Argyrios Deligkas, John Fearnley,Paul Goldberg, Paul Spirakis, and Bernhard von Stengel

What is a polymatrix game?

Polymatrix games are many-player games

For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)

They model pairwise interactions

Nodes correspond to players

Edges correspond to bimatrix games

Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games

What is a polymatrix game?

Polymatrix games are many-player games

For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)

They model pairwise interactions

Nodes correspond to players

Edges correspond to bimatrix games

Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games

What is a polymatrix game?

Polymatrix games are many-player games

For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)

They model pairwise interactions

Nodes correspond to players

Edges correspond to bimatrix games

Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games

What is a polymatrix game?

Polymatrix games are many-player games

For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)

They model pairwise interactions

Nodes correspond to players

Edges correspond to bimatrix games

Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games

What is a polymatrix game?

Polymatrix games are many-player games

For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)

They model pairwise interactions

Nodes correspond to players

Edges correspond to bimatrix games

Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games

What is a polymatrix game?

Polymatrix games are many-player games

For us, they are graphical games:player interactions are captured by an interaction graph(though sometimes this graph is assumed to be complete)

They model pairwise interactions

Nodes correspond to players

Edges correspond to bimatrix games

Each player chooses a single strategy for all hisbimatrix games and receives the sum of the payoffsfrom his bimatrix games

History of polymatrix games

Introduced in:

Janovskaya (1968)Equilibrium points in polymatrix games (in Russian)Latvian Mathematical Collection

We will touch on the following papers here:

Both classical:

Eaves 1973 [9]

Howson 1972 [15]

Howson & Rosenthal 1974 [16]

Miller & Zucker 1991 [19]

And more recent:

Cai et al 2015 [4]

Fearnley et al 2015 [8]

Mehta 2012 [18]

Govindan & Wilson 2004 [14]

Rubinstein 2015 [21]

Polymatrix game

n players i = 1, . . . , n

finite pure strategy sets Si

payoff matrices for every player i and j , i

A ij∈ R

|Si |×|Sj |

For mixed profile (x1, . . . , xn), the payoff to player i is

ui(x1, . . . , xn) =∑i,j

(xi)>A ijx j

Polymatrix game

n players i = 1, . . . , n

finite pure strategy sets Si

payoff matrices for every player i and j , i

A ij∈ R

|Si |×|Sj |

For mixed profile (x1, . . . , xn), the payoff to player i is

ui(x1, . . . , xn) =∑i,j

(xi)>A ijx j

Example polymatrix game

1 2

3

a ba 0,0 2,2b 2,2 0,0

a ba 0,0 2,2b 2,2 0,0

a ba 0,0 1,1b 1,1 0,0

Equilibria:

1 2 3

a b b

b b a

(0.5, 0.5) (0.5, 0.5) (0.5, 0.5)

Example polymatrix game

1 2

3

a ba 0,0 2,2b 2,2 0,0

a ba 0,0 2,2b 2,2 0,0

a ba 0,0 1,1b 1,1 0,0

Equilibria:

1 2 3

a b b

b b a

(0.5, 0.5) (0.5, 0.5) (0.5, 0.5)

Advantage: succinctness

In terms of the number of players, the size of a

strategic-form game is exponential

polymatrix game is polynomial (quadratic)

# players # actions(per player)

# payoffentries

strategic-formn k

n × k n

polymatrix 2k 2 × (n2)

Applications

Polymatrix games are general modelling tool for multi-playergames via pairwise interactions

We will also discuss some other applications from theliterature:

1 Relaxation Labelling Problems for Artificial Neural Networks [19]

2 Graph Transduction in Machine Learning [10]

3 To model 2-player Bayesian Games [16]

4 As a sub-routine for solving general multi-player games [14]

Take-home message

Many things carry over from bimatrix to polymatrix games:

Rational equilibria

Formulation as a Linear Complementarity Problem

Applicability of complementary pivoting algorithms (e.g.Lemke-Howson, Lemke)

Descent methods using Linear Programming for findingApproximate Equilibria

There are also important differences. For polymatrix games:

PPAD-hard to find ε-Nash equilibrium for constant ε

Finding a pure equilibrium is PLS-hard

Take-home message

Many things carry over from bimatrix to polymatrix games:

Rational equilibria

Formulation as a Linear Complementarity Problem

Applicability of complementary pivoting algorithms (e.g.Lemke-Howson, Lemke)

Descent methods using Linear Programming for findingApproximate Equilibria

There are also important differences. For polymatrix games:

PPAD-hard to find ε-Nash equilibrium for constant ε

Finding a pure equilibrium is PLS-hard

Outline

1 Nash equilibria of bimatrix games

2 Linear Complementarity Problems (LCPs)

3 The Lemke–Howson Algorithm and the class PPAD

4 Lemke’s algorithm

5 PLS-hardness of pure equilibria, Graph Transduction

6 Reduction from Polymatrix Game to LCP

7 Descent method for ε-Nash equilibria of polymatrix games

8 Other recent work on polymatrix games

Nash equilibria of bimatrix games

@@I

II

T

M

B

l r

3 31 0

2 50 2

0 64 3

Nash equilibria of bimatrix games

@@I

II

T

M

B

l r

3 31 0

2 50 2

0 64 3

Nash equilibrium =

pair of strategies x, y with

x best response to y andy best response to x

Mixed equilibria

@@I

II

T

M

B

l r

3 31 0

2 50 2

0 64 3

Ay =

3 32 50 6

( 1/3 2/3)T

=

344

xT B =

01/32/3

T 1 0

0 24 3

=(

8/3 8/3)

only only pure best responses canhave

probability > 0

Outline

1 Nash equilibria of bimatrix games

2 Linear Complementarity Problems (LCPs)

3 The Lemke–Howson Algorithm and the class PPAD

4 Lemke’s algorithm

5 PLS-hardness of pure equilibria, Graph Transduction

6 Reduction from Polymatrix Game to LCP

7 Descent method for ε-Nash equilibria of polymatrix games

8 Other recent work on polymatrix games

Linear Complementarity Problem

Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that

z ≥ 0 ⊥ w = q + Mz ≥ 0

⊥ means orthogonal:

zT w = 0⇔ ziwi = 0 all i = 1, . . . , n

If q ≥ 0, the LCP has trivial solution w = q , z = 0.

Linear Complementarity Problem

Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that

z ≥ 0 ⊥ w = q + Mz ≥ 0

⊥ means orthogonal:

zT w = 0⇔ ziwi = 0 all i = 1, . . . , n

If q ≥ 0, the LCP has trivial solution w = q , z = 0.

LP in inequality form

primal : max cT xsubject to Ax ≤ b

x ≥ 0

dual : min yT b

subject to yT A ≥ cT

y ≥ 0

LP in inequality form

primal : max cT xsubject to Ax ≤ b

x ≥ 0

dual : min yT b

subject to yT A ≥ cT

y ≥ 0

Weak duality: x, y feasible (fulfilling constraints)

⇒ cT x ≤ yT Ax ≤ yT b

LP in inequality form

primal : max cT xsubject to Ax ≤ b

x ≥ 0

dual : min yT b

subject to yT A ≥ cT

y ≥ 0

Strong duality: primal and dual feasible

⇒ ∃ feasible x, y : cT x = yT b (x, y optimal)

LCP generalizes LP

LCP encodes complementary slackness of strong duality:

cT x = yT Ax = yT b

⇔ (yT A − cT )x = 0, yT (b − Ax) = 0.

≥ 0 ≥ 0 ≥ 0 ≥ 0

LP⇔ LCP

(xy

)︸︷︷︸

z

≥ 0 ⊥(−c

b

)︸ ︷︷ ︸

q

+

(0 AT

−A 0

)︸ ︷︷ ︸

M

(xy

)︸︷︷︸

z

≥ 0

LCP generalizes LP

LCP encodes complementary slackness of strong duality:

cT x = yT Ax = yT b

⇔ (yT A − cT )x = 0, yT (b − Ax) = 0.

≥ 0 ≥ 0 ≥ 0 ≥ 0

LP⇔ LCP

(xy

)︸︷︷︸

z

≥ 0 ⊥(−c

b

)︸ ︷︷ ︸

q

+

(0 AT

−A 0

)︸ ︷︷ ︸

M

(xy

)︸︷︷︸

z

≥ 0

Outline

1 Nash equilibria of bimatrix games

2 Linear Complementarity Problems (LCPs)

3 The Lemke–Howson Algorithm and the class PPAD

4 Lemke’s algorithm

5 PLS-hardness of pure equilibria, Graph Transduction

6 Reduction from Polymatrix Game to LCP

7 Descent method for ε-Nash equilibria of polymatrix games

8 Other recent work on polymatrix games

Symmetric equilibria of symmetric games

Given: n n payoff matrix A for row player AT for column player

mixed strategy x = probability distribution on {1,...,n} x 0 , 1Tx = 1

equilibrium (x, x) x best response to x

Remark: As general as m n games (A, B).

Best responses

Given: n n payoff matrix A, mixed strategy y of column player

Ay = vector of expected payoffs against y, components (Ay)i

x best response to y

x maximizes expected payoff xTAy

best response condition:

∀i : xi > 0 (Ay)i = u = maxk (Ay)k

Symmetric equilibria as LCP solutions

equilibrium (x, x) of game with payoff matrix A x best response to x

1Tx = 1,

x 0 Ax ≤ 1u

w.l.o.g. A > 0 u > 0,

equilibrium (x, x)

z = (1/u) x ( 1/u = 1Tz ),

z 0 Az ≤ 1 "equilibrium z"

Best response polyhedron

0

2

1

1

1 2

2 0A =

1x

2x

u<>x 0,{ ( , ) |x u }1Tx= 1, x uA 1

1

Best response polyhedron

1

1

2

2

211

2

0

2

1

1

1 2

2 0A =

1x

2x

u<>x 0,{ ( , ) |x u }1Tx= 1, x uA 1

1

Best response polyhedron

1

1

2

2

21

1

2

0

2

1

1

1 2

2 0A =

1x

2x

u<>x 0,{ ( , ) |x u }1Tx= 1, x uA 1

(2/3, 1/3)

(completely labeled)equilibrium

1

Projective transformation

1 2

2 0A =

1x

2x

u<>x 0,{ ( , ) |x u }1Tx= 1, x uA 1

>x 0, <xA 1{ ( , ) |1x }1

>z 0, <zA 1

Best response polytope

{ |z }

2

1

2

1

1 2

2 0A =

2z

1z

Symmetric Lemke−Howson algorithm

1z

2z

z3

(bottom)

(back)

2

1

1

2

33

Symmetric Lemke−Howson algorithm

1missing label

1z

2z

z3

(bottom)

(back)

2

1

1

2

33

Symmetric Lemke−Howson algorithm

1missing label

1z

2z

z3

(bottom)

(back)

2

1

1

2

33

Symmetric Lemke−Howson algorithm

1missing label

1z

2z

z3

(bottom)

(back)

2

1

1

2

33

1missing label

Symmetric Lemke−Howson algorithm

1z

2z

z3

(bottom)

(back)

2

1

1

2

33

1missing label

Symmetric Lemke−Howson algorithm

1z

2z

z3

(bottom)

(back)

2

1

1

2

33

found label 1

Symmetric Lemke−Howson algorithm

1z

2z

z3

(bottom)

(back)

2

1

1

2

33

Why Lemke-Howson works

LH finds at least one Nash equilibrium because

• finitely many "vertices"

for nondegenerate (generic) games:

• unique starting edge given missing label

• unique continuation

precludes "coming back" like here:

END OF LINE (Papadimitriou 1991)

start

end

Given a graph G ofindegree/outdegree at most 1,and a start vertex of indegree 0and outdegree 1,find another vertex of degree1

END OF LINE (Papadimitriou 1991)

start0000

0101

end

Catch:graph is exponentially largedefined by two boolean circuitsS , P that take a vertex in {0, 1}n

and output its successor andpredecessor

S(0000) = 0101

P(0101) = 0000

END OF LINE (Papadimitriou 1991)

start

end

A problem belongs to PPAD if itis reducible in poly-time to ENDOF LINE; and PPAD-completeif END OF LINE is reducible toit.

END OF LINE (Papadimitriou 1991)

start

end

A problem belongs to PPAD if itis reducible in poly-time to ENDOF LINE; and PPAD-completeif END OF LINE is reducible toit.

Not to be confused with

OTHER END OF THIS LINE

output unique vertex endfound by “following the line”from the start – this isPSPACE-hard

PPAD-hardness for bimatrix games

Theorem (DGP06, CDT06 [5, 6])

It is PPAD-complete to compute an exact Nash equilibrium of abimatrix game.

Later we will see PPAD-hardness for approximate equilibriaof bimatrix and polymatrix games

Outline

1 Nash equilibria of bimatrix games

2 Linear Complementarity Problems (LCPs)

3 The Lemke–Howson Algorithm and the class PPAD

4 Lemke’s algorithm

5 PLS-hardness of pure equilibria, Graph Transduction

6 Reduction from Polymatrix Game to LCP

7 Descent method for ε-Nash equilibria of polymatrix games

8 Other recent work on polymatrix games

Costs instead of payoffs

1 2 2 1

2 0 1 3

aik 3 − aik

payoff cost

with new cost matrix A > 0 :

equilibrium z z 0 Az 1

Polyhedral view

1 + 3

2 + 1

1z ≥ 0

2z1z ≥ 1

2z1z ≥ 1

2z ≥ 0

1z

2z

1

2

1

2

Lemke's algorithm

given LCP

z 0 w = q + Mz 0

Lemke's algorithm

augmented LCP

z 0 w = q + Mz + dz0 0 z0 0

Lemke's algorithm

augmented LCP

z 0 w = q + Mz + dz0 0 z0 0

where

d > 0 covering vectorz0 extra variable

z0 = 0 z w solves original LCP

Lemke's algorithm

augmented LCP

z 0 w = q + Mz + dz0 0 z0 0

Initialization:

z 0 w = q + dz0 0

z0 0 minimal wi = 0 for some i

pivot z0 in, wi out,

can increase zi while maintaining z w .

Lemke's algorithm for

M = 2 1 , d = 2 1 3 1

w1 −1 2 1 2= + z1 + z2 + z0

w2 −1 1 3 1

w1 1 0 −5 −2= + z1 + z2 + w2

z0 1 −1 −3 −1

w1 −1 2 1 2= + z1 + z2 + z0

w2 −1 1 3 1

w1 1 0 −5 −2= + z1 + z2 + w2

z0 1 −1 −3 −1

z2 0.2 0 −0.2 −0.4= + z1 + w1 + w2

z0 0.4 −1 0.6 0.2

w1 1 0 −5 −2= + z1 + z2 + w2

z0 1 −1 −3 −1

z2 0.2 0 −0.2 −0.4= + z1 + w1 + w2

z0 0.4 −1 0.6 0.2

z2 0.2 0 −0.2 −0.4= + z0 + w1 + w2

z1 0.4 −1 0.6 0.2

Polyhedral view of Lemke

Polyhedral view of Lemke

1z

2z

1

2

1

2

Polyhedral view of Lemke

0z

1z

2z

1

2

1

2

Polyhedral view of Lemke

1z

2z

0z

1

2

1

2

Polyhedral view of Lemke

1z

2z

0z

1

2

1

2

Polyhedral view of Lemke

1z

2z

0z

1

2

1

2

Polyhedral view of Lemke

1z

2z

0z

0z = 0

1

2

1

2

Outline

1 Nash equilibria of bimatrix games

2 Linear Complementarity Problems (LCPs)

3 The Lemke–Howson Algorithm and the class PPAD

4 Lemke’s algorithm

5 PLS-hardness of pure equilibria, Graph Transduction

6 Reduction from Polymatrix Game to LCP

7 Descent method for ε-Nash equilibria of polymatrix games

8 Other recent work on polymatrix games

The class PLS (Polynomial Local Search)

s Given a starting solutions ∈ S = Σn

a P-time algorithm thatcomputes the cost c(s)

a P-time function that computesa neighbouring solutions′ ∈ N(s) with lower cost, i.e.s.t. c(s′) < c(s), or reportsthat no such neighbour exists:

find a local optimum of thecost function c

“every DAG has a sink”

Local Max Cut

Find local optimum ofMax Cut with the FLIP-neighbourhood (exactly onenode can change sides)

Schaffer and Yannakakis [22] showed that Local Max Cutis PLS-complete (via an extremely involved reduction)

Local Max Cut is to PLS what 3-SAT is to NP

1 2

3 4

1

1

−4

31

−2

Local Max Cut

Find local optimum ofMax Cut with the FLIP-neighbourhood (exactly onenode can change sides)

Schaffer and Yannakakis [22] showed that Local Max Cutis PLS-complete (via an extremely involved reduction)

Local Max Cut is to PLS what 3-SAT is to NP

1 2

3 4

1

1

−4

31

−2

Local Max Cut

Find local optimum ofMax Cut with the FLIP-neighbourhood (exactly onenode can change sides)

Schaffer and Yannakakis [22] showed that Local Max Cutis PLS-complete (via an extremely involved reduction)

Local Max Cut is to PLS what 3-SAT is to NP

1 2

3 4

1

1

−4

31

−2

Solutions:

{{1, 3, 4}, {2}} (actual Max Cut)

Local Max Cut

Find local optimum ofMax Cut with the FLIP-neighbourhood (exactly onenode can change sides)

Schaffer and Yannakakis [22] showed that Local Max Cutis PLS-complete (via an extremely involved reduction)

Local Max Cut is to PLS what 3-SAT is to NP

1 2

3 4

1

1

−4

31

−2

Solutions:

{{1, 3, 4}, {2}} (actual Max Cut){{3}, {1, 2, 4}}

Pure Equilibrium in Polymatrix Game

1 2

3

2

−1 2

Pure Equilibrium in Polymatrix Game

1 2

3

a ba 0,0 2,2b 2,2 0,0

a ba 0,0 2,2b 2,2 0,0

a ba 0,0 -1,-1b -1,-1 0,0

Pure Equilibrium in Polymatrix Game

1 2

3

a ba 0,0 2,2b 2,2 0,0

a ba 0,0 2,2b 2,2 0,0

a ba 0,0 -1,-1b -1,-1 0,0

The bimatrix games (A ,B) we used are examples of teamgames because A = B; also called coordination games

Proof that the reduction is correct

Define potential function for “team” polymatrix games

Φ(S) =12

∑i

ui(S)

This is an exact potential function:when i changes strategy then the potential functionchanges by exactly i’s change in utilityFact: in exact potential games,pure equilibria↔ local optima of exact potentialfunctionOur exact potential function value equals value of the cutfor all strategy profiles

�

Summary on PLS and polymatrix games

In contrast to bimatrix games, computing a pureequilibrium in polymatrix games is PLS-hard

Next, an application of team polymatrix games

Application: Graph Transduction

semi-supervised learning: estimate a classificationfunction defined over graph of labeled and unlabeled nodes

ie. propagate labels to unlabelled nodes in consistent way

INPUT: Weighted graph, where some nodes are labelled;

edge weights represent similarities

one approach is to use global optimization

an alternative approach is to use a polymatrix game

Note: without the labelled examples, this is a clusteringproblem; also see e.g., “Hedonic Clustering Games” [12, 2]

Application: Graph Transduction

semi-supervised learning: estimate a classificationfunction defined over graph of labeled and unlabeled nodes

ie. propagate labels to unlabelled nodes in consistent way

INPUT: Weighted graph, where some nodes are labelled;

edge weights represent similarities

one approach is to use global optimization

an alternative approach is to use a polymatrix game

Note: without the labelled examples, this is a clusteringproblem; also see e.g., “Hedonic Clustering Games” [12, 2]

Application: Graph Transduction

semi-supervised learning: estimate a classificationfunction defined over graph of labeled and unlabeled nodes

ie. propagate labels to unlabelled nodes in consistent way

INPUT: Weighted graph, where some nodes are labelled;

edge weights represent similarities

one approach is to use global optimization

an alternative approach is to use a polymatrix game

Note: without the labelled examples, this is a clusteringproblem; also see e.g., “Hedonic Clustering Games” [12, 2]

Application: Graph Transduction

1 2

3

a ba 2,2 0,0b 0,0 2,2

a ba 2,2 0,0b 0,0 2,2

a ba -1,-1 0,0b 0, 0 -1,-1

Note: asymmetric similarity measures have also beenconsidered. Then we may no longer have pure equilibria, butmixed equilibria are still considered meaningful

Application: Graph Transduction

1 2

3

a ba 2,2 0,0b 0,0 2,2

a ba 2,2 0,0b 0,0 2,2

a ba -1,-1 0,0b 0, 0 -1,-1

Note: asymmetric similarity measures have also beenconsidered. Then we may no longer have pure equilibria, butmixed equilibria are still considered meaningful

Open question for team polymatrix games

Can we compute a mixed Nash equilibrium of a teampolymatrix game in polynomial-time? [7]

Note that this problem lies in PPAD ∩ PLS so is unlikely to behard for either of them

Question:

Can anyone think of an easy mixed equilibrium for thelocal max cut game?

Suggested reading:

Daskalakis & PapadimitriouContinuous local search SODA 2011

Open question for team polymatrix games

Can we compute a mixed Nash equilibrium of a teampolymatrix game in polynomial-time? [7]

Note that this problem lies in PPAD ∩ PLS so is unlikely to behard for either of them

Question:

Can anyone think of an easy mixed equilibrium for thelocal max cut game?

Suggested reading:

Daskalakis & PapadimitriouContinuous local search SODA 2011

Open question for team polymatrix games

Can we compute a mixed Nash equilibrium of a teampolymatrix game in polynomial-time? [7]

Note that this problem lies in PPAD ∩ PLS so is unlikely to behard for either of them

Question:

Can anyone think of an easy mixed equilibrium for thelocal max cut game?

Suggested reading:

Daskalakis & PapadimitriouContinuous local search SODA 2011

Outline

1 Nash equilibria of bimatrix games

2 Linear Complementarity Problems (LCPs)

3 The Lemke–Howson Algorithm and the class PPAD

4 Lemke’s algorithm

5 PLS-hardness of pure equilibria, Graph Transduction

6 Reduction from Polymatrix Game to LCP

7 Descent method for ε-Nash equilibria of polymatrix games

8 Other recent work on polymatrix games

Polymatrix games→ LCPs

At least three different reductions to LCP; each gives analmost-complementarity algorithm

1 Howson 1972 [15]2 Eaves 1973 [9] (more general)3 Miller and Zucker 1991 [19]

Instead we are going to present bilinear games whichappeared in Ruta Mehta’s thesis [18, 13], and which are aspecialization of Eave’s games

Bilinear Games

Inspired by sequence form of Koller, Megiddo, von Stengel(1996) [17]

They turn out to be are a special case of Eaves’ polymatrixgames with joint constraints [9], where we restrict to:

two players

polytopal strategy constraint sets

Bilinear Games

A bilinear game is given by:

two m × n dimensional payoff matrices A and B

polytopal strategy constraint sets:

X = {x ∈ Rm| Ex = e, x ≥ 0}

Y = {y ∈ Rn| Fy = f , y ≥ 0}

With payoffs xT Ay and xT By

for the strategy profile (x, y) ∈ X × Y

Bilinear Games

A bilinear game is given by:

two m × n dimensional payoff matrices A and B

polytopal strategy constraint sets:

X = {x ∈ Rm| Ex = e, x ≥ 0}

Y = {y ∈ Rn| Fy = f , y ≥ 0}

(x, y) ∈ X × Y is a Nash equilibrium iff

xT Ay ≥ xT A for all x ∈ X and

xT By ≥ xT By for all y ∈ Y

An LCP for Bilinear Games

Encode best response condition via an LP:

maxx

x>(Ay)

s.t. x>E> = e>, x ≥ 0

An LCP for Bilinear Games

Encode best response condition via an LP:

maxx

x>(Ay)

s.t. x>E> = e>, x ≥ 0

The dual LP has an unconstrained vector p:

miny

e>p

s.t. E>p ≥ Ay

We will again use complementary slackness:

An LCP for Bilinear Games

Encode best response condition via an LP:

maxx

x>(Ay)

s.t. x>E> = e>, x ≥ 0

The dual LP has an unconstrained vector p:

miny

e>p

s.t. E>p ≥ Ay

We will again use complementary slackness:

Feasible x, p are optimal iff x>(Ay) = e>p = x>E>p, i.e.,

An LCP for Bilinear Games

Encode best response condition via an LP:

maxx

x>(Ay)

s.t. x>E> = e>, x ≥ 0

The dual LP has an unconstrained vector p:

miny

e>p

s.t. E>p ≥ Ay

We will again use complementary slackness:

x>(−Ay + E>p) = 0

An LCP for Bilinear Games

Given: q ∈ Rn, M ∈ Rn×n Find: z, w ∈ Rn so that

M =

−A E> −E>

−B> F> −F>

−EE−F

F

q =

00e−e

f−f

z = (x, y, p′, p′′, q′, q′′)>

wherep = p′ − p′′, q = q′ − q′′

Lemke’s algorithm for Bilinear Games

Theorem 4.1 in [17] says:

If we have

1 z>Mz ≥ 0 for all z ≥ 0, and

2 z ≥ 0, Mz ≥ 0 and z>Mz = 0 imply that z>q ≥ 0

then

Lemke’s algorithm computes an solution to the LCP M, q

Polymatrix games as Bilinear Games

Polymatrix game (with complete interaction graph):

players i = 1, . . . , n, with pure strategy sets Si

and payoff matrices for player i,

A ij∈ R

|Si |×|Sj |

for pairs of players (i, j)

let (x1, . . . , xn) in ∆(Si) × · · · ×∆(Sn) be a mixed strategyprofile, then the payoff to player i is

ui(x1, . . . , xn) =∑i,j

(xi)>A ijx j

Polymatrix games as Bilinear Games

(Symmetric) bilinear game: (A ,A>,E,E, e, e)

payoff matrices (A ,A>)

strategy constraints Ex = e

where e = 1n, and

A =

0 A12 · · · A1n

A21 0 A2n

.... . .

An1 An2 · · · 0

E =

1>

|S1|0 · · · 0

0 1>

|S2|· · · 0

.... . .

0 0 · · · 1>

|Sn |

Reductions for sparse polymatrix games

Existing reductions apply to polymatrix games oncomplete interaction graphs

For other interactions graphs, missing edges are replacedwith games with all 0 payoffs

Can we come up with more space efficient reductionsfor non-complete interaction graphs?

Outline

1 Nash equilibria of bimatrix games

2 Linear Complementarity Problems (LCPs)

3 The Lemke–Howson Algorithm and the class PPAD

4 Lemke’s algorithm

5 PLS-hardness of pure equilibria, Graph Transduction

6 Reduction from Polymatrix Game to LCP

7 Descent method for ε-Nash equilibria of polymatrix games

8 Other recent work on polymatrix games

Approximation - Background

Definition (ε-Nash equilibrium)

A strategy profile is an ε-Nash equilibrium if:

no player can gain more than ε by a unilateral deviation

(additive notion of approximation)

Approximation - Background

Definition (ε-Nash equilibrium)

A strategy profile is an ε-Nash equilibrium if:

no player can gain more than ε by a unilateral deviation

(additive notion of approximation)

Theorem (Rubinstein 2014)

There exists a constant ε such that it is PPAD-hard to find anε-Nash equilibrium of a n-player polymatrix game.

Approximation - Background

Definition (ε-Nash equilibrium)

A strategy profile is an ε-Nash equilibrium if:

no player can gain more than ε by a unilateral deviation

(additive notion of approximation)

Theorem (CDT 2006)

If there is an FPTAS for computing an ε-Nash of a bimatrixgame, then PPAD = P.

Background: bimatrix games

What is the smallest ε such that an ε-Nash equilibrium can becomputed in polynomial time (payoffs in [0, 1])?

HISTORY:

0.5 Daskalakis Mehta Papadimitriou (WINE 06)

0.382 DMP (EC 2007)

0.364 Bosse Byrka Markakis (WINE 07)

0.339 Tsaknakis Spirakis (WINE 07)

Tsaknakis & Spirakis use gradient descent

Background: bimatrix games

What is the smallest ε such that an ε-Nash equilibrium can becomputed in polynomial time (payoffs in [0, 1])?

HISTORY:

0.5 Daskalakis Mehta Papadimitriou (WINE 06)

0.382 DMP (EC 2007)

0.364 Bosse Byrka Markakis (WINE 07)

0.339 Tsaknakis Spirakis (WINE 07)

Tsaknakis & Spirakis use gradient descent

Background: many-player games

Two players: 0.3393 [Tsaknakis and Spirakis]

n players: 1 − 1/n [obvious extension of DMP]

DMP idea extends solution for n − 1 players to n players:

Three players: 0.6022

Four players: 0.7153

Guarantee goes to 1 as n goes to infinity

Next we show for the class of n-player polymatrix games:(0.5 + δ) in time polynomial in the input size and 1/δ

Background: many-player games

Two players: 0.3393 [Tsaknakis and Spirakis]

n players: 1 − 1/n [obvious extension of DMP]

DMP idea extends solution for n − 1 players to n players:

Three players: 0.6022

Four players: 0.7153

Guarantee goes to 1 as n goes to infinity

Next we show for the class of n-player polymatrix games:(0.5 + δ) in time polynomial in the input size and 1/δ

Background: many-player games

Two players: 0.3393 [Tsaknakis and Spirakis]

n players: 1 − 1/n [obvious extension of DMP]

DMP idea extends solution for n − 1 players to n players:

Three players: 0.6022

Four players: 0.7153

Guarantee goes to 1 as n goes to infinity

Next we show for the class of n-player polymatrix games:(0.5 + δ) in time polynomial in the input size and 1/δ

Gradient descent on max regret

Extend method of Tsaknakis and Spirakis

Definition

For a strategy profile x we define f(x) as the regret:

f(x) := maxi∈players

u∗i(x) − ui(x)

define δ-stationary point of f via combinatorial “gradient”

LP to find a corresponding steepest descent direction

Gradient descent on max regret

Extend method of Tsaknakis and Spirakis

Definition

For a strategy profile x we define f(x) as the regret:

f(x) := maxi∈players

u∗i(x) − ui(x)

define δ-stationary point of f via combinatorial “gradient”

LP to find a corresponding steepest descent direction

The algorithm

1 Choose an arbitrary strategy profile x ∈ ∆

2 Solve steepest descent LP with input x to obtain x′

3 Set x := x + α(x′ − x), where α = δδ+2

4 If f(x) ≤ 0.5 + δ then stop, otherwise go to step 2

The result

Theorem

A (0.5 + δ)-Nash equilibrium of a polymatrix game can befound in time polynomial in the size of the game and in 1/δ.

Proof sketch:

We do not get stuck at a bad point: Every δ-stationarypoint x∗ of f is a (0.5 + δ)-NE, i.e., f(x∗) ≤ 0.5 + δ

Each descent step makes enough progress in reducing f ,so that after polynomially many iterations f(x) ≤ 0.5 + δ

Open questions on approximate equilibria

Better upper bounds:

Constant number of players or strategies

Extend methods for bimatrix games that solve a single LP

ε-well-supported approximate equilibria

Lower bounds:

It is PPAD-hard to find an ε-Nash equilibrium of a polymatrixgame for a constant but very small ε [Rubinstein]

Improve the value of ε in such a lower bound

Application: 2-player Bayesian games

Howson and Rosenthal (1974) observed that these gamescan be written as a complete bipartite polymatrix games

Types of P1 Types of P2

1

2

3

1

2

The descent algorithm gives a 1/2-Nash but this is easilyachievable by the DMP method

Open question: do other methods for bimatrix games alsoextend to Bayesian two-player games?

Application: 2-player Bayesian games

Howson and Rosenthal (1974) observed that these gamescan be written as a complete bipartite polymatrix games

Types of P1 Types of P2

1

2

3

1

2

The descent algorithm gives a 1/2-Nash but this is easilyachievable by the DMP method

Open question: do other methods for bimatrix games alsoextend to Bayesian two-player games?

Enumerating equilibria

All methods we discussed are to find one sampleequilibrium

Often a proper analysis of a game requires anenumeration of all equilibria

Well-developed enumeration methods for bimatrixgames [1]

It is an interesting direction to develop similar methodsfor polymatrix games

Outline

1 Nash equilibria of bimatrix games

2 Linear Complementarity Problems (LCPs)

3 The Lemke–Howson Algorithm and the class PPAD

4 Lemke’s algorithm

5 PLS-hardness of pure equilibria, Graph Transduction

6 Reduction from Polymatrix Game to LCP

7 Descent method for ε-Nash equilibria of polymatrix games

8 Other recent work on polymatrix games

Other recent work on polymatrix games

Solving general multi-player games [14] (also see [11])

Zero-sum polymatrix games [4]

Efficiency of equilibria in polymatrix coordinationgames [20]

QPTAS for tree polymatrix games [3]

References I

[1] David Avis, Gabriel D. Rosenberg, Rahul Savani, and Bernhardvon Stengel.Enumeration of Nash equilibria for two-player games.Economic Theory, 42(1):9–37, 2009.

[2] Haris Aziz and Rahul Savani.Hedonic Games, chapter 15.Cambridge University Press, 2015.In press.

[3] Siddharth Barman, Katrina Ligett, and Georgios Piliouras.Approximating nash equilibria in tree polymatrix games.In Algorithmic Game Theory - 8th International Symposium,(SAGT), pages 285–296, 2015.

References II

[4] Yang Cai, Ozan Candogan, Constantinos Daskalakis, andChristos Papadimitriou.Zero-sum polymatrix games: A generalization of minmax.Mathematics of Operations Research, To appear.

[5] Xi Chen, Xiaotie Deng, and Shang-Hua Teng.Settling the complexity of computing two-player Nash equilibria.Journal of the ACM, 56(3):14:1–14:57, 2009.

[6] Constantinos Daskalakis, Paul W. Goldberg, and Christos H.Papadimitriou.The complexity of computing a Nash equilibrium.SIAM Journal on Computing, 39(1):195–259, 2009.

References III

[7] Constantinos Daskalakis and Christos Papadimitriou.Continuous local search.In Proceedings of the twenty-second annual ACM-SIAMsymposium on Discrete Algorithms, pages 790–804. SIAM,2011.

[8] Argyrios Deligkas, John Fearnley, Rahul Savani, and PaulSpirakis.Computing approximate Nash equilibria in polymatrix games.Algorithmica, 2015.Online first; Preliminary conference version appeared at WINE2014.

[9] B Curtis Eaves.Polymatrix games with joint constraints.SIAM Journal on Applied Mathematics, 24(3):418–423, 1973.

References IV

[10] Aykut Erdem and Marcello Pelillo.Graph transduction as a noncooperative game.Neural Computation, 24(3):700–723, 2012.

[11] Uriel Feige and Inbal Talgam-Cohen.A direct reduction from k-player to 2-player approximate Nashequilibrium.In Algorithmic Game Theory - Third International Symposium(SAGT), pages 138–149, 2010.

[12] Moran Feldman, Liane Lewin-Eytan, and Joseph Seffi Naor.Hedonic clustering games.In Proceedings of the 24th Annual ACM symposium onParallelism in Algorithms and Architectures SPAA, pages267–276. ACM, 2012.

References V

[13] Jugal Garg, Albert Xin Jiang, and Ruta Mehta.Bilinear games: Polynomial time algorithms for rank basedsubclasses.In Internet and Network Economics - 7th InternationalWorkshop, WINE, pages 399–407, 2011.

[14] Srihari Govindan and Robert Wilson.Computing Nash equilibria by iterated polymatrix approximation.Journal of Economic Dynamics and Control, 28(7):1229–1241,April 2004.

[15] Joseph T. Howson.Equilibria of polymatrix games.Management Science, 18(5):pp. 312–318, 1972.

References VI

[16] Jr. Howson, Joseph T. and Robert W. Rosenthal.Bayesian equilibria of finite two-person games with incompleteinformation.Management Science, 21(3):pp. 313–315, 1974.

[17] Daphne Koller, Nimrod Megiddo, and Bernhard von Stengel.Efficient computation of equilibria for extensive two-persongames.Games and Economic Behavior, 14(2):247–259, 1996.

[18] Ruta Mehta.Nash Equilibrium Computation in Various Games.PhD thesis, Dept. of CSE, IIT-Bombay, 8 2012.

References VII

[19] Douglas A. Miller and Steven W. Zucker.Copositive-plus lemke algorithm solves polymatrix games.Operations Research Letters, 10(5):285 – 290, 1991.

[20] Mona Rahn and Guido Schafer.Efficient equilibria in polymatrix coordination games.CoRR, abs/1504.07518, 2015.

[21] Aviad Rubinstein.Inapproximability of Nash equilibrium.In Proceedings of the Forty-Seventh Annual ACM onSymposium on Theory of Computing, STOC, pages 409–418,2015.

[22] Alejandro A Schaffer and Mihalis Yannakakis.Simple local search problems that are hard to solve.SIAM journal on Computing, 20(1):56–87, 1991.