equilibrium concepts in two player games

Equilibrium Equilibrium Concepts in Two Concepts in Two

Player GamesPlayer GamesKevin ByrnesKevin Byrnes

Department of Applied Mathematics & Department of Applied Mathematics & StatisticsStatistics

Nash EquilibriaNash EquilibriaA set of strategies (x*,y*) in a two A set of strategies (x*,y*) in a two player game is a player game is a Nash equilibrium Nash equilibrium

pointpoint if: if:

f(x*,y*) >=f(x,y*) for all x in Sf(x*,y*) >=f(x,y*) for all x in S11

g(x*,y*)>=g(x*,y) for all y in Sg(x*,y*)>=g(x*,y) for all y in S22

Nash EquilibriaNash EquilibriaMore generally, a set of strategies More generally, a set of strategies (x(x11*,…,x*,…,xnn*) in an n player game is *) in an n player game is

a Nash equilibrium point if:a Nash equilibrium point if:

ff11(x(x11*,…,x*,…,xnn*)>=f*)>=f11(x,...,x(x,...,xnn*) for all x *) for all x in Sin S11

ffnn(x(x11*,…,x*,…,xnn*)>=f*)>=fnn(x(x11*,…,x) for all x *,…,x) for all x in Sin Snn

Nash EquilibriaNash EquilibriaThe key propertiesThe key properties of a Nash of a Nash

equilibrium are that it is equilibrium are that it is enforceable and guaranteed to enforceable and guaranteed to exist under certain reasonable exist under certain reasonable

assumptions.assumptions.

Nash EquilibriaNash EquilibriaEnforceability comes from the Enforceability comes from the

fact that every player is fact that every player is unilaterally maximizing his payoff, unilaterally maximizing his payoff, ie no player can single-handedly ie no player can single-handedly

deviate to do better.deviate to do better.To see why it must exist, consider To see why it must exist, consider

the following sketch of a proof:the following sketch of a proof:

Nash EquilibriaNash EquilibriaLemma 1: If fLemma 1: If f11,…,f,…,fnn are concave, are concave, then the best response sets are then the best response sets are

convexconvex

Lemma 2: If fLemma 2: If f11,…,f,…,fnn are also are also continuous and the strategy continuous and the strategy

spaces are compact, then the best spaces are compact, then the best response sets form an upper hemi response sets form an upper hemi

continuous correspondencecontinuous correspondence

Nash EquilibriaNash EquilibriaExistence TheoremExistence Theorem (Nash): Assuming (Nash): Assuming that the fthat the fii are continuous and concave, are continuous and concave,

and that the strategy spaces Sand that the strategy spaces Sii are are compact, then a Nash equilibrium compact, then a Nash equilibrium

exists.exists.

Proof: By Lemmas 1 and 2 the BRProof: By Lemmas 1 and 2 the BRii(.) (.) are upper hemi continuous are upper hemi continuous

correspondences on compact sets. By correspondences on compact sets. By Kakutani’s Fixed Point Thm. we then Kakutani’s Fixed Point Thm. we then know that we must have a fixed point.know that we must have a fixed point.

Nash EquilibriaNash EquilibriaNow let us consider a special type of Now let us consider a special type of

game, namely a bimatrix game (ie both game, namely a bimatrix game (ie both players have a finite number of pure players have a finite number of pure

strategies and payoff matrices)strategies and payoff matrices)

Let A be player 1’s payoff matrix, so Let A be player 1’s payoff matrix, so AAijij=f(s=f(sii,,ssjj), where s), where sii is a pure strategy in is a pure strategy in

SS11, and , and ssjj is a pure strategy in S is a pure strategy in S22..

Let B be player 2’s payoff matrix, so Let B be player 2’s payoff matrix, so BBijij=g(s=g(sii,,ssjj))

Nash EquilibriaNash EquilibriaNow let xNow let xii denote the probability with denote the probability with

which player 1 plays swhich player 1 plays sii, and let y, and let yjj denote the probability with which denote the probability with which

player 2 plays player 2 plays ssjj. Then a Nash . Then a Nash equilibrium is a pair of strategies equilibrium is a pair of strategies

(x*,y*) such that x* satisfies (i) and y* (x*,y*) such that x* satisfies (i) and y* satisfies (ii), where we define (i) and (ii) satisfies (ii), where we define (i) and (ii)

as:as:

Nash EquilibriaNash Equilibria(i)(i) MaxMaxxx x xTTAy*Ay*

Subject to: xSubject to: x11+…+x+…+xmm=1=1x>=0x>=0

(ii)(ii) MaxMaxyy x* x*TTByBy

Subject to: ySubject to: y11+…+y+…+ynn=1=1y>=0 y>=0

Nash EquilibriaNash EquilibriaBy our existence proof, we know that By our existence proof, we know that

such equilibria exist, the question such equilibria exist, the question is, how can we find them?is, how can we find them?

Nash EquilibriaNash EquilibriaBy our existence proof, we know that By our existence proof, we know that

such equilibria exist, the question such equilibria exist, the question is, how can we find them?is, how can we find them?

It turns out that It turns out that Vorob’evVorob’ev, , KuhnKuhn, , LemkeLemke, and , and HowsonHowson (inter alia) (inter alia) have proposed algorithms for have proposed algorithms for

finding Nash equilibrium points for finding Nash equilibrium points for the special case of bimatrix games. the special case of bimatrix games. Computing such equilibria may be Computing such equilibria may be expensive, however. Thus we shall expensive, however. Thus we shall now focus on a key geometric result now focus on a key geometric result of Mangasarian that tells us which of Mangasarian that tells us which

equilibrium points we really need to equilibrium points we really need to generate. generate.

The Geometry of The Geometry of EquilibriaEquilibria

For a bimatrix game, finding an For a bimatrix game, finding an equilibrium point is equivalent to equilibrium point is equivalent to

simultaneously solving problems (i) simultaneously solving problems (i) and (ii). But each of these is just an and (ii). But each of these is just an

LP. LP.


For a bimatrix game, finding an For a bimatrix game, finding an equilibrium point is equivalent to equilibrium point is equivalent to

simultaneously solving problems (i) simultaneously solving problems (i) and (ii). But each of these is just an and (ii). But each of these is just an

LP. LP.

Now recall that to solve a single LP, we Now recall that to solve a single LP, we just need to look at the extreme just need to look at the extreme

points of the feasible region, could points of the feasible region, could we be so lucky here?we be so lucky here?


Before proceeding, it is useful to note Before proceeding, it is useful to note that if a pure strategy Nash that if a pure strategy Nash

equilibrium exists in a bimatrix equilibrium exists in a bimatrix game, it may be found in a game, it may be found in a straightforward fashion.straightforward fashion.

Consider the following game:Consider the following game:UpUp DownDown

LeftLeft (4,2)(4,2) (2,3)(2,3)

RightRight (6,-1)(6,-1) (0,0)(0,0)


A yellow box indicates the row player’s A yellow box indicates the row player’s best response to a given column best response to a given column strategy. A red box indicates the strategy. A red box indicates the

column player’s best response to a column player’s best response to a given row strategy.given row strategy.

UpUp DownDown

LeftLeft (4,2)(4,2) (2,3)(2,3)

RightRight (6,-1)(6,-1) (0,0)(0,0)


A Nash equilibrium exists at the A Nash equilibrium exists at the intersection of any of these two best intersection of any of these two best

responses.responses.

UpUp DownDown

LeftLeft (4,2)(4,2) (2,3)(2,3)

RightRight (6,-1)(6,-1) (0,0)(0,0)


First recall problems (i) and (ii):First recall problems (i) and (ii):

(i) Max(i) Maxxx x xTTAy*Ay*Subject to: eSubject to: eTTx=1x=1

x>=0x>=0

(ii) Max(ii) Maxyy x* x*TTByBySubject to: dSubject to: dTTy=1y=1

y>=0 y>=0

Where e and d are the appropriate Where e and d are the appropriate vectors of all ‘1’svectors of all ‘1’s


Now let w* equal x*Now let w* equal x*TTAy*, and let z* equal Ay*, and let z* equal x*x*TTBy* for a specific (x*,y*) solution of (i) By* for a specific (x*,y*) solution of (i)

and (ii). Then we have the following:and (ii). Then we have the following:Equivalence TheoremEquivalence Theorem: A necessary and : A necessary and

sufficient condition that (x*,y*,w*,z*) be a sufficient condition that (x*,y*,w*,z*) be a solution of (i) and (ii) is that it is a solution solution of (i) and (ii) is that it is a solution

of the programming problem:of the programming problem:

(iii) Max(iii) Maxx,y,w,zx,y,w,z {x {xTT(A+B)y-w-z|(x,z) is in S, and (A+B)y-w-z|(x,z) is in S, and (y,w) is in T}(y,w) is in T}


Where:Where:S={(x,z)|BS={(x,z)|BTTx-zd<=0, ex-zd<=0, eTTx=1, x>=0}x=1, x>=0}T={(y,w)|Ay-we<=0, dT={(y,w)|Ay-we<=0, dTTy=1, y>=0}y=1, y>=0}

Note that S and T are both convex Note that S and T are both convex polyhedral sets.polyhedral sets.


Now observe that:Now observe that:(iv) x*(iv) x*TT(A+B)y*-w*-z*=0(A+B)y*-w*-z*=0

In fact, by the In fact, by the Equivalence TheoremEquivalence Theorem, , any set of (x*,y*,w*,z*) that satisfy any set of (x*,y*,w*,z*) that satisfy

(iv) with (x*,z*) in S and (y*,w*) in T (iv) with (x*,z*) in S and (y*,w*) in T satisfy (iii).satisfy (iii).


Now we shall define an Now we shall define an extreme extreme equilibrium pointequilibrium point (x*,y*,w*,z*) as a (x*,y*,w*,z*) as a

point that satisfies (iv), and for point that satisfies (iv), and for which (x*,z*) is a vertex of S, and which (x*,z*) is a vertex of S, and

(y*,w*) is a vertex of T.(y*,w*) is a vertex of T.

Observe that by definition, there exist Observe that by definition, there exist only a finite number of extreme only a finite number of extreme

equilibrium points, as S and T only equilibrium points, as S and T only have a finite number of extreme have a finite number of extreme

points.points.


LemmaLemma (Mangasarian): All equilibrium (Mangasarian): All equilibrium points of a bimatrix game may be points of a bimatrix game may be expressed as convex combinations expressed as convex combinations

of some extreme equilibrium points.of some extreme equilibrium points.

Proof: Let (x*,y*,w*,z*) be a solution of Proof: Let (x*,y*,w*,z*) be a solution of (iii). Now if we set y=y*, and (iii). Now if we set y=y*, and

w=w*, then (iii) reduces to a w=w*, then (iii) reduces to a linearlinear programming problem in x* and z*. programming problem in x* and z*. This implies, by the extreme point This implies, by the extreme point characterization of all solutions of characterization of all solutions of an LP, that all solutions (x,y*,w*,z) an LP, that all solutions (x,y*,w*,z) must be convex combinations of must be convex combinations of

some subset U of S.some subset U of S.


Thus each vertex (x,z) of U is a solution Thus each vertex (x,z) of U is a solution of our modified (iii), and so satisfies of our modified (iii), and so satisfies

(iv):(iv):(v) x(v) xTT(A+B)y*-w*-z=0(A+B)y*-w*-z=0

In a similar fashion, we see that (y*,w*) In a similar fashion, we see that (y*,w*) must have been a convex must have been a convex

combination of vertices in V, a combination of vertices in V, a subset of T. So (v) is equal to:subset of T. So (v) is equal to:

(vi) x(vi) xTT(Ay*-w*e)+y*(Ay*-w*e)+y*TT(B(BTTx-zd)=0, for (x,z) x-zd)=0, for (x,z) in Uin U


Now note that x>=0, y>=0, and Ay*-Now note that x>=0, y>=0, and Ay*-w*e<=0 and Bw*e<=0 and BTTx-zd<=0, since x-zd<=0, since

(y*,w*) is in V and (x,z) is in U. So (y*,w*) is in V and (x,z) is in U. So (vi) implies:(vi) implies:

(vii)(vii) y*y*TT(B(BTT-zd)=0 for (x,z) in U-zd)=0 for (x,z) in U


Now note that x>=0, y>=0, and Ay*-Now note that x>=0, y>=0, and Ay*-w*e<=0 and Bw*e<=0 and BTTx-zd<=0, since x-zd<=0, since

(y*,w*) is in V and (x,z) is in U. So (y*,w*) is in V and (x,z) is in U. So (vi) implies:(vi) implies:

(vii)(vii) y*y*TT(B(BTT-zd)=0 for (x,z) in U-zd)=0 for (x,z) in U

Since (y*,w*) is a convex combination Since (y*,w*) is a convex combination of points in V, (vii) implies that:of points in V, (vii) implies that:

(viii)(viii) yyTT(B(BTTx-zd)=0 for (x,z) in U and some x-zd)=0 for (x,z) in U and some (y,w) in V(y,w) in V


Similarly, we have (can show) that:Similarly, we have (can show) that:(ix)(ix) xxTT(Ay-we)=0 for (y,w) in V and some (Ay-we)=0 for (y,w) in V and some

(x,z) in U(x,z) in U

This gives us that:This gives us that:xxTT(Ay-we)+y(Ay-we)+yTT(B(BTTx-zd)=0x-zd)=0

ie: xie: xTT(A+B)y-w-z=0(A+B)y-w-z=0 for some (y,w) in V and some (x,z) in Ufor some (y,w) in V and some (x,z) in U

Which proves the claim.Which proves the claim.

NonInferior NonInferior EquilibriaEquilibria

A set of strategies (x*,y*) in a two A set of strategies (x*,y*) in a two player game is a player game is a noninferior noninferior

equilibrium pointequilibrium point if: if:There does not exist x,y in There does not exist x,y in

SS11XSXS22such that:such that:f(x,y)>f(x*,y*) and g(x,y) f(x,y)>f(x*,y*) and g(x,y)

>=>=g(x*,y*)g(x*,y*)or f(x,y) >= f(x*,y*) and or f(x,y) >= f(x*,y*) and

g(x,y)>g(x*,y*)g(x,y)>g(x*,y*)


An example of noninferior equilibria:An example of noninferior equilibria:Suppose that we are given the following (x,y) Suppose that we are given the following (x,y)

pairs and their associated payoffs:pairs and their associated payoffs:

f(x,y)f(x,y) g(x,y)g(x,y)

(x(x11,y,y11)) 2626 22

(x(x22,y,y22)) 1212 1212

(x(x33,y,y33)) -6-6 1212

(x(x44,y,y44)) 00 00


The noninferior points are (xThe noninferior points are (x11,y,y11) and ) and (x(x22,y,y22))

f(x,y)f(x,y) g(x,y)g(x,y)

(x(x11,y,y11)) 2626 22

(x(x22,y,y22)) 1212 1212

(x(x33,y,y33)) -6-6 1212

(x(x44,y,y44)) 00 00


The noninferior points are (xThe noninferior points are (x11,y,y11) and ) and (x(x22,y,y22))

f(x,y) g(x,y)

(x1,y1) 26 2

(x2,y2) 12 12

(x3,y3) -6 12

(x4,y4) 0 0


For a bimatrix game, a noninferior set For a bimatrix game, a noninferior set of strategies (x*,y*) is a pair that of strategies (x*,y*) is a pair that

satisfies the multiobjective problem:satisfies the multiobjective problem:

(x) Max(x) Maxxx x xTTAy*, MaxAy*, Maxyy x* x*TTByBy

Subject to: xSubject to: x11+…+x+…+xmm=1=1

yy11+…+y+…+ynn=1=1x>=0x>=0y>=0y>=0


Not every Nash equilibrium is Not every Nash equilibrium is noninferior however, for example, noninferior however, for example,

the Prisoners’ Dilemmathe Prisoners’ Dilemma

ConfessConfess Don’t Don’t ConfessConfess

ConfessConfess (-2,-2)(-2,-2) (0,-10)(0,-10)

Don’t Don’t ConfessConfess

(-10,0)(-10,0) (-.5,-.5)(-.5,-.5)


Here the unique NE is (Confess, Confess), but Here the unique NE is (Confess, Confess), but its payoff (-2,-2) is strictly inferior to (-.5,-.5) its payoff (-2,-2) is strictly inferior to (-.5,-.5)

that of (Don’t Confess, Don’t Confess)that of (Don’t Confess, Don’t Confess)

ConfessConfess Don’t Don’t ConfessConfess

ConfessConfess (-2,-2)(-2,-2) (0,-10)(0,-10)

Don’t Don’t ConfessConfess

(-10,0)(-10,0) (-.5,-.5)(-.5,-.5)


The previous example demonstrates that The previous example demonstrates that noninferiority may be a better solution concept noninferiority may be a better solution concept than Nash equilibria, as it is a true maximizer. than Nash equilibria, as it is a true maximizer. The downside is that a noninferior equilibrium The downside is that a noninferior equilibrium

may note be enforceable. In our previous may note be enforceable. In our previous example, both players could benefit by example, both players could benefit by

unilaterally deviating from the noninferior unilaterally deviating from the noninferior equilibrium of (Don’t Confess, Don’t Confess)equilibrium of (Don’t Confess, Don’t Confess)This begs the question, can we perturb the This begs the question, can we perturb the

payoffs in a ‘nice’ manner to impose payoffs in a ‘nice’ manner to impose enforcability?enforcability?

Future DirectionsFuture Directions1) Can we find an appropriate penalty to 1) Can we find an appropriate penalty to

transform the noninferior equilibria of transform the noninferior equilibria of some or all games into enforceable ones?some or all games into enforceable ones?

2) Suppose that a given player knows only one 2) Suppose that a given player knows only one of A or B, is there an evolutionary strategy of A or B, is there an evolutionary strategy that maximizes his expected payoff if the that maximizes his expected payoff if the game is repeated infinitely often? What game is repeated infinitely often? What

about finitely often?about finitely often?

equilibrium concepts in two player games

Documents

nash equilibrium points

pure strategy nash equilibrium

nash equilibriaby

nash equilibriaenforceability

nash equilibrialemma

equilibrium concepts

nash equilibriamaxx

n player game