notes on game theory - personalpages.manchester.ac.uk filenash equilibrium existence i: finite games...

Notes on Game Theory

Alejandro Saporiti

Alejandro Saporiti (Copyright) Game Theory 1 / 60

Strategic-Form GamesReferences:

◮ Fudenberg, D., Tirole, J.:Game Theory. MIT Press (1991), chap. 1 &chap. 12.

◮ Reny, P.: On the existence of pure and mixed strategy Nash equilibria indiscontinuous games.Econometrica67 (5), pp. 1029-1056 (1999).

A game instrategic or normal formhas three elements:

◮ Finite setN = {1, . . . ,N} of players, with generic elementi ∈ N ;

◮ Pure strategy setsSi for each playeri ∈ N , with generic elementsi ;

◮ Thepayoff functionsui(·) for each playeri ∈ N , that givei’s vNMutility ui(s) for eachs∈ S≡

∏i∈N Si .

Usually, the structure of the strategic gameG = (Si ,ui)i∈N is assumed to becommon knowledge.


Nash Equilibrium

A mixed strategyfor playeri is a probability distributionσi overSi .

Playeri’s (expected) payoff to profile(σi , σ−i) ∈ Σ ≡∏N

i=1 Σi is

Ui(σi , σ−i) =∑

s∈S

N∏

j=1

σj(sj)

ui(s). (1)

Definition 1 (NE)A mixed strategy profileσ∗ = (σ∗

i , σ∗−i) ∈ Σ is aNash Equilibriumfor

G = (Si ,ui)i∈N if for all player i ∈ N ,

Ui(σ∗i , σ

∗−i) ≥ Ui(si , σ

∗−i) for all si ∈ Si . (2)

A pure strategy Nash equilibriumis a profile of pure strategies that satisfiesthe same conditions (recall pure strategies are degenerated mixed strategies).


Nash Equilibrium Existence I: Finite games

Theorem 1 (Nash 1950)Everyfinite strategy-form game G= (Si ,ui)i∈N has a mixed strategyequilibrium.

The idea of the proof is to applyKakutani’s fixed point theorem(FPT) the theplayers’ reaction correspondences.

Playeri’s reaction correspondence,r i , maps each strategy profileσ to the setof mixed strategies that maximize playeri’s conditional payoffUi(·, σ−i).

Let r : Σ ⇉ Σ be the Cartesian product of ther i , such thatr(σ) =∏N

i=1 r i(σ)for all σ ∈ Σ.

A fixed point ofr(·) is a profileσ ∈ Σ such thatσ ∈ r(σ) (alternatively, it is aprofileσ ∈ Σ such thatσi ∈ r i(σi , σ−i) for all i ∈ N ).



From Kakutani’s FPT,r : Σ ⇉ Σ has a fixed point if:

1. Σ is a compact, convex, and nonempty subset of a (finite-dimensional)Euclidean space;

2. r(σ) is nonempty for allσ ∈ Σ;

3. r(σ) is convex for allσ ∈ Σ;

4. r(·) has a closed graph, in the sense that for all convergent sequences(σn, σn) → (σ, σ) with σn ∈ r(σn), it follows thatσ ∈ r(σ).

N.B. Property (4) is usually referred to as u.h.c.:r(·) is u.h.c. if for anyσ∗and for any open setV ⊇ r(σ∗), there existsδ > 0 such that for allσ ∈ Bδ(σ

∗) we have thatr(σ) ⊆ V.

N.B. Recall that a subsetX of a Euclidean space is compact if any sequence inX has a subsequence that converges to a limit point inX.


Nash Equilibrium Existence I: Finite gamesCond. (1)is easy, since eachΣi is a simplex of dimension|Si | − 1.

For cond. (2), note that each conditional payoffUi(·, σ−i) is linear (andtherefore continuous) inσi; and since continuous functions on a compact setattain a maxima,r i(·) is nonempty and cond. (2) is satisfied.

For cond. (3), fix anyσ ∈ Σ, and consider any two profilesσ′, σ′′ ∈ r(σ) andany scalarα ∈ (0,1). We wish to show thatασ′ + (1− α)σ′′ ∈ r(σ).

By definition, for each playeri,

Ui(ασ′i + (1− α)σ′′

i , σ−i) =∑

s∈S

[ασ′

i (si) + (1− α)σ′′i (si)

]∏

j6=i

σj(sj)

ui(s)

= α∑

s∈S

σ′i (si)

∏

j6=i

σj(sj)

ui(s)

︸︷︷︸=Ui(σ′

i ,σ−i)

+(1− α)∑

s∈S

σ′′i (si)

∏

j6=i

σj(sj)

ui(s)

︸︷︷︸=Ui(σ′′

i ,σ−i)

.



Therefore,Ui(ασ′i + (1− α)σ′′

i , σ−i) = αUi(σ′i , σ−i) + (1− α)Ui(σ

′′i , σ−i).

Moreover, sinceσ′i , σ

′′i ∈ r i(σ), it must be thatUi(σ

′i , σ−i) = Ui(σ

′′i , σ−i).

Hence,ασ′i + (1− α)σ′′

i ∈ r i(σ) as well, which provides the desired resultsince this holds for alli ∈ N .

Finally, for cond. (4), suppose by contradiction that there exists a sequence(σn, σn) → (σ, σ) with σn ∈ r(σn), but σ 6∈ r(σ). Thus, for some playeri,σi 6∈ r i(σ). That means there must existǫ > 0 and aσ∗

i ∈ Σi such that

Ui(σ∗i , σ−i) > Ui(σi , σ−i) + 3ǫ



⇔ Ui(σ∗i , σ−i)− ǫ > Ui(σi , σ−i) + 2ǫ. (3)

Moreover, sinceUi is continuous and the sequence(σn, σn) converges to(σ, σ), for n large enough

Ui(σ∗i , σ

n−i) > Ui(σ

∗i , σ−i)− ǫ, (4)

andUi(σi , σ−i) + 2ǫ > Ui(σ

ni , σ

n−i) + ǫ. (5)

Combining (3)-(5), we have

Ui(σ∗i , σ

n−i) > Ui(σ

ni , σ

n−i) + ǫ,

implying thatσni 6∈ r i(σ

n), a contradiction.


Nash Equilibrium Existence II: Infinite games

Many economic models have an uncountable number of actions (e.g. Cournot,Bertrand, Hotelling, etc.).

These games with continuum action/strategy spaces are calledinfinite games.

Their study is important not only because they are used to represent manyeconomic situations, but also because when the infinite gamedoes not have aNash equilibrium, the equilibria corresponding to the fine,discrete grids(whose existence is guaranteed by Theorem 1), could be very sensitive towhich finite grid is actually chosen.

If there were equilibria of the finite game insensible to the grid chosen, onecould take a sequence of finer and finer grids converging to thecontinuum,and the limit of a convergent subsequence of the discrete-action-spaceequilibria would be a Nash equilibrium of the continuum game.


Nash Equilibrium Existence II: Infinite gamesConsider a strategic gameG = (Si ,ui)i∈N whose strategy spacesSi arenonempty, compact, and convex subsets ofR

m (mfinite).

Theorem 2 (Debreu 1952-Glicksberg 1952- Fan 1952)If for all i ∈ N , the payoff function ui(·) is continuousin s andquasi-concavein si , then G has a pure strategy equilibrium.

The proof is similar to that of Nash’s theorem.

◮ Continuous payoffs imply nonempty and closed-graph reactioncorrespondences;

◮ Quasi-concavity in players’ own actions implies convex valued reactioncorrespondences.

When payoffs are continuous but they are not quasi-concave,mixed strategiescan be used to ensure convex valued reaction correspondences.


Nash Equilibrium Existence II: Infinite gamesConsider a strategic gameG = (Si ,ui)i∈N whose strategy spacesSi arenonempty and compact subsets of a metric space.

Theorem 3 (Glicksberg 1952)If for all i ∈ N , the payoff functions ui(·) is continuous, then G has a mixedstrategy equilibrium.

Here the mixed strategies are (Borel) probability measuresover the purestrategies, which we endow with the topology of weak convergence.

N.B. A sequence of probability measuresµn on a metric spaceA convergesweakly to a limitµ if

∫fdµn →

∫fdµ for every real valued and continuous

function f on A.

N.B. The set of probability measures with the topology of weak convergenceis a compact set.



The proof of Theorem 3 applies again a fixed point theorem to the reactioncorrespondences.

The introduction of mixed strategies makes:

◮ The strategy sets convex;

◮ The payoffs linear in own strategy and continuous in all strategies (due toweak convergence and continuity ofui);

◮ The reaction correspondences convex valued.

With infinite many pure strategies, the space of mixed strategies is infinitedimensional, so a more powerful FPT than Kakutani’s is required.

Alternatively, one can approximate the strategy spaces by asequence of finitegrids, each of which has a mixed strategy equilibrium by Theorem 1.



Since the space of probability measures is weakly compact, the sequence ofthe discrete MSE han an accumulation point.

Finally, the limit point is a MSE of the original game due to the continuity ofthe expected payoffs in the space of probability measures.

When the payoff functionsui are discontinuous pure as well as mixed strategyequilibria can easily fail to exist.

In particular, Glicksberg’s theorem can’t be used because the discontinuity ofthe pure-strategy payoffs implies that the expected payoffs are alsodiscontinuous in the space of probability measures.

Thus, best responses may fail to exist for some of the opponents’ strategies.


Application: Election with Discontinuous PayoffsA number of games in the economic literature havediscontinuous and/ornon-quasi-concave payoff functions. Here is an example.

Consider two political candidates/parties,i = L,R, which compete in awinner-take-all election by simultaneously announcing a policy platformxi ∈ X = [0,1].

The electorate is made of a continuum of voters.

Each voter has a preferred policyθ ∈ X and a utility functionuθ(x) = −|x− θ|, where| · | denotes the absolute value onR.

Voters vote sincerely for the platform they like the most.

Candidatei wins the election if its platformxi gets more than half of the votes,with ties broken by a fair coin toss.

Apart from the uncertainty due to the possibility of a tie, candidates haveuncertainty about voters’ preferences.


Application: Election with Discontinuous PayoffsWe assume that the median voter’s ideal point, denoted byθm, is uniformlydistributed over[1/2− β,1/2+ β], with β > 0.

Thus, the probability that candidateL attaches to winning the election is:

◮ p(xL, xR) = Prob(θm ∈

[0, xL+xR

2

])if xL ≤ xR, and

◮ p(xL, xR) = Prob(θm ∈

[xL+xR

2 ,1])

if xL > xR.

CandidateR’s probability of winning is 1− p(xL, xR).

0 1

1

xL

xR

xR

=1−

xL−

2β

xR

=1−

xL

+2β

xR=

xL

p(xL, xR) = 1

p(xL, xR) = 0

p(xL, xR) = 1

2+ 1−xL−xR

4β

p(xL, xR) = 1/2

p(xL, xR) = 1

2−

1−xL−xR

4β

β→

0

β→

0

β→

0.5

β→

0.5

xR

=−

xL

xR

=2−

xL


Application: Election with Discontinuous PayoffsCandidates possessmixedor hybrid motives for running for office:

◮ They value winning the election; but

◮ They care about policy too.

Formally, the payoffs for any profile(xL, xR) ∈ X2 are:

ΠL(xL, xR) = p(xL, xR) · (uθL(xL) + χL) + [1− p(xL, xR)] · uθL(xR),

and

ΠR(xL, xR) = [1− p(xL, xR)] · (uθR(xR) + χR) + p(xL, xR) · uθR(xL),

whereθi stands for candidatei’s ideological (preferred) position onX, andχi > 0 denotes its payoff from being in power (office rents).

We assume thatθL < 1/2 < θR, andβ < min{

12 − θL +

χL2 , θR − 1

2 + χR2

}.


Application: Election with Discontinuous PayoffsUnfortunately, these payoff functions are not quasi-concave.

For instance, whenχR = 0.2,χL = 0.6, β = 0.25,θL = 0.2, andθR = 0.9,the conditional payoff functions look like this:

PLH × , xR* L

xL*0.2 0.4 0.6 0.8 1.0

-0.4

-0.3

-0.2

-0.1

Figure 1: PartyL’s conditionalpayoff function givenx∗R = 0.65.

PR HxL* , × L

xR*

0.2 0.4 0.6 0.8 1.0

-0.35

-0.30

-0.25

Figure 2: PartyR’s conditionalpayoff function givenx∗L = 0.55.

For these values of the parametersa PSE doesn’t exist.Alejandro Saporiti (Copyright) Game Theory 17 / 60

Application: Election with Discontinuous Payoffs

Nonetheless, the election game has an equilibrium in mixed strategies.

Let∆ be the space of probability measures on the Borel subsets ofX. Amixed strategy fori is a probability measureµi ∈ ∆, with supportsupp(µi) ≡ {x ∈ X : ∀ǫ > 0, µi((x− ǫ, x+ ǫ) ∩ X) > 0}.

We extend eachΠi to ∆2 by

Ui(µL, µR) =

∫

X2Πi(xL, xR)d(µL(xL)× µR(xR)).

Note thatUi is well defined because the set of discontinuities ofΠi , namely{(xL, xR) ∈ X2 : xL = xR 6= 1/2}, has measure zero.

The support of the mixed strategy equilibrium when the right-wing candidateis relatively more ideological is characterized in the nextproposition.


Application: Election with Discontinuous PayoffsIf χR/2 < β < βC

1 , the election gameG = (X,Πi)i=L,R has a mixed strategyequilibrium (µ∗

L, µ∗R) ∈ ∆2 with the property that,

(a) If β ≤ χL+χR4 , thensupp(µ∗

i ) = [x, x] for all i = L,R, with x = xL(β, χR)andx = 1

2 + β − χR2 = x∗R; and

(b) If β > χL+χR4 , thensupp(µ∗

L) = [x, x] andsupp(µ∗R) = [x, x] ∪ {x∗R}, with

x = xL(β, χR) andx = 12 − β + χL

2 = x∗L.

0

0.5

χR

2

Conver

gen

ce

One

sided

diff

.

Two sided

differentiation

χL+χR

4

χL

2ββC

1

x∗

i

x∗

R= 0.5

+ β −

χR/2

x ∗

L = 0.5− β + χ

L/2

xL(β, χR)

Probabilistic diff.

◮ βC1 ≡ χL−χR

4 +√χLχR

2 ;

◮ xL(β, χR) is the solution to

ΠR(x′L, x

∗R)− lim sup

xR→−x′L

ΠR(x′L, xR) = 0.


Nash Equilibrium Existence III: Discontinuous PayoffsSupposeeach strategy setSi is a nonempty, compact and convex subset ofsome finite dimensional Euclidean space.

With discontinuous payoffs, a compact strategy space no longer ensures that aplayer’s optimal response to his opponents’ strategies exists.

To guarantee existence, we assume that the payoffs areupper semi-continuous(u.s.c.). In words, a function is u.s.c. if itdoesn’t jump down.

Definition 2 (u.s.c.)A payoff functionui(·) on S=

∏Ni=1 Si is upper semi-continuous ats if for

any sequencesn converging tos, lim supn→∞ ui(sn) ≤ ui(s).

N.B. The limit superior (limit sup) of a sequencexn in R is the smallestx suchthat for allǫ > 0 there exists an integerN such thatxn ≤ x+ ǫ for all n > N.

Note that the conditional payoffs of Figs. 1 and 2 fail to be u.s.c. at the pointof discontinuity.


Nash Equilibrium Existence III: Discontinuous PayoffsDefine the set ofplayeri’s optimal pure reactionsto pure strategiess−i as

r∗i (s−i) =

{si ∈ Si : ui(si , s−i) ∈ max

si∈Si

ui(si , s−i)

}. (6)

Note thatr∗i differs from the reaction correspondencer i in that for a givens−i ,r i is the convex hull of the points inr∗i (i.e., includes mixed best responses).

If Si is compact andui(·, s−i) is u.s.c., thenr∗i (s−i) is nonempty.

Indeed, take a sequence{sni } ⊂ Si , such that lim

n→∞ui(sn

i , s−i) = supsi∈Si

ui(si , s−i)

(which exists due to u.s.c.). SinceSi is compact,{sni } has a convergent

subsequence, with limitsi ∈ Si say. By u.s.c.,

ui(si , s−i) ≥ lim supn→∞

ui(sni , s−i) ≥ lim

n→∞ui(s

ni , s−i) = sup

si∈Si

ui(si , s−i).

Thus,ui(si , s−i) ≥ supsi∈Siui(si , s−i), which implies that a maximand exists.


Nash Equilibrium Existence III: Discontinuous PayoffsLet r∗ : S⇉ Sbe the pure strategy reaction correspondence. Formally,r∗ isthe Cartesian product of ther∗i , such thatr∗(s) =

∏Ni=1 r∗i (s) for all s∈ S.

To prove thatr∗ has a fixed point we use Kakutani’s theorem.

Nonemptyness ofr∗(s) is ensured by u.s.c.

To guarantee thatr∗(s) is convex valuedfor all s∈ S, we assume that for alli ∈ N ands−i ∈ S−i , the conditional payoffui(·, s−i) is quasi-concave insi .

Finally, the following condition together with u.s.c. ensures thatr∗ has aclosed graph.

Definition 3A function ui(·) has acontinuous maximumif u∗i (s−i) = max

si∈Si

ui(si , s−i) is

continuous ins−i.


Nash Equilibrium Existence III: Discontinuous PayoffsA continuous maximum and u.s.c. imply thatr∗i has a closed graph. Supposenot. Then, there must exist a sequencesn → s, with sn

i ∈ r∗i (sn−i) and

si 6∈ r∗i (s−i). Sincesi is not a best response tos−i , we have

maxs′i ∈Si

ui(s′i , s−i) > ui(si , s−i). (7)

By upper semi-continuity,

ui(si , s−i) ≥ lim supn→∞

ui(sni , s−i) = lim sup

n→∞ui(s

ni , s

n−i). (8)

Moreover, since by hypothesissni ∈ r∗i (s

n−i),

lim supn→∞

ui(sni , s

n−i) = lim sup

n→∞

(max

s′iui(s

′i , s

n−i)

). (9)

Thus, combining (7)-(9), we get maxs′i∈Si

ui(s′i , s−i) > lim supn→∞

(maxs′i

ui(s′i , sn−i))

,

contradicting the assumption of a continuous maximum.Alejandro Saporiti (Copyright) Game Theory 23 / 60

Nash Equilibrium Existence III: Discontinuous PayoffsConsider a strategic gameG = (Si ,ui)i∈N whose strategy spacesSi arenonempty, compact, and convex subsets ofR

m (mfinite).

Theorem 4 (Dasgupta-Maskin 1986)If for all i ∈ N , the payoff function ui(·) is u.s.c. in s, is quasi-concave in si ,and has a continuous maximum, then G has a pure strategy equilibrium.

If either u.s.c. or quasi-concavity do not hold, one way to gois to explore theexistence of mixed equilibria.

Consider a sequence of finite approximationsSni of Si converging toSi for all i.

By Theorem 1, eachdiscretized gameGn = (Sni ,ui) has a mixed strategy

equilibriumσn; that is,

Ui(σni , σ

n−i) ≥ Ui(si , σ

n−i) for all si ∈ Sn

i , and alli ∈ N . (10)


Nash Equilibrium Existence III: Discontinuous PayoffsSince the space of probability measures onSi is compact under the topologyof weak convergence, a subsequence ofσn converges to some MSEσ∗.

If either (i) ui(·) is continuous, or (ii)σn assigns vanishingly small probabilityon the discontinuity pointsof the payoff functions, we would have that

Ui(σni , σ

n−i) → Ui(σ

∗i , σ

∗−i)

andUi(si , σ

n−i) → Ui(si , σ

∗−i).

Thus, (10) would imply thatUi(σ∗i , σ

∗−i) ≥ Ui(si , σ

∗−i) for all si ∈ Si , and all

i ∈ N . That is,σ∗ would be a MSE of the limit gameG = (Si ,ui)i∈N .

If neither (i) nor (ii) are satisfied, the challenge is to find conditions thatensure that the discontinuities do not matter in the limit game.

The two conditions needed are(1) u.s.c. of the sum of the payoffs∑

i ui , and(2) weakly lower semi-continuity(w.l.s.c.).


Nash Equilibrium Existence III: Discontinuous PayoffsThe definition of w.l.s.c. requires some notation. LetS∗∗(i) denote the set ofstrategy profiless∈ Ssuch thatui is discontinuous ats; and letS∗∗−i(si) = {s−i ∈ S−i : (si , s−i) ∈ S∗∗(i)}.

In the election game discussed above,S∗∗(i), i = L,R, would be the diagonalof the strategy space, that is,S∗∗(i) = {(xL, xR) ∈ X2 : xL = xR 6= 1/2}.

Suppose the discontinuities occur only on asubset of measure 0in which aplayer’s strategy is “related” to another player’s.

That is, for any two playersi, j ∈ N , assume there exists a finite number offunctionsf d

ij : Si → Sj , whered is an index, that are one-to-one andcontinuous, such that for each playeri ∈ N

S∗∗(i) ⊆ S∗(i) = {s∈ S : ∃j 6= i, ∃d such thatsj = f dij (si)}.

In the election game the function that relates the strategies of both players atthe discontinuities is the identity function.


Nash Equilibrium Existence III: Discontinuous PayoffsDefinition 4 (w.l.s.c.)The payoff functionui(si , s−i) is w.l.s.c. insi if for all si there existsλ ∈ [0,1]such that, for alls−i ∈ S∗∗−i(si),

ui(si , s−i) ≤ λ lim infs′i→−si

ui(s′i , s−i) + (1− λ) lim inf

s′i→+si

ui(s′i , s−i). (11)

The inequality (11) says thatui does not jumps up whens′i tends tosi eitherfrom the left, from the right, or both. Instead, ifui(si , s−i) were greater than“both lim inf”, then there would noλ ∈ [0,1] for which (11) would hold.

Roughly, w.l.s.c. implies that playeri can do as well with strategies nearsi aswith si , even if the rivals’ strategies put weight on the discontinuities ofui .

Theorem 5 (Dasgupta-Maskin 1986)Let Si = [si , si] ⊂ R. Suppose ui is continuous except on S∗∗(i) ⊆ S∗(i), isbounded, and w.l.s.c. in si . If

∑i ui(s) is u.s.c., then the game G= (Si ,ui)i∈N

has a mixed strategy equilibrium.Alejandro Saporiti (Copyright) Game Theory 27 / 60

Nash Equilibrium Existence IV: Discontinuous PayoffsAn strategic gameG = (Si ,ui)i∈N is said to becompactif Si is a nonemptycompact subset of a topological vector space, andui : S→ R is bounded.

In addition,G is said to bequasi-concaveif Si is convex and for everys−i ,ui(·, s−i) is quasi-concave onSi .

Definition 5 (Security)Playeri cansecure a payoffof α ats∈ S if there existssi ∈ Si , such thatui(si , s′−i) ≥ α for all s′−i in some open neighborhood ofs−i .

Thus, a payoff can be secured byi ats if i has a strategy that guarantees atleast that payoff even if the other players deviate slightlyfrom s.

Definition 6 (b.r.s.)A gameG = (Si ,ui)i∈N is better-reply secureif whenever(s∗,u∗) is in theclosure of the graph of its vector payoff function ands∗ is not an equilibrium,some playeri can secure a payoff strictly aboveu∗i at s∗.


Nash Equilibrium Existence IV: Discontinuous PayoffsIn words, a game is b.r.s. if for every non-equilibrium strategyx∗ and everypayoff vector limitu∗ resulting from strategies approachingx∗, some playerihas a strategy yielding a payoff strictly aboveu∗i even if the others deviateslightly from x∗.

Games with continuous payoff functions are better-reply secure, since anybetter reply will secure a payoff strictly above a player’s inferiornon-equilibrium payoff and those generated by nearby strategies.

Theorem 6 (Reny 1999)If G = (Si ,ui)i∈N is compact, quasi-concave, and better-reply secure, then itpossesses a pure strategy Nash equilibrium.

The above result on the existence of pure strategy equilibria generalizes themixed strategy equilibrium existence results of Nash 1950,Glicksberg 1952,and Dasgupta and Maskin 1986.


Nash Equilibrium Existence IV: Discontinuous PayoffsWhile b.r.s. is straightforward to verify, it is sometimes even simpler to verifyother conditions leading to it.

Better-reply security combines and generalizes two conditions,reciprocalupper semi-continuity(r.u.s.c.) andpayoff security(p.s.).

Definition 7 (p.s.)A gameG = (Si ,ui)i∈N is payoff secureif for every s∈ Sand everyǫ > 0,each playeri can secure a payoff ofui(s)− ǫ at s.

A game is p.s. if for every joint strategys, each player has a strategy thatvirtually guarantees the payoff he receives ats, even if the others play slightlydifferently than ats.

It embeds the idea of robustness of one’s payoff to perturbations in the others’strategies (a kind of “continuity” in the others’ actions).


Nash Equilibrium Existence IV: Discontinuous Payoffs

Definition 8 (r.u.s.c.)A gameG = (Si ,ui)i∈N is reciprocally upper semi-continuousif, whenever(s∗,u∗) is in the closure of the graph of its vector payoff function andui(s∗) ≤ u∗i for every playeri, thenui(s∗) = u∗i for everyi.

This condition was introduced by Simon (1987) under the nameof“complementary discontinuities.” It requires that some player’s payoff jumpsup when some other player’s payoff jumps down.

The role played by r.u.s.c. in ensuring existence of equilibrium is similar tothat played by u.s.c. of a function in guaranteeing the existence of a maximumof a real-valued function.

It generalizes the condition introduced by Dasgupta and Maskin (1986) thatthe sum of the players’ payoffs be u.s.c. Since the u.s.c.-sum condition iscardinal in nature, this generalization is a distinct improvement.


Nash Equilibrium Existence IV: Discontinuous PayoffsNone of the three conditions, b.r.s., p.s. and r.u.s.c., is an ordinal property.However, they are all virtually so in the following sense.

If fi : R → R is continuous and strictly increasing for everyi ∈ N ,(Si ,ui)i∈N is b.r.s. (resp., r.u.s.c./p.s.) if and only if(Si , fi ◦ ui)i∈N is.

Proposition 1 (Reny 1999)If G = (Si ,ui)i∈N is r.u.s.c. and p.s., then it is b.r.s.

When one moves to mixed strategies, securing any particularpayoff becomesboth easier and more difficult.

◮ It becomes easier because one can now employ mixed strategies toattempt to secure a payoff and this can increase the payoff one cansecure.

◮ On the other hand, one’s payoff must be secure against perturbationsfrom the larger set of mixed strategies of the others, which can reduce thepayoff one can secure.


Nash Equilibrium Existence IV: Discontinuous Payoffs

As a result, although r.u.s.c. of the mixed extensionG = (Σi,Ui) implies thatof G = (Si ,ui), b.r.s. (resp., p.s.) ofG neither implies nor is implied by b.r.s.(resp., p.s.) ofG.

The following result provides a convenient means for checking r.u.s.c. ofG.

Proposition 2 (Reny 1999)If∑N

i=1 ui(s) is u.s.c. in s on S, the mixed extensionG = (Σi,Ui) is r.u.s.c.

The result follows from the fact that∑N

i=1 Ui(σ) is u.s.c. inσ onΣ.

Theorem 7 (Reny 1999)Suppose that G= (Si ,ui) is a compact (Hausdorff) game. Then G possesses amixed strategy Nash equilibrium if its mixed extension,G = (Σi ,Ui), is b.r.s.Moreover,G is b.r.s. if it is both r.u.s.c. and p.s.


Nash Equilibrium Existence V: Other ApproachesAn alternative approach to the “the topological approach toexistence ofequilibrium” is based upon lattice-theoretical concepts,and at its heart liesTarski’s (1955) fixed point theorem.

Payoffs need not be quasi-concave and continuous. The key property is thatbest-replies are increasing in the opponents’ strategies.

This method typically yields theexistence of pure strategy equilibria.

Now, while lattice-theoretic methods do not require payoffs to be continuous,enough continuity must be assumed in order to guarantee the existence of bestreplies. So, typicallypayoffs are required to be u.s.c. in one’s own strategy, anassumption that fails to hold in virtually all auctions, as well as in the classicgames of Bertrand and Hotelling.

Consequently, most practical applications of lattice-theoretical techniquestend to be confined to continuous games.


Nash Equilibrium Existence V: Other Approaches

A third approach, although still fundamentally topological in nature, has beenintroduced bySimon and Zame (1990).

They modify the vector of payoffs at points of discontinuityso that theycorrespond to points inthe convex hull of limitsof nearby payoffs, whichensures amixed strategy equilibriumof such a suitably modified game.

As an example of their approach, Simon and Zame note that thediscontinuities in Hotelling’s location model arise only when two firmschoose the same location, andconsumers are indifferentbetween them.

Consequently, rather than insist that the two firms split themarket evenly,Simon and Zame suggest that one ought to be content withany division ofconsumersamong the two firms.


Nash Equilibrium Existence V: Other Approaches

Simon and Zame’s result ensures that there is asharing rulespecifying howconsumers are divided among firms when indifference arises such that theresulting game possesses a mixed strategy equilibrium.

While in some settings involving discontinuities this approach is remarkablyhelpful, i.e., Hotelling’s location game., in others it is less so.

For example, in a mechanism design environment where discontinuities aresometimes deliberately introduced auction design, for example., theparticipants must be presented with a game that fully describes the strategiesand payoffs. One cannot leave some of the payoffs unspecified, to somehowbe endogenously determined.

In addition, this method is only useful in establishing the existence of amixed, as opposed to pure, strategy equilibrium.


Dynamic Games

Reference:Acemoglu, D.:Political Economy Lecture Notes, mimeo, Ch. 3.

Consider the following class of (dynamic or stochastic) games:

◮ Finite setN = {1, . . . ,N} of players, with generic elementi ∈ N ;

◮ (Infinite) setK ⊂ Rn of state vectors, with generic elementk ∈ K;

◮ Infinite timehorizont = 1,2, . . . ;

◮ Playeri ∈ N has astrategy setAi(kt) ⊂ Rni at everyt, with generic

elementait ∈ Ai(kt);

◮ Playeri ∈ N has at timet aninstantaneous utilityui(at, kt), whereat ∈ A(kt) ≡

∏i∈N Ai(kt) is an action profile att, andui(·) is assumed to

be continuous, bounded, and time independent;


Dynamic Games

◮ Thelaw of motionof kt is given by a(Markovian) transition processq(kt+1 |at, kt), which denotes the probability density of eachkt+1 givenkt ∈ K andat ∈ At(kt), with

∫ +∞−∞ q(k |at, kt)dk= 1;

N.B. This processq(·) is said to be Markovian because it depends onlyon the current action profileat and the current statekt.

◮ Players observe the realizations of all past actions (perfect observability);

◮ LetHt be the set ofpublic historiesat timet, with generic elementht = (a0, k0, . . . ,at, kt).

◮ A mixed strategyfor i ∈ N at t is a mappingσit : Ht−1 × K → ∆(Ai)

that determines a probability distribution overAi given the entire pasthistoryht−1 and the current statekt;

If for every pair(ht−1, kt) ∈ Ht−1 × K, σit(ht−1, kt) assigns probabilityone to a single actionait ∈ Ai(kt), thenσit is said to be apure strategy;


Dynamic Games

◮ Let Si be playeri’s set of strategy profiles for the whole game, withgeneric elementσi = (σi1, σi2, . . . , σi∞);

◮ Let Si [t] be playeri’s set of continuation strategy profiles after timetinduced by eachσi ∈ Si , with generic elementσi [t] = (σit , . . . , σi∞);

◮ Playeri’s objective at timet is to maximize thediscounted payoff

Uit(σi [t], σ−i [t]) = Et

( ∞∑

s=0

βs ui(at+s, kt+s)

), (12)

whereβ ∈ (0,1) is a common discount factor, andEt(·) is theexpectation operator conditional on the information available att (notindexed byi because ofperfect observability).

◮ Playeri’s best response correspondenceis BRi(σ−i [t] |ht−1, kt) ={σi [t] ∈ Si [t] : σi[t] ∈ arg maxSi [t] Uit(σi [t], σ−i [t]) s.t.σ−i[t] ∈ S−i [t]}.


Subgame Perfect Equilibrium

Definition 9 (SPE)A subgame perfect equilibrium(SPE) is a strategy profileσ∗ = (σ∗

1, . . . , σ∗N) ∈ S≡

∏i∈N Si such thatσi[t] ∈ BRi(σ−i[t] |ht−1, kt) for

all t = 0,1, . . ., all (ht−1, kt) ∈ Ht−1 × K, and alli ∈ N .

A SPE requires strategies to be best responses to each other,at every possiblesubgame following a particular history, for all possible histories of the gamethat lead to that particular history.

Because strategies in a SPE depend on the whole history of thegame, thereare typically many SPE.

This prompted game theorists to focus on subsets of SPE, suchas MarkovPerfect Equilibrium (MPE), hoping to improve the predictions of the models.


Markov StrategiesMPE differs from SPE in only conditioning on thepayoff-relevant stateratherthan on the entire history of play.

Given the Markovian transition function assumed above, in our case thepayoff-relevant state is simplykt ∈ K.

Thus a (mixed)Markovian strategyis a mappingσi : K → ∆(Ai), and wedenote bySi the corresponding set of all those strategies.

◮ Given the stationarity of the per-period payoffui(at, kt), subscriptt hasbeen dropped from the Markovian strategy since it isn’t payoff-relevant.

◮ σi andσi have different dimensionality; the former assigns a probabilitydistribution overAi to each possible state; the second does so for eachsubgame(ht−1, kt) and allt.

◮ A Markovian strategy with the same dimension thanσi can be defined asσ′

i : K ×Ht−1 → ∆(Ai) such thatσ′i (kt,ht−1) = σi(kt) for all

ht−1 ∈ Ht−1, all kt ∈ K, and allt.


Markov Perfect Equilibrium

Definition 10 (MPE)A Markov perfect equilibriumis a profileσ = (σi , σ−i) ∈ S≡

∏i∈N Si of

Markovian strategies such that the extensionσ′ = (σ′i , σ

′−i) of these strategies

satisfy that for alli ∈ N , all (ht−1, kt) ∈ Ht−1 × K, and allt = 0,1, . . .,

σ′i [t] ∈ arg max

Si [t]Uit(σi [t], σ

′−i [t]) s.t. σ′

−i [t] ∈ S′−i [t].

◮ Playeri’s deviations are not required to be Markovian.

◮ A Markovian strategyσi must be a best response toσ−i among allstrategiesσit available at time t.

◮ MPE is a SPE.


Application: Common Pool GamesAn example of dynamic games iscommon pool games, where individualsdecide over time how much to exploit a common resource. This class ofgames is useful to illustrate how MPE can be computed in practice.

◮ Supposeui(at, kt) = log(ait), whereait denotes consumption ofindividual i ∈ N at timet.

◮ Assume society has a common resourcekt which follows thenon-stochastic law of motionkt+1 = αkt −

∑i∈N ait , whereα ≥ 1, k0 is

given, andkt ≥ 0 must be satisfied in every period.◮ α = 1 corresponds to afixed resource game, where the resource is being

run down.◮ α > 1 corresponds to a case wheregrowth in the capital stockis also

allowed.

◮ Thestage gameis as follows: at every date all players simultaneouslyannounceait ≥ 0. If αkt −

∑i∈N ait ≥ 0, then each individual consumes

ait . Otherwise,αkt is equally allocated among the players.


Single Person Decision ProblemLet’s start with thebenchmark casewhere(ait)i∈N is chosen by a benevolentplanner wishing to maximize the total discounted payoffs ofall agents,

Et

(∑

i∈N

∞∑

s=0

βs log(ai, t+s)

).

By concavity, it is clear that this planner will allocateconsumption equallyacross all individualsat every date.

Thus each individual will consumect/N, wherect denotes total consumptionat timet, i.e.,ct =

∑i∈N ait .

This implies that the program of the planner at timet = 0 is:

max(ct)

E0

( ∞∑

t=0

βt [log(ct)− log(N)]

)s.t. kt+1 = αkt − ct, k0 given. (13)

The second term in the objective function can be dropped since it is constant.Alejandro Saporiti (Copyright) Game Theory 44 / 60

Single Person Decision Problem

Since there is no uncertainty, expectation can be dropped too.

Definingsas the savings left for next period, so thatc = αk− s, we can write(13) as adynamic programming recursion

V(k) = maxs≤αk

[log(αk− s) + βV(s)]. (14)

A solution to this problem is a pair of functionsV(k) andh(k) such thats∈ h(k) is optimal.

Standard arguments of dynamic programming imply that

◮ V(k) is uniquely defined, is continuous, concave and also differentiablewhenevers∈ (0, α);

◮ h(k) is single valued.


Single Person Decision ProblemGiven these, whenevers is interior the optimal planh(k) must satisfy thefollowing necessary and sufficient first-order condition:

−1αk− s

+ βV′(s) = 0. (15)

Moreover, sinceV(k) is differentiable, using the envelope condition, we have

V′(k) =α

αk− s+

[−1

αk− s+ βV′(s)

]

︸︷︷︸=0 by (15)

·∂s∂k

=α

αk− s. (16)

Substituting the optimal plans= h(k) and (16) into (15) we have (recallV′(·)is a function of the next period’s initial stock, i.e. the current savingh(k))

1αk− h(k)

−βα

αh(k) − h(h(k))= 0. (17)

which solves forh(·) as the optimal policy function.Alejandro Saporiti (Copyright) Game Theory 46 / 60

Single Person Decision ProblemSince we know thatV(·) andh(·) are uniquely defined, if we can find one pairof functions that satisfy the necessary conditions, we are done.

Thus, using the “guess and verify” method, the solution is

h(k) = βαk; (18)

and the value function of this optimization problem is

V(k) =(1− β) log(1− β) + β log(β) + log(α)

(1− β)2 +log(k)1− β

. (19)

Going back to the original variables (rememberai =∑N

j=1 aj/N and∑j aj ≡ c = αk− s), it follows from (18) that

ait =(1− β)α

N· kt for all t = 0,1,2, . . . (20)


Multi-agent Common Pool Games: MPE

SupposeN > 1; otherwise, we are back to the benchmark case!

The (multi-agent) common pool game has some uninteresting MPEs.

For example,all individuals announcingai0 = αk0 is an MPE, since there areno profitable deviations by any agents.

◮ Since∑

i ai0 = Nαk0 > αk0, in the equilibrium candidate eachindividual gets a shareαk0/N, with instantaneous utility at time 0 equalto log(αk0)− log(N), and log(0) = −∞ for all t > 0.

◮ If N > 2, any unilateral deviation produces no change in the payoffofthe deviating player (same reasoning than above).

◮ If N = 2, the deviating player gets the same payoff if he deviates toanya′i0 > 0, and a lower payoff ifa′i0 = 0 (i mainly looses the instantaneouspayoff at time 0, as the whole pie goes to−i).


Multi-agent Common Pool Games: MPEInstead, we will look for amore interesting, continuous and symmetric MPE.

The symmetry requirement is for simplicity, and implies that all agents willuse the same Markovian strategy. Let that strategy be denoted bya(k).

As before the law of motion is

s= αk−N∑

ℓ=1

aℓ.

Thus individual consumption can be defined asai = αk−∑

j6=i aj − s, with sas the capital stock left for the next period.

Given our restriction to symmetric Markovian strategies (and droppingconditioning oni), this gives

a(k) = αk− (N − 1)a(k) − s (21)


Multi-agent Common Pool Games: MPEGiven this, each individual’s optimization problem can again be writtenrecursively as follows:

V(k) = maxs≤αk−Na(k)

[log(αk− (N − 1)a(k) − s) + βV(s)]. (22)

The solution of (22) is a value functionV(k), a consumption policya(k) and asaving policyh(k), such thats∈ h(k) is best response.

Assuming differentiability (not guaranteed yet becausea(k) is unknown), thefirst-order condition of the maximization problem in (22) is

−1αk− (N − 1)a(k) − s

+ βV′(s) = 0. (23)

Moreover, using the envelope condition, we have

V′(k) =1

αk− (N − 1)a(k) − s·

(α− (N − 1)

∂a(k)∂k

). (24)


Multi-agent Common Pool Games: MPENotice theterm∂a(k)/∂k in the numerator. This is there because individualsrealize that by their own action they will affect the state variable, and byaffecting the state variable, they will influence the consumption decision ofothers. This is where the subtlety of dynamic games come in.

Using the “guess and verify” method, the the equilibrium savings rate in theMPE of the common pool problem is

h(k) =βα

N − β(N − 1)k; (25)

Notice that whenN = 1, this is exactly equal to the optimal value obtainedfrom the single-person decision problem (or the social planner’s problem).Moreover,

∂h(k)∂N

=−(βαk)(1− β)

(N − β(N − 1))2 < 0; (26)


Multi-agent Common Pool Games: MPEThat is, the greater the number of players drawing resourcesfrom thecommon pool, the lower the savings rate of the economy. This captures thewell-knownfree-rider or tragedy of the commonsproblem.

The inability of the players in this game to coordinate theiractions leads totoo much consumption and too little savings.

For example, it is quite possible that

αβ > 1 >βα

N − β(N − 1), (27)

so that, the social planner’s solution would involve growth, while the MPEwould involve the resources shrinking over time.

Finally, it is useful to remember that there are many MPE in this game.Perhaps unsurprisingly, there also exist non-symmetric MPEs; and there mayalso exist discontinuous MPEs that implement the planner’ssolution.


Multi-agent Common Pool Games: SPE

The common pool problem can also be used to illustrate the differencebetween SPE and MPE.

We already saw that there exists an equilibrium in which all individualsreceive negative infinite utility (they all playai0 = αk0 at time 0).

This immediately implies that for any discount factorβ, any allocation can besupported as an SPE. In particular, the social planner’s solution of saving thesociety’s resources at the rateβα is an SPE.

The (grim) strategy profile that would support this SPE is onein whicheverybody follows the social planner’s allocation until one agent deviatesfrom it, and as soon as there is such a deviation, all agents switch todemanding the whole capital stock of the economy.


Some Basic ResultsSuppose that the instantaneous payoff function of each player is uniformlybounded, i.e., there existsBi < ∞ for all i ∈ N such that

supk∈K,a∈A(k)

ui(a, k) < Bi.

Theorem 8 (One-Stage Deviation Principle–SPE)A strategy profileσ∗ ∈ S is a SPE if and only if for all i∈ N , all(ht−1, kt) ∈ Ht−1 × K, all t, and all ait ∈ A(kt),

Uit(ait , σ∗i [t + 1], σ∗

−i [t]) ≤ Uit(σ∗i [t], σ

∗−i [t]).

Theorem 9 (One-Stage Deviation Principle–MPE)A strategy profileσ ∈ S is a MPE if and only if for all i∈ N , all(ht−1, kt) ∈ Ht−1 × K, all t, and all ait ∈ A(kt),

Uit(ait , σ′i [t + 1], σ′

−i [t]) ≤ Uit(σ′i [t], σ

′−i [t]).


Some Basic Results

These theorems basically imply that in dynamic games, we cancheck whethera strategy is a best response to other players’ strategies bylooking atone-stage deviations, keeping the rest of the strategy profile of the deviatingplayer as given.

The basic idea of the proof is as follows. The problem of individual i afterfixing the strategy of the other players is equivalent to a dynamic optimizationproblem. Given the uniform boundedness of instantaneous payoffs andβ < 1,limT→∞

∑Ts=0β

sui(at+s, kt+s) = 0 for all {at+s, kt+s}Ts+0 and allt. Hence, we

can apply the principle of optimality from dynamic programming, to obtain ofthe one-stage deviation principle.

The uniform boundedness assumption can be weakened to require “continuityat infinity,” which essentially means that discounted payoffs converge to zeroalong any history (and this assumption can also be relaxed further).


Some Basic Results

Theorem 10 (Existence MPE)If K and Ai(k) for all i ∈ N and k∈ K are two finite sets, then there alwaysexists a Markov perfect equilibrium.

The proof of 10 is as follows. Consider an extended game in which eachplayer is an element(i, k) ∈ N × K, with a payoff function given by playeri’soriginal payoff function starting in statek, as in (12), and a strategy setAi(k).

The set of playersN × K is finite, and the strategy setAi(k) is also finite. Theset of mixed strategies∆(Ai(k)) for player(i, k) is the simplex overAi(k).Therefore, the standard proof of existence of Nash equilibrium based onKakutani’s fixed point theorem applies and leads to the existence of anequilibrium (σ(i,k))(i,k)∈N×K in this extended game.


Some Basic Results

Now going back to the original game, construct the strategyσi for each playeri ∈ N such thatσi(k) = σ(i,k), i.e., σi : K → ∆(Ai) (in words: playeri ∈ Nplays the original game following the strategies of the players(i, k)k∈K ).

This strategy profileσ = (σi)i∈N is Markovian.

Consider the extension ofσ to σ′ as above, i.e.,σ′i (kt,ht−1) = σi(kt) for all

ht−1 ∈ Ht−1, kt ∈ K, i ∈ N andt.

Then, by construction, givenσ′−i , it is impossible to improve overσ′

i with adeviation at anyk ∈ K; thus Theorem 9 implies thatσ′

i is best response toσ′−i

for all i ∈ N , and thereforeσ′ is a MPE.

Similar existence results can be proved for countably infinite setsK andAi(k),and also for uncountable sets, but in this latter instance, some additionalrequirements are necessary.


Some Basic ResultsTheorem 11 (Markov vs. Subgame Perfect Equilibria)The set of (extended) MPE strategies is a subset of the set of SPE strategies.

This theorem follows immediately by noting that sinceσ is a MPE strategyprofile, the extended strategy profile,σ′, is such thatσ′

i is a best response toσ′−i for all ht−1 ∈ Ht−1, kt ∈ K, i ∈ N andt. Thus it is subgame perfect.

Theorem 12 (Existence SPE)If K and Ai(k) for all i ∈ N and k∈ K are two finite sets, then there alwaysexists a subgame perfect equilibrium.

Theorem 10 shows that a MPE exists and since a MPE is a SPE (Theorem11), the existence of a SPE follows.

WhenK andAi(k) are uncountable sets, existence of pure strategy SPEs canbe guaranteed by imposing compactness and convexity ofK andAi(k) andquasi-concavity ofUit(·) in σi [t] for all i ∈ N (in addition to continuity).


Some Basic ResultsIn the absence of convexity ofK andAi(k) or quasi-concavity ofUit(·), mixedstrategy equilibria can still be guaranteed to exist under some very mildadditional assumptions.

Suppose that thesame stage gameis played an infinite number of times, sothat payoffs are given by

Uit(σi [t], σ−i [t]) = Et

( ∞∑

s=0

βs ui(at+s)

), (28)

which is only different from (12) because there is no conditioning on the statevariablekt+s. Let us refer to the game{ui(a),a ∈ A} as the stage game.

Let mi = mina−i maxai ui(a) be agenti’s minimax payoffin this stage game.

DenoteV ∈ RN the set of feasible per period payoffs for theN players, with

vi being playeri’s payoff (so that the discounted payoff isvi/(1− β)).Alejandro Saporiti (Copyright) Game Theory 59 / 60

Some Basic Results

Theorem 13 (Folk Theorem for Repeated Games)If Ai is compact for all i∈ N , then for any v∈ V such that vi > mi for alli ∈ N , there existsβ ∈ [0,1) such that for allβ > β, v can be supported asthe payoff profile of a SPE.

Theorem 14 (Unique MPE in Repeated Games)If the stage game{ui(a),a ∈ A} has a unique equilibrium a∗, then there existsa unique MPE in which a∗ is played at every date.

In repeated games, there is no state vector; so Markov strategies cannot beconditioned on anything. We can only look at the strategies that are bestresponse in the stage game. If the latter has a unique equilibrium, that mustalso be the equilibrium of the repeated game.


notes on game theory - personalpages.manchester.ac.uk filenash equilibrium existence i: finite games...

Documents