extended proper equilibrium - paul milgrom · 2020. 8. 21. · of the payo s. in this way, extended...

Extended Proper Equilibrium∗

Paul Milgrom† Joshua Mollner‡

August 20, 2020

Abstract

We introduce extended proper equilibrium, which refines proper equilibrium (Myerson, 1978)by adding across-player restrictions on trembles. This refinement coincides with proper equilib-rium in games with two players but adds new restrictions in games with three or more players.One implication of these additional restrictions is that any tremble that is costless in equilibriumis regarded by all as more likely than any costly tremble, even one by a different player. Atleast one extended proper equilibrium exists in every finite game. The refinement can also becharacterized in terms of a symmetric, meta-version of the game in which players originate froma common pool: if these players tremble symmetrically and in the way of proper equilibrium,then the induced play in the original game is an extended proper equilibrium.

Keywords: equilibrium refinement, trembles, extended proper equilibrium, proper equilibrium

1 Introduction

This paper introduces a refinement of proper equilibrium. Intuitively, a proper equilibrium is atrembling-hand perfect equilibrium in which each player is much less likely to make a more costlymistake than a less costly one. Proper equilibrium, however, still permits the possibility that oneplayer has a fundamentally greater propensity to tremble than another, in the sense that each ofthe former’s mistakes (no matter how costly) are more likely than each of the latter’s mistakes(even costless “mistakes”).

To illustrate this possibility, consider the three-player game in Figure 1, in which each playerhas two strategies. The “Geo” player picks the payoff matrix—East or West. For Row, the strategyUp weakly dominates Down: the latter pays either zero or one, while the former always pays one.Similarly, for Column, Left weakly dominates Right. Geo’s decision is the focus of the analysis: Geo’sbest choice depends on what it believes the other two players will do. The undominated equilibriaof this game are those in which Row plays its weakly dominant strategy of Up, Column plays itsweakly dominant strategy of Left, and Geo mixes between East and West with any probabilities.Although all these equilibria are proper, all but (Up,Left,West) embed the idea that Geo believesit is at least as likely that Row will deviate to play Down as that Column will deviate to play Right,

∗For helpful comments, we thank Gabriel Carroll, Piotr Dworczak, Drew Fudenberg, Fuhito Kojima, Kevin Li,Barton Lipman, Ellen Muir, Andrzej Skrzypacz, Jeroen Swinkels, Alex Wolitzky, seminar participants, and anony-mous referees. Milgrom thanks the National Science Foundation for support under grant number 1525730.

†Department of Economics, Stanford University. E-mail: [email protected].‡Department of Managerial Economics and Decision Sciences, Kellogg School of Management, Northwestern Uni-

versity. E-mail: [email protected].

1

Electronic copy available at: https://ssrn.com/abstract=3035565

[email protected]

mailto:[email protected]

despite the fact that deviating from equilibrium to play Down would be a costly mistake, whiledeviating to Right would be costless.1

Figure 1: A three player game†

[West] Left RightUp 1, 1, 0 1, 1, 1

Down 0, 1, 0 1, 0, 0

[East] Left RightUp 1, 1, 0 1, 1, 0

Down 0, 1, 1 1, 0, 0

†Row’s payoffs are listed first. Column’s payoffs are listed second. Geo’s payoffsare listed third.

In contrast, our new refinement, which we call extended proper equilibrium, requires any trem-ble that is costless in equilibrium to be regarded as more likely than any tremble that is costly,with the consequence that (Up,Left,West) is the unique extended proper equilibrium of this game.The definition of extended proper equilibrium incorporates this requirement, along with others, byformalizing the idea that there should be some scaling of the players’ payoffs such that more costlymistakes are less likely, whether made by the same player or by another player. It is the italicizedcondition that represents the extra restriction in this definition compared with proper equilibrium.And because the definition permits flexibility in the chosen scaling, the set of extended proper equi-libria is invariant to positive affine transformations (henceforth, simply “affine transformations”)of the payoff matrix. For the game in Figure 1, extended proper equilibrium requires Geo to treatRow’s costly deviation to Down as less likely than Column’s costless deviation to Right, because,for any scaling, the former deviation has zero cost and the latter has positive cost.

The idea that players should be comparable in their propensities to tremble would be especiallyappropriate if the players were to originate from a common pool of ex ante identical agents. Thisidea can be modeled formally using the same approach that has been taken in biological game theoryto generalize the concept of evolutionary stability to asymmetric games. In that literature, givenany asymmetric game Γ, one constructs a meta-game Γ in which Nature moves first, randomizingthe roles of the participants. Because each player might be cast into any of the roles, a pure strategyin the meta-game corresponds to a pure strategy profile in the original game.

We define a symmetrically proper strategy of Γ to be the limit of a sequence of strategies σε

such that each profile (σε, . . . , σε) is an ε-proper equilibrium of Γ. Every symmetrically properstrategy induces some strategy profile for the original game.2 We show that a strategy profile of Γis an extended proper equilibrium if and only if it is induced by a symmetrically proper strategy ofa game Γ′, the meta-version of some game Γ′ that is equivalent to Γ up to an affine transformationof the payoffs. In this way, extended proper equilibrium can be characterized as the implication ofplay according to symmetrically proper strategies in associated meta-games.

One interpretation of this result is that some proper equilibria innately depend upon play-

1For example, to see that (Up,Left,East) is a proper equilibrium, define

σtrow =

(t

t+ 1,

1

t+ 1

), σtcol =

(t2

t2 + 1,

1

t2 + 1

), σkgeo =

(1

t+ 1,

t

t+ 1

).

For all t ∈ N, (σtrow, σtcol, σ

tgeo) is a 1

t-proper equilibrium. Taking the limit as t → ∞ establishes that (Up,Left,East)

is a proper equilibrium.2A mixed strategy σε for the meta-game Γ is a distribution over the pure strategy profiles of the original game Γ.

If σε happens to be a product distribution, then its factors constitute a mixed strategy profile of the original game,which we call the induced strategy profile. Generalizing beyond the product case, the induced strategy profile for theoriginal game is the profile of the component-wise marginal distributions of σε (cf. Definition 5).

2


ers being fundamentally asymmetric in their propensities to tremble. We also show that perfectequilibrium and Nash equilibrium can be analogously characterized, respectively, in terms of sym-metrically perfect strategies and symmetrically Nash strategies of these same meta-games. In thatsense, both perfection and Nash are consistent with across-player symmetry in the propensity totremble. It is only for properness that requiring such symmetry can lead to further restrictions,and when symmetry is required in the way that we model, extended proper equilibrium is whatemerges.

In Section 3, we state a formal definition of extended proper equilibrium. We also show thatfor finite games, such equilibria always exist and are always proper equilibria. While the aboveexample illustrates that proper equilibria may fail to be extended proper equilibria in games withthree or more players, we establish that the concepts coincide when there are precisely two players.

In Section 4, we present our results on meta-games. In Section 5, we characterize extendedproper equilibrium in terms of lexicographic probability systems (as in Blume, Brandenburger andDekel, 1991), which can be interpreted as capturing beliefs about how the game will be played.This characterization is useful in establishing one of the main results described above and may alsobe of independent interest. Finally, in Section 6, we analyze an application for our new refinement:the generalized second-price auction, which has been prominently used for internet advertising.Section 7 concludes.

2 Related Refinements

Our new refinement is closely related to proper equilibrium (Myerson, 1978): our definition notonly refines proper equilibrium but also connects to it through the meta-game model that wedescribe. Despite its various critiques,3 proper equilibrium remains a prominent refinement, hasproven its theoretical worth in many applications, and is perhaps the most classic refinement torestrict the relative likelihood of different trembles. However, its restrictions apply only whencomparing trembles of the same player. In contrast, we are motivated to seek a refinement basedon across-player tremble restrictions. In determining how to formulate such restrictions, it seemsnatural to use proper equilibrium as a starting point and exemplar. We propose a refinement thatentails not only the within-player restrictions already encoded in proper equilibrium but also novelacross-player restrictions that are in the same spirit.

A handful of other refinements of proper equilibrium have also been proposed elsewhere in theliterature. Finely settled equilibrium (Myerson and Weibull, 2015) refines proper equilibrium byrequiring that the support of the equilibrium be contained in a block (i.e., a set of strategies foreach player) that is minimal with respect to a certain property. Their fully settled equilibriumis a further refinement that adds a similar minimality requirement. Fall-back proper equilibrium(Kleppe, Borm and Hendrickx, 2017) is defined by considering a class of games in which eachaction of a player n is blocked by Nature with some independent probability δn. The limit pointsof the projections of the Nash equilibria of these blocking games are the fall-back proper equilibria,which form a subset of the proper equilibria. Strictly proper equilibrium (van Damme, 1991) refinesstrictly perfect equilibrium (Okada, 1981) by adding a certain continuity assumption; it also refinesproper equilibrium. Truly proper equilibrium and more-than-proper equilibrium (Neary, 2010) alsorefine proper equilibrium, yet are not as demanding as strictly proper equilibrium. Strongly properequilibrium (Garcıa Jurado and Prada Sanchez, 1990) refines proper equilibrium by adding the

3Notably, van Damme (1991) has shown that it cannot be justified by a micro-foundation in which players paycontrol costs to reduce their tremble probabilities. However, in subsequent work, Myerson and Weibull (2015) dosuccessfully provide a micro-foundation for proper equilibrium, based on consideration sets. (See their Remark 4.)

3


requirement that if two strategies are payoff-equivalent for a player, then they must be trembled towith the same probability. Persistent equilibrium (Kalai and Samet, 1984) requires a form of localstability. Although persistent equilibrium is not itself a refinement of proper equilibrium, everygame contains an equilibrium that is both persistent and proper. Similarly, every fully stable set(Kohlberg and Mertens, 1986) contains a proper equilibrium.

Another related refinement is test-set equilibrium (Milgrom and Mollner, 2018), which alsoincorporates the idea that a tremble by one player to a strategy that is a best response to equilibriumplay must be more likely than any costly tremble. However, test-set equilibrium incorporates anadditional idea: any single tremble to a best response is more likely than any joint tremble by twoor more players. For that reason, test-set equilibrium can be more demanding than extended properequilibrium—and in fact can be so demanding that its existence is not guaranteed in finite games,whereas every such game contains at least one extended proper equilibrium. Another differenceis that test-set equilibrium does not incorporate the idea that the trembles of different playersare statistically independent. For that reason, test-set equilibrium can be less demanding thanextended proper equilibrium (and similarly, even perfect and proper equilibrium), which do requiresuch independence.4

Finally, several other solution concepts can be thought of as imposing across-player restrictionson the likelihood of deviations akin to those contemplated by extended proper equilibrium andtest-set equilibrium. Many of the refinements for signaling games effectively impose across-type re-strictions on the sender so as to discipline the beliefs of the receiver following an off-path message.Examples include the intuitive criterion (Cho and Kreps, 1987), divine equilibrium (Banks andSobel, 1987), and the compatibility criterion (Fudenberg and He, 2018). Closely related to the lastof these is player-compatible equilibrium (Fudenberg and He, 2020), which extends the compatibil-ity criterion and is applicable in general finite normal-form games. Player-compatible equilibriumrefines trembling-hand perfect equilibrium by imposing across-player restrictions on tremble prob-abilities, but it does not refine proper equilibrium. Quantal response equilibrium (McKelvey andPalfrey, 1995) refers to a class of solution concepts that is parametrized by the profile of quan-tal response functions, which govern for each player how a strategy’s expected payoff maps intoits probability of being played. Nevertheless, the most commonly-used specification is the one inwhich each player has the same logistic quantal response function, implying a form of across-playersymmetry in the propensity to err. Likewise, many learning or evolutionary models (e.g., fictitiousplay, replicator dynamics) are typically specified in ways that imply across-player symmetry in therate of learning or evolution.

3 Extended Proper Equilibrium

3.1 Notation

A game in normal form is denoted Γ = (N , S, π), where N = {1, . . . , N} is a set of players,S = (Sn)n∈N is a profile of pure strategy sets, and π = (πn)n∈N is a profile of payoff functions.Throughout, we restrict attention to finite games, in which for all players n ∈ N , Sn is a finite set.We use S to denote

∏n∈N Sn, the set of pure strategy profiles.

We use ∆n to denote the set of mixed strategies of player n and ∆0n to denote its relative

interior, which is the set of totally mixed strategies of player n. We embed Sn in ∆n and extend

4To elaborate, any perfect equilibrium is a mixed strategy profile (so that the mixing of different players isstatistically independent), and the same is true for the associated ε-perfect equilibria. A test-set equilibrium is also astrategy profile, but the implicitly associated trembles are allowed to be certain correlated strategy profiles. (Test-setequilibrium is not defined in terms of trembles, but it can be thought of in those terms—see Appendix C.1 for details.)

4


the utility functions πn to the domain∏n∈N ∆n in the usual way. A mixed strategy profile is

denoted σ = (σ1, . . . , σN ).We use BRn(σ) for the set of player n’s best responses to σ. We use σ/σ′n for the strategy

profile constructed from σ by replacing player n’s strategy with σ′n. We also define Ln(σ) as theexpected loss for player n from playing σn instead of a best response when others play accordingto some mixed strategy profile σ:

Ln(σ) = maxsn∈Sn

πn(σ/sn)− πn(σ).

This quantity is zero when σn is a best response to σ and positive otherwise.

3.2 Definition

Given a scaling vector α ∈ RN++, an (α, ε)-extended proper equilibrium is a profile of totally mixedstrategies in which the following property holds: if a pure strategy s′l has scaled loss αlLl(σ/s

′l) ex-

ceeding that of another strategy s′′m of the same or another player, then s′l is played with probabilityat most ε times that of s′′m.

Definition 1. Let α ∈ RN++ and ε > 0. An (α, ε)-extended proper equilibrium is a profile oftotally mixed strategies σ ∈

∏n∈N ∆0

n such that for all l,m ∈ N , all s′l ∈ Sl, and all s′′m ∈ Sm, ifαlLl(σ/s

′l) > αmLm(σ/s′′m), then σl(s

′l) ≤ ε · σm(s′′m).

This definition is stronger than that of ε-proper equilibrium (Myerson, 1978). Indeed, ε-properequilibrium is equivalent to the version of this definition in which—instead of quantifying over allpairs of pure strategies s′l and s′′m—the quantification is over only all pairs for the same player.5

An extended proper equilibrium is a strategy profile that, for some scaling vector α, is a limitof (α, ε)-extended proper equilibria, as ε approaches zero.

Definition 2. A strategy profile σ ∈∏n∈N ∆n is an extended proper equilibrium if and only if

there exist α ∈ RN++ and sequences (εt)∞t=1 and (σt)∞t=1 such that:

(i) each εt > 0 and limt→∞ εt = 0,

(ii) each σt is an (α, εt)-extended proper equilibrium, and

(iii) limt→∞ σt = σ.

One restriction implied by extended proper equilibrium is that any tremble that is costless inequilibrium must be much more likely than any costly tremble, even one by a different player.6 Nosuch restriction—nor in fact any across-player restriction—is implied by proper equilibrium alone.

The scaling vector α used in the definitions above can play two roles. In traditional non-cooperative game theory, payoffs are expressed in terms of von Neumann-Morgenstern utilities,

5For purposes of comparison, we reiterate the following definitions from Myerson (1978). An ε-proper equilibriumis a profile of totally mixed strategies σ ∈

∏n∈N ∆0

n such that (∀n ∈ N )(∀s′n ∈ Sn)(∀s′′n ∈ Sn) : πn(σ/s′n) <πn(σ/s′′n) =⇒ σn(s′n) ≤ ε·σn(s′′n). Note that for any αn > 0, the antecedent inequality is equivalent to αnLn(σ/s′n) >αnLn(σ/s′′n). Similarly, an ε-perfect equilibrium is a profile of totally mixed strategies σ ∈

∏n∈N ∆0

n such that(∀n ∈ N )(∀s′n ∈ Sn)(∀s′′n ∈ Sn) : πn(σ/s′n) < πn(σ/s′′n) =⇒ σn(s′n) ≤ ε. A strategy profile is a proper equilibrium(perfect equilibrium) if and only if it is the limit of ε-proper equilibria (ε-perfect equilibria) as ε → 0, in the samesense as Definition 2.

6To see this, suppose σ is an extended proper equilibrium, so that it is the limit of a sequence (σt)∞t=1 of (α, εt)-extended proper equilibria. Suppose also that there are two players l 6= m, as well as pure strategies s′l 6∈ BRl(σ) ands′′m ∈ BRm(σ), which implies αlLl(σ/s

′l) > αmLm(σ/s′′m). By continuity, we also have αlLl(σ

t/s′l) > αmLm(σt/s′′m)—and therefore σtl (s

′l) ≤ εtσtm(s′′m)—for sufficiently large values of t.

5


which are unique only up to affine transformations. With that interpretation, game-theoreticpredictions should not depend on the particular payoff representation that is chosen. Thus, thefirst role of the scaling vector α in the above definition is to ensure that the set of extended properequilibria is invariant to affine transformations of the utility functions of the players. In somemodels, however, payoffs are understood to be normalized in some way to facilitate interpersonalwelfare comparisons. For example, with quasilinear preferences, payoffs may be normalized relativeto a transferable numeraire good. In those settings, α may instead be interpreted as representingthe propensities to tremble of the various players, and the definition as it is written allows for thepossibility that players may differ somewhat in these propensities—albeit by only a finite multiple.7

3.3 Three Basic Facts About Extended Proper Equilibrium

The first three results record some basic facts about extended proper equilibrium. First, it refinesthe existing tremble-based refinements of proper equilibrium and perfect equilibrium. Second, itsexistence is guaranteed in finite games. Third, it coincides with proper equilibrium in two-playergames.

Proposition 1. For any finite normal-form game, the extended proper equilibria form a subset ofthe proper equilibria, which form a subset of the perfect equilibria, which form a subset of the Nashequilibria. All three inclusions may be strict.

Proof. It is well known that the perfect equilibria form a subset of the Nash equilibria (e.g., Selten,1975). For the two other inclusions, it is immediate from their definitions that any (α, ε)-extendedproper equilibrium is an ε-proper equilibrium, which is an ε-perfect equilibrium. Consequently, anextended proper equilibrium, as the limit of (α, ε)-extended proper equilibria, is also the limit ofε-proper equilibria and therefore also proper. Likewise, a proper equilibrium is also perfect. Thatthe first inclusion may be strict is demonstrated by the game in Figure 1 in the introduction. Thatthe other inclusions may be strict is well known (e.g., Myerson, 1978).

The proposition implies that any extended proper equilibrium inherits the properties that arepossessed by every proper equilibrium, including the celebrated relationship between proper equi-libria of the normal form and quasi-perfect equilibria of the extensive form, as established byvan Damme (1984): an extended proper equilibrium of a normal-form game induces a quasi-perfectequilibrium (van Damme, 1984)—and hence a sequential equilibrium (Kreps and Wilson, 1982)—inevery extensive-form game having this normal form.

Although extended proper equilibrium places restrictions beyond those required by proper equi-librium, the next result states that these additional restrictions are not so strong as to be incom-patible with existence in finite games.

Theorem 2. Every finite normal-form game has at least one extended proper equilibrium.

7An alternative definition would be to require α = 1 in the definition of extended proper equilibrium. Thiscorresponds to the definition of perfectly proper equilibrium (Garcıa Jurado, 1989b). Under this definition, the selectionfails to be invariant to affine transformations of the players’ utility functions. But it may be viable for settings wherepayoffs have already been normalized. That definition is more restrictive than extended proper equilibrium, althoughnot so restrictive as to be incompatible with existence in finite games.

Also related is normalized perfectly proper equilibrium (Garcıa Jurado, 1989a), which is equivalent to requiringαn = [maxs∈S πn(s)−mins∈S πn(s)]−1, and is also more restrictive than extended proper equilibrium. In contrast tohis perfectly proper equilibrium, this definition is invariant to affine transformations of the players’ utility functions;however, it may be quite sensitive to payoffs under strategy profiles that are unrelated to equilibrium play.

6


The proof of Theorem 2 is deferred to Appendix B, as are the proofs of most subsequentresults. The proof is similar to the one used by Myerson (1978) to establish the existence of properequilibrium. In the first step, a fixed point argument is used to establish existence of an (α, ε)-extended proper equilibrium for any α ∈ RN++ and every ε > 0. Thus, for any sequence (εt)

∞t=1 of

positive numbers converging to zero, there is a corresponding sequence of (α, εt)-extended properequilibria. In the second step, we appeal to the compactness of

∏n∈N ∆n to establish existence of

a convergent subsequence, the limit of which is therefore an extended proper equilibrium.Although the extended proper equilibria may form a strict subset of the proper equilibria in

games with three or more players (e.g., the game in Figure 1), the concepts coincide in games withjust two players.

Theorem 3. In two-player games, the sets of proper equilibria and extended proper equilibriacoincide.

The motivation for extended proper equilibrium comes from questions about how one playerought to form beliefs about the likelihood of deviations from different opponents. Extended properequilibrium refines proper equilibrium by imposing additional structure on these across-opponentcomparisons, but one interpretation of Theorem 3 is that extended proper equilibrium does notimpose much more. Indeed, with two players, each player has only a single opponent (hence, noneof these across-opponent comparisons exist to be made), and the result says that extended properequilibrium reduces to proper equilibrium in that case.

Theorem 3 is, however, more subtle than the simple observation that there are no across-opponent comparisons to make in two-player games. Indeed, a given ε-proper equilibrium mightfail to be (α, ε)-extended proper, even with just two players. (We illustrate with an example in thenext section.) To prove the result, we instead show that for any convergent sequence of ε-properequilibria, one can construct a sequence of (α, ε)-extended proper equilibria with the same limit.Our construction relies on lexicographic probability systems, which are discussed in Section 5. Thedetails of the proof therefore wait until then.

3.4 Examples

To illustrate the above definitions and results, we next consider some examples.

Figure 1. We first return to the game in Figure 1 in the introduction, for which (Up,Left,West)is the unique extended proper equilibrium. We now establish this formally. Suppose α ∈ R3

++,(εt)

∞t=1, and (σt)∞t=1 together satisfy the conditions of Definition 2. Because Down and Right are

both weakly dominated strategies, we must have σtrow(Down)→ 0 and σtcol(Right)→ 0. Thus, whenplay is according to σt, Row’s scaled loss from playing Down exceeds Column’s scaled loss fromplaying Right for all sufficiently large t:

αrowLrow(σt/Down) = αrowσtcol(Left)→ αrow

αcolLcol(σt/Right) = αcolσ

tcol(Down)→ 0

Thus, for all such t, (α, εt)-extended proper equilibrium requires σtcol(Down) ≤ εtσtcol(Right). In

particular, for all sufficiently large t, σtcol(Down) < σtcol(Right). Turning our attention now to Geo,East therefore entails greater loss than West when opponents play according to σt. Thus, for allsuch t, (α, εt)-extended proper equilibrium requires σtgeo(East) ≤ εtσ

tgeo(West). In particular, this

implies σtgeo(East)→ 0. In conclusion, (Up,Left,West) must be the limit of (σt)∞t=1.

7


Of the solution concepts surveyed in Section 2, only test-set equilibrium (Milgrom and Mollner,2018) also uniquely selects (Up,Left,West) in this game.8 Indeed, both extended proper equilibriumand test-set equilibrium can be interpreted as imposing across-player restrictions on the likelihoodof trembles.9 But in general, neither concept implies the other, as subsequent examples illustrate:Figure 2(b) is a game in which extended proper equilibrium is more demanding than test-setequilibrium, and Figure 3 is an example of the reverse.

Figure 2. Next, consider the three-player extensive-form game with the centipede-like structuredepicted in Figure 2(a). For each player n ∈ {1, 2, 3}, the continuing strategy Cn is dominatedby the stopping strategy Sn. In consequence, (S1, S2, S3) is the unique perfect, proper, extendedproper, and test-set equilibrium of this game. But what we wish to emphasize with the game inFigure 2(a) is the relative probability of trembles by players 2 and 3.

Earlier discussion has focused on across-player comparisons of tremble probabilities betweenstrategy pairs consisting of a best response to the equilibrium for one player and an inferior responseto the equilibrium for another. This example illustrates that extended proper equilibrium can alsoimply meaningful restrictions even when it is two best responses to the equilibrium that are beingcompared. As Figure 2(a) suggests, such situations can naturally arise in situations based on certainextensive-form games.

Although both C2 and C3 are both best responses to the equilibrium (S1, S2, S3), extendedproper equilibrium requires player 2’s tremble to be less likely than player 3’s. To see this, supposeα ∈ R3

++, (εt)∞t=1, and (σt)∞t=1 together satisfy the conditions of Definition 2. When play is according

to σt, player 2’s scaled loss from playing C2 exceeds player 3’s scaled loss from playing C3 for allsufficiently large t:

α3L3(σt/C3)

α2L2(σt/C2)=α3σ

t1(C1)σt2(C2)

α2σt1(C1)=α3

α2σt2(C2)→ 0

Thus, for all such t, (α, εt)-extended proper equilibrium requires σt2(S2) ≤ εtσt3(S3). In contrast,

proper equilibrium is consistent with either player 2 or 3 being more likely to tremble. Similarly, therestrictions on trembles that are implicit in the definition of test-set equilibrium are also consistentwith either possibility—this is because C2 and C3 are both best responses to the equilibrium(S1, S2, S3).

These differing restrictions on relative tremble probabilities (between proper equilibrium andtest-set equilibrium on one hand and extended proper equilibrium on the other) do not translateinto differing predictions for equilibrium play in the game in Figure 2(a) itself. However, if player 1’spayoffs were instead such that its best strategy depended on the relative probabilities of trembles byplayers 2 and 3, then we would in fact obtain different predictions for equilibrium play. Figure 2(b)depicts such a game. From S1, player 1 obtains a payoff of zero, as before. But from C1, player1’s payoff now depends on the strategies of players 2 and 3. In particular, player 1 obtains a bonusof +1 if player 3 matches by playing C3, but also a penalty of −1 if player 2 matches by playing

8Like proper equilibrium, the following concepts are consistent with any mixture between East and West: finelysettled equilibrium, fully settled equilibrium, fall-back proper equilibrium, truly proper equilibrium, more-than-properequilibrium, strongly proper equilibrium, persistent equilibrium, and player-compatible equilibrium. Both East andWest (but not mixtures between them) are consistent with full stability. Like extended proper equilibrium, limitinglogit quantal response equilibrium is consistent with only West; however, it also requires Column to play a uniformmixture between Left and Right, which is a weakly dominated strategy. Strictly proper equilibrium fails to exist inthis game. Finally, as this game is not equivalent to a signaling game, the intuitive criterion, divine equilibrium, andthe compatibility criterion are not defined.

9See Appendix C for a formal definition of test-set equilibrium, more detailed explanations of what it selects inthe examples discussed in this section, and a further discussion of its relationship with extended proper equilibrium.

8


Figure 2: Two three-player games

(a) An extensive-form game

S1

S2

S3

C3

C2

C1

1

2

3

(0, 0, 0)

(−1, 0, 0)

(−1,−1, 0)

(−1,−1,−1)

(b) A normal-form game

[S1] S3 C3

S2 0, 0, 0 0, 0, 0C2 0, 0, 0 0, 0, 0

[C1] S3 C3

S2 0, 0, 0 1, 0, 0C2 −1,−1, 0 0,−1,−1

C2. Payoffs for players 2 and 3 are as before. For the reasons above, extended proper equilibriumrequires C2 to be less likely than C3, which induces player 1 to select C1 in equilibrium. In contrast,proper equilibrium and test-set equilibrium are both consistent with the opposite: C2 being morelikely than C3, and hence an equilibrium choice of S1. This is therefore a game in which extendedproper equilibrium is stronger than the intersection of proper and test-set equilibrium.

Figure 3. Theorem 2 states that every finite game has at least one extended proper equilibrium,but the same conclusion is not true for test-set equilibrium. Indeed, in Milgrom and Mollner(2018), we present an example of a game—reproduced below as Figure 3—in which no test-setequilibrium exists. (Up,Left,West) is the unique Nash equilibrium of this game, but it is not atest-set equilibrium.

Nevertheless, (Up,Left,West) is an extended proper equilibrium of the game. Indeed, for anyε > 0, the following strategy profile is an (α, ε)-extended proper equilibrium with α = (1, 1, 1) andany sufficiently large choice of t:10

σtrow =

(2t2 + 1

2t2 + 3t+ 1,

3t

2t2 + 3t+ 1

)σtcol =

(2t− 2

2t+ 1,

3

2t2 + 3t+ 1,

3t

2t2 + 3t+ 1

)σtgeo =

(2t− 2

2t+ 1,

3

2t+ 1

)10To see that this profile is indeed an (α, ε)-extended proper equilibrium, we compute the following payoffs, from

which one can easily check that all required restrictions are met:

πrow(σt/Up) = πcol(σt/Left) = πgeo(σt/West) = 0

πrow(σt/Down) = πcol(σt/Right) = πgeo(σt/East) = − 3(t− 1)

(t+ 1)(2t+ 1)2

πcol(σt/Center) = − 3(5t+ 1)

(t+ 1)(2t+ 1)2

9


Moreover, limt→∞ σt = (Up,Left,West).

Note that as t→∞, Column’s tremble to Center becomes infinitely less likely than its trembleto Right, despite the fact that both are best responses to the equilibrium (Up,Left,West). In thatsense, these trembles might be regarded as extreme. Owing to this extreme trembling, the unilateraldeviation to Center is less likely than the joint deviation to (Down,Right), which runs contrary tothe logic embedded in the definition of test-set equilibrium—but which is nevertheless consistentwith extended proper equilibrium.

Figure 3: A three player game†

[West ] Left Center RightUp 0, 0, 0 0, 0, 0 0, 0, 0

Down 0, 0, 0 −1, 1, 0 1,−1, 0

[East ] Left Center RightUp 0, 0, 0 0,−1, 1 0, 1, 0

Down −1, 0, 0 −1,−1, 0 1,−1,−1

†Row’s payoffs are listed first. Column’s payoffs are listed second. Geo’s payoffs are listed third.

Figure 4. Finally, consider the two-player game in Figure 4. For Row, Up strictly dominatesDown. For Column, Left weakly dominates Right. Thus, (Up,Left) is the game’s unique perfectequilibrium. It follows that the sets of proper and extended proper equilibria coincide: both arethe singleton {(Up,Left)}. Although it is therefore clear that the claim of Theorem 3 holds in thecase of this game, it is useful to consider the trembles that justify (Up,Left) as proper and extendedproper.

Figure 4: A two player game†

Left RightUp 1, 1 1, 1

Down 0, 1 0, 0

†Row’s payoffs are listed first. Column’s payoffs are listed second.

In particular, consider the following class of strategy profiles: for t ∈ N, let σtrow specify Downwith probability 1

t (and Up otherwise), and let σtcol similarly specify Right with probability 1t

(and Left otherwise). For any ε > 0, it is straightforward to show that σt = (σtrow, σtcol) is an

ε-proper equilibrium when t is sufficiently large. However, an analogous statement is not true for(α, ε)-extended proper equilibria. Rather, for any α ∈ R2

++ and any ε ∈ (0, 1), σt fails to be an(α, ε)-extended proper equilibrium when t is large. To see this, observe that, when play is accordingto σt, Row’s scaled loss from Down exceeds Column’s scaled loss from Right for all sufficiently larget:

αrowLrow(σt/Down) = αrow

αcolLcol(σt/Right) =

αcol

t→ 0

If σt were an (α, ε)-extended proper equilibrium, then for such t we would have 1t = σtrow(Down) ≤

ε · σtcol(Right) = εt , a contradiction.

The key to Theorem 3 lies in showing that, when there are two players, the trembles σt canbe perturbed so as to satisfy the requirements of extended proper equilibrium. In this case, it isenough to reduce the probability of Row’s trembles: let σtrow specify Down with probability 1

t2(and

10


Up otherwise). This change does not disturb the within-player restrictions that had already beensatisfied: just as σt is an ε-proper equilibrium when t is sufficiently large, so is (σtrow, σ

tcol). More

significantly, the change accommodates the across-player restrictions. Indeed, the following doesnow hold when t is sufficiently large: 1

t2= σtrow(Down) ≤ ε · σtcol(Right) = ε

t .After introducing lexicographic probability systems in Section 5, we sketch the proof of Theo-

rem 3, which entails showing how such perturbations can be constructed in general.

4 Equilibrium Refinement in Meta-Games

For some applications, it may be natural to expect players to be symmetric in their propensities totremble, but the definitions of standard tremble-based refinements do not entail any across-playerrestrictions designed to encode such symmetry. This section proposes a means of encoding symme-try into the definitions of the standard refinements, investigates the implications, and demonstratesa way in which extended proper equilibrium emerges from this analysis.

Although there might be many reasons for expecting players to have similar trembling propen-sities, one could be that they originate from a single pool of ex ante identical agents. This idea canbe modeled by embedding a game Γ into a symmetric meta-game Γ in which Γ is played only afterNature randomly assigns players to roles.

In this section, we consider the implications of the standard tremble-based refinements when twoforms of across-player symmetry are enforced. First, we apply refinements to a class of meta-gamesthat includes the meta-version of Γ itself (i.e., Γ) as well as the meta-versions of games that areequivalent to Γ up to affine transformations of the payoffs. Second, we focus on symmetric versionsof the refinements in which symmetry is required not only of the equilibrium strategy profile butalso of the trembles. The predictions of Nash equilibrium and perfect equilibrium are unaffectedby embedding symmetry in this way: applying Nash equilibrium or this form of perfection to thesemeta-games generates predictions for observed play in Γ that coincide with the predictions derivedby applying Nash equilibrium or perfect equilibrium directly to Γ. However, a corresponding resultis not true for proper equilibrium. Rather, applying this form of properness to these meta-gamesgenerates predictions for observed play in Γ that coincide with the predictions derived by applyingextended proper equilibrium directly to Γ.

4.1 Preliminaries

Given an N -player game Γ, we let Γ represent the interaction among N agents in which they playΓ after being randomly assigned to their roles. Players in Γ can be thought of as choosing theirstrategies for Γ behind a “veil of ignorance,” that is, before they discover their role assignment.Thus, pure strategies in Γ are isomorphic to pure strategy profiles in Γ.11 Likewise, mixed strategiesin Γ are isomorphic to distributions over pure strategy profiles in Γ.

Definition 3. The meta-game associated with a game Γ = (N , S, π) is the symmetric game Γconsisting of the same set of players N , and where each player n ∈ N has strategy set S =

∏n∈N Sn

and payoff function

πn(s1, . . . , sN ) =1

N !

∑τ∈Sym(N )

πτ(n)

(proj1

(sτ−1(1)

), . . . ,projN

(sτ−1(N)

) ).12

11Thus, the elements of S can be interpreted either as a strategy profile in Γ or as a strategy in Γ. We will typicallydenote such an element by s when the former interpretation is intended and by s when the latter interpretation isintended.

12In this sum, Sym(N ) denotes the set of all permutations of N . Under a permutation τ , the role of player m in

11


In the biological game theory literature, this meta-game construction is used to apply the con-cept of an evolutionarily stable strategy—which was originally defined only for symmetric games—to asymmetric games (e.g., Maynard Smith and Parker, 1976; Selten, 1980, 1983; van Damme,1991). Similarly, we exploit the symmetric structure of Γ to apply symmetric versions of the stan-dard tremble-based refinements to that game. Note that the following definitions require symmetrynot only of the equilibrium strategies but also of the trembles (as in e.g., Nachbar, 1990).

Definition 4. In a finite symmetric game Γ,

(i) a strategy σ is a symmetrically Nash strategy if (σ, . . . , σ) is a Nash equilibrium of Γ;

(ii) a strategy σ is a symmetrically perfect strategy if there exists a sequence of positive numbers(εt)

∞t=1 converging to zero and a sequence of strategies (σt)∞t=1 converging to σ such that for

all t, (σt, . . . , σt) is an εt-perfect equilibrium of Γ; and

(iii) a strategy σ is a symmetrically proper strategy if there exists a sequence of positive numbers(εt)

∞t=1 converging to zero and a sequence of strategies (σt)∞t=1 converging to σ such that for

all t, (σt, . . . , σt) is an εt-proper equilibrium of Γ.

4.2 Results

A goal of this section is to apply the concepts of symmetrically Nash, perfect, and proper strategiesto the meta-game Γ and to characterize the implications of such strategies for observed play in Γ.To do so, we observe that every mixed strategy in Γ implies a mixed strategy profile in Γ in thefollowing way.

Recall that every mixed strategy σ in Γ is equivalent to a distribution over pure strategy profilesin Γ. Unless this distribution has a product structure, it may imply some across-role correlation ofbehavior. Nevertheless, because each player assumes only one role at once in Γ, these correlationsare irrelevant to what could be observed as the induced behavior in Γ. Rather, if play in Γ isaccording to the symmetric mixed strategy profile (σ, . . . , σ), then what would be observed as theinduced behavior in Γ is the profile consisting of the component-wise marginals of σ, which wedefine below as the projection of σ.

Definition 5. Given a game Γ and a distribution σ over the pure strategy profiles of Γ, theprojection of σ is the strategy profile σ = (σ1, . . . , σN ) in Γ, where for all n ∈ N , σn is the marginalof σ on Sn.

We can then state our result, one direction of which implies that (i) symmetrically Nash strate-gies of Γ induce Nash equilibrium play in Γ, (ii) symmetrically perfect strategies of Γ induce perfectequilibrium play in Γ, and (iii) symmetrically proper strategies of Γ induce extended proper equilib-rium play in Γ. Furthermore, the same conclusions also hold for symmetrically Nash, perfect, andproper strategies of the meta-versions of games that are equivalent to Γ up to affine transformationsof the payoffs. The other direction of the result establishes a set of converses.

Theorem 4. A strategy profile σ in a finite game Γ is a Nash (or perfect or extended proper,respectively) equilibrium if and only if there exists a distribution σ over the strategy profiles of Γ,which has σ as its projection, and there exists a game Γ′, which is equivalent to Γ up to affine trans-formations of the payoffs, such that σ is a symmetrically Nash (or perfect or proper, respectively)strategy of the meta-game Γ′.

the game Γ is assigned to player τ−1(m) in the meta-game Γ, so the strategy played by role m in Γ is projm(sτ−1(m)).By averaging the payoffs to player n in Γ over all permutations, we obtain the given form for πn.

12


In other words, Nash equilibrium remains the prediction if symmetry is embedded into the Nashequilibrium logic in the way described above. Likewise, perfect equilibrium remains the prediction ifsymmetry is embedded into the perfect equilibrium logic. However, an analogous result is not truefor proper equilibrium. Rather, if players originate from the same population before being cast intothe different roles of a game Γ and if such players tremble symmetrically and in the way of properequilibrium, then some additional restrictions on trembles in Γ are implied. These restrictions areprecisely those implied by extended proper equilibrium when applied directly to Γ.13

The parts of the result pertaining to Nash equilibrium are related to the equivalence between exante and interim Bayesian Nash equilibrium in games of incomplete information, and the intuitionis similar. The parts pertaining to perfect equilibrium are also straightforward to establish. Forthose parts pertaining to extended proper equilibrium, we argue as follows:

Necessity. Suppose σ is a symmetrically proper strategy of Γ. Let (σt, . . . , σt)∞t=1 be an associ-ated sequence of εt-proper equilibria of Γ. Let σt be the projection of σt. It can be checked that σt

is an (α, |S|εt)-extended proper equilibrium of Γ with α = (1, . . . , 1). It follows that the projectionof σ is an extended proper equilibrium of Γ. If we had instead started with a symmetrically properstrategy of the meta-version of a game Γ′ that is equivalent to Γ up to affine transformations of thepayoffs, then we would simply use a correspondingly different choice for α.

Sufficiency. Suppose σ is an extended proper equilibrium of Γ. Let (σt)∞t=1 be an associated

sequence of (α, ε|S|+2t )-extended proper equilibria. Let φt =

∏n∈N σ

tn be the distribution over S

induced by σt. We would be done if we could show that (φt, . . . , φt) were an εt-proper equilibriumof Γ (or the meta-version of a game that is equivalent to Γ up to an affine transformation of thepayoffs). But this is not guaranteed to be the case. The bulk of the proof lies in using φt toconstruct another distribution over S that also has σt as its projection and that does constitutesuch an εt-proper equilibrium.14

5 Lexicographic Characterization

5.1 Framework

In this section, we employ the framework of Blume, Brandenburger and Dekel (1991)—henceforth,BBD—to characterize extended proper equilibrium directly in terms of players’ beliefs. Towardthat end, let ρ = (p1, . . . , pK) be a sequence of probability distributions over S, the set of purestrategy profiles. BBD refer to such a sequence as a lexicographic probability system (LPS).15

An LPS can be interpreted as follows: p1 represents the primary theory about how the gamewill be played, p2 represents beliefs about how the game will be played in the zero-probabilityevent that the primary theory is incorrect, and so on.16 To economize on notation, we define

13In the proof of Theorem 4, we additionally establish that extended proper equilibrium remains the prediction ifsymmetry is embedded into its logic (in the same way that symmetry changes neither Nash nor perfect equilibrium).Only for proper equilibrium does the solution change.

14In Appendix D, we provide further intuition for the proof of this part of the result by illustrating the constructionof this alternative distribution in the context of a particular example game.

15Our approach and notation differ slightly from BBD. One difference is that we directly assume that players’beliefs about the strategies of their opponents are the marginals of a “common prior LPS,” whereas BBD addthat assumption only when extending their focus beyond two-player games. Because extended proper equilibriumincorporates additional restrictions only in games with three or more players, it is natural for us to assume thiscommon prior from the outset. There are also small notational differences (e.g., while they designate players withsuperscripts and levels of an LPS with subscripts, we do the reverse).

16This interpretation is valid even if the supports of (p1, . . . , pK) overlap. BBD point out that it is necessary toallow for overlapping supports.

13


πn(sn|pk) =∑

s∈S pk(s)πn(s/sn). In words, this is player n’s payoff from sn when others’ play is

distributed according to pk (i.e., the kth level of the LPS).We define the best response set of player n ∈ N to the LPS ρ as follows, where we use the

symbol ≥L to represent the lexicographic ordering.17

BRn(ρ) =

{s′n ∈ Sn ∀s′′n ∈ Sn :

[πn(s′n|pk)

]Kk=1≥L

[πn(s′′n|pk)

]Kk=1

}.

As additional notation, given any J ⊆ N , let pkJ be the marginal distribution of pk on∏n∈J Sj .

An LPS ρ gives rise to a partial order on⋃J⊆N

∏n∈J Sj , which is the set of pure strategy profiles for

subsets of N . If sJ ∈∏n∈J Sj and sI ∈

∏n∈I Si, then we say that sJ >ρ sI , read as “sJ is infinitely

more likely than sI according to the LPS ρ,” if min{k : pkJ(sJ) > 0} < min{k : pkI (sI) > 0}. Wewrite sJ ≥ρ sI to mean it is not the case that sI >ρ sJ .

Next, we repeat a definition from BBD. A lexicographic Nash equilibrium is a pair: an LPSthat captures beliefs about how the game will be played and a strategy profile.

Definition 6. A pair (ρ, σ) is a lexicographic Nash equilibrium if

(i) for all n ∈ N and all sn ∈ Sn, p1n(sn) > 0 implies sn ∈ BRn(ρ), and

(ii) for all s ∈ S, p1(s) =∏n∈N σn(sn).

Condition (i) requires a form of rationality: under the primary theory, player n assigns zeroprobability to strategies that are not best responses to the beliefs. Condition (ii) requires a formof consistency: the primary theory for how the game will be played coincides with the strategies.

We next introduce two properties that an LPS may possess, which are progressively moredemanding. The first, “respects within-player preferences,” is equivalent to what BBD define as“respects preferences.” We use this different terminology to accentuate the distinction between thisproperty and “respects within-and-across-player preferences,” which is also defined below.

Definition 7. An LPS ρ = (p1, . . . , pK) on S respects within-player preferences if for all n ∈ Nand all s′n, s

′′n ∈ Sn, s′n ≥ρ s′′n implies[

πn(s′n|pk)]Kk=1≥L

[πn(s′′n|pk)

]Kk=1

.

Definition 8. An LPS ρ = (p1, . . . , pK) on S respects within-and-across-player preferences if thereexists some α ∈ RN++ such that for all l,m ∈ N , s∗l ∈ BRl(ρ), s∗m ∈ BRm(ρ), and all s′l ∈ Sl,s′′m ∈ Sm, s′l ≥ρ s′′m implies[

αl

(πl(s

∗l |pk)− πl(s′l|pk)

)]Kk=1≤L

[αm

(πm(s∗m|pk)− πm(s′′m|pk)

)]Kk=1

.

Definition 7 requires that under the beliefs, player n is infinitely less likely to use a strategy thatis a “worse response” to the beliefs than a strategy that is a “better response.” This strengthenscondition (i) of Definition 6, which imposed this requirement only when the latter strategy in thecomparison was a best response. Definition 8 strengthens Definition 7 by adding similar require-ments for across-player comparisons. In particular, Definition 8 requires the existence of a scalingvector α such that whenever the scaled loss (relative to a best response to the beliefs) of a strategys′l exceeds the scaled loss of another strategy s′′m, whether of the same or another player, then s′lmust be infinitely less likely than s′′m under the beliefs.

Finally, we repeat two other definitions from BBD. These are generalizations for LPSs of whatit means for a probability distribution to have full support or to be a product distribution.

17Formally, for a, b ∈ RK , a ≥L b if and only if whenever bk > ak, there exists a j < k such that aj > bj .

14


Definition 9. An LPS ρ = (p1, . . . , pK) on S has full support if for each s ∈ S, pk(s) > 0 for somek ∈ {1, . . . ,K}.

Definition 10. An LPS ρ = (p1, . . . , pK) on S satisfies strong independence if there is an equivalentF-valued probability distribution that is a product distribution, where F is some non-Archimedeanordered field that is a proper extension of R.18

To clarify the latter definition and in anticipation of subsequent discussion, it is useful to definethe following notation. If r = (r1, . . . , rK−1) is a vector and ρ = (p1, . . . , pK) is an LPS, then letr�ρ denote the probability distribution

r�ρ = (1− r1)p1 + r1[(1− r2)p2 + r2

[(1− r3)p3

+ r3[· · ·+ rK−2

[(1− rK−1)pK−1 + rK−1pK

]· · ·]]]

.(1)

As BBD show, ρ satisfies strong independence if and only if there exists a sequence r(t) ∈ (0, 1)K−1

with r(t)→ 0 such that r(t)�ρ is a product distribution for all t.

5.2 Characterizations

Using these definitions, Proposition 8 provides a characterization of extended proper equilibrium.For comparison, Propositions 5–7 are characterizations of Nash equilibrium, perfect equilibrium,and proper equilibrium, which are due to BBD.19 These characterizations are useful for two reasons.First, they simplify some proofs (particularly that of Theorem 3), since LPSs are easier to workwith than sequences of trembles. Second, compared to sequences of trembles, LPSs correspondmore closely to intuitive statements about some actions being “infinitely more likely” than others.

Proposition 5. A strategy profile σ is a Nash equilibrium if and only if there exists some LPS ρon S that satisfies strong independence for which (ρ, σ) is a lexicographic Nash equilibrium.

Proposition 6. A strategy profile σ is a perfect equilibrium if and only if there exists some LPS ρon S that satisfies strong independence and has full support for which (ρ, σ) is a lexicographic Nashequilibrium.

Proposition 7. A strategy profile σ is a proper equilibrium if and only if there exists some LPS ρon S that satisfies strong independence, has full support, and respects within-player preferences forwhich (ρ, σ) is a lexicographic Nash equilibrium.

Proposition 8. A strategy profile σ is an extended proper equilibrium if and only if there existssome LPS ρ on S that satisfies strong independence, has full support, and respects within-and-across-player preferences for which (ρ, σ) is a lexicographic Nash equilibrium.

18An ordered field F is non-Archimedean if it contains a non-zero infinitesimal (i.e., an element ε 6= 0 such that|ε| < 1

nfor all n ∈ N). An LPS ρ = (p1, . . . , pK) on S and an F-valued probability distribution p over S, are said to

be equivalent if there exists a vector of positive infinitesimals ε = (ε1, . . . , εK−1) ∈ FK−1 such that p(s) = ε�ρ(s) forall s ∈ S, where ε�ρ is defined as in equation (1).

19Propositions 6 and 7 restate Propositions 7 and 8 of BBD. While Proposition 5 is in the same spirit as theirProposition 3, which also characterizes Nash equilibrium in terms of lexicographic probability systems, it is not anidentical result. We therefore provide a separate proof. The differences are: (i) their result does not require players’beliefs about the strategies of their opponents to be the marginals of a “common prior LPS,” and (ii) their resultdoes not require the LPS to satisfy strong independence. However, the same result holds both with and withoutthese requirements. As discussed in footnote 15, it is natural for us to assume this common prior from the outset.We also require strong independence for more continuity with the other characterizations.

15


For intuition into one direction of these characterizations, suppose ρ = (p1, . . . , pK) is an LPSon S that satisfies strong independence. Hence, there exists a sequence r(t) ∈ (0, 1)K−1 withr(t) → 0 such that r(t)�ρ is a product distribution for all t, where we let σt denote its profile ofcomponent-wise marginals.

Now suppose that σ is a strategy profile and that (ρ, σ) is a lexicographic Nash equilibrium.We have r(t)�ρ→ p1, so that by condition (ii) in the definition of lexicographic Nash equilibrium,σt → σ. By condition (i) in the definition of lexicographic Nash equilibrium, σn(sn) > 0 impliessn ∈ BRn(ρ). This can be shown to imply sn ∈ BRn(σt) for all sufficiently large t.20 By conti-nuity, it follows that sn ∈ BRn(σ), so that σ is a Nash equilibrium, establishing one direction ofProposition 5.

Straightforward arguments establish the same direction of the other characterizations. Supposeρ also has full support. It can be shown that for any ε > 0, σt is an ε-perfect equilibrium for allsufficiently large t. (For instance, full support of ρ implies that σt is totally mixed.) Analogously, itcan be shown that respecting within-player preferences (respectively, respecting within-and-across-player preferences) implies that for any ε > 0, σt is an ε-proper equilibrium (respectively, an(α, ε)-extended proper equilibrium) for all sufficiently large t. Thus, in each case, the limitingstrategy profile σ satisfies the claimed refinement.

5.3 Proof Sketch of Theorem 3

We are now in a position to sketch the proof of Theorem 3, which states that proper equilibrium andextended proper equilibrium coincide in two-player games. Let σ be a proper equilibrium of sucha game. Let ρ be a corresponding LPS with the properties guaranteed by Proposition 7. To showthat σ is an extended proper equilibrium, we construct trembles of the form ((r1�ρ)1, (r2�ρ)2)with r1, r2 ∈ (0, 1)K−1.

Within-player restrictions. All sufficiently small choices of (r1, r2) guarantee that the resultingstrategy profile is an ε-proper equilibrium. To see this, focus on the perspective of player 1. Thedistribution of the opponent’s play induced by ((r1�ρ)1, (r2�ρ)2) coincides with that of r2�ρ.21

Suppose s′1 delivers a strictly lower payoff than s′′1 against such play, where r2 is small. Because ρrespects within-player preferences, it can be shown that s′1 <ρ s

′′1. And this implies that s′1 is much

less likely than s′′1 under (r1�ρ)1 when r1 is small, as required.Across-player restrictions. To complete the proof, it suffices to demonstrate that for any α ∈

R2++, there exists a choice of (r1, r2) under which the across-player restrictions are also satisfied, so

that the resulting strategy profile is an (α, ε)-extended proper equilibrium. This can be done usinga corollary of the Eilenberg and Montgomery (1946) fixed point result.22

To illustrate further, return to the two-player game in Figure 4. As mentioned, (Up,Left) isa proper equilibrium of that game. Proposition 7 therefore guarantees the existence of an LPSwith certain properties. One such LPS is ρ = (p1, p2, p3) in which p1 puts full weight on the

20In fact, it can be shown that all payoff rankings induced by ρ coincide with those induced by σt when t issufficiently large. See Proposition 1 of BBD or Lemma 12 in the appendix.

21This is where the presence of only two players becomes relevant. With three players, the distribution of play forplayers 2 and 3 induced by ((r1�ρ)1, (r2�ρ)2, (r3�ρ)3) is guaranteed to coincide with that of r�ρ for some choiceof r only if r2 = r3. Symmetric observations would limit the search to the case of r1 = r2 = r3, which removes theflexibility necessary for satisfying across-player restrictions.

22Let us briefly comment on the barrier to a more direct proof (i.e., not in terms of the LPS machinery). The proofstrategy relies on formulating a set a strategy profiles such that (i) every element of the set is close to σ, (ii) everyelement of the set is an ε-proper equilibrium, and (iii) the set is sufficiently rich to accommodate the fixed-pointargument used to satisfy the additional across-player restrictions. The LPS machinery permits a tractable formulationof such a set.

16


equilibrium (Up,Left); p2 puts equal weight on the two scenarios in which a single player trembles,(Up,Right) and (Down,Left); and p3 puts full weight on the scenario in which both players tremble,(Down,Right).23 To show that the equilibrium is also extended proper, let α = (1, 1) and use ρ toconstruct a sequence of (α, ε)-extended proper equilibria using the approach described above. Inthis case, the aforementioned fixed point construction might deliver the pair of sequences

rrow(t) =

(2t− 1

t3,

1

2t− 1

)rcol(t) =

(2t− 1

t2,

1

2t− 1

).

Then for each player n, let σtn be the marginal of rn(t)�ρ on Sn. Computing, we find that this isthe same sequence of (α, ε)-extended proper equilibria previously mentioned in conjunction withthis game: σtrow specifies Down with probability 1

t2(and Up otherwise), and σtcol specifies Right with

probability 1t (and Left otherwise).

6 Application: Generalized Second-Price Auction

Extended proper equilibrium can be productively applied to analyze the generalized second-price(GSP) auction, which is an auction format that has been widely used to sell search advertising onthe Internet. This auction was modeled and studied by Edelman, Ostrovsky and Schwarz (2007)who also proposed locally envy-free equilibrium as a refinement of the auction’s many equilibriain settings of complete information.24 That refinement, however, is defined directly in terms ofthe GSP auction game and does not apply to general non-cooperative games, rendering it difficultto assess the logic of the refinement separately from the game. In contrast, extended properequilibrium is a general refinement that, as we argue below, can do much of the same work.

6.1 Framework

There are I ad positions and N bidders. The ith position has a click rate κi > 0. Bidder n hasa per-click value vn > 0. Bidder n’s payoff from being in position i is therefore κivn minus itspayments to the auctioneer. We label positions and bidders so that click rates and bidder valuesare strictly in descending order: κ1 > · · · > κI and v1 > · · · > vN . For simplicity, we also assumeN ≤ I + 1.

Bids in a GSP auction are submitted simultaneously. Let b(i) denote the ith highest bid, andfor convenience, define b(N+1) = 0. After breaking ties uniformly at random, position i is allocatedto the ith highest bidder at a per-click price b(i+1), for a total payment κib

(i+1).In the original analysis, allowable bids are the nonnegative reals. However, the refinements

of proper equilibrium and extended proper equilibrium are defined only for finite games. Conse-quently, our analysis restricts bids to the finite set {0, 1

m ,2m , . . . ,

mMm } for positive integers m and

M . Altogether, this defines a game of complete information among the bidders.We assume M is large enough to be non-binding (e.g., M > v1 suffices). It can be shown that

if m is also sufficiently large, then ties do not occur in the pure Nash equilibria;25 we assume that

23In fact, this LPS corresponds to the sequence of ε-proper equilibria previously mentioned in conjunction withthis game. As before, let σtrow specify Down with probability 1

t(and Up otherwise), and let σtcol similarly specify Right

with probability 1t

(and Left otherwise). The probability distribution induced by σt coincides with r(t)�ρ, wherer(t) = ( 2t−1

t2, 1

2t−1).

24Varian (2007) studies the same model and makes the same selection, calling these symmetric equilibria.25Lemma 17 in the appendix shows this is the case if m > 4κ1

∆κ∆v, where ∆κ = mini∈{1,...,I−1} κi − κi+1 and

∆v = minn∈{1,...,N−1} vn − vn+1. Note that the earlier assumptions imply ∆κ > 0 and ∆v > 0.

17


m is large enough to do this. Thus, given a pure Nash equilibrium, it is well defined to let g(i)denote the identity of the ith highest bidder.

Edelman, Ostrovsky and Schwarz (2007) defined an equilibrium to be locally envy-free if nobidder would prefer to exchange bids with the bidder that is one position higher. (See their paperfor an intuitive argument for why such a refinement might be reasonable.) Thus, unlike our concept,the definition is in terms of the GSP auction itself and not in terms of trembles. To modify thatrefinement for this setting with a discrete bid set, what seems most in keeping with their originalmotivation is to slightly weaken it by instead requiring that no bidder would prefer to exchangebids with the bidder that is one position higher and then to have the latter increase its bid byone increment. Note that as m grows large (so that the discretization of the bid set becomesprogressively fine), this definition converges to their original definition for a continuous bid set.

Definition 11. A pure equilibrium of the GSP auction b = (b1, . . . , bN ) is a locally envy-free*equilibrium if for all i ∈ {2, . . . , N}, κi[vg(i) − b(i+1)] ≥ κi−1[vg(i) − (b(i) + 1

m)].

Under the aforementioned assumptions (e.g., that M and m are sufficiently large), we can thenstate the main result of this section:

Proposition 9. Every pure extended proper equilibrium of the GSP auction is a locally envy-free*equilibrium.26

What is more, as we argue below, this conclusion could not be obtained from proper equilibriumalone.

6.2 An Example

To illustrate and to sketch the proof of Proposition 9, consider the following specific example. Thereare three bidders, with per-click values (v1, v2, v3) = (4, 2, 1). There are three ad positions, withclick rates (κ1, κ2, κ3) = (4, 2, 1). For the discretized bid set, it suffices to let m = 6 and M = 3.

The equilibrium b∗ = (52 , 1,

12) is not locally envy-free*. This is because bidder 2 “envies”

bidder 1 in the sense described above: bidder 2’s equilibrium payoff is π2(52 , 1,

12) = 3, but if

bidder 2 exchanged bids with bidder 1 and bidder 1 then raised its bid by one increment, bidder 2would obtain the higher payoff π2(7

6 ,52 ,

12) = 10

3 .Nevertheless, b∗ is a proper equilibrium of this example. Indeed, in Milgrom and Mollner (2018,

Section 4.1) we argue that it is supported as a proper equilibrium by trembles in which bidder 1 ismuch less likely to tremble than bidder 2 (that is, bidder 1’s total probability of trembling is muchsmaller than bidder 2’s probability of trembling to any particular bid), and bidder 2 is much lesslikely to tremble than bidder 3. These trembles are, however, inconsistent with extended properequilibrium because they require it to be more likely that bidder 3 trembles to 7

6 (one incrementabove bidder 2’s equilibrium bid of 1), which is not a best response to b∗, than that bidder 1 makesthe same tremble, which is a best response.

What is more, b∗ is not an extended proper equilibrium at all, because it cannot be supportedby any trembles satisfying the necessary restrictions. Building on Proposition 8, suppose to the

26Milgrom and Mollner (2018, Section 3.2) show that essentially the same conclusion can be obtained from test-setequilibrium. Focusing on the case in which the allowable bids are the nonnegative reals, Theorem 6 in that paperstates that every pure test-set equilibrium of the GSP auction is a locally envy-free equilibrium. However, thereare examples of GSP auction games in which extended proper equilibrium rules out strategy profiles that test-setequilibrium does not (cf. Appendix C.3). This is, of course, in addition to the differences between the two solutionconcepts illustrated by the examples in Section 3.4.

18


contrary that ρ is an LPS that satisfies strong independence, has full support, and respects within-and-across-player preferences for which (ρ, b∗) is a lexicographic Nash equilibrium. To see that thisproduces a contradiction, suppose that bidder 2 contemplates raising its bid from b∗2 = 1 to 7

6 . Thischange makes a difference only if at least one of its opponents trembles within the set {1, 7

6}. Thus,the relevant profiles can be partitioned into four sets:

• Case 1: b1 = 76 and b3 = 1

2

• Case 2(a): b1 = 1 and b3 = 12

• Case 2(b): b1 = 52 and b3 ∈ {1, 7

6}• Case 3: b1 6= 5

2 and b3 6= 12 , with b1 ∈ {1, 7

6} or b3 ∈ {1, 76} or both

Against the profile in case 1, bidder 2 is strictly better off deviating to 76 :

π2(76 ,

76 ,

12) = 19

6 > 3 = π2(76 , 1,

12).

Moreover, each profile in the other cases is infinitely less likely under ρ than the profile in case 1.Indeed, for any profile in case 2(a) or 2(b), the deviating bidder is playing an inferior responseto b∗.27 In case 1, in contrast, only bidder 1 is deviating from b∗, and the deviation is to a bestresponse to b∗. It therefore follows from ρ satisfying strong independence and respecting within-and-across-player preferences that for any (b1, b3) in case 2(a) or 2(b), (7

6 ,12) >ρ (b1, b3). Similarly,

each profile in case 3 involves both bidders deviating from their equilibrium bids. For each suchprofile, there is a corresponding profile in one of the other cases that involves only a single bidderdeviating, which, by strong independence, must be infinitely more likely under ρ. Altogether, weconclude that 7

6 is a profitable deviation for bidder 2 against ρ, which contradicts (ρ, b∗) being alexicographic Nash equilibrium. Moreover, the preceding argument can be generalized to proveProposition 9.

7 Conclusion

We introduce a new refinement for games in normal form: extended proper equilibrium. Everyfinite game possesses at least one extended proper equilibrium. Like proper equilibrium, our newrefinement is defined using trembles and imposes the following within-player restrictions on thosetrembles: in the limit, a player should be overwhelmingly less likely to tremble to a strategythat is a more costly mistake than to a different strategy that would be a less costly mistake.However, extended proper equilibrium strengthens that concept by adding further restrictions tothe allowable trembles, which are precisely the consequences of combining proper equilibrium witha certain form of across-player symmetry in the propensity to tremble—namely, extended properequilibrium describes the play induced by symmetrically proper strategies in the meta-game modelin which players originate from a common pool. We also show that these definitions in terms oftrembles have a corresponding characterization in terms of beliefs using lexicographic probabilitysystems. In that formulation, extended proper equilibrium requires that, for some scaling of payoffs,costly mistakes should be regarded as infinitely less likely than less costly ones.

These additional restrictions have bite in games with three or more players, restricting what aplayer may believe about the relative likelihood of mistakes by two different opponents. In games

27If bidder 1 deviates from b∗1 = 52

to b1 = 1 as in case 2(a), then, assuming the others continue to play theirequilibrium bids of b∗2 = 1 and b∗3 = 1

2, bidder 1’s allocation changes from always winning the highest position to

winning the highest position only half the time. It can be shown that this is not a best response to b∗. For similarreasons, it is not a best response for bidder 3 to change the allocation by deviating to b3 ∈ {1, 7

6} as in case 2(b).

19


with only two players, each player has just a single opponent, and so there are no such beliefsto be formed. Consistent with this intuition, extended proper equilibrium coincides with properequilibrium in two-player games. Nevertheless, many games of economic interest involve more thantwo players, and in those settings, extended proper equilibrium may be a useful refinement of properequilibrium. The GSP auction model (Edelman, Ostrovsky and Schwarz, 2007; Varian, 2007) isone such application.

References

Banks, Jeffrey S. and Joel Sobel, “Equilibrium Selection in Signaling Games,” Econometrica,1987, 55 (3), 647–661.

Blume, Lawrence, Adam Brandenburger, and Eddie Dekel, “Lexicographic Probabilitiesand Equilibrium Refinements,” Econometrica, 1991, 59 (1), 81–98.

Cho, In-Koo and David M. Kreps, “Signaling Games and Stable Equilibria,” The QuarterlyJournal of Economics, 1987, 102 (2), 179–221.

Edelman, Benjamin, Michael Ostrovsky, and Michael Schwarz, “Internet Advertising andthe Generalized Second-Price Auction: Selling Billions of Dollars Worth of Keywords,” AmericanEconomic Review, 2007, 97 (1), 242–259.

Eilenberg, Samuel and Deane Montgomery, “Fixed Point Theorems for Multi-Valued Trans-formations,” American Journal of Mathematics, 1946, 68 (2), 214–222.

Fudenberg, Drew and Kevin He, “Learning and Type Compatibility in Signalling Games,”Econometrica, 2018, 86 (4), 1215–1255.

and , “Player-Compatible Equilibrium,” Working Paper, 2020. https://arxiv.org/abs/

1712.08954.

Jurado, I. Garcıa, “Nuevos Equilibrios Estables ante Tendencias al Error en Juegos no Coopera-tivos.” PhD dissertation, Universidade de Santiago de Compostela, 1989. http://eio.usc.es/

pub/io/xogos/MATERIALES/ThesisIgnacio.pdf.

, “Un Refinamiento del Concepto de Equilibrio Propio de Myerson,” Trabajos de InvestigacionOperativa, 1989, 4 (1), 11–19.

and J. M. Prada Sanchez, “A Remark on Myerson’s Concept of Proper Equilibrium,” In-ternational Journal of Game Theory, 1990, 19 (1), 11–16.

Kakutani, Shizuo, “A Generalization of Brouwer’s Fixed Point Theorem,” Duke MathematicalJournal, 1941, 8 (3), 416–427.

Kalai, Ehud and Dov Samet, “Persistent Equilibria in Strategic Games,” International Journalof Game Theory, 1984, 13 (3), 129–144.

Kleppe, John, Peter Borm, and Ruud Hendrickx, “Fall Back Proper Equilibrium,” TOP,2017, 25 (2), 402–412.

Kohlberg, Elon and Jean-Francois Mertens, “On the Strategic Stability of Equilibria,” Econo-metrica, 1986, 54 (5), 1003–1037.

Kreps, David M. and Robert Wilson, “Sequential Equilibria,” Econometrica, 1982, 50 (4),863–894.

20


https://arxiv.org/abs/1712.08954

https://arxiv.org/abs/1712.08954

http://eio.usc.es/pub/io/xogos/MATERIALES/ThesisIgnacio.pdf

http://eio.usc.es/pub/io/xogos/MATERIALES/ThesisIgnacio.pdf

Maynard Smith, John and Geoffrey A. Parker, “The Logic of Asymmetric Contests,” AnimalBehaviour, 1976, 24 (1), 159–175.

McKelvey, Richard D. and Thomas R. Palfrey, “Quantal Response Equilibria for NormalForm Games,” Games and Economic Behavior, 1995, 10 (1), 6–38.

Milgrom, Paul and Joshua Mollner, “Equilibrium Selection in Auctions and High StakesGames,” Econometrica, 2018, 86 (1), 219–261.

Myerson, Roger, “Refinements of the Nash Equilibrium Concept,” International Journal of GameTheory, 1978, 7 (2), 73–80.

and Jorgen Weibull, “Tenable Strategy Blocks and Settled Equilibria,” Econometrica, 2015,83 (3), 943–976.

Nachbar, John H., “‘Evolutionary’ Selection Dynamics in Games: Convergence and Limit Prop-erties,” International Journal of Game Theory, 1990, 19 (1), 59–89.

Neary, Philip R., “The Class of Reasonable Equilibria,” Working Paper, 2010. https://sites.google.com/site/prneary/docs/Reasonable.pdf.

Okada, Akira, “On Stability of Perfect Equilibrium Points,” International Journal of GameTheory, 1981, 10 (2), 67–73.

Reny, Philip J., “On the Existence of Monotone Pure-Strategy Equilibria in Bayesian Games,”Econometrica, 2011, 79 (2), 499–553.

Selten, Reinhard, “Reexamination of the Perfectness Concept for Equilibrium Points in ExtensiveGames,” International Journal of Game Theory, 1975, 4 (1), 25–55.

, “A Note on Evolutionarily Stable Strategies in Asymmetric Animal Conflicts,” Journal ofTheoretical Biology, 1980, 84 (1), 93–101.

, “Evolutionary Stability in Extensive Two-Person Games,” Mathematical Social Sciences, 1983,5 (3), 269–363.

van Damme, Eric, “A Relation Between Perfect Equilibria in Extensive Form Games and ProperEquilibria in Normal Form Games,” International Journal of Game Theory, 1984, 13 (1), 1–13.

, Stability and Perfection of Nash Equilibria, Springer-Verlag, 1991.

Varian, Hal R., “Position Auctions,” International Journal of Industrial Organization, 2007, 25(6), 1163–1178.

21


https://sites.google.com/site/prneary/docs/Reasonable.pdf

https://sites.google.com/site/prneary/docs/Reasonable.pdf

A Lemmas

A.1 Fixed Point Theorem

The proof of Theorem 3 requires a more powerful fixed point theorem than that of Kakutani (1941).We use the following corollary of the Eilenberg and Montgomery (1946) fixed point result.28

Lemma 10 (Eilenberg-Montgomery). Suppose that X is a nonempty, compact, and convex sub-set of a Euclidean space and that F : X ⇒ X is an upper-hemicontinuous, nonempty-valued,contractible-valued correspondence. Then F has a fixed point.

A.2 Lexicographic Probability Systems

Here, we state and prove some technical lemmas concerning LPSs. The first such lemma is animmediate consequence of the definition of the “square operator” given by equation (1):

Lemma 11. Suppose ρ = (p1, . . . , pK) is an LPS on S. Define p0 =mink∈{1,...,K}mins∈supp(pk) p

k(s). For any J ⊆ N and s′J ∈∏n∈J Sn, define k′ = min{k :

pkJ(s′J) > 0}. For any r ∈ (0, 1)K−1,

r1 · · · rk′−1(1− rk′)p0 ≤ (r�ρ)J(s′J) ≤ r1 · · · rk′−1.

Proof of Lemma 11. Because (r�ρ)J(s′J) =∑

s∈S:sJ=s′Jr�ρ(s), it can be expressed as

(1− r1)∑

s∈S:sJ=s′J

p1(s) + r1

(1− r2)∑

s∈S:sJ=s′J

p2(s) + r2

(1− r3)∑

s∈S:sJ=s′J

p3(s)

+ r3

· · ·+ rK−2

(1− rK−1)∑

s∈S:sJ=s′J

pK−1(s) + rK−1∑

s∈S:sJ=s′J

pK(s)

· · ·

= (1− r1)p1J(s′J) + r1

[(1− r2)p2

J(s′J) + r2[(1− r3)p3

J(s′J)

+ r3[· · ·+ rK−2

[(1− rK−1)pK−1

J (s′J) + rK−1pKJ (s′J)]· · ·]]]

.

Using pkJ(s′J) = 0 for all k < k′, this simplifies to

r1 · · · rk′−1[(1− rk′)pk′J (s′J) + rk

′[· · ·+ rK−2

[(1− rK−1)pK−1

J (s′J) + rK−1pKJ (s′J)]· · ·]].

On one hand, we have for all k ≥ k′ that pkJ(s′J) ≤ 1. Plugging this in, we obtain (r�ρ)J(s′J) ≤r1 · · · rk′−1. On the other hand, we have pk

′J (s′J) ≥ p0 and for all k > k′ that pkJ(s′J) ≥ 0. Plugging

these in, we obtain (r�ρ)J(s′J) ≥ r1 · · · rk′−1(1− rk′)p0.

Lemma 12.29 Given any LPS ρ = (p1, . . . , pK) on S and any α ∈ RN++, there exists a δ > 0 suchthat if r ∈ (0, δ)K−1, then (∀l ∈ N )(∀m ∈ N )(∀s′l ∈ Sl)(∀s′′m ∈ Sm)(∀s∗l ∈ BRl(ρ))(∀s∗m ∈ BRm(ρ))

αl maxsl∈Sl

∑s∈S

(r�ρ)(s)[πl(s/sl)− πl(s/s′l)] > αm maxsm∈Sm

∑s∈S

(r�ρ)(s)[πm(s/sm)− πm(s/s′′m)] (2)

28See Reny (2011) for another economic application of the Eilenberg and Montgomery (1946) result.29This result is related to Proposition 1 of BBD. The primary difference is that Lemma 12 expands focus from

within-player comparisons to within-and-across-player comparisons.

22


if and only if [αl

(πl(s


)]Kk=1

>L

[αm


)]Kk=1

. (3)

Proof of Lemma 12. Fix an LPS ρ = (p1, . . . , pK) and a scaling vector α ∈ RN++. Select forevery n ∈ N some s∗n ∈ BRn(ρ). It does not matter which element is chosen—this is because allelements of BRn(ρ) must yield identical expected payoffs against every LPS level pk, and so thespecific choice does not affect equation (3). We then define for each k ∈ {1, . . . ,K}, l,m ∈ N(possibly equal), s′l ∈ Sl, and s′′m ∈ Sm,

Dklm(s′l, s

′′m) = αl

[πl(s


]− αm

[πm(s∗m|pk)− πm(s′′m|pk)

].

If Dklm(s′l, s

′′m) = 0 for all (l,m, k, s′l, s

′′m), then neither (2) nor (3) ever applies, and the statement

holds vacuously with any δ. We therefore assume henceforth that this is not the case. Next, define

D = maxk∈{1,...,K}l,m∈N

s′l∈Sl,s′′m∈Sm

|Dklm(s′l, s

′′m)|

D = mink∈{1,...,K}l,m∈N

s′l∈Sl,s′′m∈Sm

{Dklm(s′l, s

′′m) : Dk

lm(s′l, s′′m) > 0

}.

By the previous assumption, D is a well-defined positive number. And clearly, D ≥ 0. Setδ = D

D+D> 0, and we show that the statement holds with this choice of δ. Suppose r ∈ (0, δ)K−1.

Below, part 1 establishes the statement for the case of l = m, and part 2 proceeds to the generalcase.

Part 1: We begin by observing that for all n ∈ N , s′n ∈ Sn, and s′′n ∈ Sn,

αn maxsn∈Sn

∑s∈S

(r�ρ)(s)[πn(s/sn)− πn(s/s′n)]− αn maxsn∈Sn

∑s∈S

(r�ρ)(s)[πn(s/sn)− πn(s/s′′n)]

= −αn∑s∈S

(r�ρ)(s)πn(s/s′n) + αn∑s∈S

(r�ρ)(s)πn(s/s′′n)

= αn∑s∈S

(r�ρ)(s)[πn(s/s∗n)− πn(s/s′n)]− αn∑s∈S

(r�ρ)(s)[πn(s/s∗n)− πn(s/s′′n)]

=∑s∈S

(r�ρ)(s)(αn[πn(s/s∗n)− πn(s/s′n)]− αn[πn(s/s∗n)− πn(s/s′′n)]

).

This observation will be used in the two cases we now consider. For the first case, suppose that forsome n ∈ N , s′n ∈ Sn, and s′′n ∈ Sn,[

αn

(πn(s∗n|pk)− πn(s′n|pk)

)]Kk=1

>L

[αn

(πn(s∗n|pk)− πn(s′′n|pk)

)]Kk=1

.

Then there is some k such that both Dknn(s′n, s

′′n) ≥ D > 0 and for all j < k, Dj

nn(s′n, s′′n) = 0. For

23


this k, let ρ = (pk+1, . . . , pK) and r = (rk+1, . . . , rK−1). Then∑s∈S


)

= r1 · · · rk−1

(1− rk)Dknn(s′n, s

′′n)

+rk∑s∈S


)≥ r1 · · · rk−1

[(1− rk)D − rkD

]> r1 · · · rk−1

[D − δ(D +D)

],

which is equal to zero because δ = D

D+D. We conclude that

αn maxsn∈Sn

∑s∈S

(r�ρ)(s)[πn(s/sn)− πn(s/s′n)] > αn maxsn∈Sn

∑s∈S

(r�ρ)(s)[πn(s/sn)− πn(s/s′′n)].

For the second case, suppose instead that for some n ∈ N , s′n ∈ Sn, and s′′n ∈ Sn,[αn

(πn(s∗n|pk)− πn(s′n|pk)

)]Kk=1

=L

[αn

(πn(s∗n|pk)− πn(s′′n|pk)

)]Kk=1

.

Then Dknn(s′n, s

′′n) = 0 for all k. Thus,∑s∈S


),

as a weighted average of {Dknn(s′n, s

′′n)}Kk=1, is also equal to zero. We conclude that

αn maxsn∈Sn

∑s∈S

(r�ρ)(s)[πn(s/sn)− πn(s/s′n)] = αn maxsn∈Sn

∑s∈S

(r�ρ)(s)[πn(s/sn)− πn(s/s′′n)].

The first case establishes (3) =⇒ (2). By exchanging the roles of s′n and s′′n, the two cases togetherestablish the contrapositive of (2) =⇒ (3).

Part 2: We begin by observing that for all l,m ∈ N , s′l ∈ Sl, and s′′m ∈ Sm,

αl maxsl∈Sl

∑s∈S

(r�ρ)(s)[πl(s/sl)− πl(s/s′l)]− αm maxsm∈Sm

∑s∈S

(r�ρ)(s)[πm(s/sm)− πm(s/s′′m)]

= αl∑s∈S

(r�ρ)(s)[πl(s/s∗l )− πl(s/s′l)]− αm

∑s∈S

(r�ρ)(s)[πm(s/s∗m)− πm(s/s′′m)]

=∑s∈S

(r�ρ)(s)(αl[πl(s/s

∗l )− πl(s/s′l)]− αm[πm(s/s∗m)− πm(s/s′′m)]

),

where the first step follows from part 1 of this proof, which implies that any maximizers sl and smmust be elements of BRl(ρ) and BRm(ρ), respectively. This observation will be used in the twocases we now consider. For the first case, suppose that for some l,m ∈ N , s′l ∈ Sl, and s′′m ∈ Sm,[

αl

(πl(s


)]Kk=1

>L

[αm


)]Kk=1

.

24


Then there is some k such that both Dklm(s′l, s

′′m) ≥ D > 0 and for all j < k, Dj

lm(s′l, s′′m) = 0. For

this k, let ρ = (pk+1, . . . , pK) and r = (rk+1, . . . , rK−1). Then we have that∑s∈S



)

= r1 · · · rk−1

(1− rk)Dklm(s′l, s

′′m)

+rk∑s∈S



)≥ r1 · · · rk−1

[(1− rk)D − rkD

]> r1 · · · rk−1

[D − δ(D +D)

],

which is equal to zero because δ = D

D+D. We conclude that

αl maxsl∈Sl

∑s∈S


∑s∈S

(r�ρ)(s)[πm(s/sm)− πm(s/s′′m)].

For the second case, suppose instead that for some l,m ∈ N , s′l ∈ Sl, and s′′m ∈ Sm,[αl

(πl(s


)]Kk=1

=L

[αm


)]Kk=1

.

Then Dklm(s′l, s

′′m) = 0 for all k. Thus,∑s∈S



),

as a weighted average of {Dklm(s′l, s

′′m)}Kk=1, is also equal to zero. We conclude that

αl maxsl∈Sl

∑s∈S

(r�ρ)(s)[πl(s/sl)− πl(s/s′l)] = αm maxsm∈Sm

∑s∈S


The first case establishes (3) =⇒ (2). By exchanging the roles of s′l and s′′m, the two cases togetherestablish the contrapositive of (2) =⇒ (3).

Lemma 13. Suppose an LPS ρ = (p1, . . . , pK) on S that respects within-and-across-player pref-erences and a strategy profile σ are such that (ρ, σ) is a lexicographic Nash equilibrium. For alll,m ∈ N , all s′l ∈ Sl, and all s′′m ∈ Sm, if s′l /∈ BRl(σ) and s′′m ∈ BRm(σ), then s′′m >ρ s

′l.

Proof of Lemma 13. Fix any α ∈ RN++, any s∗l ∈ BRl(ρ), and any s∗m ∈ BRm(ρ). Sup-pose s′l ∈ Sl and s′′m ∈ Sm are such that s′l /∈ BRl(σ) and s′′m ∈ BRm(σ). Because(ρ, σ) is a lexicographic Nash equilibrium, p1(s) =

∏n∈N σn(s). Thus, s′l /∈ BRl(σ) implies

that αl∑

s∈S p1(s)[πl(s/s

∗l ) − πl(s/s

′l)] > 0. On the other hand, s′′m ∈ BRm(σ) implies that

αm∑

s∈S p1(s)[πm(s/s∗m)− πm(s/s′m)] = 0. This implies[

αl

(πl(s


)]Kk=1

>L

[αm


)]Kk=1

.

Because ρ respects within-and-across-player preferences, s′′m >ρ s′l.

25


Lemma 14. Suppose an LPS ρ = (p1, . . . , pK) on S and a strategy profile σ are such that (ρ, σ) is alexicographic Nash equilibrium. For all J ⊆ N , all I ⊆ N , all s′J ∈

∏n∈J Sn, and all s′′I ∈

∏n∈I Sn,

if σn(s′′n) > 0 for all n ∈ I, then

(i) s′′I ≥ρ s′J ; and

(ii) if in addition, σm(s′m) = 0 for some m ∈ J , then s′′I >ρ s′J .

Proof of Lemma 14. Claim (i). Suppose s′′I ∈∏n∈I Sn is such that σn(s′′n) > 0 for all n ∈ I.

Because (ρ, σ) is a lexicographic Nash equilibrium, p1I(s′′I ) =

∏n∈I σn(s′′n) > 0. Therefore, s′′I ≥ρ s′J ,

regardless of the choice of s′J ∈∏n∈J Sn.

Claim (ii). Suppose in addition that s′J ∈∏n∈J Sn is such that σm(s′m) = 0 for some m ∈ J .

Thus, p1J(s′J) =

∏n∈J σn(s′n) = 0. On the other hand, we had p1

I(s′′I ) > 0. Therefore, s′′I >ρ s

′J .

Lemma 15. Suppose an LPS ρ = (p1, . . . , pK) on S is equivalent to the F-valued probabilitydistribution p on S. For all J ⊆ N , I ⊆ N , sJ ∈

∏n∈J Sj, and sI ∈

∏n∈I Si: sJ ≥ρ sI iff there

exists an N ∈ N such that pJ (sJ )pI(sI) ≥

1N .

Proof of Lemma 15. Define

p0 = mink∈{1,...,K}

mins∈supp(pk)

pk(s).

Choose N0 ∈ N to be such that p0 > 1N0

. By what it means for ρ and p to be equivalent,

there exists a vector of infinitesimals ε = (ε1, . . . εK−1) ∈ FK−1 such that p = ε�ρ. SupposesJ ≥ρ sI , so that k := min{j : pjJ(sJ) > 0} ≤ min{j : pjI(sI) > 0}. Then Lemma 11 implies both

pJ(sJ) ≥ ε1 · · · εk−1(1− εk)p0, and pI(sI) ≤ ε1 · · · εk−1(1− εk). Therefore pJ (sJ )pI(sI) ≥ p0 >

1N0

.

Suppose instead that sJ <ρ sI , so that k := min{j : pjJ(sJ) > 0} > min{j : pjI(sI) > 0}. ThenpJ(sJ) ≤ ε1 · · · εk−1(1− εk) and pI(sI) ≥ ε1 · · · εk−2(1− εk−1)p0. Consequently,

pJ(sJ)

pI(sI)≤ εk−1(1− εk)

(1− εk−1)p0,

which is an infinitesimal and less than 1N for all N ∈ N.

Lemma 16. Suppose an LPS ρ = (p1, . . . , pK) satisfies strong independence. Let I and J be disjointsubsets of N . Similarly, let I ′ and J ′ be disjoint subsets of N . For all sI ∈

∏n∈I Sn, sJ ∈

∏n∈J Sn,

sI′ ∈∏n∈I′ Sn, and sJ ′ ∈

∏n∈j′ Sn, if sI >ρ sI′ and sJ ≥ρ sJ ′ then (sI , sJ) >ρ (sI′ , sJ ′).

Proof of Lemma 16. Because ρ satisfies strong independence, it is equivalent to some F-valuedprobability distribution p on S that is a product distribution. Suppose sI >ρ sI′ and sJ ≥ρ sJ ′ .Then by Lemma 15, we have that ∀N ∈ N,

pI′ (sI′ )pI(sI) < 1

N . Similarly, ∃N0 ∈ N such that pJ (sJ )pJ′ (sJ′ )

> 1N0

.

Equivalently,pJ′ (sJ′ )pJ (sJ ) < N0. Because p is a product distribution,

pI′,J ′(sI′ , sJ ′)

pI,J(sI , sJ)=pI′(sI′)

pI(sI)

pJ ′(sJ ′)

pJ(sJ)<N0

Nfor all N ∈ N,

which implies thatpI′,J′ (sI′ ,sJ′ )

pI,J (sI ,sJ ) < 1N for all N ∈ N. Another application of Lemma 15 gives the

desired result, (sI , sJ) >ρ (sI′ , sJ ′).

26


A.3 GSP Auction

Lemma 17. Let ∆κ = mini∈{1,...,I−1} κi − κi+1 and ∆v = minn∈{1,...,N−1} vn − vn+1. Assume

m > 4κ1∆κ∆v

. If b = (b1, . . . , bN ) is a pure Nash equilibrium of the GSP auction, then for all

i ∈ {1, . . . , N − 1}, b(i) > b(i+1).

Proof of Lemma 17. Suppose, by way of contradiction, that i′ ∈ {1, . . . , N − 1} and i′′ > i′ aresuch that b(i

′) = · · · = b(i′′) = b∗ is a maximal tie. First, note that any bidder with per-click value

v who is part of the tie earns the payoff

1

i′′ − i′ + 1

[κi′′(v − b(i

′′+1)) +

i′′−1∑i=i′

κi(v − b∗)

].

If such a bidder raises its bid to b∗ + 1m , that leads to a payoff of at least

κi′

[v −

(b∗ +

1

m

)].30,31

The rest of the proof consists of two parts. First, we argue that b(i′′+1) < b∗− 1

m . Second, we arguethat at least one of the bidders who are part of the tie has a profitable deviation.

Part 1: If b(i′′+1) ≥ b∗ − 1

m ,32 then a bidder with per-click value v who is part of the tie earns apayoff that is at most

1

i′′ − i′ + 1

[κi′′

m+

i′′∑i=i′

κi(v − b∗)

].

Every such bidder must have v ≥ b∗− 1m , or else its payoff would be negative, and it could profitably

deviate to a bid of zero. There therefore exists a bidder that is part of the tie with a per-click valuev ≥ b∗ − 1

m + ∆v. We argue that it would be profitable for the latter bidder to deviate to b∗ + 1m :

κi′

[v −

(b∗ +

1

m

)]− 1

i′′ − i′ + 1

[κi′′

m+

i′′∑i=i′

κi(v − b∗)

]

=1

i′′ − i′ + 1

[i′′∑

i=i′+1

(κi′ − κi)(v − b∗)

]− 1

m

(κi′ +

κi′′

i′′ − i′ + 1

)≥ 1

i′′ − i′ + 1

[(i′′ − i′)∆κ

(∆v −

1

m

)]− κ1

m

(i′′ − i′ + 2

i′′ − i′ + 1

)=

1

i′′ − i′ + 1

[(i′′ − i′)∆κ∆v −

(i′′ − i′)∆κ + κ1(i′′ − i′ + 2)

m

]≥ i′′ − i′

i′′ − i′ + 1

[∆κ∆v −

4κ1

m

],

which is strictly positive under the assumption that m > 4κ1∆κ∆v

. In the above, the first and third

steps are algebra. The second step uses κi′ −κi ≥ ∆κ for all i > i′, v ≥ b∗− 1m + ∆v, κi′′ ≤ κ1, and

κi′ ≤ κ1. The fourth step uses ∆κ ≤ κ1 and 2i′′ − 2i′ + 2 ≤ 4(i′′ − i′).30If b(i

′−1) > b∗+ 1m

, then the payoff is exactly κi′(v− b∗). If b(i′−1) = b∗+ 1

m, then the payoff is a lottery between

that and terms of the form κi[v −

(b∗ + 1

m

)]for i < i′. In either case, the bound applies.

31It is true that this deviation to b∗+ 1m

is not possible if b∗ = M . But the deviation is necessary for the subsequentarguments only when it is profitable. And it cannot be profitable if b∗ = M , by the assumption that M > v1.

32We cannot have b(i′′+1) > b∗. But we can have b(i

′′+1) = b∗ in the case where i′′ = N and b∗ = 0. (Recall thatwe have defined b(N+1) = 0.)

27


Part 2: From the previous part of the proof, b(i′′+1) < b∗ − 1

m . Therefore, a bidder with per-clickvalue v who is part of the tie can reduce its bid to b∗ − 1

m and earn the payoff

κi′′(v − b(i′′+1)).

This would be a profitable deviation for any such bidder with per-click value v < b∗. We thereforeassume henceforth that v ≥ b∗ for all such bidders. For the following arguments, we use the factthat the following is an upper bound on such a bidder’s payoff from not deviating:

1

i′′ − i′ + 1

[κi′′(v − b(i

′′+1)) + (i′′ − i′)κi′(v − b∗)− (i′′ − i′ − 1)∆κ(v − b∗)].

Let v′ > v′′ be the per-click values of two of the tied bidders. Because the bidder with value v′ doesnot find it profitable to deviate to b∗ + 1

m ,

1

i′′ − i′ + 1

[κi′′(v

′ − b(i′′+1)) + (i′′ − i′)κi′(v′ − b∗)− (i′′ − i′ − 1)∆κ(v′ − b∗)]≥ κi′

[v −

(b∗ +

1

m

)]=⇒ κi′′(v

′ − b(i′′+1)) ≥ κi′(v′ − b∗) + (i′′ − i′ − 1)∆κ(v′ − b∗)− (i′′ − i′ + 1)κi′

m,

and therefore,

κi′′(v′ − b(i′′+1)) ≥ κi′(v′ − b∗) + (i′′ − i′ − 1)∆κ∆v − (i′′ − i′ + 1)

κ1

m

= κi′(v′ − b∗) + (i′′ − i′ − 1)

(∆κ∆v −

κ1

m

)− 2κ1

m

≥ κi′(v′ − b∗)−2κ1

m, (4)

where the first step uses κi′ ≤ κ1 and v′ − b∗ ≥ ∆v, which follows from v′ ≥ v′′ ≥ b∗; the secondstep is algebra; and the third step uses m > 4κ1

∆κ∆v. Because the bidder with value v′′ does not find

it profitable to deviate to b∗ − 1m ,

1

i′′ − i′ + 1

[κi′′(v

′′ − b(i′′+1)) + (i′′ − i′)κi′(v′′ − b∗)− (i′′ − i′ − 1)∆κ(v′′ − b∗)]≥ κi′′(v′′ − b(i

′′+1))

=⇒ κi′(v′′ − b∗) ≥ κi′′(v′′ − b(i

′′+1)) +(i′′ − i′ − 1)

i′′ − i′∆κ(v′′ − b∗),

and therefore,

κi′(v′′ − b∗) ≥ κi′′(v′′ − b(i

′′+1)), (5)

because v′′ ≥ b∗. Adding (4) and (5), then canceling like terms, we obtain

(κi′ − κi′′)(v′′ − v′) ≥ −2κ1

m,

but this is a contradiction because κi′ − κi′′ ≥ ∆κ, v′′ − v′ ≤ −∆v, and m > 4κ1∆κ∆v

.

28


B Omitted Proofs

B.1 Proofs Corresponding to Section 3

Proof of Theorem 2. Fix any α ∈ RN++ and any ε ∈ (0, 1). Let M =∑

n∈N |Sn| and δ = εM

M .For each player n ∈ N , define ∆∗n = {σn ∈ ∆n | (∀sn ∈ Sn)σn(sn) ≥ δ}, which is a nonempty andcompact subset of ∆0

n. We also define the correspondence F :∏n∈N ∆∗n ⇒

∏n∈N ∆∗n as

F (σ) =

σ ∈ ∏n∈N

∆∗n

(∀l ∈ N )(∀m ∈ N )(∀s′l ∈ Sl)(∀s′′m ∈ S′′m)if αlLl(σ/s

′l) > αmLm(σ/s′′m)

then σl(s′l) ≤ ε · σm(s′′m)

.

Claim: For all σ ∈∏n∈N ∆∗n, F (σ) is closed, convex, and nonempty.

Proof of Claim: The points in F (σ) are those that satisfy a finite collection of linear inequalities,so F (σ) is a closed, convex set. We next demonstrate that F (σ) is nonempty. For all n ∈ N andall sn ∈ Sn, define

φn(sn) = #{m ∈ N , s′m ∈ Sm | αnLn(σ/sn) > αmLm(σ/s′m)}.

Define also Φ0n = #{sn ∈ Sn |φn(sn) = 0} and

σn(sn) =

εφn(sn)

Mif φn(sn) > 0

1

Φ0n

1−∑

sn:φn(sn)>0

εφn(sn)

M

if φn(sn) = 0

To verify that σ ∈∏n∈N ∆∗n, let n ∈ N and sn ∈ Sn. If φn(sn) > 0, then

σn(sn) =εφn(sn)

M≥ εM

M= δ.

And if φn(sn) = 0, then we also have

σn(sn) =1

Φ0n

1−∑

sn:φn(sn)>0

εφn(sn)

M

≥ 1

Φ0n

(1− (M − Φ0

n)1

M

)=

1

M≥ δ.

To verify that σ ∈ F (σ), suppose l,m ∈ N , s′l ∈ Sl, and s′′m ∈ Sm are such that αlLl(σ/s′l) >

αmLm(σ/s′′m). This implies that φl(s′l) ≥ φm(s′′m) + 1. If φm(s′′m) > 0, then

σl(s′l) =

εφl(s′l)

M≤ εφm(s′′m)+1

M= ε · σm(s′′m).

And if φm(s′′m) = 0, then we also have

σl(s′l) =

εφl(s′l)

M≤ ε

M≤ ε · σm(s′′m),

where the last step uses the fact, established above, that φm(s′′m) = 0 implies σm(s′′m) ≥ 1M .

29


Claim: F is upper-hemicontinuous.

Proof of Claim: This follows from continuity of Ln(·) for each n ∈ N .

Claim: There exists an (α, ε)-extended proper equilibrium.

Proof of Claim: By the above claims, F satisfies all the conditions of the Kakutani Fixed PointTheorem (Kakutani, 1941), so there exists some σε ∈

∏n∈N ∆∗n such that σε ∈ F (σε). It follows

from the definition of F that this fixed point is an (α, ε)-extended proper equilibrium.

Claim: There exists an extended proper equilibrium.

Proof of Claim: For all t ∈ N, define εt = 1t+1 . Applying the conclusion of the previous claim,

there exists a sequence (σt)∞t=1, where each σt is an (α, εt)-extended proper equilibrium. Because∏n∈N ∆n is a compact set, there exists a convergent subsequence, the limit of which is an extended

proper equilibrium.

Proof of Theorem 3. Let σ be a proper equilibrium. Using the characterization of proper equi-librium given by Proposition 8 of BBD (and restated as Proposition 7 of this paper), there existssome LPS ρ = (p1, . . . , pK) on S1×S2 that satisfies strong independence, has full support, respectswithin-player preferences, and for which (ρ, σ) is a lexicographic Nash equilibrium. For n ∈ {1, 2}and k ∈ {1, . . . ,K}, define

ζkn = {sn ∈ Sn : pkn(sn) > 0 and pjn(sn) = 0 for j < k}.

Because ρ has full support, {ζ1n, . . . , ζ

Kn } is a partition of Sn. Because ρ respects within-player

preferences, (∀n ∈ {1, 2})(∀k ∈ {1, . . . ,K})(∀s′n ∈ ζkn)(∀s′′n ∈ ∪Kj=kζjn):[

πn(s′n|pk)]Kk=1≥L

[πn(s′′n|pk)

]Kk=1

.

That is, each s′n ∈ ζkn is optimal for player n against ρ among strategies in ∪Kj=kζjn.

Let δ be as in Lemma 12. Fix any ε ∈ (0, δ) and any α ∈ R2++. Define R = [ε2K2−3K+2, ε]K−1.

Let r ∈ R. Because ε < δ, we have, by Lemma 12, that (∀n ∈ {1, 2})(∀k ∈ {1, . . . ,K})(∀s′n ∈ζkn)(∀s′′n ∈ ∪Kj=kζ

jn): ∑

s∈S

(r�ρ)(s)πn(s/s′n) ≥∑s∈S

(r�ρ)(s)πn(s/s′′n).

In particular, if s′n, s′′n ∈ ζkn, then by symmetry we must have∑

s∈S

(r�ρ)(s)πn(s/s′n) =∑s∈S

(r�ρ)(s)πn(s/s′′n).

A consequence of the above is that there exist, for each player n ∈ {1, 2}, a nonincreasing sequenceπ1n(r) ≥ · · · ≥ πKn (r) such that (∀k ∈ {1, . . . ,K})(∀sn ∈ ζkn) :

∑s∈S(r�ρ)(s)πn(s/s′n) = πkn(r).

We then define Lkn(r) = αn[π1n(r) − πkn(r)], so that 0 = L1

n(r) ≤ · · · ≤ LKn (r). The interpretationof Lkn(r) is player n’s scaled loss (scaled according to αn) from using a strategy in ζkn when theopponent’s play is according to r�ρ rather than a best response (i.e., a strategy in ζ1

n).Using this notation, we define the correspondence F : R×R⇒ R×R by

F (r1, r2) =

(r1, r2) ∈ R×R(∀k′ ∈ {2, . . . ,K})(∀k′′ ∈ {1, . . . ,K}) :

if Lk′

1 (r2) > Lk′′

2 (r1) then r11 · · · r

k′−11 ≤ ε · r1

2 · · · rk′′−12

if Lk′

2 (r1) > Lk′′

1 (r2) then r12 · · · r

k′−12 ≤ ε · r1

1 · · · rk′′−11

.

30


To accommodate the case of k′′ = 1 in this definition, we use the convention that the empty productr1

1 · · · rk′′−11 equals one.

Claim: For all (r1, r2) ∈ R×R, F (r1, r2) is nonempty.

Proof of Claim: Fix (r1, r2). For all n ∈ {1, 2} and all k ∈ {1, . . . ,K}, define

φkn = #{m ∈ {1, 2}, j ∈ {1, . . . ,K} | Lkn(r−n) > Ljm(r−m)}.33

Note that, by construction, L1n(r−n) = 0, and so φ1

n = 0. Next, for all n ∈ {1, 2} and all k ∈{1, . . . ,K − 1}, define

rkn = ε1+(K−1)(φk+1n −φkn).

Because Lk+1n (r−n) ≥ Lkn(r−n), we have φk+1

n ≥ φkn, and so rkn ≤ ε. We also have φk+1n ≤ 2K − 1,

and so rkn = ε1+(K−1)(φk+1n −φkn) ≥ ε1+(K−1)(2K−1) = ε2K2−3K+2. From the previous two inequalities,

we conclude (r1, r2) ∈ R × R. Next, we verify that (r1, r2) ∈ F (r1, r2). To that end, it is withoutloss of generality to suppose k′ ∈ {2, . . . ,K} and k′′ ∈ {1, . . . ,K} are such that Lk

′1 (r2) > Lk

′′2 (r1).

This implies φk′

1 ≥ φk′′

2 + 1. We then have the following:

r11 · · · rk

′−11 =

k′−1∏k=1

ε1+(K−1)(φk+11 −φk1)

= εk′−1+(K−1)φk

′1

≤ εk′−1+(K−1)(φk′′

2 +1)

= ε · ε[k′−2+K−k′′]+[k′′−1+(K−1)φk′′

2 ]

≤ ε · εk′′−1+(K−1)φk′′

2

= ε ·k′′−1∏k=1

ε1+(K−1)(φk+12 −φk2)

= ε · r12 · · · rk

′′−12 ,

as required. In the above, the first and seventh (i.e., last) steps follow from the definition of rkn.The second and sixth steps use φ1

n = 0. The third step uses φk′

1 ≥ φk′′

2 + 1. The fourth step isalgebra. The fifth step uses k′ ≥ 2 and k′′ ≤ K.

Claim: For all (r1, r2) ∈ R×R, F (r1, r2) is contractible.

Proof of Claim: Fix (r1, r2). Fix some (r1, r2) ∈ F (r1, r2). Define H : F (r1, r2)× [0, 1]→ F (r1, r2)in the following way. For any r = (r1, r2) ∈ F (r1, r2), define

H(r, t) =

(((rk1)1−t(rk1)t

)K−1

k=1,(

(rk2)1−t(rk2)t)K−1

k=1

).

It follows from this definition that H is continuous, that H(r, 0) = r, and that H(r, 1) = r. Toestablish that H is a contracting homotopy and that F (r1, r2) is therefore contractible, it onlyremains to show that H(r, t) ∈ F (r1, r2).

Fix any r ∈ F (r1, r2) and t ∈ [0, 1]. Because r, r ∈ R × R, we have rkn, rkn ∈ [ε2K2−3K+2, ε] for

all n ∈ {1, 2} and k ∈ {1, . . . ,K − 1}. This implies (rkn)1−t(rkn)t ∈ [ε2K2−3K+2, ε], and therefore

33Throughout, r−n denotes the following: when n = 1 it denotes r2, and when n = 2 it denotes r1.

31


H(r, t) ∈ R×R. To finish the argument, it is without loss of generality to suppose k′ ∈ {2, . . . ,K}and k′′ ∈ {1, . . . ,K} are such that Lk

′1 (r2) > Lk

′′2 (r1). Because r, r ∈ F (r1, r2), we have r1

1 · · · rk′−11 ≤

ε · r12 · · · r

k′′−12 and r1

1 · · · rk′−11 ≤ ε · r1

2 · · · rk′′−12 We then have the following:(

(r11)1−t(r1

1)t)· · ·((rk′−1

1 )1−t(rk′−1

1 )t)

=(r1

1 · · · rk′−1

1

)1−t(r1

1 · · · rk′−1

1

)t≤(ε · r1

2 · · · rk′′−1

2

)1−t(ε · r1

2 · · · rk′′−1

2

)t= ε ·

((r1

2)1−t(r12)t)· · ·((rk′′−1

2 )1−t(rk′′−1

2 )t),

as required.

Claim: F is upper-hemicontinuous.

Proof of Claim: This follows from continuity of Lkn(·) for each n ∈ {1, 2} and k ∈ {1, . . . ,K}.Formally, let r(t) and r(t) be convergent sequences in R×R with limt→∞ r(t) = r, limt→∞ r(t) = r,and r(t) ∈ F (r(t)) for all t. We demonstrate that r ∈ F (r). It is without loss of generality tosuppose k′ ∈ {2, . . . ,K} and k′′ ∈ {1, . . . ,K} are such that Lk

′1 (r2) > Lk

′′2 (r1). By continuity,

we have Lk′

1 (r2(t)) > Lk′′

2 (r1(t)) for all sufficiently large t. Because r(t) ∈ F (r(t)), we therefore

have r11(t) · · · rk′−1

1 (t) ≤ ε · r12(t) · · · rk′′−1

2 (t) for all such t. Because limt→∞ r(t) = r, we obtain

r11 · · · r

k′−11 ≤ ε · r1

2 · · · rk′′−12 , as required.

Claim: There exists some (r1, r2) ∈ R×R such that (r1, r2) ∈ F (r1, r2).

Proof of Claim: R×R is a nonempty, compact, and convex subset of a Euclidean space. Moreover,the above claims establish that F satisfies all the conditions of Lemma 10, so F has a fixed point.

For purposes of the following claims, we define the following quantity in terms of the LPSρ = (p1, . . . , pK):

p0 = mink∈{1,...,K}

mins∈supp(pk)

pk(s)

and define γ =ε

(1− ε)p0.

Claim: If (r1, r2) ∈ F (r1, r2), then the strategy profile σ = ((r1�ρ)1, (r2�ρ)2) is an (α, γ)-extendedproper equilibrium.

Proof of Claim: That σ is totally mixed follows from the definition of R and the fact that ρ hasfull support. We next make a useful observation. Fix any n ∈ {1, 2} and any sn ∈ Sn. Let kbe such that sn ∈ ζkn. Then, observe that because this is a two-player game, player n has justa single opponent, so that both σ = ((r1�ρ)1, (r2�ρ)2) and r−n�ρ induce the same distributionof play over the opponent’s strategies. Then, given the above definition of Lkn(·), it follows thatαnLn(σ/sn) = Lkn(r−n).

To complete the argument, we consider the within-player and across-player restrictions sepa-rately. For the within-player case, it is without loss of generality to focus on player 1. First, supposes′1 ∈ S1 and s′′1 ∈ S1 are such that α1L1(σ/s′1) > α1L1(σ/s′′1). Let k′ and k′′ be such that s′1 ∈ ζk

′1

and s′′1 ∈ ζk′′

1 . We then have Lk′

1 (r2) > Lk′′

1 (r2). This requires k′ > k′′. Therefore, we have thefollowing:

σ1(s′1) = (r1�ρ)1(s′1)

≤ r11 · · · rk

′−11

≤ r11 · · · rk

′′−11 rk

′−11

32


≤ ε · r11 · · · rk

′′−11

= γ · r11 · · · rk

′′−11 (1− ε)p0

≤ γ · r11 · · · rk

′′−11 (1− rk′′1 )p0

≤ γ · (r1�ρ)1(s′′1)

= γ · σ1(s′′1),

as required. In the above, the first and eighth (i.e., last) steps follow from the definition of σ.The second and seventh steps follow from Lemma 11 and the definition of p0. The third step usesk′ > k′′. The fourth step uses rk

′−11 ≤ ε. The fifth step uses the definition of γ. The sixth step uses

rk′′

1 ≤ ε.For the across-player case, it is without loss of generality to suppose s′1 ∈ S1 and s′′2 ∈ S2 are

such that α1L1(σ/s′1) > α2L2(σ/s′′2). Let k′ and k′′ be such that s′1 ∈ ζk′

1 and s′′2 ∈ ζk′′

2 . We thenhave Lk

′1 (r2) > Lk

′′2 (r1). Note that this precludes the case of k′ = 1. Because (r1, r2) ∈ F (r1, r2),

we then have r11 · · · r

k′−11 ≤ ε ·r1

2 · · · rk′′−12 . Therefore, we have the following (where to accommodate

the accommodate the case of k′′ = K, the argument remains valid if we define rK2 = 0):

σ1(s′1) = (r1�ρ)1(s′1)

≤ r11 · · · rk

′−11

≤ ε · r12 · · · rk

′′−12

= γ · r12 · · · rk

′′−12 (1− ε)p0

≤ γ · r12 · · · rk

′′−12 (1− rk′′2 )p0

≤ γ · (r2�ρ)2(s′′2)

= γ · σ2(s′′2),

as required. The primary departure from the within-player case lies in the third step, where wenow use r1

1 · · · rk′−11 ≤ ε · r1

2 · · · rk′′−12 . As pointed out above, this is where the status of (r1, r2) as a

fixed point of F is relevant.

Claim: σ is an extended proper equilibrium.

Proof of Claim: For all t ∈ N, define εt = δt+1 and γt = εt

(1−εt)p0. Note that limt→∞ εt = limt→∞ γt =

0. By the previous claims, there exists a sequence (σt)∞t=1, where each σt is an (α, γt)-extendedproper equilibrium that can be written as σt = ((r1(t)�ρ)1, (r2(t)�ρ)2) for some (r1(t), r2(t)) ∈[ε2K2−3K+2t , εt]

K−1 × [ε2K2−3K+2t , εt]

K−1. For all n ∈ {1, 2} and all sn ∈ Sn, we therefore have

σtn(sn) = (rn(t)�ρ)n(sn) ≥ (1− r1n(t))p1

n(sn) ≥ (1− εt)p1n(sn).

We conclude that limt→∞ σt = (p1

1, p12), which by condition (ii) of Definition 6 must be σ. Conse-

quently, σ is an extended proper equilibrium.


Proof of Theorem 4. For the symmetric game Γ, we use π(σ′; σ) throughout this proof to denotethe expected payoff received from σ′ when all opponents play σ. Thus, letting σ′ and σ be,respectively, the projections of σ′ and σ, the expression for the payoff simplifies to

π(σ′; σ) =1

N

∑n∈N

πn(σ/σ′n).

33


Given a strategy profile σ in Γ, we refer to∏n∈N σn as the distribution over S induced by σ.

Necessity of Nash equilibrium. Suppose Γ′ is equivalent to Γ up to affine transformations of thepayoffs, and suppose σ is a symmetrically Nash strategy of the meta-game Γ′. Let σ be theprojection of σ, and suppose, by way of contradiction, that σ is not a Nash equilibrium of Γ. Thenσ is not a Nash equilibrium of Γ′ either. Thus, there exists some n ∈ N and some sn ∈ Sn suchthat π′n(σ/sn) > π′n(σ). Letting σ′ denote the distribution over S induced by σ/sn, this means that

π′(σ′; σ)− π′(σ; σ) =1

N

[π′n(σ/sn)− π′n(σ)

]> 0,

which contradicts (σ, . . . , σ) having been a Nash equilibrium of Γ′. We conclude that σ is a Nashequilibrium of Γ.

Sufficiency of Nash equilibrium. Let σ be a Nash equilibrium of Γ. Let σ be the distribution overS induced by σ. It suffices to show that σ is a symmetrically Nash strategy of Γ, the meta-versionof Γ itself. For any s = (s1, . . . , sN ),

π(s; σ)− π(σ; σ) =1

N

∑n∈N

[πn(σ/sn)− πn(σ)] .

Because σ is a Nash equilibrium of Γ, every element of the sum on the righthand side is nonpositive.We conclude that π(s; σ) ≤ π(σ; σ). Because the same argument can be made for all s, we concludethat (σ, . . . , σ) is a Nash equilibrium of Γ, so that σ is a symmetrically Nash strategy of Γ.

Necessity of perfect equilibrium. Suppose Γ′ is equivalent to Γ up to affine transformations of thepayoffs, and suppose σ is a symmetrically perfect strategy of Γ′. Equivalently, there exists asequence of positive numbers (εt)

∞t=1 converging to zero and a sequence of totally mixed strategies

(σt)∞t=1 converging to σ such that for all t, (σt, . . . , σt) is an εt-perfect equilibrium of Γ′. Let, forall t, σt be the projection of σt, and let σ be the projection of σ. Then (σt)∞t=1 is a sequence oftotally mixed strategy profiles in Γ′ converging to σ. We complete the proof by showing that, forall t, σt is an (|S|εt)-perfect equilibrium of Γ. To see this, suppose that n ∈ N and s′n, s

′′n ∈ Sn are

such that πn(σt/s′n) < πn(σt/s′′n). A corresponding inequality must hold in the equivalent game Γ′:π′n(σt/s′n) < π′n(σt/s′′n). For any s ∈ S, we then have

π′(s/s′′n; σt)− π′(s/s′n; σt) =1

N

[π′n(σt/s′′n)− π′n(σt/s′n)

]> 0.

Because (σt, . . . , σt) is an εt-perfect equilibrium of Γ′, we therefore have σt(s/s′n) ≤ εt. Since thesame argument can be made for all s ∈ S, and since σtn is the marginal of σt on Sn, we concludethat σtn(s′n) ≤ |S|εt, as required.

Sufficiency of perfect equilibrium. Let σ be a perfect equilibrium of Γ. Equivalently, there exists asequence of positive numbers (εt)

∞t=1 converging to zero and a sequence of totally mixed strategy

profiles (σt)∞t=1 converging to σ such that for all t, σt is an εt-perfect equilibrium of Γ. Let, forall t, σt be the distribution over S induced by σt, and let σ be the distribution over S inducedby σ. Then (σt)∞t=1 is a sequence of full support distributions converging to σ. It suffices to showthat σ is a symmetrically perfect strategy of Γ, the meta-version of Γ itself. This can be doneby showing that for all t, (σt, . . . , σt) is an εt-perfect equilibrium of Γ. Let s′ = (s′1, . . . , s

′N ) and

s′′ = (s′′1, . . . , s′′N ) be such that

0 < π(s′′; σt)− π(s′; σt) =1

N

∑n∈N

[πn(σt/s′′n)− πn(σt/s′n)

].

34


This requires there to exist some n ∈ N such that πn(σt/s′n) < πn(σt/s′′n). Because σt is an εt-perfect equilibrium of Γ, we have σtn(s′n) ≤ εt. Because σtn is the marginal of σt on Sn, we haveσt(s′) ≤ σtn(s′n), so that σt(s′) ≤ εt, as required.

Necessity of extended proper equilibrium. Suppose Γ′ is equivalent to Γ up to affine transformationsof the payoffs, and suppose σ is a symmetrically proper strategy of Γ′. Equivalently, there exists asequence of positive numbers (εt)

∞t=1 converging to zero and a sequence of totally mixed strategies

(σt)∞t=1 converging to σ such that for all t, (σt, . . . , σt) is an εt-proper equilibrium of Γ′. Let, forall t, σt be the projection of σt, and let σ be the projection of σ. Then (σt)∞t=1 is a sequence oftotally mixed strategy profiles in Γ′ converging to σ. Let α be such that Γ′ can be constructedfrom Γ by multiplying the utility function of each player n by αn and adding some constant. Wecomplete the proof by showing that, for all t, σt is an

(α, |S|εt

)-extended proper equilibrium of

Γ. To see this, suppose that for some players l and m, s′l ∈ Sl and s′′m ∈ Sm are such thatαlLl(σ

t/s′l) > αmLm(σt/s′′m). For all n ∈ N , select some s∗n ∈ BRn(σt). Define s′′ = s∗/s′′m. Thenfor all s ∈ S for which sl = s′l, it is the case that∑

n∈NαnLn(σt/sn) ≥ αlLl(σt/s′l) > αmLm(σt/s′′m) =

∑n∈N

αnLn(σt/s′′n).

Equivalently, π′(s; σt) < π′(s′′; σt). Because (σt, . . . , σt) is an εt-proper equilibrium of Γ′, wetherefore have σt(s) ≤ εtσt(s′′). Because σtm is the marginal of σt on Sm, we have σt(s′′) ≤ σtm(s′′m),so that we obtain

σt(s) ≤ εtσtm(s′′m).

Moreover, since we can make the above argument for each strategy profile s ∈ S for which sl = s′l,and since σtl is the marginal of σt on Sl, we conclude σtl (s

′l) ≤ |S|εtσtm(s′′m), as required.

Sufficiency of extended proper equilibrium. Let σ be an extended proper equilibrium of Γ. Equiva-lently, there exists an α ∈ RN++, a sequence of numbers (εt)

∞t=1 in the open unit interval converging

to zero, and a sequence of totally mixed strategy profiles (σt)∞t=1 converging to σ such that for all

t, σt is an (α, ε|S|+2t )-extended proper equilibrium of Γ. Let, for all t, φt be the distribution over S

induced by σt. For all t, we construct another distribution over S, σt, in the way described below.To foreshadow, we will subsequently establish (i) that σt has σt as its projection, and (ii) that forall sufficiently large values of t, (σt, . . . , σt) is an εt-proper equilibrium of the meta-version of agame that is equivalent to Γ up to an affine transformation of the payoffs.

To begin, let � be the partial order over S defined as follows (where its dependence on t issuppressed in the notation):

s′ � s′′ ⇐⇒∑n∈N

αnπn(σt/s′n) >∑n∈N

αnπn(σt/s′′n).

We say that s′ < s′′ if it is not the case that s′′ � s′. Next, select for all n ∈ N some s∗n ∈arg maxsn∈Sn σ

tn(sn). Note that because σt is an (α, ε

|S|+2t )-extended proper equilibrium with εt ∈

(0, 1), it must be the case that s∗n ∈ BRn(σt). It follows that s∗ is a maximal element under <(although perhaps not the unique maximal element).

Next, we partition the set of strategy profiles S into (i) s∗ itself, (ii) S1, which we define as theset of unilateral deviations from s∗, (iii) S2, which we define as the set of joint deviations from s∗

that are played relatively infrequently under σt, and (iv) S3, which we define as the set of joint

35


deviations from s∗ that are played relatively frequently under σt. Formally,

S1 ={s∗/sn : n ∈ N , sn ∈ Sn \ {s∗n}}S2 ={s ∈ S \ ({s∗} ∪ S1) : φt(s) < εtφ

t(s∗)}S3 ={s ∈ S \ ({s∗} ∪ S1) : φt(s) ≥ εtφt(s∗)}

Note that because σt is an (α, ε|S|+2t )-extended proper equilibrium, a strategy profile in which at

least one player uses an inferior response to σt can be played with probability at most ε|S|+2t φt(s∗).

Hence, the elements of S3 must all be maximal under <.To construct σt, the desired distribution over S, initialize σt(s) = φt(s). Then loop over the

elements s′ ∈ S2, at each step modifying σt in the following way:

• [Modify s′ itself:] At the step for s′ ∈ S2, select some s ∈ arg mins∈{s∗}∪S1:s<s′ φt(s) and let

` = #{s ∈ S2 : s � s′}. Set σt(s′)← ε2+`t φt(s).

• [Modify corresponding elements of S1:] For each n ∈ N for which s′n 6= s∗n, set σt(s∗/s′n) ←σt(s∗/s′n) + φt(s′)− σt(s′).

• [Modify s∗:] Set σt(s∗)← σt(s∗)−(#{n ∈ N : s′n 6= s∗n} − 1

) (φt(s′)− σt(s′)

).

In words, the step for s′ ∈ S2 alters the probability put on s′; but so as to keep the marginalsunchanged, the step also alters the probability on corresponding elements of S1 and on s∗. Theelements of S3 remain untouched throughout. Henceforth, let σt refer to its value after all steps ofthe loop have been completed, unless specified otherwise.

Claim: σt is the projection of σt.

Proof of Claim: Because σt is the projection of φt, it suffices to show that the step of the loopcorresponding to each s′ ∈ S2 does not alter the marginal distributions. For any m ∈ N and anysm ∈ Sm, we verify that σt assigns the same total probability to the strategy profiles involving smboth before and after the step. To do so, we consider separately the two cases in which s′m = s∗mand s′m 6= s∗m.

Suppose m ∈ N is such that s′m = s∗m. The step modifies σt by assigning φt(s′) − σt(s′) lessweight to one strategy profile involving s∗m (namely s′),

(#{n ∈ N : s′n 6= s∗n}− 1

) (φt(s′)− σt(s′)

)less weight to another strategy profile involving s∗m (namely s∗), as well as φt(s′) − σt(s′) moreweight to #{n ∈ N : s′n 6= s∗n} strategy profiles involving s∗m. Thus, the total weight on strategyprofiles involving s∗m remains unchanged. Moreover, the step does not adjust the weight that σt

places on any strategy profile not involving s∗m.Suppose m ∈ N is such that s′m 6= s∗m. The step modifies σt by assigning

(#{n ∈ N : s′n 6=

s∗n} − 1) (φt(s′)− σt(s′)

)less weight to one strategy profile involving s∗m (namely s∗), as well as

φt(s′) − σt(s′) more weight to #{n ∈ N : s′n 6= s∗n} − 1 strategy profiles involving s∗m. Thus, thetotal weight on strategy profiles involving s∗m remains unchanged. It also modifies σt by assigningφt(s′)− σt(s′) less weight to one strategy profile involving s′m (namely s′), as well as φt(s′)− σt(s′)more weight to another strategy profile involving s′m (namely s∗/s′m). Thus, the total weight onstrategy profiles involving s′m remains unchanged. Moreover, the step does not adjust the weightthat σt places on any strategy profile not involving either s∗m or s′m.

Claim: For all s ∈ S1, σt(s) ≤ |S|φt(s).Proof of Claim: To see this, note that every s ∈ S1 can be written as s = s∗/sm for some m ∈ Nand some sm ∈ Sm. Moreover, σt(s) is updated only at steps in the loop corresponding to s′ ∈ S2

36


where s′m = sm. At each such step, σt(s) is raised by at most φt(s′). Based on the previousdefinitions,

φt(s′) = σtm(sm)∏n6=m

σtn(s′n) ≤ σtm(sm)∏n6=m

σtn(s∗n) = φt(s).

Because σt(s) is initialized at φt(s) and because there are no more than |S| − 1 steps, we concludethat σt(s) ≤ |S|φt(s), as desired.

Claim: For all s ∈ S1, σt(s) ≥ (1− |S|ε2t )φ

t(s).

Proof of Claim: To see this, note that every s ∈ S1 can be written as s = s∗/sm for some m ∈ Nand some sm ∈ Sm. Moreover, σt(s) is updated only at steps in the loop corresponding to s′ ∈ S2

where s′m = sm. As previously observed, we have s∗n ∈ BRn(σt) for all n ∈ N . Thus,

αmπm(σt/sm) +∑n6=m

αnπn(σt/s∗n) ≥ αmπm(σt/sm) +∑n6=m

αnπn(σt/s′n),

so that s < s′. Then by construction, σt(s′) ≤ ε2tφ

t(s). At each such step, σt(s) is reduced byat most this amount. Because σt(s) is initialized at φt(s) and because there are no more than |S|steps, we conclude that σt(s) ≥ (1− |S|ε2

t )φt(s), as desired.

Claim: σt(s∗) ≥ (1−N |S|εt)φt(s∗).Proof of Claim: For all s′ ∈ S2, φt(s′) ≤ εtφt(s∗), by the definition of S2. Next, note that the stepof the loop corresponding to s′ ∈ S2 reduces σt(s∗) by no more than Nφt(s′). Because there are nomore than |S| steps, the conclusion follows.

Next, define T such that for all t ≥ T , both of the following hold. First, εt ≤ 1N |S| , which ensures

that σt(s) ≥ 0 for all s ∈ S, so that σt is a well-defined probability distribution over S. Second,

εt ≥ max

{|S|2ε|S|+2

t

1−N |S|εt,

ε2t

1−N |S|εt, |S|2ε2

t

},

which we use in the arguments below. Furthermore, let Γ′ be the game that is constructed fromΓ by multiplying the utility function of each player n by αn. Let Γ′ be the meta-version of Γ′.Because the projection of σt is σt, the payoff of some strategy s = (s1, . . . , sN ) in Γ′ when all otherplayers play according to σt is

π′(s; σt) =1

N

∑n∈N

αnπn(σt/sn).

Claim: For all t ≥ T , (σt, . . . , σt) is an εt-proper equilibrium of Γ′.34

Proof of Claim: Suppose that t ≥ T . Suppose that s′ = (s′1, . . . , s′N ) and s′′ = (s′′1, . . . , s

′′N ) are such

that π′(s′; σt) > π′(s′′; σt). Equivalently, 1N

∑n∈N αnπn(σt/s′n) > 1

N

∑n∈N αnπn(σt/s′′n), which

is also equivalent to s′ � s′′. We will show that σt(s′′) ≤ εtσt(s′), which in turn implies that

(σt, . . . , σt) is an εt-proper equilibrium of Γ′. To establish this, we consider eight cases, which aredistinguished by how s′ and s′′ fit into the partition {{s∗},S1,S2,S3}. Note that because s∗ andthe elements of S3 are maximal under <, we can have only s′′ ∈ S2 ∪ S3, which eliminates somepotential cases.

34Because Γ′ is a symmetric game and because the strategy profile in question is symmetric, we will have alsoshown that (σt, . . . , σt) is an (α, εt)-extended proper equilibrium with α = (1, . . . , 1).

37


Case 1: s′ = s∗ and s′′ ∈ S1. Note that for some player m, we can write s′′ = s∗/s′′m. Thus, s∗ � s′′

implies αmLm(σt/s∗m) < αmLm(σt/s′′m). Because σt is an (α, ε|S|+2t )-extended proper equilibrium

of Γ, we have that σtm(s′′m) ≤ ε|S|+2t σtm(s∗m). Because σtm is the marginal of φt on Sm, we also have

φt(s′′) ≤ σtm(s′′m). We also have σtm(s∗m) = φt(s∗)∏n6=m σtn(s∗n) ≤

(∏n 6=m |Sn|

)φt(s∗) ≤ |S|φt(s∗), where

the first inequality follows from s∗n ∈ arg maxsn∈Sn σtn(sn) for all n ∈ N . Further, as derived above,

σt(s′′) ≤ |S|φt(s′′) and σt(s∗) ≥ (1−N |S|εt)φt(s∗). Combining these inequalities:

σt(s′′) ≤ |S|φt(s′′) ≤ |S|σtm(s′′m) ≤ |S|ε|S|+2t σtm(s∗m) ≤ |S|2ε|S|+2

t φt(s∗) ≤ |S|2ε|S|+2t

1−N |S|εtσt(s∗).

From the definition of T , we indeed obtain σt(s′′) ≤ εtσt(s∗) for all t ≥ T .

Case 2: s′ ∈ S1 and s′′ ∈ S1. Note that for some players l and m, we can write s′ = s∗/s′land s′′ = s∗/s′′m. As previously observed, we have s∗n ∈ BRn(σt) for all n ∈ N , and so s′ � s′′

implies αlLl(σt/s′l) < αmLm(σt/s′′m). Because σt is an (α, ε

|S|+2t )-extended proper equilibrium of

Γ, we have that σtm(s′′m) ≤ ε|S|+2t σtl (s

′l). Because σtm is the marginal of φt on Sm, we also have

φt(s′′) ≤ σtm(s′′m). We also have σtl (s′l) = φt(s′)∏

n 6=l σtn(s∗n) ≤

(∏n6=l |Sn|

)φt(s′) ≤ |S|φt(s′), where the

first inequality follows from s∗n ∈ arg maxsn∈Sn σtn(sn) for all n ∈ N . Further, as derived above,

σt(s′′) ≤ |S|φt(s′′) and σt(s′) ≥ (1− |S|ε2t )φ

t(s′). Combining these inequalities:

σt(s′′) ≤ |S|φt(s′′) ≤ |S|σtm(s′′m) ≤ |S|ε|S|+2t σtl (s

′l) ≤ |S|2ε

|S|+2t φt(s′) ≤ |S|

2ε|S|+2t

1− |S|ε2t

σt(s′).

Note that|S|2ε|S|+2

t

1−|S|ε2t≤ |S|

2ε|S|+2t

1−N |S|εt. Thus, from the definition of T , we indeed obtain σt(s′′) ≤ εtσ

t(s′)

for all t ≥ T .

Case 3: s′ ∈ S1 and s′′ ∈ S2. Its construction implies σt(s′′) ≤ ε2tφ

t(s′). Further, as derived above,σt(s′) ≥ (1− |S|ε2

t )φt(s′). Combining these inequalities:

σt(s′′) ≤ ε2tφ

t(s′) ≤ ε2t

1− |S|ε2t

σt(s′).

Note thatε2t

1−|S|ε2t≤ ε2t

1−N |S|εt. Thus, from the definition of T , we indeed obtain σt(s′′) ≤ εtσt(s′) for

all t ≥ T .

Case 4: s′ ∈ S2 and s′′ ∈ S1. Let s ∈ S1 be as specified above for the step of the loop involving s′.

By its construction, σt(s′) ≥ ε|S|t φt(s). We also have s < s′ � s′′. Note that for some players l andm,we can write s = s∗/sl and s′′ = s∗/s′′m. As previously observed, we have s∗n ∈ BRn(σt) for all n ∈ N ,

and so s � s′′ implies αlLl(σt/sl) < αmLm(σt/s′′m). Because σt is an (α, ε

|S|+2t )-extended proper

equilibrium of Γ, we have that σtm(s′′m) ≤ ε|S|+2t σtl (sl). Because σtm is the marginal of φt on Sm,

we also have φt(s′′) ≤ σtm(s′′m). We also have σtl (sl) = φt(s)∏n 6=l σ

tn(s∗n) ≤

(∏n6=l |Sn|

)φt(s) ≤ |S|φt(s),

where the first inequality follows from s∗n ∈ arg maxsn∈Sn σtn(sn) for all n ∈ N . Further, as derived

above, σt(s′′) ≤ |S|φt(s′′). Combining these inequalities:

σt(s′′) ≤ |S|φt(s′′) ≤ |S|σtm(s′′m) ≤ |S|ε|S|+2t σtl (sl) ≤ |S|2ε

|S|+2t φt(s) ≤ |S|2ε2

t σt(s′).

From the definition of T , we indeed obtain σt(s′′) ≤ εtσt(s′) for all t ≥ T .

38


Case 5: s′ ∈ S2 and s′′ ∈ S2. Let (s′, `′) and (s′′, `′′) be as specified for the steps of the loopcorresponding to s′ and s′′, respectively. Because s′ � s′′, we have φt(s′) ≥ φt(s′′) and `′′ ≥ `′ + 1.Then

σt(s′′) = ε2+`′′

t φt(s′′) ≥ ε3+`′

t φt(s′′) = εtε2+`′

t φt(s′) = εtσt(s′).

Case 6: s′ = s∗ and s′′ ∈ S2. By its construction, σt(s′′) ≤ ε2tφ

t(s∗). As derived above, σt(s∗) ≥(1−N |S|εt)φt(s∗). Combining these inequalities:


t(s∗) ≤ ε2t

1−N |S|εtσt(s∗)

Thus, from the definition of T , we indeed obtain σt(s′′) ≤ εtσt(s′) for all t ≥ T .

Case 7: s′ ∈ S3 and s′′ ∈ S1. We can argue as in case 1 that σt(s′′) ≤ |S|2ε|S|+2t φt(s∗). Because

s′ ∈ S3, σt(s′) = φt(s′) ≥ εtφt(s∗). Combining these inequalities:

σt(s′′) ≤ |S|2ε|S|+2t φt(s∗) ≤ |S|2ε|S|+1

t σt(s′).

Note that |S|2ε|S|+1t ≤ |S|2ε2

t . Thus, from the definition of T , we indeed obtain σt(s′′) ≤ εtσt(s′)

for all t ≥ T .

Case 8: s′ ∈ S3 and s′′ ∈ S2. By its construction, σt(s′′) ≤ ε2tφ

t(s∗). Because s′ ∈ S3, σt(s′) =φt(s′) ≥ εtφt(s∗). Combining these inequalities:


t(s∗) ≤ εtσt(s′).

Claim: σ is the projection of a symmetrically proper strategy of Γ′.

Proof of Claim: Because∏n∈N ∆n is compact, (σt)∞t=T has a convergent subsequence, the limit of

which is therefore a symmetrically proper strategy of Γ′.35 Furthermore, the marginals of each σt

are given by the profile σt. Therefore, the projection of the aforementioned limit equals limt→∞ σt =

σ.


Proof of Proposition 5. Sufficiency. Suppose (ρ, σ) is a lexicographic Nash equilibrium, whereρ = (p1, . . . , pK) is an LPS that satisfies strong independence. For a player n ∈ N and a strategys′n ∈ Sn, if p1

n(s′n) > 0 then by condition (i) of Definition 6, for all s′′n ∈ Sn,∑s∈S

p1(s)πn(s/s′n) ≥∑s∈S

p1(s)πn(s/s′′n).

By condition (ii) of Definition 6, σ = (p11, . . . , p

1N ). Therefore, we can rephrase the above in the

following terms: if σn(s′n) > 0 then for all s′′n ∈ Sn, πn(σ/s′n) ≥ πn(σ/s′′n). Thus, σ is a Nashequilibrium.

Necessity. Suppose σ = (σ1, . . . , σN ) is a Nash equilibrium. Let K = 1 and define p1 =∏n∈N σn

as the distribution over S induced by σ. This produces an LPS ρ such that (ρ, σ) is a lexicographicNash equilibrium. Furthermore, p1 is by definition a product distribution. Because K = 1, ρtherefore satisfies strong independence.

35Following footnote 34, this limit is in fact a symmetrically extended proper strategy of Γ′, where the italicizedterm is defined in a way analogous to Definition 4.

39


Proof of Proposition 6. This is a restatement of Proposition 7 from BBD.

Proof of Proposition 7. This is a restatement of Proposition 8 from BBD.

Proof of Proposition 8. Sufficiency. Suppose (ρ, σ) is a lexicographic Nash equilibrium, whereρ = (p1, . . . , pK) is an LPS that satisfies strong independence, has full support, and respects within-and-across-player preferences. As BBD point out, ρ satisfies strong independence if and only if thereexists a sequence r(t) ∈ (0, 1)K−1 with r(t) → 0 such that r(t)�ρ is a product distribution for allt. For each n ∈ N , define σn(t) as the marginal of r(t)�ρ on Sn, and let σ(t) = (σ1(t), . . . , σN (t)).Note that limt→∞ σ(t) = (p1

1, . . . , p1N ), which by condition (ii) of Definition 6 is equal to σ. Also

define

p0 = mink∈{1,...,K}

mins∈supp(pk)

pk(s)

ε(t) = maxk∈{1,...,K−1}

{rk(t)

[1− rk(t)]p0

}Because r(t)→ 0, we also have ε(t)→ 0. Letting δ be as in the statement of Lemma 12, let T besuch that for all t ≥ T , r(t) ∈ (0, δ)K−1. In addition, let α ∈ RN++ be as in the definition of “respectswithin-and-across-player preferences.” We claim that for all t ≥ T , σ(t) is an (α, ε(t))-extendedproper equilibrium.

Fix some t ≥ T . From the fact that ρ has full support, we obtain for all n ∈ N thatσn(t) ∈ ∆0

n. Furthermore, suppose l,m ∈ N , s′l ∈ Sl, and s′′m ∈ Sm are such that αlLl(σ(t)/s′l) >αmLm(σ(t)/s′′m). Equivalently,

αl maxsl∈Sl

[πl(σ(t)/sl)− πl(σ(t)/s′l)] > αm maxsm∈Sm

[πm(σ(t)/sm)− πm(σ(t)/s′′m)].

Because r(t)�ρ is a product distribution with marginals σ1(t), . . . , σN (t), the above inequality isequivalent to

αl maxsl∈Sl

∑s∈S


∑s∈S


By Lemma 12, we obtain that for all s∗l ∈ BRl(ρ) and all s∗m ∈ BRm(ρ),[αl

(πl(s


)]Kk=1

>L

[αm


)]Kk=1

,

which, because ρ respects within-and-across-player preferences, implies that s′l <ρ s′′m. Definek′′ = min{k : pkm(s′′m) > 0}. Because ρ has full support, such a k′′ must exist. We then haveσm(t)(s′′m) ≥ r1(t) · · · rk′′−1(t)[1 − rk

′′(t)]p0, by Lemma 11. Because s′l <ρ s′′m, we also have

σl(t)(s′l) ≤ r1(t) · · · rk′′(t) by Lemma 11. Combining this with the definition of ε(t) we have

ε(t)σm(t)(s′′m) ≥ ε(t)r1(t) · · · rk′′−1(t)[1− rk′′(t)]p0

≥ rk′′(t)

[1− rk′′(t)]p0r1(t) · · · rk′′−1(t)[1− rk′′(t)]p0

= r1(t) · · · rk′′(t)≥ σl(t)(s′l).

In review, we have shown for all t ≥ T that σ(t) is a totally mixed strategy profile andαlLl(σ(t)/s′l) > αmLm(σ(t)/s′′m) implies σl(t)(s

′l) ≤ ε(t)σm(t)(s′′m). Consequently, σ(t) is an

40


(α, ε(t))-extended proper equilibrium. Because ε(t) → 0 and σ(t) → σ, σ is an extended properequilibrium.

Necessity. Suppose σ = (σ1, . . . , σN ) is an extended proper equilibrium. Then there exists ascaling vector α ∈ RN++, a sequence of positive numbers ε(t) converging to zero, and a sequence oftotally mixed strategy profiles σ(t) converging to σ where each σ(t) is an (α, ε(t))-extended properequilibrium. Let σ(t) =

∏n∈N σn(t) be the distribution over S induced by σ. By Proposition 2 of

BBD, there is an LPS ρ = (p1, . . . , pK) on S such that a subsequence σ(τ) of σ(t) can be writtenas σ(τ) = r(τ)�ρ for a sequence r(τ) ∈ (0, 1)K−1 with r(τ) → 0. We claim that ρ satisfies strongindependence, has full support, respects within-and-across-player comparisons, and is such that(ρ, σ) meets condition (ii) of Definition 6.

First, condition (ii) of Definition 6 is satisfied, since p1 = limτ→∞ r(τ)�ρ = limτ→∞ σ(τ) =limτ→∞

∏n∈N σn(τ) =

∏n∈N σn. Second, since each σ(τ) is totally mixed, each σ(τ) has full

support, and therefore ρ must have full support. Third, the definition of ρ implies that, for allτ , r(τ)�ρ is a product distribution; moreover, r(τ) → 0. BBD point out that these facts implythat ρ satisfies strong independence. Finally, we will show that ρ respects within-and-across-playerpreferences (with the same choice of α). Suppose that l,m ∈ N , s′l ∈ Sl, s′′m ∈ Sm, s∗l ∈ BRl(ρ),and s∗m ∈ BRm(ρ) are such that[

αl

(πl(s


)]Kk=1

>L

[αm


)]Kk=1

. (6)

Letting δ be as in the statement of Lemma 12, let T be such that for all τ ≥ T , r(τ) ∈ (0, δ)K−1.By Lemma 12, for all τ ≥ T ,

αl maxsl∈Sl

∑s∈S

(r(τ)�ρ)(s)[πl(s/sl)− πl(s/s′l)] > αm maxsm∈Sm

∑s∈S

(r(τ)�ρ)(s)[πm(s/sm)− πm(s/s′′m)].

Because r(τ)�ρ is a product distribution with marginals σ1(τ), . . . , σN (τ), the above inequality isequivalent to

αl maxsl∈Sl

[πl(σ(τ)/sl)− πl(σ(τ)/s′l)] > αm maxsm∈Sm

[πm(σ(τ)/sm)− πm(σ(τ)/s′′m)],

or, equivalently,

αlLl(σ(τ)/s′l) > αmLm(σ(τ)/s′′m). (7)

Define k′ = min{k : pkl (s′l) > 0}. Because ρ has full support, k′ must exist. To accommodate the

case of k′ = K, all that follows remains valid if we define rK(τ) = 0 for all τ . Because r(τ) → 0and ε(τ) → 0, there exists some τ∗ ≥ T for which ε(τ∗) < [1 − rk′(τ∗)]pk′l (s′l). Using Lemma 11and then this inequality,

σl(τ∗)(s′l) ≥ r1(τ∗) · · · rk′−1(τ∗)[1− rk′(τ∗)]pk′l (s′l)

> ε(τ∗)r1(τ∗) · · · rk′−1(τ∗).

Suppose by way of contradiction that s′′m ≤ρ s′l, so that min{k : pkm(s′′m) > 0} ≥ k′, and thusσm(τ∗)(s′′m) ≤ r1(τ∗) · · · rk′−1(τ∗) by Lemma 11. Then we would have σl(τ

∗)(s′l) > ε(τ∗)σm(s′′m).That, taken together with (7), would contradict the fact that σ(τ∗) is an (α, ε(τ∗))-extended properequilibrium. It must therefore be the case that s′l <ρ s

′′m. In review, we have shown that equation (6)

implies that s′l <ρ s′′m, which establishes that ρ respects within-and-across-player preferences.

41



Proof of Proposition 9. Assume M > v1, and assume m > 4κ1∆κ∆v

, as in footnote 25. Let b∗ be

an extended proper equilibrium.36 By Proposition 8, there exists an LPS ρ that satisfies strongindependence, has full support, and respects within-and-across-player preferences for which (ρ, b∗)is a lexicographic Nash equilibrium. Suppose, by way of contradiction, that b∗ is not locally envy-free*. Let i′ be the largest index for which the locally envy-free* inequality is violated:

κi′[vg(i′) − b∗g(i′+1)

]< κi′−1

[vg(i′) −

(b∗g(i′) +

1

m

)]. (8)

The proof consists of two parts. First, we demonstrate that bidders who bid lower than g(i′) aresorted by their values for clicks. We then argue that, against ρ, bidder g(i′) prefers the deviationb∗g(i′) + 1

m over its equilibrium bid b∗g(i′).

Part 1: Let i ∈ {i′ + 1, . . . , N}. Because i′ is the largest index for which the locally envy-free*inequality is violated,

κi

[vg(i) − b∗g(i+1)

]≥ κi−1

[vg(i) −

(b∗g(i) +

1

m

)].

Furthermore, equilibrium requires that bidder g(i− 1) cannot profit by deviating to b∗g(i). Thus:

κi−1

[vg(i−1) − b∗g(i)

]≥ κi

[vg(i−1) − b∗g(i+1)

].

Manipulating these inequalities yields

(κi−1 − κi)(vg(i−1) − vg(i)

)+κi−1

m≥ 0

We have labeled positions so that κi−1 > κi. Thus, if vg(i−1) < vg(i), we would obtain a contradic-

tion: the left-hand side would be bounded above by −∆κ∆v + κ1m , which, because m > 4κ1

∆κ∆v, is

negative. We conclude that vg(i−1) > vg(i).

Part 2: Suppose bidder g(i′) contemplates raising its bid from b∗g(i′) to b∗g(i′)+1m . This does not make

a difference against the equilibrium profile b∗ for the following reason. It could make a differenceonly if b∗g(i′−1) = b∗g(i′) + 1

m . But we cannot have b∗g(i′−1) = b∗g(i′) + 1m , for otherwise equation (8)

would imply that b∗g(i′) + 1m would be a profitable deviation from the equilibrium for bidder g(i′).

Rather, this deviation makes a difference only if at least one of bidder g(i′)’s opponents trembleswithin the set {b∗g(i′), b

∗g(i′) + 1

m}. Thus, the relevant profiles can be partitioned in the following way:

• Case 1(a): bg(i′′) = b∗g(i′) + 1m for some i′′ < i′ and bg(i) = b∗g(i) for all i /∈ {i′, i′′}

• Case 1(b): bg(i′′) = b∗g(i′) for some i′′ < i′ and bg(i) = b∗g(i) for all i /∈ {i′, i′′}

• Case 2(a): bg(i′+1) = b∗g(i′) + 1m and bg(i) = b∗g(i) for all i /∈ {i′, i′ + 1}

• Case 2(b): bg(i′+1) = b∗g(i′) and bg(i) = b∗g(i) for all i /∈ {i′, i′ + 1}

• Case 3(a): bg(i′′) = b∗g(i′) + 1m for some i′′ > i′ + 1 and bg(i) = b∗g(i) for all i /∈ {i′, i′′}

• Case 3(b): bg(i′′) = b∗g(i′) for some i′′ > i′ + 1 and bg(i) = b∗g(i) for all i /∈ {i′, i′′}

36Throughout the proof, we use Lemma 17, which says that for all i ∈ {1, . . . , N − 1}, b(i) > b(i+1). It follows thatfor all i ∈ {1, . . . , N}, it is well defined to let g(i) denote the identity of the ith highest bidder.

42


• Case 4: bg(i′′) ∈ {b∗g(i′), b∗g(i′) + 1

m} for some i′′ 6= i′ and bg(i) 6= b∗g(i) for some i /∈ {i′, i′′}We then observe the following points. Together, they show that for each of these relevant profiles:either (i) bidder g(i′) is strictly better off deviating to b∗g(i′) + 1

m than playing b∗g(i′) against that

profile, or (ii) that profile is infinitely less likely under ρ than another relevant profile against whichbidder g(i′) is strictly better off deviating to b∗g(i′) + 1

m .

• Case 4: Each profile in case 4 corresponds to a profile in one of the other cases in which strictlyfewer bidders are deviating (i.e., the profile in which g(i′′) is the only deviator). Because ρsatisfies strong independence, it follows that the former must be infinitely less likely under ρthan the latter.37

• Cases 1(a) and 1(b): Against each profile in case 1(a), bidder g(i′) is strictly better offdeviating to b∗g(i′) + 1

m . Indeed, without deviating, bidder g(i′) receives its equilibrium payoff

κi′[vg(i′) − b∗g(i′+1)

]. Deviating to b∗g(i′) + 1

m , bidder g(i′) receives a uniform lottery between

that and κi′−1

[vg(i′) −

(b∗g(i′) + 1

m

)], which is larger by equation (8). A similar argument

establishes the same conclusion for each profile in case 1(b).

A special case is the profile in case 1(a) for which i′′ = i′ − 1. Clearly, b∗g(i′) + 1m is a best

response to the equilibrium b∗ for bidder g(i′ − 1). Therefore, this profile has the followingfeature: only a single bidder is deviating from b∗ and that deviation is to a best response tob∗.

• Cases 3(a) and 3(b): For all i′′ > i′+1, b∗g(i′)+1m is an inferior response to the equilibrium b∗ for

bidder g(i′′). Indeed, bidder g(i′′)’s equilibrium payoff is κi′′[vg(i′′) − b∗g(i′′+1)

]. By deviating

from the equilibrium to b∗g(i′) + 1m , bidder g(i′′) receives the payoff κi′

[vg(i′′) − b∗g(i′)

]. Because

i′ was defined as the largest index for which the locally envy-free* inequality is violated, wehave

κi′′[vg(i′′) − b∗g(i′′+1)

]≥ κi′′−1

[vg(i′′) −

(b∗g(i′′) +

1

m

)](9)

∀i ∈ {i′ + 1, . . . , i′′ − 1} : κi

[vg(i) − b∗g(i+1)

]≥ κi−1

[vg(i) −

(b∗g(i) +

1

m

)](10)

From part 1 of this proof, we have that i ∈ {i′+ 1, . . . , i′′− 1}, vg(i) ≥ vg(i′′) + ∆v. Combiningthat with (10), we obtain

∀i ∈ {i′ + 1, . . . , i′′ − 1} : κi

[vg(i′′) − b∗g(i+1)

]≥ κi−1

[vg(i′′) −

(b∗g(i) +

1

m

)]+ ∆κ∆v

≥ κi−1

[vg(i′′) − b∗g(i)

]+ ∆κ∆v −

κ1

m(11)

Summing (9) and (11), then canceling like terms, we obtain

κi′′[vg(i′′) − b∗g(i′′+1)

]≥ κi′

[vg(i′′) − b∗g(i′+1)

]+ (i′′ − i′ − 1)

(∆κ∆v −

2κ1

m

)+ (i′′ − i′ − 2)

κ1

m

37This can be shown formally using Lemmas 14 and 16. Let b′ denote a profile in case 4. Let b′′ denote anycorresponding profile in one of the other cases in which strictly fewer bidders are deviating. Let g(i′′) denote thedeviating bidder in b′′. Let g(i′′′) denote some bidder who deviates in b′ but not in b′′. Then b′′g(i′′′) >ρ b

′g(i′′′) by

Lemma 14(ii). Because b′′g(i′′) = b′g(i′′), we also have b′′g(i′′) =ρ b′g(i′′). And we have b′′g(i) ≥ρ b′g(i) for all i /∈ {i′′, i′′′}

by Lemma 14(i). Combining all this and applying Lemma 16, we have b′′ >ρ b′.

43


Using b∗g(i′+1) < b∗g(i′), i′′ ≥ i′ + 2, and m > 4κ1

∆κ∆v, we obtain

κi′′[vg(i′′) − b∗g(i′′+1)

]> κi′

[vg(i′′) − b∗g(i′)

],

as desired. Because ρ satisfies strong independence and respects within-and-across-playerpreferences, it follows that each profile in case 3(a) is infinitely less likely under ρ thanthe special case mentioned in the previous bullet (i.e., the profile in case 1(a) for whichi′′ = i′−1).38 A similar argument establishes the same conclusion for each profile in case 3(b).

• Cases 2(a) and 2(b): For bidder g(i′ + 1), b∗g(i′) + 1m might be either an inferior response to

the equilibrium b∗ or a best response to b∗. In the case of an inferior response, the profile incase 2(a) can be handled in the same way as the profiles in case 3(a), which were discussedin the previous bullet. It remains to consider the case where b∗g(i′) + 1

m is a best response tob∗ for the deviating bidder bg(i′+1).

Bidder g(i′ + 1)’s equilibrium payoff is κi′+1

[vg(i′+1) − b∗g(i′+2)

]. By deviating from the equi-

librium to b∗g(i′) + 1m , bidder g(i′ + 1) receives the payoff κi′

[vg(i′+1) − b∗g(i′)

]. We therefore

proceed under the assumption that

κi′+1

[vg(i′+1) − b∗g(i′+2)

]= κi′

[vg(i′+1) − b∗g(i′)

]. (12)

Against the profile in case 2(a), bidder g(i′) is strictly better off deviating to b∗g(i′) + 1m .

Indeed, without deviating, bidder g(i′) receives the payoff κi′+1

[vg(i′) − b∗g(i′+2)

]. Deviating

to b∗g(i′)+1m , bidder g(i′) receives a uniform lottery between that and κi′

[vg(i′) −

(b∗g(i′) + 1

m

)].

To establish the conclusion, we argue:

κi′

[vg(i′) −

(b∗g(i′) +

1

m

)]= κi′

[vg(i′+1) − b∗g(i′)

]− κi′

m+ κi′(vg(i′) − vg(i′+1))

= κi′+1

[vg(i′+1) − b∗g(i′+2)

]− κi′

m+ κi′(vg(i′) − vg(i′+1))

= κi′+1

[vg(i′) − b∗g(i′+2)

]− κi′

m+ (κi′ − κi′+1)(vg(i′) − vg(i′+1))

≥ κi′+1

[vg(i′) − b∗g(i′+2)

]− κ1

m+ ∆κ∆v

> κi′+1

[vg(i′) − b∗g(i′+2)

],

as desired. In the above, the first and third steps are algebra. The second step uses (12). Thefourth step uses κi′ ≤ κ1, κi′ − κi′+1 ≥ ∆κ, and vg(i′) − vg(i′+1) ≥ ∆v. (Recall that step one

of this proof established vg(i′) > vg(i′+1).) The fifth step uses m > 4κ1∆κ∆v

. A similar argumentestablishes the same conclusion for the profile in case 2(b).

In summary, we have shown that for each of the relevant profiles: either (i) bidder g(i′) is strictlybetter off deviating to b∗g(i′) + 1

m than playing b∗g(i′) against that profile, or (ii) that profile is

infinitely less likely under ρ than another relevant profile against which bidder g(i′) is strictlybetter off deviating to b∗g(i′) + 1

m . It follows that bidder g(i′) lexicographically prefers b∗g(i′) + 1m over

b∗g(i′) against ρ, which contradicts (ρ, b∗) being a lexicographic Nash equilibrium.38This can be shown formally using Lemmas 13, 14, and 16. Let b′′ denote a profile in case 3(a). Let b′ denote

the aforementioned profile in case 1(a). Let g(i′′) denote the deviating bidder in b′′. By definition, g(i′ − 1) is thedeviating bidder in b′. Then b′g(i′−1) >ρ b

′′g(i′′) by Lemma 13. And we have b′N\{g(i′−1)} ≥ρ b′′N\{g(i′′)} by Lemma 14(i).

Combining all this and applying Lemma 16, we have b′ >ρ b′′.

44


C Connections to Test-Set Equilibrium

Test-set equilibrium (Milgrom and Mollner, 2018), like extended proper equilibrium, can be inter-preted as imposing across-player restrictions on the likelihood of trembles. But in general, neitherconcept implies the other. After reviewing the definition of test-set equilibrium, this appendixfurther discusses its connection to extended proper equilibrium.

C.1 Definition of Test-Set Equilibrium

We begin by reviewing the formal definition of test-set equilibrium. Given a mixed strategy profileσ, let

T (σ) =N⋃n=1

{σ/sn | sn ∈ BRn(σ)} .

We refer to T (σ) as the test set associated with σ. In words, T (σ) consists of the strategy profilesin which a single player deviates from σ and to a strategy that is a best response to σ. A strategyprofile σ satisfies the test-set condition if no player n is using a strategy that is weakly dominatedby some σn against all opponent strategy profiles consistent with some σ′ ∈ T (σ):

Definition 12. A mixed strategy profile σ satisfies the test-set condition if and only if, for alln ∈ N , there is no σn ∈ ∆n such that both

(i) for all σ′ ∈ T (σ), πn(σ′/σn) ≥ πn(σ′/σn), and

(ii) for some σ′ ∈ T (σ), πn(σ′/σn) > πn(σ′/σn).

Definition 13. A mixed strategy profile σ is a test-set equilibrium if and only if it is a Nashequilibrium in undominated strategies that satisfies the test-set condition.

For finite games, one can use the separating hyperplane theorem to recast the undominatednessrequirements of test-set equilibrium in terms of trembles. Test-set equilibrium implicitly requireseach player to use a strategy that remains optimal even though its opponents may tremble—for some mode of trembling with the structure that every strategy profile outside the test set isoverwhelmingly less likely than every profile in the test set.

Given the definition of the test set, test-set equilibrium effectively requires any unilateral de-viation to a strategy that is a best response to equilibrium to be treated as overwhelmingly morelikely than any unilateral deviation to a strategy that is an inferior response to the equilibrium,whether by the same or by another player. As observed in the text, extended proper equilibriumimplies the same. But there are also differences, and in general, neither solution concept impliesthe other in games with three or more players.

• Extended proper equilibrium may prohibit some trembles that are implicitly allowed by test-set equilibrium, and it therefore can sometimes be more demanding. For example, giventwo strategy profiles in the test set, extended proper equilibrium may require one to beoverwhelming less likely than another. This possibility was illustrated by the example ofFigure 2(b). The same can also be true for strategy profiles outside the test set, which isillustrated by the example of Figure 5, below.

• Conversely, extended proper equilibrium may allow some trembles that are implicitly prohib-ited by test-set equilibrium, and it therefore can sometimes be less demanding. For example,extended proper equilibrium is consistent with a deviation involving two non-equilibrium best

45


responses (and therefore outside the test set) being more likely than another deviation involv-ing only a single non-equilibrium best response (and therefore in the test set). This possibilityis central to the example of Figure 3 in which no test-set equilibrium exists.

C.2 Examples

In this appendix, we formally apply the definition of test-set equilibrium to the example gamesdiscussed in the main text.

Figure 1. First, return to the game in Figure 1. As mentioned, (Up,Left,East) is a properequilibrium of this game, but it is not extended proper. It is also not a test-set equilibrium.Indeed, West weakly dominates East in the test set:

T (Up,Left,East) = {(Up,Left,East), (Up,Right,East), (Up,Left,West)} .

In particular, πgeo(σ′/West) ≥ πgeo(σ′/East) for all σ′ ∈ T (Up,Left,East), and the inequality isstrict for σ′ = (Up,Right,East). An analogous argument also rules out equilibria in which Geomixes between East and West, leaving (Up,Left,West) as the lone remaining candidate. Andindeed, it is easily checked that (Up,Left,West) is in fact a test-set equilibrium.

Thus, test-set equilibrium and extended proper equilibrium coincide in this game. In words,test-set equilibrium implicitly requires Geo to believe that the test set element (Up,Right, σgeo) ismore likely than (Down,Left, σgeo), which is outside the test set. As observed in the text, extendedproper equilibrium requires the same thing.

Figure 2(b). Next, return to the game in Figure 2(b). As mentioned, (S1, S2, S3) is not anextended proper equilibrium of this game. But it is a proper equilibrium, and it is also a test-setequilibrium. In this case, players 2 and 3 are each using their dominant strategies S2 and S3, so weneed only check that player 1’s strategy S1 is neither weakly dominated in the test set nor weaklydominated in the game. The test set is

T (S1, S2, S3) = {(S1, S2, S3), (C1, S2, S3), (S1, C2, S3), (S1, S2, C3)}.

Letting σ′ denote the test set element (S1, C2, S3), we have π1(σ′/S1) > π1(σ′/C1), so that S1

is indeed not dominated in this way. In words, test-set equilibrium implicitly allows player 1 tobelieve that opponents’ trembling will be such that the test set element (S1, C2, S3) is more likelythan (S1, S2, C3). But as observed in the text, extended proper equilibrium requires the reverse.

Figure 3. Next, return to the game in Figure 3. As mentioned, (Up,Left,West) is the uniqueNash equilibrium of this game, and it is therefore also a perfect, proper, and extended properequilibrium. But it is not a test-set equilibrium. Indeed, East weakly dominates West in the testset:

T (Up,Left,West) =

{(Up,Left,West), (Down,Left,West), (Up,Center,West),

(Up,Right,West), (Up,Left,East)

}.

In particular, πgeo(σ′/East) ≥ πgeo(σ′/West) for all σ′ ∈ T (Up,Left,West), and the inequality isstrict for σ′ = (Up,Center,West). In words, test-set equilibrium implicitly requires Geo to believethat the test set element (Up,Center,West) is more likely than (Down,Right,West), which is outsidethe test set. But as observed in the text, extended proper equilibrium allows the reverse.

46


Figure 5. Finally, consider a new example: the game in Figure 5. For Row, Down is strictlydominated by Up. For Column, Right is strictly dominated by Center, which in turn is strictlydominated by Left. In consequence, (Up,Left) is the unique Nash equilibrium of this game—it isalso a perfect, proper, extended proper, and test-set equilibrium. But what we wish to illustratewith this game are two ways in which proper equilibrium (and therefore also extended properequilibrium) can sometimes prohibit trembles that test-set equilibrium implicitly allows:

• First, proper equilibrium requires a deviation by Column to Right to be less likely than adeviation to Center. Formally, if σt is an εt-proper equilibrium then

σtcol(Right) ≤ εtσtcol(Center).

In contrast, test-set equilibrium implicitly allows the reverse: both Center and Right areinferior responses to the equilibrium, so each results in a strategy profile outside the test set,and the concept does not rank profiles that are both outside the test set.

• Second, proper equilibrium requires the joint deviation to (Down,Right) to be less likelythan either unilateral deviation. Essentially, this is because proper equilibrium requires thetrembles of different players to be statistically independent, so that σt induces a productdistribution over the set of strategy profiles. Formally, we have σtcol(Right) ≤ εtσtcol(Left) andσtrow(Down) ≤ εtσtrow(Up), which together imply

σtrow(Down)σtcol(Right) ≤ εt max{σtrow(Down)σtcol(Left), σtrow(Up)σtcol(Right)}.

In contrast, test-set equilibrium implicitly allows the reverse: both a unilateral deviation to aninferior response and the joint deviation result in strategy profiles outside the test set, and theconcept does not rank profiles that are both outside the test set. Thus, test-set equilibriumdoes not embed all the restrictions that are implied if players tremble independently of eachother. In other words, the trembles associated with a test-set equilibrium may be certaincorrelated strategy profiles.

In these ways, proper equilibrium (and therefore also extended proper equilibrium) can some-times prohibit trembles that test-set equilibrium implicitly allows. What is more, these additionalrestrictions could translate into sharper predictions for equilibrium play if there were a third playerwhose best strategy depended on the relative likelihoods of the trembles considered above.39


Left Center RightUp 1, 2 1, 1 1, 0

Down 0, 2 0, 1 0, 0


39In fact, the GSP auction example discussed in Appendix C.3 is such a game. As we discuss, the predictions oftest-set equilibrium are less sharp—precisely because it does not embed all the restrictions that are implied if playerstremble independently of each other.

47


C.3 Another GSP Auction Example

In the GSP auction game, extended proper equilibrium and test-set equilibrium similarly implylocally envy-free equilibrium, in the sense described in Section 6. But despite this similarity,extended proper equilibrium can deliver conclusions that are stronger than those that test-setequilibrium can deliver.

To illustrate, we consider another specific example. There are three bidders, with per-clickvalues (v1, v2, v3) = (6, 2, 1). There are two ad positions, with click rates (κ1, κ2) = (2, 1). For thediscretized bid set, it suffices to let m = 3 and M = 4.

The profile b∗ = (3, 2, 1) is a test-set equilibrium.40 But this bid profile is not an extendedproper equilibrium. In fact, it is not even trembling-hand perfect. The subsequent analysis willshow the problem to be that bidder 1 is bidding too low if the trembles of its different opponentsmust be statistically independent.

Building on Proposition 6, suppose to the contrary that ρ is an LPS that satisfies strongindependence and has full support for which (ρ, b∗) is a lexicographic Nash equilibrium. To seethat this produces a contradiction, suppose bidder 1 contemplates raising its bid from b∗1 = 3 to 10

3 .This change makes a difference only if at least one of its opponents trembles within the set {3, 10

3 }.Thus, the relevant profiles can be partitioned into the following way:

• Case 1(a): b2 = 103 and b3 ∈ {0, 1

3}• Case 1(b): b2 ∈ {0, 1

3} and b3 = 103

• Case 2(a): b2 = 103 and b3 = 1

• Case 2(b): b2 = 2 and b3 = 103

• Case 3: all remaining profiles in which b2 ∈ {3, 103 } and/or b3 ∈ {3, 10

3 }In general, bidder 1 prefers to make a lower bid instead of a higher bid only when one of itsopponents bids relatively high and the other opponent bids relatively low, so that position 2 can beobtained for a relatively low price while position 1 can be obtained only for a relatively high price.Computing, we find that cases 1(a) and 1(b) are the only relevant profiles against which bidder 1is strictly worse off deviating to 10

3 . For example, consider b2 = 103 and b3 = 0:

π1(103 ,

103 , 0) = 17

3 < 6 = π1(3, 103 , 0).

However, note that each profile in case 1(a) involves both bidders 2 and 3 deviating from theirequilibrium bids. In contrast, the profile in case 2(a) involves the same deviation by bidder 2 butno deviation by bidder 3, which, by strong independence, must be infinitely more likely under ρ.Moreover, against the profile in case 2(a), bidder 1 is strictly better off deviating to 10

3 :

π1(103 ,

103 , 1) = 31

6 > 5 = π1(3, 103 , 1).

Similarly, each profile in case 1(b) involves both bidders 2 and 3 deviating from their equilibriumbids. In contrast, the profile in case 2(b) involves the same deviation by bidder 3 but no deviation

40No bidder is using a weakly dominated bid. Indeed: for bidder 1, the dominated bids are those greater than6 and those less than 3; for bidder 2, the dominated bids are those greater than 2 and those less than 1; and forbidder 3, the dominated bids are those greater than 1 and those less than 1

2.

Furthermore, no bidder is using a bid that is weakly dominated in the test set. Bidder 1 best responds to itsopponents’ equilibrium bids (2, 1) with any bid greater than 2. Bidder 2 best responds to its opponents’ equilibriumbids (3, 1) with any bid greater than 1 and less than 3. Bidder 3 best responds to its opponents’ equilibrium bids(3, 2) with any bid less than 2. From these observations, the test set can be constructed and the test-set conditionchecked.

48


by bidder 2, which, by strong independence, must be infinitely more likely under ρ. And againstthe profile in case 2(b), bidder 1 is strictly better off deviating to 10

3 :

π1(103 , 2,

103 ) = 14

3 > 4 = π1(3, 2, 103 ).

We have therefore shown that for each of the relevant profiles, either (i) bidder 1 is strictly betteroff deviating to 10

3 , or (ii) that profile is infinitely less likely under ρ than another relevant profileagainst which bidder 1 is strictly better off deviating to 10

3 . It follows that 103 is a profitable

deviation for bidder 1 against ρ, which contradicts (ρ, b∗) being a lexicographic Nash equilibrium.The key issue is that test-set equilibrium does not require the trembles of different players to be

statistically independent. The profiles in cases 1(a) and 2(a) are all outside the test set, as all involvebidder 2 deviating to an inferior response. Thus, test-set equilibrium implicitly allows bidder 1 tobelieve that either case is the more likely one. And bidder 1 can justify b∗1 = 3 with a belief thatcase 1(a) is more likely. In contrast, perfect equilibrium (and hence extended proper equilibrium)does require players to tremble independently of each other, and thus it requires case 2(a) to bemore likely, so that bidder 1 cannot justify b∗1 = 3. Indeed, case 2(a) involves only bidder 2 deviatingfrom equilibrium play, while case 1(a) involves bidder 2 making the same deviation as well as anadditional deviation by bidder 3.

C.4 Weak Test-Set Equilibrium

As some of the preceding examples have illustrated (e.g., the game in Figure 3), extended properequilibrium does not always imply test-set equilibrium. Nevertheless, extended proper equilibriumdoes always imply a weakened version of test-set equilibrium, which we define below. We providethis new definition as it may facilitate comparisons between extended proper equilibrium and test-set equilibrium.

C.4.1 Formalities

For any strategy profile σ, let

T (σ) =⋃J⊆N

{σ/sJ | (∀n ∈ J) sn ∈ BRn(σ)}

In words, T (σ) consists of all strategy profiles that can be constructed from σ by replacing thestrategies of any number of players with best responses to σ. We will refer to T (σ) as the generalizedtest set associated with σ. Note that T (σ) ⊆ T (σ). We also define a partial order on T (σ) in thefollowing way.

Definition 14. For σ′, σ′′ ∈ T (σ), σ′′ . σ′ if and only if

(i) for all n ∈ N , σ′′n 6= σn =⇒ σ′n = σ′′n; and

(ii) for some n ∈ N , σ′′n = σn and σ′n /∈ supp(σn).

In words, σ′′ .σ′ if σ′ entails all the deviations from σ that are entailed by σ′′, as well as at leastone additional deviation to a strategy that is played with zero probability under σ. In the casewhere σ is a pure strategy profile: (i) σ itself is an element of T (σ), and it is the unique maximumelement under .; (ii) next in the order are the remaining elements of the test set, T (σ)\{σ}, whichare single deviations from σ; (iii) finally, we have the remaining elements of the generalized testset, T (σ) \ T (σ), which are joint deviations from σ. With this notation in place, we can state thefollowing definitions:

49


Definition 15. A mixed-strategy profile σ satisfies the weak test-set condition if and only if, forall n ∈ N , there is no σn ∈ ∆n such that both

(i) for all σ′ ∈ T (σ) with πn(σ′/σn) < πn(σ′/σn), there exists a σ′′ ∈ T (σ) such that σ′′ . σ′ andπn(σ′′/σn) > πn(σ′′/σn); and

(ii) for some σ′ ∈ T (σ), πn(σ′/σn) > πn(σ′/σn).

Definition 16. A mixed strategy profile σ is a weak test-set equilibrium if and only if it is a Nashequilibrium in undominated strategies that satisfies the weak test-set condition.

The following result implies that, as the nomenclature suggests, every test-set equilibrium is aweak test-set equilibrium.

Proposition 18. Any Nash equilibrium that satisfies the test-set condition also satisfies the weaktest-set condition.

Proof of Proposition 18. We prove the contrapositive. Suppose σ is a Nash equilibrium thatdoes not satisfy the weak test-set condition. Thus, there exists some n ∈ N and some σn ∈ ∆n

that satisfies both conditions (i) and (ii) of Definition 15. We will show that the same n andthe same σn satisfy the conditions of Definition 12. Because condition (ii) of Definition 12 isidentical to condition (ii) of Definition 15, we need only check condition (i). Suppose, by way ofcontradiction, that condition (i) of Definition 12 does not hold. That is, there exists a σ′ ∈ T (σ)such that πn(σ′/σn) < πn(σ′/σn). But for σ′ ∈ T (σ), we can have σ′′ . σ′ only if σ′′ = σ.41 Andπn(σ/σn) > πn(σ/σn) contradicts the assumption that σ is a Nash equilibrium.

C.4.2 Example: The Game in Figure 3

To illustrate the definitions that we have just given, consider again the game in Figure 3. Asmentioned, (Up,Left,West) is the unique Nash equilibrium of this game. We previously pointedout that the test set associated with this strategy profile is

T (Up,Left,West) =

{(Up,Left,West), (Down,Left,West), (Up,Center,West),

(Up,Right,West), (Up,Left,East)

}.

Let us now construct the the generalized test set associated with this strategy profile. Inspectingthe payoff matrix, every strategy is a best response to (Up,Left,West). Therefore, T (Up,Left,West)is in fact the entire set of pure strategy profiles. And in this case, the partial order . onT (Up,Left,West) can be represented by the directed graph depicted in Figure 6, where σ′′ . σ′

iff there exists a directed path from σ′′ to σ′.Recall that (Up,Left,West) did not satisfy the test-set condition because East met the require-

ments set out for σn set out in Definition 12. Let us now check whether it also meets the require-ments for σn set out in Definition 15. It does not. In particular, consider σ′ = (Down,Right,West).This profile is not an element of T (σ), and so it was not relevant to Definition 12. But it is anelement of T (σ), and so it must now be considered when checking condition (i) of Definition 15.With this choice of σ′, πgeo(σ′/East) < πgeo(σ′/West). Moreover, there is no σ′′ . σ′ for whichπgeo(σ′′/East) > πgeo(σ′′/West).42 Thus, East does not meet the requirements. Furthermore, asthis argument suggests, it can be checked that (Up,Left,West) does indeed satisfy the weak test-setcondition and is a weak test-set equilibrium.

41And this is possible only if σ is itself an element of T (σ).42In particular, we have πgeo(σ′′/East) > πgeo(σ′′/West) iff (σ′′row, σ

′′col) = (Up,Center). But it is apparent from

Figure 6 that no such σ′′ also satisfies σ′′ . (Down,Right,West).

50


Figure 6: The partial order . on T (Up,Left,West) for the game in Figure 3

(Up,Left,West)

(Down,Left,West)

(Up,Center,West)

(Up,Right,West)

(Up,Left,East)

(Down,Center,West)

(Down,Right,West)

(Down,Left,East)

(Up,Center,East)

(Up,Right,East)

(Down,Center,East)

(Down,Right,East)

C.4.3 Main Result

We had previously said that (Up,Left,West) is an extended proper equilibrium of the game inFigure 3. And we have just shown that it is also a weak test-set equilibrium. The main result ofthis appendix states that this relationship is in fact true in general:

Proposition 19. Every extended proper equilibrium is a weak test-set equilibrium.

Roughly speaking, we prove the proposition by arguing that extended proper equilibrium re-quires (i) each strategy profile outside T (σ) to be infinitely less likely than every strategy profileinside T (σ),43 and (ii) if σ′, σ′′ ∈ T (σ) are such that σ′′ . σ′, then σ′ is infinitely less likely than σ′′.

In addition to facilitating comparisons between extended proper equilibrium and test-set equi-librium, Proposition 19 might also be useful in its own right. For example, it could have been usedto simplify the proof of Proposition 9: whereas our direct proof showed that every pure extendedproper equilibrium of the GSP auction game is locally envy-free*, given Proposition 19, it wouldhave sufficed to show that this property is possessed by the pure weak test-set equilibria.

Proof of Proposition 19. Suppose σ is an extended proper equilibrium. By Proposition 8, thereexists some LPS ρ that satisfies strong independence, has full support, respects within-and-across-player preferences, and for which (ρ, σ) is a lexicographic Nash equilibrium. It is immediate that σis a Nash equilibrium in undominated strategies, so it only remains to verify that the weak test-setcondition is satisfied.

Suppose, by way of contradiction, that the weak test-set condition does not hold. Thus, thereexists some n ∈ N and some σn ∈ ∆n that satisfies both conditions (i) and (ii) of Definition 15.Condition (ii) says there exists a σ′ ∈ T (σ) such that πn(σ′/σn) > πn(σ′/σn). From the definitionof T (σ), there must exist the following: some player m0 6= n, some s′m0

∈ BRm0(σ), and forall m 6= m0, some s′m ∈ Sm with σm(s′m) > 0, such that πn(s′/σn) > πn(s′/σn). On the otherhand, suppose a pure strategy profile s′′ is such that πn(s′′/σn) < πn(s′′/σn). There are two cases,depending on whether s′′/σn ∈ T (σ):

• Case 1: s′′/σn /∈ T (σ). By the definition of T (σ), there exists some player m1 6= n such thats′′m1

/∈ BRm1(σ). On the other hand, we had s′m0∈ BRm0(σ). Because ρ respects within-and-

across-player preferences, it follows from Lemma 13 that s′m0>ρ s

′′m1

. Because σm(s′m) > 0for all m 6= m0, it follows from Lemma 14(i) that s′N\{m0} ≥ρ s

′′N\{m1}. Applying Lemma 16,

we conclude s′ >ρ s′′.

43On the other hand, extended proper equilibrium does not require the stronger condition that each strategyprofile outside T (σ) be infinitely less likely than every strategy profile inside T (σ). This is why extended properequilibrium does not imply the test-set condition itself.

51


• Case 2: s′′/σn ∈ T (σ). Condition (i) of Definition 15 says there exists a σ′′′ ∈ T (σ) such thatσ′′′ . s′′/σn and πn(σ′′′/σn) > πn(σ′′′/σn). By the definition of the partial order ., there existsome J ⊆ N \{n} and some m2 ∈ N \ (J ∪{n}) with s′′m2

/∈ supp(σm2) such σ′′′ = (s′′J , σN\J).Furthermore, for all m ∈ N \ J , there must exist some s′′′m ∈ Sm with σm(s′′′m) > 0 such thatπn(s′′J , s

′′′N\J/σn) > πn(s′′J , s

′′′N\J/σn). Because σm2(s′′′m2

) > 0 but σm2(s′′m2) = 0, it follows

from Lemma 14(ii) that s′′′m2>ρ s

′′m2

. Because σm(s′′′m) > 0 for all m ∈ N \ (J ∪ {m2}), itfollows from Lemma 14(i) that s′′′N\(J∪{m2}) ≥ρ s

′′N\(J∪{m2}). Applying Lemma 16, we have

s′′′N\J >ρ s′′N\J , and (s′′J , s

′′′N\J) >ρ s

′′.

In summary, for every pure strategy profile s′′ for which πn(s′′/σn) < πn(s′′/σn), we have shownthat there exists another profile that is infinitely more likely under the LPS ρ and under whichthe inequality is reversed (i.e., s′ in case 1 and (s′′J , s

′′′N\J) in case 2). We conclude that player n

lexicographically prefers σn over σn against ρ. This contradicts (ρ, σ) being a lexicographic Nashequilibrium.

D An Illustration of the Proof of Theorem 4

Let Γ be the game in Figure 7. For Column, Left always pays 2, Center always pays 1, andRight always pays 0. For Row, Up always pays 1; Down also pays 1 if Column plays its dominantstrategy of Left but pays 0 otherwise. In consequence, σ = (Up,Left) is clearly an extended properequilibrium. We use this game to illustrate our strategy for proving the sufficiency of extendedproper equilibrium in Theorem 4. To do so, we demonstrate that σ = (Up,Left)—which clearly hasσ as its projection—is a symmetrically proper strategy of Γ.


Left Center RightUp 1, 2 1, 1 1, 0

Down 1, 2 0, 1 0, 0


The following strategy profile is an (α, δ)-extended proper equilibrium with α = (1, 1) and allsufficiently small δ > 0:

σδrow = (1− δ2, δ2) σδcol = (1− δ3 − δ4, δ3, δ4).

In particular, Down has vanishingly small loss and is played with probability δ2; Center has a lossof 1 and is played with probability δ3; and Right has a loss of 2 and is played with probability δ4.Let φδ be the distribution over S induced by σδ = (σδrow, σ

δcol):

φδ(Up,Left) = 1− δ2 − δ3 − δ4 + δ5 + δ6 φδ(Down,Left) = δ2 − δ5 − δ6

φδ(Up,Center) = δ3 − δ5 φδ(Down,Center) = δ5

φδ(Up,Right) = δ4 − δ6 φδ(Down,Right) = δ6

Mechanically, φδ is a distribution over S that has σδ as its projection (although not the unique suchdistribution). As remarked in the text, (φδ, . . . , φδ) might not itself be an ε-proper equilibrium of

52


Γ. That is indeed the case here. To see that, let s′ = (Down,Center) and s′′ = (Up,Right). Thenπ1(s′, φδ) > π1(s′′, φδ) for all sufficiently small δ > 0:

π1(s′, φδ) =1

2

[πrow

(Down, projcol(φ

δ))

+ πcol

(projrow(φδ),Center

)]=

1

2

[(1− δ3 − δ4) + 1

]π1(s′′, φδ) =

1

2

[πrow

(Up, projcol(φ

δ))

+ πcol

(projrow(φδ),Right

)]=

1

2[1 + 0]

And whereas ε-proper equilibrium would therefore require φδ(Up,Right) ≤ ε · φδ(Down,Center),we instead have φδ(Up,Right) > φδ(Down,Center) when δ > 0 is small.

More generally, given a distribution over S whose projection is σδ, a necessary condition forit to constitute the desired ε-proper equilibrium of Γ is that it satisfy the following: (Up,Left) isthe most likely profile, followed by (Down,Left), (Up,Center), (Down,Center), (Up,Right), and(Down,Right). As shown above, φδ itself does not satisfy this necessary condition when δ is small.However, φδ is not the only distribution over S whose projection is σδ. The rest of the discus-sion demonstrates how we construct another such distribution with the necessary properties. Toillustrate our approach, we partition the elements of S in the following way:

• Let s∗ = (Up,Left). This is the profile that is played in equilibrium.

• Let S1 = {(Up,Center), (Up,Right), (Down,Left)}. These are all single deviations from s∗.

• Let S2 = {(Down,Center), (Down,Right)}. These are all joint deviations from s∗.

The key is to adjust the probabilities assigned to the elements of S2 in an appropriate way, thenalso adjusting other probabilities so as to keep the projection unchanged. Let σδ,ε denote the newdistribution over S that we will obtain. Applying the proof’s general construction to this example,σδ,ε attaches the following probabilities to the elements of S2:

σδ,ε(Down,Center) = ε2(δ3 − δ5) σδ,ε(Down,Right) = ε3(δ4 − δ6)

Relative to φδ, notice that we have added the weight ε2(δ3 − δ5) − δ5 to (Down,Center). Soas to ensure that the projection will remain unchanged, we therefore subtract the same amountof weight from each of the corresponding single deviations in S1—in this case, (Up,Center) and(Down,Left)—and we also add the same amount of weight to s∗. Similarly, we have added theweight ε3(δ4 − δ6) − δ6 to (Down,Right). We therefore subtract the same amount from both(Up,Right) and (Down,Left), and we also add the same amount to s∗. In summary, we obtain

σδ,ε(Up,Left) = 1− δ2 − δ3 − δ4 + ε2(δ3 − δ5) + ε3(δ4 − δ6)

σδ,ε(Down,Left) = δ2 − ε2(δ3 − δ5)− ε3(δ4 − δ6)

σδ,ε(Up,Center) = δ3 − ε2(δ3 − δ5)

σδ,ε(Up,Right) = δ4 − ε3(δ4 − δ6)

It is easily verified that σδ is the projection of σδ,ε—just as it was the projection of φδ. Furthermore,if we choose δ to be a sufficiently high power of ε, then this constitutes the desired ε-proper

53


equilibrium of Γ. Indeed, plugging in δ = ε4, we obtain the following distribution over S:44

σε(Up,Left) = 1− ε8 − ε12 + ε14 − ε16 + ε19 − ε22 − ε27

σε(Down,Left) = ε8 − ε14 − ε19 + ε22 + ε27

σε(Up,Center) = ε12 − ε14 + ε22

σε(Down,Center) = ε14 − ε22

σε(Up,Right) = ε16 − ε19 + ε27

σε(Down,Right) = ε19 − ε27

We see that when ε > 0 is sufficiently small, σε satisfies the aforementioned necessary condition—and what is more, we have the following:

σε(Down,Right) ≤ ε · σε(Up,Right) ≤ ε · σε(Down,Center)

≤ ε · σε(Up,Center) ≤ ε · σε(Down,Left) ≤ ε · σε(Up,Left),

from which it follows that (σε, . . . , σε) is an ε-proper equilibrium of Γ. We conclude that σ =limε→0 σ

ε = (Up,Left) is a symmetrically proper strategy of Γ. Moreover, σ = (Up,Left), whichis the extended proper equilibrium of Γ that we began with, is trivially the projection of σ, asrequired.

44The general proof uses δ = ε|S|+2, which always suffices. But for this example, even δ = ε4 suffices.

54


extended proper equilibrium - paul milgrom · 2020. 8. 21. · of the payo s. in this way, extended...

Documents