computing nash-equilibria in 2-player games and beyond ...akmenon/nashequilibria...finding...
TRANSCRIPT
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Computing Nash-equilibria in 2-player games andbeyond
Algorithms Reading Group
Aditya Menon
June 3, 2007
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
IntroductionNash equilibriumA formal definition
ComplexityCharacterizing complexityPPAD
Finding equilibria in 2-player gamesLemke-Howson algorithmOther approaches
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Nash equilibriumA formal definition
Nash-equilibrium - Recap
I Refers to a special kind of state in an n-player gameI No player has an incentive to unilaterally deviate from his
current strategyI A kind of “stable” solution
I Existence depends on the type of gameI If strategies are “pure” i.e. deterministic, does not have to
exist in the gameI If strategies are “mixed” i.e. probabilistic, then it always exists
I Yet how do we find it!?!
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Nash equilibriumA formal definition
Notation
I Suppose that player p follows the mixed strategyxp = (xp1, . . . , xpnp)
I The ith entry gives the probability that player p plays move i
I Let x := (x1, . . . , xn) be the collection of strategies for allplayers
I Let the function Up(x) denote the expected utility or payoffthat player p gets when each player uses the strategy dictatedin x
I Based on up(s1, . . . , sn), which is the utility when player qplays sq
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Nash equilibriumA formal definition
Formal definition
I We say that x∗ = (x1∗, . . . , xn
∗) is a Nash equilibrium if...I “No player has an incentive to unilaterally deviate from his
current strategy”
I If player p decides to switch to a strategy yp, then write theresulting strategy set as x−p; yp
I So, x∗ is a N.E. if, for every player p, and for any mixedstrategy yp for that player, we have
Up(x∗) ≥ Up(x
∗−p; yp)
I A more symmetric version:
Up(x∗−p; x
∗p) ≥ Up(x
∗−p; yp)
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Nash equilibriumA formal definition
Why the equilibrium exists
I Based on Kakutani’s fixed point thereom
I Generalizes the following: take two pieces of paper (which are3D objects) and imagine placing them on top of each other,so that the “points” align. Crumple the top sheet withouttearing it, and place it on top of the other sheet. Then, thereis always a point that has not changed position on the twosheets.
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Questions about finding Nash equilibria
I Proof of existence is non-constructiveI So how do we find it?
I Can we find it?I Can we find it efficiently?
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Do we really care?
I Obviously an “interesting” problem, but is there a significantbenefit in being able to compute it?
I Arguably, equilibrium is not useful unless we can find itI Consider the case of a marketI “If your laptop can’t find it [the equilibrium], then neither can
the market” [Papadimitriou]
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Importance of the problem
I “Together with factoring, the complexity of finding a Nashequilibrium is in my opinion the most important concrete openquestion on the boundary of P today” [Papadimitriou]
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Complexity of the problem
I Need to define a special class, PPAD, for this problem
I Turns out that finding the Nash-equilibrium is PPAD-completeI What about NP?
I Probably not NP-completeI The decision version is in P
I Why?
I Because the equilibrium always exists!
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Complexity of the problem
I Need to define a special class, PPAD, for this problem
I Turns out that finding the Nash-equilibrium is PPAD-completeI What about NP?
I Probably not NP-completeI The decision version is in P
I Why?I Because the equilibrium always exists!
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Polynomial parity argument
I Simple fact: Every graph has an even number of odd-degreenodes
I Proof: Let W = {v ∈ V : v has odd degree }
2|E | =∑v∈W
deg(v) +∑v /∈W
deg(v)
=∑v∈W
odd + even
I Corollary: If a graph has maximum degree 2, then it musthave an even number of leaves
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
The class TFNP
I PPAD is based on the class TFNP
I Suppose we have a set of binary predicates P(x , y) where
(∀x)(∃y) : P(x , y) = TRUE
I Problems in TFNP: Given an x , find a y so that P(x , y) isTRUE
I Subclasses defined based on how we decide (∃y) : P(x , y) isTRUE
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
PPAD in terms of TFNP
I PPAD is defined by the following problemI Suppose we have an exponential-size graph where the
in-degree and out-degree of each node is at most 1I Given any node v , we have a polynomial-time algorithm that
finds the neighbours of vI Problem: Given a leaf node, output another leaf node
I Existence is guaranteed by the parity argument
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Completeness of finding Nash equilibrium
I Finding a Nash equilibrium is PPAD-completeI For 4-player games... [Papadimitriou]
I ...and 3-player games [Papadimitriou]I ...and even for 2-player games! [Chen]
I So finding the Nash equilibrium even for 2-player games is noeasier than doing it for n-players!
I At the moment, however, not much known about how “hard”a class PPAD is
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Completeness of finding Nash equilibrium
I Finding a Nash equilibrium is PPAD-completeI For 4-player games... [Papadimitriou]I ...and 3-player games [Papadimitriou]
I ...and even for 2-player games! [Chen]
I So finding the Nash equilibrium even for 2-player games is noeasier than doing it for n-players!
I At the moment, however, not much known about how “hard”a class PPAD is
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Completeness of finding Nash equilibrium
I Finding a Nash equilibrium is PPAD-completeI For 4-player games... [Papadimitriou]I ...and 3-player games [Papadimitriou]I ...and even for 2-player games! [Chen]
I So finding the Nash equilibrium even for 2-player games is noeasier than doing it for n-players!
I At the moment, however, not much known about how “hard”a class PPAD is
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Characterizing complexityPPAD
Completeness of finding Nash equilibrium
I Finding a Nash equilibrium is PPAD-completeI For 4-player games... [Papadimitriou]I ...and 3-player games [Papadimitriou]I ...and even for 2-player games! [Chen]
I So finding the Nash equilibrium even for 2-player games is noeasier than doing it for n-players!
I At the moment, however, not much known about how “hard”a class PPAD is
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Approaches to finding equilibria
I No P algorithms known!
I Most approaches are based on solving non-linear programs(for general n)
I Completeness result means even 2-player games are not (yet)“easy” to solve
I One of the earliest algorithms for finding equilibria in 2-playergames: Lemke-Howson algorithm
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Lemke-Howson algorithm
I An algorithm for finding the Nash equilibrium for a game with2 players
I Developed in 1964I Independent proof of why equilibrium must exist
I Worst-case exponential time, but in practise quite goodperformance
I Still viewed favourably, although it has been beaten of late
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
How we proceed
I We need to redefine a Nash equilibrium for 2-playersI Try and make a graph that lets us find equilibria easily
I Exploiting the convenience of the alternate definition
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Utility for 2-players
I Suppose that for a 2-player game, we have the mixedstrategies x = (s, t)
I Label the strategies by I = {1, . . . ,m} for player 1, andJ = {m + 1, . . . ,m + n} for player 2
I Expected utility for player p must be
Up(x) =∑
i
∑j
Pr[player 1 chooses i ]× Pr[player 2 chooses j ]
× Payoff for player p when 1 plays i and 2 plays j
=∑
i
∑j
s(i)t(j)up(i , j)
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Nash equilibria for two players
I We call x∗ = (s∗, t∗) a Nash equilibrium iff
(∀s)∑
i
∑j
s∗(i)t∗(j)u1(i , j) ≥∑
i
∑j
s(i)t∗(j)u1(i , j)
(∀t)∑
i
∑j
s∗(i)t∗(j)u2(i , j) ≥∑
i
∑j
s∗(i)t(j)u2(i , j)
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Claim
I Claim: If in a Nash equilibrium player p can play strategy i(non-zero probability), then if player p chooses to playstrategy i , it is the best possible strategy
I Mathematically,
s∗(i) > 0 =⇒ (∀i0)∑
j
t∗(j)u1(i , j) ≥∑
j
t∗(j)u1(i0, j) (1)
t∗(j) > 0 =⇒ (∀j0)∑
i
s∗(i)u2(i , j) ≥∑
i
s∗(i)u2(i , j0) (2)
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Lemma
I Lemma: Let πp,i denote the mixed strategy (0, . . . , 1, . . . , 0) i.e. wedeterministically choose strategy i for player p. Then, x is a NashEquilibrium iff
(∀p, πp,i)Up(x) ≥ Up(x−p;πp,i)
I Proof: (note that =⇒ direction is by definition)
Up(x−p; yp) =∑ip
yp(ip)
{∑i1...in
x1(i1) . . . xn(in)up(x−p;πp,ip)
}=
∑ip
yp(ip)Up(x−p;πp,ip)
≤ Up(x) since∑
yp(i) = 1
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Proof of claim
I Use the lemma: Up(x∗) ≥ Up(x−p;πp,i)
Up(x∗) =
∑x∗p(i)Up(x
∗) since∑
x∗p(i) = 1
≥∑
x∗p(i)Up(x∗−p;πp,i)
= Up(x∗)
I So, we have to conclude that∑x∗p(i)Up(x
∗) =∑
x∗p(i)Up(x∗−p;πp,i)
I Taking terms to one side,
x∗p(i) > 0 =⇒ Up(x∗) = Up(x
∗−p;πp,i)
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Reformulation of Nash equilibrium - I
I So, x∗ is a Nash equilibrium iffI For player 1, equation 1 holds or Pr[strategy i ] = 0I For player 2, equation 2 holds or Pr[strategy j ] = 0
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Reformulation of Nash equilibrium - II
I Define
S i = {s : s(i) = 0},S j = {s :∑
i
s(i)u2(i , j) ≥∑
i
s(i)u2(i , j0)}
T j = {t : t(j) = 0},T i = {t :∑
j
t(j)u1(i , j) ≥∑
j
t(j)u1(i0, j)}
I Then, x∗ is a Nash equilibrium iff
(∀i)s ∈ S i ∨ t ∈ T i
(∀j)s ∈ S j ∨ t ∈ T j
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Labelling
I We are claiming that x = (s, t) is an equilibrium iff...I For any k ∈ I ∪ J, either s or t (or maybe both) is in the
appropriate region Sk or T k
I Can think of these k’s as labels of pointsI Labels(s) = {k ∈ I ∪ J : s ∈ Sk}I Labels(t) = {k ∈ I ∪ J : t ∈ T k}I Note: This definition also makes sense even when s, t aren’t
valid strategies
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Reformulation of Nash equilibrium - III
I Natural label for x = Labels(s)∪ Labels(t)
I So, x is a Nash equilibrium iff it is completely labelled
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Strategy simplex
I m strategies =⇒ valid strategy space is an (m − 1)dimensional simplex
(1, 0, 0)
(0, 0, 1)
(0, 1, 0)
(0, 0, 0)
I With labelling, we can split up the simplex into regions
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Labelling - example
I For the payoff matrix A = [0 6; 2 5; 3 3]
I Label the strategy space for player 2:
1, 4
1, 2
2, 3
3, 5
4
5
(13, 2
3
)
(23, 1
3
)
(1, 0)
(0, 1)
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Reformulation of problem
I Using the labelling definition, x is a Nash equilibrium iff it iscompletely labelled
I ∼ Point that lie in the intersection of many regionsI But here there are two variables, and two sets of spacesI Can “sacrifice” one for the sake of the other
I New problem: How do we find points that are completelylabelled?
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
High-level solution
I With labelling, we have divide the strategy spaces intosub-spaces
I We want to end up with a pair of strategies that together “liein the intersection of the spaces”
I Search problem: starting off with some initial pair ofstrategies, we want to find the equilibrium
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
High-level solution
I Think of the space as a graph
I Vertices should correspond to strategy pairsI Edges should define some well-defined change in the strategies
I Which makes it easy to find equilibria
I Following a well-defined path on this graph should take us toequilibria
I Problem: How do we make such a graph?
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Graph construction
I Form the graphs GS = (VS ,ES), GT = (VT ,ET ) where:I VS ← {s ∈ Rm
+ :∑
si ≤ 1 ∧ s has exactly m labels }I ES between s1, s2 if they differ in exactly one labelI Similarly for VT ,ET
I Note: This is now “filling” the strategy simplex
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Payoff B = [1 0; 0 2; 4 3]
I Vertices lie on simplex, except for 0
(0, 0, 0) (1, 0, 0)
(0, 1, 0)
(0, 0, 1)
(23, 1
3, 0
)
(0, 1
3, 2
3
)
L = 1, 3, 5
L = 1, 2, 3
L = 1, 2, 4
L = 3, 4, 5
L = 1, 2, 4
L = 2, 3, 5
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Graph construction
I Form the product graph G = GS × GT
I V = {(s, t) : s ∈ VS , t ∈ VT}I E = {(s1, t1)→ (s1, t2) : t1 → t2} ∪ {(s1, t1)→ (s2, t1) :
s1 → s2}
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Graph and equilibria
I So now, have a graph where vertices are strategy pairs
I Know that the equilibria is one of the vertices
I Question: How do we find this vertex?
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Two definitions
I Define:I L−(k) := vertices that have all labels except, possibly, kI L := vertices that have all labels
I By definition, L ⊆ L−(k)
I L = (0, 0) ∪ {Equilibria}I We can prove some properties about these sets...
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Fact 1
I Fact: For any k, every member of L is adjacent to exactly onemember of L−(k)− L
I For any label, every (pseudo) equilibrium is adjacent to exactlyone strategy pair that is missing that label
I Proof:I Let (s, t) ∈ L. Then the label k must apply to either s or t, by
definitionI Suppose that s is labelled with k. Then, there must be an
edge between (s, t) and the point (s′, t) where s′ is missing thelabel k
I There is only one such s′ that is missing the label k - hencethe neighbour is unique
I Similar argument if t is labelled with k
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Fact 1
I Fact: For any k, every member of L is adjacent to exactly onemember of L−(k)− L
I For any label, every (pseudo) equilibrium is adjacent to exactlyone strategy pair that is missing that label
I Proof:I Let (s, t) ∈ L. Then the label k must apply to either s or t, by
definitionI Suppose that s is labelled with k. Then, there must be an
edge between (s, t) and the point (s′, t) where s′ is missing thelabel k
I There is only one such s′ that is missing the label k - hencethe neighbour is unique
I Similar argument if t is labelled with k
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Fact 2
I Fact: For any k, every member of L−(k)− L is adjacent toexactly two members of L−(k)
I Every strategy pair missing exactly one label is adjacent toexactly two other strategy pairs that are potentially missingthe same label
I Proof:I Since|Labels(s, t)| = m + n − 1 6= |Labels(s)|+ |Labels(t)| = m + n,there must be a duplicate label, `
I In the graph GS , we must have an edge from s to some otherpoint s′, where s′ does not have the label `
I Then, the edge (s, t)→ (s′, t) must belong to EI Similarly for GT - this means that the graph G has exactly two
edges that change the labelling
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Fact 2
I Fact: For any k, every member of L−(k)− L is adjacent toexactly two members of L−(k)
I Every strategy pair missing exactly one label is adjacent toexactly two other strategy pairs that are potentially missingthe same label
I Proof:I Since|Labels(s, t)| = m + n − 1 6= |Labels(s)|+ |Labels(t)| = m + n,there must be a duplicate label, `
I In the graph GS , we must have an edge from s to some otherpoint s′, where s′ does not have the label `
I Then, the edge (s, t)→ (s′, t) must belong to EI Similarly for GT - this means that the graph G has exactly two
edges that change the labelling
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Putting the facts together
I L−(k) describes a subgraph of G containing (disjoint) pathsand loops of G
I The endpoints of a path in G are (pseudo) equilibriaI Problem: How do we find this set quickly?
I Look at this later...
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Finding equilibria
I Start off at the “artificial equilibrium” (0, 0)
I Choose an arbitrary label ` ∈ I ∪ J
I Follow the path generated by the set L−(`)
I When we reach the end of the path, we necessarily havestopped at an equilibria
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Payoff matrices
A =
0 62 53 3
B =
1 00 24 3
I Choose the label 2 to be dropped i.e. move along L−(2)
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Start off at the artificial equilibrium, ((0, 0, 0), (0, 0))→ labels{1, 2, 3}, {4, 5}
(13, 2
3
)
(23, 1
3
)
(1, 0)
(0, 1)
(0, 0)
(0, 0, 0) (1, 0, 0)
(0, 1, 0)
(0, 0, 1)
(23, 1
3, 0
)
(0, 1
3, 2
3
)
L = 1, 3, 5
L = 1, 2, 3
L = 1, 2, 4
L = 3, 4, 5
L = 1, 2, 4
L = 2, 3, 5
L = 4, 5
L = 1, 4
L = 1, 2
L = 2, 3
L = 3, 5
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Step 1: ((0, 1, 0), (0, 0))→ labels {1, 3, 5}, {4, 5}; duplicate is5
(13, 2
3
)
(23, 1
3
)
(1, 0)
(0, 1)
(0, 0)
(0, 0, 0) (1, 0, 0)
(0, 1, 0)
(0, 0, 1)
(23, 1
3, 0
)
(0, 1
3, 2
3
)
L = 1, 3, 5
L = 1, 2, 3
L = 1, 2, 4
L = 3, 4, 5
L = 1, 2, 4
L = 2, 3, 5
L = 4, 5
L = 1, 4
L = 1, 2
L = 2, 3
L = 3, 5
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Step 2: ((0, 1, 0), (0, 1))→ labels {1, 3, 5}, {1, 4}; duplicate is1
(13, 2
3
)
(23, 1
3
)
(1, 0)
(0, 1)
(0, 0)
(0, 0, 0) (1, 0, 0)
(0, 1, 0)
(0, 0, 1)
(23, 1
3, 0
)
(0, 1
3, 2
3
)
L = 1, 3, 5
L = 1, 2, 3
L = 1, 2, 4
L = 3, 4, 5
L = 1, 2, 4
L = 2, 3, 5
L = 4, 5
L = 1, 4
L = 1, 2
L = 2, 3
L = 3, 5
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Step 3: ((23 , 1
3 , 0), (0, 1))→ labels {3, 4, 5}, {1, 4}; duplicate is4
(13, 2
3
)
(23, 1
3
)
(1, 0)
(0, 1)
(0, 0)
(0, 0, 0) (1, 0, 0)
(0, 1, 0)
(0, 0, 1)
(23, 1
3, 0
)
(0, 1
3, 2
3
)
L = 1, 3, 5
L = 1, 2, 3
L = 1, 2, 4
L = 3, 4, 5
L = 1, 2, 4
L = 2, 3, 5
L = 4, 5
L = 1, 4
L = 1, 2
L = 2, 3
L = 3, 5
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Step 4: ((23 , 1
3 , 0), (13 , 2
3))→ labels {3, 4, 5}, {1, 2}
I Completely labelled, and so an equilibria
(13, 2
3
)
(23, 1
3
)
(1, 0)
(0, 1)
(0, 0)
(0, 0, 0) (1, 0, 0)
(0, 1, 0)
(0, 0, 1)
(23, 1
3, 0
)
(0, 1
3, 2
3
)
L = 1, 3, 5
L = 1, 2, 3
L = 1, 2, 4
L = 3, 4, 5
L = 1, 2, 4
L = 2, 3, 5
L = 4, 5
L = 1, 4
L = 1, 2
L = 2, 3
L = 3, 5
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Step 4: ((23 , 1
3 , 0), (13 , 2
3))→ labels {3, 4, 5}, {1, 2}I Completely labelled, and so an equilibria
(13, 2
3
)
(23, 1
3
)
(1, 0)
(0, 1)
(0, 0)
(0, 0, 0) (1, 0, 0)
(0, 1, 0)
(0, 0, 1)
(23, 1
3, 0
)
(0, 1
3, 2
3
)
L = 1, 3, 5
L = 1, 2, 3
L = 1, 2, 4
L = 3, 4, 5
L = 1, 2, 4
L = 2, 3, 5
L = 4, 5
L = 1, 4
L = 1, 2
L = 2, 3
L = 3, 5
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Algorithm summary
I Consider strategies sk that have all labels except, possibly, somelabel k
I Clearly, every equilibrium belongs to this setI So too does the “artificial equilibrium”, 0
I Construct a graph from all strategies
I Then, one can show:
I Each strategy missing a label is adjacent to exactly two suchstrategies
I Each equilibrium is adjacent to only one strategy sk
I It follows that:
I Equilibria are endpoints of paths on the graph
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Generating L−(`)
I Second problem...I How do we find adjacent strategies?
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Generating L−(`)
I “Pivoting” approach
I Able to implicitly generate the graph, on-the-fly
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Optimization problem
I Suppose we know player 2’s strategy to be t
I Suppose we have payoff matrices A,B
I We need to choose our strategy to maximize the payoff, i.e.
U1(x) =∑
i
∑j
s(i)t(j)A(i , j)
=∑
i
s(i)A(i , :)t
= sT (At)
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Optimization problem
I Primal:maximize sT (At) :
∑s(i) = 1, s ≥ 0
I Dual:minimize u : u1 ≥ At
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Linear system
I We need to solves.1 = 1
t.1 = 1
u1− At ≥ 0
v1− BT s ≥ 0
s, t ≥ 0
I Equilibrium iff the above, and
sT (u1− At) = 0
tT (v1− BT s) = 0
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Linear system with slackness
I Introduce slack variables z,w:
s.1 = 1
t.1 = 1
−u1 + At + w.1 = 0
−v1 + BT s + z.1 = 0
s, z ≥ 0
t,w ≥ 0
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
System in matrix form
I Can write system as a matrix equation
I Consider player 1’s equations - same as:
s(1) + . . . + s(m) = 1
bTi .s− v + z(i) = 0, i = 1, . . . , n
s, z ≥ 0
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
System in matrix form
I Can write as Dr = c, where
D =
1 1 . . . 1 0 0 . . . 0
b11 b21 . . . bm1 −1 1 . . . 0...
b1n b2n . . . bmn −1 0 . . . 1
r =
s(1)...
s(m)v
z(1)...
z(n)
, and c =
10...0
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Connection with labelling
I Let the basis of the column space of D be {Dβ}I Call rj ...
I A basic variable if Dj is in the basisI A non-basic variable otherwise; let rj = 0 in this case
I Labels correspond to non-basic variablesI r1, . . . , rm = 0 =⇒ s(i) = 0 i.e. strategy is not playedI rm+2, . . . , rm+n+1 = 0 =⇒ z(i) = 0 i.e. strategy is
best-response
I So, removing a label corresponds to making a variable strictlypositive
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Rewriting system
I System is the same as
D−1β (Dr) = D−1
β c
I Fact: D−1β D = I +
∑j /∈β D−1
β Dj
I System is the same as
rβ = D−1β c−
∑j /∈β
D−1β Dj rj
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Why rewrite?
I What is wrong with the standard representation of thesystem?
I Nothing “wrong” with it...I Just that the new system makes it easier to find strategies
with different labelsI i.e. Generating the set L−(`)I This is the “pivot” operation
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Use earlier payoff matrix
I Matrix D for player 1 is
D =
1 1 1 0 0 01 0 4 −1 1 00 2 3 −1 0 1
I Choose column-space {2, 4, 5}
I Setting x1, x3 = 0I Setting z2 = 0
Dβ =
1 0 00 −1 12 −1 0
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Solution:
rβ =
122
−1
23
r1 −
1−13
r3 −
0−1−1
r6
I Or equivalently:
x2 = 1− x1 − x3
v = 2− 2x1 + x3 + z2
z1 = 2− 3x1 − 3x3 + z2
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example
I Can read off solution from hereI s = (0, 1, 0)I v = 2I z1 = 2, z2 = 0
I Can read the labels: 1, 3, 5
I Do the same thing for player 2
I Result: We get a pair of strategies (s, t), and we know whattheir labels are
I Question: How do we find the “adjacent” strategy, as per thegraph?
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Finding adjacent strategy
I Traversal step in the algorithm was...I Find the duplicate label in the two strategiesI Drop the label from player 1 and player 2 alternately
I Not hard to find duplicate (just read off from solution)I How to drop label?
I Just change the appropriate variable to be positiveI As large as possible while satisfying constraints
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Finding adjacent strategy
I Traversal step in the algorithm was...I Find the duplicate label in the two strategiesI Drop the label from player 1 and player 2 alternately
I Not hard to find duplicate (just read off from solution)I How to drop label?
I Just change the appropriate variable to be positive
I As large as possible while satisfying constraints
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Finding adjacent strategy
I Traversal step in the algorithm was...I Find the duplicate label in the two strategiesI Drop the label from player 1 and player 2 alternately
I Not hard to find duplicate (just read off from solution)I How to drop label?
I Just change the appropriate variable to be positiveI As large as possible while satisfying constraints
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example of pivoting
I Starting equation for player 1:
x2 = 1− x1 − x3
v = 2− 2x1 + x3 + z2
z1 = 2− 3x1 − 3x3 + z2
I Starting equation for player 2:
y2 = 1− y1
u = 6− 6y1 + w1
w2 = 1− 3y1 + w1
w3 = 3− 6y1 + w1
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example of pivoting
I Solve to get ((0, 1, 0), (0, 1))I s = (0, 1, 0), z = (2, 0)I w = (0, 1, 3), t = (0, 1)
I x1 = 0 and w1 = 0, so label 1 is duplicateI So we look to get rid of itI =⇒ Modify x1 while satisfying non-negativity constraints
I Relevant equations:x2 = 1− x1
z1 = 2− 3x1
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Example of pivoting
I For first equation: x1 ∈ (0, 1]
I For second equation: x1 ∈(0, 2
3
]I Clearly, x1 → 2
3I x2 → 1
3I But, z1 → 0!I As a result, we see that label 4 is duplicate
I Another duplicate =⇒ we repeat the process!
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Updating system
I Now x1 should appear on the LHS, since it is no longer zero
I New column basis is {D1,D2,D4}
Dβ =
1 1 01 0 −10 2 −1
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Updating system
I New system is
v =2
3− 3x3 −
2
3z1 +
1
3z2
x1 =1
3− x3 −
1
3z1 +
1
3z2
x2 =1
3+
1
3z1 −
1
3z2
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Complementary pivoting
I Newly added label will be duplicate
I e.g. Label 4 is added, and is a duplicateI Make y1 positive
I y2 = 1− y1 =⇒ y1 ∈ (0, 1]I w2 = 1− 3y1 + w1 =⇒ y1 ∈
(0, 1
3
]I So, new strategy is
(13 , 2
3
)I Now completely labelledI Hence, we have found the equilibria
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Performance of Lemke-Howson
I Worst-case exponential running timeI In practise, reasonably fast
I Of late, been beaten by a few competing algorithms e.g.[SHNP]
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Problems?
I Does not generalize to n > 2 players
I Sometimes, equilibria may be out of reach
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Other approaches
I Many more techniques...I Local-search techniques [SHNP]I Mixed integer programmingI Global Newton optimizationI Computer algebraI Markov Random FieldsI etc...
I Quite a few generalize to more than 2 players
I Nothing (as yet) tells us about the boundary of P!
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
Xi Chen, Xiaotie Deng, and Shang hua Teng.Settling the complexity of computing two-player nashequilibria.Technical report, 2007.
Ryan Porter, Eugene Nudelman, and Yoav Shoham.Simple search methods for finding a nash equilibrium.In AAAI, pages 664–669, 2004.
R. Savani and B. von Stengel.Exponentially many steps for finding a nash equilibrium in abimatrix game, 2004.
Lloyd S. Shapely.A note on the Lemke-Howson algorithm.Mathematical Programming Study 1:Pivoting and Extensions,pages 175 – 189, 1974.
Aditya Menon Computing Nash-equilibria in 2-player games and beyond
OutlineIntroductionComplexity
Finding equilibria in 2-player games
Lemke-Howson algorithmOther approaches
B. von Stengel.Computing equilibria for two-person games.Handbook of Game Theory, 3, 2002.
Aditya Menon Computing Nash-equilibria in 2-player games and beyond