evolution & economics no. 4
DESCRIPTION
Evolution & Economics No. 4. Evolutionary Stability in Repeated Games Played by Finite Automata. Automata. K. Binmore & L. Samuelson J.E.T. 1991. C. C. C,D. D. D. D. C. D. C. D. C. Grim. Tit For Tat (TFT). C. C. C. C. D. D. D. D. C. C. D. D. Tweedledum. - PowerPoint PPT PresentationTRANSCRIPT
1
Evolution & EconomicsEvolution & EconomicsNo. 4No. 4
2
Evolutionary Stability in Repeated Evolutionary Stability in Repeated Games Played by Finite AutomataGames Played by Finite Automata
K. Binmore & L. SamuelsonK. Binmore & L. SamuelsonJ.E.T. 1991J.E.T. 1991
AutomataAutomata
3
Finite Automata playing the Prisoners’ DilemmaFinite Automata playing the Prisoners’ Dilemma
C DD
C,DC
Grim
C DD
DC
Tit For Tat (TFT)
C
D CD
CC
Tat For Tit (TAFT)
D
D CD
CC
TweedledumD
states (& actions)
transitions
4
Automata playing the Prisoners’ DilemmaAutomata playing the Prisoners’ Dilemma
D CD
C,DC
CA
D CC,D
C
TweedledeeD
D
C,D
D
C
C,D
C
5
• Two Automata playing together, eventually follow a cycle
(handshake) • The payoff is the limit of the means.• The cost of an automaton is the number of his states.• The cost enters the payoffs lexicographically.
lim
Let be automata, and let be the payoff at stage
of the repeated game.
The payoff in the repeated game is the limit of the means:
The no
i
T -1
ij=0T
a,b p a,b i
1π a,b = p a,bT
..of states of an automaton is denoted by: a a
6
C D
C 2 , 2 -1 , 3
D 3 , -1 0 , 0
The Structure of Nash Equilibrium in Repeated Games with Finite AutomataDilip Abreu & Ariel Rubinstein
Econometrica,1988
In Abreu Rubinstein, The fitness satisfies
iff:
and
if
>
th
e .n
U a,b
U a,c U b,c
π a,c π b,c
π a,c =
i)
π b,
ii) c a b
7
C D
C 2 , 2 -1 , 3
D 3 , -1 0 , 0
The Structure of Nash Equilibrium in Repeated Games with Finite AutomataDilip Abreu & Ariel Rubinstein
Econometrica,1988
(0,0)
(3,-1)
(-1,3)
(2,2)
N.E. of repeated Game
N.E in Repeated Games with Finite Automata(Abreu Rubinstein)
8
An automaton is an ESS, if for all automata
and
if then
a
.
nd
if and then
a b a :
π a,a π b,a
π a,a = π b,a π a,b π b,b
π a,a = π b,a π a,b = π b,b
i)
ii)
i ) ai bi
Binmore Samuelson:
9
For any game , if automaton is an ESS,
it has a single state.
G a Lemma :
If is then any ESS must have a single state.
If has more than one action, let it have an ESS
with more than one state.
G 1x1, G
a
Proof :
Let begin with the action a x. x ?x
ay
Let be identical to except that
it acts differently when it
observes at the start.
b a,
y
x?
x
ay
If then: x?
x
by
10
For any game , if automaton is an ESS,
it has a single state.
G a Lemma :
If is then any ESS must have a single state.
If has more than one action, let it have an ESS
with more than one state.
G 1x1, G
a
Proof :
Let begin with the action a x. x ?x
ay
Let be identical to except that
it acts differently when it
observes at the start.
b a,
y
If then: x?
x
by
x?
x
ay
11
For any game , if automaton is an ESS,
it has a single state.
G a Lemma :
If is then any ESS must have a single state.
If has more than one action, let it have an ESS
with more than one state.
G 1x1, G
a
Proof :
Let begin with the action a x. x ?x
ay
Let be identical to except that
it acts differently when it
observes at the start.
b a,
y
can invade the population
hence cannot be an ESS.
b a, a Q.E.D.
12
For any game , if automaton is an ESS,
it has a single state.
G a Lemma :
For the the two singletons are: P.D.
D
C,D
D
C
C,D
C
C is not an ESS, it can be invaded by D.
D is not an ESS, it can be invaded by Tit For Tat.
C DD
DC
Tit For Tat (TFT)
C
The has no ESS. P.D.
13
A Modified ESS. MESS.Definition :
An automaton is an ESS, if for all automata
and
if then
and
if and
. then
i)
ii)
a b a :
π a,a π b,a
π a,a = π b,a π a,b π b,b
π a,a = π b,a π a,b = π b,b i) ai bi
An automaton is an , if for all automata
and
if then
and
if and t
MESS
.hen
a b a :
π a,a π b,a
π a,a = π b,a π a,b π b,b
π a,a = π b,a π a,b = π b,b
i)
ii)
i ) ai bi
14
If is a MESS then it uses all its states
when it plays against itself.
aLemma :
C
Assume that does not use its state when playing
against itself. onstruct an automaton identical to
except that state has been deleted.
a s b, a
s
Proof :
There is no difference between when playing
t .
or
Bu
a,b a b. a < b
Hence such cannot be an ESS.a
Q.E.D.
15
If is a MESS then it uses all its states
when it plays against itself.
aLemma :
C
Assume that does not use its state when playing
against itself. onstruct an automaton identical to
except that state has been deleted.
a s b, a
s
Proof :
There is no difference between when playing
t .
or
Bu
a,b a b. a < b
Hence such cannot be an ESS.a Q.E.D.
In the P.D. Tit For Tat and Grim are not MESS(they do not use one state against themselves)
C DD
C,DC
Grim
C DD
DC
Tit For Tat (TFT)
C
16
= .
An automaton for this game is a pair:
The complexity of
, .
.
1 2
1 2 2 1
1 2+
a = a a
1a,b = π a ,b + π a ,b2a : a a a
π
For a general, possibly non symmetric game G.
Define the symmetrized version of G: G # #. A player is player 1 with probability 0.5 and player 2 with probability 0.5
The previous lemmas apply to (a1,a2)
1. An ESS has a single state │a1│=│a2│=1
2. If (a1,a2) is a MESS it uses all its states when playing against itself,
i.e. a1,a2 use all their states when playing against the other.
17
max
Consider the highest payoff that can be attained in
actions of
This can be achieved by the automaton playing against itself.,
1 2
1 2
1 1 2 2 1 2s ,s
* *1 2
s ,s G.
G.1 G s ,s + G s ,s2
a* = a ,a
a utilitarian automaton.a* -
If an automaton in is a MESS then it is utilitarian.# # .GLemma :
Let be a non utilitarian MESS.
There exists a mutant which earns a higher payoff.
a b
Proof :
The mutant has the following properties:
starts with an action different to that of .i i
bb a1.
If the initial action of the opponent is not that of then it must
be and continues by imitating the utilitarian automaton
.*i
-i
-i i
a
b b a
2.
,
If the initial action of the opponent is that of then fools it to
'believe' that it plays itself, and so obtains the payoff
.
-i i
i i -i
a b
G a a
3.
18
starts with an action different to that of . i ib a1.
If the initial action of the opponent is not that of then it must
be and continues by imitating the utilitarian automaton
.*i
-i
-i i
a
b b a
2.
,
If the initial action of the opponent is that of then fools it to
'believe' that it plays itself, and so obtains the payoff
.
-i i
i i -i
a b
G a a
3.
Properties are easy to obtain.
Property is less obvious. It cannot be done with .
But goes through all its states when playin . g -i ia a
1,2
3 Grim
If initial action made move to the state If is reached when
plays who is then in state , then mimics in stat
's
e i -i -i
i i i
b a q. q a
a q* b a q.
b,a a,a a,bmimics so gets the same against as agai s n tb a, a b a.
but b,b a*,a* a,a = a,b b,b a,b
cannot be a MESS.a
Q.E.D.
19
To prove existence of a MESS is like in repeated game (Folk Theorem)
If is the minimax of , then any payoff above it can be supported a MESS.m G
In the
No 'nice' automaton can be a MESS: It can be invaded by , which is shorter.CPrisoners' Dilemma :
20
D CD
CC
Tat For Tit (TAFT)
D
is a MESS :Tat For Tit
It can be invaded by:
D CD
C,DC
CA
D CC,D
C
ACD
C DD
CC
CCD
C DD
C
CDC,D
D CC,D
C,D
AA
21
D CD
CC
Tat For Tit (TAFT)
D
is a MESS :Tat For Tit
It can be invaded by:
D CC,D
C
ACD
22
D CD
CC
Tat For Tit (TAFT)
D
is a MESS :Tat For Tit
It can be invaded by:
D CD
C,DC
CA
23
D CD
CC
Tat For Tit (TAFT)
D
is a MESS :Tat For Tit
It can be invaded by:
C DD
CC
CCD
24
D CD
CC
Tat For Tit (TAFT)
D
is a MESS :Tat For Tit
No other (longer and more sophisticated) automaton can invade.
Any exploitation of TAFT (playing D against his C) makes TAFT play D,
so the average of these two periods is (3+0)/2 = 1.5 < 2, the average of cooperating.
C D
C 2 , 2 -1 , 3
D 3 , -1 0 , 0
25
Polymorphic MESS
A population consisting of: D CD
CC
Tat For Tit (TAFT)
D
C DD
CC
CCD
C DD
C
CDC,D
can be invaded only by:
D CD
C,DC
CA
D CC,D
C
ACD
If AC invaded, it does not do well against CD
D C D C …….C D C D …….
26
Polymorphic MESS
A population consisting of: D CD
CC
Tat For Tit (TAFT)
D
C DD
CC
CCD
C DD
C
CDC,D
can be invaded only by
CAD CD
C,DC
CA
D CC,D
C,D
AA
If AA invaded, it does not do well against CC
D C C C C…….C D D D D…….
27
Polymorphic MESS
A population consisting of: D CD
CC
Tat For Tit (TAFT)
D
C DD
CC
CCD
C DD
C
CDC,D
can be invaded only by
CAD CD
C,DC
CAbut if CA invaded then a sophisticated
automaton S can invade and exploit CA .S starts with C. if it saw C it continues with C
forever (the opponent must be CD or CC ).
If it saw D, it plays D again, if the other then
plays D it must be TAFT. S plays another D
and then C forever.
If, however, after 2x D, the other played C, then it
must be CA, and S should play D forever.
28
Polymorphic MESS
A population consisting of: D CD
CC
Tat For Tit (TAFT)
D
C DD
CC
CCD
C DD
C
CDC,D
can be invaded only by
CAD CD
C,DC
CA
When S invades, CA will vanish,
and then S which is a complex automaton will die out.