Beef or Horse?Cheap talk

Beef or horse?

• In a restaurant, some customers prefer beef, some prefer horse.

• Type s likes beef, type t likes horse• Waitress asks if you want beef or horse.– For Player 1, Action A—Say beef, Action B—say horse– For Player 2, Action x—Bring horse, Acton y, bring beef

• If she brings you the kind you like, you are happier. If you are happier, you leave a bigger tip and waitress is happier.

• Simplicity of this case because payoff to Player 1, given action of Player 2 depends on one’s type, but not on action A or B.

Complications

• What if the customer is a foreigner and doesn’t know the words for beef and horse?

• Babbling equilibrum?

Bayesian waitress

• Customer is an American in Japan.• Waitress believes that only the fraction 1/ 4 of American

customers know the words for horsemeat and beef.• She believes that 9/10 of her American customers like

beef and don’t like horse. • Suppose that Americans who know the words order

what they prefer. Those who don’t are equally likely to say the word for horsemeat as that for beef.

• What should she do when an American orders beef? orders horse?

• If Americans who know the words order the meat they prefer and those who don’t order randomly, what is the probability that an American who orders horse really wants horse?

• Try Bayes’ law. Let H mean likes horse and B mean that he prefers beef. Let h mean orders horse. Then P(H|h)=P(H and h)/P(h).

• The event H and h occurs if American likes horse and either knows the words or guesses correctly.

• P(H and h)=(1/10)(1/4)+(1/10)(3/4)(1/2)=5/80• Probability that customer prefers beef and orders horse

is P(B and h)= (9/10)(3/4)1/2=27/80.• Then P(h)=P(H and h)+P(B and h)=32/8• So P(H|h)=5/32.

Babble and Horsemeat

• Since P(H|h)<1/2, the waitress will have a higher expected payoff from serving the customer beef even if he orders horsemeat.

• In equilibrium, waitress always brings beef for Americans, no matter what they say.

• Americans therefore would be indifferent about what they say.

Infinitely Repeated Games

Finitely Repeated Game

• Take any game play it, then play it again, for a specified number of times.

• A single play of the game that is repeated is known as the stage game.

• Let players observe all previous play.• For every history that you have observed, you

could have a different response.

Prisoners’ Dilemma

R, R S, T

T, S P, P

Cooperate Defect

Cooperate

Defect

PLAyER 1

Player 2

T > R > P > S “Temptation” “Reward” “Punishment” “Sucker”

Twice Repeated Prisoners’ Dilemma

Two players play two rounds of Prisoners’ dilemma. Before second round, each knows what other did on the first round. Payoff is the sum of earnings on the two rounds.

Two-Stage Prisoners’ DilemmaWorking back

Player 1

Cooperate Defect

Player 2

CooperateCooperateDefect Defect

Player 1 Player 1 Player 1 Player 1

C

C

C

C

C CD D D D

C C C D

Player 1Pl. 2 Pl 2

Pl 2 Pl 2

2R2R

D DC D C D C D DR+SR+T

R+TR+S

R+PR+P

S+RT+R

D

P+RP+R

P+SP+T

T+PP+S

2P2P

Etc…etc

Two-Stage Prisoners’ DilemmaWorking back further

Player 1

Cooperate Defect

Player 2

CooperateCooperateDefect Defect

Player 1 Player 1 Player 1 Player 1

C

C

C

C

C CD D D D

C C C D

Player 1Pl. 2 Pl 2

Pl 2 Pl 2D DC D C D C D D

1021

2110

1111

1021

022

1111

112

1111

D

220

121

1111

212

121

Longer Game

• What is the subgame perfect outcome if Prisoners’ dilemma is repeated 100 times?• Work backwards: In last round, nothing you do affects

future, so you play the dominant strategy for stage game: defect.

• Since last round is determined, nothing you do in next to last round affects future, so you play dominant strategy for stage game: defect

• Work your way back. Only subgame perfect outcome is “Defect always”.

In a repeated game that consists of four repetitions of a stage game that has a unique Nash equilibrium

A) There are four subgame perfect Nash equilibriaB) There are 24=16 subgame perfect Nash

equilibriaC) There is only one subgame perfect Nash

equilibrium.D) The number of subgame perfect Nash

equilibria varies, depending on the details of the game.

More generally

• In a subgame perfect equilibrium for a finitely repeated game where the stage game has a unique N.E, the moves in the last stage are determined for each person’s strategy. Given that the moves in the last stage don’t depend on anything that happened before, the Nash equilibrium in previous stage is uniquely determined to be the stage game equilibrium.

• And so it goes…All the way back to the beginning.

Games without a last round

• Two kinds of models– Games that continue for ever– Games that end at a random, unknown time

Infinitely repeated game

• Wouldn’t make sense to add payoffs.• You would be comparing infinities. • Usual trick. Discounted sums. • Just like in calculating present values.• We will see that cooperative outcomes can

often be sustained as Nash equilibria in infinitely repeated games.

Why consider infinite games?We only have finite lives.

• Many games do not have known end time.• Just like many human relationships.• Simple example—A favorite of game theorists• After each time the stage game is played there is

some probability d<1 that it will be played again and probability 1-d that play will stop.

• Expected payoff “discounts” payoffs in later rounds, because game is less likely to last until then.

Cleaning house as a Repeated Prisoners’ Dilemma

• Maybe a finite game if you have a fixed lease and don’t expect to see roommate again after lease expires.

• Most relationships don’t have a known last time.

• Usually some room for “residual good will.”

In a repeated game, after each round of play, a fair coin is tossed. If it comes up heads, the game continues to another

round. If it comes up tails, the game stops. What is the probability that the game is played for at least three rounds?

A) 1/3B) 2/3C) 1 /4D) 1 /2E) 1/8

Calculating sums

• In a repeated game, with probability d of continuation after each round, the probability that the game is still going at round k is dk-1

• Calculate expected winnings if you receive R so long as the game continues.R+dR+d2R+ d3R+ d4R + ….+=R(1+d +d2 + d3 + d4 + ….+ )• What is this infinite sum?

Adding forever• The series (1+d +d2 + d3 + d4 + ….+ )Is known as a geometric series. When |d|<1, this series converges. That is, to say, the limit as n approaches infinity of 1+d +d2 + d3 + d4 + ….+ dn exists. Let S= 1+d +d2 + d3 + d4 + ….+ Then dS=d +d2 + d3 + d4 + ….+ And S-dS=1.So S(1-d)=1S=1/(1-d).

Infinitely repeated prisoners’ dilemma and the “Grim Trigger Strategy”

• Suppose 2 players play repeated prisoners dilemma, where the probability is d<1 that you will play another round after the end of each round.

• The grim trigger strategy is to play cooperate on the first round and play cooperate on every round so long as the other doesn’t defect.

• If the other defects, the grim trigger strategy plays defect on all future rounds.

When is there a symmetric SPNE where all play Grim Trigger?

• Suppose that the other player is playing Grim Trigger.

• If you play Grim Trigger as well, then you will cooperate as long as the game continues and and you will receive a payoff of R.

Your expected payoff from playing Grim Trigger if the other guy is playing Grim Trigger is therefore R(1+d +d2 + d3 + d4 + ….+ )=R/(1-d)

What if you defect against Grim Trigger

• If you defect and the other guy is playing Grim Trigger, you will get a payoff of T>R the first time that you defect. But after this, the other guy will always play defect. The best you can do, then is to always defect as well.

• Your expected payoff from defecting is therefore T+ P(d +d2 + d3 + d4 + ….+ )

=T+Pd/1-d

Cooperate vs Defect• If other guy is playing Grim trigger and nobody has yet

defected, your expected payoff from playing cooperate is R/(1-d)

• If other guy is playing Grim trigger and nobody has yet defected, your expected payoff from playing defect is T+Pd/(1-d)

• Cooperate is better for you if R/(1-d)>T+Pd/(1-d) which implies d>(T-R)/(T-P)• Example If T=10, R=5, P=2, then condition is d>5/8.• If d is too small, it pays to “take the money and run”

Other equilbria?

• Grim trigger is a SPNE if d is large enough.• Are there other SPNEs?• Yes, for example both play Always Defect is an

equilibrium.• If other guy is playing Always Defect, what is

your best response in any subgame?• Another is Play Defect the first 10 rounds,

then play Grim Trigger.

Tit for Tat

• What if both players play the following strategy in infinitely repeated P.D?

• Cooperate on the first round. Then on any round do what the other guy did on the previous round.

• Suppose other guy plays tit for tat.• If I play tit for tat too, what will happen?

Payoffs

• If you play tit for tat when other guy is playing tit for tat, you get expected payoff of

R(1+d +d2 + d3 + d4 + ….+ )=R/(1-d)• Suppose instead that you choose to play “Always defect”

when other guy is tit for tat.• You will get T+ P(d +d2 + d3 + d4 + ….+ ) =T+Pd/1-dSame comparison as with Grim Trigger. Tit for tat is a better response to tit for tat than always defect if d>(T-R)/(T-P)

Another try

• Sucker punch him and then get him to forgive you.

• If other guy is playing tit for tat and you play D on first round, then C ever after, you will get payoff of T on first round, S on second round, and then R for ever. Expected payoff is T+ Sd+d2R(1+d +d2 + d3 + d4 + ….+ )=T+ Sd+d2R/(1-d).

Which is better?• Tit for tat and Cheat and ask forgiveness give same payoff

from round 3 on.• Cheat and ask for forgiveness gives T in round 1 and S in

round 2. • Tit for tat give R in all rounds.• So tit for tat is better if R+dR>T+dS, which means d(R-S)>T-R or d>(T-R)(R-S) If T=10, R=6, and S=1, this would mean if d>4/5.But if T=10, R=5, and S=1, this would be the case only if d>5/4, which can’t happen. In this case, tit for tat could not be a Nash equilibrium.

Back to Aesop’s shepherd boy

The tale

• There was once a young Shepherd Boy who tended his sheep at the foot of a mountain near a dark forest. It was rather lonely for him all day, so he thought upon a plan by which he could get a little company and some excitement. He rushed down towards the village calling out “Wolf, Wolf,” and the villagers came out to meet him, and some of them stopped with him for a considerable time.

• This pleased the boy so much that a few days afterwards he tried the same trick, and again the villagers came to his help.

• But shortly after this a Wolf actually did come out from the forest, and began to worry the sheep, and the boy of course cried out “Wolf, Wolf,” still louder than before. But this time the villagers, who had been fooled twice before, thought the boy was again deceiving them, and nobody stirred to come to his help.

The moral of the story

So the wolf made a good meal off the boy’s flock, and when the boy complained, the wise man of the village said:

“A liar will not be believed, even when he speaks the truth.”

The stage game

• Let’s draw the extensive form on the blackboard.

• What are the strategies for the boy?• What are the strategies for the villagers?

What are the equilibria if it is played just once?

• Is there an equilibrium in which the boy cries Wolf if he sees a wolf and is quiet if he sees no wolf?

• If there is an equilibrium where boy uses this strategy, what will the strategy of the villagers be?

• If that is the villagers’ strategy, what will the boy’s best reply be?

• What could be a Nash equilibrium?

Aesop’s tale is about what will happen in repeated interaction

• Suppose that there is repeated play where after each day, the probability is d that the same game will be played the next day?

• Suppose also that on any day, the wolf comes with probability p.

• Finally, suppose that the villagers use a “grim trigger strategy”. They will come when the boy cries “Wolf!” so long as he never fools them.

• If he falsely calls “Wolf”, they will come the first time, but they will never come to his aid in the future.

Equilibrium with grim trigger

• Suppose the villagers use the grim trigger strategy.

• The boy sees whether a wolf is coming. Suppose that he always cries “Wolf” when he sees a wolf, and is quiet when he doesn’t see a wolf.

• Villagers will always come when he calls wolf. His payoff in every period will be 0.

How about cheating?

• Would it ever pay him to cheat? Consider the first time that he doesn’t see a wolf. If he cries “Wolf”, he will get a payoff of 1 in the first period instead of boring old 0.

• But then nobody will ever help him in the future. • In any period in the future the wolf comes with

probability p and his payoff if the wolf comes is -10.• His total expected payoff if he cheats is therefore 1-10p(d+d2+…dn+…)=1-10pd/(1-d).

What should he do?• If he always tells truth his payoff is 0 in every period and

hence his total payoff is 0.• If he falsely cries “Wolf” his payoff is 1-10pd/(1-d)• If 1-10pd/(1-d)<0, his best response to the grim trigger is

to always tell the truth. • Suppose d=.9 and p=1/50. • 1-10pd/(1-d)=1-90/50<0 and it pays him to tell the truth. • But if p=1/100, then 1-10pd/(1-p)=1-90/100>0 andeven with the grim trigger he wouldn’t tell the truth.

Whats wrong with grim trigger?

• What if the boy makes a mistake and thinks he sees a wolf but there isn’t one?

• What if he doesn’t believe the villagers will stick to their threat?

• Anything else?

See you next week

Problems from Chapter 12

Problem 1, Chapter 12

Find a separating equilibrium• Trial-and-error. Two possible separating strategies for

Player 1:– Choose A if type s , B if type t.– Choose B if type s, A if type t.

• Player 2 sees action A or B, does not see type. If Player 2 believes that Player 1 will choose A if s and B if t, then Player 2’s best response strategy will be y if A, x if B.

• Given Player 2’s strategy, – Type s would rather have Player 2 do y than x and so would

choose A– Type t would rather have Player 2 do x than y and so would do B.

• So the strategies A if type s , B if type t for Player 1 andy if A, x if B constitute a separating Bayes-Nash equilibrium.

Problem 5, Page 384

Find a separating Bayes-Nash equilibrium

• Candidate for equilibrium behavior by Sender: Say m1 if type 1, m2 if Type 2.

• If Receiver believes that this is Sender’s strategy, best response is B if m1, C if m2.

• If Receiver plays B if m1, C if m2, then Sender’s best response is m1 if type 1, m2 if Type 2.

• This is a Bayes-Nash equilibrium. The beliefs of each player about what the other player will do are confirmed by the responses.

Find a pooling equilibrium

• Suppose that Player 1 says m1 if type 1 and m1 if type 2.• What is the best response for Player 2?

– Expected payoff from A is .6*3+.4*4=3.2– Expected payoff from B is .6*4+.4*1=2.8– Expected payoff from C is .6*0+.4*5=2

• So A if m1, A if m2, is a best response for 2.• If Player 2 always goes to A, then it doesn’t matter what

Player 1 says, so m1 if type 1 and m1 if type 2 is a best response. – So is flip a coin about what to say

• This is a pooling equilibrium. Beliefs are confirmed by outcomes.

Answer to part c• Suppose probability that Sender is type 1 is p. When is there a pooling

Bayes-Nash equilibrium where receiver always plays B?• Suppose Sender plays m1 if type 1 and m1 if type 2. • Expected payoffs to Receiver from strategies:

– From A 3p+4(1-p)– From B 4p+(1-p)– From C 0p+5(1-p)

• For what p’s is payoff from B the largest?– B beats A, 4p+1-p>3p+4(1-p) if p>3/4. – B beats C, 4p+1-p>0+5(1-p) if p>1/2

• So Always play B is a best response to Sender’s strategy of m1 if type 1 and m1 if the probability of type 1 if p>3/4.• If Receiver always plays B, Sender is indifferent about what signal to send

and might always say m1 or always say m2 or just “babble”.