extensive games with imperfect information. extensive games with imperfect info in this section, we...

Extensive Games with

Imperfect Information

Extensive Games with Imperfect Info

• In this section, we model situations in which players move sequentially but may or may not be informed about previous player’s actions.

• With perfect information, we needed to specify players, terminal histories, when players had the move, preferences and payoffs.

• With imperfect information, we need to add to that list: a specification about each player’s information about the history at every point at which she moves.

Definition: an information set for a player is a collection of decision nodes satisfying:

– The player has the move at every node in the information set, and

– When the play of the game reaches a node in the information set, the player with the move does not know which node in the

information set has (or has not) been reached


Example 314.2 in Osborne: Battle of the Sexes


I

IIII

U2

U2U2

CP

CPCP

2,1 1,20,00,0

Nash equilibria?

Subgame Perfect Nash Equilibria ?

Other Examples• 315.1 in Osborne: Card Game (Like problem set question)


Definition: Strategy in an Extensive game of Imperfect information: A (pure) strategy of player i in an extensive game is a function that assigns to each of i’s information sets, Ii , an action in A(Ii ) (ie, the set

of actions available to player i at information set Ii ).

Definition: Mixed Strategy in an Extensive game of Imperfect information: A mixed strategy of a player in an extensive game is a probability distribution over the player’s pure strategies.


Definition: Nash Equilibrium in an Extensive game of Imperfect information: The mixed strategy profile, *, in an extensive game is a (mixed strategy) Nash equilibrium if, for each player i and every mixed strategy i of player i, player i’s expected payoff to * is at least as large as her expected payoff to (i,*-i) according to a payoff function whose expected value represent player i’s preferences over lotteries.


320.2 in Osborne (Commitment and Observability)


X

Y

Y

X (3,2)

(4,3) (2,4)

(1,1)

I

II



X

Y

Y

X

NE of Simultaneous game: (Y,Y)

(3,2)

(4,3) (2,4)

(1,1)

I

II



X

Y

Y

X

SPNE outcome of extensive game (I moves first, II second) of perfect information: (X,X)

(3,2)

(4,3) (2,4)

(1,1)

I

II

I

Y

4,3 2,4

X YYX

X

IIII

1,13,2

NE of Simultaneous game: (Y,Y)

320.2 in Osborne (Commitment and Observability) -- Imperfect Information Game:


IX Y

X X

X X

X X

Y

Y Y

Y

Y

Y

II

II

ChanceChance

1-

1-

3,2

4,31,13,2

1,1 4,3 2,4

2,4

•Nash Equilibrium for < 1/4 ?

How many information sets?

How many subgames?

IX Y

X X

X X

X X

Y

Y Y

Y

Y

Y

II

II

ChanceChance

1-

1-

3,2

4,31,13,2

1,1 4,3 2,4

2,4


•Player I moves first, but his choice is imperfectly observed by player II.•Strategies: player I: (X, Y) player II: ( (X,X),(X,Y),(Y,X),(Y,Y) ). For player II, a strategy (a,b) means play a if the signal is X and play b if the signal is Y.

IX Y

X X

X X

X X

Y

Y Y

Y

Y

Y

II

II

ChanceChance

1-

1-

3,2

4,31,13,2

1,1 4,3 2,4

2,4


I\II (X,X) (X,Y) (Y,X) (Y,Y)

X 3,2 3-2e,2-e* 1+2e,1+e 1,1

Y 4,3 2+2e,4-e 4-2e,3+e 2,4

Strategic Form:

* Payoff calculated as: ( 3(1-e)+1(e), 2(1-e)+1(e) ) = (3-2e,2-e)

IX Y

X X

X X

X X

Y

Y Y

Y

Y

Y

II

II

ChanceChance

1-

1-

3,2

4,31,13,2

1,1 4,3 2,4

2,4


I\II (X,X) (X,Y) (Y,X) (Y,Y)

X 3,2 3-2e,2-e* 1+2e,1+e 1,1

Y 4,3 2+2e,4-e 4-2e,3+e** 2,4

Strategic Form:

*3-2e>2+2e -4e>-1 e < 1/4

**4-2e>1+2e -4e>-3 e < 3/4

Equilibrium = (Y,YY)



X

Y

Y

X

SPNE of extensive game (I moves first, II second) of perfect information: (X,X) outcome = (3,2)

(3,2)

(4,3) (2,4)

(1,1)

I

II

I

Y

4,3 2,4

X YYX

X

IIII

1,13,2

NE of Simultaneous game: (Y,Y) outcome = (2,4)

(Bayesian) NE outcome of sequential imperfect information game (Y,YY) outcome (2,4)

Player 1 loses his first-mover advantage!

Osborne: 317.1 / 322.1 / 323.1: Entry Game

Motivates a need for “beliefs” of players.


• In simultaneous (strategic) games of perfect information, our equilibrium concept was the Nash equilibrium (NE). When we moved to sequential games, we introduced a slightly stronger equilibrium called subgame perfect Nash equilibium (SPNE), primarily to rule out non-credible threats. In static games of imperfect information, we needed a further “refinement” which we called Bayesian Nash Equilibrium (BNE). Finally, in extensive games of imperfect information, we make a further refinement and introduce Perfect Bayesian Nash Equilibrium, or (PBE) for short.


A note on terminology:

Perfect Bayesian Equilibrium or PBE

Weak Sequential Equilibrium or WSE (Osborne’s terminology)

PBE = WSE


Perfect Bayesian Equilibrium*

• Gibbons Game, Page 176

II II

I

LM

R

R’R’L’L’

(2,1) (0,0) (0,2) (0,1)

(1,3)


• Gibbons Game, Page 176 – Strategic Form

II

I

L

M

R

R’L’

(2,1) (0,0)

(0,2) (0,1)

(1,3) (1,3)

• So 2 NE in pure strategies: (L,L’) and (R,R’)• Subgame Perfect NE? Since the game has

no subgames besides the whole game,

NE=SPNE.• But player II playing R’ is based on a non-credible threat since L’ is always optimal if player II gets to move.

So we rule out (R,R’) with the following 2 refinements.


• Requirement 1: At each information set, the player with the move must have a belief about which node in the information set has been reached by the play of the game. For a nonsingleton information set, a belief is a probability distribution over the nodes in the information set; for a singleton information set, the player’s belief puts probability one on the single decision node.


2. Requirement 2: Given their beliefs, the players’ strategies must be sequentially rational. That is, at each information set the action taken by the player with the move (and the player’s subsequent strategy) must be optimal given the player’s belief at that information set and the other players’ subsequent strategies (where a “subsequent strategy” is a complete plan of action covering every contingency that might arise after the given information set has been reached)



II II

I

LM

R

R’R’L’L’

(2,1) (0,0) (0,2) (0,1)

(1,3)

[p] [1-p]

• (p,1-p) are the beliefs of player II. p is the probability that player II puts on the history that player I has played L and (1-p) is the probability that player II puts on the history that player I has played M.



II II

I

LM

R

R’R’L’L’

(2,1) (0,0) (0,2) (0,1)

(1,3)

[p] [1-p]

• So, given player II’s beliefs, the expected payoff to player II from playing R’ is 0*p + 1*(1-p) = 1-p. The expected payoff to player II from playing L’ is 1*p + 2*(1-p) = 2-p.

• Since 2-p > 1-p, for ALL p, then R’ can never be played in equilibrium.• So (R,R’) does not satisfy the requirement #2.


Definition: for a given equilibrium in a given extensive game, an information set is on the equilibrium path if it will be reached with positive probability if the game is played according to the equilibrium strategies, and is off the equilibrium path if it is certain not to be reached if the game is played according to the equilibrium strategies.

Requirements 1 and 2 insist that the players have beliefs and act optimally given these beliefs, but not that these beliefs be reasonable. We need further requirements.


• Requirement 3: At information sets on the equilibrium path, beliefs are determined by Bayes’ rule and the players’ equilibrium strategies.

• Requirement 4 :At information sets off the equilibrium path, beliefs are determined by Bayes’ rule and the players’ equilibrium strategies where possible.



II II

I

LM

R

R’R’L’L’

(2,1) (0,0) (0,2) (0,1)

(1,3)

[p] [1-p]

• So in the SPNE where players play (L,L’), player 2’s belief must be p = 1. “Given player 1’s equilibrium strategy (L), player 2 knows which node in the information set has been reached.”

• As a second example, suppose in a mixed strategy equilibrium, player I played L with probability q1, M with probability q2, and R with probability 1-q1-q2. Then, this would force player II to set p = q1/(q1+q2), by Bayes rule.


Definition: A Perfect Bayesian Equilibrium consists of strategies and beliefs satisfying 1 through 4.

• Crucial new features of a PBE: beliefs are elevated to the level of importance of strategies in our definition of an equilibrium. An equilibrium consists of strategies and beliefs for each player at each information set at which the player has the move.

• Players must have reasonable beliefs both on and off the equilibrium path.

Signaling Games• Signaling Games: a dynamic game of

imperfect information involving a sender and a receiver, S and R.

• Nature draws a type for the S from a set of possible types. S observe his type and chooses a message to send to R. R observes the message (not the type) and then choose an action.

• Payoffs are then realized.

Signaling Games• Sender may be a worker who signals to a

potential employer (Reciever) with an education choice based on his type, possibly his productivity level. See Spence (1973): Job Market Signaling.

Signaling Games• Consider the following signaling game:

Nature Receiver

Sender t1

Sender t2

Receiver

z

1-z

m2m1

m1 m2

a1

a1 a1

a1

a2

a2a2

a2

If both types of senders send the same message in an equilibrium, we call this equilibrium “Pooling”

If a sender sends a different message depending on his type, we called this equilibrium “Separating.”

Signaling Games• Denote (p,1-p) and (q,1-q) denote R’s beliefs at each of his (two)

information sets.

Nature Receiver

Sender t1

Sender t2

Receiver

z

1-z

m2m1

m1 m2

a1

a1 a1

a1

a2

a2a2

a2

This game has four possible (pure strategy) equilibria: 1) Pooling on m1, 2) pooling on m2, 3) Separating with t1 playing m1 and t2 playing m2, 4) Separating with t1 playing m2 and t2 playing m1.

[p]

[1-p] [1-q]

[q]

Signaling Games• Lets add payoffs and solve an example.

Nature Receiver

Sender t1

Sender t2

Receiver

0.5

0.5

RL

L R

u

u u

u

d

dd

d

Solve for all PBE of this game ...

[p]

[1-p] [1-q]

[q]

1,3

1,2

1,0

0,0

2,1

0,1

2,4

4,0

Signaling GamesNeed to consider the following candidate equilibria:1) Pooling on (L,L); 2) Pooling on (R,R); 3) Separating on (L,R); 4) Separating on (R,L). For each type of candidate equilibrium we need strategies of each player

and beliefs of any playerthat is uncertain at any point in the game. In this game, the receiver is

uncertain at two of hisinformation sets (the one following the L signal and the one following the

R signal). So overall, we need:

• A strategy for the sender, (X,Y) (what he does if he’s type 1, what he does if he’s type 2).

• A strategy for the receiver, (X,Y) (what he does if he sees the L signal, what he does if he sees the R signal).

• A belief for the receiver at the left side information set, (p,1-p) (his belief that the L signal came from a type 1 sender, his belief that the L signal came from a type 2 sender).

• A belief for the receiver at the right side information set, (q,1-q) (his belief that the R signal came from a type 1 sender, his belief that the R signal came from a type 2 sender).

Signaling GamesPooling on (L,L).

• This means that the sender plays L if he’s type 1 or type 2. • On the left, the receiver doesn’t get any additional information about the sender’s type when he sees L

being played (because both types are playing L). So the receivers belief should be p = ½, the prior probability distribution of types.

• What does the receiver do if he sees the L signal? We see he always plays u on the left for any beliefs (including ½, ½).

• What does the receiver do if he sees the R signal? We see he would like to play u on the top, d on the bottom, but he cannot distinguish top from bottom. He can find beliefs (q,1-q) to make him want to do either.

• Now lets see if the sender really wants to player (L,L). Playing (L,L) results in the receiver playing u, so a type 1 sender gets a payoff of 1 and a type 2 sender gets a payoff of 2.

– If a type 1 sender deviates to R, his payoff is either 2 (if u) or 0 (if d). So in order for this to be a PBE, a type 1 sender requires that the receiver have beliefs such that he wants to play d if the receiver sees the R signal. Otherwise, the type 1 sender would want to deviate from L (and get 1) to R (and get 2). So a type 1 sender requires Erec[d|R] >= Erec[u|R] 0q+2(1-q) >= 1q+0(1-q) q <= 2/3.

– If a type 2 sender deviates to R, his payoff is either 1 (if u) or 1 (if d). These are both less than what he gets by playing L (2), so we say that a type 2 sender puts no requirement on the beliefs of the receiver following the R signal.

So we have a PBE at {(L,L), (u,d), (p,1-p), (q,1-q) | p = ½, q <= 2/3 }

Signaling GamesPooling on (R,R).

• This means that the sender plays R if he’s type 1 or type 2.

• On the right, the receiver doesn’t get any additional information about the sender’s type when he sees R being played (because both types are playing R). So the receivers belief should be q = ½, the prior probability distribution of types.

• What does the receiver do if he sees the R signal? With q = ½, E rec[u|R] = 1(1/2) + 0(1/2) = ½, while Erec[d|R] = 0(1/2) + 2(1/2) = 1. So the receiver should play d if he sees R.

• What does the receiver do if he sees the L signal? We see he always plays u on the left, no matter his beliefs (it’s a dominant strategy).

• Now lets see if the sender really wants to player (R,R). Playing (R,R) results in the receiver playing d, so a type 1 sender gets a payoff of 0 and a type 2 sender gets a payoff of 1.

– If a type 1 sender deviates to L, his payoff is 1, higher than he gets by playing R, so he would want to deviate and (R,R) cannot be part of a PBE. We could stop here …

– If a type 2 sender deviates to L, his payoff is 2, again higher than what he gets by playing R, so he would also want to deviate.

So there does not exist a PBE involving the sender pooling on R.

Signaling GamesSeparating on (L,R).

• This means that a type 1 sender plays L and a type 2 sender player R.

• Given our definition of a PBE, beliefs are determined by equilibrium strategies. Since both information sets of the receiver are ON the equilibrium path, the only beliefs that are consistent with (L,R) is p = 1 and q = 0. I.e., if only type 1 senders play L, then if a receiver sees L, he should believe that it came from a type 1 sender with probability 1. If only type 2 senders play R, then if a receiver sees R, he should believe that it came from a type 2 sender with probability 1.

• So, the receiver’s optimal strategy is (u,d).

• Now lets see if the sender really wants to player (L,R). Playing (L,R) results in the receiver playing (u,d), so a type 1 sender gets a payoff of 1 and a type 2 sender gets a payoff of 1.

– Should a type 1 sender deviate? By deviating to R, his payoff becomes 0, so he doesn’t want to deviate.– Should a type 2 sender deviate? By deviating to L, his payoff becomes 2, so he does want to deviate.

So there does not exist a PBE involving the sender separating on (L,R).

Signaling GamesSeparating on (R,L).

• This means that a type 1 sender plays R and a type 2 sender player L.

• Given our definition of a PBE, beliefs are determined by equilibrium strategies. Since both information sets of the receiver are ON the equilibrium path, the only beliefs that are consistent with (R,L) is p = 0 and q = 1. I.e., if only type 1 senders play R, then if a receiver sees R, he should believe that it came from a type 1 sender with probability 1. If only type 2 senders play L, then if a receiver sees L, he should believe that it came from a type 2 sender with probability 1.

• So, the receiver’s optimal strategy is (u,u).

• Now lets see if the sender really wants to player (R,L). Playing (R,L) results in the receiver playing (u,u), so a type 1 sender gets a payoff of 2 and a type 2 sender gets a payoff of 2.

– Should a type 1 sender deviate? By deviating to L, his payoff becomes 1, so he doesn’t want to deviate.– Should a type 2 sender deviate? By deviating to R, his payoff becomes 1, so he doesn’t want to deviate.

So there exists a PBE at {(R,L), (u,u), (p,1-p), (q,1-q) | p = 0, q = 1 }

Signaling GamesIn general,

1) Pooling on (L,L) p= the prior2) Pooling on (R,R) q= the prior3) Separating on (L,R) p=1, q=04) Separating on (R,L) q=1, p=0

The Market for Lemons• Akerlof models used car market. • Buyers value good cars at bG and lemons at bL.• Sellers value good cars at sG and lemons at sL.• Sellers know the true quality of the car and offer a price,

P. • Buyers only know a certain proportion of cars, q, are

good and a proportion, 1-q, are lemons.

The Market for Lemons• Safe assumption: bG>bL and sG > sL.• Assume that under perfect information, trade would take place for all

cars:– bL > sL

– bG > sG

• Under imperfect information, buyers buy if E[V] = q*bG+(1-q)*bL P• In a separating equilibrium, owners of good cars do not sell their cars

and owners of lemons sell their cars. This happens if:– bL = P < sG

• this is easily satisfied (see homework 5)

• How about a pooling equilibrium where all cars are sold at the price P. – Buyers Require: P q*bG+(1-q)*bL

– Sellers Require: P sG

– So: sG q*bG+(1-q)*bL

• this is hard to satisfy (see homework 5) especially if q is small.– For reasonable valuations and probabilities, a pooling equilibrium will not exist.

bG

sG

sL

bL

If q is small, q*bG+(1-q)*bL might be here

Card Game• 2 Players. Each player antes $1 into a pot.• Player 1 picks a card from a deck that is either High or Low with

equal probability. Player 2 does not observe the card.• Player 1 can then See or Raise. If he Sees and the card is High,

player 1 wins the pot for a payoff of (+1,-1). If he Sees and the card is Low, player 2 wins the pot for a payoff of (-1,+1).

• If player 1 Raises, then player 2 can either Pass or Meet. Again player 2 does not know if the card is High or Low.

• If player 2 Passes, player 1 wins the pot for a total payoff of (1,-1) no matter if the card is High or Low.

• If player 2 Meets, then if the card is High, player 1 wins the pot with a total payoff of (2,-2) and if the card is Low, player 2 wins the pot with a total payoff of (-2,2).

• What is the Nash Equilibrium (not PBE) of this game?

Card Game• Extensive Form Game:

11

Nature

High Low

See

2

See

Raise Raise

PassPassMeet Meet

+1,-1

+1,-1 +1,-1

-1,+1

+2,-2 -2,+2

•Player 1 has two information sets. Player 2 has one.•Strategies of player 2: (Pass or Meet)•Strategies of player 1: (Raise,Raise),(Raise,See),(See,Raise),(See,See) i.e., what player 1 does if the card is High and what player 1 does if the card is Low.

Card GameStrategic Form:

Player 1\2 Pass Meet

R,RR,SS,RS,S

1,-10,01,-10,0

0,01/2,-1/2-1/2,1/20,0


Player 1\2 Pass Meet

R,RR,SS,RS,S

1,-10,01,-10,0

0,01/2,-1/2-1/2,1/20,0

•So no pure strategy NE.•Mixed? Notice (S,S) is strictly dominated by for player 1 by an equal mixture of (R,R) and (R,S).


Player 1\2 Pass (q)

Meet (1-q)

1’s Expected Payoff

R,R (p1)R,S (p2)S,R (p3)

1,-10,01,-1

0,01/2,-1/2-1/2,1/2

q½(1-q)q-1/2(1-q)

•Note that if q=1 player 1 would mix on p1 and p3 with p2= 0 q=0 contradiction so q<1.•Note that if q=0 player 1 would only mix on p2 with p1=p3=0 q=1 contradiction so q>0.•So player 2 is mixing with positive weight on both pass and meet.•Also player 1 can’t be playing a pure strategy, because player 2 would want to respond with a pure strategy, but we already know that there are no PSNEs.


Player 1\2 Pass (q)

Meet (1-q)


R,R (p1)R,S (p2)S,R (p3)

1,-10,01,-1

0,01/2,-1/2-1/2,1/2

q½(1-q)q-1/2(1-q)

• Could player 1 be mixing on all 3 remaining strategies? We require: q=1/2(1-q) q=1/2 and ½ =? 1/2-1/2(1-1/2)

½ 1/4. So this cannot be a NE.•So player 1 must be mixing on 2 strategies. Given the conditional expected payoffs, E[R,R] can never equal E[S,R]. E[R,S] = R[S,R] if q = 1/2 , but E[R,R] = ½ > E[R,S] = E[S,R]. So (S,R) is never played with positive probability. Player 1 must be mixing on (R,R) and (R,S).


Player 1\2 Pass (q)

Meet (1-q)


R,R (p)R,S (1-p)

1,-10,0

0,01/2,-1/2

q½(1-q)

• E[R,R] = E[R,S] q=1/2(1-q) q=1/3.• E[Pass] = E[Meet] -p = -1/2(1-p) ½=3/2p p=1/3.

• So the unique NE of this game is

{(1/3,2/3,0,0),(1/3,2/3)}

Player 1 “bluffs” with probability 1/3.

Why PBE Still Isn’t Enough…

I

IIII

L

XX

M

YY

2,-1 0,-1-1,-2-4,-2

R

1,1[p] [1-p]


I

IIII

L

XX

M

YY

2,-1 0,-1-1,-2-4,-2

R

1,1[p] [1-p]

I\II X Y

L 2,-1 -4,-2

M -1,-2 0,-1

R 1,1 1,1


I

IIII

L

XX

M

YY

2,-1 0,-1-1,-2-4,-2

R

1,1[p] [1-p]

I\II X Y

L 2,-1 -4,-2

M -1,-2 0,-1

R 1,1 1,1

2 NE = SPNE @

(L,X) and (R,Y)


I

IIII

L

XX

M

YY

2,-1 0,-1-1,-2-4,-2

R

1,1[p] [1-p]

2 NE = SPNE @ (L,X) and (R,Y). PBE?

For (L,X), II’s information set is on the equilibrium path, so {(L,X),(p,1-p)|p=1} is a PBE.

For (R,Y), II’s information set is off the equilibrium path, so we require E[Y]>=E[X] or

-2p-(1-p) >= -p-2(1-p) or p<= 0.5. So {(R,Y),(p,1-p)|p<=0.5} is a PBE.

Problems?


I

IIII

L

XX

M

YY

2,-1 0,-1-1,-2-4,-2

R

1,1[p] [1-p]

{(L,X),(p,1-p)|p=1} is a PBE.

{(R,Y),(p,1-p)|p<=0.5} is a PBE.

R strictly dominates M for player 1! So if player 2 gets a chance to move, by forward induction, he should expect that player 1 has played L, not M. So the second equilibrium, while still a PBE, could be eliminated with further equilibrium refinements.

extensive games with imperfect information. extensive games with imperfect info in this section, we...

Documents

y y y y y y

y player

y slide

imperfect information

imperfect information

mixed strategy i of

imperfect info slide

pure strategy of player