Naoki  Masuda  (University  of  Tokyo)

Ref:  Masuda.  Proc.  R.  Soc.  B,  274,  1815-­‐1821  (2007).

Also  see  Masuda  &  Aihara,  Phys.  LeK.  A,  313,  55-­‐61  (2003).

ParNcipaNon  costs  dismiss  the  advantage  of  heterogeneous  

networks  in  evoluNon  of  cooperaNon

Prisoner’s  DilemmaCooperate Defect

Cooperate (3,  3) (0,  5)

Defect (5,  0) (1,  1)

SelfOpponent

unique  Nash  equilibrium

CC

C C

CD

DD D

D

DC

D C

DD

DD D

D

DD

D D

DD

DD D

D

Mechanisms  for  cooperaNon• Kin  selecNon  (Hamilton,  1964)

• Direct  reciprocity  (Trivers,  1971;  Axelrod  &  Hamilton  1981)

• Iterated  Prisoner’s  dilemma

• Group  selecNon  (Wilson,  1975;  Traulsen  &  Nowak,  2006)

• SpaNal  reciprocity  (Axelrod,  1984;  Nowak  &  May,  1992)

• Indirect  reciprocity  (Nowak  &  Sigmund,  1998)

• Image  scoring

• Network  reciprocity  (Lieberman,  Hauert  &  Nowak,  2005;  Santos  et  al.,  2005,  2006;  Ohtsuki  et  al.,  2006)

• Others  (punishment?,  voluntary  parNcipaNon  etc.)

Iterated  Prisoner’s  Dilemma

• Players  randomly  interact  with  others

• Discount  factor  w  (0  ≤  w  ≤  1)  to  specify  the  prob.  that  the  next  game  is  played  in  a  round

A’s  accumulated  payoff    =  3  +  5w  +  1w2  +  3w3  +  1w4  +  0w5  +  …

A  plays C D D C D C

B  plays C C D C D D

A  gets 3 5 1 3 1 0

acNon C D

C (3,  3) (0,  5)

D (5,  0) (1,  1)

• SelecNon  based  on  accumulated  payoff  aher  each  round

• Replicator  dynamics

• Best-­‐response  dynamics

• Nice,  retalitatory,  and  forgiving  strategies  (e.g.  Tit-­‐for-­‐Tat)  are  generally  strong  (but  not  the  strongest).

SpaNal  Prisoner’s  Dilemma(Axelrod,  1984;  Nowak  &  May,  1992)• e.g.  square  laice

• Either  cooperator  or  defector  on  each  vertex

• Each  player  plays  against  all  (4  or  8)  neighbors.

C:24 C:18 D:24 D:12

C:24 C:21 C:12 D:16

C:24 C:21 C:12 D:16

C:21 C:15 D:20 D:16

• Successful  strategies  propagate  aher  one  generaNon.

• Result:  Cs  form  “clusters”  to  resist  invasion  by  Ds.

• Note:  2-­‐neighbor  CA.  More  complex  than  1-­‐neighbor  dynamics  such  as  spin  (opinion)  dynamics  and  disease  dynamics

PD  on  the  WaKs-­‐Strogatz  small-­‐world  network(Masuda  and  Aihara,  Phys.  LeK.  A,  2003)

Note:  The  degrees  of  all  the  nodes  are  the  same  

regardless  of  the  rewiring.

acNon C D

C (1,  1) (0,  T)

D (T,  0) (0,  0)p  =  0 p:  small p  ≈  1

0

0.2

0.4

0.6

0.8

1

1 1.5 2 2.5

%C

T

p=0p=0.01p=0.9

0

0.2

0.4

0.6

0.8

1

0 100 200 300 400

%C

generation

T=1.1

0

0.2

0.4

0.6

0.8

1

0 100 200 300 400

%C

generation

T=1.70

0.2

0.4

0.6

0.8

1

0 100 200 300 400

%C

generation

T=3.0

•  1-­‐dim  ring

•  QualitaNvely  the  same  results  for  2-­‐dim  networks

small  p

small  p

small  p

large  p large  p

large  p

Clustering  helps  cooperaNonmore  C more  D

Small  distance  (L)  accelerates  whatever  propagaNon

slow fast

Social  dilemma  games  on  scale-­‐free  networks?

Scale-­‐free  networks  promote  cooperaNon

acNon Cooperate Defect

Cooperate (1,  1) (S,  T)

Defect (T,  S) (0,  0)

Originally  by  Santos,  Pacheco  &  Lenaerts,  PNAS  2006T

S

(a)

0 1 2-1

0

1

T

S

(b)

0 1 2-1

0

1

Regular  RG(or  the  complete  graph) scale-­‐free

stag  hunt

snowdrihno  dilemma

PD

Two  assumpNons  underlie  the  enhanced  coopraNon  in  SF  nets

AssumpNon  1:  addiNve  payoff  scheme

• AddiNve:  add  the  payoffs  gained  via  all  the  neighbors  

• Nowak,  Bonhoeffer  &  May  1994;  Abramson  &  Kuperman  2001;  Ebel  &  Bornholdt,  2002;  Ihi,  Killingback  &  Doebeli  2004;  Durán  &  Mulet,  2005;  Santos  et  al.,  2005;  2006;  Ohtsuki  et  al.,  2006

• Average:  divide  the  summed  payoffs  by  the  number  of  neighbors

• Kim  et  al.,  2002;  Holme  et  al.,  2003;  Vukov  &  Szabó,  2005;  Taylor  &  Nowak,  2006

acNon Cooperate Defect

Cooperate (3,  3) (0,  5)

Defect (5,  0) (1,  1)

k = 1001 2v v

10 90

1 k = 22

= C= D

addiNve

payoff  scheme

average

3×10+0×90  =  30

(3×10+0×90)/100  =  0.3

5×2  =  10

(5×2)/2  =  5

• Average  payoff  diminishes  cooperaNon  in  heterogeneous  networks  (Santos  &  Pacheco,  J.  Evol.  Biol.,  2006).

• Originate  from  the  translaNon  invariance  of  replicator  dynamics

AssumpNon  2:  PosiNvely  biased  payoffs

→  Nme  rescaling

→  parNcipaNon  cost

acNon C D

C (1,  1) (0,  T)

D (T,  0) (0,  0)

acNon C D

C (1,  1) (S,  T)

D (T,  S) (0,  0)

✓a bc d

◆!

✓a� h b� hc� h d� h

◆

✓a bc d

◆!

✓ka kbkc kd

◆

✓a bc d

◆!

✓a+ k b+ kc d

◆

C D

C a b

D c d

!!(⇡C = axC + bxD

⇡D = cxC + dxD

h⇡i = ⇡CxC + ⇡DxD(xC = xC (⇡C � h⇡i)xD = xD (⇡D � h⇡i)

Reason  for  enhanced  cooperaNon

• Hubs  earn  more  than  leaves.

• C  on  hubs  (with  at  least  some  C  neighbors)  are  stable.

• C  spreads  from  hubs  to  leaves.

acNon C D

C (1,  1) (0,  T)

D (T,  0) (0,  0)

k = 1001 2v v

10 90

1 k = 22

= C= D

Payoff  matrix  is  not  invariant  on  heterogeneous  networks

acNon C D

C (1,  1) (0,  T)

D (T,  0) (0,  0)

→  Nme  rescaling

→  parNcipaNon  cost

✓a bc d

◆!

✓a� h b� hc� h d� h

◆

✓a bc d

◆!

✓ka kbkc kd

◆

✓a bc d

◆!

✓a+ k b+ kc d

◆

C D

C a b

D c d

!!(⇡C = axC + bxD

⇡D = cxC + dxD

h⇡i = ⇡CxC + ⇡DxD(xC = xC (⇡C � h⇡i)xD = xD (⇡D � h⇡i)

✔

NG

NG

Our  assumpNons• AddiNve  payoff  scheme

• Introduce  the  parNcipaNon  cost

• Do  numerics

• N  =  5000  players

• Each  player  plays  against  all  the  neighbors.

• Replicator-­‐type  update  rule:  player  i  copies  player  j’s  strategy  with  prob

 (πj-­‐πi)/[max(ki,kj)  *  (max  possible  payoff  –  min  possible  payoff)]

T

h

(a)

0.7 1 1.3 1.6

0

1

2

3

T

h

(a)

0.7 1 1.3 1.6

0

1

2

3cf

0

0.25

0.5

0.75

1

Simplified  prisoner’s  dilemma

acNon C D

C (1-h,  1-h) (-h,  T-h)D (T-h,  -h) (-h,  -h)

regular  random  net

scale-­‐free  net 3  (roughly  separated)  regimes

Strong  influence  of  iniNal  cnds  due  to  long  transients

Somewhat  reduced  cooperaNon

Enhanced  cooperaNon  (prev  results)

1

2

3

From  leaves  to  hubs.    PD  payoff  structure  is  most  relevant.

T

h

(a)

0.7 1 1.3 1.6

0

1

2

3 1

2

3

Strategy  spreads  from  stubborn  leaves  to  hubs.

From  hub  cooperators  to  leaves.

-50

0

50

100

0 50 100 150 200

gene

ratio

n pa

yoff

# neighbors

(a)

0

50

10 100

# fli

ps

# neighbors

(b)

T  =  1.5

h  =  0h  =  0.23h  =  0.2

h  =  0.24h  =  0.25

h  =  0.3h  =  0.5

h  =  0h  =  0.2

h  =  0.23

h  =  0.24h  =  0.25

h  =  0.3

h  =  0.5

General  matrix  game• Homogeneous  (in  degree)  →  2  parameters  (S  

and  T)

• e.g.  well-­‐mixed,  square  laice,  regular  random  graph

• Heterogeneous  →  3  parameters  (S,  T,  and  h)

• e.g.  ER  random  graph,  scale-­‐free

• PD,  snowdrih  game,  hawk-­‐dove  game  included

acNon C D

C (1-h,  1-h) (S-h,  T-h)D (T-h,  S-h) (-h,  -h)

T

S

(a)

0 1 2-1

0

1

T

S

(c)

0 1 2-1

0

1

T

S

(e)

0 1 2-1

0

1

T

S

(d)

0 1 2-1

0

1

T

S

(b)

0 1 2-1

0

1

1 2

3

Regular  RG  (h  =  0) SF  (h  =  0) SF  (h  =  0.5)

SF  (h  =  1) SF  (h  =  2)

stag  hunt

snowdrihno  dilemma

PD

cf

0

0.25

0.5

0.75

1

1

2

consistent  with  Santos  et  al.,  PNAS  (2006)

Thoughts  about  the  payoff  bias  

• Naturally  understood  as  the  parNcipaNon  cost

• Payoffs  may  be  negaNve  in  many  pracNcal  situaNons.

• Environmental  problems?

• InternaNonal  relaNons?

• When  one  is  ‘forced’  to  play  games

Conclusions• Games  with  parNcipaNon  costs  on  networks  

• More  C  for  small  parNcipaNon  cost  h  (previous  work).

• Networks  determine  dynamics  for  small  and  large  h.

• Payoff  matrix  is  most  relevant  for  intermediate  h.

• Think  twice  about  the  use  of  simplified  PD  payoff  matrices.