evolutionary algorithms vs. poker games

EVOLUTIONARY ALGORITHMSVS.POKER GAMES

Yikan Chen (yc2r@virginia.edu)Weikeng Qin (wq7yt@virginia.edu)

1

OUTLINE

2

Evolutionary Algorithm

Poker!

Artificial Neural

Network

E-ANN

EVOLUTIONARY ALGORITHM

3

EVOLUTIONARY ALGORITHM Evolution Process

4

Crossover

Mutation

Natural Selection

Evolutionary Algorithm

EVOLUTIONARY ALGORITHM Encoding and Crossover

5

1 1 1 0 0 1 1 0

0 1 0 0 1 0 1 1

0 0 1 1 0

0 1 0 1 1

0 1 0

1 1 1

EVOLUTIONARY ALGORITHM Mutation

6

1 1 1 0 0 1 1 0

1 1 0 0 0 1 1 1

EVOLUTIONARY ALGORITHM Natural Selection

7

Run the roulette-wheel selection based on the fitness value of candidates

EVOLUTIONARY ALGORITHM Important Parameters

Crossover rate Mutation rate Elite rate Fitness function

Demohttp://userweb.elec.gla.ac.uk/y/yunli/

ga_demo/

8

EVOLUTIONARY ALGORITHM & POKER AKQ 2-player game

$1 blinds for each player Player1 bet or fold Player2 call or fold

9

EVOLUTIONARY ALGORITHM & POKER Derive the optimal strategy using EA Chromosomal representations

Fij: fold threshold when Pi got Cardj

Fitness functions

10

Card1

Card2

Card3

P1 2/3 0 0P2 1 2/3 0

EVOLUTIONARY ALGORITHM & POKER Fitness functions

Fi: fitness function Wij: money won by candidate I against

candidate j

11

12

13

EVOLUTIONARY ALGORITHM & POKER

14

Decreased fluctuation

Further decreased fluctuation

400-500 generations

Var(f11) ; Var(f22)

Mean(f11);Mean(f22)

Count only wins

.065;

.067.67;.60

Penalize failure

.037;

.035.67;.70

Penalize Failure heavier

.028;

.024.67;.74

EVOLUTIONARY ALGORITHM & POKER Real Texas Hold’em Encoding Strategy (Turn and River)

Hand strength (player confidence) Fraction of opponent raise (opponent

confidence) Total raise (profit)

15

EVOLUTIONARY ALGORITHM & POKER Fitness Criterion

16

EVOLUTIONARY ALGORITHM & POKER Performance

17

ARTIFICIAL NEURAL NETWORK: REVIEW

18

ARTIFICIAL NEURAL NETWORK: REVIEW

19

∑

w1

w2

wn

b

……

a1

a2

an

1

f output

ARTIFICIAL NEURAL NETWORK: REVIEW

20

Inputoutput

Hidden Layer

EvolvingTopology

E-ANN (EVOLUTIONARY ANN) Simplest Encoding Method

21

a b c d d c b a

NEAT E-ANN http://www.cs.utexas.edu/users/nn/ Neuro Evolution of Augmenting

Topologies Encoding Strategy: Node-based

Neuron gene table Link gene table

Innovation number Global database of innovations Each innovation has unique ID number

22

NEAT E-ANN

23

NEAT E-ANN Mutation

Perturb weights Add a link gene Add a neuron gene

Crossover By innovation number

24

NEAT E-ANN Crossover

25

2 3

56

4

31 2

5

4

1

11->4

22->4

33->4

42->5

55->4

81->5

11->4

22->4

33->4

42->5

55->4

65->6

76->4

93>5

101->6

NEAT E-ANN Crossover

26

2 3

5

6

4

1

81->5

11->4

22->4

33->4

42->5

55->4

65->6

76->4

93>5

101->6

E-ANN & POKER Simplified Poker Model

1-10 Initial credit: 10 chips One chip ante at the beginning Call, raise (1 chip each time), fold Tournament

27

E-ANN & POKER

28

Two player game

E-ANN & POKER

29

E-ANN & POKER Four different types of

opponents

30

Tight Aggressive (TA) Tight Passive (TP)Loose Aggressive (LP) Loose Passive (LP)

E-ANN & POKER α: min win probability to call β: min win probability to raise

31

E-ANN & POKER

32

A: player typeB: player action

E-ANN & POKER

33

E-ANN & POKER Bluffing……

34

35

Thanks!