selective search in games of different complexity maarten schadd

Selective Search in Games of Different Complexity

Maarten Schadd

Playing Chess

Computer vs. Human

• Intuition• Feelings

• Only few variations• Aggressive Pruning

• Selective Search

• Calculator• Fast

• Examines most variations

• Safe Pruning

• Brute-Force Search

Problem Statement

How can we improve selective-search methods in such a way that programs increase their performance in

domains of different complexity ?

Domains of Different Complexity

One-Player GameNo Chance

Perfect Information

Two-Player GameNo Chance

Perfect Information

Two-Player GameChance or

Imperfect Information

Multi-Player GameNo Chance

Perfect Information

One-Player GameNo Chance – Perfect Information

Research question 1 How can we adapt Monte-Carlo Tree Search

for a one-player game?

One player games

• No opponent!

• No uncertainty!

• Why not use all time at the beginning?

• Deviation on the score of moves

SameGame

Single-Player Monte Carlo Tree Search

• Selection Strategy–

• Expansion Strategy– Same

• Simulation Strategy– TabuColourRandom Policy

• Back-Propagation Strategy– Average Score, Sum of Squared Results and

Best Result achieved so far

Experiments – Simulation Strategy

• 250 random positions

• 10 million nodes in memory

One search or several?

Parameter tuning

Highscores

• DBS 72,816• SP-MCTS(1) 73,998• SP-MCTS(2) 76,352• MC-RWS 76,764• Nested MC 77,934• SP-MCTS(3) 78,012• Spurious AI 84,414• HGSTS 84,718

Position 1 – Move 0

Position 1 – Move 10

Position 1 – Move 20

Position 1 – Move 30

Position 1 – Move 40

Position 1 – Move 52

Position 1 – Move 53

Position 1 – Move 63

Two-Player GameNo Chance – Perfect Information

Research Question 2 How can we solve a two-player game by

using Proof-Number Search in combination with endgame database?

Two-Player GameNo Chance – Perfect Information

• Proof-Number Search • Endgame Databases

Fanorona

Average Branchin Factor

Average Number of Pieces

Endgame Database Statistics

Two-Player GameNo Chance – Perfect Information

• 130,820,097,938 nodes

• Fanorona solved – Draw!

Two-Player GameChance or Imperfect Information

Research Question 3How can we perform forward pruning at chance nodes in the expectimax framework?

0.9 0.1

Two-Player GameChance or Imperfect Information

• ChanceProbCut• Predictions based on

shallow search

ChanceProbCut

Stratego

Predicting Stratego

Node Reduction

Performance gain

Multi-Player GameNo Chance – Perfect Information

Research Question 4How can we improve search for multi-player games?

What games do you play?

Coalitions

Multi-Player GameNo Chance – Perfect Information

MaxN

Multi-Player GameNo Chance – Perfect Information

Paranoid

Max^n

1

2 2

3 3 3 3

4 4 4 4 4 4 4 4

6,2,6,3 5,5,1,2 4,1,6,81,1,3,1 7,2,9,5 4,5,6,7 1,5,0,8 5,2,1,4

6,2,6,3

4,1,6,8

7,2,9,5

6,2,6,3

5,2,1,4

7,2,9,5

7,2,9,5

Paranoid

1

7,2,9,5

7 -9

Paranoid

1

2 2

3 3 3 3

4 4 4 4 4 4 4 4

-5 -3 -110 -9 -14 -12 -2

-5

-11

-14

-11 <= -14

-11

Multi-Player GameNo Chance – Perfect Information

Best-Reply Search

Best-Reply Search

• Only 1 opponent plays

• Chose opponent– Strongest counter move

• Other opponents have to pass

• Long term planning

• Less paranoid

• Pruning possible

Best-Reply Search

1

2,3,4

-5

-11 <= -14

-11

2,3,4

1 1 1 1 1 1 1 1 1 1 1

-4 -11 -6 2 -7 -14 -3 3 -4 -7 -1

2 2 3 3 4 4 2 2 3 3 4 4

1

Chinese Checkers

Focus

Rolit

Experiments

3 Players: 6 setups

4 Players: 14 setups

6 Players: 62 setups

Validation

Average Depth

Average Depth

Average Depth

BRS vs. Max^n

BRS vs. Max^n

BRS vs. Max^n

BRS vs. Paranoid

BRS vs. Paranoid

BRS vs. Paranoid

BRS vs. Max^n vs. Paranoid

BRS vs. Max^n vs. Paranoid

BRS vs. Max^n vs. Paranoid

Multi-Player GameNo Chance – Perfect Information

• New Search Algorithm: Best-Reply Search

• Ignoring Opponents

• Long-Term Planning

• Illegal Positions don’t disturb

• Generally Stronger than Max^n and Paranoid

Conclusions

• We have investigated four ways to improve selective search methods– Single-Player Monte-Carlo Tree Search– Proof-Number search + endgame databases– ChanceProbCut– Best-Reply Search

Future Research

• Testing selective search methods in other domains– Other games in the same complexity level– Games of other complexity levels

Thank you for your attention!