selective search in games of different complexity maarten schadd
TRANSCRIPT
Selective Search in Games of Different Complexity
Maarten Schadd
Playing Chess
Computer vs. Human
• Intuition• Feelings
• Only few variations• Aggressive Pruning
• Selective Search
• Calculator• Fast
• Examines most variations
• Safe Pruning
• Brute-Force Search
Problem Statement
How can we improve selective-search methods in such a way that programs increase their performance in
domains of different complexity ?
Domains of Different Complexity
One-Player GameNo Chance
Perfect Information
Two-Player GameNo Chance
Perfect Information
Two-Player GameChance or
Imperfect Information
Multi-Player GameNo Chance
Perfect Information
One-Player GameNo Chance – Perfect Information
Research question 1 How can we adapt Monte-Carlo Tree Search
for a one-player game?
One player games
• No opponent!
• No uncertainty!
• Why not use all time at the beginning?
• Deviation on the score of moves
SameGame
Single-Player Monte Carlo Tree Search
• Selection Strategy–
• Expansion Strategy– Same
• Simulation Strategy– TabuColourRandom Policy
• Back-Propagation Strategy– Average Score, Sum of Squared Results and
Best Result achieved so far
Experiments – Simulation Strategy
• 250 random positions
• 10 million nodes in memory
One search or several?
Parameter tuning
Highscores
• DBS 72,816• SP-MCTS(1) 73,998• SP-MCTS(2) 76,352• MC-RWS 76,764• Nested MC 77,934• SP-MCTS(3) 78,012• Spurious AI 84,414• HGSTS 84,718
Position 1 – Move 0
Position 1 – Move 10
Position 1 – Move 20
Position 1 – Move 30
Position 1 – Move 40
Position 1 – Move 52
Position 1 – Move 53
Position 1 – Move 63
Two-Player GameNo Chance – Perfect Information
Research Question 2 How can we solve a two-player game by
using Proof-Number Search in combination with endgame database?
Two-Player GameNo Chance – Perfect Information
• Proof-Number Search • Endgame Databases
Fanorona
Average Branchin Factor
Average Number of Pieces
Endgame Database Statistics
Two-Player GameNo Chance – Perfect Information
• 130,820,097,938 nodes
• Fanorona solved – Draw!
Two-Player GameChance or Imperfect Information
Research Question 3How can we perform forward pruning at chance nodes in the expectimax framework?
0.9 0.1
Two-Player GameChance or Imperfect Information
• ChanceProbCut• Predictions based on
shallow search
ChanceProbCut
Stratego
Predicting Stratego
Node Reduction
Performance gain
Multi-Player GameNo Chance – Perfect Information
Research Question 4How can we improve search for multi-player games?
What games do you play?
Coalitions
Multi-Player GameNo Chance – Perfect Information
MaxN
Multi-Player GameNo Chance – Perfect Information
Paranoid
Max^n
1
2 2
3 3 3 3
4 4 4 4 4 4 4 4
6,2,6,3 5,5,1,2 4,1,6,81,1,3,1 7,2,9,5 4,5,6,7 1,5,0,8 5,2,1,4
6,2,6,3
4,1,6,8
7,2,9,5
6,2,6,3
5,2,1,4
7,2,9,5
7,2,9,5
Paranoid
1
7,2,9,5
7 -9
Paranoid
1
2 2
3 3 3 3
4 4 4 4 4 4 4 4
-5 -3 -110 -9 -14 -12 -2
-5
-11
-14
-11 <= -14
-11
Multi-Player GameNo Chance – Perfect Information
Best-Reply Search
Best-Reply Search
• Only 1 opponent plays
• Chose opponent– Strongest counter move
• Other opponents have to pass
• Long term planning
• Less paranoid
• Pruning possible
Best-Reply Search
1
2,3,4
-5
-11 <= -14
-11
2,3,4
1 1 1 1 1 1 1 1 1 1 1
-4 -11 -6 2 -7 -14 -3 3 -4 -7 -1
2 2 3 3 4 4 2 2 3 3 4 4
1
Chinese Checkers
Focus
Rolit
Experiments
3 Players: 6 setups
4 Players: 14 setups
6 Players: 62 setups
Validation
Average Depth
Average Depth
Average Depth
BRS vs. Max^n
BRS vs. Max^n
BRS vs. Max^n
BRS vs. Paranoid
BRS vs. Paranoid
BRS vs. Paranoid
BRS vs. Max^n vs. Paranoid
BRS vs. Max^n vs. Paranoid
BRS vs. Max^n vs. Paranoid
Multi-Player GameNo Chance – Perfect Information
• New Search Algorithm: Best-Reply Search
• Ignoring Opponents
• Long-Term Planning
• Illegal Positions don’t disturb
• Generally Stronger than Max^n and Paranoid
Conclusions
• We have investigated four ways to improve selective search methods– Single-Player Monte-Carlo Tree Search– Proof-Number search + endgame databases– ChanceProbCut– Best-Reply Search
Future Research
• Testing selective search methods in other domains– Other games in the same complexity level– Games of other complexity levels
Thank you for your attention!