University College Cork (Ireland) Department of Civil and Environmental Engineering
Course:Engineering Artificial Intelligence
Dr. Radu Marinescu
Lecture 5
University College Cork (Ireland) Department of Civil and Environmental Engineering
Today’s class
o Game-playing
R. Marinescu October-2009 page 2
University College Cork (Ireland) Department of Civil and Environmental Engineering
Overview
o Computer programs which play 2-player games• game-playing as search• with the complication of an opponent
o General principles of game-playing and search• evaluation functions• mini-max principle• alpha-beta-pruning• heuristic techniques
o Status of Game-Playing Systems• in chess, checkers, backgammon, Othello, etc, computers routinely defeat
leading world playerso Applications?
• think of “nature” as an opponent• economics, war-gaming, medical drug treatment
R. Marinescu October-2009 page 3
University College Cork (Ireland) Department of Civil and Environmental Engineering
History
R. Marinescu October-2009 page 4
1949 – Shannon paper1951 – Turing paper1958 – Bernstein program55-60 – Simon-Newell program (α – β McCarthy?)1961 – Soviet program66-67 – MacHack 6 (MIT AI Lab)70’s – NW Chess 4.580’s – Crazy Blitz90’s – Belle, Hitech, Deep Thought, Deep Blue
University College Cork (Ireland) Department of Civil and Environmental Engineering
Solving 2-player games
o Two players, perfect informationo Examples:
• e.g., chess, checkers, tic-tac-toeo Configuration of the board = unique
arrangement of “pieces”
o Games as a Search Problem• States = board configurations• Operators = legal moves• Initial State = current configuration• Goal = winning configuration• payoff function = gives numerical value of outcome of the game
R. Marinescu October-2009 page 5
University College Cork (Ireland) Department of Civil and Environmental Engineering
Game tree search
o Game tree: encodes all possible games
o We are not looking for a path, only the next move to make (that hopefully leads to a winning position)
o Our best move depends on what the other player does
R. Marinescu October-2009 page 6
University College Cork (Ireland) Department of Civil and Environmental Engineering
Move generation
R. Marinescu October-2009 page 7
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example: partial game tree for Tic-Tac-Toe
R. Marinescu October-2009 page 8
MAX (O)
MIN (X)
MAX (O)
MIN (X)
University College Cork (Ireland) Department of Civil and Environmental Engineering
Scoring function
R. Marinescu October-2009 page 9
We will use the same scoring function for both players, simply negating the values to represent the opponent's scores.
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example: scoring function for Tic-Tac-Toe
R. Marinescu October-2009 page 10
Leaves are either win (+1), loss (-1) or draw (0)
University College Cork (Ireland) Department of Civil and Environmental Engineering
Min-Max: an optimal procedure
o Designed to find the optimal strategy for MAX player and find best move:
• 1. Generate the whole game tree to leaves• 2. Apply scoring (payoff) function to leaves• 3. Back-up values from leaves toward the root:
a MAX node computes the max of its child values a MIN node computes the min of its child values
• 4. When value reaches the root: choose max value and the corresponding move.
R. Marinescu October-2009 page 11
University College Cork (Ireland) Department of Civil and Environmental Engineering
Min-Max
R. Marinescu October-2009 page 12
University College Cork (Ireland) Department of Civil and Environmental Engineering
Properties of Min-Max
o Complete? Yes (if tree is finite)o Optimal? Yes (against an optimal opponent)o Time complexity? O(bd)o Space complexity? O(bd) (depth-first exploration)
o For chess, b ≈ 35, m ≈100 for "reasonable" games exact solution completely infeasible
• Chess: b ~ 35 (average branching factor) d ~ 100 (depth of game tree for typical game) bd ~ 35100 ~10154 nodes!!
• Tic-Tac-Toe ~5 legal moves, total of 9 moves 59 = 1,953,125 9! = 362,880 (Computer goes first) 8! = 40,320 (Computer goes second)
R. Marinescu October-2009 page 13
University College Cork (Ireland) Department of Civil and Environmental Engineering
Min-Max
o Naïve Min-Max assumes that it is possible to search the full tree, where the scoring function of the leaves is either win (+1), loss (-1) or draw (0)
o However: For most games, it is impossible to develop the whole search tree
o Instead develop part of the tree (up to a certain depth or a number of plys) and evaluate promise of leaves using a static evaluation function.
R. Marinescu October-2009 page 14
University College Cork (Ireland) Department of Civil and Environmental Engineering
Static evaluation function
o The static evaluation function:• Estimates how good the current board configuration is for a player.• Typically, one figures how good it is for the player, and how good it
is for the opponent, and subtracts the opponents score from the players
• Othello: Number of white pieces - Number of black pieces• Chess: Value of all white pieces - Value of all black pieces
o Typical values from -infinity (loss) to +infinity (win) or [-1, +1].
o If the board evaluation is X for a player, it’s -X for the opponent
o Example: • Evaluating chess boards, Checkers, Tic-tac-toe
R. Marinescu October-2009 page 15
University College Cork (Ireland) Department of Civil and Environmental Engineering
Evaluation functions: chess
R. Marinescu October-2009 page 16
For Chess, typically a linear weighted sum of features
n
iiinn sfwsfwsfwsfwsEval
12211 )()(...)()()(
e.g.,)queensblack ofnumber ()queens whiteofnumber ()(
9
1
1
sf
w weight of the piece (Q = 9, P=1, K=3, B=3.5, R=5)
University College Cork (Ireland) Department of Civil and Environmental Engineering
Evaluation functions: Tic-Tac-Toe
R. Marinescu October-2009 page 17
nnE
nO
nM
nOnMnE
statefor evaluation total theis )(
(O) lines winningpossible sOpponent' of total theis )(
(X) lines winningpossibleMy of total theis )( where
)()()(
Another example
E(n) =
University College Cork (Ireland) Department of Civil and Environmental Engineering
Backup values
R. Marinescu October-2009 page 18
Two-ply minimax applied to the opening move of tic-tac-toe
University College Cork (Ireland) Department of Civil and Environmental Engineering
Backup values
R. Marinescu October-2009 page 19
Two-ply minimax applied to the X’s second move
University College Cork (Ireland) Department of Civil and Environmental Engineering
Backup values
R. Marinescu October-2009 page 20
Two-ply minimax applied to X’s move near end game
University College Cork (Ireland) Department of Civil and Environmental Engineering
Impact of tree depth
R. Marinescu October-2009 page 21
University College Cork (Ireland) Department of Civil and Environmental Engineering
Deep Blue
o 32 SP2 processors• Each with 8 dedicated chess processors = 256 CP
o 50 – 100 billion moves in 3 min• 13-30 ply search
R. Marinescu October-2009 page 22
University College Cork (Ireland) Department of Civil and Environmental Engineering
α – β pruning
o In Min-Max there is a separation between node generation and evaluation.
R. Marinescu October-2009 page 23
Backup Values
University College Cork (Ireland) Department of Civil and Environmental Engineering
α – β pruning
o Idea: • Do depth-first search to generate partial game tree• Give static evaluation function to leaves• Compute bound on internal nodes
o Alpha (α), Beta (β) bounds:• α value for MAX node means that max real value is at least alpha• β for MIN node means that MIN can guarantee a value below beta
o Computation:• Alpha of a MAX node is the maximum value of its seen children• Beta of a MIN node is the lowest value seen of its child node
R. Marinescu October-2009 page 24
University College Cork (Ireland) Department of Civil and Environmental Engineering
When to prune
o Pruning
• Below a MIN node whose beta value is lower than or equal to the alpha value of its ancestors.
• Below a MAX node having an alpha value greater than or equal to the beta value of any of its MIN nodes ancestors.
R. Marinescu October-2009 page 25
University College Cork (Ireland) Department of Civil and Environmental Engineering
α – β pruning
R. Marinescu October-2009 page 26
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 27
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 28
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 29
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 30
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 31
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 32
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 33
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 34
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 35
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 36
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 37
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 38
University College Cork (Ireland) Department of Civil and Environmental Engineering
Example
R. Marinescu October-2009 page 39
University College Cork (Ireland) Department of Civil and Environmental Engineering
α – β properties
o Guaranteed same value as Min-Max
o In a perfectly ordered tree, expected work is O(bd/2), vs O(bd) for Min-Max, so can search twice as deep with the same effort
o With good move orderings, the actual running time is close to the optimistic estimate
R. Marinescu October-2009 page 40
University College Cork (Ireland) Department of Civil and Environmental Engineering
Game program
o Move generator (ordered moves) 50%o Static evaluation 40%o Search control 10%
R. Marinescu October-2009 page 41
OpeningsEnd games
databases
[all in place by late 60’s]
University College Cork (Ireland) Department of Civil and Environmental Engineering
Move generator
o Legal moveso Ordered by
• Most valuable victim• Least valuable aggressor
o Killer heuristics
R. Marinescu October-2009 page 42
University College Cork (Ireland) Department of Civil and Environmental Engineering
Static evaluation
o Initially - very complex
o 70’s - very simple (material)
o Now - Deep searchers: moderately complex
(hardware)
PC programs: elaborate, hand tuned
R. Marinescu October-2009 page 43
University College Cork (Ireland) Department of Civil and Environmental Engineering
Other games
o Backgammon• Involves randomness – dice rolls• Machine-learning based player was able to draw the world
champion human playero Bridge
• Involves hidden information – other players’ cards – and communication during bidding
• Computer players play well but do not bid wello Go
• No new elements but huge branching factor• No good computer players exist
R. Marinescu October-2009 page 44
University College Cork (Ireland) Department of Civil and Environmental Engineering
Observations
o Computers excel in well-defined activities where rules are clear
• Chess• Mathematics
o Success comes after a long period of gradual refinement
R. Marinescu October-2009 page 45
University College Cork (Ireland) Department of Civil and Environmental Engineering
Summary
o Game playing is best modeled as a search problem
o Game trees represent alternate computer/opponent moves
o Evaluation functions estimate the quality of a given board configuration for the MAX player.
o Min-Max is a procedure which chooses moves by assuming that the opponent will always choose the move which is best for them
o Alpha-Beta is a procedure which can prune large parts of the search tree and allow search to go deeper
o For many well-known games, computer algorithms based on heuristic search match or out-perform human world experts
o Reading: R&N Chapter 6
R. Marinescu October-2009 page 46