04- adversarial search ai

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E

CSE 481ARTIFICIAL INTELLIGENCE

ADVERSARIAL SEARCH AND GAMES

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E22

Conventional Search

In conventional search

There is no adversary to consider

A solution is a method for finding goal

Heuristic techniques

Could find an optimal solution but not always

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E33

Games: Adversarial Search

• Games are adversarial

– A solution involves strategy that specifies a move for every

possible opponent reply

– We can find an exact solution

• Using search with alternating player moves and assuming an

opponent playing perfectly

• The search explores the game tree to the conclusion and selects

the move with the guaranteed best outcome

– Time limits may force an approximate solution

• A heuristic evaluation function

– Replaces the utility function

• It evaluates the goodness of a game position

– Examples: chess, checkers … etc.

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E44

The Game Prototype

• Formulating games as search problems

1. Initial State

• The board position + the player to move

2. Successor Function

• A list of (move, state) pairs specifying legal moves

3. Terminal Test

• Detects a terminal state (i.e., detects if the game is finished)

4. Utility Function

• Returns a numerical value for terminal states

• Possibly as simple as 1 for win, 0 for lose, and -1 for draw

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E55

Game Search & Game Trees

From the initial state and successor function

We generate the game tree

Each turn is 1 ply (a layer of nodes representing states)

We use the search tree to determine the next move:

1. Search to terminal states

2. Use their utility function values + assumptions about opponent

moves

3. Pick the optimal next move

Example: a complete tree for an example game called

NIM (http://illuminations.nctm.org/ActivityDetail.aspx?ID=140)

http://illuminations.nctm.org/ActivityDetail.aspx?ID=140










CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E66

A very small game: NIM Game Rules

Consider a game called NIM

It is played by two players

In this game a number of stones are placed in a pile

In each player’s turn, player can remove either 1 or 2

stones from the pile

The player who removes the last stone wins

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E77

A very small game: NIM Game Representation

Consider the game tree for

the 5 stone version of NIM.

[VAL] represents the first

player (We) node or MAX

node

(VAL) represents the

Opponent node or MIN

node

VAL is the current number

of stones in the pile

We start the game

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E88

A very small game: NIM Node Evaluation

If a node corresponds to MAXmove and the value is [0], thenthat node indicates a Lose forMAX and a utility value of 0 isassigned to this node

Similarly a node with (0) is aWin for MAX and a utility valueof 1 is assigned to this node

To propagate these values backup the list from bottom up asfollow:

if we are evaluating a MAX node, findthe maximum of the utility function ofall branches going out from this node

if we are evaluating a MIN node findthe minimum of the utility function ofall branches going out from this node

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E99

Optimal Strategies: minimax

minimax values for a simple 2-ply example

here's the game tree

with terminal values from a utility function

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1010



now we back up the values for min nodes

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1111



then we back up values for max node

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1212


The optimal play assumption

Optimal play for MAX

Assumes that MIN also plays optimally

i.e. maximizes the worst-case outcome for MAX

What if the assumption doesn't hold

What if MIN does not play optimally?

In this case, MAX will do even better

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1313

Extending minimax

• Pruning the game search tree

– The number of game states is

• Exponential in the number of moves

– Thus, for real non-trivial games

• There are just too many nodes

– So, how to avoid examining every node in the tree?

• We can still compute the correct minimax decision without

expanding all the nodes

– We can prune the game tree by not expanding portions that

wouldn't make any difference (- pruning algorithm)

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1414

α-β pruning example

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1515


CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1616


CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1717


CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1818


CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E1919 - Pruning Algorithm

Idea: if current path is already worse than some

other known path, stop expanding it.

Improves the speed of the mini-max search

algorithm

is the max lower bound of all solutions. i.e. the value of

the best move so far at any choice point on MAX path

is the min upper bound of all solutions. i.e. value for the

best move so far at any choice point on MIN path

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E2020 - Pruning Algorithm Example

1) Setup phase: Assign to each left-most internal node of the tree,

variables: = -, = +

MAX

MIN

MAX

MIN

MAX

=-

=+

=-

=+

=-

=+

=-

=+

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


2) Look at first computed final configuration value. It’s 3. Parent is a MIN node, so

set the (min) value to 3

MAX

MIN

MAX

MIN

MAX3

=-

=+

=-

=+

=-

=+

=-

= 3

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


3) Look at next value, 5. Since parent is a MIN node, we want the minimum of

3 and 5 which is 3. Parent MIN node is done – fill (max) value of its parent MAX

node. Always set for MAX nodes and for MIN nodes. Copy the state of the MAX parent

node into the second unevaluated MIN child

3

3 5

=-

=+

=-

=+

= 3

=+

=-

= 3

= 3

=+

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


4) Look at next value, 2. Since parent node is MIN with =+, 2 is smaller, change

3

3 5 2

=-

=+

=-

=+

= 3

=+

=-

= 3

= 3

= 2

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


5) Now, the MIN parent node has a max value of 3 and MIN value of 2. The value of the 2nd child

does not matter. if it is >2, 2 will be selected for MIN node. If it is <2, it will be selected for MIN node,

but since it is <3 it will not get selected for the parent max node. Thus, we prune the right sub-tree of

the MIN node. Propagate MAX value up the tree

3

3 2

3 5 2

=-

=+

=-

=+

= 3

=+

=-

= 3

= 3

= 2

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


6) MAX node is now done and we can set the value of its parent and propagate node

state to sibling sub-tree’s left-most path

3

3 2

3 5 2

=-

=+

=-

= 3

= 3

=+

=-

= 3

= 3

= 2

=-

= 3

=-

= 3

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


7) The next node is 10. 10 is not smaller than 3, so state of parent does not change. We

still have to look at the 2nd child since is still –

3

3 2

3 5 2 10

=-

=+

=-

= 3

= 3

=+

=-

= 3

= 3

= 2

=-

= 3

=-

= 3

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


8) The next node is 4. Smallest value goes to the parent MIN node. MIN sub-tree is

done, so the parent MAX node gets the (max) value from the child. Note that if the

MAX node had a 2nd sub-tree, we can prune it since >

3 4

3 2 4

3 5 2 10 4

=-

=+

=-

= 3

= 3

=+

=-

= 3

= 3

= 2

= 4

= 3

=-

= 3

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


9) Continue propagating value up the tree, modifying the corresponding and values.

Also, propagate the state of root node down the left-most path of the right sub-tree

3 4

3

3 2 4

3 5 2 10 4

= 3

=+

=-

= 3

= 3

=+

=-

= 3

= 3

= 2

= 4

= 3

=-

= 3

= 3

=+

= 3

=+

= 3

=+

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


10) Next value is a 2. We set the (min) value of the MIN parent to 2. Since no other

children exist, we propagate the value up the tree

3 4 2

3

3 2 4 2

3 5 2 10 4 2

= 3

=+

=-

= 3

= 3

=+

=-

= 3

= 3

= 2

= 4

= 3

=-

= 3

= 3

=+

= 3

=+

= 3

= 2

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


11) We have a value for the 3rd level MAX node, now we can modify the (min) value of

the MIN parent to 2. Now, we have a situation that > and thus the value of the

rightmost sub-tree of the MIN node does not matter, so we prune the whole sub-tree

3 4 2

3

3 2 4 2

3 5 2 10 4 2

= 3

=+

=-

= 3

= 3

=+

=-

= 3

= 3

= 2

= 4

= 3

=-

= 3

= 3

= 2

= 3

=+

= 3

= 2

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC


12) Finally, no more nodes remain, we propagate values up the tree. The root has a value of 3 that

comes from the left-most child. Thus, the player should choose the left-most child’s move in order to

maximize his/her winnings. The result is the same as with the original minimax algorithm, but we did not

visit all nodes of the tree

3

3 4 2

3 2

3 2 4 2

3 5 2 10 4 2

= 3

=+

=-

= 3

= 3

=+

=-

= 3

= 3

= 2

= 4

= 3

=-

= 3

= 3

= 2

= 3

=+

= 3

= 2

MAX

MIN

MAX

MIN

MAX

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E3232

Comments on - Pruning Algorithm

Pruning does not change the eventual results though

entire sub-trees can be pruned

Its effectiveness depends partly on

– The order of examining successors

– Perfect ordering would be best successors first which allows

earliest pruning

• With "perfect ordering" time complexity is O(bm/2)

The branching factor reduces to sqrt(b)

From an alternative view, given - pruning and a fixed

amount of time we can look twice as far ahead as with simple

minimax

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E3333

Dealing with games with large game trees

What to do when the game tree is too large

Complete search is not feasible

We can't guarantee optimal play

Replace the utility function with a heuristic evaluation

function that assigns an estimate to intermediate nodes

without going all the way down to the terminal nodes

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E3434

Non-Optimal Game Trees

Heuristic evaluation function

Evaluation function should

1. order terminal states as the utility function would

2. be economical to compute

3. when applied to non-terminals

– Be highly correlated with the actual chance of winning

This introduces uncertainty about the outcome

This uncertainty is due to computational limitations

The game is still characterized by complete information

CS

E 4

81

: A

RT

IF

IC

IA

L I

NT

EL

LI

GE

NC

E3535

Non-Optimal Game Trees

Cutoff depth and heuristic evaluation function

The simplest cutoff is a fixed depth limit

Chosen to allow a move choice within the time limit

A more robust approach

Uses iterative deepening

Selects its move from deepest search completed in time

04- adversarial search ai

Documents