csc411artificial intelligence1 chapter 4 heuristic search contents hill-climbing dynamic programming...

CSC411CSC411 Artificial IntelligenceArtificial Intelligence 11

Chapter 4

HEURISTIC SEARCH

Contents• Hill-climbing• Dynamic programming• Heuristic search algorithm• Admissibility, Monotonicity, and Informedness• Using Heuristics In Games• Complexity Issues


HeuristicsHeuristicsRules for choosing paths in a state Rules for choosing paths in a state space that most likely lead to an space that most likely lead to an acceptable problem solutionacceptable problem solutionPurposePurpose– Reduce the search spaceReduce the search space

ReasonsReasons– May not have exact solutions, need May not have exact solutions, need

approximationsapproximations– Computational cost is too highComputational cost is too high


First three levels of the tic-tac-toe state space First three levels of the tic-tac-toe state space reduced by symmetryreduced by symmetry


The “most wins” heuristic applied to the first children in tic-tac-toe


Heuristically reduced state space for tic-tac-toe


Hill-ClimbingHill-ClimbingAnalogAnalog– Go uphill along the steepest possible path until Go uphill along the steepest possible path until

no farther upno farther up

PrinciplePrinciple– Expand the current state of the search and Expand the current state of the search and

evaluate its childrenevaluate its children– Select the best child, ignore its siblings and Select the best child, ignore its siblings and

parentparent– No history for backtrackingNo history for backtracking

ProblemProblem– Local maxima – not the best solutionLocal maxima – not the best solution


The local maximum problem for hill-climbing with 3-level look ahead


Dynamic ProgrammingDynamic ProgrammingForward-backward searchingForward-backward searchingDivide-and-conquer: Divided problems into multiple Divide-and-conquer: Divided problems into multiple interacting and related subproblemsinteracting and related subproblemsAddress issues of reusing subproblems solutions Address issues of reusing subproblems solutions An example: Fibonacci seriesAn example: Fibonacci series– F(0) = 1; F(1)=1; F(n)=F(n-1)+F(n-2)F(0) = 1; F(1)=1; F(n)=F(n-1)+F(n-2)– Keep track of the computation F(n-1) and F(n-2), and Keep track of the computation F(n-1) and F(n-2), and

reuse their results to compute F(n)reuse their results to compute F(n)– Compare with recursion, much more efficientCompare with recursion, much more efficient

Applications:Applications:– String matchingString matching– Spell checkingSpell checking– Nature language processing and understandingNature language processing and understanding– planningplanning


• Find an optimal global alignment of two character strings

• Data structure: (n+1)(m+1) array, each element reflects the global alignment success to that point

• Three possible costs for the current state• If a character is shifted along in the shorter string for better

possible alignment, the cost is 1 and recorded in the column score; and If a new character is inserted, cost is 1 and reflected in the row score

• If the characters to be aligned are different, shift and insert, the cost is 2

• If identical, the cost is 0• The initialization stage and first step in completing the

array for character alignment using dynamic programming.

Global Alignment of StringsGlobal Alignment of Strings


• The initialization stage and first step in completing the array for character alignment using dynamic programming.


The completed array reflecting the maximum alignment information for the strings.


A completed backward component of the dynamic programming example giving one (of several possible) string alignments.


Minimum Edit DifferenceMinimum Edit DifferenceDetermine the best approximate words of Determine the best approximate words of s misspelling word in spelling checkers misspelling word in spelling checker

Specified as the number of character Specified as the number of character insertion, deletion, and replacements insertion, deletion, and replacements necessary to turn the first string into the necessary to turn the first string into the secondsecond

Cost: 1 for insertion and deletion, and 2 for Cost: 1 for insertion and deletion, and 2 for replacementreplacement

Determine the minimum cost differenceDetermine the minimum cost difference

Data structure: array Data structure: array


Initialization of minimum edit difference matrix between intention and execution


Array elements are the costs of the minimum editing to that point plus the minimum cost of either an insertion, deletion or replacement

Cost(x, y) = min{ Cost(x-1, y) + 1 (insertion cost),

Cost(x-1, y-1) + 2 (replacement cost),

Cost(x, y-1) + 1 (deletion cost) }


Intention

ntention delete I, cost 1

etention replace n with e, cost 2

exention replace t with x, cost 2

exenution insert u, cost 1

execution replace n with c, cost 2

Complete array of minimum edit difference between intention and execution (of several possible) string alignments.


The Best-First SearchThe Best-First Search

Also heuristic search – use heuristic Also heuristic search – use heuristic (evaluation) function to select the (evaluation) function to select the best state to explorebest state to explore

Can be implemented with a priority Can be implemented with a priority queuequeue– Breadth-first implemented with a queueBreadth-first implemented with a queue– Depth-first implemented with a stackDepth-first implemented with a stack


The best-The best-first first search search algorithmalgorithm


Heuristic search of a hypothetical state space

13


A trace of the execution of best-first-search


Heuristic search of a hypothetical state space with open and closed states highlighted


Implement Heuristic Evaluation FunctionImplement Heuristic Evaluation Function

Heuristics can be evaluated in different Heuristics can be evaluated in different waysways8-puzzle problem8-puzzle problem– Heuristic 1: count the tiles out of places Heuristic 1: count the tiles out of places

compared with the goal statecompared with the goal state– Heuristic 2: sum all the distances by which the Heuristic 2: sum all the distances by which the

tiles are out of pace, one for each square a tile tiles are out of pace, one for each square a tile must be moved to reach its position in the goal must be moved to reach its position in the goal statestate

– Heuristic 3: multiply a small number (say, 2) Heuristic 3: multiply a small number (say, 2) times each direct tile reversal (where two times each direct tile reversal (where two adjacent tiles must be exchanged to be in the adjacent tiles must be exchanged to be in the order of the goal)order of the goal)


The start state, first moves, and goal state for an example-8 puzzle


Three heuristics applied to states in the 8-puzzle


Heuristic DesignHeuristic DesignUse the limited information available in a single Use the limited information available in a single state to make intelligent choicesstate to make intelligent choicesEmpirical, judgment, and intuitionEmpirical, judgment, and intuitionMust be its actual performance on problem instancesMust be its actual performance on problem instancesThe solution path consists of two parts: from the The solution path consists of two parts: from the starting state to the current state, and from the starting state to the current state, and from the current state to the goal statecurrent state to the goal stateThe first part can be evaluated using the known The first part can be evaluated using the known informationinformationThe second part must be estimated using unknown The second part must be estimated using unknown informationinformationThe total evaluation can be The total evaluation can be

f(n) = g(n) + h(n)f(n) = g(n) + h(n)g(n) – from the starting state to the current state ng(n) – from the starting state to the current state nh(n) – from the current state n to the goal stateh(n) – from the current state n to the goal state


The heuristic f applied to states in the 8-puzzle


State space generated in heuristic search of the 8-puzzle graph


The successive stages of open and closed that generate the graph are:


Open and closed as they appear after the 3rd iteration of heuristic search


Heuristic Design SummaryHeuristic Design Summaryf(n) is computed as the sum of g(n) and h(n)f(n) is computed as the sum of g(n) and h(n)

g(n) is the depth of n in the search space and has g(n) is the depth of n in the search space and has the search more of a breadth-first flavor. the search more of a breadth-first flavor.

h(n) is the heuristic estimate of the distance from h(n) is the heuristic estimate of the distance from n to a goaln to a goal

The h value guides search toward heuristically The h value guides search toward heuristically promising statespromising states

The g value grows to determine h and force The g value grows to determine h and force search back to a shorter path, and thus prevents search back to a shorter path, and thus prevents search from persisting indefinitely on a fruitless search from persisting indefinitely on a fruitless pathpath


Admissibility, Monotonicity, and Admissibility, Monotonicity, and InformednessInformedness

A best-first search algorithm A best-first search algorithm guarantee to find a best path, if guarantee to find a best path, if exists, if the algorithm is admissibleexists, if the algorithm is admissible

A best-first search algorithm is A best-first search algorithm is admissible if its heuristic function h is admissible if its heuristic function h is monotonemonotone


Admissibility and Algorithm A*Admissibility and Algorithm A*


Monotonicity and InformednessMonotonicity and Informedness


Comparison of state space searched using heuristic search with space searched by breadth-first search. The proportion of the graph searched heuristically is shaded. The optimal search selection is in bold. Heuristic used is f(n) = g(n) + h(n) where h(n) is tiles out of place.


Minimax ProcedureMinimax ProcedureGames Games – Two players attempting to winTwo players attempting to win– Two opponents are referred to as MAX Two opponents are referred to as MAX

and MINand MIN

A variant of game nimA variant of game nim– A number of tokens on a table between A number of tokens on a table between

the 2 opponentsthe 2 opponents– Each player divides a pile of tokens into Each player divides a pile of tokens into

two nonempty piles of different sizestwo nonempty piles of different sizes– The player who cannot make division The player who cannot make division

losseslosses


State space for a variant of nim. Each state partitions the seven matches into one or more piles

Exhaustive SearchExhaustive Search


Maxmin SearchMaxmin SearchPrinciplesPrinciples– MAX tries to win by maximizing her score, moves to a MAX tries to win by maximizing her score, moves to a

state that is best for MAXstate that is best for MAX– MIN, the opponent, tries to minimize the MAX’s score, MIN, the opponent, tries to minimize the MAX’s score,

moves to a state that is worst for MAXmoves to a state that is worst for MAX– Both share the same informationBoth share the same information– MIN moves firstMIN moves first– The terminating state that MAX wins is scored 1, The terminating state that MAX wins is scored 1,

otherwise 0otherwise 0– Other states are valued by propagating the value of Other states are valued by propagating the value of

terminating statesterminating states

Value propagating rulesValue propagating rules– If the parent state is a MAX node, it is given the If the parent state is a MAX node, it is given the

maximum value among its childrenmaximum value among its children– If the parent state is a MIN state, it is given the minimum If the parent state is a MIN state, it is given the minimum

value of its children value of its children


Exhaustive minimax for the game of nim. Bold lines indicate forced win for MAX. Each node is marked with its derived value (0 or 1) under minimax.


Minmaxing to Fixed Ply DepthMinmaxing to Fixed Ply DepthIf cannot expand the state space to terminating (leaf) nodes (explosive), can use the fixed ply depthSearch to a predefined number, n, of levels from the starting state, n-ply look-aheadThe problem is how to value the nodes at the predefined level – heuristicsPropagating values is similar – Maximum children for MAX nodes– Minimum children for MIN nodes


Minimax to a hypothetical state space. Leaf states show heuristic values; internal states show backed-up values.


Heuristic measuring conflict applied to states of tic-tac-toe


Two-ply minimax applied to the opening move of tic-tac-toe


Two ply minimax, and one of two possible MAX second moves


Two-ply minimax applied to X’s move near the end of the game


Alpha-Beta ProcedureAlpha-Beta ProcedureAlpha-beta pruning to improve search efficiencyAlpha-beta pruning to improve search efficiencyProceeds in a depth-first fashion and creates two Proceeds in a depth-first fashion and creates two values values alphaalpha and and betabeta during the search during the searchAlpha associated with MAX nodes, and never Alpha associated with MAX nodes, and never decreasesdecreasesBeta associated with MIN nodes, never increasesBeta associated with MIN nodes, never increasesTo begin, descend to full ply depth in a depth-first To begin, descend to full ply depth in a depth-first search, and apply heuristic evaluation to a state and search, and apply heuristic evaluation to a state and all its siblings. The value propagation is the same as all its siblings. The value propagation is the same as minimax procedureminimax procedureNext, descend to other grandchildren and terminate Next, descend to other grandchildren and terminate exploration if any of their values is >= this beta valueexploration if any of their values is >= this beta valueTerminating criteriaTerminating criteria– Below any MIN node having beta <= alpha of any of its MAX Below any MIN node having beta <= alpha of any of its MAX

ancestorsancestors– Below any MAX node having alpha >= beta of any of its MIN Below any MAX node having alpha >= beta of any of its MIN

ancestorsancestors


Alpha-beta pruning applied. States without numbers are not evaluated.

csc411artificial intelligence1 chapter 4 heuristic search contents hill-climbing dynamic programming...

Documents