ITCS 3153Artificial Intelligence

Lecture 7Lecture 7

Informed SearchesInformed Searches

Lecture 7Lecture 7

Informed SearchesInformed Searches

Subproblems

• Is 4-piece subproblem an admissible heuristic?Is 4-piece subproblem an admissible heuristic?

– it can never overestimate the true costit can never overestimate the true cost

• Is it consistent?Is it consistent?

– h(n) <= c(n, a, n’) + h(n’)h(n) <= c(n, a, n’) + h(n’)

• Is 4-piece subproblem an admissible heuristic?Is 4-piece subproblem an admissible heuristic?

– it can never overestimate the true costit can never overestimate the true cost

• Is it consistent?Is it consistent?

– h(n) <= c(n, a, n’) + h(n’)h(n) <= c(n, a, n’) + h(n’)

Genetic Algorithms (GAs)

Another randomized search algorithmAnother randomized search algorithm

Start with k initial guessesStart with k initial guesses• they form a they form a populationpopulation

• each each individualindividual from the population is a fixed-length string from the population is a fixed-length string (gene)(gene)

• each individual’s each individual’s fitness fitness is evaluatedis evaluated

• successors are generated from individuals according to successors are generated from individuals according to fitness functionfitness function results results

Another randomized search algorithmAnother randomized search algorithm

Start with k initial guessesStart with k initial guesses• they form a they form a populationpopulation

• each each individualindividual from the population is a fixed-length string from the population is a fixed-length string (gene)(gene)

• each individual’s each individual’s fitness fitness is evaluatedis evaluated

• successors are generated from individuals according to successors are generated from individuals according to fitness functionfitness function results results

Genetic Algorithms

• ReproductionReproduction

– ReuseReuse

– CrossoverCrossover

• MutationMutation

• ReproductionReproduction

– ReuseReuse

– CrossoverCrossover

• MutationMutation

Crossover

• Early states are diverseEarly states are diverse

– Crossover explores state broadlyCrossover explores state broadly

• Later stages are more similarLater stages are more similar

– Crossover fine tunes in small regionCrossover fine tunes in small region

• Early states are diverseEarly states are diverse

– Crossover explores state broadlyCrossover explores state broadly

• Later stages are more similarLater stages are more similar

– Crossover fine tunes in small regionCrossover fine tunes in small region} Like simulated

annealing

Mutation

Could screw up a good solutionCould screw up a good solution

• Like metropolis step in simulated annealingLike metropolis step in simulated annealing

Could explore untapped part of search spaceCould explore untapped part of search space

Could screw up a good solutionCould screw up a good solution

• Like metropolis step in simulated annealingLike metropolis step in simulated annealing

Could explore untapped part of search spaceCould explore untapped part of search space

GA Analysis

CombinesCombines

• uphill tendencyuphill tendency

• random explorationrandom exploration

• exchange information between multiple threadsexchange information between multiple threads

– like stochastic beam searchlike stochastic beam search

Crossover is not needed – theoreticallyCrossover is not needed – theoretically

• if starting states are sufficiently randomif starting states are sufficiently random

CombinesCombines

• uphill tendencyuphill tendency

• random explorationrandom exploration

• exchange information between multiple threadsexchange information between multiple threads

– like stochastic beam searchlike stochastic beam search

Crossover is not needed – theoreticallyCrossover is not needed – theoretically

• if starting states are sufficiently randomif starting states are sufficiently random

GA Analysis

It’s all in the representationIt’s all in the representation

• GA works best if representation stores related pieces of the GA works best if representation stores related pieces of the puzzle in neighboring cells of stringpuzzle in neighboring cells of string

• Not all problems are amenable to crossoverNot all problems are amenable to crossover

– TSPTSP

It’s all in the representationIt’s all in the representation

• GA works best if representation stores related pieces of the GA works best if representation stores related pieces of the puzzle in neighboring cells of stringpuzzle in neighboring cells of string

• Not all problems are amenable to crossoverNot all problems are amenable to crossover

– TSPTSP

Continuous Spaces

What does continuous mean to you?What does continuous mean to you?What does continuous mean to you?What does continuous mean to you?

A function is continuous if its graph can be drawn without lifting A function is continuous if its graph can be drawn without lifting the pencil from the paper Descartethe pencil from the paper Descarte

A function is continuous if its graph can be drawn without lifting A function is continuous if its graph can be drawn without lifting the pencil from the paper Descartethe pencil from the paper Descarte

In terms of searching?

Continuous search spaces have neighbors for all Continuous search spaces have neighbors for all statesstates

That means they have derivativesThat means they have derivatives

Can the derivative help out here?Can the derivative help out here?

Continuous search spaces have neighbors for all Continuous search spaces have neighbors for all statesstates

That means they have derivativesThat means they have derivatives

Can the derivative help out here?Can the derivative help out here?

Derivative directs future steps

One dimensional functionOne dimensional function

• Left or right?Left or right?

Two dimensional functionTwo dimensional function

• Direction in 3-spaceDirection in 3-space

N-dimensional functionN-dimensional function

• GradientGradient

One dimensional functionOne dimensional function

• Left or right?Left or right?

Two dimensional functionTwo dimensional function

• Direction in 3-spaceDirection in 3-space

N-dimensional functionN-dimensional function

• GradientGradient ),...,,(21 nx

f

x

f

x

ff

An Example

Place three airportsPlace three airports

• minimize sum of minimize sum of squared distances from squared distances from each city to closest each city to closest airport, f(airport, f(xx))

• find (xfind (x11, y, y11, x, x22, y, y22, x, x33, y, y33))

Place three airportsPlace three airports

• minimize sum of minimize sum of squared distances from squared distances from each city to closest each city to closest airport, f(airport, f(xx))

• find (xfind (x11, y, y11, x, x22, y, y22, x, x33, y, y33))

Airport Example

• Simulated AnnealingSimulated Annealing

– Trial and error experimentation with six valuesTrial and error experimentation with six values

• Genetic AlgorithmsGenetic Algorithms

– Create a gene with six values and crossover/mutateCreate a gene with six values and crossover/mutate

• Discretize some slight change in position, Discretize some slight change in position,

– each of six parameters has three values (+,-,same)each of six parameters has three values (+,-,same)

– branching factor of 18branching factor of 18

– A* searchA* search

• Simulated AnnealingSimulated Annealing

– Trial and error experimentation with six valuesTrial and error experimentation with six values

• Genetic AlgorithmsGenetic Algorithms

– Create a gene with six values and crossover/mutateCreate a gene with six values and crossover/mutate

• Discretize some slight change in position, Discretize some slight change in position,

– each of six parameters has three values (+,-,same)each of six parameters has three values (+,-,same)

– branching factor of 18branching factor of 18

– A* searchA* search

Airport Example

Derivatives!Derivatives!

• For given f (a, b, c, d, e, f), compute gradientFor given f (a, b, c, d, e, f), compute gradient

– change in f in response to small change in a, then b, …change in f in response to small change in a, then b, …

• Update the vector, Update the vector, xx

• Beware of jumping too far if Beware of jumping too far if too large too large

• Beware of local minBeware of local min

– each parameter may settle in its own local mineach parameter may settle in its own local min

Derivatives!Derivatives!

• For given f (a, b, c, d, e, f), compute gradientFor given f (a, b, c, d, e, f), compute gradient

– change in f in response to small change in a, then b, …change in f in response to small change in a, then b, …

• Update the vector, Update the vector, xx

• Beware of jumping too far if Beware of jumping too far if too large too large

• Beware of local minBeware of local min

– each parameter may settle in its own local mineach parameter may settle in its own local min

)( oldoldnew xfxx

Computing the Gradient

• Difficult to solve in Difficult to solve in closedclosed form form

– compute f’(compute f’(xx)=)=y y that works for all that works for all xx

• We can usually compute We can usually compute locallylocally

– compute f’(compute f’(xx)=)=y y that works for only that works for only x x near near zz

• We can also compute We can also compute empiricallyempirically

– pick some small offset to add to each element of x and pick some small offset to add to each element of x and compute difference beween f(compute difference beween f(xx) and f() and f(x+x+))

• Difficult to solve in Difficult to solve in closedclosed form form

– compute f’(compute f’(xx)=)=y y that works for all that works for all xx

• We can usually compute We can usually compute locallylocally

– compute f’(compute f’(xx)=)=y y that works for only that works for only x x near near zz

• We can also compute We can also compute empiricallyempirically

– pick some small offset to add to each element of x and pick some small offset to add to each element of x and compute difference beween f(compute difference beween f(xx) and f() and f(x+x+))

Derivative = 0 at Max/Min

Newton-RaphsonNewton-Raphson

• Find the zero of an equation (where it crosses x-axis)Find the zero of an equation (where it crosses x-axis)

• set p = pset p = p00 – f(p – f(p00) / f’(p) / f’(p00))

– if p is close to pif p is close to p00 then return p then return p

• else set pelse set p00 = p and repeat = p and repeat

Newton-RaphsonNewton-Raphson

• Find the zero of an equation (where it crosses x-axis)Find the zero of an equation (where it crosses x-axis)

• set p = pset p = p00 – f(p – f(p00) / f’(p) / f’(p00))

– if p is close to pif p is close to p00 then return p then return p

• else set pelse set p00 = p and repeat = p and repeat

Newton-Raphson

Why set p = pWhy set p = p00 – f(p – f(p00) / f’(p) / f’(p00))

• y-intercept equation of line: y = mx + by-intercept equation of line: y = mx + b

• let y-intercept, b = f (plet y-intercept, b = f (p00))

• let slope, m = f’(plet slope, m = f’(p00))

• we want to find x-value where y-value = 0we want to find x-value where y-value = 0

– let y = 0let y = 0

• 0 = f’(p0 = f’(p00) * x + f(p) * x + f(p00))

– solve for x: -f(psolve for x: -f(p00) / f’(p) / f’(p00))

Why set p = pWhy set p = p00 – f(p – f(p00) / f’(p) / f’(p00))

• y-intercept equation of line: y = mx + by-intercept equation of line: y = mx + b

• let y-intercept, b = f (plet y-intercept, b = f (p00))

• let slope, m = f’(plet slope, m = f’(p00))

• we want to find x-value where y-value = 0we want to find x-value where y-value = 0

– let y = 0let y = 0

• 0 = f’(p0 = f’(p00) * x + f(p) * x + f(p00))

– solve for x: -f(psolve for x: -f(p00) / f’(p) / f’(p00))

But we’re not finding Zero of f(x)

We’re finding zero of gradient(x)We’re finding zero of gradient(x)

• So, replace f(x) with gradient of xSo, replace f(x) with gradient of x

• replace f’(x) with second derivative of xreplace f’(x) with second derivative of x

– HessianHessian

We’re finding zero of gradient(x)We’re finding zero of gradient(x)

• So, replace f(x) with gradient of xSo, replace f(x) with gradient of x

• replace f’(x) with second derivative of xreplace f’(x) with second derivative of x

– HessianHessian

Hessian

Second derivative of a multivariable functionSecond derivative of a multivariable function

HHff(x) = Hessian = (x) = Hessian =

Second derivative of a multivariable functionSecond derivative of a multivariable function

HHff(x) = Hessian = (x) = Hessian =

23

2

23

2

13

232

2

22

2

12

231

2

21

2

21

2

x

f

xx

f

xx

fxx

f

x

f

xx

fxx

f

xx

f

x

f

Newton Raphson

Final EquationFinal EquationFinal EquationFinal Equation

)()(1 xfxHxx f

Online Searches

States and Actions are unknown apriori (before)States and Actions are unknown apriori (before)

• A real robot finding its way through a mazeA real robot finding its way through a maze

State is difficult to changeState is difficult to change

• A real robot cannot jump across state space at will to explore best A real robot cannot jump across state space at will to explore best potential pathspotential paths

State is difficult/impossible to reverseState is difficult/impossible to reverse

• Can you ride your bike backwards? Do you have space to turn around?Can you ride your bike backwards? Do you have space to turn around?

• When spelunking, do you send the smallest person through the tunnel When spelunking, do you send the smallest person through the tunnel first?first?

States and Actions are unknown apriori (before)States and Actions are unknown apriori (before)

• A real robot finding its way through a mazeA real robot finding its way through a maze

State is difficult to changeState is difficult to change

• A real robot cannot jump across state space at will to explore best A real robot cannot jump across state space at will to explore best potential pathspotential paths

State is difficult/impossible to reverseState is difficult/impossible to reverse

• Can you ride your bike backwards? Do you have space to turn around?Can you ride your bike backwards? Do you have space to turn around?

• When spelunking, do you send the smallest person through the tunnel When spelunking, do you send the smallest person through the tunnel first?first?

Online Searches

• Difficult to skip around when using A*Difficult to skip around when using A*

• Potential of irreversible dead end with depth-first searchPotential of irreversible dead end with depth-first search

• Local search is perfect for online searchesLocal search is perfect for online searches

– stop when you cannot improve any furtherstop when you cannot improve any further

– chances are high of stopping at local solutionchances are high of stopping at local solution

– add memory to permit continued exploration with ability to add memory to permit continued exploration with ability to return to best solution (the Brady Bunch again)return to best solution (the Brady Bunch again)

• Difficult to skip around when using A*Difficult to skip around when using A*

• Potential of irreversible dead end with depth-first searchPotential of irreversible dead end with depth-first search

• Local search is perfect for online searchesLocal search is perfect for online searches

– stop when you cannot improve any furtherstop when you cannot improve any further

– chances are high of stopping at local solutionchances are high of stopping at local solution

– add memory to permit continued exploration with ability to add memory to permit continued exploration with ability to return to best solution (the Brady Bunch again)return to best solution (the Brady Bunch again)

Learning in Online Search

Online agents must resolve ignoranceOnline agents must resolve ignorance

• exploreexplore the world the world

• build a build a mapmap

– mapping of (state, action) to resultsmapping of (state, action) to results

– also called a also called a modelmodel relating (state, action) to results relating (state, action) to results

Online agents must resolve ignoranceOnline agents must resolve ignorance

• exploreexplore the world the world

• build a build a mapmap

– mapping of (state, action) to resultsmapping of (state, action) to results

– also called a also called a modelmodel relating (state, action) to results relating (state, action) to results