![Page 1: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/1.jpg)
Heuristic Search
Rob PlattNortheastern University
Some images and slides are used from:AIMA
![Page 2: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/2.jpg)
Recap: What is graph search?
Graph search: find a path from start to goal
– what are the states?
– what are the actions (transitions)?
– how is this a graph?
Start state Goal state
![Page 3: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/3.jpg)
Recap: What is graph search?
Graph search: find a path from start to goal
– what are the states?
– what are the actions (transitions)?
– how is this a graph?
Start state
Goal state
![Page 4: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/4.jpg)
Recap: BFS/UCS
start
goal
It's like this
– search in all directions equally until discovering goal
![Page 5: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/5.jpg)
Idea
Is it possible to use additional information to decide which direction to search in?
![Page 6: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/6.jpg)
Idea
Is it possible to use additional information to decide which direction to search in?
Yes!
Instead of searching in all directions, let's bias search in the direction of the goal.
![Page 7: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/7.jpg)
Example
Stright-line distances to Bucharest
![Page 8: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/8.jpg)
Example
Start state
Goal state
Expand states in order of their distance to the goal
– for each state that you put on the fringe: calculate straight-line distance to the goal
– expand the state on the fringe closest to the goal
![Page 9: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/9.jpg)
Example
Start state
Goal state
Expand states in order of their distance to the goal
– for each state that you put on the fringe: calculate straight-line distance to the goal
– expand the state on the fringe closest to the goal
Heuristic:
Greedy search
![Page 10: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/10.jpg)
Greedy Search
![Page 11: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/11.jpg)
Greedy Search
Each time you expand a state, calculate the heuristic for each of the states that you add to the fringe.
– heuristic:
– on each step, choose to expand the state with the lowest heuristic value.
i.e. distance to Bucharest
![Page 12: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/12.jpg)
Greedy Search
Each time you expand a state, calculate the heuristic for each of the states that you add to the fringe.
– heuristic:
– on each step, choose to expand the state with the lowest heuristic value.
i.e. distance to Bucharest
This is like a guess about how far the state is from the goal
![Page 13: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/13.jpg)
Example: Greedy Search
![Page 14: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/14.jpg)
Example: Greedy Search
![Page 15: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/15.jpg)
Example: Greedy Search
![Page 16: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/16.jpg)
Example: Greedy Search
Path: A-S-F-B
![Page 17: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/17.jpg)
Example: Greedy Search
Notice that this is not the optimal path!
Path: A-S-F-B
![Page 18: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/18.jpg)
Example: Greedy Search
Notice that this is not the optimal path!
Path: A-S-F-B
Greedy Search:– Not optimal– Not complete– But, it can be very fast
![Page 19: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/19.jpg)
Greedy vs UCS
Greedy Search:– Not optimal– Not complete– But, it can be very fast
UCS:– Optimal– Complete– Usually very slow
![Page 20: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/20.jpg)
Greedy vs UCS
Greedy Search:– Not optimal– Not complete– But, it can be very fast
UCS:– Optimal– Complete– Usually very slow
Can we combine greedy and UCS???
![Page 21: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/21.jpg)
Greedy vs UCS
Greedy Search:– Not optimal– Not complete– But, it can be very fast
UCS:– Optimal– Complete– Usually very slow
Can we combine greedy and UCS???
YES: A*
![Page 22: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/22.jpg)
A*
![Page 23: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/23.jpg)
A*
: a state
: minimum cost from start to
: heuristic at (i.e. an estimate of remainingcost-to-go)
UCS: expand states in order of
Greedy: expand states in order of
A*: expand states in order of
![Page 24: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/24.jpg)
A*
: a state
: minimum cost from start to
: heuristic at (i.e. an estimate of remainingcost-to-go)
UCS: expand states in order of
Greedy: expand states in order of
A*: expand states in order of
What is “cost-to-go”?
![Page 25: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/25.jpg)
A*
: a state
: minimum cost from start to
: heuristic at (i.e. an estimate of remainingcost-to-go)
UCS: expand states in order of
Greedy: expand states in order of
A*: expand states in order of
What is “cost-to-go”?– minimum cost required
to reach a goal state
![Page 26: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/26.jpg)
A*
h=0
S a d
b
Gh=5
h=6
h=2
1
8
1
1
2
h=6
c
h=7
3
e h=11
• Uniform-cost orders by path cost from Start: g(n)
• Greedy orders by estimated cost to goal: h(n)
• A* orders by the sum: f(n) = g(n) + h(n)
Modified from: Teg Grenager
![Page 27: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/27.jpg)
When should A* terminate?
Slide: Adapted from Berkeley CS188 course notes (downloaded Summer 2015)
S
B
A
G
2
3
2
2h = 1
h = 2
h = 0h = 3
Should we stop when we enqueue a goal?
No: only stop when we dequeue a goal
![Page 28: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/28.jpg)
Is A* optimal?
A
GS
1 3
h = 6
h = 0
5
h = 7
Image: Adapted from Berkeley CS188 course notes (downloaded Summer 2015)
What went wrong here?
![Page 29: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/29.jpg)
When is A* optimal?
It depends on whether we are using the tree search or the graph search version of the algorithm.
Recall: – in tree search, we do not track the explored set– in graph search, we do
![Page 30: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/30.jpg)
Recall: Breadth first search (BFS)
What is the purpose of the explored set?
![Page 31: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/31.jpg)
When is A* optimal?
It depends on whether we are using the tree search or the graph search version of the algorithm.
Optimal if h is admissibleOptimal if h is consistent
![Page 32: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/32.jpg)
When is A* optimal?
It depends on whether we are using the tree search or the graph search version of the algorithm.
Optimal if h is admissible
– h(s) is an underestimate of the true cost-to-go.
Optimal if h is consistent
– h(s) is an underestimate of the cost of each arc.
![Page 33: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/33.jpg)
When is A* optimal?
It depends on whether we are using the tree search or the graph search version of the algorithm.
Optimal if h is admissible
– h(s) is an underestimate of the true cost-to-go.
Optimal if h is consistent
– h(s) is an underestimate of the cost of each arc.
What is “cost-to-go”?– minimum cost required
to reach a goal state
![Page 34: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/34.jpg)
When is A* optimal?
It depends on whether we are using the tree search or the graph search version of the algorithm.
Optimal if h is admissible
– h(s) is an underestimate of the true cost-to-go.
Optimal if h is consistent
– h(s) is an underestimate of the cost of each arc.
More on this later...
![Page 35: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/35.jpg)
Admissibility: Example
Stright-line distances to Bucharest
h(s) = straight-line distance to goal state (Bucharest)
![Page 36: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/36.jpg)
Admissibility
Stright-line distances to Bucharest
h(s) = straight-line distance to goal state (Bucharest)
Is this heuristic admissible???
![Page 37: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/37.jpg)
Admissibility
Stright-line distances to Bucharest
h(s) = straight-line distance to goal state (Bucharest)
Is this heuristic admissible???YES! Why?
![Page 38: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/38.jpg)
Admissibility: Example
h(s) = ?
Start state Goal state
Can you think of an admissible heuristic for this problem?
![Page 39: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/39.jpg)
Admissibility
Why isn't this heuristic admissible?
A
GS
1 3
h = 6
h = 0
5
h = 7
Image: Adapted from Berkeley CS188 course notes (downloaded Summer 2015)
![Page 40: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/40.jpg)
Consistency
What went wrong?
Slide: Adapted from Berkeley CS188 course notes (downloaded Summer 2015)
S
A
B
C
G
1
1
1
23
h=2
h=1
h=4h=1
h=0
S (0+2)
A (1+4) B (1+1)
C (2+1)
G (5+0)
C (3+1)
G (6+0)
State space graph Search tree
![Page 41: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/41.jpg)
Consistency
Cost of going from s to s'
s s'
![Page 42: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/42.jpg)
Consistency
Rearrange terms
![Page 43: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/43.jpg)
Consistency
Cost of going from s to s' implied by heuristic
Actual cost of going from s to s'
![Page 44: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/44.jpg)
Consistency
Cost of going from s to s' implied by heuristic
Actual cost of going from s to s'
![Page 45: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/45.jpg)
Consistency
Consistency implies that the “f-cost” never decreases along any path to a goal state.– the optimal path gives a goal state its lowest f-cost.
A* expands states in order of their f-cost.
Given any goal state, A* expands states that reach the goal state optimally before expanding states the reach the goal state suboptimally.
![Page 46: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/46.jpg)
Consistency implies admissibility
Suppose:
Then:
![Page 47: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/47.jpg)
Consistency implies admissibility
Suppose:
Then:
![Page 48: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/48.jpg)
Consistency implies admissibility
Suppose:
Then: admissible
![Page 49: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/49.jpg)
Consistency implies admissibility
Suppose:
Then:
![Page 50: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/50.jpg)
Consistency implies admissibility
Suppose:
Then:
admissible
![Page 51: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/51.jpg)
Consistency implies admissibility
Suppose:
Then:
admissibleadmissible
![Page 52: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/52.jpg)
Consistency implies admissibility
Suppose:
Then:
![Page 53: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/53.jpg)
A* vs UCS
Greedy UCS A*
start
goalgoal
start start
goal
![Page 54: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/54.jpg)
Choosing a heuristic
The right heuristic is often problem-specific.
But it is very important to select a good heuristic!
![Page 55: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/55.jpg)
Choosing a heuristic
How much better is ?
Consider the 8-puzzle:
: number of misplaced tiles
: sum of manhattan distances between each tile and its goal.
![Page 56: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/56.jpg)
Choosing a heuristic
Consider the 8-puzzle:
: number of misplaced tiles
: sum of manhattan distances between each tile and its goal.
Average # states expanded on a random depth-24 puzzle:
(by depth 12)
![Page 57: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/57.jpg)
Choosing a heuristic
Consider the 8-puzzle:
: number of misplaced tiles
: sum of manhattan distances between each tile and its goal.
Average # states expanded on a random depth-24 puzzle:
(by depth 12)
So, getting the heuristic right can speed things up by multiple orders of magnitude!
![Page 58: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/58.jpg)
Choosing a heuristic
Consider the 8-puzzle:
: number of misplaced tiles
: sum of manhattan distances between each tile and its goal.
Why not use the actual cost to goal as a heuristic?
![Page 59: Heuristic Search - Northeastern University College of ... Search Rob Platt Northeastern University Some images and slides are used from: AIMA](https://reader034.vdocuments.us/reader034/viewer/2022051600/5aa1e13c7f8b9a84398c485d/html5/thumbnails/59.jpg)
How to choose a heuristic?
Nobody has an answer that always works.
A couple of best-practices:– solve a relaxed version of the problem– solve a subproblem