planning algorithms: when optimal just isn’t good enoughruml/papers/uml-2011-talk.pdf ·...
TRANSCRIPT
![Page 1: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/1.jpg)
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 1 / 33
Planning Algorithms:When Optimal Just Isn’t Good Enough
Wheeler Ruml
Department of Computer Science
Joint work with the UNH AI Group. Funded by NSF and DARPA.
![Page 2: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/2.jpg)
Where are the Walking Talking Robots?
Introduction
■ Robots?
■ Toddler Steps
■ An Agent
Planning as Search
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 2 / 33
Lt. Commander Data Lt. Sharon Valerii Zoe GreystoneStar Trek: TNG Battlestar Galactica Caprica
![Page 3: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/3.jpg)
Toddler Steps
Introduction
■ Robots?
■ Toddler Steps
■ An Agent
Planning as Search
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 3 / 33
![Page 4: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/4.jpg)
The Role of Planning
Introduction
■ Robots?
■ Toddler Steps
■ An Agent
Planning as Search
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 4 / 33
world model
planner
agent
world
actions
sensing
![Page 5: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/5.jpg)
The Role of Planning
Introduction
■ Robots?
■ Toddler Steps
■ An Agent
Planning as Search
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 4 / 33
world model
planner
search
agent
world
actions
sensing
![Page 6: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/6.jpg)
Planning as Heuristic Graph Search
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 5 / 33
![Page 7: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/7.jpg)
Planning
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 6 / 33
Given:
■ current state of the world■ models of available actions
preconditions, effects, costs
■ desired state of the world (partially specified?)
Find:
■ cheapest plan
![Page 8: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/8.jpg)
Graph Search
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 7 / 33
Given:
■ start state: an explicit node■ expand function: generates children and their costs
assume arc cost ≥ 0■ goal test: predicate on nodes
Find:
■ cheapest path to a goal node
current
next
action a b
![Page 9: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/9.jpg)
Graph Search
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 7 / 33
Given:
■ start state: an explicit node■ expand function: generates children and their costs
assume arc cost ≥ 0■ goal test: predicate on nodes
Find:
■ cheapest path to a goal node
root
a
a
a b
b
a b
b
a
a b
b
a b
![Page 10: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/10.jpg)
Graph Search
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 7 / 33
Given:
■ start state: an explicit node■ expand function: generates children and their costs
assume arc cost ≥ 0■ goal test: predicate on nodes
Find:
■ cheapest path to a goal node
root = 0
1
2
2 2
1
3 1
2
1
2 3
2
1 3
![Page 11: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/11.jpg)
Example: Motion Planning
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 8 / 33
(S. LaValle)
![Page 12: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/12.jpg)
The Problem: Exponential Growth
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 9 / 33
Ever wonder why no one draws trees more than 3 levels deep?
![Page 13: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/13.jpg)
The Problem: Exponential Growth
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 9 / 33
Ever wonder why no one draws trees more than 3 levels deep?
number of actionsplan length
14100 > 1080
![Page 14: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/14.jpg)
A Tale of Three Algorithms
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 10 / 33
1. Uniform Cost Search (Dijkstra, 1959)
2. A* Search (Hart, Nilsson, and Raphael, 1968)
3. Explicit Estimation Search (Thayer and Ruml, 2011)
![Page 15: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/15.jpg)
Uniform Cost Search (Dijkstra, 1959)
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 11 / 33
![Page 16: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/16.jpg)
Uniform Cost Search (Dijkstra, 1959)
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 11 / 33
Explore nodes in increasing order of cost-so-far (g(n)):
![Page 17: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/17.jpg)
Uniform Cost Search (Dijkstra, 1959)
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 11 / 33
Explore nodes in increasing order of cost-so-far (g(n)):
Q← ordered list containing the initial stateLoop
If Q is empty, return failureNode ← pop cheapest node off Q
If Node is a goal, return it (or path to it)Children ← Expand(Node).Merge Children into Q, keeping sorted by g(n).
1
2
2 2
1
3 1
2
1
2 3
2
1 3
![Page 18: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/18.jpg)
Dijkstra Behavior
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 12 / 33
![Page 19: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/19.jpg)
Dijkstra Behavior
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 12 / 33
![Page 20: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/20.jpg)
Dijkstra Behavior
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 12 / 33
![Page 21: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/21.jpg)
Dijkstra: Pathfinding in Warcraft
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 13 / 33
![Page 22: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/22.jpg)
Problem-Specific Background Knowledge
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 14 / 33
![Page 23: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/23.jpg)
Lower Bounds on Cost-to-go
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 15 / 33
h(n) < f*(n)
![Page 24: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/24.jpg)
Heuristic Search: A* (Hart, Nilsson, and Raphael, 1968)
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 16 / 33
![Page 25: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/25.jpg)
Heuristic Search: A* (Hart, Nilsson, and Raphael, 1968)
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 16 / 33
Cost should include both cost-so-far and cost-to-go:
![Page 26: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/26.jpg)
Heuristic Search: A* (Hart, Nilsson, and Raphael, 1968)
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 16 / 33
Cost should include both cost-so-far and cost-to-go:
Q← ordered list containing the initial stateLoop
If Q is empty, return failureNode ← pop cheapest node off Q
If Node is a goal, return it (or path to it)Children ← Expand(Node)Merge Children into Q, keeping sorted by f(n)
f(n) = g(n) + h(n)g(n) = cost incurred so farh(n) = lower bound on cost to goal
![Page 27: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/27.jpg)
A* Behavior
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 17 / 33
![Page 28: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/28.jpg)
A* Behavior
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 17 / 33
![Page 29: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/29.jpg)
A* Behavior
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 17 / 33
![Page 30: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/30.jpg)
Dijkstra vs A*: Pathfinding in Warcraft
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 18 / 33
![Page 31: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/31.jpg)
Dijkstra vs A*: Pathfinding in Warcraft
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 18 / 33
![Page 32: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/32.jpg)
Quick Note: A Lower Bound on Solution Cost
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 19 / 33
f(n) = g(n) + h(n)
g(n) is actual cost-so-far, sowhen h(n) is a lower bound on cost-to-go,f(n) is a lower bound on final solution cost
lowest f(n) on frontier gives lower bound for entire problem!
bestf = argminn∈open
f(n)
![Page 33: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/33.jpg)
Problems with A*
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33
![Page 34: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/34.jpg)
Problems with A*
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33
A* takes exponential memory
![Page 35: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/35.jpg)
Problems with A*
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33
A* takes exponential memory
Can sometimes be fixed: see ‘iterative deepening’
![Page 36: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/36.jpg)
Problems with A*
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33
A* takes exponential memory
Can sometimes be fixed: see ‘iterative deepening’
A* takes exponential time
![Page 37: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/37.jpg)
Problems with A*
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33
A* takes exponential memory
Can sometimes be fixed: see ‘iterative deepening’
A* takes exponential time
We must trade cost for time.
![Page 38: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/38.jpg)
A New Generation of Problem Settings
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 21 / 33
optimal: minimize solution costmust expand all with f(n) < f∗(opt)
![Page 39: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/39.jpg)
A New Generation of Problem Settings
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 21 / 33
optimal: minimize solution costmust expand all with f(n) < f∗(opt)
greedy: minimize solving time
bounded suboptimal: minimize time subject to relative costbound (factor of optimal)
bounded cost: minimize time subject to absolute cost bound
contract: minimize cost subject to absolute time bound
utility function: maximize utility function of cost and timeeg, goal achievement time =
plan makespan + search time
![Page 40: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/40.jpg)
A New Generation of Problem Settings
Introduction
Planning as Search
■ Planning
■ Graph Search
■ Example
■ The Problem
■ 3 Algorithms
■ Uniform Search
■ Dijkstra Behavior
■ Warcraft
■ Knowledge
■ Lower Bounds
■ Heuristic Search
■ A* Behavior
■ Dijkstra vs A*
■ Lower Bound
■ Problems with A*
■ Problem Settings
Bounded Suboptimal
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 21 / 33
optimal: minimize solution costmust expand all with f(n) < f∗(opt)
greedy: minimize solving time
bounded suboptimal: minimize time subject to relative costbound (factor of optimal)
bounded cost: minimize time subject to absolute cost bound
contract: minimize cost subject to absolute time bound
utility function: maximize utility function of cost and timeeg, goal achievement time =
plan makespan + search time
![Page 41: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/41.jpg)
Bounded Suboptimal Search
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 22 / 33
![Page 42: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/42.jpg)
The Intuition Behind EES (Thayer and Ruml, 2011)
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 23 / 33
■ unbiased estimates can be more informed than lower bounds■ nearest goal is the easiest to find
![Page 43: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/43.jpg)
The Intuition Behind EES (Thayer and Ruml, 2011)
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 23 / 33
■ unbiased estimates can be more informed than lower bounds■ nearest goal is the easiest to find
Heavy Vacuum
Size302010
total ra
w cpu tim
e
400
200
admiss. h
inadmiss. h
![Page 44: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/44.jpg)
The Intuition Behind EES (Thayer and Ruml, 2011)
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 23 / 33
■ unbiased estimates can be more informed than lower bounds■ nearest goal is the easiest to find
Heavy Vacuum
Size302010
total ra
w cpu tim
e
400
200
admiss. h
inadmiss. h
d
![Page 45: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/45.jpg)
The Intuition Behind EES (Thayer and Ruml, 2011)
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 23 / 33
■ unbiased estimates can be more informed than lower bounds■ nearest goal is the easiest to find
Explicit Estimation Search (EES) combines these ideas
to minimize solving time subject to cost ≤ w·optimal:
pursue the nearest goal within the bound
need more information than just lower bound on cost (h(n))!
![Page 46: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/46.jpg)
Three Heuristic Sources of Information
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 24 / 33
1. h: a lower bound on cost-to-gof(n) = g(n) + h(n)the traditional optimal A* lower bound
![Page 47: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/47.jpg)
Three Heuristic Sources of Information
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 24 / 33
1. h: a lower bound on cost-to-gof(n) = g(n) + h(n)the traditional optimal A* lower bound
2. h: an estimate of cost-to-gounbiased estimates can be more informedf(n) = g(n) + h(n)(Thayer and Ruml, ICAPS-11)
![Page 48: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/48.jpg)
Three Heuristic Sources of Information
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 24 / 33
1. h: a lower bound on cost-to-gof(n) = g(n) + h(n)the traditional optimal A* lower bound
2. h: an estimate of cost-to-gounbiased estimates can be more informedf(n) = g(n) + h(n)(Thayer and Ruml, ICAPS-11)
3. d: an estimate of distance-to-gonearest goal is the easiest to find(Pearl and Kim, IEEE PAMI 1982,Thayer et al, ICAPS-09)
![Page 49: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/49.jpg)
Three Heuristic Sources of Information
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 24 / 33
1. h: a lower bound on cost-to-gof(n) = g(n) + h(n)the traditional optimal A* lower bound
2. h: an estimate of cost-to-gounbiased estimates can be more informedf(n) = g(n) + h(n)(Thayer and Ruml, ICAPS-11)
3. d: an estimate of distance-to-gonearest goal is the easiest to find(Pearl and Kim, IEEE PAMI 1982,Thayer et al, ICAPS-09)
pursue the nearest goal within the bound
![Page 50: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/50.jpg)
Finding bestd
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 25 / 33
bestf : open node giving lower bound on cost
argminn∈open
f(n)
![Page 51: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/51.jpg)
Finding bestd
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 25 / 33
bestf : open node giving lower bound on cost
argminn∈open
f(n)
bestf: open node giving estimated optimal cost
argminn∈open
f(n)
![Page 52: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/52.jpg)
Finding bestd
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 25 / 33
bestf : open node giving lower bound on cost
argminn∈open
f(n)
bestf: open node giving estimated optimal cost
argminn∈open
f(n)
pursue the nearest goal within the bound
bestd: estimated w-suboptimal node with minimum d
argminn∈open∧f(n)≤w·f(best
f)
d(n)
![Page 53: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/53.jpg)
EES Expansion Order
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 26 / 33
bestf : open node giving lower bound on costbest
f: open node giving estimated optimal cost
bestd: estimated w-suboptimal node with minimum d
node to expand next:
1. pursue the shortest solution that is within the bound2.3.
in other words:
1. bestd
2.3.
![Page 54: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/54.jpg)
EES Expansion Order
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 26 / 33
bestf : open node giving lower bound on costbest
f: open node giving estimated optimal cost
bestd: estimated w-suboptimal node with minimum d
node to expand next:
1. pursue the shortest solution that is within the bound2.3.
in other words:
1. if f(bestd) ≤ w · f(bestf ) then best
d
2.3.
![Page 55: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/55.jpg)
EES Expansion Order
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 26 / 33
bestf : open node giving lower bound on costbest
f: open node giving estimated optimal cost
bestd: estimated w-suboptimal node with minimum d
node to expand next:
1. pursue the shortest solution that is within the bound2. pursue the estimated optimal solution3.
in other words:
1. if f(bestd) ≤ w · f(bestf ) then best
d
2. else if f(bestf) ≤ w · f(bestf ) then best
f
3.
![Page 56: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/56.jpg)
EES Expansion Order
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 26 / 33
bestf : open node giving lower bound on costbest
f: open node giving estimated optimal cost
bestd: estimated w-suboptimal node with minimum d
node to expand next:
1. pursue the shortest solution that is within the bound2. pursue the estimated optimal solution3. raise the lower bound on optimal solution cost
in other words:
1. if f(bestd) ≤ w · f(bestf ) then best
d
2. else if f(bestf) ≤ w · f(bestf ) then best
f
3. else bestf
see paper for further justification
![Page 57: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/57.jpg)
EES Respects the Suboptimality Bound
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 27 / 33
how does f(n) ≤ w · f(bestf ) ensure the suboptimality bound?
![Page 58: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/58.jpg)
EES Respects the Suboptimality Bound
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 27 / 33
how does f(n) ≤ w · f(bestf ) ensure the suboptimality bound?
f(n) ≤ f(n) f(n) is a lower bound for n
f(n) ≤ w · f(bestf ) expansion criterionw · f(bestf ) ≤ w · f∗(opt) because f(bestf ) is a lower
bound for the entire problem
f(n) ≤ w · f∗(opt) suboptimality bound
![Page 59: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/59.jpg)
EES Performance
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 28 / 33
bounded suboptimal search:minimize time subject to
relative cost bound (factor of optimal)
![Page 60: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/60.jpg)
EES Performance
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 28 / 33
Dock Robot
Suboptimality2.41.6
total raw cpu tim
e300
200
100
A* eps
wA*
Optimistic
Skeptical
EES
EES Opt.
![Page 61: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/61.jpg)
EES Performance
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 28 / 33
Vacuum World
Suboptimality42
log10 total raw cpu tim
e
0.6
0
-0.6
A* eps
Optimistic
Skeptical
wA*
EES
EES Opt.
![Page 62: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/62.jpg)
EES Performance
Introduction
Planning as Search
Bounded Suboptimal
■ Intuition
■ Three Heuristics
■ EES
■ Expansion Order
■ Bound
■ EES Performance
Conclusion
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 28 / 33
Heavy Vacuum World
Suboptimality168
log10 total raw cpu tim
e
1
0
-1
wA*
Optimistic
Skeptical
A* eps
EES
EES Opt.
![Page 63: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/63.jpg)
Conclusion
Introduction
Planning as Search
Bounded Suboptimal
Conclusion
■ Summary
■ The AI Vision
■ Algs as Agents
■ UNH
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 29 / 33
![Page 64: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/64.jpg)
Summary
Introduction
Planning as Search
Bounded Suboptimal
Conclusion
■ Summary
■ The AI Vision
■ Algs as Agents
■ UNH
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 30 / 33
Who’s afraid of big bad NP-hardness?
bounded suboptimal planning
■ fast solving■ control over cost
a new generation of planning algorithms
■ beyond optimal■ estimates, in addition to lower bounds■ new sources of heuristic information■ principled, provable performance
![Page 65: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/65.jpg)
The AI Vision
Introduction
Planning as Search
Bounded Suboptimal
Conclusion
■ Summary
■ The AI Vision
■ Algs as Agents
■ UNH
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 31 / 33
world model
planner
search
agent
world
actions
sensing
![Page 66: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/66.jpg)
Search Algorithms as Agents
Introduction
Planning as Search
Bounded Suboptimal
Conclusion
■ Summary
■ The AI Vision
■ Algs as Agents
■ UNH
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 32 / 33
model of space
meta-reasoner
open list
search algorithm
domain
expand
g, h values
children,
![Page 67: Planning Algorithms: When Optimal Just Isn’t Good Enoughruml/papers/uml-2011-talk.pdf · 2011-11-15 · WheelerRuml(UNH) WhenOptimalJustIsn’tGoodEnough–1/33 Planning Algorithms:](https://reader034.vdocuments.us/reader034/viewer/2022042105/5e82f4acd5cd447f8c4f9ead/html5/thumbnails/67.jpg)
The University of New Hampshire
Introduction
Planning as Search
Bounded Suboptimal
Conclusion
■ Summary
■ The AI Vision
■ Algs as Agents
■ UNH
Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 33 / 33
tell your students to apply to grad school in CS at UNH!
■ friendly faculty■ funding■ individual attention■ beautiful campus■ low cost of living■ easy access to Boston,
White Mountains■ strong in AI, infoviz,
networking,bioinformatics