planning algorithms: when optimal just isn’t good enoughruml/papers/uml-2011-talk.pdf ·...

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 1 / 33

Planning Algorithms:When Optimal Just Isn’t Good Enough

Wheeler Ruml

Department of Computer Science

Joint work with the UNH AI Group. Funded by NSF and DARPA.

Where are the Walking Talking Robots?

Introduction

■ Robots?

■ Toddler Steps

■ An Agent

Planning as Search

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 2 / 33

Lt. Commander Data Lt. Sharon Valerii Zoe GreystoneStar Trek: TNG Battlestar Galactica Caprica

Toddler Steps

Introduction

■ Robots?

■ Toddler Steps

■ An Agent

Planning as Search

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 3 / 33

The Role of Planning

Introduction

■ Robots?

■ Toddler Steps

■ An Agent

Planning as Search

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 4 / 33

world model

planner

agent

world

actions

sensing

The Role of Planning

Introduction

■ Robots?

■ Toddler Steps

■ An Agent

Planning as Search

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 4 / 33

world model

planner

search

agent

world

actions

sensing

Planning as Heuristic Graph Search

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 5 / 33

Planning

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 6 / 33

Given:

■ current state of the world■ models of available actions

preconditions, effects, costs

■ desired state of the world (partially specified?)

Find:

■ cheapest plan

Graph Search

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 7 / 33

Given:

■ start state: an explicit node■ expand function: generates children and their costs

assume arc cost ≥ 0■ goal test: predicate on nodes

Find:

■ cheapest path to a goal node

current

next

action a b

Graph Search

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 7 / 33

Given:

■ start state: an explicit node■ expand function: generates children and their costs

assume arc cost ≥ 0■ goal test: predicate on nodes

Find:

■ cheapest path to a goal node

root

a

a

a b

b

a b

b

a

a b

b

a b

Graph Search

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 7 / 33

Given:

■ start state: an explicit node■ expand function: generates children and their costs

assume arc cost ≥ 0■ goal test: predicate on nodes

Find:

■ cheapest path to a goal node

root = 0

1

2

2 2

1

3 1

2

1

2 3

2

1 3

Example: Motion Planning

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 8 / 33

(S. LaValle)

The Problem: Exponential Growth

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 9 / 33

Ever wonder why no one draws trees more than 3 levels deep?

The Problem: Exponential Growth

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 9 / 33

Ever wonder why no one draws trees more than 3 levels deep?

number of actionsplan length

14100 > 1080

A Tale of Three Algorithms

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 10 / 33

1. Uniform Cost Search (Dijkstra, 1959)

2. A* Search (Hart, Nilsson, and Raphael, 1968)

3. Explicit Estimation Search (Thayer and Ruml, 2011)

Uniform Cost Search (Dijkstra, 1959)

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 11 / 33

Uniform Cost Search (Dijkstra, 1959)

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 11 / 33

Explore nodes in increasing order of cost-so-far (g(n)):

Uniform Cost Search (Dijkstra, 1959)

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 11 / 33

Explore nodes in increasing order of cost-so-far (g(n)):

Q← ordered list containing the initial stateLoop

If Q is empty, return failureNode ← pop cheapest node off Q

If Node is a goal, return it (or path to it)Children ← Expand(Node).Merge Children into Q, keeping sorted by g(n).

1

2

2 2

1

3 1

2

1

2 3

2

1 3

Dijkstra Behavior

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 12 / 33

Dijkstra Behavior

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 12 / 33

Dijkstra Behavior

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 12 / 33

Dijkstra: Pathfinding in Warcraft

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 13 / 33

Problem-Specific Background Knowledge

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 14 / 33

Lower Bounds on Cost-to-go

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 15 / 33

h(n) < f*(n)

Heuristic Search: A* (Hart, Nilsson, and Raphael, 1968)

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 16 / 33

Heuristic Search: A* (Hart, Nilsson, and Raphael, 1968)

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 16 / 33

Cost should include both cost-so-far and cost-to-go:

Heuristic Search: A* (Hart, Nilsson, and Raphael, 1968)

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 16 / 33

Cost should include both cost-so-far and cost-to-go:

Q← ordered list containing the initial stateLoop

If Q is empty, return failureNode ← pop cheapest node off Q

If Node is a goal, return it (or path to it)Children ← Expand(Node)Merge Children into Q, keeping sorted by f(n)

f(n) = g(n) + h(n)g(n) = cost incurred so farh(n) = lower bound on cost to goal

A* Behavior

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 17 / 33

A* Behavior

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 17 / 33

A* Behavior

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 17 / 33

Dijkstra vs A*: Pathfinding in Warcraft

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 18 / 33

Dijkstra vs A*: Pathfinding in Warcraft

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 18 / 33

Quick Note: A Lower Bound on Solution Cost

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 19 / 33

f(n) = g(n) + h(n)

g(n) is actual cost-so-far, sowhen h(n) is a lower bound on cost-to-go,f(n) is a lower bound on final solution cost

lowest f(n) on frontier gives lower bound for entire problem!

bestf = argminn∈open

f(n)

Problems with A*

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33

Problems with A*

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33

A* takes exponential memory

Problems with A*

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33

A* takes exponential memory

Can sometimes be fixed: see ‘iterative deepening’

Problems with A*

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33

A* takes exponential memory

Can sometimes be fixed: see ‘iterative deepening’

A* takes exponential time

Problems with A*

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 20 / 33

A* takes exponential memory

Can sometimes be fixed: see ‘iterative deepening’

A* takes exponential time

We must trade cost for time.

A New Generation of Problem Settings

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 21 / 33

optimal: minimize solution costmust expand all with f(n) < f∗(opt)

A New Generation of Problem Settings

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 21 / 33

optimal: minimize solution costmust expand all with f(n) < f∗(opt)

greedy: minimize solving time

bounded suboptimal: minimize time subject to relative costbound (factor of optimal)

bounded cost: minimize time subject to absolute cost bound

contract: minimize cost subject to absolute time bound

utility function: maximize utility function of cost and timeeg, goal achievement time =

plan makespan + search time

A New Generation of Problem Settings

Introduction

Planning as Search

■ Planning

■ Graph Search

■ Example

■ The Problem

■ 3 Algorithms

■ Uniform Search

■ Dijkstra Behavior

■ Warcraft

■ Knowledge

■ Lower Bounds

■ Heuristic Search

■ A* Behavior

■ Dijkstra vs A*

■ Lower Bound

■ Problems with A*

■ Problem Settings

Bounded Suboptimal

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 21 / 33

optimal: minimize solution costmust expand all with f(n) < f∗(opt)

greedy: minimize solving time

bounded suboptimal: minimize time subject to relative costbound (factor of optimal)

bounded cost: minimize time subject to absolute cost bound

contract: minimize cost subject to absolute time bound

utility function: maximize utility function of cost and timeeg, goal achievement time =

plan makespan + search time

Bounded Suboptimal Search

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 22 / 33

The Intuition Behind EES (Thayer and Ruml, 2011)

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 23 / 33

■ unbiased estimates can be more informed than lower bounds■ nearest goal is the easiest to find

The Intuition Behind EES (Thayer and Ruml, 2011)

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 23 / 33

■ unbiased estimates can be more informed than lower bounds■ nearest goal is the easiest to find

Heavy Vacuum

Size302010

total ra

w cpu tim

e

400

200

admiss. h

inadmiss. h

The Intuition Behind EES (Thayer and Ruml, 2011)

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 23 / 33

■ unbiased estimates can be more informed than lower bounds■ nearest goal is the easiest to find

Heavy Vacuum

Size302010

total ra

w cpu tim

e

400

200

admiss. h

inadmiss. h

d

The Intuition Behind EES (Thayer and Ruml, 2011)

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 23 / 33

■ unbiased estimates can be more informed than lower bounds■ nearest goal is the easiest to find

Explicit Estimation Search (EES) combines these ideas

to minimize solving time subject to cost ≤ w·optimal:

pursue the nearest goal within the bound

need more information than just lower bound on cost (h(n))!

Three Heuristic Sources of Information

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 24 / 33

1. h: a lower bound on cost-to-gof(n) = g(n) + h(n)the traditional optimal A* lower bound

Three Heuristic Sources of Information

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 24 / 33

1. h: a lower bound on cost-to-gof(n) = g(n) + h(n)the traditional optimal A* lower bound

2. h: an estimate of cost-to-gounbiased estimates can be more informedf(n) = g(n) + h(n)(Thayer and Ruml, ICAPS-11)

Three Heuristic Sources of Information

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 24 / 33

1. h: a lower bound on cost-to-gof(n) = g(n) + h(n)the traditional optimal A* lower bound

2. h: an estimate of cost-to-gounbiased estimates can be more informedf(n) = g(n) + h(n)(Thayer and Ruml, ICAPS-11)

3. d: an estimate of distance-to-gonearest goal is the easiest to find(Pearl and Kim, IEEE PAMI 1982,Thayer et al, ICAPS-09)

Three Heuristic Sources of Information

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 24 / 33

1. h: a lower bound on cost-to-gof(n) = g(n) + h(n)the traditional optimal A* lower bound

2. h: an estimate of cost-to-gounbiased estimates can be more informedf(n) = g(n) + h(n)(Thayer and Ruml, ICAPS-11)

3. d: an estimate of distance-to-gonearest goal is the easiest to find(Pearl and Kim, IEEE PAMI 1982,Thayer et al, ICAPS-09)

pursue the nearest goal within the bound

Finding bestd

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 25 / 33

bestf : open node giving lower bound on cost

argminn∈open

f(n)

Finding bestd

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 25 / 33

bestf : open node giving lower bound on cost

argminn∈open

f(n)

bestf: open node giving estimated optimal cost

argminn∈open

f(n)

Finding bestd

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 25 / 33

bestf : open node giving lower bound on cost

argminn∈open

f(n)

bestf: open node giving estimated optimal cost

argminn∈open

f(n)

pursue the nearest goal within the bound

bestd: estimated w-suboptimal node with minimum d

argminn∈open∧f(n)≤w·f(best

f)

d(n)

EES Expansion Order

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 26 / 33

bestf : open node giving lower bound on costbest

f: open node giving estimated optimal cost

bestd: estimated w-suboptimal node with minimum d

node to expand next:

1. pursue the shortest solution that is within the bound2.3.

in other words:

1. bestd

2.3.

EES Expansion Order

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 26 / 33

bestf : open node giving lower bound on costbest

f: open node giving estimated optimal cost

bestd: estimated w-suboptimal node with minimum d

node to expand next:

1. pursue the shortest solution that is within the bound2.3.

in other words:

1. if f(bestd) ≤ w · f(bestf ) then best

d

2.3.

EES Expansion Order

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 26 / 33

bestf : open node giving lower bound on costbest

f: open node giving estimated optimal cost

bestd: estimated w-suboptimal node with minimum d

node to expand next:

1. pursue the shortest solution that is within the bound2. pursue the estimated optimal solution3.

in other words:

1. if f(bestd) ≤ w · f(bestf ) then best

d

2. else if f(bestf) ≤ w · f(bestf ) then best

f

3.

EES Expansion Order

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 26 / 33

bestf : open node giving lower bound on costbest

f: open node giving estimated optimal cost

bestd: estimated w-suboptimal node with minimum d

node to expand next:

1. pursue the shortest solution that is within the bound2. pursue the estimated optimal solution3. raise the lower bound on optimal solution cost

in other words:

1. if f(bestd) ≤ w · f(bestf ) then best

d

2. else if f(bestf) ≤ w · f(bestf ) then best

f

3. else bestf

see paper for further justification

EES Respects the Suboptimality Bound

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 27 / 33

how does f(n) ≤ w · f(bestf ) ensure the suboptimality bound?

EES Respects the Suboptimality Bound

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 27 / 33

how does f(n) ≤ w · f(bestf ) ensure the suboptimality bound?

f(n) ≤ f(n) f(n) is a lower bound for n

f(n) ≤ w · f(bestf ) expansion criterionw · f(bestf ) ≤ w · f∗(opt) because f(bestf ) is a lower

bound for the entire problem

f(n) ≤ w · f∗(opt) suboptimality bound

EES Performance

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 28 / 33

bounded suboptimal search:minimize time subject to

relative cost bound (factor of optimal)

EES Performance

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 28 / 33

Dock Robot

Suboptimality2.41.6

total raw cpu tim

e300

200

100

A* eps

wA*

Optimistic

Skeptical

EES

EES Opt.

EES Performance

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 28 / 33

Vacuum World

Suboptimality42

log10 total raw cpu tim

e

0.6

0

-0.6

A* eps

Optimistic

Skeptical

wA*

EES

EES Opt.

EES Performance

Introduction

Planning as Search

Bounded Suboptimal

■ Intuition

■ Three Heuristics

■ EES

■ Expansion Order

■ Bound

■ EES Performance

Conclusion

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 28 / 33

Heavy Vacuum World

Suboptimality168

log10 total raw cpu tim

e

1

0

-1

wA*

Optimistic

Skeptical

A* eps

EES

EES Opt.

Conclusion

Introduction

Planning as Search

Bounded Suboptimal

Conclusion

■ Summary

■ The AI Vision

■ Algs as Agents

■ UNH

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 29 / 33

Summary

Introduction

Planning as Search

Bounded Suboptimal

Conclusion

■ Summary

■ The AI Vision

■ Algs as Agents

■ UNH

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 30 / 33

Who’s afraid of big bad NP-hardness?

bounded suboptimal planning

■ fast solving■ control over cost

a new generation of planning algorithms

■ beyond optimal■ estimates, in addition to lower bounds■ new sources of heuristic information■ principled, provable performance

The AI Vision

Introduction

Planning as Search

Bounded Suboptimal

Conclusion

■ Summary

■ The AI Vision

■ Algs as Agents

■ UNH

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 31 / 33

world model

planner

search

agent

world

actions

sensing

Search Algorithms as Agents

Introduction

Planning as Search

Bounded Suboptimal

Conclusion

■ Summary

■ The AI Vision

■ Algs as Agents

■ UNH

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 32 / 33

model of space

meta-reasoner

open list

search algorithm

domain

expand

g, h values

children,

The University of New Hampshire

Introduction

Planning as Search

Bounded Suboptimal

Conclusion

■ Summary

■ The AI Vision

■ Algs as Agents

■ UNH

Wheeler Ruml (UNH) When Optimal Just Isn’t Good Enough – 33 / 33

tell your students to apply to grad school in CS at UNH!

■ friendly faculty■ funding■ individual attention■ beautiful campus■ low cost of living■ easy access to Boston,

White Mountains■ strong in AI, infoviz,

networking,bioinformatics