Download - P l a nn i ng (A I M A Ch. 10 )
Planning (AIMA Ch. 10)
1
• Planning problem defined
• Simple planning agent• Types of plans• Graph planning
What we have so far
2
• Can TELL KB about new percepts about the world
• KB maintains model of the current world state
• Can ASK KB about any fact that can be inferred from KB
How can we use these components to build a planning agent?
i.e., an agent that constructs plans that can achieve its goals, and that then executes these plans?
Planning Problem
3
• Find a sequence of actions that achieves a given goal when executed from a given initial world state. That is, given◦a set of operator descriptions (defining the possible
primitive actions by the agent),◦an initial state description, and◦a goal state description or predicate,
• compute a plan, which is◦a sequence of operator instances, such that
executing them in the initial state will change the world to a state satisfying the goal-state description.
• Goals are usually specified as a conjunction of subgoals to be achieved
Planning vs. Problem Solving
4
• Planning and problem solving methods can often solve the same sorts of problems
• Planning is more powerful because of the representations and methods used
• States, goals, and actions are decomposed into sets of sentences (usually in first-order logic)
• Search often proceeds through plan space rather than state space (though there are also state-space planners)
• Subgoals can be planned independently, reducing the complexity of the planning problem
Remember: Problem-Solving Agent
5
function SIM PLE-PROBLEM -SOLV ING-A GENT ( percept) ret urns an actionst at ic: seq, an action sequence, init ial ly empt y
state, some descript ion of t he cur rent wor ld st at egoal, a goal, init ial ly nullproblem, a problem formulat ionstate f.. U PDA T E-ST A T E(state,
percept)if seq is empt y t hen
goal f.. FORM ULA T E-GOA L(state)
// What is the current state?
problem f.. FORM U LA T E-PROB LEM (state, goal) seq f.. SEA RCH ( problem)
action f.. RECOM M EN D A T I ON (seq, state)seq f.. REM A I N D ER (seq, state)ret urn action
Note: This is offline problem-solving. Online problem-solving involves acting w/o complete knowledge of the problem and environment
// From LA to San Diego (given curr. state)
// e.g., Gas usage
// If fails to reach goal, update
Simple planning agent
6
• Use percepts to build model of current world state
• IDEAL-PLANNER: Given a goal, algorithm generates plan of action
• STATE-DESCRIPTION: given percept, return initial state description in format required by planner
• MAKE-GOAL-QUERY: used to ask KB what next goal should be
A Simple Planning Agent
7
function SIMPLE-PLANNING-AGENT(percept) returns an actionstatic: KB, a knowledge base (includes action descriptions)
p, a plan (initially, NoPlan)t, a time counter (initially 0)
local variables:G, a goalcurrent, a current state description
TELL(KB, MAKE-PERCEPT-SENTENCE(percept, t)) current STATE-DESCRIPTION(KB, t)if p = NoPlan then
G ASK(KB, MAKE-GOAL-QUERY(t))p IDEAL-PLANNER(current, G, KB)
if p = NoPlan or p is empty thenaction NoOp
elseaction FIRST(p) p REST(p)
TELL(KB, MAKE-ACTION-SENTENCE(action, t))t t+1return action
Like popping from a stack
Goal of Planning
8
• Choose actions to achieve a certain goal• But isn’t it exactly the same goal as for
problem solving?• Some difficulties with problem solving:
◦The successor function is a black box: it must be “applied” to a state to know which actions are possible in that state and what are the effects of each one
Goal of Planning
9
• Choose actions to achieve a certain goalut isn’t it exactly the same goal as for oblem solving?
ome difficulties with problem solving:
• B • Suppose that the goal is HAVE(milk).prthe successor function must be repeatedly applied
to• S eventually generate a state where HAVE(milk) is satisfied.
• From some initial state where HAVE(milk) is not satisfied,
• An explicit representation of the possible actions and◦ their effects would help the problem solver select
the relevant actions.possible in that state and what are the effects ofeac
h oneOtherwise, in the real world an agent would be overwhelmed by irrelevant actions
Goal of Planning
• Choose actions to achieve a certain goal• But isn’t it exactly the same goal as for
problem solving?• Some difficulties with problem solving:
◦The goal test is another black-box function, states are domain-specific data structures, and heuristics must be supplied for each new problem
10
Goal of Planning
• Choose actions to achieve a certain goal• But isn’t it exactly the same goal as for
problem solving?• Some difficulties with problem solving:
◦The goal test is another black-box function, states are domain-specific data structures, and heuristics must be supplied for each new problem
11
• Suppose that the goal is HAVE(milk) ^ HAVE(book).• Without an explicit representation of the goal, theproblem solver cannot know that a state where HAVE(milk)is already achieved is more promising than a state whereneither HAVE(milk) nor HAVE(book) is achieved.
Goal of Planning
• Choose actions to achieve a certain goal• But isn’t it exactly the same goal as for
problem solving?• Some difficulties with problem solving:
◦The goal may consist of several nearly independent subgoals, but there is no way for the problem solver to know it
12
Goal of Planning
13
• Choose actions to achieve a certain goal• But isn’t it exactly the same goal as for
pro• So
blem solving?me difficulties with problem solving:
HAVE(milk) and HAVE(book) may be achieved by two nearly independent
sequences of actions
◦The goal may consist of several nearly independent subgoals, but there is no way for the problem solver to know it
Representations in Planning
14
Planning opens up the black boxes by usinglogic to represent:• Actions• States• Goals
Problem solving Logic representation
Planning
How to represent Planning Problem
• Classical planning: we will start with problems that are full observable, deterministic, statics environment with single agents
15
PDDL
PDDL = Planning Domain Definition Language ← standard encoding language for “classical” planning tasksComponents of a PDDL planning task:• Objects: Things in the world that interest us.• Predicates: Properties of objects that we are interested in; can be
true or false.• Initial state: The state of the world that we start in.• Goal specification: Things that we want to be true.• Actions/Operators: Ways of changing the state of the world.PDDL can describe what we need for search algorithm: 1) initial state 2) actions that are available in a state 3) result of applying an action 4) and goal test
16
PDDL operators
States: Represented as a conjunction of fluent (ground, functionless atoms).Following are not allowed: At(x, y) (non-ground), ¬ Poor (negation), and At(Father(Fred), Sydney) (use function symbol)
• Use closed-world assumption: Any fluent that are not mentioned are false.• Use unique names assumption: States named differently are distinct.
*Fluents: Predicates and functions whose values vary from by time (situation). *Ground term: a term with no variables. *Literal: atomic sentence.Initial state is conjunction of ground atoms.Goal is also a conjunction ( ) of literal (positive or negative) that may ∧contain variables.
17
PDDL operators cont.
• Actions: described by a set of action schemas that implicitly define the ACTIONS(s) and RESULT(s, a).
• Action schema: composed of 1) action name, 2) list of all the variable used in the schema, 3)a precondition, and 4) an effect
• Action(Fly(p, from, to),PRECOND: At(p, from) Plane(p) Airport(from) Airport(to)∧ ∧ ∧EFFECT: ¬ At(p, from) At(p, to)∧
• Value substituted for the variables: Action(Fly(P1, SFO, JFK),PRECOND: At(P1, SFO) Plane(P1) Airport(SFO) Airport(JFK)∧ ∧ ∧EFFECT: ¬ At(P1, SFO) At(p, JFK)∧
Action a is applicable in state s if the preconditions are satisfied by s. *preconditions: Condition which makes an action possible.
18
PDDL operators cont.
• Result of executing action a in state s is defined as a state s which is ′represented by the set of fluents formed by starting with s, removing the fluents what appear as negative literals in the action’s effects.
• For example: in Fly(P1, SFO, JFK) we remove At(P1, SFO) and add At(P1, JFK).
a Actions(s) s |= Precond(a)∈ ⇔Result(s, a) = (s − Del(a)) Add(a)∪
where Del(a) is the list of literals which appear negatively in the effect of a, and Add(a) is the list of positive literals in the effect of a.• * The precondition always refers to time t and the effect to time t + 1.
19
PDDL: Cargo transportation planning problem
20
Major approaches
21
• Planning graphs• State space planning• Situation calculus• Partial order planning• Hierarchical decomposition (HTN planning)• Reactive planning
Basic representations for planning
22
• Classic approach first used in STRIPS planner circa 1970• States represented as a conjunction of ground literals
◦ at(Home)• Goals are conjunctions of literals, but may have
existentially quantified variables◦ at(?x) ^ have(Milk) ^ have(bananas) ...
• Do not need to fully specify state◦ Non-specified either don’t-care or assumed false◦ Represent many cases in small storage◦ Often only represent changes in state rather than entire
situation• Unlike theorem prover, not seeking whether the goal is
true, but is there a sequence of actions to attain it
C
A B
Start State
On(C, A)On(A, Table) On(B, Table) Clear(C) Clear(B)
Sparse encoding, but
complete state spec
A
C
B
Goal
Planning Problems
23
Which goal first?
ACTION: MoveToTable(b,x)PRECONDITIONS: On(b,x), Clear(b), Block(b), Block(x), (bx) POSTCONDITIONS: On(b,Table), Clear(x)
On(b,x)
Action schema, instantiates to give
specific ground actions
On(B, C)On(A, B)
Set of goal states, only requirements
specified (think unary constraints)
Operator/action representation
24
• Operators contain three components:◦ Action description◦ Precondition - conjunction of positive literals◦ Effect - conjunction of positive or negative literals which
describe how situation changes when operator is applied• Example:
Op[Action: Go(there),
Precond: At(here) ^ Path(here,there), Effect: At(there) ^ ~At(here)]
• All variables are universally quantified
• Situation variables are implicit
At(here) Path(here,there)
Go(there)At(there) ~At(here)
◦ preconditions must be true in the state immediately before operator is applied; effects are true immediately after
C
A B
Start State
Sequential Plan
MoveToTable(C,A) > Move(B,Table,C) > Move(A,Table,B)
Types of Plans
25
On(C, A)On(A, Table) On(B, Table) Clear(C) Clear(B) Block(A)…
Partial-Order Plan
MoveToTable(C,A)> Move(A,Table,B)
Move(B,Table,C)
Forward Search
Forward (progression) state-space search: ♦ Search through the space of states, starting in the initial
state and using the problem’s actions to search forward for a member of the set of goal states.
♦ Prone to explore irrelevant actions ♦ Planning problems often have large state space. ♦ The state-space is concrete,
i.e. there is no unassigned variables in the states.
26
C
A B
Start State
On(C, A)On(A, Table)On(B, Table)Clear(C)Clear(B)Block(A)
Forward Search
27
A CB
MoveToBlock(C,A,B)
Applicable actions
Backward Search
• Backward (regression) relevant-state search: ♦ Search through set of relevant states, staring at the set of
states representing the goal and using the inverse of the actions to search backward for the initial state.
♦ Only considers actions that are relevant to the goal (or current state).
♦Works only when we know how to regress from a state description to the predecessor state description.
♦ Need to deal with partially instantiated actions and state, not just ground ones.
(Some variable may not be assigned values.)
28
A
C
B
Goal State
MoveToBlock(A,Table,B) MoveToBlock(A,x’,B)
ACTION: MoveToBlock(b,x,y) PRECONDITIONS: On(b,x), Clear(b), Clear(y),
Block(b), Block(y), (bx), (by), (xy) POSTCONDITIONS: On(b,y), Clear(x)
On(b,x), Clear(y)
Backward Search
29
A C
B?
On(B, C)On(A, B)Relevant
actions
Start:HaveCake
Goal: AteCake, HaveCake
Action: Eat Pre: HaveCake Add: AteCake
Have=T, Ate=F
Have=F, Ate=T
{Eat} {}
Have=T, Ate=F
Planning “Tree”
30
Del: HaveCake
Action: Bake Pre: HaveCake Add: HaveCake
Have=T, Ate=T
{Bake} {}
Have=F, Ate=T
Have=F, Ate=T
{Eat}
Have=T, Ate=F
{}
Have=T, Ate=F
{Eat} {}
Have=T, Ate=F
Reachable State Sets
31
Have=F, Ate=T
Have=T Ate=F
Have=T, Ate=T
{Bake} {}
Have=F, Ate=T
Have=F, Ate=T
{Eat}
Have=T, Ate=F
{}
Have=T, Ate=F
Have=F, Ate=T
Have=T, Ate=T
Have=F, Ate=T
Have=T, Ate=F
Have=T, Ate=F
Have={T}, Ate={F}
Approximate Reachable Sets
32
Have=T, Ate=F
Have=F, Ate=T
Have=T, Ate=T
Have=F, Ate=T
Have=T, Ate=F
Have={T,F},
Ate={T,F}
Have={T,F},Ate={T,F}
(Have, Ate) not (T,T)(Have, Ate) not (F,F)
(Have,Ate) not (F,F)
Planning Graphs
33
Start:HaveCake
Goal: AteCake, HaveCake
Action: EatPre: HaveCake Add: AteCake Del: HaveCake
HaveCake
Eat
HaveCake
HaveCake
Action: BakePre: HaveCake Add: HaveCake
S0
AteCake AteCake
AteCake
A0 S1
Mutual Exclusion (Mutex)
34
HaveCake
Eat
HaveCake
HaveCake
NEGATIONLiterals and their negations can’t be true at the
S0
AteCake AteCake
AteCake
A0 S1
same time
P
P
Mutual Exclusion (Mutex)
35
INCONSISTENT EFFECTSAn effect of one negates the effect of the other
HaveCake
Eat
HaveCake
HaveCake
S0
AteCake AteCake
AteCake
A0 S1
Mutual Exclusion (Mutex)
36
INCONSISTENT SUPPORTAll pairs of actions that achieve two literals are mutex
HaveCake
Eat
HaveCake
HaveCake
S0
AteCake AteCake
AteCake
A0 S1
Planning Graph
37
Eat
HaveCake
HaveCake
Bake
HaveCake
Eat
HaveCake
HaveCake
AteCake
AteCake
A1 S2S0
AteCake AteCake
AteCake
A0 S1
Mutual Exclusion (Mutex)
38
COMPETITIONPreconditions are mutex; cannot both hold
INCONSISTENT EFFECTSAn effect of one negates the effect of the other
Mutual Exclusion (Mutex)
39
INTERFERENCEOne deletes a precondition of the other
Planning Graph
40
Observation 1
41
Observation 2
42
Actions monotonically increase
(if they applied before, they still do)
Observation 3
43
p
q
rA
p
q
r
p
q
r
Proposition mutex relationships monotonically decrease
… … …
Observation 4
44
p
q
…B
p
q
r
p
q
r
A
B
A
p
q
rB
A
Action mutex relationships monotonically decrease
s
…
s
…
C C s
…
C
Observation 5
45
• Claim: planning graph “levels off”◦ After some time k all levels are identical◦ Because it’s a finite space, the set of literals
cannot increase indefinitely, nor can the mutexes decrease indefinitely
• Claim: if goal literal never appears, or goal literals never become non-mutex, no plan exists◦ If a plan existed, it would eventually achieve all goal literals
(and remove goal mutexes – less obvious)◦ Converse not true: goal literals all appearing non-mutex
does not imply a plan exists
• Relax problem by ignoring preconditions
• Can drop all or just some preconditions• Can solve in closed form or with set-cover methods
Heuristics: Ignore Preconditions
46
• Relax problem by not deleting falsified literals
• Can’t undo progress, so solve with hill-climbing (non-admissible)
COn(C, A)On(A, Table) On(B, Table)
Heuristics: No-Delete
47
A BClear(C) Clear(B)
ACTION: MoveToBlock(b,x,y) PRECONDITIONS: On(b,x), Clear(b), Clear(y),
Block(b), Block(y), (bx), (by), (xy) POSTCONDITIONS: On(b,y), Clear(x)
On(b,x), Clear(y)
• Independent subgoals?• Partition goal literals
• Find plans for each subset
• cost(all) < cost(any)?
• cost(all) < sum-cost(each)?
A
C
B
Goal State
Heuristics: Independent Goals
48
On(B, C)On(A, B)
On(A, B) On(B, C)
Heuristics: Level Costs
49
S1
• Planning graphs enable powerful heuristics◦ Level cost of a literal is the smallest S in which it appears◦ Max-level: goal cannot be realized before largest goal
conjunct level cost (admissible)◦ Sum-level: if subgoals are independent, goal cannot be
realized faster than the sum of goal conjunct level costs (not admissible)
◦ Set-level: goal cannot be realized before all conjuncts are non-mutex (admissible)
Bake
HaveCake HaveCake HaveCake
Eat HaveCake Eat HaveCake
AteCake AteCake
AteCake
A1 S2
AteCake
S0
AteCake
A0
Graphplan
50
• Graphplan directly extracts plans from a planning graph
• Graphplan searches for layered plans (often called parallel plans)◦ More general than totally-ordered plans, less general than
partially-ordered plans• A layered plan is a sequence of sets of actions
◦ actions in the same set must be compatible◦ all sequential orderings of compatible actions gives same
resultA C ? D
B D A C B
move(A,B,TABLE) move(C,D,TABLE)
move(B,TABLE,A) move(D,TABLE,C);
Layered Plan: (a two layer plan)
Search problem:Start state: goal set at last level Actions: conflict-free ways of
achieving the current goal set Terminal test: at S0 with goal set
entailed by initial planning state
Note: may need to start much deeper than the leveling-off point!
Caching, good ordering is important
Solution Extraction: Backward Search
51
An example domain: the blocks world
The domain consists of:1. A table, a set of cubic blocks and a robot arm;2. Each block is either on the table or stacked on top of
another block3. The arm can pick up a block and move it to another
position either on the table or on top of another block;4. The arm can only pick up one block at time, so it
cannot pick up a block which has another block on top.
A goal is a request to build one or more stacks of blocks specified in terms of what blocks are on top of what other blocks
52
State descriptions
•Blocks are represented by constants A, B, C, . . . etc. and an additional constant T able representing the table.
•The following predicates are used to describe states:
On(b, x) block b is on x, where x is either another block or the table
Clear (x) there is a clear space on x to hold a block
53
PDDL: Block-world problem
54
Start State Goal State
C
B A
Initial state:On(C, A) ∧ OnT able(A) ∧ OnT able(B ) ∧ Clear (B ) ∧ Clear (C )
Goal:On(A, B ) ∧ On(B , C ) ∧ OnT able(C )
A
B
C
Actions schemas
• Actions schemas:MOVE(b, x, y):
Precond: On(b, x) Clear(b) Clear(y)∧ ∧Effect: On(b, y) Clear(x)∧ ∧
¬ On(b, x)∧ ¬ Clear(y)MOVE-TO-TABLE(b, x):
Precond: On(b, x) Clear(b)∧Effect: On(b, Table) Clear(x) ∧ ∧
¬ On(b, x)
55
PDDL: Block-world problem
56
Start State Goal State
C
B A
Initial state:On(C, A) ∧ OnT able(A) ∧ OnT able(B ) ∧ Clear (B ) ∧ Clear (C )
Goal:On(A, B ) ∧ On(B , C ) ∧ OnT able(C )
A
B
C
Goal stack planning
• We work backwards from the goal, looking for an operator which has one or more of the goal literals as one of its effects and then trying to satisfy the preconditions of the operator. The preconditions of the operator become sub-goals that must be satisfied. We keep doing this until we reach the initial state.
• Goal stack planning uses a stack to hold goals and actions to satisfy the goals, and a knowledge base to hold the current state, action schemas and domain axioms
• Goal stack is like a node in a search tree; if there is a choice of action, we create branches
57
Goal stack planning pseudo-code
• Push the original goal on the stack. Repeat until the stack is empty:• If stack top is a compound goal, push its unsatisfied sub-goals on the
stack.• If stack top is a single unsatisfied goal, replace it by an action that
makes it satisfied and push the actions precondition on the stack.• If stack top is an action, pop it from the stack, execute it and change
the knowledge base by the actions effects.• If stack top is a satisfied goal, pop it from the stack.
58
Goal stack planning 1
• (We do pushing the sub-goals at the same step as the compound goal.) The order of sub-goals is up to us; we could have put On(B,C) on top
On(A,B)On(B,C)OnTable(C)On(A,B) On(B,C) OnTable(C)∧ ∧
KB = {On(C,A),OnTable(A),OnTable(B),Clear(B), Clear(C)}plan = [ ]The top of the stack is a single unsatisfied goal. We push the action which would achieve it, and its preconditions.
59
Goal stack planning 2
On(A, Table)Clear(B)Clear(A)On(A, Table) Clear(A) Clear(B)∧ ∧MOVE(A, x,B)On(A,B)On(B,C)OnTable(C)On(A,B) On(B,C) OnTable(C)∧ ∧
KB = {On(C,A),OnTable(A),OnTable(B),Clear(B), Clear(C)}plan = [ ]The top of the stack is a satisfied goal. We pop the stack (twice).
60
Goal stack planning 3
Clear(A)On(A, Table) Clear(A) Clear(B)∧ ∧MOVE(A, Table,B)On(A,B)On(B,C)OnTable(C)On(A,B) On(B,C) OnTable(C)∧ ∧
KB = {On(C,A),OnTable(A),OnTable(B),Clear(B), Clear(C)}plan = [ ]
The top of the stack is an unsatisfied goal. We push the action which wouldachieve it, and its preconditions. 61
Goal stack planning 4
On(C,A)Clear(C)On(C,A) Clear(C)∧MOVE-TO-TABLE(C,A)Clear(A)On(A, Table) Clear(A) Clear(B)∧ ∧MOVE(A, Table,B)On(A,B)On(B,C)OnTable(C)On(A,B) On(B,C) OnTable(C)∧ ∧
KB = {On(C,A),OnTable(A),OnTable(B),Clear(B), Clear(C)}plan = [ ]
The top of the stack is a satisfied goal. We pop the stack (three times).62
Goal stack planning 5
MOVE-TO-TABLE(C,A)Clear(A)On(A, Table) Clear(A) Clear(B)∧ ∧MOVE(A, Table,B)On(A,B)On(B,C)OnTable(C)On(A,B) On(B,C) OnTable(C)∧ ∧
KB = {On(C,A),OnTable(A),OnTable(B),Clear(B), Clear(C)}plan = [ ]
The top of the stack is an action. We execute it, update the KB with itseffects, and add it to the plan. 63
Goal stack planning 6
Clear(A)On(A, Table) Clear(A) Clear(B)∧ ∧MOVE(A, Table,B)On(A,B)On(B,C)OnTable(C)On(A,B) On(B,C) OnTable(C)∧ ∧
KB = {OnTable(C),OnTable(A),OnTable(B),Clear(A), Clear(B), Clear(C)}plan = [MOVE-TO-TABLE(C,A)]
The top of the stack is a satisfied goal. We pop the stack (twice).
64
Goal stack planning 7
MOVE(A, Table,B)On(A,B)On(B,C)OnTable(C)On(A,B) On(B,C) OnTable(C)∧ ∧
KB = {OnTable(C),OnTable(A),OnTable(B),Clear(A), Clear(B), Clear(C)}plan = [MOVE-TO-TABLE(C,A)]
The top of the stack is an action. We execute it, update the KB with itseffects, and add it to the plan.
65
Goal stack planning 8
On(A,B)On(B,C)OnTable(C)On(A,B) On(B,C) OnTable(C)∧ ∧
KB = {On(A,B), OnTable(C), OnTable(B), Clear(A), Clear(C)}plan = [MOVE-TO-TABLE(C,A),MOVE(A, Table,B)]the current state is
66
Goal stack planning 9
If we follow the same process for the On(B,C) goal, we end up in the state
On(B,C) OnTable(A) OnTable(C)∧ ∧The remaining goal on the stack,
On(A,B) On(B,C) OnTable(C)∧ ∧is not satisfied. So On(A,B) will be pushed on the stack again.
67
Goal stack planning 10
Now we finally can move A on top of B, but the resulting plan is redundant:
MOVE-TO-TABLE(C,A)MOVE(A, Table,B)MOVE-TO-TABLE(A,B)MOVE(B, Table,C)MOVE(A, Table,B)
There are techniques for ‘fixing’ inefficient plans (where something is done and then undone) but it is difficult in general (when it is not straight one after another)
68