INSTITUTE OF THEORETICAL INFORMATICS

Automated Planning and SchedulingLecture 10: Advanced problem models

Tomas Balyo, Dominik Schreiber | January 10, 2019

KIT – University of the State of Baden-Wuerttemberg and

National Laboratory of the Helmholtz Association

www.kit.edu

Outline

Advanced modeling and planning techniquesEnhancing our problem logic: Axioms, functions,numerical and temporal planningEnhancing our objectives: Complex metrics, preferences, constraints

Practical view: “Everything that exists in PDDL 3.1”

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 2/34

Problem models: What we know

Classical planning

STRIPS-style actions: preconditions + effects

Conditional effects

Equality predicates

ADL: Quantifications and disjunctive conditions

Action costs for cost-optimal planning

Extensions of classical planning

Hierarchical planning: Actions + Methods + Tasks

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 3/34

Enhancing our problem logic

From STRIPS to ADL: Logic describing the problem gainsmore expressive power

STRIPS: Pure “sets of literals” – Flat conjunctionsStructure of expressions: F ∈ {a, ¬a, F1 ∧ F2}(a: atom; F1, F2: expressions)

ADL (Action Description Language): Function-free First Order LogicF ∈ {a, ¬F1, F1 ∧ F2, F1 ∨ F2, ∀xF1(x), ∃xF1(x)}

Can we use even more powerful logics in planning?Logics with axiomsFirst Order Logic with functionsNon-classical logics: Temporal logic

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 4/34

Enhancing our problem logic

From STRIPS to ADL: Logic describing the problem gainsmore expressive power

STRIPS: Pure “sets of literals” – Flat conjunctionsStructure of expressions: F ∈ {a, ¬a, F1 ∧ F2}(a: atom; F1, F2: expressions)

ADL (Action Description Language): Function-free First Order LogicF ∈ {a, ¬F1, F1 ∧ F2, F1 ∨ F2, ∀xF1(x), ∃xF1(x)}

Can we use even more powerful logics in planning?Logics with axiomsFirst Order Logic with functionsNon-classical logics: Temporal logic

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 4/34

Axioms in Planning [THN05]

Navigation example: How to model reachability on a graph?

Given a predicate (connected ?x ?y) to denote edges in graph,define key predicate (reachable ?x ?y) for any node pair

Need actions to explicitly define p step by stepa1 := (direct-reach ?x ?y),pre(a1) = {(connected ?x ?y)},eff(a1) = {(reachable ?x ?y)}a2 := (indirect-reach ?x ?y ?z),pre(a2) = {(connected ?x ?z), (reachable ?z ?y)},eff(a2) = {(reachable ?x ?y)}

Idea: Derive value of (reachable ?x ?y)

directly from other atoms (using same logic as pre(a1) and pre(a2))

Add such derived predicates to domain definition as axioms

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 5/34

Axioms in Planning [THN05]

Navigation example: How to model reachability on a graph?

Given a predicate (connected ?x ?y) to denote edges in graph,define key predicate (reachable ?x ?y) for any node pairNeed actions to explicitly define p step by step

a1 := (direct-reach ?x ?y),pre(a1) = {(connected ?x ?y)},eff(a1) = {(reachable ?x ?y)}a2 := (indirect-reach ?x ?y ?z),pre(a2) = {(connected ?x ?z), (reachable ?z ?y)},eff(a2) = {(reachable ?x ?y)}

Idea: Derive value of (reachable ?x ?y)

directly from other atoms (using same logic as pre(a1) and pre(a2))

Add such derived predicates to domain definition as axioms

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 5/34

Axioms in Planning [THN05]

Navigation example: How to model reachability on a graph?

Given a predicate (connected ?x ?y) to denote edges in graph,define key predicate (reachable ?x ?y) for any node pairNeed actions to explicitly define p step by step

a1 := (direct-reach ?x ?y),pre(a1) = {(connected ?x ?y)},eff(a1) = {(reachable ?x ?y)}a2 := (indirect-reach ?x ?y ?z),pre(a2) = {(connected ?x ?z), (reachable ?z ?y)},eff(a2) = {(reachable ?x ?y)}

Idea: Derive value of (reachable ?x ?y)

directly from other atoms (using same logic as pre(a1) and pre(a2))

Add such derived predicates to domain definition as axioms

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 5/34

Derived Predicates in PDDL

Notation in PDDL domains:(:predicates (connected ?x ?y - node))

(:derived (reachable ?x ?y - node)

(or

; direct reach

(connected ?x ?y)

; indirect reach

(exists (?z - node) (and

(connected ?x ?z) (reachable ?z ?y)

))

)

)

Can then use (reachable ?x ?y) like a normal predicatein preconditions and goals (but not in effects)

Definitions may recurse and/or reference other derived predicates

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 6/34

Derived Predicates in PDDL

Notation in PDDL domains:(:predicates (connected ?x ?y - node))

(:derived (reachable ?x ?y - node)

(or

; direct reach

(connected ?x ?y)

; indirect reach

(exists (?z - node) (and

(connected ?x ?z) (reachable ?z ?y)

))

)

)

Can then use (reachable ?x ?y) like a normal predicatein preconditions and goals (but not in effects)

Definitions may recurse and/or reference other derived predicates

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 6/34

Axioms: Discussion

Good way to separate actual action effects from merelogical consequences; cleaner and more succinct actions

More than syntactic sugar: Compiling them away can lead toexponential blow-up of ground problem [THN05]

Semantics can become complicated in recursive derivationsDefinition must not be “cyclic”

Ways to handle axioms inside a planner:Replace derived predicates with helper actionsPreserve derived predicates, evaluate during planning

More information: [THN05]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 7/34

Axioms: Discussion

Good way to separate actual action effects from merelogical consequences; cleaner and more succinct actions

More than syntactic sugar: Compiling them away can lead toexponential blow-up of ground problem [THN05]Semantics can become complicated in recursive derivations

Definition must not be “cyclic”

Ways to handle axioms inside a planner:Replace derived predicates with helper actionsPreserve derived predicates, evaluate during planning

More information: [THN05]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 7/34

Axioms: Discussion

Good way to separate actual action effects from merelogical consequences; cleaner and more succinct actions

More than syntactic sugar: Compiling them away can lead toexponential blow-up of ground problem [THN05]Semantics can become complicated in recursive derivations

Definition must not be “cyclic”

Ways to handle axioms inside a planner:Replace derived predicates with helper actionsPreserve derived predicates, evaluate during planning

More information: [THN05]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 7/34

Intermission

All PDDL features which we covered up to now are (more or less)what the planning system Fast Downward supports.

This planner and its enhancements represent the state of the artin classical planning.

Next up: Non-classical planning models;numeric, temporal, and preference-based planning.

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 8/34

Functions in Planning Domains

Mainly two types of functions:Numeric fluents f : C∗ → R

Maps list of constants 〈c1, . . . , cn〉 to real number(can be different in each state! hence the name fluent)E.g. (:function (capacity ?t - truck) - number):Maps a truck object to its current package capacity

Special case n = 0: parameter-less, plain numeric variableUsed to build numeric preconditions and effects,important for cost-optimal planningE.g. used by Metric-FF, SHOP2 and temporal planners

Object fluents f : C∗ → CMaps set of constants to another constantE.g. (:function (neighbor-left ?c - cell) - cell):Maps a cell object to its left neighbor cellRaises expressiveness to Full First Order LogicUsed by very few planners[citation needed]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 9/34

Functions in Planning Domains

Mainly two types of functions:Numeric fluents f : C∗ → R

Maps list of constants 〈c1, . . . , cn〉 to real number(can be different in each state! hence the name fluent)E.g. (:function (capacity ?t - truck) - number):Maps a truck object to its current package capacitySpecial case n = 0: parameter-less, plain numeric variable

Used to build numeric preconditions and effects,important for cost-optimal planningE.g. used by Metric-FF, SHOP2 and temporal planners

Object fluents f : C∗ → CMaps set of constants to another constantE.g. (:function (neighbor-left ?c - cell) - cell):Maps a cell object to its left neighbor cellRaises expressiveness to Full First Order LogicUsed by very few planners[citation needed]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 9/34

Functions in Planning Domains

Mainly two types of functions:Numeric fluents f : C∗ → R

Maps list of constants 〈c1, . . . , cn〉 to real number(can be different in each state! hence the name fluent)E.g. (:function (capacity ?t - truck) - number):Maps a truck object to its current package capacitySpecial case n = 0: parameter-less, plain numeric variableUsed to build numeric preconditions and effects,important for cost-optimal planningE.g. used by Metric-FF, SHOP2 and temporal planners

Object fluents f : C∗ → CMaps set of constants to another constantE.g. (:function (neighbor-left ?c - cell) - cell):Maps a cell object to its left neighbor cellRaises expressiveness to Full First Order LogicUsed by very few planners[citation needed]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 9/34

Functions in Planning Domains

Mainly two types of functions:Numeric fluents f : C∗ → R

Maps list of constants 〈c1, . . . , cn〉 to real number(can be different in each state! hence the name fluent)E.g. (:function (capacity ?t - truck) - number):Maps a truck object to its current package capacitySpecial case n = 0: parameter-less, plain numeric variableUsed to build numeric preconditions and effects,important for cost-optimal planningE.g. used by Metric-FF, SHOP2 and temporal planners

Object fluents f : C∗ → CMaps set of constants to another constantE.g. (:function (neighbor-left ?c - cell) - cell):Maps a cell object to its left neighbor cell

Raises expressiveness to Full First Order LogicUsed by very few planners[citation needed]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 9/34

Functions in Planning Domains

Mainly two types of functions:Numeric fluents f : C∗ → R

Maps list of constants 〈c1, . . . , cn〉 to real number(can be different in each state! hence the name fluent)E.g. (:function (capacity ?t - truck) - number):Maps a truck object to its current package capacitySpecial case n = 0: parameter-less, plain numeric variableUsed to build numeric preconditions and effects,important for cost-optimal planningE.g. used by Metric-FF, SHOP2 and temporal planners

Object fluents f : C∗ → CMaps set of constants to another constantE.g. (:function (neighbor-left ?c - cell) - cell):Maps a cell object to its left neighbor cellRaises expressiveness to Full First Order LogicUsed by very few planners[citation needed]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 9/34

Numeric Planning with PDDL

Define initial fluent values in initial state:(= (capacity truck1) 3) (= (capacity truck2) 5) ...

Do numeric comparisons in preconditions::precondition (and (>= (capacity ?truck) 1) ...)

Update numeric fluents in effects::effect (and (decrease (capacity ?truck) 1) ...)

Keywords: assign, increase/decrease (+/-), scaleUp/scaleDown (*/:)

Numeric expressions can be composite (note prefix operators):(assign (hp ?hero) (- (hp ?hero) (attack ?monster)))

(optional) Specify some metric for the planner to optimize:(:metric minimize (+

(distance-traveled truck1) (* 2 (fuel-used truck1))

)) (weighs used fuel twice as much as traveled distance)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 10/34

Numeric Planning with PDDL

Define initial fluent values in initial state:(= (capacity truck1) 3) (= (capacity truck2) 5) ...

Do numeric comparisons in preconditions::precondition (and (>= (capacity ?truck) 1) ...)

Update numeric fluents in effects::effect (and (decrease (capacity ?truck) 1) ...)

Keywords: assign, increase/decrease (+/-), scaleUp/scaleDown (*/:)

Numeric expressions can be composite (note prefix operators):(assign (hp ?hero) (- (hp ?hero) (attack ?monster)))

(optional) Specify some metric for the planner to optimize:(:metric minimize (+

(distance-traveled truck1) (* 2 (fuel-used truck1))

)) (weighs used fuel twice as much as traveled distance)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 10/34

Numeric Planning with PDDL

Define initial fluent values in initial state:(= (capacity truck1) 3) (= (capacity truck2) 5) ...

Do numeric comparisons in preconditions::precondition (and (>= (capacity ?truck) 1) ...)

Update numeric fluents in effects::effect (and (decrease (capacity ?truck) 1) ...)

Keywords: assign, increase/decrease (+/-), scaleUp/scaleDown (*/:)

Numeric expressions can be composite (note prefix operators):(assign (hp ?hero) (- (hp ?hero) (attack ?monster)))

(optional) Specify some metric for the planner to optimize:(:metric minimize (+

(distance-traveled truck1) (* 2 (fuel-used truck1))

)) (weighs used fuel twice as much as traveled distance)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 10/34

Numeric Planning with PDDL

Define initial fluent values in initial state:(= (capacity truck1) 3) (= (capacity truck2) 5) ...

Do numeric comparisons in preconditions::precondition (and (>= (capacity ?truck) 1) ...)

Update numeric fluents in effects::effect (and (decrease (capacity ?truck) 1) ...)

Keywords: assign, increase/decrease (+/-), scaleUp/scaleDown (*/:)

Numeric expressions can be composite (note prefix operators):(assign (hp ?hero) (- (hp ?hero) (attack ?monster)))

(optional) Specify some metric for the planner to optimize:(:metric minimize (+

(distance-traveled truck1) (* 2 (fuel-used truck1))

)) (weighs used fuel twice as much as traveled distance)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 10/34

Numeric Planning with PDDL

Define initial fluent values in initial state:(= (capacity truck1) 3) (= (capacity truck2) 5) ...

Do numeric comparisons in preconditions::precondition (and (>= (capacity ?truck) 1) ...)

Update numeric fluents in effects::effect (and (decrease (capacity ?truck) 1) ...)

Keywords: assign, increase/decrease (+/-), scaleUp/scaleDown (*/:)

Numeric expressions can be composite (note prefix operators):(assign (hp ?hero) (- (hp ?hero) (attack ?monster)))

(optional) Specify some metric for the planner to optimize:(:metric minimize (+

(distance-traveled truck1) (* 2 (fuel-used truck1))

)) (weighs used fuel twice as much as traveled distance)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 10/34

Numeric Planning and State Space

t@A, p1@A, p2@B

t@A, p1@T, p2@B

t@B, p1@A, p2@B

load(p1)

move(A,B)

move(B,A)

…

Classical planning: Finite state space;can eventually do exhaustive search (in theory)

Numeric planning: Infinite state space!Unlike atoms, numeric fluents have infinite domainSearch space is even less manageable than before

Is that a problem for forward state space search algorithms?Not for correctness, but for termination

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 11/34

Numeric Planning and State Space

t@A, p1@A, p2@B, traveled=0,

capacity=1

t@A, p1@T, p2@B, traveled=0,

capacity=0

t@B, p1@A, p2@B, traveled=1,

capacity=1

t@B, p1@A, p2@A, traveled=2,

capacity=1

load(p1)

move(A,B)

move(B,A) move(...)

…

Classical planning: Finite state space;can eventually do exhaustive search (in theory)Numeric planning: Infinite state space!

Unlike atoms, numeric fluents have infinite domainSearch space is even less manageable than before

Is that a problem for forward state space search algorithms?Not for correctness, but for termination

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 11/34

Numeric Planning and State Space

t@A, p1@A, p2@B, traveled=0,

capacity=1

t@A, p1@T, p2@B, traveled=0,

capacity=0

t@B, p1@A, p2@B, traveled=1,

capacity=1

t@B, p1@A, p2@A, traveled=2,

capacity=1

load(p1)

move(A,B)

move(B,A) move(...)

…

Classical planning: Finite state space;can eventually do exhaustive search (in theory)Numeric planning: Infinite state space!

Unlike atoms, numeric fluents have infinite domainSearch space is even less manageable than before

Is that a problem for forward state space search algorithms?Not for correctness, but for termination

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 11/34

Numeric Planning

Can we plan numerically with classical techniques?

Ground functions just like predicates and operatorsResults in flat set of numeric variablesAnalogy: Range of atoms is {true, false},Range of numeric variables is R

World state: All true atoms + all numeric variable valuesExtend forward state-space search to this new kind of states

Applicability of an action: Evaluate numeric conditionsw.r.t. variable values in current stateApplication of an action: Update variables according to effects

Example for this style of numeric planning: FF-Metric [Hof03]Uses extended definition of delete-relaxation for FF heuristic:Ignore preconditions and effects which decrease variable values

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 12/34

Numeric Planning

Can we plan numerically with classical techniques?Ground functions just like predicates and operators

Results in flat set of numeric variablesAnalogy: Range of atoms is {true, false},Range of numeric variables is R

World state: All true atoms + all numeric variable valuesExtend forward state-space search to this new kind of states

Applicability of an action: Evaluate numeric conditionsw.r.t. variable values in current stateApplication of an action: Update variables according to effects

Example for this style of numeric planning: FF-Metric [Hof03]Uses extended definition of delete-relaxation for FF heuristic:Ignore preconditions and effects which decrease variable values

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 12/34

Numeric Planning

Can we plan numerically with classical techniques?Ground functions just like predicates and operators

Results in flat set of numeric variablesAnalogy: Range of atoms is {true, false},Range of numeric variables is R

World state: All true atoms + all numeric variable values

Extend forward state-space search to this new kind of statesApplicability of an action: Evaluate numeric conditionsw.r.t. variable values in current stateApplication of an action: Update variables according to effects

Example for this style of numeric planning: FF-Metric [Hof03]Uses extended definition of delete-relaxation for FF heuristic:Ignore preconditions and effects which decrease variable values

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 12/34

Numeric Planning

Can we plan numerically with classical techniques?Ground functions just like predicates and operators

Results in flat set of numeric variablesAnalogy: Range of atoms is {true, false},Range of numeric variables is R

World state: All true atoms + all numeric variable valuesExtend forward state-space search to this new kind of states

Applicability of an action: Evaluate numeric conditionsw.r.t. variable values in current stateApplication of an action: Update variables according to effects

Example for this style of numeric planning: FF-Metric [Hof03]Uses extended definition of delete-relaxation for FF heuristic:Ignore preconditions and effects which decrease variable values

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 12/34

Numeric Planning

Can we plan numerically with classical techniques?Ground functions just like predicates and operators

Results in flat set of numeric variablesAnalogy: Range of atoms is {true, false},Range of numeric variables is R

World state: All true atoms + all numeric variable valuesExtend forward state-space search to this new kind of states

Applicability of an action: Evaluate numeric conditionsw.r.t. variable values in current stateApplication of an action: Update variables according to effects

Example for this style of numeric planning: FF-Metric [Hof03]Uses extended definition of delete-relaxation for FF heuristic:Ignore preconditions and effects which decrease variable values

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 12/34

Numeric Planning: Trucking Example

Operators: written as { pre(a) } a { eff(a) }{ t@x } move(x ,y ) { ¬t@x , t@y , traveled+=1 }

{ t@x , p@x , capacity>0 } load(p,x) { ¬p@x , p@t, capacity-=1 }{ t@x , p@t } drop(p,x) { ¬p@t, p@x , capacity+=1 }

A plan and its intermediate states:

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 13/34

Numeric Planning: Trucking Example

Operators: written as { pre(a) } a { eff(a) }{ t@x } move(x ,y ) { ¬t@x , t@y , traveled+=1 }

{ t@x , p@x , capacity>0 } load(p,x) { ¬p@x , p@t, capacity-=1 }{ t@x , p@t } drop(p,x) { ¬p@t, p@x , capacity+=1 }

A plan and its intermediate states:

t@A, p1@A, p2@B, traveled=0,

capacity=1

t@A, p1@T, p2@B, traveled=0,

capacity=0

load(p1) t@B, p1@T, p2@B, traveled=1,

capacity=0

move(A,B)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 13/34

Numeric Planning: Trucking Example

Operators: written as { pre(a) } a { eff(a) }{ t@x } move(x ,y ) { ¬t@x , t@y , traveled+=1 }

{ t@x , p@x , capacity>0 } load(p,x) { ¬p@x , p@t, capacity-=1 }{ t@x , p@t } drop(p,x) { ¬p@t, p@x , capacity+=1 }

A plan and its intermediate states:

t@A, p1@A, p2@B, traveled=0,

capacity=1

t@A, p1@T, p2@B, traveled=0,

capacity=0

load(p1) t@B, p1@T, p2@B, traveled=1,

capacity=0

move(A,B)

t@C, p1@T, p2@B, traveled=2,

capacity=0

move(B,C) t@C, p1@C, p2@B, traveled=2,

capacity=1

drop(p1) t@B, p1@C, p2@B, traveled=3,

capacity=1

move(C,B)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 13/34

Numeric Planning: Trucking Example

Operators: written as { pre(a) } a { eff(a) }{ t@x } move(x ,y ) { ¬t@x , t@y , traveled+=1 }

{ t@x , p@x , capacity>0 } load(p,x) { ¬p@x , p@t, capacity-=1 }{ t@x , p@t } drop(p,x) { ¬p@t, p@x , capacity+=1 }

A plan and its intermediate states:

t@A, p1@A, p2@B, traveled=0,

capacity=1

t@A, p1@T, p2@B, traveled=0,

capacity=0

load(p1) t@B, p1@T, p2@B, traveled=1,

capacity=0

move(A,B)

t@C, p1@T, p2@B, traveled=2,

capacity=0

move(B,C) t@C, p1@C, p2@B, traveled=2,

capacity=1

drop(p1) t@B, p1@C, p2@B, traveled=3,

capacity=1

move(C,B)

t@B, p1@C, p2@T, traveled=3,

capacity=0

load(p2) t@C, p1@C, p2@T, traveled=4,

capacity=0

move(B,C) t@C, p1@C, p2@C, traveled=4,

capacity=1

drop(p2)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 13/34

Numeric Planning: Discussion

Opens up vast possibilities for modeling complex domainsDealing with resources, non-discrete quantitiesArithmetic action conditions: Quantitative object relationshipsSophisticated plan quality metrics

Optimal planning becomes much more complicatedHeuristic must be admissible with respect to metricWhat about decreasing metric values?

Weaker form of function support: Rigid (constant) function valuesNot really numeric planningStill useful for defining interesting action costs

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 14/34

Numeric Planning: Discussion

Opens up vast possibilities for modeling complex domainsDealing with resources, non-discrete quantitiesArithmetic action conditions: Quantitative object relationshipsSophisticated plan quality metrics

Optimal planning becomes much more complicatedHeuristic must be admissible with respect to metricWhat about decreasing metric values?

Weaker form of function support: Rigid (constant) function valuesNot really numeric planningStill useful for defining interesting action costs

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 14/34

Numeric Planning: Discussion

Opens up vast possibilities for modeling complex domainsDealing with resources, non-discrete quantitiesArithmetic action conditions: Quantitative object relationshipsSophisticated plan quality metrics

Optimal planning becomes much more complicatedHeuristic must be admissible with respect to metricWhat about decreasing metric values?

Weaker form of function support: Rigid (constant) function valuesNot really numeric planningStill useful for defining interesting action costs

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 14/34

Advanced Plan Quality Metrics

Plan quality can involve more than just some numeric quantityExamples:

The hero must reach the dungeon exit, and if possible,he should have the treasure.Any plan involving a long break for the trucker in between twodeliveries is to be preferred.I would rather not use the expensive rum for mixing that drink,but I could if I needed to.

Idea: Embed soft constraints about the plan into the definitionso planners can try and find the overall best planRealization in PDDL:

Specify preferences in preconditions (what we would like to hold)Specify a plan quality metric based on number of violationsof each defined preference

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 15/34

Advanced Plan Quality Metrics

Plan quality can involve more than just some numeric quantityExamples:

The hero must reach the dungeon exit, and if possible,he should have the treasure.Any plan involving a long break for the trucker in between twodeliveries is to be preferred.I would rather not use the expensive rum for mixing that drink,but I could if I needed to.

Idea: Embed soft constraints about the plan into the definitionso planners can try and find the overall best plan

Realization in PDDL:Specify preferences in preconditions (what we would like to hold)Specify a plan quality metric based on number of violationsof each defined preference

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 15/34

Advanced Plan Quality Metrics

Plan quality can involve more than just some numeric quantityExamples:

The hero must reach the dungeon exit, and if possible,he should have the treasure.Any plan involving a long break for the trucker in between twodeliveries is to be preferred.I would rather not use the expensive rum for mixing that drink,but I could if I needed to.

Idea: Embed soft constraints about the plan into the definitionso planners can try and find the overall best planRealization in PDDL:

Specify preferences in preconditions (what we would like to hold)Specify a plan quality metric based on number of violationsof each defined preference

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 15/34

Preferences in PDDL [GL05]

Syntax for preference: (:preference name condition )

Example, inside an operator mix cocktail::precondition (and

...

(:preference CheapRum (not (expensive ?rum)))

)

Specify metric in the problem definition:(:metric minimize (is-violated CheapRum))

(is-violated ?Pref): “built-in function”, maps preferenceto its number of violations in an action sequenceObjective here: minimize amount of times where ?rum is expensive

Build complex metrics from multiple preference violations:(:metric minimize (+ (is-violated CheapVodka)

(* 2 (is-violated CheapRum)))

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 16/34

Preferences in PDDL [GL05]

Syntax for preference: (:preference name condition )

Example, inside an operator mix cocktail::precondition (and

...

(:preference CheapRum (not (expensive ?rum)))

)

Specify metric in the problem definition:(:metric minimize (is-violated CheapRum))

(is-violated ?Pref): “built-in function”, maps preferenceto its number of violations in an action sequenceObjective here: minimize amount of times where ?rum is expensive

Build complex metrics from multiple preference violations:(:metric minimize (+ (is-violated CheapVodka)

(* 2 (is-violated CheapRum)))

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 16/34

Preferences in PDDL [GL05]

Syntax for preference: (:preference name condition )

Example, inside an operator mix cocktail::precondition (and

...

(:preference CheapRum (not (expensive ?rum)))

)

Specify metric in the problem definition:(:metric minimize (is-violated CheapRum))

(is-violated ?Pref): “built-in function”, maps preferenceto its number of violations in an action sequenceObjective here: minimize amount of times where ?rum is expensive

Build complex metrics from multiple preference violations:(:metric minimize (+ (is-violated CheapVodka)

(* 2 (is-violated CheapRum)))

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 16/34

Next Up: Temporal Planning

Some temporal aspects of planning

Background: Linear Temporal Logic (LTL)

Application: Plan trajectory constraints

Time models in planning

Basic Temporal Planning with durative actions

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 17/34

Linear Temporal Logic (1)

Linear Temporal LogicGiven a set of propositional variables V , the set of Linear Temporal Logic(LTL) formulae is defined recursively as follows:

1 For v ∈ V , v is an LTL formula.2 For LTL formulae α and β, the following are LTL formulae as well:

¬α ; α ∧ β ; α ∨ βX α (read: next alpha)G α (read: globally alpha)α U β (read: alpha until beta)

Intuitive meanings:

X α : α holds at the next time point from now

G α : α holds at all time points from now

α U β : α holds until, at some time point, β holds

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 18/34

Linear Temporal Logic (1)

Linear Temporal LogicGiven a set of propositional variables V , the set of Linear Temporal Logic(LTL) formulae is defined recursively as follows:

1 For v ∈ V , v is an LTL formula.2 For LTL formulae α and β, the following are LTL formulae as well:

¬α ; α ∧ β ; α ∨ βX α (read: next alpha)G α (read: globally alpha)α U β (read: alpha until beta)

Intuitive meanings:

X α : α holds at the next time point from now

G α : α holds at all time points from now

α U β : α holds until, at some time point, β holds

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 18/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0

α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:

Within(t) α :=∨

0≤i≤t XiαSomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U α

Eventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G α

AtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α))

. . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

Linear Temporal Logic (2)

LTL formulae are evaluated under an infinite sequence of variablevalues (states) W := 〈w0,w1,w2, . . .〉:

A prop. formula F holds in w0 iff it is satisfied under the values in w0

X α holds in w0 iff α holds in state w1

G α holds in w0 iff α holds in all states wk , k ≥ 0α U β holds in w0 iff, for some k ≥ 0, α holds in all states{w0, . . . ,wk−1} and β holds in state wk

LTL formulae as described form a basis for additional operators:Within(t) α :=

∨0≤i≤t Xiα

SomeTime α := ¬α U αEventually α := SomeTime G αAtMostOnce α := SomeTime (α ∧ X (G ¬α)). . .(If you like this, you should try the Formal Systems lecture)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 19/34

LTL Applied to Planning

Plan trajectoryGiven a plan or a prefix of a plan π, the trajectory of π is the sequence ofintermediate states that are visited during the execution of π.

Idea: Use LTL operators on a plan trajectory to express constraints, e.g.

We want to reach the treasure within 12 time units.

When a cocktail is mixed, it is eventually served.

The fuel of a truck must be non-negative at all times.

The turbo gear can be activated over 3 time units at most.

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 20/34

LTL Applied to Planning

Plan trajectoryGiven a plan or a prefix of a plan π, the trajectory of π is the sequence ofintermediate states that are visited during the execution of π.

Idea: Use LTL operators on a plan trajectory to express constraints, e.g.

We want to reach the treasure within 12 time units.

When a cocktail is mixed, it is eventually served.

The fuel of a truck must be non-negative at all times.

The turbo gear can be activated over 3 time units at most.

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 20/34

Plan Trajectory Constraints [GL05]

Using LTL, specify and validate temporal constraints on plans

Let π = 〈a0, . . . , an−1〉 be a plan candidateand 〈s0, s1, . . . , sn〉 its trajectory

Define LTL state sequence W := 〈s0, s1, . . . , sn−1, sn, sn, sn, . . .〉Given a set of constraints written as an LTL formula α under W ,evaluate α in s0 to check if constraints are satisfied (how? not here)

Incorporate constraints into planning:

For current plan prefix candidate (unfinished plan),check if there is some way to satisfy α in the future

If yes: Continue planning on this path(Example: un-served cocktail might still be served later)If not: Plan candidate is no good, backtrack(Example: if truck fuel is already negative, cannot be fixed)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 21/34

Plan Trajectory Constraints [GL05]

Using LTL, specify and validate temporal constraints on plans

Let π = 〈a0, . . . , an−1〉 be a plan candidateand 〈s0, s1, . . . , sn〉 its trajectory

Define LTL state sequence W := 〈s0, s1, . . . , sn−1, sn, sn, sn, . . .〉

Given a set of constraints written as an LTL formula α under W ,evaluate α in s0 to check if constraints are satisfied (how? not here)

Incorporate constraints into planning:

For current plan prefix candidate (unfinished plan),check if there is some way to satisfy α in the future

If yes: Continue planning on this path(Example: un-served cocktail might still be served later)If not: Plan candidate is no good, backtrack(Example: if truck fuel is already negative, cannot be fixed)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 21/34

Plan Trajectory Constraints [GL05]

Using LTL, specify and validate temporal constraints on plans

Let π = 〈a0, . . . , an−1〉 be a plan candidateand 〈s0, s1, . . . , sn〉 its trajectory

Define LTL state sequence W := 〈s0, s1, . . . , sn−1, sn, sn, sn, . . .〉Given a set of constraints written as an LTL formula α under W ,evaluate α in s0 to check if constraints are satisfied (how? not here)

Incorporate constraints into planning:

For current plan prefix candidate (unfinished plan),check if there is some way to satisfy α in the future

If yes: Continue planning on this path(Example: un-served cocktail might still be served later)If not: Plan candidate is no good, backtrack(Example: if truck fuel is already negative, cannot be fixed)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 21/34

Plan Trajectory Constraints [GL05]

Using LTL, specify and validate temporal constraints on plans

Let π = 〈a0, . . . , an−1〉 be a plan candidateand 〈s0, s1, . . . , sn〉 its trajectory

Define LTL state sequence W := 〈s0, s1, . . . , sn−1, sn, sn, sn, . . .〉Given a set of constraints written as an LTL formula α under W ,evaluate α in s0 to check if constraints are satisfied (how? not here)

Incorporate constraints into planning:

For current plan prefix candidate (unfinished plan),check if there is some way to satisfy α in the future

If yes: Continue planning on this path(Example: un-served cocktail might still be served later)If not: Plan candidate is no good, backtrack(Example: if truck fuel is already negative, cannot be fixed)

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 21/34

Plan Trajectory Constraints: Examples

Some possible (:constraints ) in PDDL problem files

We want to reach the treasure within 12 time units:(within 12 (hero-has-treasure))

When a cocktail is mixed, it is eventually served:(sometime-after (mixed cocktail) (served cocktail))

The fuel of a truck must be non-negative at all times:(always (>= (fuel) 0))

The turbo gear can be activated over 3 time units at most:(always-within 3 (turbo-gear on) (turbo-gear off))

Combine preferences and constraints

If possible, use turbo gear at most once:(preference Eco (at-most-once (turbo-gear on)))

Some example PDDL domains: [GDHS06]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 22/34

Plan Trajectory Constraints: Examples

Some possible (:constraints ) in PDDL problem files

We want to reach the treasure within 12 time units:(within 12 (hero-has-treasure))

When a cocktail is mixed, it is eventually served:(sometime-after (mixed cocktail) (served cocktail))

The fuel of a truck must be non-negative at all times:(always (>= (fuel) 0))

The turbo gear can be activated over 3 time units at most:(always-within 3 (turbo-gear on) (turbo-gear off))

Combine preferences and constraints

If possible, use turbo gear at most once:(preference Eco (at-most-once (turbo-gear on)))

Some example PDDL domains: [GDHS06]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 22/34

Plan Trajectory Constraints: Examples

Some possible (:constraints ) in PDDL problem files

We want to reach the treasure within 12 time units:(within 12 (hero-has-treasure))

When a cocktail is mixed, it is eventually served:(sometime-after (mixed cocktail) (served cocktail))

The fuel of a truck must be non-negative at all times:(always (>= (fuel) 0))

The turbo gear can be activated over 3 time units at most:(always-within 3 (turbo-gear on) (turbo-gear off))

Combine preferences and constraints

If possible, use turbo gear at most once:(preference Eco (at-most-once (turbo-gear on)))

Some example PDDL domains: [GDHS06]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 22/34

Plan Trajectory Constraints: Examples

Some possible (:constraints ) in PDDL problem files

We want to reach the treasure within 12 time units:(within 12 (hero-has-treasure))

When a cocktail is mixed, it is eventually served:(sometime-after (mixed cocktail) (served cocktail))

The fuel of a truck must be non-negative at all times:(always (>= (fuel) 0))

The turbo gear can be activated over 3 time units at most:(always-within 3 (turbo-gear on) (turbo-gear off))

Combine preferences and constraints

If possible, use turbo gear at most once:(preference Eco (at-most-once (turbo-gear on)))

Some example PDDL domains: [GDHS06]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 22/34

Plan Trajectory Constraints: Examples

Some possible (:constraints ) in PDDL problem files

We want to reach the treasure within 12 time units:(within 12 (hero-has-treasure))

When a cocktail is mixed, it is eventually served:(sometime-after (mixed cocktail) (served cocktail))

The fuel of a truck must be non-negative at all times:(always (>= (fuel) 0))

The turbo gear can be activated over 3 time units at most:(always-within 3 (turbo-gear on) (turbo-gear off))

Combine preferences and constraints

If possible, use turbo gear at most once:(preference Eco (at-most-once (turbo-gear on)))

Some example PDDL domains: [GDHS06]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 22/34

Plan Trajectory Constraints: Examples

Some possible (:constraints ) in PDDL problem files

We want to reach the treasure within 12 time units:(within 12 (hero-has-treasure))

When a cocktail is mixed, it is eventually served:(sometime-after (mixed cocktail) (served cocktail))

The fuel of a truck must be non-negative at all times:(always (>= (fuel) 0))

The turbo gear can be activated over 3 time units at most:(always-within 3 (turbo-gear on) (turbo-gear off))

Combine preferences and constraints

If possible, use turbo gear at most once:(preference Eco (at-most-once (turbo-gear on)))

Some example PDDL domains: [GDHS06]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 22/34

Trajectory Constraints: Discussion

Nice tool to model complex constraints of problem domainsNatural restrictions, e.g. non-negative resources and capacitiesSafety restrictions, e.g. temperatures, mutual exclusions

Enables versatile and realistic assessment of plan qualitywhen combined with preferences (Preference-based planning)

Just-in-time constraintsEconomic usage of resources (time, fuel, machines, . . . )“Slack” for handling unexpected problems during plan execution

Many constraints could also be realized in a classical way(e.g. at-most-once, sometime can be enforced in actions and goal)

Requires modification of planning algorithm,non-trivial to evaluate LTL formulae

Raises questions on semantics of time in planning

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 23/34

Trajectory Constraints: Discussion

Nice tool to model complex constraints of problem domainsNatural restrictions, e.g. non-negative resources and capacitiesSafety restrictions, e.g. temperatures, mutual exclusions

Enables versatile and realistic assessment of plan qualitywhen combined with preferences (Preference-based planning)

Just-in-time constraintsEconomic usage of resources (time, fuel, machines, . . . )“Slack” for handling unexpected problems during plan execution

Many constraints could also be realized in a classical way(e.g. at-most-once, sometime can be enforced in actions and goal)

Requires modification of planning algorithm,non-trivial to evaluate LTL formulae

Raises questions on semantics of time in planning

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 23/34

Trajectory Constraints: Discussion

Nice tool to model complex constraints of problem domainsNatural restrictions, e.g. non-negative resources and capacitiesSafety restrictions, e.g. temperatures, mutual exclusions

Enables versatile and realistic assessment of plan qualitywhen combined with preferences (Preference-based planning)

Just-in-time constraintsEconomic usage of resources (time, fuel, machines, . . . )“Slack” for handling unexpected problems during plan execution

Many constraints could also be realized in a classical way(e.g. at-most-once, sometime can be enforced in actions and goal)

Requires modification of planning algorithm,non-trivial to evaluate LTL formulae

Raises questions on semantics of time in planning

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 23/34

Trajectory Constraints: Discussion

Nice tool to model complex constraints of problem domainsNatural restrictions, e.g. non-negative resources and capacitiesSafety restrictions, e.g. temperatures, mutual exclusions

Enables versatile and realistic assessment of plan qualitywhen combined with preferences (Preference-based planning)

Just-in-time constraintsEconomic usage of resources (time, fuel, machines, . . . )“Slack” for handling unexpected problems during plan execution

Many constraints could also be realized in a classical way(e.g. at-most-once, sometime can be enforced in actions and goal)

Requires modification of planning algorithm,non-trivial to evaluate LTL formulae

Raises questions on semantics of time in planning

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 23/34

Time Models in Planning

0 1 2 3 …

What is our understanding of time in automated planning?

Ray with linear sequence of non-negative time points

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 24/34

Time Models in Planning

0s

0

1s

1

2s

2

3 …s

3

a1

a2

a3

…

Classical, sequential planningInteger time pointsOne state and one action at each time pointExamples: Forward SSS, naıve SAT planning

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 24/34

Time Models in Planning

a12

a2

a33

…

a11

a32

a31

0s

0

1s

1

2s

2

3 …s

3

Classical planning with parallel actionsInteger time pointsOne state and set of actions at each time pointExamples: Graphplan, SAT planning with parallel semantics

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 24/34

Time Models in Planning

0 1 2 3 …

a1

…

a2

a3

a4

s0

s1

s2 s

3s

4

s5

Temporal planningReal valued time points (in general)Actions have certain durations, may overlapNo bijection between time points and states any more:dynamic introduction of states whenever something might change

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 24/34

Towards Temporal Planning

Model of time and actions for temporal planning

0 1 2 3 …

a1

…

a2

a3

a4

s0

s1

s2 s

3s

4

s5

How to describe actions in such a model?

Preconditions and effects?

Durations?

Concurrency?

States?

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 25/34

Temporal Planning with PDDL

Anatomy of a temporal operator (durative action) in PDDL:

(:durative-action drive

:parameters (?t - truck ?from ?to - location)

:duration (= ?duration (drive-time ?from ?to))

:condition (and

(at start (at ?t ?from))

(over all (connected ?from ?to)))

:effect (and

(at start (not (at ?t ?from)))

(at end (at ?t ?to))))

Duration: How long does the action take? (numeric expression)

Condition: Preconditions, before and during the action execution

Effect: Effects, at the action’s start and end points

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 26/34

Semantics of Durative Actions

td

td+d

(drive ?t ?from ?to)

(at ?t ?from) (at ?t ?to)

(connected ?from ?to)

Condition (at start (at ?t ?from)):For some ε > 0, (at ?t ?from) holds during [td − ε, td)

Condition (over all (connected ?from ?to)):(connected ?from ?to) holds during [td , td + d)

Effect (at start (not (at ?t ?from))):(at ?t ?from) is deleted at time point tdEffect (at end (at ?t ?to)):(at ?t ?to) is added at time point td + d

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 27/34

Semantics of Durative Actions

td

td+d

(drive ?t ?from ?to)

(at ?t ?from) (at ?t ?to)

(connected ?from ?to)

Condition (at start (at ?t ?from)):For some ε > 0, (at ?t ?from) holds during [td − ε, td)Condition (over all (connected ?from ?to)):(connected ?from ?to) holds during [td , td + d)

Effect (at start (not (at ?t ?from))):(at ?t ?from) is deleted at time point tdEffect (at end (at ?t ?to)):(at ?t ?to) is added at time point td + d

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 27/34

Semantics of Durative Actions

td

td+d

(drive ?t ?from ?to)

(at ?t ?from) (at ?t ?to)

(connected ?from ?to)

Condition (at start (at ?t ?from)):For some ε > 0, (at ?t ?from) holds during [td − ε, td)Condition (over all (connected ?from ?to)):(connected ?from ?to) holds during [td , td + d)

Effect (at start (not (at ?t ?from))):(at ?t ?from) is deleted at time point td

Effect (at end (at ?t ?to)):(at ?t ?to) is added at time point td + d

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 27/34

Semantics of Durative Actions

td

td+d

(drive ?t ?from ?to)

(at ?t ?from) (at ?t ?to)

(connected ?from ?to)

Condition (at start (at ?t ?from)):For some ε > 0, (at ?t ?from) holds during [td − ε, td)Condition (over all (connected ?from ?to)):(connected ?from ?to) holds during [td , td + d)

Effect (at start (not (at ?t ?from))):(at ?t ?from) is deleted at time point tdEffect (at end (at ?t ?to)):(at ?t ?to) is added at time point td + d

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 27/34

Temporal Planning

Some (informal) notions of temporal planning:Domain: As in classical planning, but with durative actions

Each action has a certain duration dEffects, preconditions associated with time points relative to d

Planning Problem: Initial state and goals as in classical planning

Plan: A set π = {(t0, a0), (t1, a1), . . . , (tn, an)},meaning that each action ai starts at time point ti

Temporal planning approaches?

State-space based: Extend Forward Search to temporal domains

Plan-space based: Extend PSP to temporal domains

Constraint-based: Satisfiability Modulo Theories (SMT),Integer Linear Programming (ILP), . . . [Rin17]

Here: Brief idea on temporal forward SSS planning

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 28/34

Temporal Planning

Some (informal) notions of temporal planning:Domain: As in classical planning, but with durative actions

Each action has a certain duration dEffects, preconditions associated with time points relative to d

Planning Problem: Initial state and goals as in classical planning

Plan: A set π = {(t0, a0), (t1, a1), . . . , (tn, an)},meaning that each action ai starts at time point ti

Temporal planning approaches?

State-space based: Extend Forward Search to temporal domains

Plan-space based: Extend PSP to temporal domains

Constraint-based: Satisfiability Modulo Theories (SMT),Integer Linear Programming (ILP), . . . [Rin17]

Here: Brief idea on temporal forward SSS planning

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 28/34

Temporal Planning

Some (informal) notions of temporal planning:Domain: As in classical planning, but with durative actions

Each action has a certain duration dEffects, preconditions associated with time points relative to d

Planning Problem: Initial state and goals as in classical planning

Plan: A set π = {(t0, a0), (t1, a1), . . . , (tn, an)},meaning that each action ai starts at time point ti

Temporal planning approaches?

State-space based: Extend Forward Search to temporal domains

Plan-space based: Extend PSP to temporal domains

Constraint-based: Satisfiability Modulo Theories (SMT),Integer Linear Programming (ILP), . . . [Rin17]

Here: Brief idea on temporal forward SSS planning

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 28/34

The Idea of Temporal SSS (1)

Preliminaries for state-space based temporal planning:

Action ClocksGiven a durative action a, an action clock c of a features a timer valuet ≥ 0 and a set of timed events {(t1, e1), . . . , (tk , ek)}. Each event ei iseither a precondition of a to check or an effect of a to apply.

Timed StatesA timed state τ = (s,C) is a classical planning state s associated with aset of action clocks C.

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 29/34

The Idea of Temporal SSS (2)

Temporal State-space search planning (see [Rin17])

Initial search node: Timed state τI = (sI , {})

Options for one exploration step:1 Initiate an action a: add a new clock c for a with timer value of 0,

process and remove all of its events (ti , ei) with ti = 02 Advance time by some amount t > 0: increase timer of all clocks by t ;

chronologically process and remove all events (ti , ei) with ti ≤ t

Leads to chronological exploration procedure

Backtrack whenever condition of an event is violated

When state (s, {}) is reached s.t. g ⊆ s,return timer value and associated action of each action clock

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 30/34

The Idea of Temporal SSS (2)

Temporal State-space search planning (see [Rin17])

Initial search node: Timed state τI = (sI , {})Options for one exploration step:

1 Initiate an action a: add a new clock c for a with timer value of 0,process and remove all of its events (ti , ei) with ti = 0

2 Advance time by some amount t > 0: increase timer of all clocks by t ;chronologically process and remove all events (ti , ei) with ti ≤ t

Leads to chronological exploration procedure

Backtrack whenever condition of an event is violated

When state (s, {}) is reached s.t. g ⊆ s,return timer value and associated action of each action clock

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 30/34

The Idea of Temporal SSS (2)

Temporal State-space search planning (see [Rin17])

Initial search node: Timed state τI = (sI , {})Options for one exploration step:

1 Initiate an action a: add a new clock c for a with timer value of 0,process and remove all of its events (ti , ei) with ti = 0

2 Advance time by some amount t > 0: increase timer of all clocks by t ;chronologically process and remove all events (ti , ei) with ti ≤ t

Leads to chronological exploration procedure

Backtrack whenever condition of an event is violated

When state (s, {}) is reached s.t. g ⊆ s,return timer value and associated action of each action clock

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 30/34

The Idea of Temporal SSS (2)

Temporal State-space search planning (see [Rin17])

Initial search node: Timed state τI = (sI , {})Options for one exploration step:

1 Initiate an action a: add a new clock c for a with timer value of 0,process and remove all of its events (ti , ei) with ti = 0

2 Advance time by some amount t > 0: increase timer of all clocks by t ;chronologically process and remove all events (ti , ei) with ti ≤ t

Leads to chronological exploration procedure

Backtrack whenever condition of an event is violated

When state (s, {}) is reached s.t. g ⊆ s,return timer value and associated action of each action clock

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 30/34

The Idea of Temporal SSS (2)

Temporal State-space search planning (see [Rin17])

Initial search node: Timed state τI = (sI , {})Options for one exploration step:

1 Initiate an action a: add a new clock c for a with timer value of 0,process and remove all of its events (ti , ei) with ti = 0

2 Advance time by some amount t > 0: increase timer of all clocks by t ;chronologically process and remove all events (ti , ei) with ti ≤ t

Leads to chronological exploration procedure

Backtrack whenever condition of an event is violated

When state (s, {}) is reached s.t. g ⊆ s,return timer value and associated action of each action clock

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 30/34

Temporal Planning: Discussion

Generalizes notion of time in planningEnables seamless, well-defined concurrencyMore realism than instantaneous actions

Requires different / new planning approaches

Durative actions as modeled in PDDL have some shortcomings(see exercises)

Examples for temporal PDDL domains: [CCM18]

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 31/34

Deterministic Planning: Brief Taxonomy

Common notions in classical planningSTRIPS (preconditions, effects, goals)ADL (equality, disjunctive conditions, quantifications)Axioms / derived predicates

Hierarchical planningHierarchical task networks

Numeric planningNumeric fluents (functions) and conditionsAdvanced (arithmetic) plan quality metrics

Temporal planningDurative and concurrent actions

Preference-based planningPreferences / soft plan constraintsPlan trajectories / hard plan constraints

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 32/34

Stay tuned!

Next lecture: Introduction to Scheduling

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 33/34

References IAndrew Coles, Amanda Coles, and Moises Martinez, International planningcompetition 2018: Temporal tracks,https://ipc2018-temporal.bitbucket.io/#domains, 2018.

Alfonso Gerevini, Yannis Dimopoulos, Patrik Haslum, and Alessandro Saetti,Benchmark domains and problems of IPC-5,http://idm-lab.org/wiki/icaps/ipc2006/deterministic/, 2006.

Alfonso Gerevini and Derek Long, Plan constraints and preferences in PDDL3, Tech.report, Technical Report 2005-08-07, Department of Electronics for Automation . . . ,2005.

Jorg Hoffmann, The Metric-FF planning system: Translating “ignoring delete lists” tonumeric state variables, ”Journal of Artificial Intelligence Research” 20 (2003),291–341.

Jussi Rintanen, A brief overview of temporal planning in AI,https://users.aalto.fi/~rintanj1/temporalplanning.html, 2017.

Sylvie Thiebaux, Jorg Hoffmann, and Bernhard Nebel, In defense of PDDL axioms,Artificial Intelligence 168 (2005), no. 1-2, 38–69.

Tomas Balyo, Dominik Schreiber – Planning and Scheduling January 10, 2019 34/34